I just hope it's not the engine being limited, but on a 12 player melee map with 11 AI bots, at the 20 minute mark around, everything goes to hell... performance just takes a dump. 1 Second it's fine, the other second it freezes, and it loops.... When you pause it, framerate goes back to normal but the second you unpause hold on to your burritos cause it's going back down again and freezes... seriously Blizzard? I understand custom maps that does memory leaks but this is a simple melee game that I was playing at work (nothing else to do at work lol.)
i was going to give you a hard time and tell you you have a shitty computer, but just to be sure i played a 12p ffa with 11 insane computers and the exact same thing happened to me. it's not a hardware issue, i have an i7 950 @ 4ghz with 12 gigs of ram. war3 doesn't even have cpu spikes or a memory leak, it's chilling at 374 megs of ram and 3-5% cpu during it. it comes and goes after ~20 minutes, but it's definitely as you described.
Do the bots have enough build space? I only know they can go haywire when they want to place structures but do not find the place for it and therefore try it over and over again with the pathing algorithm.