Glancing around it seems like people have caught most of the glaring issues, but I'll chip in:
1. AI players don't build anything beefier than single units for the most part. Only the very top performers on the map seem to get that far.
2. The AI barely understands armor. It knows what atk value is good for and how to get it. But it can't think any higher than basic tier armor.
3. The AI doesn't understand it's under threat once war is declared. It reacts to players just fine. It will attack armies it can reasonably beat. It will make sure there are people on its cities. But it doesn't alter its production scheme. It doesn't make concerted efforts to capture player territory unless it's SCREAMING to the AI to be taken. (I.e. undefended. I see AIs taking cities from each other sometimes...but rarely from me.) As of the last beta, all they did was mob it with pioneers. It doesn't mass armies once war is declared, it generally just keeps expanding its armies at the normal rate it would have. When a player has just cut down 50% of their military, and they're still at war, the AI needs to re-up it's military as a priority.
4. AIs still freeze. For example, in my last game, I went to war with Umber. I destroyed two cities, that then put two food resources in completely unclaimed territory between our cities. I left those resources alone, to see who would claim them. No one did, hundreds of turns later. Umber never sent people to rebuild on them, and none of the other 4 AIs tried to claim them either, even though there was another third player very close by with clear access to it. If nothing else, I expected Umber to reclaim them. He had enough resources to build armies, that he sent the opposite direction against another player. He even had pioneers running around. So yeah. Not sure what happened there. But in general some of the AIs in the mid-game just seem to stop doing anything other than produce. Their units stop moving, they stop claiming stuff. In that same game, the Pariden player had a few units camped off my borders for 300 turns, no kidding. Probably more. They never moved by the time I abandoned that game. I see the same behavior elsewhere, from other AIs. Some just stop moving when they've run out of easy things to do. Even if the AI isn't up to something, it's units should still be doing something.
And as a last request....monsters and AI players need to treat each other as threats. They don't. Monsters are there for the AIs to kill for experience, and that's totally unfair to the player. Monsters that could ransack AI empires ignore them. Maybe you think that wouldn't be fun, to have AI empires fall to monsters. But it's not any more fun to be the only thing they pay attention to. If a faction falls to monsters, I consider that how the world shook out this time around. Plus, if AIs are actually building militaries to defeat monsters....maybe they won't be such push overs when the players arrive. Monsters are THE reason as a player that I have strong militaries by the time I meet the other factions, because I had to kill my there to just find them.
And I guess a final note of caution. We want AIs that feel challenging....but we also want AIs that feel different. There isn't an inherent problem with AIs that don't build super strong armies. The problem is that almost no AIs do it. We want variety in how factions play, we don't want them all be l33t military planners and tacticians. But there has to be a base level of understanding every AI has about what it needs to do to survive. The resourcing end, I think that's *mostly* sorted out. Military design is still one place it needs a lot of work. Even if an AI is meant to be non-aggressive, it should still realize it has to build army-sized units if the other players are.
On cities:
What I've seen is this. In lieu of actual resources as a reason for building cities, the AI builds research centers. That's its basic, end of thought build plan. It goes "Hrm I need research and I can't put anymore in this city/don't have the population for it. Welp, I'll just start an outpost and build another research lab!" Eventually they find food and their research labs expand to become really large cities built entirely on research buildings and nothing else.
It seems like the AI is, above all things, terrified of falling behind in tech (magic secondarily.) So it will screw up its entire empire (and probably use its city upgrades) in the name of !SCIENCE! Which is funny because then the AI often can't, or just fails to, do anything with all that fabulous research they've been doing. Battle axes, Umberdoths, and the city upgrades, are about the most tangible thing I can point to that comes out of their research, and the occasional squad-sized army.
It's appetite for research seems endless too. Even if it has 3 Lost Libraries (which it sort of optimizes for since it's focusing on research anyways) it will happily build more cities and fill them with research.