A 4-hour game is extraordinarily long in the first place... and I'd wager that by that point the bottleneck is probably one of the players' CPUs, rather than a problem with Sins itself. If there are tens of thousands of ships/particles etc in the galaxy, the lower-performing CPUs simply can't keep up and game speeds slow to a crawl.

I really recommend playing shorter games, like the kinds you see online, which rarely last more than an hour and a bit but have a much more engaging strategic setting.

Are yall trimming your empire trees?

Perhaps try a few games so that you all play each other 1 on 1 with however many CPUs. If it is just that one of you has a slow PC/Internet connection, two of those games should be the same while 1 of them should be better. Of course I suppose it could be more than 1 of you having problems but if that's the case there is probably no easy solution.

Trimming the feature tree (i.e. do not use it, unpin everything and turn-off autopinning) does wonders, as does running on low graphics.

What is truly a big killer is the game speed settings as they are trying to cram more and more cycles through the cpu.