The number of people doesn't matter too much as long as everyone is pulling their weight (so avoid that guy with the 4* pulsar who isn't doing much)

Having damage bonuses will make it easier to kill enemies quicker (so use some Offense-Armor lolololz) especially if you're doing 4 person elite since enemies have a lot of hp.
brandish/vortexes are commonly used to kill enemies quickly. Max out ctr usually through chaos set or trinkets.
You can also combo this up with sprite skills, so use them when you can.

knowing the spawns will help too (charging blitz to kill a trojan when it appears, or drop a vortex when you know a group of zombies will come out)

It helps if you run with the same people since you know what weapons/styles they are using and you usually end up taking roles (one person watering the mask instead of nobody or everybody trying to do it, etc) and this cuts down on time wasted.
Try for a time of 30-40 minutes (yeah it's been done quicker, but that's good enough without having to feel rushed)