starting as babies – at a young age, they would have a "very" hard time chasing down food in a cage that large.
Once the male becomes mature enough to breed (while the females are too young) he would probably try to breed them which could cause egg binding and/or extra stress.
Housing any group of beardies together throughout their growing period will cause stress for the less dominant ones and cause them to eat less (and usually grow slower).
It's best to raise the females separate from the male until they're all full grown adults. Then you can place them all in the same cage... if the male is overly aggressive, he should only be with the females during breeding.
Hope this helps.