I wouldn't worry too much about removing the other 2.
You might want to if they are trying to hide with the nesting female, or otherwise disturbing her.