Couple quick questions...

Are there any kinds of paint to stay away from for painting the foam rocks?

and

With caulking, i'm going to put these foam rocks in and should I caulk the mdf box (painted with clear coat) or caulk the fake rocks or do I caulk both?