Hello,
First of all, you might want to get a slightly bigger cage. You are going to need a fluorescent light, which gives the lizards UVA and UVB rays necessary for them survive. Normally, when they are outside, the sun gives off these rays. The lizards will eventually get sick without them and die (I lost my first Alligator lizard because of this since I didn't know). For my lizards, I have a big fish tank style hood that has a fluorescent light, but I think you can now purchase heat lamp bulbs that give off both UVA and UVB rays without needing a big extra hood. You are going to need a heat lamp in addition, unless you can find those new heat lamps that have both UVA and UVB rays. Check your local pet store and read the boxes. They will say what kind of rays the bulb gives out. It is important for the lizards to have a basking spot under the heat lamp so they can properly digest their food once they've eaten (heat is vital to getting the lizards bodies functioning). The reason you are going to need a bigger cage is so it doesn't get too hot in there. With a small cage, the heat lamp could fry the little guys (since they don't have anywhere to escape from the heat). A 10-gallon cage should be about right for them. The cage temperature should be about 75-85 around the basking spot, but cooler on the other side of the cage.
You can pretty much use any type of substrate for these lizards, but I would recommend either sand (calcium sand), or lizard carpet. I personally prefer lizard carpet. It isn't too expensive, and it's easy to clean (you just soak the carpet in water during each cleaning). Lizard carpet also might be less irritating to the lizards, unlike other substrates. Lots of hiding spots are good for Alligator lizards, but I noticed that if you give them too many deep caves and tunnels to hide in you will never see them. I fix this by giving my Alligator lizards hiding spots that I can still see them in (like under leaves or in tight crevices that aren't too deep).
For food, you will need to feed your Alligator lizards about 2-3 small crickets every other day. When they get bigger, they can eat large crickets. Mealworms are ok also, but it's more fun watching the Alligator Lizards hunt down crickets. You should probably buy something for the crickets called calcium dust. It's a dust that you drop the crickets in right before you feed them to your lizards. Your lizards could get sick without calcium.
Alligator lizards will drink out of a water dish, so some sort of lid or anything with water will work. It's best if you give them enough water to swim in and out of. Don't make it too deep though, incase they can't get out.
I took a picture of my cage for you to look at. I labeled some of it to be more helpful. Your cage decorations don't have to be this fancy. You can find things outside if you are short on money. Just make sure you wash things before you put them in the cage, or else your lizards could pick up mites.

I hope this helps! I've taken care of Alligator lizards for about 6 years now. They are all very healthy and active. Hopefully yours will be the same 