I agree. He really needs to adjust to his new environment - and you. Think of it this way - from his point of view: some other big creature had him and then he ended up with you, another big creature, who put him in another new home. Before the last one you don't know how often or how many big creatures he had been with or if he'd been snatched from the wild and just flew in after a whirlwind tour of the globe. So, don't you think you would need a little down time? LOL! 
I know it is VERY tempting to want to hold him and have him climb on you and make him happy, but he will be happier if you let him do what he wants to do - especially in the beginning so he knows that this big creature isn't going to pass him off to another one.
-----
0.1 veiled - Luna
0.2 house geckos - Gaia & Tia (MIA 
0.2 felines - Kyndra and Líta