He couldn't stand the urge to touch her in her pajamas, and it got the best of him