Surely they couldn't have lost Sherlock in the beach house. It wasn't big enough to lose a child in. There wasn't even a first floor.
One can tell Siger doesn't spend all that much time with the kids!

“I want to be a crab,” he said.
“Well, you're well on your way, personality-wise,” Siger said.
Can't argue with that!

Yes, little Sherlock is even more difficult than little Mycroft. Siger gets there, though.