Actually there's a thought, if all these people are refugees living in a hiding place, why do they need to appear as Victorians? If no one ever comes there and those who do get their memory wiped?
Anyway, the idea of the Doctor no longer being there and him being the Warrior again, the threat of the re-emergence[...]
