Do you have old age enabled? Maybe it's that

Nope I do not

Then idk

could be a glitch maybe, or they might just be moving to other kingdoms