There is a reason why the foundation of sects is the control they have on the only five secret realms. In summary, the advantage that sects have is that they have access to an environment with a high concentration of mana.
There is mana in the world. But in this region of the world, it has reached at most rank 2 in concentration. That means, it can only support those who are in the second mortal coil to cultivate.
With this kind of concentration, valuable plants and materials that are rank 3 can't grow outside secret realms. Energy stone mines also are very scarce.
So the current environment is not enough to support cultivators at the third mortal coil. The only option is to gain control of a secret realm or move closer to the Ethera tree.
For families with weak strength like theirs, they can't gain control of secret realms, so they would likely be destroyed if they moved closer to the Ethera tree.
