The "Uncle Zhengren" that Yagami Takuji mentioned refers to Yagyu Zhengren, but this person is anything but upright.
The only responsibility he has in the Yagami family is assassination.
Moreover, Yagyu Zhengren is a late-stage Gang Qi Realm martial artist, stronger than Irudi and others.
However, Yagyu Zhengren also has a flaw: he desires to become the Family Head.
When Yagami Takuji ascended to power, the factions they represented clashed several times.
Even now, their relationship is not very good.
But with the Yagami family now facing a life-or-death crisis, if Su Fan were to come for revenge, he clearly wouldn't care whether Yagyu Zhengren has a grudge against Yagami Takuji.
Another reason is that if Yagyu Zhengren could kill Su Fan, it would be ideal.
If Yagyu Zhengren were to die at Su Fan's hand, that would also be good.
Without Yagyu Zhengren, there would be no internal opposition in the Yagami family, and everyone could unite to face this crisis.