Since the Lannister Family can poison Jon Alyn, they can actually threaten King Robert as well, since Queen Cersei and Jam, one of the Royal Forest Iron Guard, are part of the Lannister Family.
They are all people who can be in close contact with him!
"We cannot neglect the Desolation Wall, it is our duty, but I cannot just watch Robert fall into the pit. He will come to me, which means that there aren't many people around him whom he can trust..."
Ed Stark also had his own perspective in mind.
The King personally came to invite him to be the Hand of the King, which actually showed his respect for him.
In the end, Ed decided to wait at Winterfell for King Robert to visit, and make clear the situation at the Desolation Wall face-to-face...
——
"Huff~ Huff~"
Bran Stark opened his eyes, sat up on the bed, and began to breathe heavily.