It frustrated Xavier. He couldn't let down his walls with someone else watching, so instead of breaking down, he redirected that tension by bickering with Alexandre. Ironically, without realizing it, the exchange helped ease the tightness in his chest. It wasn't the emotional release he expected, but it was something, and it left him feeling just a little lighter.
As for Alexandre, his refusal to leave wasn't out of disregard for Xavier's emotions. The investigation into Hera's accident was still ongoing, and he wanted to be the first to update Xavier and the rest of Hera's close circle and her harem with any findings. Maybe it was strange, even to himself, that he was being so considerate of his love rivals. But deep down, he understood: if the others couldn't accept him, Hera might never fully accept him either.
