They kinda forgot

Biggest missed opportunity in the series, seeing as I'm pretty sure Jon will learn his parentage from Howland Reed in the novels, as he's the last survivor from the Tower of Joy.