I think what happened was that Meera was pleading for Bran to warg into present-day Hodor, but because Bran was "observing" in the past there was some sort of weird-ass time travel paradox and Wylis got affected as well. Time travel is supposed to make your head hurt and this was no exception. This revelation has spawned plenty of theories including:

(i) that Bran and Brandon the Builder are one and the same;
(ii) Bran was the voice in the Mad King's head; and
(iii) that Meera may be Jon's twin sister.