Those of us who grew up with gaming in the 1990s relied on magazines such as Nintendo Power and Electronic Gaming Monthly for our news on new titles, and often those titles were teased in print before they had official titles. Everyone was talking about Zelda 3 before it was The Legend of Zelda: A Link to the Past and, and before we bought The Legend of Zelda: Ocarina of Time we knew it as Zelda 64. The thing is, these shorthand titles tend to stick. Clyde Mandelin discusses the staying power of temporary titles over at Legends of Localization.
I’m not alone, actually – many other gamers my age still refer to A Link to the Past as “Zelda III” out of habit. A large part of it is that magazines at the time called it “Zelda III” even though it had a subtitle. This led everyone to call it that on playgrounds, in homes, at workplaces, etc. Basically, for a short period, “Zelda III” was what almost everyone called it, at least for short.
A similar thing happened with “Link’s Awakening” and “Zelda IV”. My recollection is that this numerated title didn’t stick around nearly as long, so it’s rarer to hear anyone refer it to as “Zelda IV” these days.
While this kind of thing isn't exclusive to the Zelda franchise (Super Mario World was colloquially called Super Mario Bros. 4 before its debut and the game even carries that subtitle in Japan), the wordy titles of the franchise tend to make the numerical shorthand something of a necessity. Somehow I expect that if the third Sonic the Hedgehog game was entitled Sonic the Hedgehog and the Floating Island, we'd still refer to it in passing as Sonic 3. There are too many Zelda games today to count them off in sequential order, but for a while there it was a handy tool.