Peter, Red  Caboose Was across the street up stairs. in the early eighties while in school I used to visit it on lunch breaks once in a while. I happened on one of those days to see the street blocked off and fire trucks pumping water into the building. Yes it was unfortunate that a diamond district jewelers company  (?) was on fire directly above the old train store! The water damage was a catastrophic event! What a mess with tons of diamonds in the gravitational water pressure!  I can only imagine the clean up and insurance claims!  Many months later the store reopened across the street where it is now.  The difference was there was a large ground story display window and no displays on the stairs. What a fantastic place and history. Alan is a great guy to talk to. The shop is awesome. 
back in the early eighties I happened to visit the shop for the first time out of curiosity and bought a model craftsman mag that started a lifetime of enjoyment. I did shop there when I lived in NYC. 
go check it out! 

Ah, the Model Railroad Equipment Corporation, Carmen Webster, and Herb.  Herb was the main clerk during the 40's and 50's when I lived in the area.  He knew where every nut, bolt, and screw was stashed away ... and could instantly produce them -- with kindness -- even for a teenager.  And that was in the days when non-adults were supposed to be seen but not heard.

Another hobby shop (forget the name) opened upstairs across the street after WWII, but it didn't have nearly the range of trains as MRE.  Not really a place to look and linger like being in the basement across the street.

A good walk down 5th Avenue brought you to Polk's Hobby Store, which had 5 floors devoted to hobbies.  IIRC, trains were on the 2nd floor.  Further on were the American Flyer and Lionel stores and, of course, Macy's and Wanamaker's during the Christmas season. 

Macy's was where I got my 1st exposure to MR, HO Scale trains, and the New York Society of Model Engineers right after WWII.  I still remember the Mantua display layout which had an automated yard to prove the reliability of their hook and loop couplers.  IIRC, the NYSME layout was still in a Manhattan basement.  It my first encounter with what was the closest thing to prototype modeling at that time.

Good to hear that Red Caboose still carries on, at least for the moment.  Don't expect to be able to visit it with my feet planted about 3000 miles away.  Tempus fugits.

Polk’s hobby store! 5 th avenue. 23rd street ish?essentially a 5 story brownstone building among  bigger looming buildings.They had a train floor on I believe the second floor, mostly for  HO enthusiasts. Lots of parts and hobby tools for layouts. Small amount of O gage kits some Lionel. I don’t remember seeing European stuff, Now the  top floor had a raceway for 1/24 (?) slot cars, not home table type variety. Lots of  car kits, parts, models. Cool stuff. I remember the narrow stair case always being pounded by kids up to the top floor and down. It had a particular sound! Always an overly busy place on Saturdays!

I spent most of my time up stairs racing my RTR COX green lotus car with replaced silicone tires and a motor a friend of mine rewound, and some special gears that were well lubed! Zoom.  Good memories.

It would be exciting to have a pro race that you would pay to enter. It would get heated when a car would wipe out and then wipe out others during the races! 
I recall a brawl that happened there in such a circumstance out in the stair case! I squeezed by that while peace and restraint were being made....I remember hiding in the train dept while that was going on. They had to cancel the races, no one got trophies.  No body won that day.

but I enjoyed the train stuff.

I remember one summer probably '69 or '70 my friend was running the slot car floor at Polk's.  He was a factory racer for Russkit.  Now he makes some of the best stereo speakers.

Before Covid I was teaching courses on two campuses in NYC, and Red Caboose was on a viable pedestrian route between my two destinations.  Such luck to be able to just walk in as often as twice a week!  I mostly bought current magazine issues, but occasionally I would need some obscure MicroTrains conversion kits or some such and he'd usually have them.  Maybe once Covid lets up ...

VK

Was never at Red Caboose, but this post reminded me of several hobby shops I was in many years ago that were jaw dropping in their inventory.  When entering had that same feeling I had as a kid going into a toy store.  All long gone.  Kind of sad, and showing my age.

Wilmington, DE   Mitchells

Chicago, IL  Downtown Hobby

Denver, CO  Caboose Hobbies

Maybe in the end with EBAY and internet it is actually easier.  But I miss those days.

Don't forget Caboose (hobbies) - well, they didn't last long.

I loved your line:

"I always find something I did not know I was looking for"!

Been there, done that.

I was last in NYC in 2014, one afternoon while my wife was in Macy's I went over to check it out and: IT WAS CLOSED! Sort of like the fish that got away!

I have to put in my $.02 here. I grew up on Long Island and during the late 70's early '80's I often traveled into Manhattan on the LIRR (What can be better than riding a train to shop trains in the big City!) to visit the three shops on the same street (45th IIRC) Ma Barkers MREC shop was at street level and was packed with stuff. Bought my first piece of Brass there, a NJ Custom LIRR G53sd (Paid $89). Still have it although it took a trip to the floor and has seen better days. I was a member of the West Island club in Farmingdale (Now in Hicksville). I would visit all three shops, including the Roundhouse across the street one flight up and then the Red Caboose that you had to ride a small elevator to get to. The elevator opened into the shop. All three stores had copious amount of brass and I would often shop all three before making my purchase. Those were good times for brass buyers, never to be seen again. The guy who ran the Red Caboose was grumpy and got pissed off when I purchased a magazine. He always turned me off with his  attitude (Although the other shop clerks were not much better). Once I was about to pull the trigger on a piece of Brass and he was so obnoxious that I went elsewhere to a store in the classifieds of MR magazine to purchase the piece. The trip was fun but eventually the roundhouse closed and took over the MREC space. Stopped going there after a few years so missed out on the owners shuffle that ended up with Red Caboose where it is today. I miss having those stores and Willis Hobbies in Mineola. I am in PA now, no remotely decent hobby shops in my AO.