and decided to fix up an ancient one sitting in my parents attic rather than buy a new one. It's a compact with a decent enough record player. So I bought a few records to play with it (because fixing up that mechanism was the most fun part). That got me started.

I don't think it's "better" by any objective measure, however, there are a couple of things for why I enjoy it.
1) You are more or less committing to listen to a whole album (or at least half). This type of listening isn't always better than streaming, but it's fun sometimes. It doesn't sound like poo. It doesn't sound better, but if it sounds like poo, your equipment is trash.
2) Owning music has value. It's a waste when you can stream, but the collecting is fun. It's less disposable. You have to make choices; you remember what you've listened to. Curating a collection makes what you have more enjoyable even though it's limited.

Why vinyl over CDs or tapes? I think if you are going to bother having streaming and physical available to you, you might as well have two completely different experiences. A CD feels like a crappy version of streaming; holding a record feels different.

So yes, it's based pretty much entirely on feelings. Therefore it's worthwhile, but also stupid.