Honestly, it’s just a feeling. I’ve been playing this series for like 20 years now, and it’s hard for me to say that there’s any objective scale that I use. It’s all about how I felt when playing those games and how I feel when replaying them. I didn’t enjoy Black and White for example; that’s not to say they’re bad in an objective sense, because they’re certainly not. But subjectively, they’ve never evoked the same feelings as prior games in the series did, for me - I didn’t feel the same sense of wonder and excitement. I thought this might have been me growing out of the series, but then I did feel that again when I played titles like Moon, which became one of my favorite gaming experiences ever. A similar thing comes up when people talk about the quality of the story, specifically, of any given game in the series. There’s tons of praise for the Black and White story, but it’s not really for me, where Sun and Moon were my favorite narrative experience in the series. At the same time, I get why there was value in Black and White pushing forward with a deeper and more serious narrative than seen before.
So yeah, it’s hard to pin down a definitive quality or set of qualities that makes me feel the magic in a Pokémon game. I certainly can see why it makes little sense - especially to the large portion of the fanbase who really love Black and White, since I consider that one of my least favorite experiences.
If I were to try really hard to be objective, I feel like BDSP still wouldn’t be worst (hello Let’s Go!) but it would probably be ranked lower than BW pretty easily. The latter tried to do new things, reinvent the series a bit, challenge our notions of what a Pokémon game is and should be. BDSP, on the other hand, willfully fail to do anything meaningfully new or unique, and they look visually terrible at the same time.
To expand on the FRLG mention specifically though - it’s more because of Kanto than anything else. I don’t like the visual style’s differences from RSE, I don’t enjoy much at all about Kanto, and I feel more inclined to be critical of FRLG than Gen 1 Kanto experiences because I feel they could have been far more ambitious in remaking the experience, whereas RGBY were the foundation of the series’ concept. I suppose, though, FRLG did literally give us the entire concept of a remake in a new generation, so I can be thankful for that.