First, I’ll note that holding HGSS in particularly high regard is actually less popular now than it once was. As time passes, fans move in and out of online engagement with the series, and we’ve recently seen an influx of younger fans who view the fifth generation as the pinnacle of the series, in many spaces. One such place would be the Pokémon subreddit. I’ve seen a lot of discourse surrounding the fourth generation lately that’s explicitly negative; for HGSS, this particularly centers around the alleged sins of the weird level curve in mid-to-late Johto and the fact that HMs were still expected to traverse the region.
Nonetheless, I think there’s still a huge portion of the fanbase who do view HGSS very favorably, and indeed many who view them as the best games in the series as you mention. As for why they have that reputation in the first place…
I think it all comes down to length and quantity of content, in the end. HGSS have a relatively average main story, as far as the time it takes one to get from the start to being the champion. But beyond that, they have an entire second region to explore, complete with a full set of Gym Leaders, a couple side-quest style tasks to complete, and plenty of Pokémon to catch that you can’t find in the first region. That’s what you’ll see held up as HGSS’s greatest accomplishment most of the time, that they technically have the most extensive post-game experience of any game in the franchise. In addition to Kanto, you also open up the Battle Frontier, where trainers can spend hundreds of hours trying to beat each facility’s Brain, one of the most heralded post-game experiences in Pokémon (though I know many people prefer the original Emerald Frontier). Simply put, the sheer amount of things you can do even after beating the Elite Four and Lance is nearly unparalleled within the series so far. I would personally estimate that only Black 2 and White 2, so far, have enough to do when you’re “done” with the game to even compare to HGSS, barring perhaps Galar should one choose to preserve both DLC locations until after the final battle with Leon.
Beyond that, I think that they retain a significant amount of the charm that a lot of people feel has been lost along the way as Pokémon has evolved, for various reasons. Part of this is because they aren’t particularly pretentious in their treatment of Legendary Pokémon; you’re not going out to take down a titan that will flood the world, or stop the evil team of the moment from destroying this universe to make way for his own. Nobody’s city gets frozen, there’s no possession by Ultra Beasts, there’s no terrible ancient weapon to contend with. That’s not to say any of those things is bad, but rather that HGSS just don’t try as hard to raise the stakes, and that’s an experience that can be enjoyable as well. They’re a little closer to the pure experience of being a trainer trying to overcome two sets of Gym Leaders, a Pokémon League, a rival with an attitude problem, and a legendary trainer hidden away at the peak of a mountain - you just happen to get sucked into helping out with the dying flails of Team Rocket as well. Along the way, you may also encounter the mysterious and powerful legends of the Johto region. It’s a mellower experience, and one that I think lends itself to nostalgic memories of Pokémon battles rather than one that demands your deeper engagement with the plot. Further, agency is entirely given to the player; you’re the one doing all this because you want to, not because you’re being herded around by adult characters who tease at interesting happenings but then tell you to go away while they handle it.
The rival I mentioned is also an important point. Many people think that Silver is the way a rival should be - less friendly, more antagonistic, though I think a lot of people still prefer the classic Blue attitude. That’s one of several things that HGSS do right that a lot of people feel is not done right in recent games. The antagonistic rival; the presence of mildly complex dungeons; regions you can explore with relative freedom of choice instead of forced linear progression; the “difficulty” of many of the Gym Leaders and E4; an appealing visual style free of obvious aesthetic flaws or technical limitation issues like pop-in; the lack of a forced Experience Share; these things and more are small points in the games’ favor for those looking backward from the current games.
Finally, a significant amount of their status is rooted in nostalgia. I take issue with those that would imply that the only reason they’re so respected is nostalgia, as there’s certainly a number of newer fans who mistakenly make that argument. But the truth of the matter is that it is a factor, and clouds even my own judgment of the games. Gold and Silver were hugely popular; HGSS capitalized on the childhood memories of many, many players who remain engaged with the series, and were even the first games for some fans themselves. Nostalgia is a powerful, reality-bending kind of force - it can make simply good experiences feel perfect just because you’re comfortable being in a place where everything feels familiar and right.
TL;DR: There’s a lot to do in HGSS, they do a lot of things right that people think aren’t done right these days, and nostalgia is one hell of a drug.
Hopefully this is helpful to inspire thought and maybe even a little discussion, coming from someone who enjoys these two games more than virtually any other video games bar one or two other standout titles.