Might as well just say the game's biggest problem was that Arceus was announced immediately afterward and you would've gotten to the root of it all.
The deal is that BDSP had some big shoes to fill. Each remake was made in the style of the current generation's presentation - FRLG didn't look anything unlike what RS offered, HGSS looked like it was built directly off of DPPt, and while ORAS didn't fulled embrace XY's presentation it did follow through in areas to make it look/feel like a Gen 6 game.
So when you present to an audience that wanted Diamond and Pearl remakes to be just as grand if not better than Sword and Shield (as some mock up trailers showcased following BDSP's announcement, with one having a more dynamic camera, while another had more real-time battles) and you present them with "Smol Dawn" and go on about how they're aiming for "faithful remakes" while at the same time add-in the broken Affection mechanic being mixed in with Happiness since Sword and Shield and the always on EXP Share - well people are bound to get their feathers ruffled.
As for arguing that Let's Go really holds that title? I mean, let's be honest with ourselves, we've had unrealistic expectations since the Switch was even announced. Remember when Game Journalists, the moment the Switch was announced and Nintendo was all hush hush about the games until January, kept on saying they knew what games were coming to the Switch? How they said Sun and Moon, two of GameFreak's most talked about games all throughout 2016, were going to be ported to the thing?
Then we all thought we were getting Pokemon Stars on the Switch? Only to be outraged when they didn't announce it yet they opted to announce Ultra Sun and Ultra Moon on the 3DS? Even though we all said the 3DS was a dead system since the Switch was released by that point? By the way that's the real reason USUM were hated too. Ironic since the announcement of the eShop closing...
Point is, we've had high expectations for a while, and it was clear that GameFreak wasn't going to fulfill them, so they let ILCA do it while PLA essentially became a glorified tech demo for Gen 9.
That's a little presumptuous to say the reason people didn't like LGPE and BDSP is because they had sky high expectations for Switch games. There's some of that, but there's a lot more going on with them to simply conclude that the expectations were unrealistic.
For one, it's totally reasonable to expect the other Switch games to have SwSh-tier graphics. Not just because of past convention, but because of the nature of the console market that they were now transitioning to on the Switch. Console games, especially console adventure games, have traditionally been large scale, open affairs with relatively realistic graphics/proportions. You never really see a chibi, tile based console adventure game because console hardware has always been capable of more than that. Even the 5th gen consoles (the N64, PS1, etc.), with their jagged low-poly graphics that look positively repulsive today, still did not go for that kind of simplistic style of graphics and game design. And really, if you look at the big picture of what a home console is and what they're trying to sell, they kind of can't. New hardware is constantly introduced to the market with the latest and greatest tech to encourage you to buy the new one before you might naturally consider doing so. So the developers really kind of have to maximize the hardware to show that it can play games that the past gen hardware can't. And consoles especially need to do this because they're large boxes that hook up to your TV for you to sit down and play them for hours and hours. They're like the gaming equivalent of movies. They're supposed to be big and high end and impressive to justify the cost and the time. Games like SwSh, LA, and SV, while perhaps not quite what fans are expecting, still reasonably satisfy this kind of experience and visibly look and feel like something that could only be done on the Switch. Games like LGPE and BDSP on the other hand, do not and feel like they could've been made on the 3DS, undermining incentive to buy a Switch to play these games.
Beyond that though, these remakes regress in other ways unrelated to the hardware and not only do they not feel like an improvement over other remakes, they actually feel like a regression. Beyond the chibi graphics, they've been starting to exclude content from third versions and in LGPE's case, FRLG (ORAS was also guilty of this really but that doesn't excuse it, in fact that just lumps ORAS in with this), and they insist that the games are "only remakes of X version" (RS, Yellow, DP). Again, looking at the big picture of the point of a remake, this design philosophy doesn't really hold up. The point of a remake is to take an old, outdated game from a past console not being sold anymore, update it for the current console, and add it back into circulation. And not only are the games they chose to base it on outdated, but other versions of that game are as well. Emerald, FRLG, and Platinum are old outdate games no longer on the market too, so how are players supposed to experience those games now? The remakes really have to be the definitive versions of those games to properly incentivize people to buy them, otherwise again there's no reason to buy the remakes over the updated versions in question.
Additionally in LGPE's case, the design philosophy of heavily basing the gameplay off Pokemon Go and regressing certain gameplay mechanics back to the way they were in Gen 1 (no Pokemon from later generations whatsoever except Meltan/Melmetal, no hold items, no abilities, etc.) ended up being very controversial, and while the games sold well enough, you'd be hard pressed to say that they really succeeded in bringing over mobile fans to the main games in a way that a more traditional Kanto remake wouldn't given that they sold about the same as other remakes. So essentially all of that controversy was for nothing. Again, in the big picture it makes sense that this would happen, mobile and console gamers have almost completely opposite preferences so putting a mobile style game on console was a total mismatch.
None of these things are exactly asking for heaven and earth. They just don't seem to have the right design philosophy for their remakes. And given how the likes of SwSh, LA, and SV are much better received (especially the latter two) it's not really a problem of the fans expecting too much, to the contrary Game Freak/ILCA are delivering too little.