I think that they are great additions to the game.
Competitively, of course, they are crucial. EVs and IVs are basically your stat boards and how you customize a Pokemon. Many Pokemon can run different sets, and essentially customizing the EVs and IVs can really help you. For example, EVing a Pokemon to be able to survive a hit from a certain Pokemon with just enough HP to KO that Pokemon, or giving a Pokemon a certain IV stat so that, under Trick Room, it just barely outspeeds a Pokemon you need to outspeed while underspeeding everything else. Things like that. Without EVs and IVs, the game would seriously turn into a numbers game based greatly on which side of the coin flip you got. Will you or will you not outspeed? Will you or will you not survive? There is no making sure that a Pokemon can do this or that; it would seriously be "it does this, let's hope the RNG doesn't screw me over." if EVs did not exist.
Now, casually, canonically, spiritually, etc. If you view Pokemon not as a game but as an alternate reality. EVs and IVs are still essential.
Wait how?
They are what gives each Pokemon its characteristic, its personality. Those are based off of the IVs a Pokemon has when it is caught or hatched.
EVs, on the other hand, is special training. If you put a Pokemon on a treadmill everyday, it would get faster and its endurance would increase. If you made a Pokemon lift weights or punch a punching bag everyday, its strength would increase. If you made a Pokemon sit under a waterfall, its endurance would increase. If you made a Pokemon sit down and use Flamethrower all the time, its ability to use Flamethrower would grow. Things like that. Special training means that they will gain a specific benefit. And that's what the EVs are; special training.
Now let's compare it to real life. If you, as a person, do not do any sort of special training (EV training), you will either not gain any special benefit, or you would receive a diminished amount of special training spread across different areas. You would have no specialization. Now, if you undergo special training, you will see that special benefit. Basically, that is the end result to EV training.
Now why does that matter in the casual/canonical/imaginative part of the fandom? Because it adds another texture to the Pokemon world. The headcanon if you will. A Pokemon who has not undergone any form of special training by their trainer will more than likely struggle to perform the task a Pokemon who has undergone the special training will. The Pokemon just isn't raised that way. So if you're worried about that sense of Pokemon, it is still important. It shows that Pokemon does have that extra layer, setting a difference between the normal Pokemon and the "specially trained" Pokemon.
It also brings that "Hometown Hero, World-class Zero" element to Pokemon. That sounds mean because I can't think of a catchier way to say it, so bare with me. Let me explain.
By not EV training your Pokemon, you can be very good on a lower level. You can have the basic concepts down, have a semblance of strategy, synergy and such, but you're not in the big leagues. Like how in Pokemon, there are trainers who do well in their small hometowns, but do not add up when pitted against world class trainers. "Hometown Heroes" represent those Pokemon trainers who are good on the hometown level. Nothing to be ashamed of, this is where the vast majority of players sit anyways.
The "World Class" trainers aren't exactly worldclass. They just get the competitive aspect, play under it, and thus can "technically" stand up against the big boys. They're the trainers who have the EVs and IVs down, they get that whole concept of competitive battling. The Hometown Heroes are at a significant handicap while playing against these trainers, although that doesn't mean they can't win. It's possible that they could, just highly unlikely.
Without EV training and IV breeding, that wouldn't be possible. For some, this sucks, because they want everything to come easy to them. For others, this is great, because it does add that dimension, and for those of us who deal with it and work hard, we get to reap the rewards of putting in the hard work.