The "friends." Save for Calem/Serena, they were all incredibly stereotypical anime friend characters. You had the cute, cheerful, overexcited girl. The comic relief guy, and the small, nerdy boy. I found all of them annoying and could have done without them, if they were more unique and less stereotypical, I would have liked them better, but that is not the case.
The rival wasn't much better though. Rivals have just been going downhill since gen 1 (and I'm most definitely not a "genwunner"). Blue defined what it meant to be a good rival, he wasn't overly mean, but he was competitive and cocky. Rivalry is the core of the relationship between him and Red(the PC). Calem/Serena were definitely more of a friend to the player than a rival, and they have been doing this since gen III. I don't entirely hate Calem/Serena though, I actually prefer them to the Gen VI and V rivals simply because they help you against Team Flare quite a bit, much more so than past rivals have with enemy organizations. But even though I liked that aspect, I was disappointed that I had more double battles alongside him as opposed to battles against him. And even when I did battle him, he wasn't difficult at all (even with exp share off). Not to mention, he always called me to let me know ahead of time that he wanted to battle as opposed to challenging me at random (and often inconvenient) times like past rivals. Too easy! Rivals are supposed to be at the same level as you, that's why they are a rival in the first place. Not to mention, his personality was really bland and he showed no character development whatsoever. Honestly, I wish they would have just implemented the romantic subplot with the rival. I wasn't a huge fan of the idea in the first place, but perhaps it would have added more depth to their personality of that were there.
The legendaries. I cannot express with words how disappointed I when I encountered Xerneas and found out that he was 10 levels below the lowest leveled pokemon on my team and then I caught him at full health with ONE ultra ball. The legendaries after that weren't much better, Zygrade took me 4 ultra balls and Mewtwo took around 10. I ended up using my masterball on a random pokemon in the wild because I didn't need it for the legendaries. I preferred it when legendaries were higher leveled than your pokemon when encountering them and they were actually hard to catch. It's much more exciting that way and makes the feeling of when you do finally catch them so much better.
And lastly, the overall easiness of the game. This ties in with my complaints about the legendaries and rival. Even with EXP Share and O-Powers off, the game was way way too easy. The combination of all those exp boosts and mega evolution was just too much. None of the gym leaders were challenging at all and I was able to oneshot the elite four without any of my pokemon fainting and without using any items. That should never ever happen. I like to actually feel good about earning my badges because I deleted a tough gym leader, or excited about beating the champion because he/she was so challenging. XY didn't have that, and that was very disappointing to me.
It's a lot of complaints I know. I still really like XY, they improved a lot of things. It's just a shame they felt like they had to tone down the difficulty of pokemon games. A high difficulty combined with XY graphics would have been amazing.