If I recall correctly, Azurill needs six IQ Stars, not four, in order to evolve. Different Pokémon that evolve by IQ require different amounts of IQ Stars, ranging from four to six, before they can evolve. To see if a Pokémon has enough IQ to evolve, check its status; if it says "Evolution: Not Now," then it requires more IQ Stars.
By the way, are you feeding your Azurill White Gummis? Many people think that Azurill is a Water-type, but it is actually, oddly enough, a Normal-type; as a result, White Gummis are best for quickly boosting its IQ.