If you are self-milling like in a Lightsworn deck, I guess that you shouldn 't have more than 50 cards, because the Lightsworns benefit themselves from the number of cards in your graveyard. Also, never forget that Judgment Dragon needs the Graveyard.Can't 50+ be an advantage for a mill deck, though? You don't get milled as fast, anyway, but it does lower your chances of drawing what you need. Might be solvable with a handful of search cards tossed in, though.
If you are milling your opponent, ALWAYS have a number of cards a bit over your opponent 's number. This way, even if you don 't get your milling funtioning perfectly (the opponent uses Skill Drain, in a Example) all you have to do is stall for time, like with Waboku, Swords of Revealing Ligh or Wall of Revealing Light.