Don't worry about it. If you're REALLY paranoid just change your password to something about 8 characters long using Capitals, symbols ($%^# etc) and numbers. A password like that would literally take 200 years to brute force hack into, unless they know more advanced methods, which are completely illegal (and again its not hard for us to retrieve a hacked profile).
Also hacking isn't like they make it seem on TV. For one hes likely full of crap and isn't going to do anything, but in case he does try a "brute force" password cracker its literally impossible if your password has some random numbers and letters. If by some miracle he does get your password, its easy enough to tell and we can get it back for you.