Honestly I've never gotten past what is played on the radio with Prince. Give me a couple songs to check out.
You could do worse than getting The Hits/The B-Sides, a 3 disc set from '93. It doesn't have the last 2 decades, but it definitely covers his most successful periods. As for later stuff, I'd say get the album Musicology, and I really enjoyed PLECTRUMELECTRUM from a couple of years ago.
Specific tracks? Geez. OK. I'll list Track - Album, and I'll put a * next to the ones that might be offensive to those who don't like blatant sexuality. I'm just scrolling thru the albums, so they'll be in alphabetical order by album. I'm not going to bother listing anything from Purple Rain, because every track is amazing.
Free - 1999 (Really, anything on this album is phenomenal)
Controversy - Controversy
When You Were Mine - Dirty Mind
Head - Dirty Mind*
Musicology - Musicology
PRETZELBODYLOGIC - PLECTRUMELECTRIC (The lyrics are crap, but the sound is phenomenal)
I Wanna Be Your Lover - Prince*
Sign O The Times - Sign O The Times
If I Was Ur Girlfriend - Sign O The Times