All the Mario Karts are worth getting, but yes, I was one of those guys who loved the SNES one to bits, played it to death with anyone willing to step near the console, and was strangely horrified by the 64 version.
The tracks are long and empty (apart from Moo Moo Farm and the Special Cup tracks, none are all that memorable), the karts are slower, I finished the single player in 3 days (without mastering any of the shortcuts), 4-player mode was strangely lifeless without music and the Blue Shell is just cheap.
Sorry to say, but I always thought Diddy Kong Racing was better than MK64, even though the multiplayer arenas aren't up to much.
However, I still downloaded MK64 and enjoyed playing through again, even though it did reinforce all my old beliefs. It's fun, but the weakest of the Mario Karts. However, ask me again once Super Mario Kart is downloadable - my memory may be ever so slightly rose-tinted.