And I'm strictly talking PS2 V PSP in this thread. No XBOX, no Wii, no PS3 no whatever. It's just that if you want to buy one of those NOW, you should go for the PSP. The PS2, despite being a great console, is no longer worth it. Unless you find it for a terrific price and you so desperately want to play a few games
But if you buy a PS2, you need to have older games. Who would go buy GT3 now? OR GTA Liberty City? Those two were my favourite solo games, I still have them and when I'm in the mood I play them. Now we're going to have GT5, on the PS3... It's all about logic and evolution. I wouldn't go buy the Nintendo64 now, no games are out and it's very hard to find the older ones. This will be the case with the PS2 in 2-3 years. Unless you want the PS2 to play a couple of games only, then there's no reason to buy it at this moment, if you didn't own it before(therefore have PS2 games). Yet the PSP is something newer, and I don't think that there's something SONY will launch a superior version.
So strictly PSP Vs PS2, I'd go for PSP, despite any past I had with the consoles.
Get PS3 if you are still undecided (and want to go for SONY)