The first thing would grab my attention would (naturally) be the graphics. However, I know that although some sites can look really pretty, their organisation might not be the best — so that's the second thing I look for: organisation. If one isn't too organised and everything is clumped together, it really turns me off.
The third thing I look at is the clarity of the rules: if the rules aren't too clear, then I'm not joining because I'd have too many questions and most of them tend to go unanswered or even if they were answered, it sounds wishy-washy and not firm. The fourth thing I look at is the activity level of the members as well — is one member playing too much characters? Because there have been cases were on a site, I feel like I'm writing with just one person because they hold the majority of the characters on a site.
Another key thing I look at is moderation and creativity as well. While I do appreciate special events on a site, I also like it if there was an actual overall plot for all members to participate in while having their own personal character arcs as well — it also means there's a lot of material to fall back on as well — and that is for the creativity section. As for moderation, there needs to be a limit to make it fair to everyone, in regards to their characters and activity. For instance, I'm a fan of this sort of moderation: if someone cannot handle, let's say, three characters... just archive one and make sure that they focus on those two characters before they're allowed to make another one.
I think I have a lot more but these are the general guidelines I've set for myself when looking at RP sites so far. I also look at the writing levels of the site; not too picky about this because writing can be improved; it's just a matter of whether a person wants to or not.