If I worked for Twitter, here's what I would have done.
I would have grabbed the follower id list of the large accounts (those
that usually kicked back 502s) and written them to flat files once
every 5 or so minutes.
When an API request comes in for that list, I'd just grab it from the
flat file, instead of asking the DB to select 2+ million ids from
amongst a few billion records, while it's trying to do a few thousand
other selects at the same time.
That's one way of getting rid of 502s on large social graph lists.
Okay, the data is going to be 5 minutes out-dated. To that I say, so