Hi, Abraham, and everyone.

I'm crawling twitter. (But who isn't, right?) Us social graph geeks
have our own advantages, and our own set of challenges.
For example, I would not want to manage the vastness of tweet volumes.
But I do get neck-deep in social graph data. Which means I crawl with
this:
http://twitter.com/friends/ids.xml/?user_id=12345
x20,000/hr.
so far i've discovered existance of 51 million accounts, and examined
13 million of these. if i need two scrapes to determine account
activity, then i've got just 89 million captures to go! that's 6
months at full speed.

inactive accounts can live with a vastly slower refresh cycle.
so really what would benefit me (and twitter, as i see it) is a cheat
sheet of active vs. inactive accounts.
download the file, and know the integers within it are active
accounts.

in one move, through occasional publication of one file, twitter saves
6 months of scrapes for anyone who can leverage a quick-start list of
which accounts are active, and which are inactive. i imagine people
could, in many scenarios, limit their entire set of inquiries to these
active accounts, saving millions of calls to twitter's api.

maybe it's bad p.r. to state explicitly which accounts merit resources
and which are dead.

i guess once it's over i won't look back and perhaps it is i who can
publish this dataset to some other newbie. but what a great efficiency
for twitter to avoid this for everyone in my shoes. which are small
shoes, i accept.

best regards,
john






On Mar 23, 11:56 am, Abraham Williams <4bra...@gmail.com> wrote:
> Bulk lookup of social graphs seems like it would be a pretty resource
> intensive call. I would not hold my breath for Twitter to implement it.
>
> Abraham
>
> On Tue, Mar 23, 2010 at 08:21, Orian Marx (@orian) <or...@orianmarx.com>wrote:
>
>
>
>
>
> > Thanks Abraham, don't worry I'm watching Intersect closely ;)
>
> > Unfortunately, this doesn't currently address what I'm getting at,
> > namely, if I use the bulk user lookup, I'd like to similarly get
> > accurate friend / follower info for each of those users (relative to
> > the user making the bulk lookup) in one call.
>
> > On Mar 22, 11:00 pm, Abraham Williams <4bra...@gmail.com> wrote:
> > > I provide a simple API that returns common friends and follower of two
> > > specific Twitter users. It currently works for the 5000 most recent
> > > (although soon to be increasing) and only on public accounts.
>
> > >http://github.com/abraham/intersect/blob/master/README
>
> > > <http://github.com/abraham/intersect/blob/master/README>Abraham
>
> > > On Mon, Mar 22, 2010 at 19:41, Orian Marx (@orian) <or...@orianmarx.com
> > >wrote:
>
> > > > The bulk users/lookup call recently added to the API is a great new
> > > > tool for developers. This call would become even more useful with a
> > > > corresponding bulk lookup for user relationships. Are there any plans
> > > > for this?
>
> > > > Also, I'm assuming that the <following> and <notifications> nodes
> > > > returned in the user objects of the users/lookup call should be
> > > > considered unreliable as is stated for users/show.
>
> > > > Thanks,
> > > > @orian
>
> > > > To unsubscribe from this group, send email to twitter-development-talk+
> > > > unsubscribegooglegroups.com or reply to this email with the words
> > "REMOVE
> > > > ME" as the subject.
>
> > > --
> > > Abraham Williams | Community Advocate |http://abrah.am
> > > TwitterOAuth |http://github.com/abraham/twitteroauth
> > > This email is: [ ] shareable [x] ask first [ ] private.
>
> > To unsubscribe from this group, send email to twitter-development-talk+
> > unsubscribegooglegroups.com or reply to this email with the words "REMOVE
> > ME" as the subject.
>
> --
> Abraham Williams | Community Advocate |http://abrah.am
> TwitterOAuth |http://github.com/abraham/twitteroauth
> This email is: [ ] shareable [x] ask first [ ] private.- Hide quoted text -
>
> - Show quoted text -


-- 
To unsubscribe, reply using "remove me" as the subject.

Reply via email to