Thank you for the conjecturing :) Abraham ------------- Abraham Williams | Hacker Advocate | http://abrah.am @abraham | http://projects.abrah.am | http://blog.abrah.am This email is: [ ] shareable [x] ask first [ ] private.
On Thu, Sep 30, 2010 at 14:57, Taylor Singletary < taylorsinglet...@twitter.com> wrote: > Of course :) > > I'll arm chair conjecture a bit also, since while I do work here I'm > definitely not the caliber of engineer as my colleagues, and certainly > not very knowledge in what it takes to scale a service like Twitter: > > Things like followers/ids and friends/ids are likely accessible easily > because the social graph is a necessary component to delivering tweets > properly: who gets what, in what way, on what timeline, etc. The > entire historical archive of tweets for a given user is not required > to remain in memory to perform these mission critical tasks -- whether > that's just ids of tweets or otherwise. > > Again. Don't know nothing about nothing, but this conjecture seems > reasonable. > > Taylor > > On Thu, Sep 30, 2010 at 2:35 PM, Abraham Williams <4bra...@gmail.com> > wrote: > > Us outsiders have to get our pokes and prods in while we can :-P > > > > Abraham > > ------------- > > Abraham Williams | Hacker Advocate | http://abrah.am > > @abraham | http://projects.abrah.am | http://blog.abrah.am > > This email is: [ ] shareable [x] ask first [ ] private. > > > > > > On Thu, Sep 30, 2010 at 14:19, Taylor Singletary > > <taylorsinglet...@twitter.com> wrote: > >> > >> Theorizing from the outside-in on our capacity issues aside, I'm a big > >> advocate for a bulk status/show or lookup function. We're definitely > >> giving that a lot of thought at the moment. > >> > >> Taylor > >> > >> On Thu, Sep 30, 2010 at 2:10 PM, Abraham Williams <4bra...@gmail.com> > >> wrote: > >> > Considering Twitter can support returning the ids of almost 300,000 > >> > followers then 40,000 tweets should be easy. > >> > http://api.twitter.com/1/followers/ids.json?screen_name=rsarver > >> > > >> > Abraham > >> > ------------- > >> > Abraham Williams | Hacker Advocate | http://abrah.am > >> > @abraham | http://projects.abrah.am | http://blog.abrah.am > >> > This email is: [ ] shareable [x] ask first [ ] private. > >> > > >> > > >> > On Thu, Sep 30, 2010 at 14:04, Orian Marx (@orian) < > or...@orianmarx.com> > >> > wrote: > >> >> > >> >> This should probably be a separate thread, but... what about a bulk > >> >> tweet lookup using status_ids for one very specific use case: turning > >> >> search results into proper tweets and avoiding all the other issues > >> >> that exist with the current implementation? I know bulk tweet lookup > >> >> by id has been asked before and the response is it's the same > capacity > >> >> issue of accessing old tweets from storage. Okay, so what about a > bulk > >> >> tweet lookup limited to timestamps that fall within whatever range > >> >> search capacity is currently at (ya know, like the last seven days)? > I > >> >> bet this could actually reduce the number of calls developers are > >> >> making when dealing with converting search result items into proper > >> >> status objects. > >> >> > >> >> On Sep 30, 4:57 pm, Taylor Singletary <taylorsinglet...@twitter.com> > >> >> wrote: > >> >> > I'll agree with all of you that'd it be valuable for us to do this. > >> >> > > >> >> > The current state of availability of tweets is a capacity issue. > It's > >> >> > not in anyway a deliberate prevention of access. > >> >> > > >> >> > As for since_id in this context -- it'd be great if it'd work to > just > >> >> > use since_id=1, but it doesn't. I don't even think it'd work if you > >> >> > knew the very first status id that the user had ever posted.. > because > >> >> > in most cases, that status id would be unknown to the back end > >> >> > transport layer (for the same reason it can't access the tweet > >> >> > itself). > >> >> > > >> >> > If we could provide a means today to allow complete traversal of a > >> >> > user_timeline for a specific user with over ~3200 tweets, we would > -- > >> >> > and I'm pretty confident we will. But not today and not tomorrow. > >> >> > > >> >> > Taylor > >> >> > > >> >> > On Thu, Sep 30, 2010 at 1:43 PM, M. Edward (Ed) Borasky > >> >> > > >> >> > > >> >> > > >> >> > <zn...@borasky-research.net> wrote: > >> >> > > Retrieving one's GMail, for example, if one wants to delete the > >> >> > > account, is > >> >> > > a decidedly non-trivial exercise. Perhaps it's simple with > Outlook, > >> >> > > but > >> >> > > KMail, Evolution and Thunderbird all gave me a ration of doo-doo > in > >> >> > > the > >> >> > > process, and I'm not sure I did it right. > >> >> > > -- > >> >> > > M. Edward (Ed) Borasky > >> >> > >http://borasky-research.nethttp://twitter.com/znmeb > >> >> > > >> >> > > "A mathematician is a device for turning coffee into theorems." - > >> >> > > Paul > >> >> > > Erdos > >> >> > > >> >> > > Quoting "Orian Marx (@orian)" <or...@orianmarx.com>: > >> >> > > >> >> > >> It remains a good idea. Imagine if Gmail only let you retrieve > >> >> > >> your > >> >> > >> last 3200 messages even if you had 40,000. > >> >> > > >> >> > >> On Sep 30, 4:20 pm, "M. Edward (Ed) Borasky" <zn...@borasky- > >> >> > >> research.net> wrote: > >> >> > > >> >> > >>> I also posted a request a long time ago that an authenticated > >> >> > >>> user > >> >> > >>> be > >> >> > >>> able to retrieve all of his own tweets, back beyond the 16-page > >> >> > >>> limit. > >> >> > >>> In retrospect, now that I'm within shooting distance of 40,000 > >> >> > >>> I'm > >> >> > >>> not > >> >> > >>> sure how good an idea that is. ;-) > >> >> > > >> >> > >>> -- > >> >> > >>> M. Edward (Ed) > >> >> > >>> Boraskyhttp://borasky-research.nethttp://twitter.com/znmeb > >> >> > > >> >> > >>> "A mathematician is a device for turning coffee into theorems." > - > >> >> > >>> Paul > >> >> > >>> Erdos > >> >> > > >> >> > >>> Quoting Abraham Williams <4bra...@gmail.com>: > >> >> > > >> >> > >>> > Here is a closed feature request from forever ago to return > the > >> >> > >>> > status_id of > >> >> > >>> > all statuses for a user. Maybe Twitter wil reconsider the > >> >> > >>> > request. > >> >> > > >> >> > >>> >http://code.google.com/p/twitter-api/issues/detail?id=379 > >> >> > > >> >> > >>> > Abraham > >> >> > >>> > ------------- > >> >> > >>> > Abraham Williams | Hacker Advocate |http://abrah.am > >> >> > >>> > @abraham |http://projects.abrah.am|http://blog.abrah.am > >> >> > >>> > This email is: [ ] shareable [x] ask first [ ] private. > >> >> > > >> >> > >>> > On Thu, Sep 30, 2010 at 11:32, linuslive > >> >> > >>> > <michael.c....@gmail.com> > >> >> > >>> > wrote: > >> >> > > >> >> > >>> >> Hmmm...but there is no way to know the ids of the tweets > >> >> > >>> >> without > >> >> > >>> >> having a list of all of the tweets, which would kind of > defeat > >> >> > >>> >> the > >> >> > >>> >> purpose of the limit. > >> >> > > >> >> > >>> >> Oh well... > >> >> > > >> >> > >>> >> On Sep 30, 12:39 pm, Tom van der Woerdt <i...@tvdw.eu> > wrote: > >> >> > >>> >> > Only the 3200 most recent ones, unless you know the IDs of > >> >> > >>> >> > the > >> >> > >>> >> > tweets. > >> >> > > >> >> > >>> >> > Tom > >> >> > > >> >> > >>> >> > On 9/30/10 5:09 PM, linuslive wrote: > >> >> > > >> >> > >>> >> > > Good morning all! > >> >> > > >> >> > >>> >> > > Before the OAuth change, I wrote a twitter archiver that > >> >> > >>> >> > > would > >> >> > >>> >> > > grab my > >> >> > >>> >> > > tweets and dump them into a database. Back then I only > >> >> > >>> >> > > had > >> >> > >>> >> > > about > >> >> > >>> >> > > 1K > >> >> > >>> >> > > tweets so I was able to grab all my tweets without > hitting > >> >> > >>> >> > > the > >> >> > >>> >> > > limit. > >> >> > >>> >> > > And of course since then, I don't post more than 20 > tweets > >> >> > >>> >> > > a > >> >> > >>> >> > > day. > >> >> > > >> >> > >>> >> > > But today, I was curious to see if I could use the new > >> >> > >>> >> > > OAuth > >> >> > >>> >> > > to > >> >> > >>> >> > > retrieve my timeline, starting from my first tweet. I > >> >> > >>> >> > > can't > >> >> > >>> >> > > seem > >> >> > >>> >> > > to > >> >> > >>> >> > > get it. > >> >> > > >> >> > >>> >> > > Here are my parameters: > >> >> > > >> >> > >>> >> > > > >> >> > >>> >> > > > > >> >> > >>> >> > > > > > > http://api.twitter.com/1/statuses/user_timeline.json?screen_name=linu. > >> >> > >>> >> .. > >> >> > > >> >> > >>> >> > > When I do this, it only gives me the top 200 tweets. It > >> >> > >>> >> > > doesn't > >> >> > >>> >> > > start > >> >> > >>> >> > > from the beginning and give me my 200 oldest tweets, as > I > >> >> > >>> >> > > would > >> >> > >>> >> > > expect. > >> >> > > >> >> > >>> >> > > I've tried various since_id's, even retrieving my first > >> >> > >>> >> > > tweet: > >> >> > >>> >> > >http://twitter.com/linuslive/statuses/872090728 > >> >> > > >> >> > >>> >> > > Which equals > >> >> > > >> >> > >>> >> > > > >> >> > >>> >> > > > > >> >> > >>> >> > > > > > > http://api.twitter.com/1/statuses/user_timeline.json?screen_name=linu. > >> >> > >>> >> .. > >> >> > > >> >> > >>> >> > > And I get the same response. > >> >> > > >> >> > >>> >> > > According to twitter, I have 3682 tweets, just over the > >> >> > >>> >> > > 3200 > >> >> > >>> >> > > limit > >> >> > >>> >> > > according > >> >> > >>> >> > > tohttp://dev.twitter.com/doc/get/statuses/user_timeline > . > >> >> > >>> >> > > The confusing thing is that the documentation does not > >> >> > >>> >> > > make > >> >> > >>> >> > > it > >> >> > >>> >> > > clear > >> >> > >>> >> > > that it's 3200 of the most recent tweets, or just 3200 > per > >> >> > >>> >> > > session. > >> >> > > >> >> > >>> >> > > Could I get a clarification on this? > >> >> > > >> >> > >>> >> > > Thanks!- Hide quoted text - > >> >> > > >> >> > >>> >> > - Show quoted text - > >> >> > > >> >> > >>> >> -- > >> >> > >>> >> Twitter developer documentation and > >> >> > >>> >> resources:http://dev.twitter.com/doc > >> >> > >>> >> API updates via Twitter:http://twitter.com/twitterapi > >> >> > >>> >> Issues/Enhancements Tracker: > >> >> > >>> >>http://code.google.com/p/twitter-api/issues/list > >> >> > >>> >> Change your membership to this group: > >> >> > >>> >>http://groups.google.com/group/twitter-development-talk > >> >> > > >> >> > >>> > -- > >> >> > >>> > Twitter developer documentation and > >> >> > >>> > resources:http://dev.twitter.com/doc > >> >> > >>> > API updates via Twitter:http://twitter.com/twitterapi > >> >> > >>> > Issues/Enhancements > >> >> > >>> > Tracker:http://code.google.com/p/twitter-api/issues/list > >> >> > >>> > Change your membership to this group: > >> >> > >>> >http://groups.google.com/group/twitter-development-talk > >> >> > > >> >> > >> -- > >> >> > >> Twitter developer documentation and > >> >> > >> resources:http://dev.twitter.com/doc > >> >> > >> API updates via Twitter:http://twitter.com/twitterapi > >> >> > >> Issues/Enhancements Tracker: > >> >> > >>http://code.google.com/p/twitter-api/issues/list > >> >> > >> Change your membership to this group: > >> >> > >> http://groups.google.com/group/twitter-development-talk > >> >> > > >> >> > > -- > >> >> > > Twitter developer documentation and > >> >> > > resources:http://dev.twitter.com/doc > >> >> > > API updates via Twitter:http://twitter.com/twitterapi > >> >> > > Issues/Enhancements Tracker: > >> >> > >http://code.google.com/p/twitter-api/issues/list > >> >> > > Change your membership to this group: > >> >> > >http://groups.google.com/group/twitter-development-talk > >> >> > >> >> -- > >> >> Twitter developer documentation and resources: > >> >> http://dev.twitter.com/doc > >> >> API updates via Twitter: http://twitter.com/twitterapi > >> >> Issues/Enhancements Tracker: > >> >> http://code.google.com/p/twitter-api/issues/list > >> >> Change your membership to this group: > >> >> http://groups.google.com/group/twitter-development-talk > >> > > >> > -- > >> > Twitter developer documentation and resources: > >> > http://dev.twitter.com/doc > >> > API updates via Twitter: http://twitter.com/twitterapi > >> > Issues/Enhancements Tracker: > >> > http://code.google.com/p/twitter-api/issues/list > >> > Change your membership to this group: > >> > http://groups.google.com/group/twitter-development-talk > >> > > >> > >> -- > >> Twitter developer documentation and resources: > http://dev.twitter.com/doc > >> API updates via Twitter: http://twitter.com/twitterapi > >> Issues/Enhancements Tracker: > >> http://code.google.com/p/twitter-api/issues/list > >> Change your membership to this group: > >> http://groups.google.com/group/twitter-development-talk > > > > -- > > Twitter developer documentation and resources: > http://dev.twitter.com/doc > > API updates via Twitter: http://twitter.com/twitterapi > > Issues/Enhancements Tracker: > > http://code.google.com/p/twitter-api/issues/list > > Change your membership to this group: > > http://groups.google.com/group/twitter-development-talk > > > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk > -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
