Re: [twitter-dev] Re: Can't access old tweets via statuses/user_timeline
I have over 20,000 tweets under my belt. Even whitelisted at 20,000 API calls per second ... I'm thinking Twitter could charge a dollar a thousand for a full backdump like that - I'd pay $20 to get a full CSV file of them. -- M. Edward (Ed) Borasky borasky-research.net/m-edward-ed-borasky/ "A mathematician is a device for turning coffee into theorems." ~ Paul Erd?s Quoting Raffi Krikorian : it is on our roadmap to be able to retrieve all your tweets. just to be precise, however, all your tweets are retrievable, if you know their IDs -- you can then retrieve them using statuses/show endpoint. what we don't maintain, however, is retrieval of more than 3200 tweets via the timeline calls. this is a limitation of our timeline and storage architecture, but it is actively being addressed. On Sat, Mar 6, 2010 at 11:09 AM, M. Edward (Ed) Borasky wrote: I filed a request for the ability for an authenticated user to be able to retrieve *all* of his or her tweets a long time ago. I can find the date and other details on this group if necessary. Someone back then from Twitter said that or some equivalent was on the road map. -- M. Edward (Ed) Borasky borasky-research.net/m-edward-ed-borasky/ "A mathematician is a device for turning coffee into theorems." ~ Paul Erd?s Quoting "@seiz" : Thanks all for the replies. Mark: Then the API Documentation is misleading as it says "Clients may request up to 3,200 statuses via the page…". My assumption was i can get a maximum result set of 3200 tweets but could change the timeframe of the result set by using since_id and max_id. What you say seems to be the case, but then the wording in the docs should be changed to clearly state: Only the newest 3200 tweets can be received via the API. Older Tweets are still archived at Twitter but not accessible via the archive. After all, this is not a Pagination limit at all then! It's simply a limit of how many tweets are accessible via the API. Since there might be cases, where archiving tweets might even be required by law (in corporate enviroments e.g.), accessing old tweets might be a rare but needed case and i could see the following as a solution not affecting your performance. Allow access to an "archive database" of tweets via a new set of API Methods - e.g.: http://api.twitter.com/1/statuses/home_timeline_archive.format And implement a different rate limit to these API Methods – e.g. 4,000 requests per Month This should suffice most usecases, as downloading old tweets is often a one time thing in order to catch up. Once othe old archive is retrieved, we can use the regular API Methods more frequent to keep our backup archive current and stay within the 3200 tweets boundary. What do you think? Stefan. -- Raffi Krikorian Twitter Platform Team http://twitter.com/raffi
Re: [twitter-dev] Re: Can't access old tweets via statuses/user_timeline
it is on our roadmap to be able to retrieve all your tweets. just to be precise, however, all your tweets are retrievable, if you know their IDs -- you can then retrieve them using statuses/show endpoint. what we don't maintain, however, is retrieval of more than 3200 tweets via the timeline calls. this is a limitation of our timeline and storage architecture, but it is actively being addressed. On Sat, Mar 6, 2010 at 11:09 AM, M. Edward (Ed) Borasky wrote: > I filed a request for the ability for an authenticated user to be able to > retrieve *all* of his or her tweets a long time ago. I can find the date and > other details on this group if necessary. Someone back then from Twitter > said that or some equivalent was on the road map. > > -- > M. Edward (Ed) Borasky > borasky-research.net/m-edward-ed-borasky/ > > "A mathematician is a device for turning coffee into theorems." ~ Paul > Erd?s > > > Quoting "@seiz" : > > Thanks all for the replies. >> >> Mark: Then the API Documentation is misleading as it says "Clients may >> request up to 3,200 statuses via the page…". My assumption was i can >> get a maximum result set of 3200 tweets but could change the timeframe >> of the result set by using since_id and max_id. >> >> What you say seems to be the case, but then the wording in the docs >> should be changed to clearly state: >>Only the newest 3200 tweets can be received via the API. Older >> Tweets are still archived at Twitter but not accessible via the >> archive. >> >> After all, this is not a Pagination limit at all then! It's simply a >> limit of how many tweets are accessible via the API. >> >> Since there might be cases, where archiving tweets might even be >> required by law (in corporate enviroments e.g.), accessing old tweets >> might be a rare but needed case and i could see the following as a >> solution not affecting your performance. >> >> Allow access to an "archive database" of tweets via a new set of API >> Methods - e.g.: >> http://api.twitter.com/1/statuses/home_timeline_archive.format >> And implement a different rate limit to these API Methods – e.g. 4,000 >> requests per Month >> This should suffice most usecases, as downloading old tweets is often >> a one time thing in order to catch up. Once othe old archive is >> retrieved, we can use the regular API Methods more frequent to keep >> our backup archive current and stay within the 3200 tweets boundary. >> >> What do you think? >> >> Stefan. >> >> > -- Raffi Krikorian Twitter Platform Team http://twitter.com/raffi
Re: [twitter-dev] Re: Can't access old tweets via statuses/user_timeline
> Ah, i see. I thought only the first post would be moderated (and only > with that i got a notification about the moderation). The group is configured to auto-promote after a certain number of accepted posts. The number is internally determined but is in the ballpark of 20 or 30. I will make this clearer on the group home page. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- He is rising from affluence to poverty. -- Mark Twain --
Re: [twitter-dev] Re: Can't access old tweets via statuses/user_timeline
> Apologies if my reply comes through multiple times. But google tells > me my reply was posted but i never see it arrive here. Just remember that the group is moderated (I removed your duplicate). -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- armadillo, n. the act of providing weapons to a Spanish pickle.
Re: [twitter-dev] Re: Can't access old tweets via statuses/user_timeline
I filed a request for the ability for an authenticated user to be able to retrieve *all* of his or her tweets a long time ago. I can find the date and other details on this group if necessary. Someone back then from Twitter said that or some equivalent was on the road map. -- M. Edward (Ed) Borasky borasky-research.net/m-edward-ed-borasky/ "A mathematician is a device for turning coffee into theorems." ~ Paul Erd?s Quoting "@seiz" : Thanks all for the replies. Mark: Then the API Documentation is misleading as it says "Clients may request up to 3,200 statuses via the page…". My assumption was i can get a maximum result set of 3200 tweets but could change the timeframe of the result set by using since_id and max_id. What you say seems to be the case, but then the wording in the docs should be changed to clearly state: Only the newest 3200 tweets can be received via the API. Older Tweets are still archived at Twitter but not accessible via the archive. After all, this is not a Pagination limit at all then! It's simply a limit of how many tweets are accessible via the API. Since there might be cases, where archiving tweets might even be required by law (in corporate enviroments e.g.), accessing old tweets might be a rare but needed case and i could see the following as a solution not affecting your performance. Allow access to an "archive database" of tweets via a new set of API Methods - e.g.: http://api.twitter.com/1/statuses/home_timeline_archive.format And implement a different rate limit to these API Methods – e.g. 4,000 requests per Month This should suffice most usecases, as downloading old tweets is often a one time thing in order to catch up. Once othe old archive is retrieved, we can use the regular API Methods more frequent to keep our backup archive current and stay within the 3200 tweets boundary. What do you think? Stefan.
