John, because no offense, but frankly I don't trust the Twitter API. I've been burned too many times by things that were "supposed to work", code pushed into production that wasn't tested properly, etc. that I know better to do all I can to account for Twitter's mistakes. There's no telling if at some point that next_cursor returns nothing, but in reality it was supposed to return something, and my users accidentally unfollow all their friends because of it when they weren't intending to do so. Having that number in there ensures, without a doubt (unless the number itself is wrong, which I can't do anything about), that I know if Twitter is right or not when I retrieve that next_cursor value. I hope that makes sense - it's nothing against Twitter, I've just seen it too many times to know that I need to have backup error checking in place to be sure I know Twitter's return data is correct.
Regarding the user being removed before finished, I thought the whole purpose of these cursors was to provide a snapshot of a social graph at a given point of time, so unfollowed users don't show up until after the list is retrieved - is that not the case? Also, my experience has been that pulling the user's friend and follower count ahead of time pulls a number that is not the same as the number of followers/friends I actually pull from the API. Having you guys do a count on the set ahead of time will help ensure that's the correct number. Thanks, Jesse On Sun, Oct 4, 2009 at 8:24 AM, John Kalucki <[email protected]> wrote: > > Curious -- why isn't the end of list indicator a reliable enough > indication? "Iterate until" seems simple and reliable. > > Can you request the denormalized count via the API before you begin? > (Not familiar enough with the API, but the back-end store offers this > for all sorts of purposes.) You'd have to apply some heuristic to > allow for high-velocity sets. > > The last user in the list could be removed before iteration completes, > setting up a race-condition that you'd have to allow for as well. > > -John Kalucki > http://twitter.com/jkalucki > Services, Twitter Inc. > > > On Oct 4, 1:29 am, Jesse Stay <[email protected]> wrote: > > I was wondering if it might be possible to include, at least in the first > > page, but if it's easier it could be on all pages, either a total > expected > > number of followers/friends, or a total expected number of returned pages > > when the cursor parameter is provided for friends/ids and followers/ids? > I'm > > assuming since you're moving to the cursor-based approach you ought to be > > able to accurately count this now since it's a snapshot of the data at > that > > time. > > The reason I think that would be useful is that occasionally Twitter goes > > down or introduces code that could break this. This would enable us to > be > > absolutely sure we've hit the end of the entire set. I guess another > > approach could also be to just list the last expected cursor ID in the > set > > so we can be looking for that. > > > > Thanks, > > > > Jesse >
