Hi all, is there some recommended best practice for paging? With the various issues involved I cannot see how i could reliably "process once, in order" all twitter direct messages from my app. This seems like a pretty basic thing, but AFAICS is defeated by a few things. Specifically,
1) Paging is in reverse order, and i have no way to find out what the "last" page is. 2) If i have to retrieve multiple pages, what happens if my app is restarted between pages? If i keep since_id the same, i will reprocess duplicate messages. If i update it, i lose messages. 3) How to tell that there is "no more data"? Pages can be empty if users/messages are deleted, how can the "many deleted messages" be distinguished from a "end of data set" case? Thanks for any help, this seems like a big API gap afaict...
