I understand. There's lots of sausage making going on behind the scenes. You make it the best you can. Often for reasons that I cannot hope to understand. ;-)
Let me stop being metaphysical and boil this down to a simple bug report: There is no documentation of how to get from the schema of the User Stream API to the schema of the REST API. Here's why this is a problem: The REST API divides and filters data, sometimes in some non-obvious ways. User streams divide the data less. It's the clients job to fetch backfill from one API, stream from the other, and display the result such that the display is consistent no matter which API the tweets came from. In practice when I compare the clients that use streams, they're all falling down in a few places. Here's an example 1: I retweet a tweet from my Home timeline. In Twitter.com it does NOT appear In Client A it DOES appear (but not if fetched from backfill) In Client B it does NOT appear. And example 2: I retweet a tweet from a list that is not in my Home timeline. In Twitter.com it DOES appear In Client A it DOES appear In Client B it does NOT appear. (but does if fetched from backfill) So, it looks to me like twitter.com seems to be de-duplicating the original tweets of retweets. Client A de-duplicates only regular tweets letting all retweets through. Client B remove all retweets that i create. Both clients have different behavior for streaming than they do for backfill. * Client A is my client (Kiwi), Client B is a very popular streaming client on the Mac. I have not been able to find any client that behaves consistently between backfill and streaming. All fail on at least one corner case that I'm aware of. There are subtle behaviors, some undocumented, that each developer must find, analyze and implement. It's difficult. We're not doing so well. In order to make bug-free clients we need Twitter to make the schema the same, or (if that sausage is too expensive) document how they differ. isaiah http://twitter.com/isaiah On Dec 18, 2010, at 7:48 AM, John Kalucki wrote: > Yes, our model does externalize some development and hosting costs onto > clients. But, we tend to only externalize cost when issues would be far > cheaper, in aggregate, to solve on the client, or would be intractable to > solve on our end and might otherwise prevent the launch of the feature. We > try to balance this cost externalization very carefully and with all due > concern for everyone's time. Our resources are limited, and our reasoning may > not always be immediately obvious, but we're trying to get you as much data > as possible, as efficiently as possible for everyone. > > -John Kalucki > http://twitter.com/jkalucki > Twitter, Inc. > > > > > > > On Tue, Dec 14, 2010 at 2:07 PM, Isaiah Carew <isa...@me.com> wrote: > > In other words, if I want to disambiguate the stream, I have to filter it > myself. Well, humph… > > Not impossible, just a pain in the butt. > > From an information organization standpoint, it seems odd: The REST API is > broken out into separate calls. The stream has everything glommed together. > > It would be no big deal if you only needed one or the other, but you have to > do backfill with the REST API, so you always need both. > > The REST API has hierarchy in the endpoints. The Stream API has hierarchy in > the schema. Why not make the hierarchies (at least roughly) the same? You > don't have to answer, I'm just mouthing off. I'll get back to work writing a > track-term to nspredicate converter. ;-) > > > isaiah > http://twitter.com/isaiah > > On Dec 13, 2010, at 9:30 AM, John Kalucki wrote: > >> Roughly: >> If the tweet is from a following, place it in the home timeline. >> If the tweet refers to the user (to or from), or contains the @screenname >> place it in mentions >> If it's a message -> messages. >> What remains is probably a track term. >> >> -John Kalucki >> http://twitter.com/jkalucki >> Twitter, Inc. >> >> >> >> >> On Fri, Dec 10, 2010 at 5:58 PM, isaiah <isa...@mac.com> wrote: >> >> Hi, >> >> I'm implementing user streams in my client and looking for some advice >> on best practices. My client supports viewing multiple timelines at >> the same time, so it's quite possible to, for example: view a saved >> search, the user's own home timeline, and another user's recent >> tweets. >> >> Of course, I'd love to implement these in user streams. My concern is >> that if each of these timelines were to open a separate stream >> simultaneously, then the user could easily cross over their limit of >> active streams. Another potential solution seems to be adding the >> search and the second user as tracking parameters to a single user >> stream. That works fine and the track parameter limitations seem to >> be similar to the limitations of the UI/UX of my app, so it seemed >> like a good fit. >> >> The challenge is that once track parameters are added to the stream I >> get a whole bunch of new statuses returned but i can't tell which are >> associated with each parameter. Or, well, I couldn't figure out how >> to tell short of building a regex for each of my track parameters and >> trying to sort the items by hand (yuck!). >> >> So my question: >> 1. Is there some way to disambiguate statuses returned as a specific >> track parameter from those returned for other reasons? >> 2. Is there some other way to skin this cat that I'm missing? >> >> Thanks, >> Isaiah >> >> -- >> Twitter developer documentation and resources: http://dev.twitter.com/doc >> API updates via Twitter: http://twitter.com/twitterapi >> Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list >> Change your membership to this group: >> http://groups.google.com/group/twitter-development-talk >> >> >> -- >> Twitter developer documentation and resources: http://dev.twitter.com/doc >> API updates via Twitter: http://twitter.com/twitterapi >> Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list >> Change your membership to this group: >> http://groups.google.com/group/twitter-development-talk > > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk > > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
