Our client would make even less sense to you then. It's written in Scala!
On Sun, Jan 17, 2010 at 9:56 PM, M. Edward (Ed) Borasky <zzn...@gmail.com>wrote: > As an aside, could Twitter release the streaming client they use under > some open source license, so we can use it as a prototype? I took a > look at the one Tom May of Gist wrote using Apache HttpClient and it > didn't make much sense to me - it was importing a bunch of Java > libraries and I'm not a Java programmer. > > On Jan 16, 10:18 pm, John Kalucki <j...@twitter.com> wrote: > > Given a reasonable stack, it shouldn't be all that hard to build > something > > robust. Our internal streaming client, which transits every tweet that > you > > see on the streaming api, seems to work just fine through various forms > of > > abuse, and it's, roughly, a few hundred lines wrapped around Apache > > httpclient. > > > > On the other hand, I suspect that dependability is all but impossible on > > some stacks, or will require some heroism on the part of a library > > developer. > > > > As a community, we need clients that trivially allow robustness in a > variety > > of stacks. We'll get there soon enough. > > > > On Sat, Jan 16, 2010 at 10:05 PM, M. Edward (Ed) Borasky > > <zzn...@gmail.com>wrote: > > > > > > > > > On Jan 16, 7:28 pm, John Kalucki <j...@twitter.com> wrote: > > > > I'd strongly suggest consuming the Streaming API only from persistent > > > > processes that write into some form of durable asynchronous queue (of > any > > > > type) for your application to consume. Running curl periodically is > > > unlikely > > > > to be a robust solution. > > > > > > Select one of the existing Streaming API clients out there and wrap > it in > > > a > > > > durable process. Write to rotated log files, a message queue, or > whatever > > > > other mechanism that you choose, to buffer the arrival of new > statuses > > > > before consumption by your application. This will allow you to > restart > > > your > > > > application at will without data loss. > > > > > I don't know that there are any open source libraries out there yet > > > that are robust enough to do that. At the moment, I'm working > > > exclusively in Perl, and "AnyEvent::Twitter::Stream" seems to be the > > > only Perl Streaming API consumer with any kind of mileage on it. As > > > you point out, real-time programming for robustness is a non-trivial > > > exercise. It would be nice if someone would build a C library and SWIG > > > ".i" files. ;-) > > > > > -- > > > M. Edward (Ed) Borasky > > >http://borasky-research.net/smart-at-znmeb > > > > > "A mathematician is a device for turning coffee into theorems." ~ Paul > > > Erdős >
