[twitter-dev] Re: streaming API track method missing statuses?

2009-08-14 Thread John Kalucki
Track keywords are logically ORed together. Give it a try. It should work as you expect. If not, let's figure it out. (OTOH: You have to do logical ANDs via post processing on your end.) -John Kalucki twitter.com/jkalucki Services, Twitter, Inc. On Aug 13, 9:56 pm, Andrew McCloud

[twitter-dev] Re: streaming API track method missing statuses?

2009-08-13 Thread John Kalucki
Andrew, If your test account statuses show up in search, but not in track, and you haven't been rate limited in track, then there's a problem. If the test account doesn't show up in search, it won't show up in the Streaming API either. I don't understand your second question. -John Kalucki

[twitter-dev] Re: streaming API track method missing statuses?

2009-08-13 Thread Chad Etzel
Hi Andrew, What are your account names? Feel free to unicast me. -Chad On Thu, Aug 13, 2009 at 5:02 PM, Andrew McCloudand...@amccloud.com wrote: Hello Everyone, I can't seem to figure out why the tweets from my test account do not show up in the streaming track method. If I use my personal

[twitter-dev] Re: Streaming API -- Recheck your clients -- post DDoS cleanup

2009-08-07 Thread dean.j.robinson
What about those using the regular API, via both Basic Auth and OAuth, is there anything at all we can do to stop getting endless 408's ? I'm guessing that since even twitter.com itself is still very inconsistent, for lack of a better word, theres probably nothing much more we can do than just

[twitter-dev] Re: Streaming API -- Recheck your clients -- post DDoS cleanup

2009-08-06 Thread Shannon Clark
Not specific to only developers but at the moment http://search.twitter.com is not loading on my iPhone though search via an iPhone app (twitterfon is what I tried) is working. Shannon Sent from my iPhone On Aug 6, 2009, at 2:19 PM, John Kalucki jkalu...@gmail.com wrote: Some users

[twitter-dev] Re: streaming API for DM for multiple users ?

2009-07-28 Thread Andres B
I'd support the creation of such a method. My latest focus is receiving orders/commands through DM and having a hose would give us a much snappier response than polling every 30 secs. On Jul 27, 8:55 pm, Doug Williams d...@twitter.com wrote: There is currently no Streaming API to receive DMs

[twitter-dev] Re: streaming API for DM for multiple users ?

2009-07-28 Thread Ben Hall
Random idea, but wouldn't a streaming API for DMs allow IM style clients to be implemented on top of the twitter platform? I know I use DMs instead of MSN now, the delay is a bit of a pain but being able to move the conversation from public to private is great, plus sometimes you do want delayed

[twitter-dev] Re: streaming API for DM for multiple users ?

2009-07-28 Thread Fabien Penso
On Tue, Jul 28, 2009 at 3:55 AM, Doug Williamsd...@twitter.com wrote: There is currently no Streaming API to receive DMs for a given user. If you have a great use case for this please share it here. We like to have justification for new streaming methods. If you have ideas to help augment a

[twitter-dev] Re: streaming API for DM for multiple users ?

2009-07-28 Thread Fabien Penso
On Tue, Jul 28, 2009 at 4:39 AM, Ben Hallben200...@googlemail.com wrote: Random idea, but wouldn't a streaming API for DMs allow IM style clients to be implemented on top of the twitter platform? It would.

[twitter-dev] Re: streaming API for DM for multiple users ?

2009-07-27 Thread Doug Williams
There is currently no Streaming API to receive DMs for a given user. If you have a great use case for this please share it here. We like to have justification for new streaming methods. If you have ideas to help augment a business case for engineering resources, we would love to know about them.

[twitter-dev] Re: Streaming API + PHP and Python

2009-07-24 Thread Joseph
Have you resolved this problem? suggestion: did you try writing the raw output to a file (like every hour, and then create another file, and so on), and then have another script process the JSON? On Jun 8, 1:16 pm, Chad Etzel jazzyc...@gmail.com wrote: Hi All, I am stumped.  For several days

[twitter-dev] Re: Streaming API -- Additional markup added -- Deletion notifications on track streams.

2009-07-14 Thread John Kalucki
Yes, the two are related. Good sleuthing. On Jul 13, 10:08 pm, Damon Clinkscales sca...@pobox.com wrote: On Mon, Jul 13, 2009 at 9:53 PM, John Kaluckijkalu...@gmail.com wrote: Deletions will be enabled on or after Thursday July 16th, as previously scheduled. From the wiki,  

[twitter-dev] Re: Streaming API -- Additional markup added -- Deletion notifications on track streams.

2009-07-13 Thread Joel Strellner
Wow John, that was quick. Thank you. -Joel On Jul 13, 2009, at 7:53 PM, John Kalucki jkalu...@gmail.com wrote: In addition to deletion notices, limitation notices will be added to track streams. These notices will be enabled on or after Tuesday July 14th. Deletions will be enabled on or

[twitter-dev] Re: Streaming API -- Additional markup added -- Deletion notifications on track streams.

2009-07-13 Thread Damon Clinkscales
On Mon, Jul 13, 2009 at 9:53 PM, John Kaluckijkalu...@gmail.com wrote: Deletions will be enabled on or after Thursday July 16th, as previously scheduled. From the wiki,  http://apiwiki.twitter.com/Streaming-API-Documentation: Streams may also contain status deletion notices. Clients are

[twitter-dev] Re: Streaming API -- New objects in markup stream. Test your code

2009-07-11 Thread Laurent Eschenauer
Hi John, I can't find such thing as a status or delete object in the JSON feed. There is indeed a status enveloppe in the XML, but the corresponding JSON object seems to be already one level deeper, only encapsulating the data from the status itself. Could you please clarify what we should be

[twitter-dev] Re: Streaming API -- New objects in markup stream. Test your code

2009-07-11 Thread John Kalucki
Laurent, There are examples of the new objects on the Streaming API wiki. The XML and JSON formats are, sadly, not orthogonal. The objects aren't flowing to give developers time to adjust. We'll probably enable this in the middle of next week. -John On Jul 11, 12:34 am, Laurent Eschenauer

[twitter-dev] Re: Streaming API -- New objects in markup stream. Test your code

2009-07-11 Thread Joel Strellner
John, Any chance you can allow us to send an additional variable when we connect and you guys send it in the new format? This would allow for overlap and testing. -Joel On Jul 11, 2009, at 7:04 AM, John Kalucki jkalu...@gmail.com wrote: Laurent, There are examples of the new objects

[twitter-dev] Re: Streaming API/ track method - results Tweets from Dec. 2008

2009-07-11 Thread John Kalucki
Unless there's been a server restart and some unusual resulting backlog, nearly all tweets from /track will be forwarded within a second or two of being posted. I suspect you are conflating the created_at of the user with the created_at of the status. I've done this many times. -John Kalucki

[twitter-dev] Re: Streaming API -- New objects in markup stream. Test your code

2009-07-11 Thread John Kalucki
We'll mail again and post on the @twitterapi account before we cry havoc and let slip the hounds. -John On Jul 11, 2:26 pm, Joel Strellner j...@twitturly.com wrote: John, Any chance you can allow us to send an additional variable when we   connect and you guys send it in the new format?

[twitter-dev] Re: Streaming API/ track method - results Tweets from Dec. 2008

2009-07-11 Thread Germig
Sorry, I should have seen it. You are right. Thanks a lot. Martin On 12 Jul., 03:25, John Kalucki jkalu...@gmail.com wrote: Unless there's been a server restart and some unusual resulting backlog, nearly all tweets from /track will be forwarded within a second or two of being posted. I

[twitter-dev] Re: Streaming API: updating request parameters

2009-07-10 Thread John Kalucki
This isn't implemented. I don't think we could, either, given our server framework. Create a new connection with the updated parameters. As soon as you receive data from the new connection, close the old connection. Keep the churn rate low, don't reconnect too often, and you should stay out of

[twitter-dev] Re: Streaming API -- Code Change Required: follow parameter delimiter

2009-07-07 Thread John Kalucki
A reminder: Support for space separated track and follow lists will be dropped in this afternoon's Streaming API deploy. Only comma separated lists will be accepted. On Jun 12, 11:53 am, John Kalucki jkalu...@gmail.com wrote: The follow post parameter now takes acommaseparated list of userids

[twitter-dev] Re: Streaming API -- Code Change Required: follow parameter delimiter

2009-07-07 Thread Cary Knoop
A prelude to perhaps allowing searching for phrases at a later time? Certainly something that many would find useful and would effectively limit the stream sent over the wires. For instance, at this time, if I would want to capture the phrase Joe Jackson I have to search for the keyword Jackson

[twitter-dev] Re: streaming api multiple connections

2009-06-30 Thread John Kalucki
Hardeep, What is your use case? If you are attempting to get around limits in / track or /follow, contact us to discuss higher access levels. If you are building a service that creates a connection per user, let's discuss how you can achieve the same ends with a single connection. If you require

[twitter-dev] Re: streaming api multiple connections

2009-06-29 Thread Arunachalam
As per my understanding, all multiple connection will have same sampled output. Cheers, Arunachalam On Tue, Jun 30, 2009 at 10:36 AM, Hardeep Khehra hardeep.s.khe...@gmail.com wrote: will the streaming track api allow multiple connections from the same IP using different user accounts and

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-06-23 Thread கார்த்திக் . மு
John, Can we have two concurrent connections, one each to the methods gardenhose and shadow from a single Twitter account? Thanks, Karthik On May 10, 9:04 am, John Kalucki jkalu...@gmail.com wrote: Note: TheStreamingAPI is currently under a limited alpha test, details below. Multiple

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-06-23 Thread John Kalucki
You can't have multiple logins per account at the moment. In general, contact the API group for help with this. In this case, I'll set things up for you. -John Kalucki Services, Twitter Inc. On Jun 23, 7:26 am, கார்த்திக்.மு fermis...@gmail.com wrote: John, Can we have two concurrent

[twitter-dev] Re: Streaming API + PHP and Python

2009-06-10 Thread Jason Emerick
I believe I had to set the default locale of my system to use UTF-8 by setting the appropriate environment variable. I believe it was the following on an ubuntu server: LANG=en_US.UTF-8 The other option as Nick pointed out is using the following: foo = foo.encode(utf-8) Jason Emerick The

[twitter-dev] Re: Streaming API -- /follow /birddog and /shadow also match in_reply_to field.

2009-06-10 Thread John Kalucki
This is a known defect. Quick catch! The fix is easy enough and required for future features. Streaming API clients need to de-duplicate for a whole host of other reasons, especially those who use the count parameter. The service errs on the side of overdelivery. -John Service, Twitter Inc.

[twitter-dev] Re: Streaming API -- /follow /birddog and /shadow also match in_reply_to field.

2009-06-10 Thread Chad Etzel
Cool, thanks. I have noticed that the duplicates have always (thus far) come in back-to-back, so I've just started checking the tweet id against the last received tweet id to see if they match. Do you know if that should always be the case (until the fix), or if I have just been getting lucky

[twitter-dev] Re: Streaming API -- /follow /birddog and /shadow also match in_reply_to field.

2009-06-10 Thread John Kalucki
For this issue, and all filtering-driven issues, they'll always be back-to-back. For other causes of overdelivery, duplicates can be scattered pretty far apart. Overdelivery is usually triggered by lifecycle events -- servers restarting, your client reconnecting, that sort of thing. This type of

[twitter-dev] Re: Streaming API -- /follow /birddog and /shadow also match in_reply_to field.

2009-06-09 Thread Chad Etzel
Neato! Would it be possible to add some sort of attribute to the status object which indicates when this is the case? (i.e. this update is being sent to you, but the user id of the sender is not explicitly in the follow id list?) Would be handy, perhaps. -Chad On Tue, Jun 9, 2009 at 10:46 PM,

[twitter-dev] Re: Streaming API -- /follow /birddog and /shadow also match in_reply_to field.

2009-06-09 Thread John Kalucki
Unlikely. In general, we treat a status as immutable, but removable. Hosebird doesn't re-write statuses. Clients can determine this by themselves. Too many other things to do! -John On Jun 9, 8:10 pm, Chad Etzel jazzyc...@gmail.com wrote: Neato! Would it be possible to add some sort of

[twitter-dev] Re: Streaming API -- /follow /birddog and /shadow also match in_reply_to field.

2009-06-09 Thread Chad Etzel
Makes sense. One thing I'm noticing that now this feature is live: If userA and userB are both in my follow id list, and then if userA makes an explicit reply to userB, I get userA's update twice. Just something to be aware of for everyone. This duplicate update also happens if you have the

[twitter-dev] Re: Streaming API + PHP and Python

2009-06-08 Thread John Kalucki
A theory: The PHP client has stopped reading data, for whatever reason. The TCP buffers fill on the client host, the TCP window closes, and wireshark shows no data flowing. netstat(1) will show the number of bytes waiting in the local TCP buffer. Baseless speculation: There's a limitation in the

[twitter-dev] Re: Streaming API + PHP and Python

2009-06-08 Thread Chad Etzel
I thought those things, too... but the following things made me think otherwise: a) The stream stops after a different number of updates/bytes each time, and will happily go on forever if I put an error-catching loop in the script. b) The same thing is happening in the python script. c)

[twitter-dev] Re: Streaming API + PHP and Python

2009-06-08 Thread jstrellner
Hi Chad, We too have noticed the same behavior in PHP. Initially I wrote something very similar to your example, and noticed that I'd get a random time's worth of data before it disconnected. Then I rewrote it, which you can see at the below URL (modified to remove irrelevant code to this

[twitter-dev] Re: Streaming API + PHP and Python

2009-06-08 Thread Chad Etzel
Well, glad I'm not the only one :) But still a bummer it's happening... Another strange thing is that his does *not* seem to happen with the /follow streams. I have a PHP script running (same source, just requesting /follow instead of /spritzer) that has been connected for over 2 days. Of

[twitter-dev] Re: Streaming API + PHP and Python

2009-06-08 Thread Jason Emerick
Here is some rough python code that I quickly wrote last weekend to handle the json spritzer feed: http://gist.github.com/126173 During the 3 or so days that I ran it, I didn't notice it die at any time... Jason Emerick The information transmitted (including attachments) is covered by the

[twitter-dev] Re: Streaming API + PHP and Python

2009-06-08 Thread Chad Etzel
Hi Jason, Thanks! I've tried it out, and it seems that it doesn't like unicode characters? Here's the traceback I get: Exception in thread Thread-2: Traceback (most recent call last): File /usr/lib/python2.5/threading.py, line 486, in __bootstrap_inner self.run() File spritzer.py,

[twitter-dev] Re: Streaming API + PHP and Python

2009-06-08 Thread Nick Arnett
Try calling encode(utf-8) on the strings before you do anything else with them but when you do, you may find that you have to add Python components. In other words, if the string is foo, do this: foo = foo.encode(utf-8) Nick On Mon, Jun 8, 2009 at 4:52 PM, Chad Etzel jazzyc...@gmail.com

[twitter-dev] Re: Streaming API: missing updates non-indexed user correlation?

2009-06-06 Thread @Jalada
I'd like to also voice my agreement with Chad here, if you're specifically wanting to follow someone/thing you should get it. I agree with filtering in search, but in this case you are purposefully stating you wish to follow someones updates, and the result should reflect that, regardless of how

[twitter-dev] Re: Streaming API: missing updates non-indexed user correlation?

2009-06-05 Thread John Kalucki
There are multiple bits set for accounts that control various levels of access and all kinds of folderol. It's complicated and for mostly understandable reasons, purposefully opaque. Search and Hosebird currently have identical access rules, but that's subject to change. In this case, it appears

[twitter-dev] Re: Streaming API: missing updates non-indexed user correlation?

2009-06-05 Thread Chad Etzel
Ok, at least that confirms my suspicions about why those updates were not being delivered. If I may make an argument to separate the policy between Search and Hosebird (at least for the /follow methods)... In the case of the /follow methods (as opposed to the unfiltered /(fire|garden)hose

[twitter-dev] Re: Streaming API: missing updates non-indexed user correlation?

2009-06-05 Thread John Kalucki
A very good point. I'll take this up with product. On Jun 5, 10:58 am, Chad Etzel jazzyc...@gmail.com wrote: Ok, at least that confirms my suspicions about why those updates were not being delivered. If I may make an argument to separate the policy between Search and Hosebird (at least for

[twitter-dev] Re: Streaming API: Spritzer-stream coverage

2009-05-27 Thread Brendan O'Connor
On Tue, May 26, 2009 at 10:07 PM, elversatile elversat...@gmail.com wrote: Makes sense. I was assuming the same. Thanks people! John from Twitter said that spritzer is 1/3 of the gardenhose, which makes it 15%. So I guess statistical insignificance of spritzer is due to its low percentage.

[twitter-dev] Re: Streaming API: Spritzer-stream coverage

2009-05-27 Thread John Kalucki
Folks, The significant/insignificant language currently isn't that important or clear, as we're preparing for future changes. The spritzer will likely remain a small public sample, the gardenhose will likely remain a larger sample that requires an EULA. The proportions, however, are subject to

[twitter-dev] Re: Streaming API: /spritzer sending nothing but keep-alives after a while

2009-05-26 Thread Chad Etzel
New debugging output shows I just received 44,574 tweets before getting only keep-alive newlines every 20 seconds for about 30 minutes before I killed the script and restarted it. -Chad On Tue, May 26, 2009 at 5:48 PM, Chad Etzel jazzyc...@gmail.com wrote: Hi John (and others playing w/ the

[twitter-dev] Re: Streaming API: Spritzer-stream coverage

2009-05-26 Thread stephane
Hi Sven, well I merely assumed that the easiest way for twitter to send a subset of tweets on spitzer was to send them based on their ids (autoincrement integer)... watching at the stream, I noticed that all the ids where ending with 000,001,002,003,004, 100,102, ... 900,901,... 904 I did not

[twitter-dev] Re: Streaming API: /spritzer sending nothing but keep-alives after a while

2009-05-26 Thread John Kalucki
Digging into this... -John Kalucki Services, Twitter Inc. On May 26, 4:58 pm, Chad Etzel jazzyc...@gmail.com wrote: New debugging output shows I just received 44,574 tweets before getting only keep-alive newlines every 20 seconds for about 30 minutes before I killed the script and restarted

[twitter-dev] Re: Streaming API: Spritzer-stream coverage

2009-05-26 Thread elversatile
Makes sense. I was assuming the same. Thanks people! John from Twitter said that spritzer is 1/3 of the gardenhose, which makes it 15%. So I guess statistical insignificance of spritzer is due to its low percentage. Any explanation directly from Twitter? On May 26, 6:01 pm, stephane

[twitter-dev] Re: Streaming API: Spritzer-stream coverage

2009-05-25 Thread elversatile
How are spritzer statuses sampled? Are they picked uniformly at random? Or is there some logic behind it? Also, what makes it statistically insignificant? Is it its percentage in relation to the entire stream or the way it is sampled? Thanks, -Eldar On May 24, 8:23 pm, John Kalucki

[twitter-dev] Re: Streaming API: Spritzer-stream coverage

2009-05-25 Thread stephane
looking at the tweet ids it looks like the spitzer stream delivers 5 tweets every hundreds this would make it a 5% of the firehose am i correct? Stephane http://www.twazzup.com On May 25, 12:17 am, elversatile elversat...@gmail.com wrote: How are spritzer statuses sampled? Are they picked

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-05-14 Thread John Kalucki
Damon, Interesting use cases are starting to pop up! For now, yes, the concurrent connection limit applies to all resources. If you want streams for various sets of users, we'd ask that you just set up one stream and multiplex all your requests over the same connection. You'll have to do the

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-05-11 Thread Hwee-Boon Yar
Oh god. Please share where is this twitter-announce list? -- Hwee-Boon On May 10, 12:51 pm, Jesse Stay jesses...@gmail.com wrote: Not to be picky, but can we get these announcements on the twitter-announce list in the future? Who is this John and is he a real Twitter employee? On Sat, May

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-05-11 Thread dean.j.robinson
Nice work guys, talk about the firehose has been floating around for ages, great to see it finally appear and with numerous variants available (thats a bonus). I personally don't have any use for it (yet) but I'm sure it'll please quite a few. On May 10, 2:04 pm, John Kalucki

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-05-10 Thread Doug Williams
Jesse, We will announce this to the announce list when we feel it is ready for a larger audience. John is a Twitter employee and is developing the streaming service. Be sure to give him a thanks if you use his stuff. Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-05-10 Thread benburkert
Just to clarify, are we limited to one resource per account? If i connect to /spritzer on a test server, will that knock off my connection to /gardenhose on my production server if I use the same account? -Ben On May 9, 11:04 pm, John Kalucki jkalu...@gmail.com wrote: Note: The Streaming API

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-05-10 Thread John Kalucki
Once the service fully enforces the intended policy, then yes, a / spritzer login will knock off a /gardenhose login, or visa versa. The current intention is for developers to create a /gardenhose production account and a testing /spritzer account. At first glance, this requirement does not

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-05-10 Thread Jesse Stay
Thanks Doug - I just wanted to be sure we weren't venturing away from this practice. I'll often miss it if it's not on the announce list, and it's a great way to know for sure it's coming from Twitter. Thanks for your work John! I don't use your service now but if I ever do I think the streaming

[twitter-dev] Re: Streaming API Terms Of Service change - multiple simultaneous logins discouraged

2009-05-09 Thread Jesse Stay
Not to be picky, but can we get these announcements on the twitter-announce list in the future? Who is this John and is he a real Twitter employee? On Sat, May 9, 2009 at 10:04 PM, John Kalucki jkalu...@gmail.com wrote: Note: The Streaming API is currently under a limited alpha test, details

<    1   2   3