[twitter-dev] Streaming API track parameter Options
According to the docs, searching for twitter should NOT return www.twitter.com (http://dev.twitter.com/pages/streaming_api_methods#track) However this is exactly what happens: Searching for amzn returns tweets with www.amzn.co.jp Searching for Google Maps returns tweets with maps.google.com Any idea? -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Streaming API with start_date and end_date
Hi Guys, is it possible to query streaming api with filter and include start_date and end_date, so that developer can track or filter by date range? If yes, that could be awesome, if no, would you mind please to add this feature? Why it is needed because it would be great if we can track keyword from previous date where we have missed the tweets. For event organizer and finance reports application like us, this feature is very valuable. Many Thanks, Jack -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Streaming API with start_date and end_date
This would be a great feature, any word on this yet? Trevor Dean | Director big time design communication Inc. 647 234 8198 Visit http://www.bigtimedesign.ca for more information On 2011-06-23, at 7:13 AM, JackRabbit yacobus.reinh...@gmail.com wrote: Hi Guys, is it possible to query streaming api with filter and include start_date and end_date, so that developer can track or filter by date range? If yes, that could be awesome, if no, would you mind please to add this feature? Why it is needed because it would be great if we can track keyword from previous date where we have missed the tweets. For event organizer and finance reports application like us, this feature is very valuable. Many Thanks, Jack -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Streaming API with start_date and end_date
This is not within the scope of the real-time streaming API which serves the purpose of streaming tweets related events as they happen with very limited support for any kind of rewind behavior. On Thu, Jun 23, 2011 at 9:42 AM, Trevor Dean trevord...@gmail.com wrote: This would be a great feature, any word on this yet? Trevor Dean | Director big time design communication Inc. 647 234 8198 Visit http://www.bigtimedesign.ca for more information On 2011-06-23, at 7:13 AM, JackRabbit yacobus.reinh...@gmail.com wrote: Hi Guys, is it possible to query streaming api with filter and include start_date and end_date, so that developer can track or filter by date range? If yes, that could be awesome, if no, would you mind please to add this feature? Why it is needed because it would be great if we can track keyword from previous date where we have missed the tweets. For event organizer and finance reports application like us, this feature is very valuable. Many Thanks, Jack -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Streaming API w/ two dedicated IP's
It is kosher to connect to the Twitter Streaming API using two dedicated IP's on two separate servers or does this violate the terms of usage? I've looked for something telling me I can't do this and nothing has cropped up, but I'd like to have this confirmed. -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Streaming API w/ two dedicated IP's
Connecting from two IPs, with two different users will be perfectly fine and fall within the current rules. On 14 Jun 2011, at 20:39, @dbbradle wrote: It is kosher to connect to the Twitter Streaming API using two dedicated IP's on two separate servers or does this violate the terms of usage? I've looked for something telling me I can't do this and nothing has cropped up, but I'd like to have this confirmed. -- Scott Wilcox @dordotky | sc...@dor.ky | http://dor.ky +44 (0) 7538 842418 | +1 (646) 827-0580 -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Streaming API, Following just a couple of people
I I start following just 1 or 2 people using the streaming API I do not get any of their tweets. Is there a buffer that needs to be filled before I get these? Ray Slakinski -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Streaming API, Following just a couple of people
Hey Ray, As soon as the connection is established, you start receiving public statuses that match your filter predicates. Are you sure these users were actually tweeting during the time you were consuming the stream? Arnaud / @rno http://twitter.com/rno On Mon, Jun 6, 2011 at 6:04 AM, Ray Slakinski ray.slakin...@gmail.comwrote: I I start following just 1 or 2 people using the streaming API I do not get any of their tweets. Is there a buffer that needs to be filled before I get these? Ray Slakinski -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Streaming API, Following just a couple of people
Hi Ray, There isn't a buffer that has to be filled before the Streaming API delivers tweets. Only public tweets created after you open a connection will be delivered. Have the users you are following Tweeted since you connected, and are they public accounts (not protected)? On Jun 6, 2011, at 6:04, Ray Slakinski ray.slakin...@gmail.com wrote: I I start following just 1 or 2 people using the streaming API I do not get any of their tweets. Is there a buffer that needs to be filled before I get these? Ray Slakinski -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Streaming API credentials
I'd like to use the streaming API to track certain terms that I'll ultimately present to all of my web application's users. For instance, I want my app to display all tweets for some event, identified by some hashed term. I see that the streaming API (unlike the search API) requires authentication, either Basic or OAuth. For tinkering purposes, I've just used my own OAuth token/secret to hit the streaming API. But which credentials should my app use? Since the stream will be presented to all of my app's users, it doesn't make sense for it to use a single user's credentials. It also doesn't make much sense to open up individual, but identical streams for each user. Is there a way to consume the streaming API with some app-level credentials? -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Streaming API credentials
You would create a twitter app at https://dev.twitter.com/apps After you create it, there is a My Access Token button on the details page for your application. I /believe/ that will get you what you want. James On Mon, May 23, 2011 at 3:04 PM, Craig Walls hab...@gmail.com wrote: I'd like to use the streaming API to track certain terms that I'll ultimately present to all of my web application's users. For instance, I want my app to display all tweets for some event, identified by some hashed term. I see that the streaming API (unlike the search API) requires authentication, either Basic or OAuth. For tinkering purposes, I've just used my own OAuth token/secret to hit the streaming API. But which credentials should my app use? Since the stream will be presented to all of my app's users, it doesn't make sense for it to use a single user's credentials. It also doesn't make much sense to open up individual, but identical streams for each user. Is there a way to consume the streaming API with some app-level credentials? -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Streaming API length
Hi there, Is there a maximum limit on how long we can maintain the connection to the Streaming API or on the number of tweets consumed? Thanks -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Streaming API length
No. On 19 May 2011, at 20:26, Tereno wrote: Hi there, Is there a maximum limit on how long we can maintain the connection to the Streaming API or on the number of tweets consumed? Thanks -- Scott Wilcox @dordotky | sc...@dor.ky | http://dor.ky +44 (0) 7538 842418 | +1 (646) 827-0580 -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Streaming API timeout issues
Hi
I'm trying the following code snippet to connect to your API but it is
timing out while waiting for the response.
string url = http://stream.twitter.com/1/statuses/filter.json?
follow=12;
WebRequest req = WebRequest.Create(url);
req.Credentials = new NetworkCredential(ConnectTweetPRO,
Preston71);
WebResponse response = req.GetResponse();
Encoding encode = Encoding.GetEncoding(utf-8);
var responseStream = new StreamReader(response.GetResponseStream(),
encode);
int count = 0;
while (!responseStream.EndOfStream count 2)
{
DateTime end = DateTime.Now + TimeSpan.FromSeconds(60);
while (DateTime.Now end)
Thread.Sleep(1000);
Response.Write(responseStream.ReadLine());
count++;
}
Can you help please? I'm sure I'm missing something.
Ta
--
Twitter developer documentation and resources: https://dev.twitter.com/doc
API updates via Twitter: https://twitter.com/twitterapi
Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Streaming API timeout issues
Hey there,
Please be careful when you share pieces of code on this Mailing List. I had
to reset your Twitter password because you just shared it publicly. Please
go to http://twitter.com/account/resend_password so we can send you password
reset instructions.
Concerning your issue, the Streaming API doesn't timeout. Our servers hold
the connection open indefinitely. Cf this documentation page for more
information: http://dev.twitter.com/pages/streaming_api_concepts#connecting
Looks like you're disconnecting in your while loop. There are a couple of
resources out there (like this Shannon Whitley's blog post:
http://bit.ly/ghavU1) that might help you to build your Twitter Streaming
API Client :)
Hope that helps,
Arnaud / @rno http://twitter.com/rno
On Thu, May 19, 2011 at 10:12 AM, yippanion yippan...@hotmail.com wrote:
Hi
I'm trying the following code snippet to connect to your API but it is
timing out while waiting for the response.
string url = http://stream.twitter.com/1/statuses/filter.json?
follow=12;
WebRequest req = WebRequest.Create(url);
req.Credentials = new NetworkCredential(ConnectTweetPRO,
Preston71);
WebResponse response = req.GetResponse();
Encoding encode = Encoding.GetEncoding(utf-8);
var responseStream = new StreamReader(response.GetResponseStream(),
encode);
int count = 0;
while (!responseStream.EndOfStream count 2)
{
DateTime end = DateTime.Now + TimeSpan.FromSeconds(60);
while (DateTime.Now end)
Thread.Sleep(1000);
Response.Write(responseStream.ReadLine());
count++;
}
Can you help please? I'm sure I'm missing something.
Ta
--
Twitter developer documentation and resources: https://dev.twitter.com/doc
API updates via Twitter: https://twitter.com/twitterapi
Issues/Enhancements Tracker:
https://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
https://groups.google.com/forum/#!forum/twitter-development-talk
--
Twitter developer documentation and resources: https://dev.twitter.com/doc
API updates via Twitter: https://twitter.com/twitterapi
Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Streaming API truncating tweets
Why does the streaming API truncate certain tweets that are not truncated on twitter.com or through other APIs? Thanks! -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Streaming API truncating tweets
Where are you seeing this truncation? Can you share some examples of the
JSON with the truncated (I assume status body) field?
In the case of retweets in the streaming API, you may see the top-level
text node with truncated tweet text prepended with RT -- but the full
text of the original referenced status will be embedded in the
retweeted_status/text node:
See this example:
{
in_reply_to_status_id_str:null,
coordinates:null,
truncated:true,
text:RT @chocobo_bot: \u3053\u306e \u3076\u3093\u3057\u3087\u3046
\u306f \u306b\u3093\u3052\u3093 \u306f \u3082\u3058 \u3092
\u306b\u3093\u3057\u304d \u3059\u308b \u3068\u304d \u305d\u306e
\u3055\u3044\u3057\u3087 \u3068 \u3055\u3044\u3054 \u306e
\u3082\u3058\u3055\u3048 \u3042\u3063\u3066\u3044\u308c\u3070
\u3058\u3085\u3093\u3070\u3093 \u306f \u3081\u3061\u3083\u304f\u3061\u3083
\u3067\u3082 \u3061\u3083\u3093\u3068 \u3088\u3081\u308b \u3068\u3044\u3046
\u3051\u3093\u304d\u3085\u3046 \u3092 \u3057\u3063\u3066\u3044\u308b
\u3072\u3068 \u3092 \u3072\u3063\u304b\u3051\u308b \u3044 ...,
contributors:null,
retweeted:false,
in_reply_to_user_id_str:null,
id_str:70499845542125568,
entities:
{
user_mentions:
[
{
indices:
[
3,
15
],
id_str:76918274,
name:\u30c1\u30e7\u30b3\u30dc,
screen_name:chocobo_bot,
id:76918274
}
],
hashtags:
[
],
urls:
[
]
},
retweeted_status:
{
in_reply_to_status_id_str:null,
coordinates:null,
truncated:false,
text:\u3053\u306e \u3076\u3093\u3057\u3087\u3046 \u306f
\u306b\u3093\u3052\u3093 \u306f \u3082\u3058 \u3092 \u306b\u3093\u3057\u304d
\u3059\u308b \u3068\u304d \u305d\u306e \u3055\u3044\u3057\u3087 \u3068
\u3055\u3044\u3054 \u306e \u3082\u3058\u3055\u3048
\u3042\u3063\u3066\u3044\u308c\u3070 \u3058\u3085\u3093\u3070\u3093 \u306f
\u3081\u3061\u3083\u304f\u3061\u3083 \u3067\u3082 \u3061\u3083\u3093\u3068
\u3088\u3081\u308b \u3068\u3044\u3046 \u3051\u3093\u304d\u3085\u3046 \u3092
\u3057\u3063\u3066\u3044\u308b \u3072\u3068 \u3092
\u3072\u3063\u304b\u3051\u308b \u3044\u3068 \u304c
\u3042\u308a\u307e\u3059,
contributors:null,
retweeted:false,
in_reply_to_user_id_str:null,
id_str:68712208368021504,
entities:
{
user_mentions:
[
],
hashtags:
[
],
urls:
[
]
},
in_reply_to_status_id:null,
created_at:Thu May 12 16:20:41 + 2011,
place:null,
source:\u003Ca href=\http:\/\/slum.in\/cybervirus\/\
rel=\nofollow\\u003E\u30c1\u30e7\u30b3\u30dc\u003C\/a\u003E,
in_reply_to_user_id:null,
favorited:false,
geo:null,
user:
{
contributors_enabled:false,
profile_background_image_url:http:\/\/a2.twimg.com
\/images\/themes\/theme14\/bg.gif,
location:,
show_all_inline_media:false,
follow_request_sent:null,
geo_enabled:false,
notifications:null,
id_str:76918274,
favourites_count:3200,
profile_text_color:33,
lang:ja,
created_at:Thu Sep 24 11:39:23 + 2009,
profile_sidebar_fill_color:efefef,
description:\u30c1\u30e7\u30b3\u30dc\u306e\u4eba\u5de5\u7121\u8133bot\u3067\u3059\u3002\u300c\u30d5\u30a9\u30ed\u30fc\u3057\u3066\u300d\u3068\u30ea\u30d7\u30e9\u30a4\u3059\u308b\u3068\u30d5\u30a9\u30ed\u30fc\u8fd4\u3057\u3092\u3057\u307e\u3059\u3002\r\n\u300c\u30ad\u30fc\u30ef\u30fc\u30c9===\u8fd4\u4fe1\u30d1\u30bf\u30fc\u30f3\u300d\u306e\u3088\u3046\u306b\u30ea\u30d7\u30e9\u30a4\u3059\u308b\u3068\u3001\u8fd4\u4fe1\u30d1\u30bf\u30fc\u30f3\u3092\u5b66\u7fd2\u3057\u307e\u3059\u3002,
statuses_count:215641,
profile_background_tile:true,
listed_count:556,
default_profile:false,
following:null,
verified:false,
time_zone:Tokyo,
profile_link_color:00,
profile_image_url:http:\/\/a0.twimg.com
\/profile_images\/123566\/df_normal.png,
friends_count:2295,
profile_sidebar_border_color:ee,
protected:false,
is_translator:false,
url:http:\/\/bit.ly\/cyRus,
screen_name:chocobo_bot,
name:\u30c1\u30e7\u30b3\u30dc,
default_profile_image:false,
profile_use_background_image:true,
id:76918274,
utc_offset:32400,
profile_background_color:131516,
followers_count:4897
},
retweet_count:100+,
id:68712208368021504,
in_reply_to_screen_name:null
},
in_reply_to_status_id:null,
[twitter-dev] Streaming API JSON Samples
Hello there, Does anyone know of some sort of community maintained repository of message types sent by the Streaming API? With a sample of every known type of json message found in the site/user/filter streams. - Juliano -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API JSON Samples
Hi Juliano, From filter stream we received just two types of messages: 'status' (tweets itself) and 'limit' (show how many tweets was suppressed since last reconnection). Abraços da UFRGS!! On Sun, May 1, 2011 at 3:40 PM, Juliano Bortolozzo Solanho juliano.sola...@gmail.com wrote: Hello there, Does anyone know of some sort of community maintained repository of message types sent by the Streaming API? With a sample of every known type of json message found in the site/user/filter streams. - Juliano -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- 氣 -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API JSON Samples
From sample you will receive delete messages. From User Streams you will receive numerous types of events, as well as tweets and DMs. I haven't looked at the documentation recently, but last time I did Twitter was still reserving the right to add message types and recommended you have a code path for message types you had never seen. On Mon, May 2, 2011 at 12:34 PM, Augusto Santos augu...@gemeos.org wrote: Hi Juliano, From filter stream we received just two types of messages: 'status' (tweets itself) and 'limit' (show how many tweets was suppressed since last reconnection). Abraços da UFRGS!! On Sun, May 1, 2011 at 3:40 PM, Juliano Bortolozzo Solanho juliano.sola...@gmail.com wrote: Hello there, Does anyone know of some sort of community maintained repository of message types sent by the Streaming API? With a sample of every known type of json message found in the site/user/filter streams. - Juliano -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- 氣 -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- http://twitter.com/znmeb http://borasky-research.net A mathematician is a device for turning coffee into theorems. -- Paul Erdős -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API 420 error
Hello, We've been using some of the streaming API (specifically /statuses/filter.json and occasionally /statuses/sample.json), and recently I have noticed that we are seeing only 420 errors from our auth tokens for all of our dev, staging, and prod applications. Two sets of those auth tokens are never used for streaming unless I run them manually, sometimes with an hour between tests. status.twitter.com says that there *was* a streaming error, but it was resolved. Anyone else having issues? Thank you, - Josiah -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API statuses/fileter method, follow parameter
Hi All, The streaming api documentation says that mentions, implicit retweets and implicit replies will NOT be returned. However, it looks like they are being returned. For our application we do need to keep track of mentions and the streaming api seems to work fine. Can we rely on it being present always? Is the documentation dated and need to be updated? Thanks, Kumar. - http://dev.twitter.com/pages/streaming_api_methods#follow References unmatched are statuses that were: Mentions (“Hello @user!”) Implicit retweets (“RT @user Says Helloes” without pressing a retweet button) Implicit replies (“@user Hello!”, created without pressing a reply “swoosh” button to set the in_reply_to field) --- -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API statuses/fileter method, follow parameter
With the follow parameter, you should only get real replies retweets. If you need to track all mentions, try the track parameter (i.e. track=@user) Arnaud / @rno http://twitter.com/rno On Tue, Apr 19, 2011 at 9:35 PM, Kumar kumar...@gmail.com wrote: Hi All, The streaming api documentation says that mentions, implicit retweets and implicit replies will NOT be returned. However, it looks like they are being returned. For our application we do need to keep track of mentions and the streaming api seems to work fine. Can we rely on it being present always? Is the documentation dated and need to be updated? Thanks, Kumar. - http://dev.twitter.com/pages/streaming_api_methods#follow References unmatched are statuses that were: Mentions (“Hello @user!”) Implicit retweets (“RT @user Says Helloes” without pressing a retweet button) Implicit replies (“@user Hello!”, created without pressing a reply “swoosh” button to set the in_reply_to field) --- -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API Rate Limiting
Essentially, I'd like to know how rate limiting on the streaming API works. As a research group at a large university, we want to collect data from Twitter for various different projects. Some involve setting different bounding boxes for location queries, and some involve tracking different keywords. For example, we have one project that looks at tweets in the US, and another project that looks at tweets in Switzerland. I would not like the data gathering for one project to affect another project. If I'm rate limited depending on how much data my single connection to the data stream receives - this would affect all projects. If this is the case, can I make multiple connections to the streaming API? The IP addresses would be similar but not the same. Since the projects are different, the accounts would be different too. I find the description of rate limiting and multiple connections on the Streaming API documentation a little confusing. Note: We considered buying data from Gnip, but they don't support the bounding box location queries. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API Rate Limiting
On Thu, Mar 31, 2011 at 1:49 PM, Data Gatherer gatherer...@gmail.com wrote: ... I would not like the data gathering for one project to affect another project. If I'm rate limited depending on how much data my single connection to the data stream receives - this would affect all projects. If this is the case, can I make multiple connections to the streaming API? The IP addresses would be similar but not the same. Since the projects are different, the accounts would be different too. I find the description of rate limiting and multiple connections on the Streaming API documentation a little confusing. I just did a stream search for the and received 2200 tweets in 30 seconds. If that sustained, it would be 6,336,000 per day. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API Rate Limiting
No it won't. Streaming has rate limit with around 1% of firehose, if your search term os too much generic. If your search term or bouding box get too many tweets, you will start receive 'limit' status message as doc said. http://dev.twitter.com/pages/streaming_api_concepts#parsing-responses On Thu, Mar 31, 2011 at 4:27 PM, Jeremy Dunck jdu...@gmail.com wrote: On Thu, Mar 31, 2011 at 1:49 PM, Data Gatherer gatherer...@gmail.com wrote: ... I would not like the data gathering for one project to affect another project. If I'm rate limited depending on how much data my single connection to the data stream receives - this would affect all projects. If this is the case, can I make multiple connections to the streaming API? The IP addresses would be similar but not the same. Since the projects are different, the accounts would be different too. I find the description of rate limiting and multiple connections on the Streaming API documentation a little confusing. I just did a stream search for the and received 2200 tweets in 30 seconds. If that sustained, it would be 6,336,000 per day. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- 氣 -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API Rate Limiting
On Thu, Mar 31, 2011 at 2:48 PM, Augusto Santos augu...@gemeos.org wrote: No it won't. Streaming has rate limit with around 1% of firehose, if your search term os too much generic. If your search term or bouding box get too many tweets, you will start receive 'limit' status message as doc said. http://dev.twitter.com/pages/streaming_api_concepts#parsing-responses Sure, I understand that, I just meant to say that 1% of all tweets is a lot (140M average per day now). If your terms are not very general, you have a lot of head room. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API
Hi, I would like to have a stream of geo-located tweets filtered against pre-defined keywords. As stated in the documentation: Bounding boxes are logical ORs. A locations parameter may be combined with track parameters, but note that all terms are logically ORd, so the query string track=twitteramp;locations=-122.75,36.8,-121.75,37.8 would match any tweets containing the term Twitter (even non-geo tweets) OR coming from the San Francisco area Any idea how to have logical ANDs? I found two ways: 1. filter with the locations parameter and then select only those tweets containing the keywords. Unfortunately with this I will miss a lot of tweets 2. filter with the track parameter and then find the tweet geolocation with Yahoo! Placemaker for non-geo tweets. In this case I will have mostly imprecise locations and a slower service (yahoo! placemaker is slower than a twitter stream) Any idea? carlo -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
Thanks, figured it out. Another question, how many connections are allowed with a shared IP? Any suggestions on multiple streams in one machine with one IP? J On Mon, Feb 28, 2011 at 1:50 PM, John Kalucki j...@twitter.com wrote: Are you specifying the IDs in the URL or in a POST parameter? There's a limit to the URL length that we'll parse, but we'll take huge POST parameters. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Tue, Feb 22, 2011 at 11:22 PM, aquajach aquaj...@gmail.com wrote: Hi, Just started to play with streaming API, but get confused on how many followers id could be tracked with one connection. In basic level of filter, http://dev.twitter.com/doc/post/statuses/filter says 400 followers ids http://dev.twitter.com/pages/streaming_api_methods says 5,000 followers ids Then I tried in local machine, could only follow around 320 ids ( receive 413 if more) and seems multiple connections in one IP are not allowed. Any body here know: Is there any ways to follow a few thousands ids for each authenticated account (with oauth)? Or how to apply for higher access level? Any experience share or answers are appreciated! J -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
Hi J, Glad you worked it out. The Streaming API (stream.twitter.com) does not support multiple streams - only one connection is permitted. This is explained in more detail on our developer resources site: http://dev.twitter.com/pages/streaming_api_concepts#access-rate-limiting Best, @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Mon, Feb 28, 2011 at 12:54 AM, Chen Jack S Y aquaj...@gmail.com wrote: Thanks, figured it out. Another question, how many connections are allowed with a shared IP? Any suggestions on multiple streams in one machine with one IP? J On Mon, Feb 28, 2011 at 1:50 PM, John Kalucki j...@twitter.com wrote: Are you specifying the IDs in the URL or in a POST parameter? There's a limit to the URL length that we'll parse, but we'll take huge POST parameters. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Tue, Feb 22, 2011 at 11:22 PM, aquajach aquaj...@gmail.com wrote: Hi, Just started to play with streaming API, but get confused on how many followers id could be tracked with one connection. In basic level of filter, http://dev.twitter.com/doc/post/statuses/filter says 400 followers ids http://dev.twitter.com/pages/streaming_api_methods says 5,000 followers ids Then I tried in local machine, could only follow around 320 ids ( receive 413 if more) and seems multiple connections in one IP are not allowed. Any body here know: Is there any ways to follow a few thousands ids for each authenticated account (with oauth)? Or how to apply for higher access level? Any experience share or answers are appreciated! J -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
Although this is specified at streaming API docs, it's possible to connect two diferent users at the same IP address. On Mon, Feb 28, 2011 at 5:25 PM, Matt Harris thematthar...@twitter.comwrote: Hi J, Glad you worked it out. The Streaming API (stream.twitter.com) does not support multiple streams - only one connection is permitted. This is explained in more detail on our developer resources site: http://dev.twitter.com/pages/streaming_api_concepts#access-rate-limiting Best, @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Mon, Feb 28, 2011 at 12:54 AM, Chen Jack S Y aquaj...@gmail.comwrote: Thanks, figured it out. Another question, how many connections are allowed with a shared IP? Any suggestions on multiple streams in one machine with one IP? J On Mon, Feb 28, 2011 at 1:50 PM, John Kalucki j...@twitter.com wrote: Are you specifying the IDs in the URL or in a POST parameter? There's a limit to the URL length that we'll parse, but we'll take huge POST parameters. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Tue, Feb 22, 2011 at 11:22 PM, aquajach aquaj...@gmail.com wrote: Hi, Just started to play with streaming API, but get confused on how many followers id could be tracked with one connection. In basic level of filter, http://dev.twitter.com/doc/post/statuses/filter says 400 followers ids http://dev.twitter.com/pages/streaming_api_methods says 5,000 followers ids Then I tried in local machine, could only follow around 320 ids ( receive 413 if more) and seems multiple connections in one IP are not allowed. Any body here know: Is there any ways to follow a few thousands ids for each authenticated account (with oauth)? Or how to apply for higher access level? Any experience share or answers are appreciated! J -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- 氣 -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
From the developer resource, it is said that 'Each account may create only one standing connection to the Streaming API'. While it is possible to have a few streams with different users' account through OAuth? If yes, what's the limit? Thanks, J On Tue, Mar 1, 2011 at 4:25 AM, Matt Harris thematthar...@twitter.com wrote: Hi J, Glad you worked it out. The Streaming API (stream.twitter.com) does not support multiple streams - only one connection is permitted. This is explained in more detail on our developer resources site: http://dev.twitter.com/pages/streaming_api_concepts#access-rate-limiting Best, @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Mon, Feb 28, 2011 at 12:54 AM, Chen Jack S Y aquaj...@gmail.com wrote: Thanks, figured it out. Another question, how many connections are allowed with a shared IP? Any suggestions on multiple streams in one machine with one IP? J On Mon, Feb 28, 2011 at 1:50 PM, John Kalucki j...@twitter.com wrote: Are you specifying the IDs in the URL or in a POST parameter? There's a limit to the URL length that we'll parse, but we'll take huge POST parameters. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Tue, Feb 22, 2011 at 11:22 PM, aquajach aquaj...@gmail.com wrote: Hi, Just started to play with streaming API, but get confused on how many followers id could be tracked with one connection. In basic level of filter, http://dev.twitter.com/doc/post/statuses/filter says 400 followers ids http://dev.twitter.com/pages/streaming_api_methods says 5,000 followers ids Then I tried in local machine, could only follow around 320 ids ( receive 413 if more) and seems multiple connections in one IP are not allowed. Any body here know: Is there any ways to follow a few thousands ids for each authenticated account (with oauth)? Or how to apply for higher access level? Any experience share or answers are appreciated! J -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
Are you specifying the IDs in the URL or in a POST parameter? There's a limit to the URL length that we'll parse, but we'll take huge POST parameters. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Tue, Feb 22, 2011 at 11:22 PM, aquajach aquaj...@gmail.com wrote: Hi, Just started to play with streaming API, but get confused on how many followers id could be tracked with one connection. In basic level of filter, http://dev.twitter.com/doc/post/statuses/filter says 400 followers ids http://dev.twitter.com/pages/streaming_api_methods says 5,000 followers ids Then I tried in local machine, could only follow around 320 ids ( receive 413 if more) and seems multiple connections in one IP are not allowed. Any body here know: Is there any ways to follow a few thousands ids for each authenticated account (with oauth)? Or how to apply for higher access level? Any experience share or answers are appreciated! J -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API limits...
This is documented in painful detail here: http://dev.twitter.com/pages/streaming_api_concepts#updating-filter-predicates . If you connect a second time, you should get a TCP Close or Reset on the first connection. It sounds like your client library isn't detecting the connection close. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Thu, Feb 24, 2011 at 1:08 PM, Josiah Carlson josiah.carl...@gmail.comwrote: Now that I've got OAuth with statuses/follow.json working, I've been working through building a small part of our app. Part of the streaming API docs state that only one connection is allowed (reasonable). Upon making a second connection, the first no longer receives any data (not even anti-timeout newlines), nor does it get connected by the server. On my end of things, I've written an async client which can detect such a condition (it watches a shared Redis key looking for a changed state when it doesn't receive any data for a while), and automatically disconnects. The streaming API docs also state that repeated reconnections, etc., are frowned upon and may result in banning. My question is simple: how often can I reconnect to follow different people/keywords? Obviously ten times a second is well beyond reasonable and would probably get us banned in seconds. But isat most once every 5 minutes okay? At most once every minute? At what level would we be safe? Thank you, - Josiah -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API limits...
John, Thank you for getting back to me. The doc lists a For example, reconnect no more than twice every four minutes, or three times per six minutes, or some similar metric. but doesn't give a Don't reconnect more than a few times a minute unless you are retrying automatically due to failures, but remember to use exponential backoff to a reasonable upper limit depending on the type of failure. ;) (also, mistype earlier: nor does it get connected by the server should have read nor does it get disconnected by the server) The funny thing is, I've actually discarded the client that I was using, and have written one from scratch using Python's asynchronous sockets. Every other time I've written async socket clients/servers, when a connection is closed, the socket will be seen as readable but when read will return zero bytes; this is a semantic that has worked for me every other time in the past, is the standard behavior for countless async (and sync libraries), and is the most reliable cross-platform way to detect socket disconnect. Worst-case scenario, the socket will come up as being in an error condition (I handle that too, but it hasn't come up when connecting to stream.twitter.com). Looking at a wireshark dump, I see some sort of disconnect-like packet, but the socket is never seen as readable or in an error condition according to select (Python wraps the standard system routines), and both lsof AND netstat see the sockets as being established (as opposed to disconnected, time_wait, etc.) I have worked around the seeming lack of connect, I was just reporting what I was experiencing. Regards, - Josiah On Sunday, February 27, 2011 9:56:04 PM UTC-8, John Kalucki wrote: This is documented in painful detail here: http://dev.twitter.com/pages/streaming_api_concepts#updating-filter-predicates . If you connect a second time, you should get a TCP Close or Reset on the first connection. It sounds like your client library isn't detecting the connection close. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Thu, Feb 24, 2011 at 1:08 PM, Josiah Carlson josiah@gmail.comwrote: Now that I've got OAuth with statuses/follow.json working, I've been working through building a small part of our app. Part of the streaming API docs state that only one connection is allowed (reasonable). Upon making a second connection, the first no longer receives any data (not even anti-timeout newlines), nor does it get connected by the server. On my end of things, I've written an async client which can detect such a condition (it watches a shared Redis key looking for a changed state when it doesn't receive any data for a while), and automatically disconnects. The streaming API docs also state that repeated reconnections, etc., are frowned upon and may result in banning. My question is simple: how often can I reconnect to follow different people/keywords? Obviously ten times a second is well beyond reasonable and would probably get us banned in seconds. But isat most once every 5 minutes okay? At most once every minute? At what level would we be safe? Thank you, - Josiah -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
Hey dude. You gave me a hint, but not tweetstream, that is twitterstream, which is newer and works for me. On Fri, Feb 25, 2011 at 2:12 AM, M. Edward (Ed) Borasky zn...@borasky-research.net wrote: On Thu, 24 Feb 2011 17:34:52 +0800, Chen Jack S Y aquaj...@gmail.com wrote: Thanks, dude. My problem is still there though. When I try the streaming api with curl in command line, everything goes well and it tracks a few thousands of ids successfully. While using eventmachine (together with em-http-request) ruby gem, haven't found any solutions to track more 400 ids but keep receiving 413 response errors. Kind of weird. Is this the tweetstream Ruby gem? If their repository is still on Github, it hasn't been updated in over a year. In particular, they haven't added code for User Streams or oAuth. Could they be using an incorrect endpoint or something like that? -- http://twitter.com/znmeb http://borasky-research.net A mathematician is a device for turning coffee into theorems. -- Paul Erdős -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
Thanks, dude. My problem is still there though. When I try the streaming api with curl in command line, everything goes well and it tracks a few thousands of ids successfully. While using eventmachine (together with em-http-request) ruby gem, haven't found any solutions to track more 400 ids but keep receiving 413 response errors. Kind of weird. J On Thu, Feb 24, 2011 at 5:52 AM, Matt Harris thematthar...@twitter.comwrote: Hi J, The authoritative information for the Streaming API is under the /pages/ path and you should use that for guidance. The number of connections you are allowed to the Streaming API is described in the Streaming API Concepts document: http://dev.twitter.com/pages/streaming_api_concepts It says: Each account may create only one standing connection to the Streaming API. Subsequent connections from the same account may cause previously established connections to be disconnected. Excessive connection attempts, regardless of success, will result in an automatic ban of the client's IP address. Continually failing connections will result in your IP address being blacklisted from all Twitter access. When tracking users using the Streaming API the default level allows 5000 follower IDs to be tracked. Make sure the user_ids are specified with the follow parameter and not the track parameter. Best, @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Tue, Feb 22, 2011 at 11:22 PM, aquajach aquaj...@gmail.com wrote: Hi, Just started to play with streaming API, but get confused on how many followers id could be tracked with one connection. In basic level of filter, http://dev.twitter.com/doc/post/statuses/filter says 400 followers ids http://dev.twitter.com/pages/streaming_api_methods says 5,000 followers ids Then I tried in local machine, could only follow around 320 ids ( receive 413 if more) and seems multiple connections in one IP are not allowed. Any body here know: Is there any ways to follow a few thousands ids for each authenticated account (with oauth)? Or how to apply for higher access level? Any experience share or answers are appreciated! J -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
If it's working for you in curl, then it's likely something either in your code or the library you're using. Are you using OAuth to authenticate or basic auth? Either way, if you can get a trace of the exact POST body and URL you are sending when issuing the request from eventmachine, it will likely contain the clues as to what's going wrong. @episod http://twitter.com/episod - Taylor Singletary - Twitter Developer Advocate On Thu, Feb 24, 2011 at 1:34 AM, Chen Jack S Y aquaj...@gmail.com wrote: Thanks, dude. My problem is still there though. When I try the streaming api with curl in command line, everything goes well and it tracks a few thousands of ids successfully. While using eventmachine (together with em-http-request) ruby gem, haven't found any solutions to track more 400 ids but keep receiving 413 response errors. Kind of weird. J On Thu, Feb 24, 2011 at 5:52 AM, Matt Harris thematthar...@twitter.comwrote: Hi J, The authoritative information for the Streaming API is under the /pages/ path and you should use that for guidance. The number of connections you are allowed to the Streaming API is described in the Streaming API Concepts document: http://dev.twitter.com/pages/streaming_api_concepts It says: Each account may create only one standing connection to the Streaming API. Subsequent connections from the same account may cause previously established connections to be disconnected. Excessive connection attempts, regardless of success, will result in an automatic ban of the client's IP address. Continually failing connections will result in your IP address being blacklisted from all Twitter access. When tracking users using the Streaming API the default level allows 5000 follower IDs to be tracked. Make sure the user_ids are specified with the follow parameter and not the track parameter. Best, @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Tue, Feb 22, 2011 at 11:22 PM, aquajach aquaj...@gmail.com wrote: Hi, Just started to play with streaming API, but get confused on how many followers id could be tracked with one connection. In basic level of filter, http://dev.twitter.com/doc/post/statuses/filter says 400 followers ids http://dev.twitter.com/pages/streaming_api_methods says 5,000 followers ids Then I tried in local machine, could only follow around 320 ids ( receive 413 if more) and seems multiple connections in one IP are not allowed. Any body here know: Is there any ways to follow a few thousands ids for each authenticated account (with oauth)? Or how to apply for higher access level? Any experience share or answers are appreciated! J -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
On Thu, 24 Feb 2011 17:34:52 +0800, Chen Jack S Y aquaj...@gmail.com wrote: Thanks, dude. My problem is still there though. When I try the streaming api with curl in command line, everything goes well and it tracks a few thousands of ids successfully. While using eventmachine (together with em-http-request) ruby gem, haven't found any solutions to track more 400 ids but keep receiving 413 response errors. Kind of weird. Is this the tweetstream Ruby gem? If their repository is still on Github, it hasn't been updated in over a year. In particular, they haven't added code for User Streams or oAuth. Could they be using an incorrect endpoint or something like that? -- http://twitter.com/znmeb http://borasky-research.net A mathematician is a device for turning coffee into theorems. -- Paul Erdős -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API limits...
Now that I've got OAuth with statuses/follow.json working, I've been working through building a small part of our app. Part of the streaming API docs state that only one connection is allowed (reasonable). Upon making a second connection, the first no longer receives any data (not even anti-timeout newlines), nor does it get connected by the server. On my end of things, I've written an async client which can detect such a condition (it watches a shared Redis key looking for a changed state when it doesn't receive any data for a while), and automatically disconnects. The streaming API docs also state that repeated reconnections, etc., are frowned upon and may result in banning. My question is simple: how often can I reconnect to follow different people/keywords? Obviously ten times a second is well beyond reasonable and would probably get us banned in seconds. But isat most once every 5 minutes okay? At most once every minute? At what level would we be safe? Thank you, - Josiah -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API access level limit
Hi, Just started to play with streaming API, but get confused on how many followers id could be tracked with one connection. In basic level of filter, http://dev.twitter.com/doc/post/statuses/filter says 400 followers ids http://dev.twitter.com/pages/streaming_api_methods says 5,000 followers ids Then I tried in local machine, could only follow around 320 ids ( receive 413 if more) and seems multiple connections in one IP are not allowed. Any body here know: Is there any ways to follow a few thousands ids for each authenticated account (with oauth)? Or how to apply for higher access level? Any experience share or answers are appreciated! J -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API access level limit
Hi J, The authoritative information for the Streaming API is under the /pages/ path and you should use that for guidance. The number of connections you are allowed to the Streaming API is described in the Streaming API Concepts document: http://dev.twitter.com/pages/streaming_api_concepts It says: Each account may create only one standing connection to the Streaming API. Subsequent connections from the same account may cause previously established connections to be disconnected. Excessive connection attempts, regardless of success, will result in an automatic ban of the client's IP address. Continually failing connections will result in your IP address being blacklisted from all Twitter access. When tracking users using the Streaming API the default level allows 5000 follower IDs to be tracked. Make sure the user_ids are specified with the follow parameter and not the track parameter. Best, @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Tue, Feb 22, 2011 at 11:22 PM, aquajach aquaj...@gmail.com wrote: Hi, Just started to play with streaming API, but get confused on how many followers id could be tracked with one connection. In basic level of filter, http://dev.twitter.com/doc/post/statuses/filter says 400 followers ids http://dev.twitter.com/pages/streaming_api_methods says 5,000 followers ids Then I tried in local machine, could only follow around 320 ids ( receive 413 if more) and seems multiple connections in one IP are not allowed. Any body here know: Is there any ways to follow a few thousands ids for each authenticated account (with oauth)? Or how to apply for higher access level? Any experience share or answers are appreciated! J -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming Api returning tweets with NULL value for object place
Hello, I am using the streaming api in order to be able to save tweets that uses the geoJSON place key of the returned json object. Tt was working fine last Tuesday, Feb 15, But now, there seems to be a problem with the place tag of the tweet object. I was just wondering if it's just me or is there some sort of bug or changes have been made in the API? Aci Cartagena -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API vs. Search API: no API returns 95% of intented tweets
Hi,
this problem was already posted to the twitter4j mailing list [1]. Not
sure if it is an issue with my code, twitter4j or an API issue... user
reported similar problems in the past [2].
First:
I'm doing a 100 tweet search (without paging) every 5 minutes e.g.
against 'twitter search'. I get a set of tweets A - excluding the
duplicates, of course. I get approx 5 new tweets for every 5 minutes,
so 100 tweets as pageSize should be perfectly sufficient to get all
tweets.
Second:
When I'm doing a streaming filter request for the same terms 'twitter
search' then I'm getting a set of tweets B.
The problem is: combining A and B ('C=A v B') gives me a set C where
the count of C is more than 10% larger then A or B, which means that
neither with search nor streaming API I can catch a nearly complete
set of tweets.
E.g. doing this for 3 hours I'm getting 254 tweets (A) for the search
and 257 tweets (B) for the streaming but the combined set C has 337
tweets!
Is this a bug in my code or could this be an API issue?
BTW: I don't assume 100% correctness, I only want something above
90% :) especially for such relatively infrequent terms, where users
can, should and have noticed it.
Regards,
Peter.
[1]
http://groups.google.com/group/twitter4j/msg/d959e6257ceb452f
[2]
http://groups.google.com/group/twitter-development-talk/browse_thread/thread/71ab5cc666113c9e
http://blog.tweetsmarter.com/twitter-downtime/twitters-dirty-secret-they-dont-show-you-all-tweets/
--
http://jetwick.com Twitter Search without Noise
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API vs. Search API: no API returns 95% of intented tweets
I don't get that big a discrepancy, but I do get different results
from search and streaming. I use streaming for real-time delivery, and
then either search or user timelines to backfill missing tweets. As
long as the flow makes this possible within rate limits this gets me
the greatest number of results, but still not 100%. I accept that 100%
ain't gonna happen. You should get within your desired 95% though.
That is a realistic goal.
On Tue, Feb 15, 2011 at 6:36 AM, Karussell tableyourt...@googlemail.com wrote:
Hi,
this problem was already posted to the twitter4j mailing list [1]. Not
sure if it is an issue with my code, twitter4j or an API issue... user
reported similar problems in the past [2].
First:
I'm doing a 100 tweet search (without paging) every 5 minutes e.g.
against 'twitter search'. I get a set of tweets A - excluding the
duplicates, of course. I get approx 5 new tweets for every 5 minutes,
so 100 tweets as pageSize should be perfectly sufficient to get all
tweets.
Second:
When I'm doing a streaming filter request for the same terms 'twitter
search' then I'm getting a set of tweets B.
The problem is: combining A and B ('C=A v B') gives me a set C where
the count of C is more than 10% larger then A or B, which means that
neither with search nor streaming API I can catch a nearly complete
set of tweets.
E.g. doing this for 3 hours I'm getting 254 tweets (A) for the search
and 257 tweets (B) for the streaming but the combined set C has 337
tweets!
Is this a bug in my code or could this be an API issue?
BTW: I don't assume 100% correctness, I only want something above
90% :) especially for such relatively infrequent terms, where users
can, should and have noticed it.
Regards,
Peter.
[1]
http://groups.google.com/group/twitter4j/msg/d959e6257ceb452f
[2]
http://groups.google.com/group/twitter-development-talk/browse_thread/thread/71ab5cc666113c9e
http://blog.tweetsmarter.com/twitter-downtime/twitters-dirty-secret-they-dont-show-you-all-tweets/
--
http://jetwick.com Twitter Search without Noise
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
--
Adam Green
Twitter API Consultant and Trainer
http://140dev.com
@140dev
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API vs. Search API: no API returns 95% of intented tweets
If you examine set C, do they contain matches on fields other than the Tweet
text? To increase recall, search sometimes includes keywords in followed
links and other techniques.
Also, are you getting rate limit messages on the Streaming API?
-John Kalucki
http://twitter.com/jkalucki
Twitter, Inc.
On Tue, Feb 15, 2011 at 3:36 AM, Karussell tableyourt...@googlemail.comwrote:
Hi,
this problem was already posted to the twitter4j mailing list [1]. Not
sure if it is an issue with my code, twitter4j or an API issue... user
reported similar problems in the past [2].
First:
I'm doing a 100 tweet search (without paging) every 5 minutes e.g.
against 'twitter search'. I get a set of tweets A - excluding the
duplicates, of course. I get approx 5 new tweets for every 5 minutes,
so 100 tweets as pageSize should be perfectly sufficient to get all
tweets.
Second:
When I'm doing a streaming filter request for the same terms 'twitter
search' then I'm getting a set of tweets B.
The problem is: combining A and B ('C=A v B') gives me a set C where
the count of C is more than 10% larger then A or B, which means that
neither with search nor streaming API I can catch a nearly complete
set of tweets.
E.g. doing this for 3 hours I'm getting 254 tweets (A) for the search
and 257 tweets (B) for the streaming but the combined set C has 337
tweets!
Is this a bug in my code or could this be an API issue?
BTW: I don't assume 100% correctness, I only want something above
90% :) especially for such relatively infrequent terms, where users
can, should and have noticed it.
Regards,
Peter.
[1]
http://groups.google.com/group/twitter4j/msg/d959e6257ceb452f
[2]
http://groups.google.com/group/twitter-development-talk/browse_thread/thread/71ab5cc666113c9e
http://blog.tweetsmarter.com/twitter-downtime/twitters-dirty-secret-they-dont-show-you-all-tweets/
--
http://jetwick.com Twitter Search without Noise
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker:
http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API setLocations Accuracy
Good Afternoon I am using the StreamingAPI with a boundary box but finding it to be inaccurate. I don't mind thinking outside the box :) but my a large % of my results are 20 miles outside my selected boundary box. I have not set any other search criteria. Any thoughts? -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API setLocations Accuracy
Any exaples of tweets outside the box and the box itself? On Mon, Feb 14, 2011 at 6:59 PM, 1537 News 1537n...@gmail.com wrote: Good Afternoon I am using the StreamingAPI with a boundary box but finding it to be inaccurate. I don't mind thinking outside the box :) but my a large % of my results are 20 miles outside my selected boundary box. I have not set any other search criteria. Any thoughts? -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- 氣 -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API maintenance: brief delivery pause and an increased likelihood for duplicate tweets
We are performing a maintenance activity shortly that will increase the likelihood of duplicate tweets and other messages on all Streaming APIs: User Streams, Site Streams, and stream.twitter.com. There may also be a brief pause in delivery. No tweets or other messages will be lost during this maintenance event. The maintenance window is predicted to be approximately 2 minutes long and may occur between 1:15pm PST / 21:15 UTC and 3:30pm PST / 23:15 UTC. Note that this possibility of duplications has always been documented on the Streaming API at: http://dev.twitter.com/pages/streaming_api_concepts#quality-of-service. -John Kalucki Twitter, Inc. http://twitter.com/jkalucki -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API filter terms
Hi, I have a question about the streaming API. We're currently consuming the Spritzer stream, this is fine. However, if we use the filter with a keyword filter, let's say 'boats', does this give us just a subset of the spritzer stream or does it give 'boats' across the whole of Twitter? Saving CPU cycles isn't a problem, we can handle the stream, but we would like to get the most data we can around our filter keyword(s). Thanks, S. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API stopwords
Hello! Does anyone have experience using a list of stopwords to reduce noise when making streaming API requests to statuses/filter? I have a basic list (e.g. a,an, and, etc.) but wonder if anyone out there is using something more comprehensive. Thanks, Kenny -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API: retweeted always false
Hi, I'm with a problem using streaming API, I had tracked 'amazon' and no tweets were returned with attribute 'retweeted' equals true, when in fact some tweets were retweeted. Does this occur to anyone else? -- Regards, * Fabrício Ferrari de Campos* | @fabricioffc http://twitter.com/FabricioFFC *Blog:* QualidadeBR http://qualidadebr.wordpress.com/ *LinkedIn:* br.linkedin.com/in/fabricioferraridecampos -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API - Multiple connections on 1 IP?
Hello everyone, Just a simple question: can I launch multiple connections to Streaming API using different accounts, but on the same machine? The official document seems don't say this clearly: Each account may create only one standing connection to the Streaming API. Subsequent connections from the same account may cause previously established connections to be disconnected. Excessive connection attempts, regardless of success, will result in an automatic ban of the client's IP address. Continually failing connections will result in your IP address being blacklisted from all Twitter access. Many thanks for the help! epomqo -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API - Multiple connections on 1 IP?
Yes you can. On Thu, Dec 16, 2010 at 8:23 AM, epomqo wenzi0...@gmail.com wrote: Hello everyone, Just a simple question: can I launch multiple connections to Streaming API using different accounts, but on the same machine? The official document seems don't say this clearly: Each account may create only one standing connection to the Streaming API. Subsequent connections from the same account may cause previously established connections to be disconnected. Excessive connection attempts, regardless of success, will result in an automatic ban of the client's IP address. Continually failing connections will result in your IP address being blacklisted from all Twitter access. Many thanks for the help! epomqo -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- 氣 -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API, GeoLocation Filtering
Hello all, Just a general question about a location-based predicate. According to the streaming API docs... Only tweets that are both created using the Geotagging API and are placed from within a tracked bounding box will be included in the stream... But, as other have pointed out, a lot/most of the statuses that come back from such a filtered query don't always contain GeoLocation data. So... what's going on here? Am I missing something? I'm not necessarily concerned that I'm getting more data than I can use (although it's unfortunate from a rate-limiting standpoint), but my curiosity is certainly piqued. For the project I'm working on (I'm a grad student), I'm trying to listen to activity in different locations and sonify it on the fly. In keeping with best practices, I build one filter predicate for a handful of locations and try to make sense of the data as it comes by, rather than opening a handful of streams. As a result, I end up with a great many tweets I can't attribute to a geographic source, sans GeoLocation data. Based on the wording above though, I understand it to mean a status that clears the filter should have that information. Any insight would be greatly appreciated. Best, Eric Humphrey -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API firehose visibility
Yes, where firehose is the stream of all public statuses, with some low-quality accounts removed. On Wed, Dec 1, 2010 at 3:52 PM, dburkes dbur...@gmail.com wrote: If I am using the statuses/filter streaming API, with a track= query that is not overly broad, and my client never receives any limit responses, can I assume that the results returned represent all the results from the entire firehose? In other words, in the absence of limit response, is my visibility into the firehose 100%? Thanks- Danny Burkes -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API firehose visibility
If I am using the statuses/filter streaming API, with a track= query that is not overly broad, and my client never receives any limit responses, can I assume that the results returned represent all the results from the entire firehose? In other words, in the absence of limit response, is my visibility into the firehose 100%? Thanks- Danny Burkes -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API 'restricted track' 'partner track' Levels
Hi, I was wondering about what happens to the developers/companies who are at 'restricted track' or 'partner track' levels for streaming API access? Also, If a developer wants to have his applications Streaming api access-roles elevated, who does he contact - Twitter or Gnip? And finally, for the keywords with very lesser frequencies, Streaming API, for last two weeks, is dropping almost everything. So how random is this sample that I'm looking for? Does it make sense to detect these kind of keywords early and then use Search-API tto discover conversations? Thanks, dj -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] streaming api delete status notice problem
We aggregate tweets of a few thousands of people using Streaming API follow. Streaming api gives us tweets of other people who mention our set of users. The problem rises when the other people delete their tweets. These delete notifications do not reach us and we can not delete those tweets. We just try to show who said what about a twitter-er. But not the deleted ones. A similar problem occurs while we try to aggregate retweets. The undo retweet messages of unfollowed users are not receieved. We do want to show true and accurate results. Any quick fix suggestion is appreciated. Thanks, -- Furkan Kuru -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API and OAuth
Please update your documentation [1] for more detail information on authenticating on the Streaming API with OAuth. We need to know the same type of information that you currently provide [2] for REST OAuth. [1] http://developer.twitter.com/pages/stre0aming_api_concepts#authentication [2] http://developer.twitter.com/pages/auth -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API and OAuth
Streaming API doesn't differ from the REST API with it's authentication. Both use OAuth 1.0. Tom On 11/8/10 11:55 PM, Dewald Pretorius wrote: Please update your documentation [1] for more detail information on authenticating on the Streaming API with OAuth. We need to know the same type of information that you currently provide [2] for REST OAuth. [1] http://developer.twitter.com/pages/stre0aming_api_concepts#authentication [2] http://developer.twitter.com/pages/auth -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API and authentication
I've been using the streaming library in one of my apps. My understanding is that the streaming API still supports Basic Authentication, so I don't need to make any changes. My app did stop working with messages indicating that I am not being properly authenticated. Do I need to switch to oAuth here as well (not user streams)? -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API and authentication
The streaming API supports basic auth *now*. At some point in the future it will not. If you're developing something new with basic auth you're setting yourself up for more work in the not too distant future. As far as the present message, only userstreams and sitestreams require oauth currently, so it's not likely an oauth issue. ---Mark http://twitter.com/mccv On Tue, Nov 2, 2010 at 2:27 PM, EastSideDev eastside...@gmail.com wrote: I've been using the streaming library in one of my apps. My understanding is that the streaming API still supports Basic Authentication, so I don't need to make any changes. My app did stop working with messages indicating that I am not being properly authenticated. Do I need to switch to oAuth here as well (not user streams)? -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API, Basic Auth Ok, OAuth Unauthorized?
Hey Bradley, This is another instance of the the ongoing (and as yet un-answered sadly) question I have in the mailing list about my client (which iirc you're using) See : http://github.com/ciaranj/node-oauth/issues#issue/7 and http://groups.google.co.uk/group/twitter-development-talk/browse_thread/thread/fe6e50d60d1e95fa/f06e93e761183bf1?hl=enlnk=gstq=javajunky#f06e93e761183bf1 oh and also ( :( ) http://groups.google.co.uk/group/twitter-development-talk/browse_thread/thread/4aedc185cee34d81?hl=en# The bad news is twitter don't seem to want to tell me if I'm wrong, or they're wrong (I don't care, just want to know what to fix ! :( ) .. the good news is the work around is to url encode your parameters before you pass them off to my client ( you won't need to do this with any other OAuth provider I've yet come across fwiw, but if they come back and say yes, thats deliberate, yes its different, I'll hardcode it into the client so you don't need to worry about it *sigh* (or even better, the client is wrong, we're right and we do it the same as everyone else..which would be an ideal outcome) ) Take Care - cj. On Mon, Oct 25, 2010 at 10:20 PM, bradley.meck bradley.m...@gmail.com wrote: I have a simple oauth client that I use to post status updates currently, however, when I added the ability to track statuses with the Stream api using OAuth I noticed I could not connect, with Unauthorized 401 being the reply to anything I sent it. I looked into the documentation and it seems to be a simple request using the same OAuth style as the normal api. After searching threads I noticed the rate limiting and so I have left my app alone for extended periods of time and still I get 401s. I tested against basic auth, and the code worked! M, that was odd. So unless I am mistaken I am doing something wrong, but I am posting to the right url and mirroring my basic auth test to no avail. The code is at: http://github.com/bmeck/Simple-Bot/blob/master/modules/twitter.js the track() function is the boilerplate that is in question oa.post is a simple rest wrapper for oauth POST. Any help or directions as to where to go from here is much appreciated. Cheers, Bradley -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API sampling and filter limiting algorithms switched to new status id
The new status id format, previewed as new_id, requires slightly different algorithms for sampling and imposing filter limits on the Streaming API. In preparation for the big switch later today, we've cut over to using the new_id for these cases at about 6:30am PDT, 13:30 UTC. Only the most careful observer of the sampled streams should notice a difference. Consumers of the Streaming filter endpoint (track, loc, etc.) that aggressively push the rate limits may notice some minor differences in when the limits are imposed. We may tweak this algorithm and the associated Filter limits in the future. Overall, the results for Sample and Filter should be very similar to the previous sequentially generated status id system. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API new_id test
We streamed the new_id field for about 15 minutes this morning, starting at about 10:05 PDT, 17:05 UTC until about 10:15 / 17:15 UTC. If your streaming consumer had problems during this period: 1) Check your markup parser. 2) Respond to this thread. Barring any issues, we'll nail this setting up tomorrow at about the same time. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API test: Adding new_id field to statuses at 17:00 UTC Sept 29
Tomorrow, Wednesday September 29, at 10:00 AM PDT / 17:00 UTC, we will briefly introduce a field called new_id to statuses delivered over the Streaming API. If this 10 minute test is successful, we will enable the new_id field continuously on Thursday September 30th at about the same time. Note that timelines returned by the REST and Search APIs will not contain this field. This new_id field will allow applications to preview the new status id generation scheme before the primary key transition scheduled for Tuesday October 12th. For more information on our status id transition: http://groups.google.com/group/twitter-api-announce/browse_thread/thread/daf6298d0fdcbc87 http://groups.google.com/group/twitter-api-announce/browse_thread/thread/7982e3b037eeef95 -John Kalucki http://twitter.com/jkalucki Twitter, Inc. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API question..
hi there, I am using the streaming API (the statuses/filter), and I get a lot of tweets in spanish. I wanted to know if there is a way to get results only in English? I tried to use the geo-location of the USA only, but it didn't help much. anybody? thanks, Omri -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API question..
Hi Omri, Sorry, there's no option currently to filter by language. Taylor On Mon, Sep 27, 2010 at 1:50 AM, omri omridek...@gmail.com wrote: hi there, I am using the streaming API (the statuses/filter), and I get a lot of tweets in spanish. I wanted to know if there is a way to get results only in English? I tried to use the geo-location of the USA only, but it didn't help much. anybody? thanks, Omri -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API and oAuth
Hello, Does the streaming API still support Basic Auth? Is there going to be any python module like tweepy that will support oAuth and the streaming API? Thanks. Joel -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en
[twitter-dev] Streaming API
Hi All, I am a bit torn on which API to use for my application. Have looked in twitter dev pages and did search internet but couldn't find comprehensive information. I need to get tweets from certain users in my application. I don't know how many users at this moment but I would like to build a scalable solution. I don't care if the tweets are not received real time as long as the delay is not in hours or days. My understanding is that I could either use streaming API (statuses/ filter) or timelines (friends_timeline). 1) Is there a reason for someone to use one over other (other than real time updates using streaming api)? 2) Which method twitter team recommends developer to use for future applications? 3) Are streaming APIs available for production? Sorry if some or all of these questions are already answered somewhere. I couldn't locate the information hence posting here. Any inputs in this regard will be greatly appreciated. Thanks in advance, Rajat -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en
Re: [twitter-dev] Streaming API
You should use the Streaming API for large scale integrations. We're not doing much whitelisting for the REST API at the moment, but the shadow role on the Streaming API is generally available. If all of your users are public, use the shadow role on stream.twitter.com/1/statuses to get all of their statuses, but not their timelines. This feature has been in production for over a year. If you have OAuth tokens for the users, you can use the Site Streams beta to get protected user's statuses, but currently you will also get a lot of other information for them as well. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Tue, Sep 7, 2010 at 6:39 AM, Tom van der Woerdt i...@tvdw.eu wrote: 1) Yes: the timelines are easier to use and more flexible, while the streaming API gives you realtime statuses and no problems with rate limits. 2) Streams. I'm not part of the Twitter team, but I really think that they would recommend Streams. Of course, it depends a bit on the situation. 3) I think so, yes. Tom On 9/7/10 3:31 PM, Rajat wrote: Hi All, I am a bit torn on which API to use for my application. Have looked in twitter dev pages and did search internet but couldn't find comprehensive information. I need to get tweets from certain users in my application. I don't know how many users at this moment but I would like to build a scalable solution. I don't care if the tweets are not received real time as long as the delay is not in hours or days. My understanding is that I could either use streaming API (statuses/ filter) or timelines (friends_timeline). 1) Is there a reason for someone to use one over other (other than real time updates using streaming api)? 2) Which method twitter team recommends developer to use for future applications? 3) Are streaming APIs available for production? Sorry if some or all of these questions are already answered somewhere. I couldn't locate the information hence posting here. Any inputs in this regard will be greatly appreciated. Thanks in advance, Rajat -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en
[twitter-dev] Streaming api returning 401 error on phrase tracking(logical AND)
I am getting strange results when using streaming api with and without Oauth. Without oauth i am able to track phrases whereas with oauth i get a 401 error, able to track normal words with oauth.Is there any restriction on phrase tracking? Could not find any pointers in the docs.btw i use twitter4j. Thanks, Karthik -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en
Re: [twitter-dev] Streaming API limits
On 8/16/10 4:38 PM, Thiago Souza wrote: Hi, I know that it's possible to track up to 200 keywords per account, but how many accounts per IP is allowed? Regards, Thiago Souza Hi Thiago, I don't think that Twitter will give you these numbers. Just don't create more connections than necessary, and if you need more track keywords, you should consider asking Twitter for an upgrade of your account. Tom
Re: [twitter-dev] Streaming API limits
Hi Thiago, We allow each account to have one standing connection to the streaming API. Multiple connections receive the same stream of Tweets so connecting multiple times would not give access to any more Tweets. If you need more to have more track keywords you can contact us explaining your use case so the team can make a decision and if necessary grant you higher access. You can read more about access and rate limiting on the streaming API in our developer documentation: http://dev.twitter.com/pages/streaming_api_concepts#access-rate-limiting Hope that helps, Matt On Mon, Aug 16, 2010 at 10:05 AM, Tom van der Woerdt i...@tvdw.eu wrote: On 8/16/10 4:38 PM, Thiago Souza wrote: Hi, I know that it's possible to track up to 200 keywords per account, but how many accounts per IP is allowed? Regards, Thiago Souza Hi Thiago, I don't think that Twitter will give you these numbers. Just don't create more connections than necessary, and if you need more track keywords, you should consider asking Twitter for an upgrade of your account. Tom -- Matt Harris Developer Advocate, Twitter http://twitter.com/themattharris
Re: [twitter-dev] Streaming API (filter) and retweets, JSON Output
A retweet will have an embedded retweeted_status object. Example
(first one from spritzer, not necessarily a favorite):
{
coordinates: null,
favorited: false,
created_at: Mon Aug 16 04:03:17 + 2010,
truncated: false,
retweeted_status: {
coordinates: null,
favorited: false,
created_at: Sun Aug 15 22:58:25 + 2010,
truncated: false,
contributors: null,
text: Do u ever feel like u don't belong? When u #LoveYourself
your ppl will find you! Belonging starts within! #TDL,
id: 21266102648,
geo: null,
in_reply_to_user_id: null,
user: {
profile_sidebar_fill_color: ff,
name: Mastin Kipp,
profile_background_tile: true,
profile_sidebar_border_color: 838581,
location: The wise classy place within,
profile_image_url:
http://a0.twimg.com/profile_images/993737856/Picture_45_normal.png;,
created_at: Mon Oct 20 17:22:57 + 2008,
follow_request_sent: null,
profile_link_color: 56a6ec,
favourites_count: 559,
contributors_enabled: false,
url: http://www.TheDailyLove.com;,
utc_offset: -32400,
id: 16870682,
profile_use_background_image: true,
listed_count: 5701,
protected: false,
followers_count: 224270,
profile_text_color: 707070,
lang: en,
verified: false,
profile_background_color: ff,
description: To make each day a fresh start sign up for our
FREE daily email at www.TheDailyLove.com - TDL was founded by 28yr old
@MastinKipp,
time_zone: Alaska,
geo_enabled: false,
notifications: null,
friends_count: 651,
statuses_count: 7696,
profile_background_image_url:
http://a1.twimg.com/profile_background_images/112686910/thedailylove_final_smllrwitter.jpg;,
screen_name: TheDailyLove,
following: null,
show_all_inline_media: false
},
place: null,
in_reply_to_screen_name: null,
source: web,
in_reply_to_status_id: null
},
contributors: null,
text: RT @TheDailyLove: Do u ever feel like u don't belong? When
u #LoveYourself your ppl will find you! Belonging starts within!
#TDL,
id: 21285825400,
geo: null,
in_reply_to_user_id: null,
user: {
profile_sidebar_fill_color: 3b0705,
name: Emillie Immaculate,
profile_background_tile: true,
profile_sidebar_border_color: 00,
location: Empire State Of Mind,
profile_image_url:
http://a2.twimg.com/profile_images/1069286198/ME_MYSELF_AND_I_normal.jpg;,
created_at: Mon Jun 15 07:41:33 + 2009,
follow_request_sent: null,
profile_link_color: b8100d,
favourites_count: 19,
contributors_enabled: false,
url: http://www.facebook.com/EmilyImmaculate;,
utc_offset: -18000,
id: 47286045,
profile_use_background_image: true,
listed_count: 5,
protected: false,
followers_count: 1225,
profile_text_color: 08,
lang: en,
verified: false,
profile_background_color: 663737,
description: Mind of a Billionaire,, Young entrepreneur in the making.,
time_zone: Eastern Time (US Canada),
geo_enabled: false,
notifications: null,
friends_count: 62,
statuses_count: 3657,
profile_background_image_url:
http://a1.twimg.com/profile_background_images/133632534/wallpaper_marilyn_monroe.jpg;,
screen_name: EmilyImmaculate,
following: null,
show_all_inline_media: false
},
place: null,
in_reply_to_screen_name: null,
source: web,
in_reply_to_status_id: null
}
---Mark
http://twitter.com/mccv
On Fri, Aug 13, 2010 at 9:54 AM, Joe joe.vivona.a...@gmail.com wrote:
If we are listening to the stream api/filter using track and getting
JSON output, I cannot see anywhere that I can determine if something
is a retweet except explicitly checking the text of the tweet.
Am I missing something or should I simultansously be listing to the
retweet stream and try to cross match the 2 (that seems like a waste
of resources).
Any advice?
[twitter-dev] Streaming API (filter) and retweets, JSON Output
If we are listening to the stream api/filter using track and getting JSON output, I cannot see anywhere that I can determine if something is a retweet except explicitly checking the text of the tweet. Am I missing something or should I simultansously be listing to the retweet stream and try to cross match the 2 (that seems like a waste of resources). Any advice?
[twitter-dev] Streaming API and lists
Hello there, Does anyone knows if it's possible to use the streaming api to follow a list instead of a user? I already tried using the list id as the follow parameter with no success... Regards, Thiago Souza
Re: [twitter-dev] Streaming API and lists
If everyone on the list is public, you can fetch the user ids via REST, then use follow. Protected accounts won't show, of course. Also, on User Streams, you cannot specify your follow list. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Thu, Aug 12, 2010 at 8:39 AM, thiago tcostaso...@gmail.com wrote: Hello there, Does anyone knows if it's possible to use the streaming api to follow a list instead of a user? I already tried using the list id as the follow parameter with no success... Regards, Thiago Souza
Re: [twitter-dev] Streaming API and lists
Hi Thiago, The streaming API allows you to follow user IDs and track keywords but not lists directly. Instead you need to follow all the user IDs of the list and then assemble their Tweets on your server to recreate the list. Hope that helps, Matt On Thu, Aug 12, 2010 at 8:39 AM, thiago tcostaso...@gmail.com wrote: Hello there, Does anyone knows if it's possible to use the streaming api to follow a list instead of a user? I already tried using the list id as the follow parameter with no success... Regards, Thiago Souza -- Matt Harris Developer Advocate, Twitter http://twitter.com/themattharris
[twitter-dev] streaming API help (regular API works)
Hi - I hope I am not posting a question that has previously been answered - I tried searching the archives but to no avail. I am trying to get the 'sample' stream API working but am getting 401 Unathorized errors. For debugging purposes, I am using curl for now. The following command fails (401): curl 'http://stream.twitter.com/1/statuses/sample.json? delimited=length' -H 'Authorization: OAuth realm=Twitter API, oauth_nonce=24599946, oauth_timestamp=1281319798, oauth_consumer_key=, oauth_signature_method=HMAC-SHA1, oauth_version=1.0, oauth_token=175905996- JkrGAl8ZXCgIjeZl3o7fMCD8HbyfVeDbkP9Y13mX, oauth_signature=i %2BVzWX23sp5t8%2Fz0swJl%2FDHloOo%3D' However, I believe that my OAuth stuff is (hopefully) correct because the following command works, where I have reused the exact same OAuth header (all I changed was the URL): curl 'http://api.twitter.com/1/statuses/user_timeline.json' -H 'Authorization: OAuth realm=Twitter API, oauth_nonce=24599946, oauth_timestamp=1281319798, oauth_consumer_key=, oauth_signature_method=HMAC-SHA1, oauth_version=1.0, oauth_token=175905996-JkrGAl8ZXCgIjeZl3o7fMCD8HbyfVeDbkP9Y13mX, oauth_signature=i%2BVzWX23sp5t8%2Fz0swJl%2FDHloOo%3D' So what does this mean? Are the authentication requirements at all different for these two API calls? In case its relevant, note that I am using my account's single access token to create these OAuth signatures as opposed to a real customer key/secret pair. Any suggestions on what else I can do to try and debug this? Many thanks! - Ian
[twitter-dev] Streaming API for lists
Hey there! I wanted to know if Twitter will integrate lists into the streaming API. Thx.
Re: [twitter-dev] Streaming API for lists
We were planning to do lists, but we postponed the feature to get the bulk of User Streams to market sooner and to also get User Streams out to more users. We'll consider lists as an add-on later. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Fri, Jul 30, 2010 at 4:13 AM, sexyprout abom...@gmail.com wrote: Hey there! I wanted to know if Twitter will integrate lists into the streaming API. Thx.
[twitter-dev] Streaming API time drifting problem and possible solutions
Hi everyone, I have a program calling the statuses/sample method of a garden hose of the Streaming API, and I am experiencing the following problem: the timestamps of the tweets that I downloaded constantly drift behind real-time, the time drift keeps increasing until it reaches around 25 minutes, and then I get a timeout from the request, sleep for 5 seconds and reset the connection. The time drift is also reset to 0 when the connection is reset. One solution for this I have now is to proactively reset the connection more frequently, e.g., if I reconnect every 1 minute, the time drift I get will be at most 1 minute. But I am not sure whether this is allow by the API. So could anyone tell me if you have the same problem as mine or I am using the API in the wrong way. And is it OK to reset connection every minute? I am using Tweepy (http://github.com/joshthecoder/tweepy) as the library for accessing the Streaming API. Thanks a lot! -Larry
Re: [twitter-dev] Streaming API time drifting problem and possible solutions
Absolutely do not reset the connection and reconnect. Connections should be long-lived on the Streaming API. This is almost certainly a problem with the read throughput of your client, or, less likely, with bandwidth from your system. Run curl(1) from the same system and grep for the date field. It will almost certainly not fall behind. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Thu, Jul 8, 2010 at 8:31 AM, Larry Zhang yuelizh...@gmail.com wrote: Hi everyone, I have a program calling the statuses/sample method of a garden hose of the Streaming API, and I am experiencing the following problem: the timestamps of the tweets that I downloaded constantly drift behind real-time, the time drift keeps increasing until it reaches around 25 minutes, and then I get a timeout from the request, sleep for 5 seconds and reset the connection. The time drift is also reset to 0 when the connection is reset. One solution for this I have now is to proactively reset the connection more frequently, e.g., if I reconnect every 1 minute, the time drift I get will be at most 1 minute. But I am not sure whether this is allow by the API. So could anyone tell me if you have the same problem as mine or I am using the API in the wrong way. And is it OK to reset connection every minute? I am using Tweepy (http://github.com/joshthecoder/tweepy) as the library for accessing the Streaming API. Thanks a lot! -Larry
Re: [twitter-dev] Streaming API time drifting problem and possible solutions
Larry, have you decoupled the processing code from tweepy's StreamListener, for example using a Queue.Queue oder some message queue server? Pascal On Jul 8, 2010, at 17:31 , Larry Zhang wrote: Hi everyone, I have a program calling the statuses/sample method of a garden hose of the Streaming API, and I am experiencing the following problem: the timestamps of the tweets that I downloaded constantly drift behind real-time, the time drift keeps increasing until it reaches around 25 minutes, and then I get a timeout from the request, sleep for 5 seconds and reset the connection. The time drift is also reset to 0 when the connection is reset. One solution for this I have now is to proactively reset the connection more frequently, e.g., if I reconnect every 1 minute, the time drift I get will be at most 1 minute. But I am not sure whether this is allow by the API. So could anyone tell me if you have the same problem as mine or I am using the API in the wrong way. And is it OK to reset connection every minute? I am using Tweepy (http://github.com/joshthecoder/tweepy) as the library for accessing the Streaming API. Thanks a lot! -Larry
Re: [twitter-dev] Streaming API time drifting problem and possible solutions
Larry, moreover, I assume you checked I/O and CPU load. But even if that's not the issue, you should absolutely check if you have simplejson with c extension installed. The python included version is 1.9 which is decidedly slower than the new 2.x branch. You might see json decoding load drop by 50% or more. Pascal On Jul 8, 2010, at 17:31 , Larry Zhang wrote: Hi everyone, I have a program calling the statuses/sample method of a garden hose of the Streaming API, and I am experiencing the following problem: the timestamps of the tweets that I downloaded constantly drift behind real-time, the time drift keeps increasing until it reaches around 25 minutes, and then I get a timeout from the request, sleep for 5 seconds and reset the connection. The time drift is also reset to 0 when the connection is reset. One solution for this I have now is to proactively reset the connection more frequently, e.g., if I reconnect every 1 minute, the time drift I get will be at most 1 minute. But I am not sure whether this is allow by the API. So could anyone tell me if you have the same problem as mine or I am using the API in the wrong way. And is it OK to reset connection every minute? I am using Tweepy (http://github.com/joshthecoder/tweepy) as the library for accessing the Streaming API. Thanks a lot! -Larry
[twitter-dev] Streaming API and Oauth
The Oauth Overview page http://dev.twitter.com/pages/auth_overview has sections for three APIs: REST, Search, and Streaming. The bottom of the page displays a ribbon stating that The @twitterapi team will be shutting of basic authentication for the Twitter API. Does this mean all of the Twitter APIs (REST, Search, and Streaming)? or just the REST API? Most specifically, while I know that the Streaming API end-point now supports OAuth, I do not know if Streaming will require OAuth come August 16th... can someone please clarify. TIA.
Re: [twitter-dev] Streaming API and Oauth
Quoting John Kalucki: We haven't announced our plans for streaming and oAuth, beyond stating that User Streams will only be on oAuth. Right now, basic auth and oAuth both work on streaming, and that won't change when basic for REST turns off. Since there's no set shutdown date yet for basic/streaming, I wouldn't expect it to happen soon. Pascal On Jul 5, 2010, at 20:25 , Zhami wrote: The Oauth Overview page http://dev.twitter.com/pages/auth_overview has sections for three APIs: REST, Search, and Streaming. The bottom of the page displays a ribbon stating that The @twitterapi team will be shutting of basic authentication for the Twitter API. Does this mean all of the Twitter APIs (REST, Search, and Streaming)? or just the REST API? Most specifically, while I know that the Streaming API end-point now supports OAuth, I do not know if Streaming will require OAuth come August 16th... can someone please clarify. TIA.
[twitter-dev] streaming api - track - whitelisting for 10,000 keywords
Hello, Can someone point me to the right location for increasing streaming api, track keywords limit? Thanks Peter
Re: [twitter-dev] streaming api - track - whitelisting for 10,000 keywords
Thanks Taylor and John. On Fri, Jul 2, 2010 at 7:03 AM, Taylor Singletary taylorsinglet...@twitter.com wrote: Hi Peter, First, make sure that you've used this form acquire garden hose access: https://twitter.com/help/request_streaming Second, send an email to a...@twitter.com from the email address associated with the same Twitter account you filled that form out with, requesting Track Restricted streaming access, which allows 10,000 keywords. Be sure and include a description of how you'll be using the data and any other information that can help the team best service your request. Taylor On Fri, Jul 2, 2010 at 12:18 AM, Peter Denton petermden...@gmail.com wrote: Hello, Can someone point me to the right location for increasing streaming api, track keywords limit? Thanks Peter -- Peter Denton Co-Founder, Product Marketing www.mombo.com cell: (206) 427-3866 twitter @Mombo_movies twitter - personal: @petermdenton
[twitter-dev] Streaming API: Why not Curl?
I am just getting started with the streaming API, and I was a bit puzzled by this line in the documentation: While a client can be built around cycling connections, perhaps using curl for transport, the overall reliability will tend to be poor due to operational gotchas. Save curl for debugging and build upon a persistent process that does not require periodic reconnections. I am not at all familiar with the internals of how libcurl works, so maybe I'm missing something quite obvious, but can't curl/libcurl keep a persistent connection? Why does using it require periodic reconnections? In fact, many examples around the web of how to consume the streaming API seem to use libcurl. (I'm using Python so have been looking at this example in particular: http://arstechnica.com/open-source/guides/2010/04/tutorial-use-twitters-new-real-time-stream-api-in-python.ars)
Re: [twitter-dev] Streaming API: Why not Curl?
It's not libcurl, per se, that's the problem, but just using curl(1) from a shell script doesn't necessarily give you the control that you might desire to build a stable client. You may want to set a connect timeout, socket timeout, perform parsing before persisting to disk, use overlapping connections to update predicates without data loss, provide an estimate for the count parameter, deal with events from downstream processes, rotate journal files, or any number of other things that are non-obvious or difficult to do with a single threaded shell script wrapping a curl(1) process. You require periodic reconnections to deal with predicate change on your end, or operational bumps (server restarts after code deploys) on our end. Most of the blogged Streaming API examples I've seen contain no error handling and would cause you to get locked out and your IP address eventually banned. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Wed, Jun 30, 2010 at 2:47 AM, Mark Linsey mjlin...@gmail.com wrote: I am just getting started with the streaming API, and I was a bit puzzled by this line in the documentation: While a client can be built around cycling connections, perhaps using curl for transport, the overall reliability will tend to be poor due to operational gotchas. Save curl for debugging and build upon a persistent process that does not require periodic reconnections. I am not at all familiar with the internals of how libcurl works, so maybe I'm missing something quite obvious, but can't curl/libcurl keep a persistent connection? Why does using it require periodic reconnections? In fact, many examples around the web of how to consume the streaming API seem to use libcurl. (I'm using Python so have been looking at this example in particular: http://arstechnica.com/open-source/guides/2010/04/tutorial-use-twitters-new-real-time-stream-api-in-python.ars )
[twitter-dev] Streaming API track logical AND
This may have been lost in all the hubbub around the Chirp conference, but we added logical AND to streaming track queries in mid-April. The documentation has been updated and moved over to http://dev.twitter.com/pages/streaming_api. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc.
[twitter-dev] Streaming API OAuth
Can I somehow use the OAuth implementation in my client to use Streaming API without prompting for user password too? -- Subscription settings: http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
Re: [twitter-dev] Streaming API OAuth
Hi Jumpa, OAuth isn't supported for the Streaming API yet. We'll let everyone know the appropriate new access methods when they're fully baked. Taylor Singletary Developer Advocate, Twitter http://twitter.com/episod On Mon, Apr 26, 2010 at 3:17 AM, Jumpa giampa.ma...@gmail.com wrote: Can I somehow use the OAuth implementation in my client to use Streaming API without prompting for user password too? -- Subscription settings: http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
[twitter-dev] Streaming API
Dear Sirs, Hello! My name is Xiaowen, I have been playing with Twitter for some time and I am writing to ask a small question regarding Streaming API :) From the documentation I know we can use the command curl -d @locations http://stream.twitter.com/1/statuses/filter.json - uAnyTwitterUser:Password to filter geo-tagged tweets in a bounded box area, we could also use similar method to follow tweets of specific users. These are two parameters which can be implemented by statuses/ filter method. My question is, if I want to incorporate both of these two parameters in one command, how to put them together? I have tried something like curl -d @locations @following http://stream.twitter.com/1/statuses/filter.json -uAnyTwitterUser:Password but it doesn't work :( Later I got response from Twitter support, suggesting that: One Streaming API connection can filter both geo-tagged tweets and tweets from specific users. You may want to combine both of your parameters into one file so that you do not need to reference two in your command (for example @locandfollow instead of @locations @following). But then what is the correct way to put the parameters in one file? I tried simply put them together, like this: locations=-122.75,36.8,-121.75,37.8; follow=...,...,... but still it doesn't work. Many thanks for your advices! -- Subscription settings: http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
Re: [twitter-dev] Streaming API
The curl man page explains how to send multiple parameters. If you just put them in one file, curl runs then all together. You have to separate them with , or have curl do it for you by specifying multiple -d @ pairs: adrift.local:/tmp head foo bar == foo == follow=1234 == bar == track=north adrift.local:/tmp curl -v -x -d @foo -d @bar stream.twitter.com/1/statuses/filter.xml -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Sat, Apr 24, 2010 at 5:27 AM, epomqo wenzi0...@gmail.com wrote: Dear Sirs, Hello! My name is Xiaowen, I have been playing with Twitter for some time and I am writing to ask a small question regarding Streaming API :) From the documentation I know we can use the command curl -d @locations http://stream.twitter.com/1/statuses/filter.json - uAnyTwitterUser:Password to filter geo-tagged tweets in a bounded box area, we could also use similar method to follow tweets of specific users. These are two parameters which can be implemented by statuses/ filter method. My question is, if I want to incorporate both of these two parameters in one command, how to put them together? I have tried something like curl -d @locations @following http://stream.twitter.com/1/statuses/filter.json -uAnyTwitterUser:Password but it doesn't work :( Later I got response from Twitter support, suggesting that: One Streaming API connection can filter both geo-tagged tweets and tweets from specific users. You may want to combine both of your parameters into one file so that you do not need to reference two in your command (for example @locandfollow instead of @locations @following). But then what is the correct way to put the parameters in one file? I tried simply put them together, like this: locations=-122.75,36.8,-121.75,37.8; follow=...,...,... but still it doesn't work. Many thanks for your advices! -- Subscription settings: http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
[twitter-dev] Streaming API OAuth explanation?
One thing I meant to find out @chirp last week--what will oauth look like for the Streaming API? I'm having a hard time visualizing how that will work. Thanks, Jonathon Hill @compwright Company52 http://company52.com -- Subscription settings: http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
Re: [twitter-dev] Streaming API OAuth explanation?
Hi Jonathon, For Streaming API access that isn't from the perspective of a user's account, you would use two-legged OAuth to establish authentication instead of basic auth. A two-legged OAuth request is very similar to other OAuth requests: you have a specific resource you are trying to access, you have some parameters you want to pass to that resource, and you have an OAuth consumer key and OAuth consumer secret. Which is unlike three-legged OAuth where you also have oauth_tokens representing either a user/access_token or a request token in addition to the rest. But the rules remain the same. You take all the OAuth parameters and the parameters you are sending to the resource, organize them, build a signature base string, then sign that with your consumer secret and send the request on to Twitter properly signed. The only difference is that there is no oauth_token and oauth_token_secret getting involved in the mix. This is essentially what a two-legged request to the streaming API would look like: Signature Base String GEThttp%3A%2F%2Fstream.twitter.com %2F1%2Fstatuses%2Fsample.jsonoauth_consumer_key%3Dri8JxYK2zzwSV5xIUfNNvQ%26oauth_nonce%3DSJJqJPdaZrYuIogToapS6ueJRyWB4Rs2ox4HEbu4nW8%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1271783743%26oauth_version%3D1.0 Signature Xi5jfuw2XqtU5KpNX9ZCtTptJS0= Authorization Header OAuth oauth_nonce=SJJqJPdaZrYuIogToapS6ueJRyWB4Rs2ox4HEbu4nW8, oauth_signature_method=HMAC-SHA1, oauth_timestamp=1271783743, oauth_consumer_key=ri8JxYK2zzwSV5xIUfNNvQ, oauth_signature=Xi5jfuw2XqtU5KpNX9ZCtTptJS0%3D, oauth_version=1.0 Taylor Singletary Developer Advocate, Twitter http://twitter.com/episod On Tue, Apr 20, 2010 at 10:05 AM, Jonathon Hill jhill9...@gmail.com wrote: One thing I meant to find out @chirp last week--what will oauth look like for the Streaming API? I'm having a hard time visualizing how that will work. Thanks, Jonathon Hill @compwright Company52 http://company52.com -- Subscription settings: http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
