[twitter-dev] Re: all replies by friends
Umm, how do we discover new people?
[twitter-dev] Re: all replies by friends
@chad You just wanted to link to my typo :) @all: This is not an API issue so the discussion is somewhat off topic here. This option's fate was a Twitter.com product decision and was axed by the product folks, not the API team. @ev has said that he hears the small but vocal minority of users that relied on this option for discovery. I'm sure we as a company will talk about the impact, our options, and how to move forward tomorrow. Secondly, where is the entrepreneurial spirit to turn this into an opportunity? For the users of this option, think about the ways having @replies to strangers in your timeline helped you find people. How can you use the API make similar connections? The data is still there, present it in a compelling way. I'd use a service that sent an occasional email digest like: The following 3 users were mentioned at least X number of times by the people you follow, check 'em out. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Tue, May 12, 2009 at 11:51 PM, Sarah Isaacson sarahisaac...@gmail.comwrote: I'm wondering what the justification is for removing the option for filtering replies? Surely with the way that you guys have included mentions, people will just circumvent this by not placing the @username at the start of the tweet? That leaves those who previously used the Show me @replies only for people I'm following with pretty much no option for any kind of filtering as people get around this. I think we'll particularly see this with the power users. Maybe I missed something? Regards, Sarah -- Sarah Isaacson Flickr : http://www.flickr.com/photos/out/ Twitter: http://www.twitter.com/velvetsarah Weblog: http://www.velvet.id.au Not a work of art or a showcase of my design skills, just a place for me to put my thoughts and get on my soapbox sometimes!
[twitter-dev] Re: all replies by friends
Yeah, lots of protest out there, just watch the 'Links to this post:' below the Twitter blog post. There also is a hashtag to protest against this decision: #fixreplies . It is already in the 'Trending Topics' sidebar. Let's see how this evolves... Jack On May 13, 7:19 am, Chad Etzel jazzyc...@gmail.com wrote: In regard to Doug's recent tweet regarding this change:http://twitter.com/dougw/status/1781551902 As much as I hate for people to be upset with Twitter, I am partially happy to see I'm not the only one upset by this behavior change... I could *swear* that show all @replies was the default at some point (maybe a long time ago when I joined). Now my friend's timeline is just a closed loop, and for (at least) the power users and people that understood the setting, the timeline has gotten a lot less interesting to watch. I know I'm screaming into the wind... I'm just glad I'm not alone. -Chad
[twitter-dev] Re: Getting id for last status
I just tried posting from another server and it worked just fine! *grrr* So I guess that means there's something going on with my server and the cURL install, maybe? Can someone try pointing me in the right direction to see if I can get this corrected? (No sense in having a development server if it's not working properly.) :/ Thanks! On May 12, 7:40 pm, Peter Denton petermden...@gmail.com wrote: was that the problem? I ran a test with that script under a few different accounts and it was working fine. Did you try posting from another server?
[twitter-dev] in lieu of @replies, need better filters for applications
Hi, The recent @replies issue is a big problem for applications. I think either the user needs to be given control, or API access needs to be more flexible. There are a multitude of apps that act like an echo of a set of users. That is a really hard problem to solve if you aren't following the whole set. In place of getting the friend stream, you would need to poll each individually. With the API access limits, that inherently prevents an app to scale beyond a few thousand users. For example, in Tipjoy, we look for payment commands. We do this in 3 ways: - Poll the search API across all Twitter users for keyword like 'tip' 'give' ,etc. Lots of users tweet the letter 'p' though - Poll that same search but for an individual user in Tipjoy. This takes a long time to cycle through all our users - Poll @tipjoy's friends_timeline looking for payments The third is by far the best option. But the change hurts the effort as any payment from a user formatted like: @ev p $2 wouldn't get caught. Another app I'm building is @tatatweet**. It takes a set of twitter users, and feeds their tweets to a single account. For example, @TeamTipjoy is a test account that feeds tweets from Tipjoy employees. Rather than poll each account individually, the app will make the target account follow the sources. But every tweet starting with an @reply will get filtered out, for no good reason as far as the app is concerned. I think one potential solution (in case the product people don't act) is to give apps the ability to poll a set of users, with raw tweets and all @replies. The model of having an app follow a user is a bit klunky anyway. Enable api endpoints to create, edit and share filters, which are sets of users and perhaps a regex in place to filter the users. This would also prevent the annoying following limits from being a problem. If a user account is protected, mandate that your service authenticate as the user to add them to the filter. I'd pay for this. Generally, changes put in place for a better user experience shouldn't hurt applications. Users and apps are different. Best, Ivan http://tipjoy.com ** @tatatweet is an open source premium twitter app built using Tipjoy's API http://bit.ly/git_tatatweet Please copy the code
[twitter-dev] Status ID closing in on maximum unsigned integer
Hi all, Just a quick heads up. Our status id is fast approaching the maximum unsigned integer value. Based on all of the feedback on scaling I've received from this list I'm assuming everyone will be fine. One thing to check is your table definitions … no amount of interpreted language will save you from that. I think the tipping point will be this afternoon sometime but with the planned maintenance it might end up being later in the day. Thanks; — Matt Sanford / @mzsanford
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
Make that *signed* integer. Writing too many emails at once, sorry. On May 13, 7:38 am, Matt Sanford m...@twitter.com wrote: Hi all, Just a quick heads up. Our status id is fast approaching the maximum unsigned integer value. Based on all of the feedback on scaling I've received from this list I'm assuming everyone will be fine. One thing to check is your table definitions … no amount of interpreted language will save you from that. I think the tipping point will be this afternoon sometime but with the planned maintenance it might end up being later in the day. Thanks; — Matt Sanford / @mzsanford
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
I'm assuming this is just a friendly warning to developers... and that the status IDs will continue to monotonically increase past (2^31 - 1), i.e. not rollover? -Chad On Wed, May 13, 2009 at 10:40 AM, Matt Sanford m...@twitter.com wrote: Make that *signed* integer. Writing too many emails at once, sorry. On May 13, 7:38 am, Matt Sanford m...@twitter.com wrote: Hi all, Just a quick heads up. Our status id is fast approaching the maximum unsigned integer value. Based on all of the feedback on scaling I've received from this list I'm assuming everyone will be fine. One thing to check is your table definitions … no amount of interpreted language will save you from that. I think the tipping point will be this afternoon sometime but with the planned maintenance it might end up being later in the day. Thanks; — Matt Sanford / @mzsanford
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
When you say integer what are you referring to? I'm _assuming_ 32-bit integers, not 64-bit integers, because ILP-32 is so commonplace. When you're about to breach the 2^64 in your ID space, please give us a heads-up around 2^63 so we can upgrade our systems. :) On 5/13/09 10:40 AM, Matt Sanford wrote: Make that *signed* integer. Writing too many emails at once, sorry. On May 13, 7:38 am, Matt Sanfordm...@twitter.com wrote: Hi all, Just a quick heads up. Our status id is fast approaching the maximum unsigned integer value. Based on all of the feedback on scaling I've received from this list I'm assuming everyone will be fine. One thing to check is your table definitions … no amount of interpreted language will save you from that. I think the tipping point will be this afternoon sometime but with the planned maintenance it might end up being later in the day. Thanks; — Matt Sanford / @mzsanford -- Dossy Shiobara | do...@panoptic.com | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70)
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
On Wed, May 13, 2009 at 7:40 AM, Matt Sanford m...@twitter.com wrote: Make that *signed* integer. Writing too many emails at once, sorry. That would be the MySQL (and any 32-bit OS) maximum signed integer, or 2,147,483,647, I'm assuming. So if we're using an unsigned INT in our tables, we're fine for a while (months?) - status IDs will always be positive, right? And BIGINT, especially unsigned, is safe for a long, long time? Just curious... does this mean Twitter really is closing in on 2 billion tweets, or were some IDs skipped? Is there a prize for whoever posts tweet number 2 billion? ;-) Nick
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
Hi Nick, As mentioned to Dossy (probably still wending its way thought Google Groups) I did mean 32-bit signed integers. Unsigned ints in MySQL will be fine but the rails default is int(10) if I recall which would be a problem. As mentioned to Chad (Google Groups seems to be slow delivering my messages :/) status IDs will continue to increment as positive numbers. BIGINT and unsigned number should be fine for the time being, yes. You know we don't comment on the number of users and tweets … but you almost tricked me. Sneaky, sneaky. Thanks; – Matt Sanford / @mzsanford Twitter Dev On May 13, 2009, at 8:13 AM, Nick Arnett wrote: On Wed, May 13, 2009 at 7:40 AM, Matt Sanford m...@twitter.com wrote: Make that *signed* integer. Writing too many emails at once, sorry. That would be the MySQL (and any 32-bit OS) maximum signed integer, or 2,147,483,647, I'm assuming. So if we're using an unsigned INT in our tables, we're fine for a while (months?) - status IDs will always be positive, right? And BIGINT, especially unsigned, is safe for a long, long time? Just curious... does this mean Twitter really is closing in on 2 billion tweets, or were some IDs skipped? Is there a prize for whoever posts tweet number 2 billion? ;-) Nick
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
On Wed, May 13, 2009 at 11:08 AM, Matt Sanford m...@twitter.com wrote: When we breach 2^64 I'm not telling anyone … it seems to create more work for me when I send out friendly warnings :) Well, if you *did* send out a warning at 2^64, it would mean either: a) you have found the fountain of youth b) you're a zombie c) you wrote the most awesome cron job ever d) you have a Delorean -Chad
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
Have you not heard, we're tweeting from space these days. Birth rate changes are now considered optional. No Humans Required™ — Matt On May 13, 2009, at 8:26 AM, Andrew Badera wrote: or e) Twitter becomes even more massively popular while the world also undergoes a birth rate revoluion. Thanks- - Andy Badera - and...@badera.us - Google me: http://www.google.com/search?q=andrew+badera - This email is: [ ] bloggable [x] ask first [ ] private On Wed, May 13, 2009 at 11:21 AM, Chad Etzel jazzyc...@gmail.com wrote: On Wed, May 13, 2009 at 11:08 AM, Matt Sanford m...@twitter.com wrote: When we breach 2^64 I'm not telling anyone … it seems to create more work for me when I send out friendly warnings :) Well, if you *did* send out a warning at 2^64, it would mean either: a) you have found the fountain of youth b) you're a zombie c) you wrote the most awesome cron job ever d) you have a Delorean -Chad
[twitter-dev] Re: Source request - no wait, listen...
So does this mean soon we will soon be able to use source=space and have it look like we built our own rocket? On Tue, May 12, 2009 at 23:50, Arik Fraimovich arik...@gmail.com wrote: You guys rock ! :) On May 13, 2:10 am, Doug Williams d...@twitter.com wrote: Wait, no really... I have a NASA contact that worked with us to make tweets flow from space. I'll ping him and see if we can get a source parameter patched into the update logic. Yay geeks! Thanks, Doug -- Doug Williams Twitter Platform Supporthttp://twitter.com/dougw On Tue, May 12, 2009 at 3:52 PM, Chad Etzel jazzyc...@gmail.com wrote: I know this may be goofy, but I think it would be awesome... You (twitter) should make a rule that when @Astro_Mike posts, it should say: xx minutes ago from SPACE! zomg that would rock. -Chad -- Abraham Williams | http://the.hackerconundrum.com Hacker | http://abrah.am | http://twitter.com/abraham Web608 | Community Evangelist | http://web608.org This email is: [ ] blogable [x] ask first [ ] private.
[twitter-dev] API FAQ can be reached in office but cann't at home
http://apiwiki.twitter.com/FAQ can be reached in office,where IE PROXY is needed .but it cann't be reached at home(direct internet coonnection),where i cann't use the proxy.Is there any access restriction to access the API site . PS: the http://twitter.com/ can be reached both office and home
[twitter-dev] Re: [OAuth] GET parameters
On May 12, 10:40 pm, Matt Sanford m...@twitter.com wrote: Hi there, The parameters should be required for the OAuth signature. Can you please provide the request and response headers for an example? Hi Matt, Forget about that nonsense, the fault was on my side. Now works as expected. When I've tried to collect headers and all other debug info, I've noticed that GET parameters was just not appended to the URL. So actual URL was http://twitter.com/statuses/friends_timeline.xml, not surprisingly that you did not include my GET parameters into signature calculation :) -- Best regards, Dmitriy V'jukov
[twitter-dev] Oauth SIgn-in
Has anyone noticed that the Oauth Sign in for users is again constantly asking to allow / deny even though the account has already granted access? I know this was the case before, but then it started detecting. And now it's back to this. Anyone else experiencing this? Jason.
[twitter-dev] twitter on mobile
Hi all, I had come here few days back also. now again after preparing my design. I want to write twitter application which will contain my own feature also, for MOBILE. for the time being, just forget which mobile I am going to use. Instead of J2ME, I will use similar framework. From here only I had got info about list of API's to develop twitter application. But I need API's for J2ME in fact. I have not gone in depth of these technical terms till now as far as mobile and twitter api's are concern because I was working with GUI and data flow of my own concept (which dont need any space on web server now because i have changed it a bit). So, would you please help me from where I will get those API's which will run on J2ME platform? please guide me properly considering I have basic idea of J2ME and developing twitter application as well. Thanks and Regards, Omi learner.
[twitter-dev] Few questions about rate limiting
Hi, I'm using Sign in with twitter functionality - since account/ verify_credentials request is rate limited, how can I otherwise get info about authenticated user? How exactly requests are limited? I.e. I got 100 users logged in in my web app and they all display page with their friends/followers list. Will these requests be counted for these users' accounts or IP of my server? What's the limit for search requests? We're using jsonp for fetching search results, so at least in this case the limit is for client's IP. Is it ok to make search request i.e. every 15s? Thanks in advance
[twitter-dev] Invalid JSON
Consider this status: http://twitter.com/primerano/status/1784283306 The JSON for this, as found at http://twitter.com/statuses/show/1784283306.json, is below (prettified by JSONLint - which, by the way, calls it valid). Both Crack and ActiveSupport::JSON refuse to parse it, returning the error Invalid JSON string. Can anyone see why? My first thought was the emoticon in the status text, :-\, but even with that removed the parsers fail. Console: json_string = EOS { text: loving that I turned my 22 inch monitor sideways. great for reading pdfs but the sta nd doesn't support this configuration., in_reply_to_status_id: null, user: { description: web geek, accomplishment junkie, parent and System Architect at AOL, time_zone: Eastern Time (US Canada), utc_offset: -18000, profile_sidebar_fill_color: e0ff92, following: 0, favourites_count: 0, profile_sidebar_border_color: 87bc44, profile_background_tile: true, followers_count: 146, url: http:\/\/tonycode.com\/, name: Tony Primerano, protected: false, friends_count: 112, profile_background_color: 9ae4e8, created_at: Fri Jun 22 20:36:59 + 2007, notifications: false, statuses_count: 1973, profile_background_image_url: http:\/\/s3.amazonaws.com\/ twitter_production\/profile_bac kground_images\/1259422\/pattern2.JPG, profile_text_color: 00, profile_image_url: http:\/\/s3.amazonaws.com\/ twitter_production\/profile_images\/745684 29\/fire-1_normal.jpg, location: DC Area, id: 7023582, profile_link_color: ff, screen_name: primerano }, in_reply_to_user_id: null, favorited: false, in_reply_to_screen_name: null, created_at: Wed May 13 13:33:03 + 2009, id: 1784283306, truncated: false, source: a href=\http:\/\/www.tweetdeck.com\/\TweetDeck\/ a } EOS Crack::JSON.parse(json_string) Crack::ParseError: Invalid JSON string from /usr/lib/ruby/gems/1.8/gems/crack-0.1.1/lib/crack/json.rb: 14:in `parse' from (irb):40 ActiveSupport::JSON.decode(json_string) ActiveSupport::JSON::ParseError: Invalid JSON string from /usr/lib/ruby/gems/1.8/gems/activesupport-2.3.2/lib/ active_support/json/decoding.rb:14:in `decode' from (irb):41
[twitter-dev] Re: Streaming API's XML format
On Wed, May 13, 2009 at 9:38 AM, bnagykek...@gmail.com bnagykek...@gmail.com wrote: Hi, Let me give an other point of view. By definition an XML must contain one and only one root document. ...Only from the point of view of the parser. If the parser is reset upon each new status, then it wouldn't really care. -Chad
[twitter-dev] Re: Streaming API's XML format
I'm trying the delimited parameter but I don't see any difference in the feed. I find it's description in the docs a little confusing: Indicates that statuses should be delimited in the stream. Statuses are represented by a length, in bytes, a newline, and the status text that is exactly length bytes. Note that keep-alive newlines may be inserted before each length. Values: length in bytes (integer) Makes it sound like I have to use http://stream.twitter.com/spritzer.xml?delimiter=length But what is length in this case? No matter what I use, the feed always looks the same. In terms of efficiency, it would seem to me that letting the parser read directly from one never ending XML document using a callback every time it parses a status is better than having to wrap the parser inside another one that splits the feed into documents then parsing each document individually. Ianiv Schweber ia...@blogaholics.ca Skype: ianivs Public Key: http://www.blogaholics.ca/ianivpubkey.asc On Wed, May 13, 2009 at 6:24 AM, John Kalucki jkalu...@gmail.com wrote: Ianiv, I'm glad you are giving the XML feed a close examination. Nearly all consumers so far have been on the JSON feed. The statuses begin their life upstream from Hosebird as these fully- formed documents. Hosebird doesn't alter their contents, as it's conceptualized as middleware and not a data-changing application. Both the XML and JSON feed are pretty symmetrical. I've found it easiest to parse the markup by using the delimited parameter to read an entire status at once, and then pass that on to the XML parser. If it is considerably more efficient to do something else, perhaps we can make a change. -John On May 12, 9:38 pm, Ianiv Schweber ian...@gmail.com wrote: Currently the spritzer stream looks like: ?xml version=1.0 encoding=UTF-8? status /status \n ?xml version=1.0 encoding=UTF-8? status /status \n ... I'm wondering why it contains a stream of XML documents instead of just one never ending document with the same format as the public timeline: ?xml version=1.0 encoding=UTF-8? statuses status.../status status.../status status.../status ... To me it seems like it would be a lot easier just passing a stream like this to a parser. Instead, with the current stream of documents one has to look out for new prologs, split the stream at that point, parse that doc, reset the parser and continue. Not an insurmountable problem, but it seems like a lot of extra work that shouldn't really be needed. Just wondering what everyone else thinks. Ianiv Schweber
[twitter-dev] Re: Invalid JSON
Hi Bill, My first guess would be the URL fields. We escape the / in the URL which is optional [1]. I don't know either of the libraries showing the problem very well but you might want to try without the URL fields and see if that works to confirm. Thanks; – Matt Sanford / @mzsanford Twitter Dev [1] - http://tech.groups.yahoo.com/group/json/message/249 On May 13, 2009, at 8:52 AM, Bill Kocik wrote: Consider this status: http://twitter.com/primerano/status/1784283306 The JSON for this, as found at http://twitter.com/statuses/show/1784283306.json , is below (prettified by JSONLint - which, by the way, calls it valid). Both Crack and ActiveSupport::JSON refuse to parse it, returning the error Invalid JSON string. Can anyone see why? My first thought was the emoticon in the status text, :-\, but even with that removed the parsers fail. Console: json_string = EOS { text: loving that I turned my 22 inch monitor sideways. great for reading pdfs but the sta nd doesn't support this configuration., in_reply_to_status_id: null, user: { description: web geek, accomplishment junkie, parent and System Architect at AOL, time_zone: Eastern Time (US Canada), utc_offset: -18000, profile_sidebar_fill_color: e0ff92, following: 0, favourites_count: 0, profile_sidebar_border_color: 87bc44, profile_background_tile: true, followers_count: 146, url: http:\/\/tonycode.com\/, name: Tony Primerano, protected: false, friends_count: 112, profile_background_color: 9ae4e8, created_at: Fri Jun 22 20:36:59 + 2007, notifications: false, statuses_count: 1973, profile_background_image_url: http:\/\/s3.amazonaws.com\/ twitter_production\/profile_bac kground_images\/1259422\/pattern2.JPG, profile_text_color: 00, profile_image_url: http:\/\/s3.amazonaws.com\/ twitter_production\/profile_images\/745684 29\/fire-1_normal.jpg, location: DC Area, id: 7023582, profile_link_color: ff, screen_name: primerano }, in_reply_to_user_id: null, favorited: false, in_reply_to_screen_name: null, created_at: Wed May 13 13:33:03 + 2009, id: 1784283306, truncated: false, source: a href=\http:\/\/www.tweetdeck.com\/\TweetDeck\/ a } EOS Crack::JSON.parse(json_string) Crack::ParseError: Invalid JSON string from /usr/lib/ruby/gems/1.8/gems/crack-0.1.1/lib/crack/json.rb: 14:in `parse' from (irb):40 ActiveSupport::JSON.decode(json_string) ActiveSupport::JSON::ParseError: Invalid JSON string from /usr/lib/ruby/gems/1.8/gems/activesupport-2.3.2/lib/ active_support/json/decoding.rb:14:in `decode' from (irb):41
[twitter-dev] Re: Oauth SIgn-in
Hi there, There was a fix yesterday that fixed an error where the previous logged in users information was being returned when using force_login. When verifying that fix in production we came across the same issue you're reporting and a fix is waiting to go out. I've also managed to create a reproducible test case so we can keep this from happening again in the future. With the maintence window today it might be tomorrow before the fix goes out. I'll update the change log when it does. Thanks; – Matt Sanford / @mzsanford Twitter Dev On May 13, 2009, at 7:11 AM, Jason Wong wrote: Has anyone noticed that the Oauth Sign in for users is again constantly asking to allow / deny even though the account has already granted access? I know this was the case before, but then it started detecting. And now it's back to this. Anyone else experiencing this? Jason.
[twitter-dev] Re: Streaming API's XML format
The delimited=length option causes the length, in bytes, of the next status to be placed in the stream. You should see: ... /status 1704 ?xml version=1.0 encoding=UTF-8? status ... Whereas, without the parameter, you see: ... /status ?xml version=1.0 encoding=UTF-8? status ... The length allows you process the stream deterministically, with no error-prone pre-parsing and with the minimum number of reads. This also ensures that you call your parser exactly once per status. There's some pseudo code in the API docs that shows how this might work: while (true) { do { lengthBytes = readline() } while (lengthBytes.length 1) parseMarkup(read(Integer(lengthBytes).parseInt())) } -John On May 13, 9:00 am, Ianiv Schweber ian...@gmail.com wrote: I'm trying the delimited parameter but I don't see any difference in the feed. I find it's description in the docs a little confusing: Indicates that statuses should be delimited in the stream. Statuses are represented by a length, in bytes, a newline, and the status text that is exactly length bytes. Note that keep-alive newlines may be inserted before each length. Values: length in bytes (integer) Makes it sound like I have to use http://stream.twitter.com/spritzer.xml?delimiter=length But what is length in this case? No matter what I use, the feed always looks the same. In terms of efficiency, it would seem to me that letting the parser read directly from one never ending XML document using a callback every time it parses a status is better than having to wrap the parser inside another one that splits the feed into documents then parsing each document individually. Ianiv Schweber ia...@blogaholics.ca Skype: ianivs Public Key:http://www.blogaholics.ca/ianivpubkey.asc On Wed, May 13, 2009 at 6:24 AM, John Kalucki jkalu...@gmail.com wrote: Ianiv, I'm glad you are giving the XML feed a close examination. Nearly all consumers so far have been on the JSON feed. The statuses begin their life upstream from Hosebird as these fully- formed documents. Hosebird doesn't alter their contents, as it's conceptualized as middleware and not a data-changing application. Both the XML and JSON feed are pretty symmetrical. I've found it easiest to parse the markup by using the delimited parameter to read an entire status at once, and then pass that on to the XML parser. If it is considerably more efficient to do something else, perhaps we can make a change. -John On May 12, 9:38 pm, Ianiv Schweber ian...@gmail.com wrote: Currently the spritzer stream looks like: ?xml version=1.0 encoding=UTF-8? status /status \n ?xml version=1.0 encoding=UTF-8? status /status \n ... I'm wondering why it contains a stream of XML documents instead of just one never ending document with the same format as the public timeline: ?xml version=1.0 encoding=UTF-8? statuses status.../status status.../status status.../status ... To me it seems like it would be a lot easier just passing a stream like this to a parser. Instead, with the current stream of documents one has to look out for new prologs, split the stream at that point, parse that doc, reset the parser and continue. Not an insurmountable problem, but it seems like a lot of extra work that shouldn't really be needed. Just wondering what everyone else thinks. Ianiv Schweber
[twitter-dev] Re: Streaming API's XML format
Ok, so delimited=length does work. The docs say the value for delimited must be a number, which is why I was confused. Ianiv Schweber ia...@blogaholics.ca Skype: ianivs Public Key: http://www.blogaholics.ca/ianivpubkey.asc On Wed, May 13, 2009 at 9:31 AM, John Kalucki jkalu...@gmail.com wrote: The delimited=length option causes the length, in bytes, of the next status to be placed in the stream. You should see: ... /status 1704 ?xml version=1.0 encoding=UTF-8? status ... Whereas, without the parameter, you see: ... /status ?xml version=1.0 encoding=UTF-8? status ... The length allows you process the stream deterministically, with no error-prone pre-parsing and with the minimum number of reads. This also ensures that you call your parser exactly once per status. There's some pseudo code in the API docs that shows how this might work: while (true) { do { lengthBytes = readline() } while (lengthBytes.length 1) parseMarkup(read(Integer(lengthBytes).parseInt())) } -John On May 13, 9:00 am, Ianiv Schweber ian...@gmail.com wrote: I'm trying the delimited parameter but I don't see any difference in the feed. I find it's description in the docs a little confusing: Indicates that statuses should be delimited in the stream. Statuses are represented by a length, in bytes, a newline, and the status text that is exactly length bytes. Note that keep-alive newlines may be inserted before each length. Values: length in bytes (integer) Makes it sound like I have to use http://stream.twitter.com/spritzer.xml?delimiter=length But what is length in this case? No matter what I use, the feed always looks the same. In terms of efficiency, it would seem to me that letting the parser read directly from one never ending XML document using a callback every time it parses a status is better than having to wrap the parser inside another one that splits the feed into documents then parsing each document individually. Ianiv Schweber ia...@blogaholics.ca Skype: ianivs Public Key:http://www.blogaholics.ca/ianivpubkey.asc On Wed, May 13, 2009 at 6:24 AM, John Kalucki jkalu...@gmail.com wrote: Ianiv, I'm glad you are giving the XML feed a close examination. Nearly all consumers so far have been on the JSON feed. The statuses begin their life upstream from Hosebird as these fully- formed documents. Hosebird doesn't alter their contents, as it's conceptualized as middleware and not a data-changing application. Both the XML and JSON feed are pretty symmetrical. I've found it easiest to parse the markup by using the delimited parameter to read an entire status at once, and then pass that on to the XML parser. If it is considerably more efficient to do something else, perhaps we can make a change. -John On May 12, 9:38 pm, Ianiv Schweber ian...@gmail.com wrote: Currently the spritzer stream looks like: ?xml version=1.0 encoding=UTF-8? status /status \n ?xml version=1.0 encoding=UTF-8? status /status \n ... I'm wondering why it contains a stream of XML documents instead of just one never ending document with the same format as the public timeline: ?xml version=1.0 encoding=UTF-8? statuses status.../status status.../status status.../status ... To me it seems like it would be a lot easier just passing a stream like this to a parser. Instead, with the current stream of documents one has to look out for new prologs, split the stream at that point, parse that doc, reset the parser and continue. Not an insurmountable problem, but it seems like a lot of extra work that shouldn't really be needed. Just wondering what everyone else thinks. Ianiv Schweber
[twitter-dev] Re: Streaming API's XML format
Another benefit for large-scale feeds: You can get some parallelism fairly easily. You can pipeline your parsing step by having one thread read fully-formed statuses and pass the markup parsing and subsequent processing off to a thread pool. This may not seem line an issue yet, but the /firehose will soon, in all likelihood, swamp a single core. -John On May 13, 9:00 am, Ianiv Schweber ian...@gmail.com wrote: I'm trying the delimited parameter but I don't see any difference in the feed. I find it's description in the docs a little confusing: Indicates that statuses should be delimited in the stream. Statuses are represented by a length, in bytes, a newline, and the status text that is exactly length bytes. Note that keep-alive newlines may be inserted before each length. Values: length in bytes (integer) Makes it sound like I have to use http://stream.twitter.com/spritzer.xml?delimiter=length But what is length in this case? No matter what I use, the feed always looks the same. In terms of efficiency, it would seem to me that letting the parser read directly from one never ending XML document using a callback every time it parses a status is better than having to wrap the parser inside another one that splits the feed into documents then parsing each document individually. Ianiv Schweber ia...@blogaholics.ca Skype: ianivs Public Key:http://www.blogaholics.ca/ianivpubkey.asc On Wed, May 13, 2009 at 6:24 AM, John Kalucki jkalu...@gmail.com wrote: Ianiv, I'm glad you are giving the XML feed a close examination. Nearly all consumers so far have been on the JSON feed. The statuses begin their life upstream from Hosebird as these fully- formed documents. Hosebird doesn't alter their contents, as it's conceptualized as middleware and not a data-changing application. Both the XML and JSON feed are pretty symmetrical. I've found it easiest to parse the markup by using the delimited parameter to read an entire status at once, and then pass that on to the XML parser. If it is considerably more efficient to do something else, perhaps we can make a change. -John On May 12, 9:38 pm, Ianiv Schweber ian...@gmail.com wrote: Currently the spritzer stream looks like: ?xml version=1.0 encoding=UTF-8? status /status \n ?xml version=1.0 encoding=UTF-8? status /status \n ... I'm wondering why it contains a stream of XML documents instead of just one never ending document with the same format as the public timeline: ?xml version=1.0 encoding=UTF-8? statuses status.../status status.../status status.../status ... To me it seems like it would be a lot easier just passing a stream like this to a parser. Instead, with the current stream of documents one has to look out for new prologs, split the stream at that point, parse that doc, reset the parser and continue. Not an insurmountable problem, but it seems like a lot of extra work that shouldn't really be needed. Just wondering what everyone else thinks. Ianiv Schweber
[twitter-dev] Re: Few questions about rate limiting
The hits are charged to the user's account if you authenticate. If you don't provide authentication credentials then the hits are charged to your IP address. You are limited to 100 requests per a hour. This limit applies to all API endpoints that are rate limited. So calling a search every 15s will cause you to run out of requests pretty quick. Hope that clears things up for you. On Wed, May 13, 2009 at 9:59 AM, szimek szi...@gmail.com wrote: Hi, I'm using Sign in with twitter functionality - since account/ verify_credentials request is rate limited, how can I otherwise get info about authenticated user? How exactly requests are limited? I.e. I got 100 users logged in in my web app and they all display page with their friends/followers list. Will these requests be counted for these users' accounts or IP of my server? What's the limit for search requests? We're using jsonp for fetching search results, so at least in this case the limit is for client's IP. Is it ok to make search request i.e. every 15s? Thanks in advance
[twitter-dev] Re: Few questions about rate limiting
Correction: Search API uses a higher limit then the other endpoints. It is not documented in the API, so I'm not sure what the limit would be. http://apiwiki.twitter.com/Rate-limiting On Wed, May 13, 2009 at 12:23 PM, Josh Roesslein jroessl...@gmail.comwrote: The hits are charged to the user's account if you authenticate. If you don't provide authentication credentials then the hits are charged to your IP address. You are limited to 100 requests per a hour. This limit applies to all API endpoints that are rate limited. So calling a search every 15s will cause you to run out of requests pretty quick. Hope that clears things up for you. On Wed, May 13, 2009 at 9:59 AM, szimek szi...@gmail.com wrote: Hi, I'm using Sign in with twitter functionality - since account/ verify_credentials request is rate limited, how can I otherwise get info about authenticated user? How exactly requests are limited? I.e. I got 100 users logged in in my web app and they all display page with their friends/followers list. Will these requests be counted for these users' accounts or IP of my server? What's the limit for search requests? We're using jsonp for fetching search results, so at least in this case the limit is for client's IP. Is it ok to make search request i.e. every 15s? Thanks in advance
[twitter-dev] User Icon in RSS Feed?
I'm building a little sidebar on a website with a twitter RSS feed, and I'd love to be able to use the user icons with it. I do not see this included in the feed, nor do they appear to have consistent URLs. Is there anyway to do this? I only found one post on the matter from two years ago in which it was said that the developers were getting to it. Has that happened yet?
[twitter-dev] Re: Invalid JSON
I'm waiting on a JS expert I know to get back to his desk to tell me whether {a:b\\} is valid JSON or not (as before, JSONLint says it is). I read it as valid (i.e., DOUBLEQUOTE b BACKSLASH DOUBLEQUOTE). -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- Those are my principles. If you don't like them, I have others. -- G. Marx -
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
I see that the product manager/APIs position is still on your site... does that mean the position is still open? Does teasing the API team help as a qualification? It may get you an interview, but only so they can surreptitiously get a photo and send it to the nice Italian men with the kneecappers. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- They told me I was gullible ... and I believed them. ---
[twitter-dev] Re: Invalid JSON
Yeah, JSONLint calls it valid, and every JS person I've talked to says it should be valid - so it seems there's a bug in ActiveSupport. In any case, this isn't Twitter's problem. Thanks... On May 13, 1:42 pm, Cameron Kaiser spec...@floodgap.com wrote: I'm waiting on a JS expert I know to get back to his desk to tell me whether {a:b\\} is valid JSON or not (as before, JSONLint says it is). I read it as valid (i.e., DOUBLEQUOTE b BACKSLASH DOUBLEQUOTE). -- personal:http://www.cameronkaiser.com/-- Cameron Kaiser * Floodgap Systems *www.floodgap.com* ckai...@floodgap.com -- Those are my principles. If you don't like them, I have others. -- G. Marx -
[twitter-dev] Re: New Public Streaming API Resource - Follow
John, this looks pretty interesting! Two questions: 1) you are requiring to send a username and password for Basic Auth - how does that map to apps / services using OAuth, as they won't have access to a user's passwords? (and related, how does this fit into your general roadmap to move everything to OAuth?) 2) the docs only mention http as a protocol, not https. In combination with requiring passwords, only making http available seems quite unsecure. Any plans to also support https soon (or any other mechanism which gives better security?) Thanks, Marco 2009/5/13 John Kalucki jkalu...@gmail.com Chad, Yes, I think this is called POSTDATA in browsers. I don't recall what the actual name of this part of the HTTP protocol is, but it's the body section after the headers. I corrected the file name error. Thanks. -John On May 12, 8:49 pm, Chad Etzel jazzyc...@gmail.com wrote: Hi John, /follow looks very interesting. Since you're asking for feedback I'm copying the follow parameter example documentation: Example: Create a file called 'follow' that contains, exactly and excluding the quotation marks: follow=12 13 15 16 20 87. Execute: curl -d @followinghttp://stream.twitter.com/follow.json -uAnyTwitterUser:Password.You will receive JSON updates from Jack Biz, Crystal, Ev, Krissy, but not from Jeremy, as he's a private user. I'm assuming that follow is just a POSTDATA variable in the normal case (you're just using curl's file posting ability in the example)? In the example, should the file be called following instead of follow (since you are using -d @following in the curl line)? Thanks, -Chad On Tue, May 12, 2009 at 11:24 PM, John Kalucki jkalu...@gmail.com wrote: Note: The Streaming API is currently under a limited alpha test, details below. The /follow Streaming API resource is now publicly available. This resource streams near-real-time public updates posted by an arbitrary set of users. Streaming by user_id may be interesting to a variety of developers who wish to provide a nearly instantaneous experience without the drawbacks of continuous polling, polling rate limits, auto- following and follow limits. For example, a desktop client could simulate a user's /home timeline, minus private updates and mentions, via the /follow resource. Continuous polling would no longer be necessary or desired. Upon receipt of a new streamed message, the REST API may be periodically polled to back-fill mentions, private statuses and other updates not available via the Streaming API. This stream may also be interesting to service developers that follow their subscribers solely to receive their replies or for data mining purposes. Auto-following, following limit and rate limit hassles could be exchanged for real-time streaming subscriber updates. Currently this resource is limited to following 200 user_ids. Developers requiring considerably more followings and/or back-filling via the count parameter should consider applying for the restricted /shadow resource. Feedback is encouraged as we determine the ease-of-use, value, tuning and operational viability of this resource. With any luck, streaming might also be easier on the Twitter service. Our flock of orange whale- hoisting birds are pretty tuckered out. Important Alpha Test Note: The Streaming API (aka Hosebird) is currently under an alpha test. All developers using the Streaming API must tolerate possible unannounced and extended periods of unavailability, especially during off-hours, Pacific Time. New features, resources and policies are being deployed on very little, if any, notice. Any developer may experiment with the unrestricted resources and provide feedback via this list. Access to restricted resources is extremely limited and is only granted on a case-by-case basis after acceptance of an additional terms of service document. Documentation is available: http://apiwiki.twitter.com/Streaming-API-Documentation.
[twitter-dev] Re: twitter on mobile
One more silly question.. I still wonder how to find friends_timeline (xml/json). http://twitter.com/statuses/friends_timeline.xml I mean, http://twitter.com/statuses/ after this URL.. how I will know that which name (with .xml or .json) I am suppose to write?? from where I will get those lists where names like friends_timeline is mentioned?? I am new so please help me. On May 13, 6:59 pm, Omkaaraa badshaah.k...@gmail.com wrote: Hi all, I had come here few days back also. now again after preparing my design. I want to write twitter application which will contain my own feature also, for MOBILE. for the time being, just forget which mobile I am going to use. Instead of J2ME, I will use similar framework. From here only I had got info about list of API's to develop twitter application. But I need API's for J2ME in fact. I have not gone in depth of these technical terms till now as far as mobile and twitter api's are concern because I was working with GUI and data flow of my own concept (which dont need any space on web server now because i have changed it a bit). So, would you please help me from where I will get those API's which will run on J2ME platform? please guide me properly considering I have basic idea of J2ME and developing twitter application as well. Thanks and Regards, Omi learner.
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
Quick update … While looking at the code that reminded me of this error I see it had some bugs of its own. We seem to have a matter of weeks rather than days before this change. Mobile developers and other who deride our lack of early notice take heed … now is the time to go unsigned. Thanks; – Matt Sanford / @mzsanford Twitter Dev On May 13, 2009, at 10:49 AM, Cameron Kaiser wrote: I see that the product manager/APIs position is still on your site... does that mean the position is still open? Does teasing the API team help as a qualification? It may get you an interview, but only so they can surreptitiously get a photo and send it to the nice Italian men with the kneecappers. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- They told me I was gullible ... and I believed them. ---
[twitter-dev] Re: all replies by friends
On May 13, 2009, at 2:50 AM, voorwiel wrote: Yeah, lots of protest out there, just watch the 'Links to this post:' below the Twitter blog post. There also is a hashtag to protest against this decision: #fixreplies . It is already in the 'Trending Topics' sidebar. Let's see how this evolves... I'm happy to say that I created that hashtag. @biz has posted and claimed that the reason for this change has to do with the underlying system architecture, which wasn't going to support viewing replies to unfollowed people much longer. (Personally, my mind boggles at that, filtering replies by a dynamically changing list of people is *easier* than not filtering them at all? Whatever.) Clearly this is a side effect of an architecture change in the past, since originally @foo didn't mean anything at all to the database. So somebody made an architecture decision sometime ago without considering the social ramifications. That sucks. My recommendation is to back the change out real quick to mollify folks, and then have an emergency meeting about how you're going to fix the architecture before whatever you did wrong breaks it--because you just took the viral out of Twitter. (And worse, made #followfriday a permanent fixture!) How major a change is this? Well, I wouldn't have my current job, house or girlfriend without being able to see other people's replies. I wouldn't have hired my virtual admin, or had a vacation this summer in Florida. I have two sets of friends who wouldn't be engaged without that feature. There are high-level analysts following me because I interjected something useful into a conversation they were having with a client--that wouldn't have happened. There are people I don't know, whose problems I solved because I saw a friend talking with them--that wouldn't have happened. On May 13, 2009, at 3:08 AM, Doug Williams wrote: Secondly, where is the entrepreneurial spirit to turn this into an opportunity? For the users of this option, think about the ways having @replies to strangers in your timeline helped you find people. How can you use the API make similar connections? The data is still there, present it in a compelling way. I'd use a service that sent an occasional email digest like: The following 3 users were mentioned at least X number of times by the people you follow, check 'em out. Somewhere along the line, Twitter appears to have fallen for the add 10,000 followers in one month crap. This isn't about how many people you can follow. This isn't some grade school popularity competition. This is about conversations. It's about listening to people talk to each other, learning who you like, who you don't. It's about learning new things and helping other people. There *is* no replacement for this feature that will provide the same functionality, or even *close* to the functionality. This isn't an opportunity for a third-party to use the API, this is an opportunity for Google to eat your lunch. Were lots of people using the feature? I have no idea. Certainly your core early adopters were. Certainly the social media folks (who write most of your press) were. Are the new folks who come to follow celebs or just chat with their friends using it? No, probably not. The fact of the matter is the UI for that setting sucked rocks. First you have the fundamental confusion between followers (a noun) and following (a verb AND a noun). Put the two together and you can figure it out, have following by itself, and you have no idea. That you get used to, but it's confusing to new users. Add in that sometimes the site talks about friends--g*d knows what those are. Then you attempted to explain the settings in a menu item instead of using a simple English sentence, and finally you had the wrong default (don't see replies sent to people I don't follow) for new users. Every time I told someone about that setting, they were like Oh! *That's* why I never saw anyone talking! I *knew* what the settings did and I still found the text confusing. Whenever I told someone about it, I had to reread the site to make sure I got it right. Blaming lack of use on a bad UI is just lazy. Twitter is a transport. Different people use it very differently. What Twitter just did is like someone taking email and deciding that mailing lists aren't really very important, since they are used by very few people, so we'll just remove the ability to have a mailing list. After all, it simplifies authentication, and it reduces traffic. Unfortunately, a core set of very important people really, really need mailing lists. It's how they network, meet people, develop ideas, and improve software like... Twitter. Oops.
[twitter-dev] Re: User Icon in RSS Feed?
I don't think anything has changed. You can take a look at SPIURL - http://code.google.com/p/spiurl/ On May 13, 10:35 am, 51productions chrisgr...@gmail.com wrote: I'm building a little sidebar on a website with a twitter RSS feed, and I'd love to be able to use the user icons with it. I do not see this included in the feed, nor do they appear to have consistent URLs. Is there anyway to do this? I only found one post on the matter from two years ago in which it was said that the developers were getting to it. Has that happened yet?
[twitter-dev] Re: all replies by friends
As I mentioned on Twitter: only 3% of users had opted to see replies to users they weren't following. Given that this setting was putting the hurt on our servers, we decided to remove it in the interest of a faster, more reliable Twitter experience for the vast majority of users. We have, however, heard the feedback. We have a conversation underway internally about bringing back this feature. My impression is that if it's technically feasible to do so, we'll take the user feedback we've received under consideration and bring it back. If it's not, we'll try to deliver an experience that's similar but supportable in the long term. We're aware of this issue and the arguments involved. As this discussion isn't directly related to the API, let's move on. Thanks. On Wed, May 13, 2009 at 12:31, Kee Hinckley naz...@somewhere.com wrote: On May 13, 2009, at 2:50 AM, voorwiel wrote: Yeah, lots of protest out there, just watch the 'Links to this post:' below the Twitter blog post. There also is a hashtag to protest against this decision: #fixreplies . It is already in the 'Trending Topics' sidebar. Let's see how this evolves... I'm happy to say that I created that hashtag. @biz has posted and claimed that the reason for this change has to do with the underlying system architecture, which wasn't going to support viewing replies to unfollowed people much longer. (Personally, my mind boggles at that, filtering replies by a dynamically changing list of people is *easier* than not filtering them at all? Whatever.) Clearly this is a side effect of an architecture change in the past, since originally @foo didn't mean anything at all to the database. So somebody made an architecture decision sometime ago without considering the social ramifications. That sucks. My recommendation is to back the change out real quick to mollify folks, and then have an emergency meeting about how you're going to fix the architecture before whatever you did wrong breaks it--because you just took the viral out of Twitter. (And worse, made #followfriday a permanent fixture!) How major a change is this? Well, I wouldn't have my current job, house or girlfriend without being able to see other people's replies. I wouldn't have hired my virtual admin, or had a vacation this summer in Florida. I have two sets of friends who wouldn't be engaged without that feature. There are high-level analysts following me because I interjected something useful into a conversation they were having with a client--that wouldn't have happened. There are people I don't know, whose problems I solved because I saw a friend talking with them--that wouldn't have happened. On May 13, 2009, at 3:08 AM, Doug Williams wrote: Secondly, where is the entrepreneurial spirit to turn this into an opportunity? For the users of this option, think about the ways having @replies to strangers in your timeline helped you find people. How can you use the API make similar connections? The data is still there, present it in a compelling way. I'd use a service that sent an occasional email digest like: The following 3 users were mentioned at least X number of times by the people you follow, check 'em out. Somewhere along the line, Twitter appears to have fallen for the add 10,000 followers in one month crap. This isn't about how many people you can follow. This isn't some grade school popularity competition. This is about conversations. It's about listening to people talk to each other, learning who you like, who you don't. It's about learning new things and helping other people. There *is* no replacement for this feature that will provide the same functionality, or even *close* to the functionality. This isn't an opportunity for a third-party to use the API, this is an opportunity for Google to eat your lunch. Were lots of people using the feature? I have no idea. Certainly your core early adopters were. Certainly the social media folks (who write most of your press) were. Are the new folks who come to follow celebs or just chat with their friends using it? No, probably not. The fact of the matter is the UI for that setting sucked rocks. First you have the fundamental confusion between followers (a noun) and following (a verb AND a noun). Put the two together and you can figure it out, have following by itself, and you have no idea. That you get used to, but it's confusing to new users. Add in that sometimes the site talks about friends--g*d knows what those are. Then you attempted to explain the settings in a menu item instead of using a simple English sentence, and finally you had the wrong default (don't see replies sent to people I don't follow) for new users. Every time I told someone about that setting, they were like Oh! *That's* why I never saw anyone talking! I *knew* what the settings did and I still found the text confusing. Whenever I told someone about it, I had to reread the site to make sure I got it
[twitter-dev] Can't reach twitter.com from Amazon Cloud instance after twitter maintenance
I have a couple of Amazon Cloud instances that haven't been able to contact twitter.com after Twitter came back up this afternoon from the database maintenance. I can reach it fine through other machines not on the Amazon cloud. Anyone else having this trouble? Just a simple telnet www.twitter.com 80 goes unresponded to. A browser trying to get to twitter.com also fails on two differen Amazon instances. Any ideas on what might be happening or have I just jumped the gun before Twitter is all the way back up? Randy
[twitter-dev] Re: New Public Streaming API Resource - Follow
The main API does support HTTPS. On Wed, May 13, 2009 at 16:50, John Kalucki jkalu...@gmail.com wrote: I'll attempt to answer these questions, but I can only do so with some speculation and humble ignorance. 1) OAuth allows clients to authenticate with the Twitter REST API via third-party services. These services should not also need to interact with the Streaming API on a per client basis. Instead, the service should establish a single query that satisfies all clients' needs. This may not be practical in all cases, but I suspect we can approximate the desired behavior with the current set of primitives. 2) There are no immediate plans to support HTTPS, mainly because we're not really trying to keep the data private. Also, and I am probably totally wrong here, I don't think we use HTTPS on the main WWW site or on the REST API, so this doesn't make things much worse than they already are. A possible workaround would be for sensitive service to create an account just for streaming. Should the password be compromised, there's only a denial of service risk and no further risk. -John On May 13, 11:18 am, Marco Kaiser kaiser.ma...@gmail.com wrote: John, this looks pretty interesting! Two questions: 1) you are requiring to send a username and password for Basic Auth - how does that map to apps / services using OAuth, as they won't have access to a user's passwords? (and related, how does this fit into your general roadmap to move everything to OAuth?) 2) the docs only mention http as a protocol, not https. In combination with requiring passwords, only making http available seems quite unsecure. Any plans to also support https soon (or any other mechanism which gives better security?) Thanks, Marco 2009/5/13 John Kalucki jkalu...@gmail.com Chad, Yes, I think this is called POSTDATA in browsers. I don't recall what the actual name of this part of the HTTP protocol is, but it's the body section after the headers. I corrected the file name error. Thanks. -John On May 12, 8:49 pm, Chad Etzel jazzyc...@gmail.com wrote: Hi John, /follow looks very interesting. Since you're asking for feedback I'm copying the follow parameter example documentation: Example: Create a file called 'follow' that contains, exactly and excluding the quotation marks: follow=12 13 15 16 20 87. Execute: curl -d @followinghttp://stream.twitter.com/follow.json -uAnyTwitterUser:Password.You will receive JSON updates from Jack Biz, Crystal, Ev, Krissy, but not from Jeremy, as he's a private user. I'm assuming that follow is just a POSTDATA variable in the normal case (you're just using curl's file posting ability in the example)? In the example, should the file be called following instead of follow (since you are using -d @following in the curl line)? Thanks, -Chad On Tue, May 12, 2009 at 11:24 PM, John Kalucki jkalu...@gmail.com wrote: Note: The Streaming API is currently under a limited alpha test, details below. The /follow Streaming API resource is now publicly available. This resource streams near-real-time public updates posted by an arbitrary set of users. Streaming by user_id may be interesting to a variety of developers who wish to provide a nearly instantaneous experience without the drawbacks of continuous polling, polling rate limits, auto- following and follow limits. For example, a desktop client could simulate a user's /home timeline, minus private updates and mentions, via the /follow resource. Continuous polling would no longer be necessary or desired. Upon receipt of a new streamed message, the REST API may be periodically polled to back-fill mentions, private statuses and other updates not available via the Streaming API. This stream may also be interesting to service developers that follow their subscribers solely to receive their replies or for data mining purposes. Auto-following, following limit and rate limit hassles could be exchanged for real-time streaming subscriber updates. Currently this resource is limited to following 200 user_ids. Developers requiring considerably more followings and/or back-filling via the count parameter should consider applying for the restricted /shadow resource. Feedback is encouraged as we determine the ease-of-use, value, tuning and operational viability of this resource. With any luck, streaming might also be easier on the Twitter service. Our flock of orange whale- hoisting birds are pretty tuckered out. Important Alpha Test Note: The Streaming API (aka Hosebird) is currently under an alpha test. All developers using the Streaming API must tolerate possible unannounced and extended periods of unavailability, especially during
[twitter-dev] Re: New Public Streaming API Resource - Follow
On Wed, May 13, 2009 at 5:50 PM, John Kalucki jkalu...@gmail.com wrote: 2) There are no immediate plans to support HTTPS, mainly because we're not really trying to keep the data private. Also, and I am probably totally wrong here, I don't think we use HTTPS on the main WWW site or on the REST API, so this doesn't make things much worse than they already are. A possible workaround would be for sensitive service to create an account just for streaming. Should the password be compromised, there's only a denial of service risk and no further risk. Yes, there is HTTPS available on both the main site and the REST API, and several clients/apps have been heavily chided when it was discovered they were not using it to authenticate users. I don't think the concern is keeping the data private, but keeping the credentials private. But, as you said, another account could be used instead... However, since this is an open stream, it may be problematic to use HTTPS since the normal encryption case would encrypt/sign the entire response headers + body... which it can't really do in this case. That certainly is an interesting problem for long-poll solutions like this. -Chad
[twitter-dev] Re: New Public Streaming API Resource - Follow
I'll attempt to answer these questions, but I can only do so with some speculation and humble ignorance. 1) OAuth allows clients to authenticate with the Twitter REST API via third-party services. These services should not also need to interact with the Streaming API on a per client basis. Instead, the service should establish a single query that satisfies all clients' needs. This may not be practical in all cases, but I suspect we can approximate the desired behavior with the current set of primitives. 2) There are no immediate plans to support HTTPS, mainly because we're not really trying to keep the data private. Also, and I am probably totally wrong here, I don't think we use HTTPS on the main WWW site or on the REST API, so this doesn't make things much worse than they already are. A possible workaround would be for sensitive service to create an account just for streaming. Should the password be compromised, there's only a denial of service risk and no further risk. -John On May 13, 11:18 am, Marco Kaiser kaiser.ma...@gmail.com wrote: John, this looks pretty interesting! Two questions: 1) you are requiring to send a username and password for Basic Auth - how does that map to apps / services using OAuth, as they won't have access to a user's passwords? (and related, how does this fit into your general roadmap to move everything to OAuth?) 2) the docs only mention http as a protocol, not https. In combination with requiring passwords, only making http available seems quite unsecure. Any plans to also support https soon (or any other mechanism which gives better security?) Thanks, Marco 2009/5/13 John Kalucki jkalu...@gmail.com Chad, Yes, I think this is called POSTDATA in browsers. I don't recall what the actual name of this part of the HTTP protocol is, but it's the body section after the headers. I corrected the file name error. Thanks. -John On May 12, 8:49 pm, Chad Etzel jazzyc...@gmail.com wrote: Hi John, /follow looks very interesting. Since you're asking for feedback I'm copying the follow parameter example documentation: Example: Create a file called 'follow' that contains, exactly and excluding the quotation marks: follow=12 13 15 16 20 87. Execute: curl -d @followinghttp://stream.twitter.com/follow.json -uAnyTwitterUser:Password.You will receive JSON updates from Jack Biz, Crystal, Ev, Krissy, but not from Jeremy, as he's a private user. I'm assuming that follow is just a POSTDATA variable in the normal case (you're just using curl's file posting ability in the example)? In the example, should the file be called following instead of follow (since you are using -d @following in the curl line)? Thanks, -Chad On Tue, May 12, 2009 at 11:24 PM, John Kalucki jkalu...@gmail.com wrote: Note: The Streaming API is currently under a limited alpha test, details below. The /follow Streaming API resource is now publicly available. This resource streams near-real-time public updates posted by an arbitrary set of users. Streaming by user_id may be interesting to a variety of developers who wish to provide a nearly instantaneous experience without the drawbacks of continuous polling, polling rate limits, auto- following and follow limits. For example, a desktop client could simulate a user's /home timeline, minus private updates and mentions, via the /follow resource. Continuous polling would no longer be necessary or desired. Upon receipt of a new streamed message, the REST API may be periodically polled to back-fill mentions, private statuses and other updates not available via the Streaming API. This stream may also be interesting to service developers that follow their subscribers solely to receive their replies or for data mining purposes. Auto-following, following limit and rate limit hassles could be exchanged for real-time streaming subscriber updates. Currently this resource is limited to following 200 user_ids. Developers requiring considerably more followings and/or back-filling via the count parameter should consider applying for the restricted /shadow resource. Feedback is encouraged as we determine the ease-of-use, value, tuning and operational viability of this resource. With any luck, streaming might also be easier on the Twitter service. Our flock of orange whale- hoisting birds are pretty tuckered out. Important Alpha Test Note: The Streaming API (aka Hosebird) is currently under an alpha test. All developers using the Streaming API must tolerate possible unannounced and extended periods of unavailability, especially during off-hours, Pacific Time. New features, resources and policies are being deployed on very little, if any, notice. Any developer may experiment with the unrestricted resources and provide feedback via this list. Access to
[twitter-dev] Re: twitter on mobile
This might help you get started: http://apiwiki.twitter.com/Getting-Started On Wed, May 13, 2009 at 13:08, Omkaaraa badshaah.k...@gmail.com wrote: One more silly question.. I still wonder how to find friends_timeline (xml/json). http://twitter.com/statuses/friends_timeline.xml I mean, http://twitter.com/statuses/ after this URL.. how I will know that which name (with .xml or .json) I am suppose to write?? from where I will get those lists where names like friends_timeline is mentioned?? I am new so please help me. On May 13, 6:59 pm, Omkaaraa badshaah.k...@gmail.com wrote: Hi all, I had come here few days back also. now again after preparing my design. I want to write twitter application which will contain my own feature also, for MOBILE. for the time being, just forget which mobile I am going to use. Instead of J2ME, I will use similar framework. From here only I had got info about list of API's to develop twitter application. But I need API's for J2ME in fact. I have not gone in depth of these technical terms till now as far as mobile and twitter api's are concern because I was working with GUI and data flow of my own concept (which dont need any space on web server now because i have changed it a bit). So, would you please help me from where I will get those API's which will run on J2ME platform? please guide me properly considering I have basic idea of J2ME and developing twitter application as well. Thanks and Regards, Omi learner. -- Abraham Williams | http://the.hackerconundrum.com Hacker | http://abrah.am | http://twitter.com/abraham Web608 | Community Evangelist | http://web608.org This email is: [ ] blogable [x] ask first [ ] private.
[twitter-dev] replies blog post clarification...
from @biz and his post : First, we're making a change such that any updates beginning with @username (that are not explicitly created by clicking on the reply icon) will be seen by everyone following that account. for us folks that fiddle with the API. Can we read this as Tweets that start with @username, but don't have the in_reply_to_status_id parameter set? -steve
[twitter-dev] Re: Can't reach twitter.com from Amazon Cloud instance after twitter maintenance
Evidently Jumped the gun is the answer. It was about 20 minutes after other machines were able to get to Twitter that the Amazon instances were getting a response from Twitter. On May 13, 2:30 pm, RC bioscienceupda...@gmail.com wrote: I have a couple of Amazon Cloud instances that haven't been able to contact twitter.com after Twitter came back up this afternoon from the database maintenance. I can reach it fine through other machines not on the Amazon cloud. Anyone else having this trouble? Just a simple telnetwww.twitter.com80 goes unresponded to. A browser trying to get to twitter.com also fails on two differen Amazon instances. Any ideas on what might be happening or have I just jumped the gun before Twitter is all the way back up? Randy
[twitter-dev] Re: Bad Celebrity Search Results
I'm also curious about this phenomenon. It does seem that certain high-profile users are indexed poorly in search. It doesn't seem to correlate exactly with follower numbers but I've not done much in the way of empirical analysis on that. It's causing me some trouble on tweetreach.com as I often get requests to run reach reports about things tweeted by high profile users only to find that tweets from those users aren't returned by search. Should I file an issue for this? Hayes On Tue, May 12, 2009 at 1:54 PM, rob robsew...@gmail.com wrote: Hi, I'm doing a project that deals specifically with P Diddy (twitter.com/ iamdiddy). When I do a search, no tweets newer than May 6th show up. However, looking at his timeline, more recent tweets exist. This isn't a problem with my tweets (twitter.com/robseward). Also, it appears search is not returning accurate results for other celebrities online. Does anyone know what's going on here? Is the problem isolated to users with a high number of followers or is it something non celebrity- related. Some examples below: P Diddy: http://search.twitter.com/search?q=from%3Aiamdiddy http://twitter.com/iamdiddy Ashton Kutcher: http://search.twitter.com/search?q=from%3Aaplusk http://twitter.com/aplusk Shaq: http://search.twitter.com/search?q=from=THE_REAL_SHAQ twitter.com/THE_REAL_SHAQ Me (not a celebrity. Accurate search results). http://search.twitter.com/search?q=+from%3Arobseward twitter.com/robseward Rob
[twitter-dev] Re: Status ID closing in on maximum unsigned integer
Let me be the first to say THANK YOU for this advance notice. I found and fixed some bugs in our iPhone client today because of it — and I'm very happy to hear that we have some time to get the code approved in the App Store. We love to complain when the process goes awry, but it's also important to remember to give credit when it's well and truly due. Thanks again! -ch On May 13, 12:12 pm, Matt Sanford m...@twitter.com wrote: Quick update … While looking at the code that reminded me of this error I see it had some bugs of its own. We seem to have a matter of weeks rather than days before this change. Mobile developers and other who deride our lack of early notice take heed … now is the time to go unsigned. Thanks; – Matt Sanford / @mzsanford Twitter Dev On May 13, 2009, at 10:49 AM, Cameron Kaiser wrote: I see that the product manager/APIs position is still on your site... does that mean the position is still open? Does teasing the API team help as a qualification? It may get you an interview, but only so they can surreptitiously get a photo and send it to the nice Italian men with the kneecappers. -- personal:http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems *www.floodgap.com* ckai...@floodgap.com -- They told me I was gullible ... and I believed them. ---
[twitter-dev] Re: replies blog post clarification...
Non-confirmed @replies will still be seen by all. Confirmed @replies, or those with in_reply_to_status_id set, will only be seen by mutual followers. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Wed, May 13, 2009 at 3:25 PM, Steve Brunton sbrun...@gmail.com wrote: from @biz and his post : First, we're making a change such that any updates beginning with @username (that are not explicitly created by clicking on the reply icon) will be seen by everyone following that account. for us folks that fiddle with the API. Can we read this as Tweets that start with @username, but don't have the in_reply_to_status_id parameter set? -steve
[twitter-dev] twitter digest not available
Hi, thanks to twitter's api and the api team, the data feed for data mining is just wonderful. I have put together a real time system that takes in the feed and does some NLP analysis on tweets using open tools like Open Calais and openNLP. The results are freely available on http://web2express.org/. Using this twiiter web app, you can spot daily hot topics and for each hot topic, quickly find the top contributing twitter users. I hope this real time information will help users to understand the popular topics at any given moment and easily identify who to follow. Please let me know if you have any comment. -aj AJ Chen, PhD Co-Chair, Semantic Web SIG, sdforum.org http://web2express.org Palo Alto, CA
[twitter-dev] Re: Bad Celebrity Search Results
i just found out some high-volume users aren't indexed at all. for example: http://search.twitter.com/search?q=+from%3Athe_real_shaq and it's not just the from: operators that's broken. if I search for some text from one of his most recent tweets, nothing from him shows up. for this tweet: http://twitter.com/THE_REAL_SHAQ/status/1785949479 this search doesn't get it: http://search.twitter.com/search?q=Listen+in+tonite+7p+to+9p+Shaq -Brendan On Wed, May 13, 2009 at 2:04 PM, Hayes Davis ha...@appozite.com wrote: I'm also curious about this phenomenon. It does seem that certain high-profile users are indexed poorly in search. It doesn't seem to correlate exactly with follower numbers but I've not done much in the way of empirical analysis on that. It's causing me some trouble on tweetreach.com as I often get requests to run reach reports about things tweeted by high profile users only to find that tweets from those users aren't returned by search. Should I file an issue for this? Hayes On Tue, May 12, 2009 at 1:54 PM, rob robsew...@gmail.com wrote: Hi, I'm doing a project that deals specifically with P Diddy (twitter.com/ iamdiddy). When I do a search, no tweets newer than May 6th show up. However, looking at his timeline, more recent tweets exist. This isn't a problem with my tweets (twitter.com/robseward). Also, it appears search is not returning accurate results for other celebrities online. Does anyone know what's going on here? Is the problem isolated to users with a high number of followers or is it something non celebrity- related. Some examples below: P Diddy: http://search.twitter.com/search?q=from%3Aiamdiddy http://twitter.com/iamdiddy Ashton Kutcher: http://search.twitter.com/search?q=from%3Aaplusk http://twitter.com/aplusk Shaq: http://search.twitter.com/search?q=from=THE_REAL_SHAQ twitter.com/THE_REAL_SHAQ Me (not a celebrity. Accurate search results). http://search.twitter.com/search?q=+from%3Arobseward twitter.com/robseward Rob
[twitter-dev] Re: Bad Celebrity Search Results
Hi Brendan, found out how? references? or merely observed per the link? It's curious because it might be throwing off my calculated 'coolness vector' - I noticed the coolness vector of a tweet containing a celebs name seemed lower than anticipiated - however it wasn't the point of the experiment, mind you - I just found the downplay sorta odd and mad- ening. :-) Thanks Waitman On May 13, 6:44 pm, Brendan O'Connor breno...@gmail.com wrote: i just found out some high-volume users aren't indexed at all. for example:
[twitter-dev] Re: Bad Celebrity Search Results
i wanted to get all of shaq's tweets with the search api and i couldn't :) going through random ones on twitterholic .. from:aplusk barely works from:scobleizer works from:adventuregirl works from:timoreilly works haven't tried the standard API for these. bleah Brendan On Wed, May 13, 2009 at 6:57 PM, explicious avail4...@gmail.com wrote: Hi Brendan, found out how? references? or merely observed per the link? It's curious because it might be throwing off my calculated 'coolness vector' - I noticed the coolness vector of a tweet containing a celebs name seemed lower than anticipiated - however it wasn't the point of the experiment, mind you - I just found the downplay sorta odd and mad- ening. :-) Thanks Waitman On May 13, 6:44 pm, Brendan O'Connor breno...@gmail.com wrote: i just found out some high-volume users aren't indexed at all. for example:
[twitter-dev] Re: replies blog post clarification...
Riddle me this, what happens in this scenario: I'm following user A and user B 1. User A sends an @reply to user B. 2. User B changes his name. 3. Tweet is going to user B's old name. Will I see the tweet? Zac Bowling On Wed, May 13, 2009 at 6:26 PM, Doug Williams d...@twitter.com wrote: Non-confirmed @replies will still be seen by all. Confirmed @replies, or those with in_reply_to_status_id set, will only be seen by mutual followers. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Wed, May 13, 2009 at 3:25 PM, Steve Brunton sbrun...@gmail.com wrote: from @biz and his post : First, we're making a change such that any updates beginning with @username (that are not explicitly created by clicking on the reply icon) will be seen by everyone following that account. for us folks that fiddle with the API. Can we read this as Tweets that start with @username, but don't have the in_reply_to_status_id parameter set? -steve
[twitter-dev] oauth just to post status
Hi all. My app, once it gets going, is mainly going to post status updates as mentions to followers. For instance, the app, called DummyOauthApp will post a statuses/update like '@tayknight blah blah blah'. So, i don't need my app to authenticate itself to oauth as tayknight, but as itself DummyOauthApp. Does this mean I need to generate an access token/secret pair for the app itself and continually reuse this pair to call statuses/update? Thanks.
[twitter-dev] Re: replies blog post clarification...
This change is not owned by the API team. If you test this, please report back. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Wed, May 13, 2009 at 7:37 PM, Zac Bowling zbowl...@gmail.com wrote: Riddle me this, what happens in this scenario: I'm following user A and user B 1. User A sends an @reply to user B. 2. User B changes his name. 3. Tweet is going to user B's old name. Will I see the tweet? Zac Bowling On Wed, May 13, 2009 at 6:26 PM, Doug Williams d...@twitter.com wrote: Non-confirmed @replies will still be seen by all. Confirmed @replies, or those with in_reply_to_status_id set, will only be seen by mutual followers. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Wed, May 13, 2009 at 3:25 PM, Steve Brunton sbrun...@gmail.com wrote: from @biz and his post : First, we're making a change such that any updates beginning with @username (that are not explicitly created by clicking on the reply icon) will be seen by everyone following that account. for us folks that fiddle with the API. Can we read this as Tweets that start with @username, but don't have the in_reply_to_status_id parameter set? -steve
[twitter-dev] Re: oauth just to post status
Updates have to come from a twitter account. Your app can not post updates by itself. You will either have to use the OAuth flow for what ever account you use or you could use basic auth. On Wed, May 13, 2009 at 21:50, tayknight taykni...@gmail.com wrote: Hi all. My app, once it gets going, is mainly going to post status updates as mentions to followers. For instance, the app, called DummyOauthApp will post a statuses/update like '@tayknight blah blah blah'. So, i don't need my app to authenticate itself to oauth as tayknight, but as itself DummyOauthApp. Does this mean I need to generate an access token/secret pair for the app itself and continually reuse this pair to call statuses/update? Thanks. -- Abraham Williams | http://the.hackerconundrum.com Hacker | http://abrah.am | http://twitter.com/abraham Web608 | Community Evangelist | http://web608.org This email is: [ ] blogable [x] ask first [ ] private. Sent from Madison, WI, United States
[twitter-dev] Timelines/search frozen?
I haven't gotten any tweets in over an hour, and search is frozen, too. Individual profile pages show new tweets, and public timeline seems to have them, too. Am I the only one seeing this? -- ivey
[twitter-dev] Re: Timelines/search frozen?
I haven't gotten any tweets in over an hour, and search is frozen, too. Individual profile pages show new tweets, and public timeline seems to have them, too. Am I the only one seeing this? No, I'm noticing it's stuck also. DMs are coming through however. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- There are few problems that the liberal usage of high explosives can't cure.
[twitter-dev] It looks like timelines and search are stuck
Guys, I don't see any notice on the status blog or other official channels so I figured I might as well say something. None of my account timelines have updated in more than 45 minutes and all search queries I'm looking at are that old as well. Hayes
[twitter-dev] Re: API FAQ can be reached in office but cann't at home
is apiwiki.twitter.com just a workplace for twitter.com api hosted by pbworks? so it's the problem with pbwork.com,not with twitter.com? On May 13, 6:50 pm, Coonay fla...@gmail.com wrote: http://apiwiki.twitter.com/FAQcan be reached in office,where IE PROXY is needed .but it cann't be reached at home(direct internet coonnection),where i cann't use the proxy.Is there any access restriction to access the API site . PS: thehttp://twitter.com/can be reached both office and home
[twitter-dev] Re: API FAQ can be reached in office but cann't at home
use https://proxytea.appspot.com/ to access it if you have the problem also On May 13, 6:50 pm, Coonay fla...@gmail.com wrote: http://apiwiki.twitter.com/FAQcan be reached in office,where IE PROXY is needed .but it cann't be reached at home(direct internet coonnection),where i cann't use the proxy.Is there any access restriction to access the API site . PS: thehttp://twitter.com/can be reached both office and home
[twitter-dev] Re: oauth just to post status
I meant to add, I want to use oauth for this so I get the 'via DummyOauthApp' line in the tweet. On May 13, 9:50 pm, tayknight taykni...@gmail.com wrote: Hi all. My app, once it gets going, is mainly going to post status updates as mentions to followers. For instance, the app, called DummyOauthApp will post a statuses/update like '@tayknight blah blah blah'. So, i don't need my app to authenticate itself to oauth as tayknight, but as itself DummyOauthApp. Does this mean I need to generate an access token/secret pair for the app itself and continually reuse this pair to call statuses/update? Thanks.
[twitter-dev] Re: It looks like timelines and search are stuck
Ops guys are on it, fixing a problem with back end machines. Thanks for the report. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Wed, May 13, 2009 at 8:44 PM, Hayes Davis ha...@appozite.com wrote: Guys, I don't see any notice on the status blog or other official channels so I figured I might as well say something. None of my account timelines have updated in more than 45 minutes and all search queries I'm looking at are that old as well. Hayes
[twitter-dev] Re: It looks like timelines and search are stuck
Confirming the same issue. I've been observing this for the last 24 hours or so. On May 13, 9:18 pm, Doug Williams d...@twitter.com wrote: Ops guys are on it, fixing a problem with back end machines. Thanks for the report. Thanks, Doug -- Doug Williams Twitter Platform Supporthttp://twitter.com/dougw On Wed, May 13, 2009 at 8:44 PM, Hayes Davis ha...@appozite.com wrote: Guys, I don't see any notice on the status blog or other official channels so I figured I might as well say something. None of my account timelines have updated in more than 45 minutes and all search queries I'm looking at are that old as well. Hayes
[twitter-dev] Re: Timelines/search frozen?
Repairs are finished. Things should be on their way back to normal once the queues empty. Thanks, Doug -- Sent from my mobile device. On Wed, May 13, 2009 at 9:17 PM, Doug Williams d...@twitter.com wrote: Looks like there are some problems on the back end right now. Ops guys are on it. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Wed, May 13, 2009 at 9:07 PM, Cameron Kaiser spec...@floodgap.comwrote: I haven't gotten any tweets in over an hour, and search is frozen, too. Individual profile pages show new tweets, and public timeline seems to have them, too. Am I the only one seeing this? No, I'm noticing it's stuck also. DMs are coming through however. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- There are few problems that the liberal usage of high explosives can't cure.