Re: [twitter-dev] Introduce yourself!
Great idea Abraham! I'm Jonathan Markwell I first experimented with the the API in in Summer 2007 when I made our office Nabaztag Rabbit speak all the tweets in my timeline (it wasn't long before it was thrown out the window). I pay the bills as an independent Ruby on Rails developer, mostly working on Twitter integrations. I'm really interested in the challenges of scaling developer communities and have just started a series of blog posts on the topic here: http://blog.jot.is/the-unsolved-scaling-challenge I enjoy organising real world developer events and activities inspired by the barcamp, coworking, hack space and seed accelerator movements mostly in Brighton and London in the UK. These include: The Twitter Developer Nest: http://twitterdevelopernest.com WarbleCamp - The Twitter Developer Unconference: http://warblecamp.com Developer Mission - A group of UK hackers travelling to Chirp: http://developermission.com The Skiff coworking space: http://theskiff.org BootCycle - Peer support for early stage technology products: http://bootcycle.com BarCampBrighton: http://barcampbrighton.org Please let me know if you are interested in participating in any of the above or replicating them in your part of the world. The feature I'd most like to see added to the API is a way for us to add freeform metadata to tweets and users. This would enable the developer community to experiment with the potential of new features enabling us to innovate in the same way that users have with notations such as @ and #. I'm also working on my own app called Smidgn which is an approach to achieving some of the benefits the above feature would provide. Looking forward to meeting many of you at Chirp, I'm also travelling to SXSWi for the first time this year, would be great to meet any of you that are there. @jot On Fri, Feb 19, 2010 at 8:20 PM, Abraham Williams 4bra...@gmail.com wrote: We have not had an introductions thread in a long time (or ever that I could find) so I'm starting one. Don't forget to add an answer to the tools thread [1](Gmail link [2]) as well. I'm Abraham Williams, I've been working with the Twitter API and this group since early 2008. I do mostly freelance Drupal and Twitter API integration and personal projects. I love seeing the creative projects developers build or integrate with the API and look forward to meeting many of you at Chirp. TwitterOAuth [3] the first PHP library to support OAuth is built and maintained by me, and will hopefully see a new release soon. I also built a fun Chrome extension [4] that integrates common friends and followers into Twitter profiles. The feature I would most like added to the API is a conversation method to get replies to a specific status. So. Who are you, what do you do, what have you built, and what feature do you most want to see added? @Abraham [1] http://groups.google.com/group/twitter-development-talk/browse_thread/thread/c7cdaa0840f0de84/ [2] https://mail.google.com/mail/#inbox/12680cd0fa59011e [3] https://chrome.google.com/extensions/detail/npdjhmblakdjfnnajeomfbogokloiggg [4] http://code.google.com/p/twitter-api/issues/detail?id=142 -- Abraham Williams | Community Advocate | http://abrah.am Project | Out Loud | http://outloud.labs.poseurtech.com This email is: [ ] shareable [x] ask first [ ] private. Sent from Seattle, WA, United States -- Jonathan Markwell Engineer | Founder | Connector Inuda Innovations Ltd, Brighton, UK Web application development support Twitter Facebook integration specialists http://inuda.com Organising the world's first events for the Twitter developer Community http://TwitterDeveloperNest.com Providing a nice little place to work in the middle of Brighton - http://theskiff.org Measuring your brand's visibility on the social web - http://HowSociable.com mob: 07766 021 485 | tel: 01273 704 549 | fax: 01273 376 953 skype: jlmarkwell | twitter: http://twitter.com/jot
Re: [twitter-dev] Introduce yourself!
Hi gals guys, I'm Aral Balkan and I'm an interaction designer and iPhone/Flash/Web developer. I just launched my second iPhone app called Feathers (decorate your tweets). It's Twitter client for posting fun updates – in different text styles, including upside-down, and even Morse code (the 1.1 update brings 1337 text too) using Unicode symbols: http://feathersapp.com I started playing with the Twitter API back in the Flash days, where I added Twitter API support to SWX (the native data format for Flash I created; like JSON but data is stored in SWF bytecode). I'm not maintaining SWX any longer but it is still being used and you can find more info at http://apiwiki.twitter.com/SWX+Twitter+API and at http://swxformat.org Having contributed heavily to the Flash community over the last decade, I share some of Jon's concerns and would love to see the Twitter developer community scale well. I wasn't a big fan of the current oAuth policy (as it pertains to mobile desktop apps) but I'm very interested in xAuth (see http://aralbalkan.com/3057) and support Twitter fully on it. I hope to implement xAuth in Feathers the moment I'm let in to the beta (hint, hint, nudge, nudge) :) I blog on http://aralbalkan.com I'm going to be attending Chirp and I look forward to meeting some of you there :) Take care, Aral On Sat, Feb 20, 2010 at 2:55 PM, Jonathan Markwell j.l.markw...@inuda.comwrote: Great idea Abraham! I'm Jonathan Markwell I first experimented with the the API in in Summer 2007 when I made our office Nabaztag Rabbit speak all the tweets in my timeline (it wasn't long before it was thrown out the window). I pay the bills as an independent Ruby on Rails developer, mostly working on Twitter integrations. snip
[twitter-dev] Migrating to the Twitter Streaming API: A Primer
via TechCunct IT (http://www.techcrunchit.com/2010/02/18/gnips-manual- on-the-twitter-streaming-api/) comes a post on Gnip's blog (a API aggregation platform) : Migrating to the Twitter Streaming API: A Primer http://blog.gnip.com/2010/02/15/migrating-to-the-twitter-streaming-api-a-primer/ Ian http://twitter.com/ianirving http://www.FalsePositives.com : Code and Culture : http://www.ConnectedThinking.com : Building the People and Data Driven Web
[twitter-dev] Re: Introduce yourself!
My name is Maurice Wright. I'm a mechanical engineer turned web designer, turned web developer, turned web marketer. Most of this time was spent in the financial services industry. I first really found out about the Twitter API early last year. Early this year I began working on a Twitter advertising platform (http://www.pay4tweet.com) that will focus on ease of use rather than landing celebrity Tweeters. I spend most of my other free web time maintaining a design awards site (http://www.moluv.com - 10 years) and honing my internet marketing skills. I was relocated out of my job and December, and it feels good...at least until I have to start paying bills again. Anyone need online marketing help? The feature I'd like to most see from Twitter is a non-feature. Please keep it simple. Google has done a great job with this over the years with their main search page. Hopefully, Twitter can do the same. @moluv1 @moluv00 @pay4tweet On Feb 19, 12:20 pm, Abraham Williams 4bra...@gmail.com wrote: We have not had an introductions thread in a long time (or ever that I could find) so I'm starting one. Don't forget to add an answer to the tools thread [1](Gmail link [2]) as well. I'm Abraham Williams, I've been working with the Twitter API and this group since early 2008. I do mostly freelance Drupal and Twitter API integration and personal projects. I love seeing the creative projects developers build or integrate with the API and look forward to meeting many of you at Chirp. TwitterOAuth [3] the first PHP library to support OAuth is built and maintained by me, and will hopefully see a new release soon. I also built a fun Chrome extension [4] that integrates common friends and followers into Twitter profiles. The feature I would most like added to the API is a conversation method to get replies to a specific status. So. Who are you, what do you do, what have you built, and what feature do you most want to see added? @Abraham [1]http://groups.google.com/group/twitter-development-talk/browse_thread... [2]https://mail.google.com/mail/#inbox/12680cd0fa59011e [3]https://chrome.google.com/extensions/detail/npdjhmblakdjfnnajeomfbogo... [4]http://code.google.com/p/twitter-api/issues/detail?id=142 -- Abraham Williams | Community Advocate |http://abrah.am Project | Out Loud |http://outloud.labs.poseurtech.com This email is: [ ] shareable [x] ask first [ ] private. Sent from Seattle, WA, United States
[twitter-dev] Limiting API Response
Hello, I am developing a twitter client that runs on an embedded microprocessor (Arduino) which has only 2K of RAM. The problem is that Twitter's API response (even if only for a single @ mention) is too large to store to a string with only 2K RAM. Is there a way I can make my API call and limit the response to certain pertinent return values? Such as only screen name, text and time? This would greatly cutdown on the memory needed to store the API response. Thank you, Matt23
[twitter-dev] Re: Introduce yourself!
Yong Su Kim - Founder of http://HanPerson.com where we create applications and games for the Social Web. Current Twitter related Projects include http://TwitIQ.com and http://TwitHive.com. We're also working on another Twitter related project around conversations that will be launched soon. I have lots of feature requests but the three top ones that would help me immediately would be: - Display of in_reply_to_status_id in search results if tweets are replies to other tweets - Better Twitter paging support so that you can retrieve tweets in date order starting from a since_id - Better people search ranking geared towards twitter names (eg. search for yong and it should return yongsu first) Yong Su Kim - http://www.HanPerson.com ysk at hanperson.com
[twitter-dev] Why do you sent no-cache headers?
Hello Twitter dev team! I am wondering, why don't you take advantage of Cache control headers when processing requests for user's profiles? For example, a url of profile (which does not require api key or any type of authentication and can be used by anyone) http://api.twitter.com/1/users/show.json?user_id=52146756 It has this header: Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, post- check=0 So a browser or server cannot cache this, but why? It looks like you do report the Last-Modified header and even the Etag So it would be much more efficient if you would just reply with a 304 Not modified header if request contains the If-Modified-Since value Do you support conditional requests with If-Modified-Since and with If- None-Match headers?
[twitter-dev] Re: The XML for user settings would be helpful
How can it be counted if no api key is used? Do you mean its counted against the ip address? On Feb 19, 12:06 pm, alexro arodyg...@gmail.com wrote: Dmitri, I believe such request still counts against your usage limit. Just to remember to stay within the boundaries :) On Feb 18, 10:15 pm, Dmitri Snytkine d.snytk...@gmail.com wrote: Sorry to bother you, but I found out that this feature is already available Turns out I can easily get user's profile as json or xml without using oAuth or API Very simple, like this: http://api.twitter.com/1/users/show/MythBusters.json This is just great! On Feb 18, 3:36 pm, Dmitri Snytkine d.snytk...@gmail.com wrote: I just though of something that would be very helpful to developers: what if there was a url to get xml or json of user's profile, background image, color settings and avatar. I mean similar to regular RSS feed, only for the current user's settings. This way we don't even need to use API if we want to generate a page that looks like user's own twitter page. And because it would be static files, they could be served from Twitter very fast and make use last-modified and etag headers. Currently if I want to style a page to mimin user's twitter page, I have to access thehttps://twitter.com/account/verify_credentials.json and for that I have to use oAuth call. But this is an overkill. Why do I even need to have user's token and secret just to get his latest profile that is basically available on his twitter page, I just don't want to to and scrape it from the actual twitter page. Why not give us the url to get these settings as json or xml the same way we can get the RSS for user's latest messages without having to use API
[twitter-dev] Re: Introduce yourself!
Hello all, My name is Andrew Seigner. I created http://heypic.me to display tweets on a Google Map, with iPhone integration and a public API supporting XML and KML. The site uses python-twitter and oauth-python-twitter. I contributed minor changes to each, adding Geo and Trends support to python-twitter and a bug fix to oauth-python-twitter: http://code.google.com/p/python-twitter/ http://code.google.com/p/oauth-python-twitter/ The feature I would most like to see was actually fulfilled a few days ago with the release of the mobile OAuth login page, go team! The next feature I'd like to see is delegated OAuth, so the iPhone app could post photos to TwitPic or YFrog, rather than just using ImageShack. Nice to meet everyone, Andrew http://twitter.com/siggy_sf http://heypic.me/user/siggy_sf
Re: [twitter-dev] Limiting API Response
Can you even run TCP or a JSON parser in 2k of RAM? In any case, I think a proxy server is going to be your best bet. -John Typos by iPhone. On Feb 20, 2010, at 10:54 AM, Matt23 mrichardso...@gmail.com wrote: Hello, I am developing a twitter client that runs on an embedded microprocessor (Arduino) which has only 2K of RAM. The problem is that Twitter's API response (even if only for a single @ mention) is too large to store to a string with only 2K RAM. Is there a way I can make my API call and limit the response to certain pertinent return values? Such as only screen name, text and time? This would greatly cutdown on the memory needed to store the API response. Thank you, Matt23
Re: [twitter-dev] Limiting API Response
Can you even run TCP or a JSON parser in 2k of RAM? In any case, I think a proxy server is going to be your best bet. You would be surprised: http://www.sics.se/~adam/uip/index.php/Main_Page though JSON would seem to be a bit of a stretch. -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- Today's forecast is total crap! -- Strong Bad, Homestar Runner Menu #11 --
Re: [twitter-dev] Introduce yourself!
Marc Mims. @semifor. Author and maintainer of Net::Twitter [1], the Perl interface to the Twitter API. I'm a freelance software developer specializing in modern perl (Moose, Catalyst, DBIx::Class, and Net::Twitter, of course). I'm a Linux enthusiast and run Debian on my personal systems with xmonad, Vim, Firefox+Vimperator. Give me text, and take the mouse with you. :) My first Twitter app was Twirc [2], a local IRC gateway for Twitter. It's the Twitter client I use. Chris Thompson wrote and maintained Net::Twitter through version 2.12. I wrote replacement, from scratch, and Chris handed off the project to me. So I authored Net::Twitter 3.0 and maintain it, currently. Net::Twitter and the community of users and applications they've built have been a joy to work with. I'm registered for Chirp and I'm doing a little fund raising from the Net::Twitter users to help cover the costs of attending [3]. In my spare time, I'm working on packrati.us (@packratius) [4], a Twitter + Delicious mashup. It started as a simple learning project for OAuth and the Streaming API. I'm really pleased with the backend code, which is where most of the effort has gone. If I can shake loose some more time, I need to do plenty of front end work (a designer I am not) and add some features to make it useful to a wider audience. I'll 2nd @Abraham's feature request for a conversation method returning replies for a specified status. I'd also like to see some consistency in error handling. Getting Not authorized for user_timeline can mean (1) the account is protected and the authenticated user is not authorized, (2) the account is suspended, (3) the account has 0 tweets. Getting a 500 response with HTML content from the Search API is unhelpful; getting an error response in the requested format would be much saner. There are many other inconsistencies in error responses. Life would be easier if they were addressed. Looking forward to Chirp and meeting some other Twitter devs there. @semifor [1] http://search.cpan.org/dist/Net-Twitter [2] http://search.cpan.org/dist/POE-Component-Server-Twirc/ [3] http://semifor.posterous.com/how-1000-became-0-and-how-it-can-be-fixed-for [4] http://packrati.us
Re: [twitter-dev] Introduce yourself!
I'm Neal Rauhauser, @StrandedWind. Iowa State software engineering back when - dodged a punchcard programming class by one semester. Cisco Certified Network Design Professional for a decade, recently lapsed as the cert is devalued at both resellers and in general due to their failure to protect the brand. I pay the bills operating a rural triple play carrier. I'm a founding member of the Blog Workers Industrial Union, which came together at the end of Netroots Nation 2009. We've spawned Progressive PST, a social media consulting operation for Progressive Democratic candidates causes. PeanutButterPAC is another offshoot that I'm only tangentially involved with, and we're chipping away at a workflow management system that is Twitter enabled. We very purposefully don't have a web site for the tools we build. Some of the BWIU are quite activist and they draw attention to themselves - we don't want to bring that kind of heat down on our hosting. Any complex data is prepared for the back end in Google Docs and access is triggered by commands in direct messages to controlled Twitter accounts. The security model is implemented using private lists. Results and logs return from the gmail accounts associated with a particular set of Twitter IDs. Those operating the systems have no idea where the backend servers are physically located. Twitter tolerates us having two whitelisted IPs, only one of which is active at any given moment. The systems are quite geographically diverse and we've been doing a good bit of cross training. We've got low frequency, high value automated message placement - think public service announcements for political campaigns and such. There's a one to many direct message utility that permits the receivers to go onduty/offduty with a single message, no matter how many tasks groups they've joined. Applications accessible by API can be triggered remotely by non-technical users and they receive reports via email. None of this stuff is particularly complex - any elegant looking code we might have is due to Net:Twitter and the help Marc Mims has provided. I am the resident programmer but you wouldn't have to follow me very long to learn that I'm wrestling with #Lyme. I'd really like to find some more stuff like Marc's Net:Twtter module - things that are simple to use, things based on perl, thing that behave if I want to put them in a chain of unix tools running in the background. Oh, and we've got a couple of million plus users organizations that would like us to do certain things, and I could really use an Oauth app but nothing off the shelf is going to fit my needs ... On Sat, Feb 20, 2010 at 2:45 PM, Marc Mims marc.m...@gmail.com wrote: Marc Mims. @semifor. Author and maintainer of Net::Twitter [1], the Perl interface to the Twitter API. I'm a freelance software developer specializing in modern perl (Moose, Catalyst, DBIx::Class, and Net::Twitter, of course). I'm a Linux enthusiast and run Debian on my personal systems with xmonad, Vim, Firefox+Vimperator. Give me text, and take the mouse with you. :) My first Twitter app was Twirc [2], a local IRC gateway for Twitter. It's the Twitter client I use. Chris Thompson wrote and maintained Net::Twitter through version 2.12. I wrote replacement, from scratch, and Chris handed off the project to me. So I authored Net::Twitter 3.0 and maintain it, currently. Net::Twitter and the community of users and applications they've built have been a joy to work with. I'm registered for Chirp and I'm doing a little fund raising from the Net::Twitter users to help cover the costs of attending [3]. In my spare time, I'm working on packrati.us (@packratius) [4], a Twitter + Delicious mashup. It started as a simple learning project for OAuth and the Streaming API. I'm really pleased with the backend code, which is where most of the effort has gone. If I can shake loose some more time, I need to do plenty of front end work (a designer I am not) and add some features to make it useful to a wider audience. I'll 2nd @Abraham's feature request for a conversation method returning replies for a specified status. I'd also like to see some consistency in error handling. Getting Not authorized for user_timeline can mean (1) the account is protected and the authenticated user is not authorized, (2) the account is suspended, (3) the account has 0 tweets. Getting a 500 response with HTML content from the Search API is unhelpful; getting an error response in the requested format would be much saner. There are many other inconsistencies in error responses. Life would be easier if they were addressed. Looking forward to Chirp and meeting some other Twitter devs there. @semifor [1] http://search.cpan.org/dist/Net-Twitter [2] http://search.cpan.org/dist/POE-Component-Server-Twirc/ [3] http://semifor.posterous.com/how-1000-became-0-and-how-it-can-be-fixed-for [4] http://packrati.us --
[twitter-dev] Re: Introduce yourself!
I'm @unfollowr developer who getting too tired about not whitelisting DM sending. I've even deleted my own twitter account already On Feb 19, 10:20 pm, Abraham Williams 4bra...@gmail.com wrote: We have not had an introductions thread in a long time (or ever that I could find) so I'm starting one. Don't forget to add an answer to the tools thread [1](Gmail link [2]) as well. I'm Abraham Williams, I've been working with the Twitter API and this group since early 2008. I do mostly freelance Drupal and Twitter API integration and personal projects. I love seeing the creative projects developers build or integrate with the API and look forward to meeting many of you at Chirp. TwitterOAuth [3] the first PHP library to support OAuth is built and maintained by me, and will hopefully see a new release soon. I also built a fun Chrome extension [4] that integrates common friends and followers into Twitter profiles. The feature I would most like added to the API is a conversation method to get replies to a specific status. So. Who are you, what do you do, what have you built, and what feature do you most want to see added? @Abraham [1]http://groups.google.com/group/twitter-development-talk/browse_thread... [2]https://mail.google.com/mail/#inbox/12680cd0fa59011e [3]https://chrome.google.com/extensions/detail/npdjhmblakdjfnnajeomfbogo... [4]http://code.google.com/p/twitter-api/issues/detail?id=142 -- Abraham Williams | Community Advocate |http://abrah.am Project | Out Loud |http://outloud.labs.poseurtech.com This email is: [ ] shareable [x] ask first [ ] private. Sent from Seattle, WA, United States
[twitter-dev] Suspension whilst testing Oauth...
Hi Guys I am toward the end of writing an AIR application using FLEX. I have bought signature certificates, I have the site URL etc. I have tried to add the application to my own twitter account and it has been suspended. The only task of the application right now is to test the Oauth process, which it passed and then would not tweet. Then when I looked a bit deeper I found the application was suspended. I emailed Twitter and have been given a ticket number and although the ticket does not seem to exist I have had some really fast replies - thank you. It seems that your team is making judgements on me because of my company name, which is historic, not what the application does. Can someone point me in the right direction. Have I taken it down the wrong development route here? I started hand coding original microprocessors back in the 80s. I have written a lot of software over the years. I consider myself competent but fallible to bloomers from time to time!
Re: [twitter-dev] Introduce yourself!
I maintain TTYtter, a Perl Twitter command-line client and application platform, and act as one of the list moderators. My day job has nothing to do with either one of those roles. :) http://www.floodgap.com/software/ttytter/ -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- Remember, kids: for great justice take off every zig! --
Re: [twitter-dev] Introduce yourself!
Hi, I'm Scott Wilcox (@dordotky). I'm a freelance developer and currently run and maintain the http://tweekly.fm and http://laststat.us services. I developer mostly in PHP over the majority of my projects but plan to switch to either Ruby or Python this year. I'm also an iPhone developer and plan to release a few apps this year. I use both the REST API and Streaming API regularly and agree with the comments on standardising the errors across the platform (the user_timeline as mention by Marc is a particular pet hate). I've also been doing some research in to awareless of embedded EXIF data in images that are posted to Twitter via services such as twitpic.com. I'll be publishing these finds towards the end of the month. I sadly won't be attending Chirp due to it being too far to travel from England and not enough funds to do so :( Hopefully one of you will create a webcast for me to watch! Scott. smime.p7s Description: S/MIME cryptographic signature
[twitter-dev] Re: Introduce yourself!
Hello folks, I'm Daniel, I started the TweetSharp project (http://tweetsharp.com) which I work on with Jason. If you work with .NET, you probably want to use TweetSharp. We're happily supporting users like Seesmic, TidyTweet, Sobees, and beginners alike so they can build great Twitter applications. Our API has grown to include fluent and service-based support for 100% of the Twitter API, and most recently Yammer as well. We're going to do our best to make it to Chirp this year. @dimebrain and @jdiller On Feb 19, 3:20 pm, Abraham Williams 4bra...@gmail.com wrote: We have not had an introductions thread in a long time (or ever that I could find) so I'm starting one. Don't forget to add an answer to the tools thread [1](Gmail link [2]) as well. I'm Abraham Williams, I've been working with the Twitter API and this group since early 2008. I do mostly freelance Drupal and Twitter API integration and personal projects. I love seeing the creative projects developers build or integrate with the API and look forward to meeting many of you at Chirp. TwitterOAuth [3] the first PHP library to support OAuth is built and maintained by me, and will hopefully see a new release soon. I also built a fun Chrome extension [4] that integrates common friends and followers into Twitter profiles. The feature I would most like added to the API is a conversation method to get replies to a specific status. So. Who are you, what do you do, what have you built, and what feature do you most want to see added? @Abraham [1]http://groups.google.com/group/twitter-development-talk/browse_thread... [2]https://mail.google.com/mail/#inbox/12680cd0fa59011e [3]https://chrome.google.com/extensions/detail/npdjhmblakdjfnnajeomfbogo... [4]http://code.google.com/p/twitter-api/issues/detail?id=142 -- Abraham Williams | Community Advocate |http://abrah.am Project | Out Loud |http://outloud.labs.poseurtech.com This email is: [ ] shareable [x] ask first [ ] private. Sent from Seattle, WA, United States
Re: [twitter-dev] Introduce yourself!
I'm Marco Kaiser (@marco), started playing with the API in Summer 2007 and developed AIR-based twhirl back then. It was acquired by Seesmic almost two years ago now, and I joined the company, too. Did a couple more Twitter desktop apps since then... :) I am based in Germany, and I also act as a moderator on this list. I'll be at Chirp. Cheers, Marco On Sat, Feb 20, 2010 at 10:17 PM, Scott Wilcox sc...@tig.gr wrote: Hi, I'm Scott Wilcox (@dordotky). I'm a freelance developer and currently run and maintain the http://tweekly.fm and http://laststat.us services. I developer mostly in PHP over the majority of my projects but plan to switch to either Ruby or Python this year. I'm also an iPhone developer and plan to release a few apps this year. I use both the REST API and Streaming API regularly and agree with the comments on standardising the errors across the platform (the user_timeline as mention by Marc is a particular pet hate). I've also been doing some research in to awareless of embedded EXIF data in images that are posted to Twitter via services such as twitpic.com. I'll be publishing these finds towards the end of the month. I sadly won't be attending Chirp due to it being too far to travel from England and not enough funds to do so :( Hopefully one of you will create a webcast for me to watch! Scott.
[twitter-dev] Where is the group for end user ?
I can only find this development group, I cannot find any group for end user, so I post my question here. It's now two weeks I have created a twitter account at http://twitter.com/helenecambodge and when I search cambodge I cannot find my tweets ! Thanks.
[twitter-dev] Re: Streaming API question
Sorry, John but this is really happening and I am having it on a daily basis in the last 2 weeks on both dev machine and production. I am using Apache httpClient and this code was working fine for months before this started to happen 2 weeks ago. I have a java call String line =reader.readLine(); and this call at least once a day just does not return (100% sure since I am logging all the call chain in debug mode). So please take a look at whatever data you are monitoring to see if anything can explain this. In the meantime, I am building a monitoring thread to force a restart on the connection which is really saddening Thanks On Feb 19, 2:55 pm, John Kalucki j...@twitter.com wrote: This shouldn't be happening, and having developers build these sorts of workarounds saddens me. It is possible that the server side is holding dead connections open, but I doubt it -- as I've a considerable amount of data to the contrary. I suspect that the socket code does not detect a close, driven by either a TCP Close or a TCP Reset. I've run connections over the public internet with close monitoring and rarely noticed a timeout. If you point the same client at a file of streaming data on a web server, does the client detect the end of file at the correct point? -John Kaluckihttp://twitter.com/jkalucki Infrastructure, Twitter Inc. On Fri, Feb 19, 2010 at 9:16 AM, Dima Brodsky ddbrod...@gmail.com wrote: Hi, Yup, I saw it the last couple of weeks, this week has been considerably better. I use the delimited stream so I do read(entry size), read(entry), repeat ... I just put a 30 second timeout on the read operation (this is all in python) and if a read fails I close the stream and reconnect. This seems to work quite well ... depending on the chunk size you are reading you could probably lower the timeout if you are afraid of losing data. ttyl Dima On 19-Feb-10, at 7:36 AM, rob wrote: Has anyone else ran into an issue where over time the Streaming API just stops sending results? We are using a Ruby library to connect (twitter-stream) which uses EventMachine to open a persistent connection to the API (we are tracking and following). The library properly handles reconnection (from dropped connections) and the various error conditions. All works well for a period of time (8+ hours in some cases, sometimes a full day) after that the connection does no get dropped but no data gets sent. (At least that's what is seems as EventMachine feels its still connected) If we just drop the connection and reconnect all is well and the data starts to flow again. (Which we could do but that seems like a hack) Anyone else have this issue or should I dive into the EventMachine code and see if there is an issue in there? Thanks in advance, Rob -- ddbrod...@gmail.com The price of reliability is the pursuit of the utmost simplicity. It is a price which the very rich find the most hard to pay. (Sir Antony Hoare, 1980)
Re: [twitter-dev] Re: Streaming API question
I don't doubt that this is happening, but, whenever I've looked into this in the past, the issue boiled down to a poorly written client that didn't detect a TCP Close or TCP Reset when the server, for whatever reason, ended the connection. However, it appears that something different is happening now. There are reports across multiple client library of this behavior occurring -- low velocity streams where the server and the client think that the connection is open, but no data is read by the client. Now these reports are coincident in time, and do not correlate with server maintenance. This morning, I drilled down with the Phirehose PHP client people on their mailing list. ( http://groups.google.com/group/phirehose-users/browse_frm/thread/6a29c7b9f50d0368) They've provided some good data so far. After I do a little more experimentation, I'll start a fresh thread on this list to address this issue. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Sat, Feb 20, 2010 at 6:06 PM, Sami sami.ben.romdh...@gmail.com wrote: Sorry, John but this is really happening and I am having it on a daily basis in the last 2 weeks on both dev machine and production. I am using Apache httpClient and this code was working fine for months before this started to happen 2 weeks ago. I have a java call String line =reader.readLine(); and this call at least once a day just does not return (100% sure since I am logging all the call chain in debug mode). So please take a look at whatever data you are monitoring to see if anything can explain this. In the meantime, I am building a monitoring thread to force a restart on the connection which is really saddening Thanks On Feb 19, 2:55 pm, John Kalucki j...@twitter.com wrote: This shouldn't be happening, and having developers build these sorts of workarounds saddens me. It is possible that the server side is holding dead connections open, but I doubt it -- as I've a considerable amount of data to the contrary. I suspect that the socket code does not detect a close, driven by either a TCP Close or a TCP Reset. I've run connections over the public internet with close monitoring and rarely noticed a timeout. If you point the same client at a file of streaming data on a web server, does the client detect the end of file at the correct point? -John Kaluckihttp://twitter.com/jkalucki Infrastructure, Twitter Inc. On Fri, Feb 19, 2010 at 9:16 AM, Dima Brodsky ddbrod...@gmail.com wrote: Hi, Yup, I saw it the last couple of weeks, this week has been considerably better. I use the delimited stream so I do read(entry size), read(entry), repeat ... I just put a 30 second timeout on the read operation (this is all in python) and if a read fails I close the stream and reconnect. This seems to work quite well ... depending on the chunk size you are reading you could probably lower the timeout if you are afraid of losing data. ttyl Dima On 19-Feb-10, at 7:36 AM, rob wrote: Has anyone else ran into an issue where over time the Streaming API just stops sending results? We are using a Ruby library to connect (twitter-stream) which uses EventMachine to open a persistent connection to the API (we are tracking and following). The library properly handles reconnection (from dropped connections) and the various error conditions. All works well for a period of time (8+ hours in some cases, sometimes a full day) after that the connection does no get dropped but no data gets sent. (At least that's what is seems as EventMachine feels its still connected) If we just drop the connection and reconnect all is well and the data starts to flow again. (Which we could do but that seems like a hack) Anyone else have this issue or should I dive into the EventMachine code and see if there is an issue in there? Thanks in advance, Rob -- ddbrod...@gmail.com The price of reliability is the pursuit of the utmost simplicity. It is a price which the very rich find the most hard to pay. (Sir Antony Hoare, 1980)
Re: [twitter-dev] Streaming API question
* rob robert.bag...@gmail.com [100219 08:56]: Has anyone else ran into an issue where over time the Streaming API just stops sending results? Yes. I'm seeing the same thing. I've set up a 45 second timeout. The following entries were extracted from the application log. I'm currently following 100 users, so periods of inactivity are not unusual. Receipt of keep alive packets are indicated by ping. If there's no activity for 45 seconds, I drop the connection and reconnect. Times are PST. @semifor 2010/02/09 14:39:44 connecting 2010/02/09 14:40:29 timeout 2010/02/09 14:40:29 Waiting 0 seconds before reconnecting... -- 2010/02/09 16:09:10 ping 2010/02/09 16:09:40 ping 2010/02/09 16:10:25 timeout 2010/02/09 16:10:25 Waiting 0 seconds before reconnecting... 2010/02/09 16:10:25 connecting 2010/02/09 16:11:10 timeout 2010/02/09 16:11:10 Waiting 0 seconds before reconnecting... -- 2010/02/11 08:54:04 8968466186: @BarackObama. Gd morning MR President. OMG u twi 2010/02/11 08:54:49 timeout 2010/02/11 08:54:49 Waiting 0 seconds before reconnecting... 2010/02/11 08:54:49 connecting 2010/02/11 08:55:34 timeout 2010/02/11 08:55:34 Waiting 0 seconds before reconnecting... -- 2010/02/12 08:50:40 ping 2010/02/12 08:51:10 ping 2010/02/12 08:51:55 timeout 2010/02/12 08:51:55 Waiting 0 seconds before reconnecting... -- 2010/02/12 11:45:05 ping 2010/02/12 11:45:35 ping 2010/02/12 11:46:20 timeout 2010/02/12 11:46:20 Waiting 0 seconds before reconnecting... 2010/02/12 11:46:20 connecting 2010/02/12 11:47:05 timeout 2010/02/12 11:47:05 Waiting 0 seconds before reconnecting... -- 2010/02/16 07:11:25 9188373420: @alexpriest oh Alex, you're a fool. Haha. :D Dri 2010/02/16 07:11:55 ping 2010/02/16 07:12:40 timeout 2010/02/16 07:12:40 Waiting 0 seconds before reconnecting... -- 2010/02/16 08:38:49 Waiting 0 seconds before reconnecting... 2010/02/16 08:38:49 connecting 2010/02/16 08:39:34 timeout 2010/02/16 08:39:34 Waiting 0 seconds before reconnecting... -- 2010/02/17 07:28:03 9238057534: @BarackObama President B. Pls explain this to us 2010/02/17 07:28:48 timeout 2010/02/17 07:28:48 Waiting 0 seconds before reconnecting... -- 2010/02/18 09:24:21 9292626812: @SuzieLin no problem! So far can't complain, jus 2010/02/18 09:24:50 9292645527: @BarackObama Another terrorism actack? How many 2010/02/18 09:25:35 timeout 2010/02/18 09:25:35 Waiting 0 seconds before reconnecting... -- 2010/02/19 09:36:16 9344797252: @BarackObama We The People want the #PNHP at 2010/02/19 09:37:01 timeout 2010/02/19 09:37:01 Waiting 0 seconds before reconnecting... 2010/02/19 09:37:01 connecting 2010/02/19 09:37:46 timeout 2010/02/19 09:37:46 Waiting 0 seconds before reconnecting...
Re: [twitter-dev] Streaming API question
A 45 second period of inactivity is not unusual when following just 100, or even 100,000 users. The keep-alive newlines are only sent once every 10 minutes. You should not reconnect so aggressively. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Sat, Feb 20, 2010 at 7:59 PM, Marc Mims marc.m...@gmail.com wrote: * rob robert.bag...@gmail.com [100219 08:56]: Has anyone else ran into an issue where over time the Streaming API just stops sending results? Yes. I'm seeing the same thing. I've set up a 45 second timeout. The following entries were extracted from the application log. I'm currently following 100 users, so periods of inactivity are not unusual. Receipt of keep alive packets are indicated by ping. If there's no activity for 45 seconds, I drop the connection and reconnect. Times are PST. @semifor 2010/02/09 14:39:44 connecting 2010/02/09 14:40:29 timeout 2010/02/09 14:40:29 Waiting 0 seconds before reconnecting... -- 2010/02/09 16:09:10 ping 2010/02/09 16:09:40 ping 2010/02/09 16:10:25 timeout 2010/02/09 16:10:25 Waiting 0 seconds before reconnecting... 2010/02/09 16:10:25 connecting 2010/02/09 16:11:10 timeout 2010/02/09 16:11:10 Waiting 0 seconds before reconnecting... -- 2010/02/11 08:54:04 8968466186: @BarackObama. Gd morning MR President. OMG u twi 2010/02/11 08:54:49 timeout 2010/02/11 08:54:49 Waiting 0 seconds before reconnecting... 2010/02/11 08:54:49 connecting 2010/02/11 08:55:34 timeout 2010/02/11 08:55:34 Waiting 0 seconds before reconnecting... -- 2010/02/12 08:50:40 ping 2010/02/12 08:51:10 ping 2010/02/12 08:51:55 timeout 2010/02/12 08:51:55 Waiting 0 seconds before reconnecting... -- 2010/02/12 11:45:05 ping 2010/02/12 11:45:35 ping 2010/02/12 11:46:20 timeout 2010/02/12 11:46:20 Waiting 0 seconds before reconnecting... 2010/02/12 11:46:20 connecting 2010/02/12 11:47:05 timeout 2010/02/12 11:47:05 Waiting 0 seconds before reconnecting... -- 2010/02/16 07:11:25 9188373420: @alexpriest oh Alex, you're a fool. Haha. :D Dri 2010/02/16 07:11:55 ping 2010/02/16 07:12:40 timeout 2010/02/16 07:12:40 Waiting 0 seconds before reconnecting... -- 2010/02/16 08:38:49 Waiting 0 seconds before reconnecting... 2010/02/16 08:38:49 connecting 2010/02/16 08:39:34 timeout 2010/02/16 08:39:34 Waiting 0 seconds before reconnecting... -- 2010/02/17 07:28:03 9238057534: @BarackObama President B. Pls explain this to us 2010/02/17 07:28:48 timeout 2010/02/17 07:28:48 Waiting 0 seconds before reconnecting... -- 2010/02/18 09:24:21 9292626812: @SuzieLin no problem! So far can't complain, jus 2010/02/18 09:24:50 9292645527: @BarackObama Another terrorism actack? How many 2010/02/18 09:25:35 timeout 2010/02/18 09:25:35 Waiting 0 seconds before reconnecting... -- 2010/02/19 09:36:16 9344797252: @BarackObama We The People want the #PNHP at 2010/02/19 09:37:01 timeout 2010/02/19 09:37:01 Waiting 0 seconds before reconnecting... 2010/02/19 09:37:01 connecting 2010/02/19 09:37:46 timeout 2010/02/19 09:37:46 Waiting 0 seconds before reconnecting...
Re: [twitter-dev] Streaming API question
* John Kalucki j...@twitter.com [100220 20:24]: A 45 second period of inactivity is not unusual when following just 100, or even 100,000 users. The keep-alive newlines are only sent once every 10 minutes. You should not reconnect so aggressively. I can certainly set the time out to 10 minutes. I'm seeing newlines in the stream every 30 seconds, except for rare occasions. I understood those to be keep-alive packets. Apparently they are not and should not be relied on? @semifor
Re: [twitter-dev] Streaming API question
Arg. This is what I get for not checking the configuration each time. Yes, it's currently set to send a newline every 30 seconds. On Sat, Feb 20, 2010 at 8:45 PM, Marc Mims marc.m...@gmail.com wrote: * John Kalucki j...@twitter.com [100220 20:24]: A 45 second period of inactivity is not unusual when following just 100, or even 100,000 users. The keep-alive newlines are only sent once every 10 minutes. You should not reconnect so aggressively. I can certainly set the time out to 10 minutes. I'm seeing newlines in the stream every 30 seconds, except for rare occasions. I understood those to be keep-alive packets. Apparently they are not and should not be relied on? @semifor
[twitter-dev] Re: Streaming API question
I was looking at Gist's Java code the other day (http://gistinc.github.com/TwitterClient/) and they had a similar coding - they were seeing newlines every 30 seconds so they set the timeout to one minute. // HttpClient has no way to set SO_KEEPALIVE on our // socket, and even if it did the TCP keepalive interval // may be too long, so we need to set a timeout at this // level. Twitter will send periodic newlines for // keepalive if there is no traffic, but they don't say // how often. Looking at the stream, it's every 30 // seconds, so we use a read timeout of twice that. -- M. Edward (Ed) Borasky borasky-research.net/m-edward-ed-borasky/ A mathematician is a device for turning coffee into theorems. ~ Paul Erdos On Feb 20, 8:45 pm, Marc Mims marc.m...@gmail.com wrote: * John Kalucki j...@twitter.com [100220 20:24]: A 45 second period of inactivity is not unusual when following just 100, or even 100,000 users. The keep-alive newlines are only sent once every 10 minutes. You should not reconnect so aggressively. I can certainly set the time out to 10 minutes. I'm seeing newlines in the stream every 30 seconds, except for rare occasions. I understood those to be keep-alive packets. Apparently they are not and should not be relied on? @semifor
Re: [twitter-dev] Streaming API question
* John Kalucki j...@twitter.com [100220 21:02]: Arg. This is what I get for not checking the configuration each time. Yes, it's currently set to send a newline every 30 seconds. Ok. Sorry to drag this out, but what, then, is an appropriate timeout value for the application? @semifor
[twitter-dev] Re: Streaming API question
I've got the tweetstream Ruby gem installed and I have a test driver program. I can fire this up if it will give anything useful. Is this happening just on filter or would it happen on sample too? On Feb 20, 9:02 pm, John Kalucki j...@twitter.com wrote: Arg. This is what I get for not checking the configuration each time. Yes, it's currently set to send a newline every 30 seconds. On Sat, Feb 20, 2010 at 8:45 PM, Marc Mims marc.m...@gmail.com wrote: * John Kalucki j...@twitter.com [100220 20:24]: A 45 second period of inactivity is not unusual when following just 100, or even 100,000 users. The keep-alive newlines are only sent once every 10 minutes. You should not reconnect so aggressively. I can certainly set the time out to 10 minutes. I'm seeing newlines in the stream every 30 seconds, except for rare occasions. I understood those to be keep-alive packets. Apparently they are not and should not be relied on? @semifor
Re: [twitter-dev] Re: Streaming API question
I have a hunch that this doesn't happen on sample, or, if it does so, it happens much more rarely. On Sat, Feb 20, 2010 at 9:26 PM, M. Edward (Ed) Borasky zzn...@gmail.comwrote: I've got the tweetstream Ruby gem installed and I have a test driver program. I can fire this up if it will give anything useful. Is this happening just on filter or would it happen on sample too? On Feb 20, 9:02 pm, John Kalucki j...@twitter.com wrote: Arg. This is what I get for not checking the configuration each time. Yes, it's currently set to send a newline every 30 seconds. On Sat, Feb 20, 2010 at 8:45 PM, Marc Mims marc.m...@gmail.com wrote: * John Kalucki j...@twitter.com [100220 20:24]: A 45 second period of inactivity is not unusual when following just 100, or even 100,000 users. The keep-alive newlines are only sent once every 10 minutes. You should not reconnect so aggressively. I can certainly set the time out to 10 minutes. I'm seeing newlines in the stream every 30 seconds, except for rare occasions. I understood those to be keep-alive packets. Apparently they are not and should not be relied on? @semifor
Re: [twitter-dev] Streaming API question
60 or 90 seconds seems reasonable, but your code should also detect a socket close immediately and reconnect immediately. The common case for a connection drop -- a server restart -- should cause your socket to close, the client to detect the closure, and reconnect, all within about a second. On Sat, Feb 20, 2010 at 9:09 PM, Marc Mims marc.m...@gmail.com wrote: * John Kalucki j...@twitter.com [100220 21:02]: Arg. This is what I get for not checking the configuration each time. Yes, it's currently set to send a newline every 30 seconds. Ok. Sorry to drag this out, but what, then, is an appropriate timeout value for the application? @semifor
[twitter-dev] Re: Streaming API question
I've got my tweetstream test started - filter keyword is haiti - it's delivering tweets about 2 - 5 per minute at the moment. On Feb 20, 10:16 pm, John Kalucki j...@twitter.com wrote: I have a hunch that this doesn't happen on sample, or, if it does so, it happens much more rarely. On Sat, Feb 20, 2010 at 9:26 PM, M. Edward (Ed) Borasky zzn...@gmail.comwrote: I've got the tweetstream Ruby gem installed and I have a test driver program. I can fire this up if it will give anything useful. Is this happening just on filter or would it happen on sample too? On Feb 20, 9:02 pm, John Kalucki j...@twitter.com wrote: Arg. This is what I get for not checking the configuration each time. Yes, it's currently set to send a newline every 30 seconds. On Sat, Feb 20, 2010 at 8:45 PM, Marc Mims marc.m...@gmail.com wrote: * John Kalucki j...@twitter.com [100220 20:24]: A 45 second period of inactivity is not unusual when following just 100, or even 100,000 users. The keep-alive newlines are only sent once every 10 minutes. You should not reconnect so aggressively. I can certainly set the time out to 10 minutes. I'm seeing newlines in the stream every 30 seconds, except for rare occasions. I understood those to be keep-alive packets. Apparently they are not and should not be relied on? @semifor
