[twitter-dev] Re: Streaming API + PHP and Python
Have you resolved this problem? suggestion: did you try writing the raw output to a file (like every hour, and then create another file, and so on), and then have another script process the JSON? On Jun 8, 1:16 pm, Chad Etzel jazzyc...@gmail.com wrote: Hi All, I am stumped. For several days I have tried to write a simple PHP script that can interact with the Streaming API by just pulling in the stream and echoing out the contents. This is the script I have: http://pastie.org/private/9owdxerouwhitz5nfacrw Right now it just pulls in the feed and echos it. I am not parsing anything at the moment. This works great for a while, then the fread will start timing out every 60 seconds (I have set the stream_timeout to 60). It will do this after an undeterministic number of updates or number of bytes received. netstat shows I am still connected to stream.twitter.com but Wireshark shows that no new data is arriving. I have tried this on 3 different machines (2 behind the same NAT/firewall, and 1 remote server) all with the same results. I even scraped together a simple python script which should do the same thing here: http://pastie.org/private/k0p5286ljlhdyurlagnq Same results works for a while, then it stops. Strangely, if I use CURL or telnet to open a raw socket to /spritzer or /gardenhose it stays up forever. I had a telnet socket open on /spritzer all weekend with no disconnects... In the PHP script, if I add code to detect the time-outs and immediately disconnect the socket and reconnect, the updates start flowing in again... This is nice for error checking, but I'd really like to figure out a more robust solution. 1) Can anyone find anything wrong with the scripts I've posted? 2) Does anyone have an example PHP script they are using to connect to the Streaming API which stays up indefinitely? I would like to thank John K at Twitter for helping me debug thus far. Thanks, -Chad
[twitter-dev] Re: API limit confusion
Isn't this what I said? -- Hwee-Boon On Jul 24, 2:36 pm, srikanth reddy srikanth.yara...@gmail.com wrote: @jim.renkel. Thanks a ton. I think now it is clear. It appears to me that each user of a white-listed site gets 20k requests per hour, independent of any other users of that site or *any other uses of the twitter API at other sites by that user * probably this is what they mean by *IP whitelisting takes precedence to account rate limits*. *GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users*. If the IP limit is for the consumer then it will lead to denial of service attacks. This is how we wanted it to work. Srikanth On Fri, Jul 24, 2009 at 12:52 AM, jim.renkel james.ren...@gmail.com wrote: My experience with this is, I think, a little bit different than what you describe. It appears to me that each user of a white-listed site gets 20k requests per hour, independent of any other users of that site or any other uses of the twitter API at other sites by that user. I didn't think this was what twitter intended and reported it as a bug (See:http://code.google.com/p/twitter-api/issues/detail?id=617), but the twitter folk said Yup, working as intended. After you log in athttp://twxlate.com, the site reports rate limit information on every page view, so you can see how this works there. Comments expected and welcome. Jim Renkel On Jul 23, 3:48 am, jmathai jmat...@gmail.com wrote: In other words, you have a web app running on a single server with a single IP. You make authenticated requests using each user's account. If your IP is whitelisted, the calls go towards your 20k limit, if it is not whitelisted, it goes against the current 150 limit for the respective accounts. That's what it means by IP whitelisting takes precedence to account rate limits. I don't believe that is true. If your web app is running on a whitelisted IP then you get up to 20k GET calls per hour. POST requests (status or DM) are counted against the user being authenticated. You CANNOT retrieve a user's rate limit status.
[twitter-dev] Re: API Curl: Status update result: http_code =0!
Thx worked On 20 Jul., 20:27, Chad Etzel jazzyc...@gmail.com wrote: Ah, I think I may have an idea... You both are setting the status in the query string (more like a GET request) and setting curl_setopt($ch, CURLOPT_POST, 1). But, It doesn't look like either of you are setting a POSTFIELDS option. I gothttp_code=0sometime ago when I realized I was not sending a POST body along with a POST request. So, try setting curl_setopt(CURLOPT_POSTFIELDS, ); (yes, it's just an empty string, but it creates the content-length header in the request) and then see what happens. -Chad On Mon, Jul 20, 2009 at 11:23 AM, ldas...@schultschik.de wrote: nobody an idea?
[twitter-dev] Re: oauth_token and secret length
http://groups.google.com/group/oauth/browse_thread/thread/d4b7f1f9174041ca?hl=en Side note, top-posted for your pleasure simply because of the demanding, twice-accented-by-exclamation point note in your sig. I prefer a bottom quote myself, but I think it's your exclamation points that pushed me over the hump. Thanks- - Andy Badera - and...@badera.us - Google me: http://www.google.com/search?q=andrew+badera - This email is: [ ] bloggable [x] ask first [ ] private On Fri, Jul 24, 2009 at 6:59 AM, shiplu shiplu@gmail.com wrote: is there any fixed length of oauth_token and secret?? is it fixed by twitter or by oauth standard?? -- A K M Mokaddim http://talk.cmyweb.net http://twitter.com/shiplu Stop Top Posting !! বাংলিশ লেখার চাইতে বাংলা লেখা অনেক ভাল Sent from Dhaka, Bangladesh
[twitter-dev] Re: oauth_token and secret length
On Fri, Jul 24, 2009 at 6:03 PM, Andrew Baderaand...@badera.us wrote: http://groups.google.com/group/oauth/browse_thread/thread/d4b7f1f9174041ca?hl=en Thanks. But I have read that already. I thought twitter would have a max length defined. -- A K M Mokaddim http://talk.cmyweb.net http://twitter.com/shiplu Stop Top Posting !! বাংলিশ লেখার চাইতে বাংলা লেখা অনেক ভাল Sent from Dhaka, Bangladesh
[twitter-dev] Re: Updating the APIs authentication limiting policy
Why will it not apply to OAuth? OAuth is having this problem too!! Not happy! On Jul 23, 12:15 am, Doug Williams d...@twitter.com wrote: Scott,This change will only affect Basic Auth, and will not affect OAuth applications. Thanks, Doug On Tue, Jul 21, 2009 at 4:27 PM, Scott haw...@gmail.com wrote: Thanks for the update Doug. Does this still apply to OAuth apps? Also, if a user goes through an app and unsuccessfully attempts to login 15 times will that app be blocked from authenticating anybody for an hour or just that user? The previous change seemed to block the entire app from making an authentication request on anybody once the limit had been hit.
[twitter-dev] Re: Updating the APIs authentication limiting policy
What do you mean the change won't affect OAuth? My application has been suffering from this issue ever since you made the limit change. My application has the ability to use either Basic or OAuth. My twitter users get blocked with the 403 error after a few minutes of usage because they reach the 15 limit Authentication limit. But It does this wether I am using OAuth or Basic? Can you just clarify that I am understanding correctly that your rollback will fix OAuth problems too? On Jul 23, 12:15 am, Doug Williams d...@twitter.com wrote: Scott,This change will only affect Basic Auth, and will not affect OAuth applications. Thanks, Doug On Tue, Jul 21, 2009 at 4:27 PM, Scott haw...@gmail.com wrote: Thanks for the update Doug. Does this still apply to OAuth apps? Also, if a user goes through an app and unsuccessfully attempts to login 15 times will that app be blocked from authenticating anybody for an hour or just that user? The previous change seemed to block the entire app from making an authentication request on anybody once the limit had been hit.
[twitter-dev] Re: Change your avatar's Twitter
I would like to know if anyone knows an example using PHP to change image 2009/7/23 Cameron Kaiser spec...@floodgap.com: How to replace the image of the avatar through the Twitter API or any other form automatically? http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-account%C2%A0update_profile_image -- personal: http://www.cameronkaiser.com/ -- Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com -- Courage is being scared to death, and saddling up anyway. -- John Wayne -- Amplexos, Leo Baiano http://www.leobaiano.com http://www.blog.ljunior.com http://www.mcelebridades.com http://www.twitter.com/leobaiano
[twitter-dev] need help with OAuth library usage in cocoa app
Dear Twitter developers, First, I apologize if I misplace the question. I've cloned Isaiah's git repository of his AOuth implementation from http://github.com/yourhead/OAuth_ObjC_Test_App/tree/master but I experienced an issue that after the web sheet closed, there is no place that I can put the PIN retrieved from the authentication result, or anywhere in the code that I need to pass the oauth_verifier parameter along with other params. the debugger said that ther is Operation could not be completed. (NSURLErrorDomain error -1012.) Thank you in advance for any response, Cheers, Fares
[twitter-dev] Re: need help with OAuth library usage in cocoa app
My example was built right as the pin code method was invented/ implemented in the API. So my example still uses the Browser method that doesn't require a pin code. If you go to your application settings page in twitter and set your Application Type to be Browser you should be good to go. As I understand it the PIN code was invented to help clients that couldn't detect if the browser had been sent to the success callback URL. However, my example doesn't have this issue. My example embeds the browser and communicates directly with it to determine when the callback URL is sent. This technique obviates the need for the pin code. I like to think of my example as a hybrid app -- neither purely a desktop client nor really a web app -- but a little bit of both in the right places. ;-) I've considered adding the pin code, but it seemed to further complicate an already challenging UI without adding any value. If you have any other issues with the example code, please feel free to email me directly. I'd be happy to help out. Isaiah YourHead Software supp...@yourhead.com http://www.yourhead.com On Jul 24, 2009, at 12:04 AM, Fares Farhan wrote: Dear Twitter developers, First, I apologize if I misplace the question. I've cloned Isaiah's git repository of his AOuth implementation from http://github.com/yourhead/OAuth_ObjC_Test_App/tree/master but I experienced an issue that after the web sheet closed, there is no place that I can put the PIN retrieved from the authentication result, or anywhere in the code that I need to pass the oauth_verifier parameter along with other params. the debugger said that ther is Operation could not be completed. (NSURLErrorDomain error -1012.) Thank you in advance for any response, Cheers, Fares
[twitter-dev] Re: Updating the APIs authentication limiting policy
Jim raised a huge weakness with the authentication rate limiting that could essentially break third-party apps. Anybody can try to add anybody else's Twitter account to a third-party app using an invalid password. If they do that 15 times with a Twitter account, the real owner of that Twitter account, who may have added his account a long time ago with the correct password, is locked out from using that app for an hour. I believe you will absolutely have to reset / remove the lock as soon as the Twitter account uses the correct password. On Jul 22, 4:58 pm, jim.renkel james.ren...@gmail.com wrote: My concern with this proposal is that it opens up denials of service, not to twitter.com, but to associated sites such as twitpic, or my site twxlate, among others For example, Lance Armstrong is a heavy user of twitpic. It is very easy for anyone to find Lance's twitter ID (@lancearmstrong), view his status updates, and see that he is a frequent user of twitpic. Now, someone that is unhappy with Lance, say one of George Hincapie's ardent fans that really believes that Lance was a significant contributor to George not winning the maillot jeune last Sunday, could go to twitpic, fail to login as Lance the requisite number of times, and deny Lance access to twitpic. Not only celebrities would or could be subject to such denials of service. I notice that @dougw occasionally uses twitpic! :-) One solution to this problem is to add to each twitter account another private ID. By default this private ID would be equal to the existing (public) ID (If not equal to the account's public ID, it would have to be unique among all twitter IDs, both public and private.). The public ID would be used just as the existing twitter ID is now: others would use it to follow, mention, DM, etc., the user. But the user MUST use their private ID for authenticated requests through the API, and CAN also use it for non-authenticated requests. In either case, twitter would treat a request from a private ID as if it came from the corresponding public ID. Blocking the public ID because of excessive authentication failures would NOT block the associated private ID unless they were equal. Changing your public ID would also change your private ID if the two were the same before the change, i.e., they would remain the same after the change. It may seem onerous to require all users to also have a private ID, but since it defaults to be the same as their public ID, only those concerned about their service being denied would change it and subsequently use it instead of their public ID to access associated sites such as twitpic or twxlate. In fact, I think this change, though potentially large on the twitter side, could be implemented without any changes to users or associated sites, with one small, obscure exception: now, if I attempt to create a new twitter account or change the ID of an existing account, and find that the ID I want is in use, I can view that account; if this were implemented and I attempted to use a private ID that was not the same as its associated public ID, I could not view the account using the denied ID. Comments expected and welcome. Jim Renkel On Jul 21, 6:00 pm, Doug Williams d...@twitter.com wrote: Devs --A change shipped last week that limited the number of times a user could access the account/verify_credentials method [1] in a given hour. This change proved hasty and short-sighted as pointed out by the subsequent discussion [2]. We apologize to any developer that was adversely affected. Given the problems, we want to fix this in a public and transparent manner. Like most web services, we limit the number of attempts users can make to login to their accounts on Twitter.com to prevent brute force dictionary attacks. This same security is not extended to the platform and leaves accounts vulnerable to the same method of attack through the API. The change we shipped to limit user accounts to 15 calls an hour to the account/verify_credentials method [1] was intended to mitigate this risk. It was thought to limit the number of tests a potential attack could run in the hour, even in a distributed fashion. However, we only protected a single resource which still leaves all other authenticated methods exposed as a vector of attack (limited only by the API rate limit). Our thinking is now that we will limit the total number of unsuccessful attempts to access authenticated resources to 15 an hour per user per IP address. If a single IP address makes 15 attempts to access a protected resource unsuccessfully for a given user (as indicated by an HTTP 401), then the user will be locked out of authenticated resources from that IP address for 1 hour. This scheme has all of the positive effects that we need, however we want to make sure that we have thought through all of the potential problems on the developer's side before
[twitter-dev] I'd like to get on the Developer for Hire page
I believe I have to request this, cause requesting a develoeprs account with the raised limits and whitelisted IP didn't get me into that list. I'm a Twitter developer and I'd like to be listed there together with my recent Twitter app called Foller.me (http:// foller.me) Thank you.
[twitter-dev] Re: Update multiple users at once
You can use curl_multi_* to make multiple requests in parallel. http://us3.php.net/manual/en/function.curl-multi-exec.php On Thu, Jul 23, 2009 at 06:30, DavidH david.h...@gmail.com wrote: Cheers for that: it's what I thought but just wanted to check. Guess I'll have to queue separate cron jobs if things start to get too big. On Jul 23, 1:31 pm, Cameron Kaiser spec...@floodgap.com wrote: If I want to update multiple Twitter user accounts at once (with a different message for each), is there anyway to do it other than making multiple posts to update.xml? Nope. -- personal: http://www.cameronkaiser.com/-- Cameron Kaiser * Floodgap Systems *www.floodgap.com* ckai...@floodgap.com -- Humor is a drug which it's the fashion to abuse. -- William Gilbert -- Abraham Williams | Community Evangelist | http://web608.org Hacker | http://abrah.am | http://twitter.com/abraham Project | http://fireeagle.labs.poseurtech.com This email is: [ ] blogable [x] ask first [ ] private. Sent from Seattle, WA, United States
[twitter-dev] New Twist To Follow Terms Violations
On Twitter's new site, http://business.twitter.com, under the heading Best Practices, the following is listed as a spamming practice: Following churn: Following and unfollowing the same people repeatedly, as well as following and unfollowing those who don't follow back, are both violations of our terms of service. Take note devs, the ...unfollowing those who don't follow back... statement is posing a risk for any of your apps that do bulk unfollow. On that point, I would like to get clear guidance from Twitter whether unfollowing someone who has stopped following you, i.e., unfollowed you first, would also constitute a violation of Twitter terms.
[twitter-dev] Re: statuses/friends page count?
I'm experiencing the same issue with implu. With 14,408 follows, I should go up to page 145. However, the last page of data is 101 and 102 onwards returns nothing. http://twitter.com/statuses/friends/implu.xml?page=102 The following call does seem to return all the friends/ids however. http://twitter.com/friends/ids/implu.xml Any thoughts? On Jul 10, 10:47 am, Karthik Murugan fermis...@gmail.com wrote: It's also possible, that some intermediate pages return empty result set. Try this,http://twitter.com/statuses/followers.json?id=billcrosbypage=124 This profile has 44K followers and some of the intermediate pages return empty result sets. Not sure why, but my obvious guess is that all the followers in this page are suspended. So, empty result set doesn't mean that you are done with the traversal. I've modified my scripts to crawl N number of pages, where N is (number of followers/100). Total number of followers can be retrieved by users/show - Karthik On Jul 6, 11:31 pm, Doug Williams d...@twitter.com wrote: Duane,Yes, you will get an empty result set if you step off the end: doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=43; | grep /user | wc -l 100 doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=44; | grep /user | wc -l 17 doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=45; | grep /user | wc -l 0 Thanks, Doug -- Do you follow me?http://twitter.com/dougw On Mon, Jul 6, 2009 at 10:32 AM, Duane Roelands duane.roela...@gmail.comwrote: Do you ever get an empty response set? I was experimenting with the pagination and I found that if you request page 20 (for example) for someone who only has one page of friends, you simply get the page 1 response set. On Jul 6, 1:09 pm, Doug Williams d...@twitter.com wrote: You should either page throughstatuses/friendsuntil you get an empty response set or use statuses/show to get the number of friends expected and intelligently page to the end of the list. Thanks, Doug
[twitter-dev] Re: follow limits
If you have 7000 people following you already it's not the 2000/1900 follow restriction it's just the daily follow limit API. Wait until tomorrow and try again etc. This will obvious take 5 days for you to get all follow backs implemented. Regards, Dean Collins Cognation Inc d...@cognation.net mailto:d...@cognation.net +1-212-203-4357 New York +61-2-9016-5642 (Sydney in-dial). +44-20-3129-6001 (London in-dial). From: twitter-development-talk@googlegroups.com [mailto:twitter-development-t...@googlegroups.com] On Behalf Of Mario Menti Sent: Friday, July 24, 2009 12:36 PM To: twitter-development-talk@googlegroups.com Subject: [twitter-dev] follow limits Hi there, I have a twitter account I use for my company, which has just over 7,000 current followers. Because we're just about to launch a DM-based support service on that account, I'm trying to programmatically follow all these 7,000 users back, but am stuck at about 1,500, and am getting the You are unable to follow more people at this time message. The account is whitelisted, but it seems the follow limits are separate to that? Is there anything that can be done to help this - this is a legit account, and since these particular support functions (essentially reminder-type questions) require proof of a user's twitter account, a DM bot would be an ideal and quick solution, but will obviously only work if we can follow all our existing followers back. Any help/pointers appreciated, thanks, Mario.
[twitter-dev] Re: statuses/friends page count?
Just as aside, does anyone know if each call to a new page counts against the API limit? On Jul 24, 8:08 am, st...@implu.com st...@implu.com wrote: I'm experiencing the same issue with implu. With 14,408 follows, I should go up to page 145. However, the last page of data is 101 and 102 onwards returns nothing. http://twitter.com/statuses/friends/implu.xml?page=102 The following call does seem to return all the friends/ids however. http://twitter.com/friends/ids/implu.xml Any thoughts? On Jul 10, 10:47 am, Karthik Murugan fermis...@gmail.com wrote: It's also possible, that some intermediate pages return empty result set. Try this,http://twitter.com/statuses/followers.json?id=billcrosbypage=124 This profile has 44K followers and some of the intermediate pages return empty result sets. Not sure why, but my obvious guess is that all the followers in this page are suspended. So, empty result set doesn't mean that you are done with the traversal. I've modified my scripts to crawl N number of pages, where N is (number of followers/100). Total number of followers can be retrieved by users/show - Karthik On Jul 6, 11:31 pm, Doug Williams d...@twitter.com wrote: Duane,Yes, you will get an empty result set if you step off the end: doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=43; | grep /user | wc -l 100 doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=44; | grep /user | wc -l 17 doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=45; | grep /user | wc -l 0 Thanks, Doug -- Do you follow me?http://twitter.com/dougw On Mon, Jul 6, 2009 at 10:32 AM, Duane Roelands duane.roela...@gmail.comwrote: Do you ever get an empty response set? I was experimenting with the pagination and I found that if you request page 20 (for example) for someone who only has one page of friends, you simply get the page 1 response set. On Jul 6, 1:09 pm, Doug Williams d...@twitter.com wrote: You should either page throughstatuses/friendsuntil you get an empty response set or use statuses/show to get the number of friends expected and intelligently page to the end of the list. Thanks, Doug
[twitter-dev] Re: New Twist To Follow Terms Violations
What? Re: as well as following and unfollowing those who don't follow back, are both violations of our terms of service. What gives Twitter the right to dictate who you want to follow or not? That is like Gmail saying you can't remove contacts from your contacts list. When I signed up it suggested a list of people to follow but I didn't find the tweets interesting so I un-followed them (they didn't follow me back, but that was not the reason I un-followed them). I should have the right to decide who I want to follow or not unless Twitter is under a communist regime? Is there also a term that if someone posts a link I have to click it also? I also followed iGoogle for a while but didn't find the tweets that interesting so I un-followed them, they never followed me back, so if Twitter wants to delete my account (for TOS violations) then fine go a head, do so right now then, but I feel it is my right to decide who I do and do not want to follow and that will not change. They need to post a message when you sign up that you are not allowed to un-follow people. Why is there even that option then? Regards, Vision Jinx @visionjinx (In case Twitter wants to delete my account for feeling I have the right to decide who I follow, fine then do it now.) I also, un- followed someone because they kept posting the same tweets over again so who's the bigger offender there then? On Jul 24, 10:22 am, Dewald Pretorius dpr...@gmail.com wrote: On Twitter's new site,http://business.twitter.com, under the heading Best Practices, the following is listed as a spamming practice: Following churn: Following and unfollowing the same people repeatedly, as well as following and unfollowing those who don't follow back, are both violations of our terms of service. Take note devs, the ...unfollowing those who don't follow back... statement is posing a risk for any of your apps that do bulk unfollow. On that point, I would like to get clear guidance from Twitter whether unfollowing someone who has stopped following you, i.e., unfollowed you first, would also constitute a violation of Twitter terms.
[twitter-dev] Re: New Twist To Follow Terms Violations
The way I read the actual text it suggests that following and then unfollowing in a short period of time is the violation...not that you can't unfollow someone who didn't follow you. In fact, isn't following someone who doesn't want to follow you back a form of stalking for some people? I've always taken it as a given that it's a courtesy to not continue following someone indefinitely if they're not following back. There are exceptions of course depending on the person or topic and how that person is operating their account and whether they are a public or private entity. http://twitter.zendesk.com/forums/26257/entries/18311
[twitter-dev] Re: New Twist To Follow Terms Violations
On Fri, Jul 24, 2009 at 1:13 PM, Vision Jinx vjn...@gmail.com wrote: What? Re: as well as following and unfollowing those who don't follow back, are both violations of our terms of service. What gives Twitter the right to dictate who you want to follow or not? That is like Gmail saying you can't remove contacts from your contacts list. When I signed up it suggested a list of people to follow but I didn't find the tweets interesting so I un-followed them (they didn't follow me back, but that was not the reason I un-followed them). I should have the right to decide who I want to follow or not unless Twitter is under a communist regime? Is there also a term that if someone posts a link I have to click it also? I also followed iGoogle for a while but didn't find the tweets that interesting so I un-followed them, they never followed me back, so if Twitter wants to delete my account (for TOS violations) then fine go a head, do so right now then, but I feel it is my right to decide who I do and do not want to follow and that will not change. They need to post a message when you sign up that you are not allowed to un-follow people. Why is there even that option then? Regards, Vision Jinx @visionjinx (In case Twitter wants to delete my account for feeling I have the right to decide who I follow, fine then do it now.) I also, un- followed someone because they kept posting the same tweets over again so who's the bigger offender there then? Wow, ridiculous much? Those terms are in place to prevent spam-friendly follow-bot whores from ruining the ecosystem for all of us. If anything about the spirit or intention of those terms is disagreeable to you, I'm going to go ahead and label you an inconsiderate spammer. Thanks- - Andy Badera - and...@badera.us - Google me: http://www.google.com/search?q=andrew+badera - This email is: [ ] bloggable [x] ask first [ ] private
[twitter-dev] Re: New Twist To Follow Terms Violations
you knew that was going to happen. How about bulk follows, if that its done in a thoughtful way? On Fri, Jul 24, 2009 at 9:22 AM, Dewald Pretorius dpr...@gmail.com wrote: On Twitter's new site, http://business.twitter.com, under the heading Best Practices, the following is listed as a spamming practice: Following churn: Following and unfollowing the same people repeatedly, as well as following and unfollowing those who don't follow back, are both violations of our terms of service. Take note devs, the ...unfollowing those who don't follow back... statement is posing a risk for any of your apps that do bulk unfollow. On that point, I would like to get clear guidance from Twitter whether unfollowing someone who has stopped following you, i.e., unfollowed you first, would also constitute a violation of Twitter terms. -- Dale Merritt Fol.la MeDia, LLC
[twitter-dev] Re: Updating the APIs authentication limiting policy
Jim's concern is valid, fortunately OAuth is immune to brute-force attacks once the access key has been issued to an application. For this reason alone I would urge people to switch to OAuth if at all possible. I would hope (and assume) that if login attempts for an account are locked out that a user would still be able to successfully use an already authorized OAuth driven application. Unfortunately allowing a successful un/pw login while an account is locked out even when the correct password is presented effectively bypasses the whole reason for a lockout in the first place, preventing brute-force password attempts. If an attacker used a dictionary or brute-force attack and the account was locked out after 15 attempts, then they could continue trying even though the system replied locked out; if they eventually sent the correct password it would just bypass the lockout and they would then know the correct password. Perhaps Twitter could implement a selective captcha, I know they are annoying but if executed properly it could be effective protection against brute-force and dictionary attacks. Say after 3 or 4 failed attempts without a captch the API would then include a captcha image URL in it's response that the application would then need to show to the person and include the user's response with the next authentication attempt as a header or POST variable. The site stackoverflow.com does this to great effect, if you create posts quicker than a certain threshold which a person would not exceed then they pop a captcha up, in the normal use of the site you will never see one; I've only hit two captchas in the last in the last 8 months using the site. Josh Dewald Pretorius wrote: Jim raised a huge weakness with the authentication rate limiting that could essentially break third-party apps. Anybody can try to add anybody else's Twitter account to a third-party app using an invalid password. If they do that 15 times with a Twitter account, the real owner of that Twitter account, who may have added his account a long time ago with the correct password, is locked out from using that app for an hour. I believe you will absolutely have to reset / remove the lock as soon as the Twitter account uses the correct password. On Jul 22, 4:58 pm, jim.renkel james.ren...@gmail.com wrote: My concern with this proposal is that it opens up denials of service, not to twitter.com, but to associated sites such as twitpic, or my site twxlate, among others For example, Lance Armstrong is a heavy user of twitpic. It is very easy for anyone to find Lance's twitter ID (@lancearmstrong), view his status updates, and see that he is a frequent user of twitpic. Now, someone that is unhappy with Lance, say one of George Hincapie's ardent fans that really believes that Lance was a significant contributor to George not winning the maillot jeune last Sunday, could go to twitpic, fail to login as Lance the requisite number of times, and deny Lance access to twitpic. Not only celebrities would or could be subject to such denials of service. I notice that @dougw occasionally uses twitpic! :-) One solution to this problem is to add to each twitter account another private ID. By default this private ID would be equal to the existing (public) ID (If not equal to the account's public ID, it would have to be unique among all twitter IDs, both public and private.). The public ID would be used just as the existing twitter ID is now: others would use it to follow, mention, DM, etc., the user. But the user MUST use their private ID for authenticated requests through the API, and CAN also use it for non-authenticated requests. In either case, twitter would treat a request from a private ID as if it came from the corresponding public ID. Blocking the public ID because of excessive authentication failures would NOT block the associated private ID unless they were equal. Changing your public ID would also change your private ID if the two were the same before the change, i.e., they would remain the same after the change. It may seem onerous to require all users to also have a private ID, but since it defaults to be the same as their public ID, only those concerned about their service being denied would change it and subsequently use it instead of their public ID to access associated sites such as twitpic or twxlate. In fact, I think this change, though potentially large on the twitter side, could be implemented without any changes to users or associated sites, with one small, obscure exception: now, if I attempt to create a new twitter account or change the ID of an existing account, and find that the ID I want is in use, I can view that account; if this were implemented and I attempted to use a private ID that was not the same as its associated public ID, I could not view the account using the denied ID. Comments expected and welcome. Jim Renkel On Jul 21, 6:00 pm, Doug Williams d...@twitter.com
[twitter-dev] Re: New Twist To Follow Terms Violations
Think about a bot who just bulk follows random people, it then would kept track of users who didn't blindly or automatically follow back and dump them quickly and try following another batch of users so that it wouldn't bust it's follow ratio limit. Using this strategy a bot could eventually build a very large following/followers list for someone while still keeping it's ratio within the boundaries set. I believe that the second part of that term is to protect against this scenario. Vision Jinx wrote: What? Re: as well as following and unfollowing those who don't follow back, are both violations of our terms of service. What gives Twitter the right to dictate who you want to follow or not? That is like Gmail saying you can't remove contacts from your contacts list. When I signed up it suggested a list of people to follow but I didn't find the tweets interesting so I un-followed them (they didn't follow me back, but that was not the reason I un-followed them). I should have the right to decide who I want to follow or not unless Twitter is under a communist regime? Is there also a term that if someone posts a link I have to click it also? I also followed iGoogle for a while but didn't find the tweets that interesting so I un-followed them, they never followed me back, so if Twitter wants to delete my account (for TOS violations) then fine go a head, do so right now then, but I feel it is my right to decide who I do and do not want to follow and that will not change. They need to post a message when you sign up that you are not allowed to un-follow people. Why is there even that option then? Regards, Vision Jinx @visionjinx (In case Twitter wants to delete my account for feeling I have the right to decide who I follow, fine then do it now.) I also, un- followed someone because they kept posting the same tweets over again so who's the bigger offender there then? On Jul 24, 10:22 am, Dewald Pretorius dpr...@gmail.com wrote: On Twitter's new site,http://business.twitter.com, under the heading Best Practices, the following is listed as a spamming practice: Following churn: Following and unfollowing the same people repeatedly, as well as following and unfollowing those who don't follow back, are both violations of our terms of service. Take note devs, the ...unfollowing those who don't follow back... statement is posing a risk for any of your apps that do bulk unfollow. On that point, I would like to get clear guidance from Twitter whether unfollowing someone who has stopped following you, i.e., unfollowed you first, would also constitute a violation of Twitter terms.
[twitter-dev] Re: statuses/friends page count?
yes, each page request counts against the API limit On Fri, Jul 24, 2009 at 10:25 PM, Joseph northwest...@gmail.com wrote: Just as aside, does anyone know if each call to a new page counts against the API limit? On Jul 24, 8:08 am, st...@implu.com st...@implu.com wrote: I'm experiencing the same issue with implu. With 14,408 follows, I should go up to page 145. However, the last page of data is 101 and 102 onwards returns nothing. http://twitter.com/statuses/friends/implu.xml?page=102 The following call does seem to return all the friends/ids however. http://twitter.com/friends/ids/implu.xml Any thoughts? On Jul 10, 10:47 am, Karthik Murugan fermis...@gmail.com wrote: It's also possible, that some intermediate pages return empty result set. Try this, http://twitter.com/statuses/followers.json?id=billcrosbypage=124 This profile has 44K followers and some of the intermediate pages return empty result sets. Not sure why, but my obvious guess is that all the followers in this page are suspended. So, empty result set doesn't mean that you are done with the traversal. I've modified my scripts to crawl N number of pages, where N is (number of followers/100). Total number of followers can be retrieved by users/show - Karthik On Jul 6, 11:31 pm, Doug Williams d...@twitter.com wrote: Duane,Yes, you will get an empty result set if you step off the end: doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=43; | grep /user | wc -l 100 doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=44; | grep /user | wc -l 17 doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=45; | grep /user | wc -l 0 Thanks, Doug -- Do you follow me?http://twitter.com/dougw On Mon, Jul 6, 2009 at 10:32 AM, Duane Roelands duane.roela...@gmail.comwrote: Do you ever get an empty response set? I was experimenting with the pagination and I found that if you request page 20 (for example) for someone who only has one page of friends, you simply get the page 1 response set. On Jul 6, 1:09 pm, Doug Williams d...@twitter.com wrote: You should either page throughstatuses/friendsuntil you get an empty response set or use statuses/show to get the number of friends expected and intelligently page to the end of the list. Thanks, Doug
[twitter-dev] Re: New Twist To Follow Terms Violations
What gives Twitter the right to dictate who you want to follow or not? Its their service. They can dictate what they want. Their playground, their rules. The ToS clearly says they can alter their terms at any time and if you don't want to comply you can leave. That being said, this is to prevent people who are massively following and unfollowing (as I see happen dozens of times daily on my account). You aren't going to get banned for not following someone back, or for unfollowing someone that bothers you. dave On Jul 24, 2:15 pm, Joshua Perry j...@6bit.com wrote: Think about a bot who just bulk follows random people, it then would kept track of users who didn't blindly or automatically follow back and dump them quickly and try following another batch of users so that it wouldn't bust it's follow ratio limit. Using this strategy a bot could eventually build a very large following/followers list for someone while still keeping it's ratio within the boundaries set. I believe that the second part of that term is to protect against this scenario. Vision Jinx wrote: What? Re: as well as following and unfollowing those who don't follow back, are both violations of our terms of service. What gives Twitter the right to dictate who you want to follow or not? That is like Gmail saying you can't remove contacts from your contacts list. When I signed up it suggested a list of people to follow but I didn't find the tweets interesting so I un-followed them (they didn't follow me back, but that was not the reason I un-followed them). I should have the right to decide who I want to follow or not unless Twitter is under a communist regime? Is there also a term that if someone posts a link I have to click it also? I also followed iGoogle for a while but didn't find the tweets that interesting so I un-followed them, they never followed me back, so if Twitter wants to delete my account (for TOS violations) then fine go a head, do so right now then, but I feel it is my right to decide who I do and do not want to follow and that will not change. They need to post a message when you sign up that you are not allowed to un-follow people. Why is there even that option then? Regards, Vision Jinx @visionjinx (In case Twitter wants to delete my account for feeling I have the right to decide who I follow, fine then do it now.) I also, un- followed someone because they kept posting the same tweets over again so who's the bigger offender there then? On Jul 24, 10:22 am, Dewald Pretorius dpr...@gmail.com wrote: On Twitter's new site,http://business.twitter.com, under the heading Best Practices, the following is listed as a spamming practice: Following churn: Following and unfollowing the same people repeatedly, as well as following and unfollowing those who don't follow back, are both violations of our terms of service. Take note devs, the ...unfollowing those who don't follow back... statement is posing a risk for any of your apps that do bulk unfollow. On that point, I would like to get clear guidance from Twitter whether unfollowing someone who has stopped following you, i.e., unfollowed you first, would also constitute a violation of Twitter terms.
[twitter-dev] Re: New Twist To Follow Terms Violations
Thanks for the comments RandyC and Joshua Perry! I just started a Twitter account so I could develop free Twitter apps and tweet things I thought my friends would find interesting and in a short period of time I gathered a few followers (more then I actually expected). I don't know the twitter etiquette if I am suppose to follow them back or only the ones I think have interesting tweets and ones I do want to follow for the information they provide. It does not bother me if people follow me for the information I tweet. As for the Terms, I tend to take them literally and not read them as in the spirit of or read between the lines so my bad if I misinterpreted them but I would like to see an official response from a Twitter rep if possible (for personal clarification). What you say Joshua Perry does makes a lot of sense! :) Thanks for the link RandyC I have re-read it: *If you have followed and unfollowed people in a short time period (I have not) *If your updates consist mainly of links, and not personal updates; (I do post links to things I think are interesting, but so does @mashable and @Google etc, my friends do like the info. I tweet. If required I can add more fluff like my cat is watching my fish, my cat meowed at me and wants food, stuck in traffic etc to off set the links I post if required, but looking at my followers so far I don't believe that would interest them) I seriously did not mean that in any condescending way at all. :) @Andy Badera - Not to start a flame war, but are you implying I'm an inconsiderate spammer? I assure you there is a real person here and not an automated bot making my posts, Also being a Google Trusted Developer and one of their API Gurus (and friends with several Google Devs) I spend a lot of time helping developers with APIs and Mashups (and would like to do the same in the Twitter Dev community, and make friends here also) and if you check my groups profile you will see I have a pretty decent rating far from which would suggest that I am a spammer of any kind. But if you feel the need to report me an inconsiderate spammer well, what ever floats your boat man. I do appreciate what your saying about the spam bots though and I don't like getting spam followers any more then anyone else, I assure you that was not what my comments were regarding. I took the terms post as being literally translated (like how a legal team would read it) and if Twitter states un-following people is a violation of their terms well I'm sorry but I did interpret it as being such. I ask for clarification then for the record. Best regards everyone, Vision Jinx (Please forgive the length of my reply and any negative connotations I may have given off earlier) On Jul 24, 12:15 pm, Joshua Perry j...@6bit.com wrote: Think about a bot who just bulk follows random people, it then would kept track of users who didn't blindly or automatically follow back and dump them quickly and try following another batch of users so that it wouldn't bust it's follow ratio limit. Using this strategy a bot could eventually build a very large following/followers list for someone while still keeping it's ratio within the boundaries set. I believe that the second part of that term is to protect against this scenario. Vision Jinx wrote: What? Re: as well as following and unfollowing those who don't follow back, are both violations of our terms of service. What gives Twitter the right to dictate who you want to follow or not? That is like Gmail saying you can't remove contacts from your contacts list. When I signed up it suggested a list of people to follow but I didn't find the tweets interesting so I un-followed them (they didn't follow me back, but that was not the reason I un-followed them). I should have the right to decide who I want to follow or not unless Twitter is under a communist regime? Is there also a term that if someone posts a link I have to click it also? I also followed iGoogle for a while but didn't find the tweets that interesting so I un-followed them, they never followed me back, so if Twitter wants to delete my account (for TOS violations) then fine go a head, do so right now then, but I feel it is my right to decide who I do and do not want to follow and that will not change. They need to post a message when you sign up that you are not allowed to un-follow people. Why is there even that option then? Regards, Vision Jinx @visionjinx (In case Twitter wants to delete my account for feeling I have the right to decide who I follow, fine then do it now.) I also, un- followed someone because they kept posting the same tweets over again so who's the bigger offender there then? On Jul 24, 10:22 am, Dewald Pretorius dpr...@gmail.com wrote: On Twitter's new site,http://business.twitter.com, under the heading Best Practices, the following is listed as a spamming practice: Following churn: Following and unfollowing the same
[twitter-dev] Re: API limit confusion
If this is correct (and I don't think it is), then it's very different from what has always been my understanding. I've stated a few times on this list my belief that if you're going to be supporting a significant number of simultaneous users, whitelisting works against you. No one has ever challenged that assertion. This is in the rate limit documentation Abraham linked to: Each whitelisted entity, whether an account or IP address, is allowed 2 requests per hour. It is my understanding that if your IP is whitelisted, you get one pool of 20,000 requests per each hour to divide up amongst all your users - NOT 20,000 requests for each of them. I could be wrong, but I don't think I am. If you have a whitelisted IP, here's an experiment you can run: 1. Check the current rate limit using two different Twitter accounts from your IP (using curl or whichever tool you choose). You should get the same number (probably 20,000 unless you've been making requests recently from that IP) 2. Make a request that counts against the rate limit from ONE and only ONE of the two accounts (grab their statuses/friends_timeline, for example) 3. Repeat step 1. Do both users now see 19,999? Or does one see 19,999 and one see 20,000? If one user still sees 20,000, then I'm wrong, and you've got 20,000 requests per hour per user (and I'm gonna go apply for whitelisting :). If they both see 19,999, then I'm right - your IP has a single pool of 20,000 requests from which all of your users draw. On Jul 24, 2:36 am, srikanth reddy srikanth.yara...@gmail.com wrote: @jim.renkel. Thanks a ton. I think now it is clear. It appears to me that each user of a white-listed site gets 20k requests per hour, independent of any other users of that site or *any other uses of the twitter API at other sites by that user * probably this is what they mean by *IP whitelisting takes precedence to account rate limits*. *GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users*. If the IP limit is for the consumer then it will lead to denial of service attacks. This is how we wanted it to work. Srikanth On Fri, Jul 24, 2009 at 12:52 AM, jim.renkel james.ren...@gmail.com wrote: My experience with this is, I think, a little bit different than what you describe. It appears to me that each user of a white-listed site gets 20k requests per hour, independent of any other users of that site or any other uses of the twitter API at other sites by that user. I didn't think this was what twitter intended and reported it as a bug (See:http://code.google.com/p/twitter-api/issues/detail?id=617), but the twitter folk said Yup, working as intended. After you log in athttp://twxlate.com, the site reports rate limit information on every page view, so you can see how this works there. Comments expected and welcome. Jim Renkel On Jul 23, 3:48 am, jmathai jmat...@gmail.com wrote: In other words, you have a web app running on a single server with a single IP. You make authenticated requests using each user's account. If your IP is whitelisted, the calls go towards your 20k limit, if it is not whitelisted, it goes against the current 150 limit for the respective accounts. That's what it means by IP whitelisting takes precedence to account rate limits. I don't believe that is true. If your web app is running on a whitelisted IP then you get up to 20k GET calls per hour. POST requests (status or DM) are counted against the user being authenticated. You CANNOT retrieve a user's rate limit status.
[twitter-dev] Re: API limit confusion
On Jul 24, 4:13 am, Hwee-Boon Yar hweeb...@gmail.com wrote: Isn't this what I said? I don't think it is. I think your take is correct. What's telling is this bit of text from up the chain: It appears to me that each user of a white-listed site gets 20k requests per hour. I don't believe it's true that each user on a site whitelisted by IP gets their own 20k requests to use. I think it's true that *all* users of that whitelisted site are drawing from the *same single pool* of 20k requests allocated to that IP. I think that's where Srikanth is confused. But I'm always willing concede that I might be wrong. I'm wrong a lot. I would actually like to be wrong on this one.
[twitter-dev] Re: statuses/friends page count?
The API removes suspended accounts lazily at read time which is why you may receive incomplete pages (less users than the count parameter specifies). We are still working to report correct follower numbers in a timely matter which was the reason for last night's maintenance work (see http://status.twitter.com). Thanks, Doug On Fri, Jul 24, 2009 at 11:31 AM, Karthik Muruganfermis...@gmail.com wrote: yes, each page request counts against the API limit On Fri, Jul 24, 2009 at 10:25 PM, Joseph northwest...@gmail.com wrote: Just as aside, does anyone know if each call to a new page counts against the API limit? On Jul 24, 8:08 am, st...@implu.com st...@implu.com wrote: I'm experiencing the same issue with implu. With 14,408 follows, I should go up to page 145. However, the last page of data is 101 and 102 onwards returns nothing. http://twitter.com/statuses/friends/implu.xml?page=102 The following call does seem to return all the friends/ids however. http://twitter.com/friends/ids/implu.xml Any thoughts? On Jul 10, 10:47 am, Karthik Murugan fermis...@gmail.com wrote: It's also possible, that some intermediate pages return empty result set. Try this,http://twitter.com/statuses/followers.json?id=billcrosbypage=124 This profile has 44K followers and some of the intermediate pages return empty result sets. Not sure why, but my obvious guess is that all the followers in this page are suspended. So, empty result set doesn't mean that you are done with the traversal. I've modified my scripts to crawl N number of pages, where N is (number of followers/100). Total number of followers can be retrieved by users/show - Karthik On Jul 6, 11:31 pm, Doug Williams d...@twitter.com wrote: Duane,Yes, you will get an empty result set if you step off the end: doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=43; | grep /user | wc -l 100 doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=44; | grep /user | wc -l 17 doug-williamss-macbook-pro:~ igudo$ curl -u dougw:PASSWORD http://twitter.com/statuses/followers.xml?count=100page=45; | grep /user | wc -l 0 Thanks, Doug -- Do you follow me?http://twitter.com/dougw On Mon, Jul 6, 2009 at 10:32 AM, Duane Roelands duane.roela...@gmail.comwrote: Do you ever get an empty response set? I was experimenting with the pagination and I found that if you request page 20 (for example) for someone who only has one page of friends, you simply get the page 1 response set. On Jul 6, 1:09 pm, Doug Williams d...@twitter.com wrote: You should either page throughstatuses/friendsuntil you get an empty response set or use statuses/show to get the number of friends expected and intelligently page to the end of the list. Thanks, Doug
[twitter-dev] Re: statuses/friends page count?
I suspect this lazy updating of the summary data (follower counts) explains why people keep saying I used Twitter Karma, and all of a sudden I lost hundreds of followers! :-) On 7/24/09 4:39 PM, Doug Williams wrote: The API removes suspended accounts lazily at read time which is why you may receive incomplete pages (less users than the count parameter specifies). We are still working to report correct follower numbers in a timely matter which was the reason for last night's maintenance work (see http://status.twitter.com). -- Dossy Shiobara | do...@panoptic.com | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70)
[twitter-dev] Re: Updating the APIs authentication limiting policy
Well said Joshua. Dewald, you have identified the risk of using basic authentication. If your users being locked out due to malicious behavior, you should either implement further user-level rate limiting on your side or adopt OAuth. Are there any other glaring omissions in our thinking or should we proceed with this as our solution? Thanks, Doug On Fri, Jul 24, 2009 at 11:08 AM, Joshua Perryj...@6bit.com wrote: Jim's concern is valid, fortunately OAuth is immune to brute-force attacks once the access key has been issued to an application. For this reason alone I would urge people to switch to OAuth if at all possible. I would hope (and assume) that if login attempts for an account are locked out that a user would still be able to successfully use an already authorized OAuth driven application. Unfortunately allowing a successful un/pw login while an account is locked out even when the correct password is presented effectively bypasses the whole reason for a lockout in the first place, preventing brute-force password attempts. If an attacker used a dictionary or brute-force attack and the account was locked out after 15 attempts, then they could continue trying even though the system replied locked out; if they eventually sent the correct password it would just bypass the lockout and they would then know the correct password. Perhaps Twitter could implement a selective captcha, I know they are annoying but if executed properly it could be effective protection against brute-force and dictionary attacks. Say after 3 or 4 failed attempts without a captch the API would then include a captcha image URL in it's response that the application would then need to show to the person and include the user's response with the next authentication attempt as a header or POST variable. The site stackoverflow.com does this to great effect, if you create posts quicker than a certain threshold which a person would not exceed then they pop a captcha up, in the normal use of the site you will never see one; I've only hit two captchas in the last in the last 8 months using the site. Josh Dewald Pretorius wrote: Jim raised a huge weakness with the authentication rate limiting that could essentially break third-party apps. Anybody can try to add anybody else's Twitter account to a third-party app using an invalid password. If they do that 15 times with a Twitter account, the real owner of that Twitter account, who may have added his account a long time ago with the correct password, is locked out from using that app for an hour. I believe you will absolutely have to reset / remove the lock as soon as the Twitter account uses the correct password. On Jul 22, 4:58 pm, jim.renkel james.ren...@gmail.com wrote: My concern with this proposal is that it opens up denials of service, not to twitter.com, but to associated sites such as twitpic, or my site twxlate, among others For example, Lance Armstrong is a heavy user of twitpic. It is very easy for anyone to find Lance's twitter ID (@lancearmstrong), view his status updates, and see that he is a frequent user of twitpic. Now, someone that is unhappy with Lance, say one of George Hincapie's ardent fans that really believes that Lance was a significant contributor to George not winning the maillot jeune last Sunday, could go to twitpic, fail to login as Lance the requisite number of times, and deny Lance access to twitpic. Not only celebrities would or could be subject to such denials of service. I notice that @dougw occasionally uses twitpic! :-) One solution to this problem is to add to each twitter account another private ID. By default this private ID would be equal to the existing (public) ID (If not equal to the account's public ID, it would have to be unique among all twitter IDs, both public and private.). The public ID would be used just as the existing twitter ID is now: others would use it to follow, mention, DM, etc., the user. But the user MUST use their private ID for authenticated requests through the API, and CAN also use it for non-authenticated requests. In either case, twitter would treat a request from a private ID as if it came from the corresponding public ID. Blocking the public ID because of excessive authentication failures would NOT block the associated private ID unless they were equal. Changing your public ID would also change your private ID if the two were the same before the change, i.e., they would remain the same after the change. It may seem onerous to require all users to also have a private ID, but since it defaults to be the same as their public ID, only those concerned about their service being denied would change it and subsequently use it instead of their public ID to access associated sites such as twitpic or twxlate. In fact, I think this change, though potentially large on the twitter side, could be implemented without any changes to users or
[twitter-dev] Anyone experiencing any issues with API performance just now?
At twibbon.com we are seeing long latencies and frequent failures on API calls, as of 15 minutes ago. No mention on status.twitter.com. Anyone else experiencing any issues? Jonathan Joyce Founder Twibbon.com
[twitter-dev] Is it possible to receive a list of twitter users starting with a given string ?
Hello, For example, I'd like to display all the twitter users starting with guy, in AJAX. I would have a text input. the user would write down guy, I would send that string to the twitter API, that would send me back a list of all the users with the name starting with guy. Is this thing possible ?
[twitter-dev] The remote server returned an error: (401) Unauthorized.
Working in C# trying to send a tweet using Basic Auth with my own
current credentials from a Vista machine running IIS7. Any reason why
I am getting 401 Unauthorized?
try
{
string user = Convert.ToBase64String
(System.Text.Encoding.UTF8.GetBytes(metroUserName + : +
metroPassword));
byte[] bytes = System.Text.Encoding.ASCII.GetBytes
(status= + tweetText);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create
(http://twitter.com/statuses/update.xml;);
request.Method = POST;
request.ServicePoint.Expect100Continue = false;
request.Headers.Add(Authorization, Basic + user);
request.ContentType = application/x-www-form-urlencoded;
request.ContentLength = bytes.Length;
Stream reqStream = request.GetRequestStream();
reqStream.Write(bytes, 0, bytes.Length);
reqStream.Close();
HttpWebResponse response = (HttpWebResponse)
request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader returned = new StreamReader(responseStream);
// what is returned at this point
validationLabel.Text = returned.ReadToEnd();
}
catch (Exception ex)
{
validationLabel.Text = ex.Message;
}
[twitter-dev] Re: need help with OAuth library usage in cocoa app
Thank you for your prompt reply Isaiah, Thank you also for the offer of emailing you directly, but I think your code is also precious for other newbie like me who want to know more about this OAuth thing :) About changing application type to be Browser, then there is Callback URL: box that I have to fill, which I have no idea what do I have to fill in there. But I tried it anyway, and same result. Fares On Jul 24, 10:58 pm, Isaiah supp...@yourhead.com wrote: My example was built right as the pin code method was invented/ implemented in the API. So my example still uses the Browser method that doesn't require a pin code. If you go to your application settings page in twitter and set your Application Type to be Browser you should be good to go. As I understand it the PIN code was invented to help clients that couldn't detect if the browser had been sent to the success callback URL. However, my example doesn't have this issue. My example embeds the browser and communicates directly with it to determine when the callback URL is sent. This technique obviates the need for the pin code. I like to think of my example as a hybrid app -- neither purely a desktop client nor really a web app -- but a little bit of both in the right places. ;-) I've considered adding the pin code, but it seemed to further complicate an already challenging UI without adding any value. If you have any other issues with the example code, please feel free to email me directly. I'd be happy to help out. Isaiah YourHead Software supp...@yourhead.comhttp://www.yourhead.com On Jul 24, 2009, at 12:04 AM, Fares Farhan wrote: Dear Twitter developers, First, I apologize if I misplace the question. I've cloned Isaiah's git repository of his AOuth implementation from http://github.com/yourhead/OAuth_ObjC_Test_App/tree/master but I experienced an issue that after the web sheet closed, there is no place that I can put the PIN retrieved from the authentication result, or anywhere in the code that I need to pass the oauth_verifier parameter along with other params. the debugger said that ther is Operation could not be completed. (NSURLErrorDomain error -1012.) Thank you in advance for any response, Cheers, Fares
[twitter-dev] Re: The remote server returned an error: (401) Unauthorized.
On Fri, 2009-07-24 at 13:43 -0700, HatMan wrote:
Working in C# trying to send a tweet using Basic Auth with my own
current credentials from a Vista machine running IIS7. Any reason why
I am getting 401 Unauthorized?
try
{
string user = Convert.ToBase64String
(System.Text.Encoding.UTF8.GetBytes(metroUserName + : +
metroPassword));
byte[] bytes = System.Text.Encoding.ASCII.GetBytes
(status= + tweetText);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create
(http://twitter.com/statuses/update.xml;);
request.Method = POST;
request.ServicePoint.Expect100Continue = false;
request.Headers.Add(Authorization, Basic + user);
request.ContentType = application/x-www-form-urlencoded;
request.ContentLength = bytes.Length;
Stream reqStream = request.GetRequestStream();
reqStream.Write(bytes, 0, bytes.Length);
reqStream.Close();
HttpWebResponse response = (HttpWebResponse)
request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader returned = new StreamReader(responseStream);
// what is returned at this point
validationLabel.Text = returned.ReadToEnd();
}
catch (Exception ex)
{
validationLabel.Text = ex.Message;
}
You're actually going the wrong way about sending credentials--you
shouldn't be adding headers willy nilly, let the .NET Framework classes
negotiate that for you. This is how I do Basic Auth in my Twitter API
implementation:
var webRequest = WebRequest.Create (uri);
var ccache = new CredentialCache ();
ccache.Add (uri, Basic, new NetworkCredential (userName, password));
webRequest.Credentials = ccache;
From here, you can use your code to send the POST request and read the
response. :)
If you want to use my Twitter API implementation (written in C#), you
can check it out at http://code.google.com/p/twarp/ and fetch the source
code. There aren't really many examples at the moment, but the XML
documentation is pretty much all there, and I think the API is simple
enough to explore via IntelliSense.
Regards,
--
Bojan Rajkovic boj...@brandeis.edu
Biochemistry '10, Brandeis University
PGP Signature Key ID: 0x8783D016
PGP Encryption Key ID: 0x2497B8B2
signature.asc
Description: This is a digitally signed message part
[twitter-dev] Re: need help with OAuth library usage in cocoa app
Thank you for your prompt reply Isaiah, No worries. Glad to help out. Thank you also for the offer of emailing you directly, but I think your code is also precious for other newbie like me who want to know more about this OAuth thing :) It wasn't so easy to figure out a nice way to do this stuff. I figured it would be a good thing to share. Glad it's helped. About changing application type to be Browser, then there is Callback URL: box that I have to fill, which I have no idea what do I have to fill in there. But I tried it anyway, and same result. The callback URL can be any URL you like. After a successful authorization this is the URL that's loaded. You should specify the URL both in the twitter app setup and also as a constant in the YHTwitter.m file on line 25: #define kYHOAuthTwitterCallbackSuccessURL @http://twitter.com/; Through the browser delegate methods the client will know that the callback URL was reached and the authorization was granted by the user. This triggers the example app to close the browser window and save the OAuth info to the keychain. Although I would imagine you would want to change it to your own web site for practical reasons, leaving it as http://twitter.com/ should *work* at least for testing purposes. Isaiah YourHead Software supp...@yourhead.com http://www.yourhead.com On Jul 24, 2009, at 1:06 PM, Fares Farhan wrote: Thank you for your prompt reply Isaiah, Thank you also for the offer of emailing you directly, but I think your code is also precious for other newbie like me who want to know more about this OAuth thing :) About changing application type to be Browser, then there is Callback URL: box that I have to fill, which I have no idea what do I have to fill in there. But I tried it anyway, and same result. Fares On Jul 24, 10:58 pm, Isaiah supp...@yourhead.com wrote: My example was built right as the pin code method was invented/ implemented in the API. So my example still uses the Browser method that doesn't require a pin code. If you go to your application settings page in twitter and set your Application Type to be Browser you should be good to go. As I understand it the PIN code was invented to help clients that couldn't detect if the browser had been sent to the success callback URL. However, my example doesn't have this issue. My example embeds the browser and communicates directly with it to determine when the callback URL is sent. This technique obviates the need for the pin code. I like to think of my example as a hybrid app -- neither purely a desktop client nor really a web app -- but a little bit of both in the right places. ;-) I've considered adding the pin code, but it seemed to further complicate an already challenging UI without adding any value. If you have any other issues with the example code, please feel free to email me directly. I'd be happy to help out. Isaiah YourHead Software supp...@yourhead.comhttp://www.yourhead.com On Jul 24, 2009, at 12:04 AM, Fares Farhan wrote: Dear Twitter developers, First, I apologize if I misplace the question. I've cloned Isaiah's git repository of his AOuth implementation from http://github.com/yourhead/OAuth_ObjC_Test_App/tree/master but I experienced an issue that after the web sheet closed, there is no place that I can put the PIN retrieved from the authentication result, or anywhere in the code that I need to pass the oauth_verifier parameter along with other params. the debugger said that ther is Operation could not be completed. (NSURLErrorDomain error -1012.) Thank you in advance for any response, Cheers, Fares
[twitter-dev] Re: Updating the APIs authentication limiting policy
I think Dewald's concern is very valid - and even though OAuth might solve it, the reality is that most (if not all) desktop and mobile apps are using Basic Auth today for various reasons, so if you implement this policy as described, there's a pretty high risk that many users can be locked out of twitter from their usual ways to access it. Also, again a reminder that AFAIK the last official status re: OAuth from Twitter was that it is still in beta, and therefore not recommended for production use - or has there been another announcement that I missed? Marco 2009/7/24 Doug Williams d...@twitter.com Well said Joshua. Dewald, you have identified the risk of using basic authentication. If your users being locked out due to malicious behavior, you should either implement further user-level rate limiting on your side or adopt OAuth. Are there any other glaring omissions in our thinking or should we proceed with this as our solution? Thanks, Doug On Fri, Jul 24, 2009 at 11:08 AM, Joshua Perryj...@6bit.com wrote: Jim's concern is valid, fortunately OAuth is immune to brute-force attacks once the access key has been issued to an application. For this reason alone I would urge people to switch to OAuth if at all possible. I would hope (and assume) that if login attempts for an account are locked out that a user would still be able to successfully use an already authorized OAuth driven application. Unfortunately allowing a successful un/pw login while an account is locked out even when the correct password is presented effectively bypasses the whole reason for a lockout in the first place, preventing brute-force password attempts. If an attacker used a dictionary or brute-force attack and the account was locked out after 15 attempts, then they could continue trying even though the system replied locked out; if they eventually sent the correct password it would just bypass the lockout and they would then know the correct password. Perhaps Twitter could implement a selective captcha, I know they are annoying but if executed properly it could be effective protection against brute-force and dictionary attacks. Say after 3 or 4 failed attempts without a captch the API would then include a captcha image URL in it's response that the application would then need to show to the person and include the user's response with the next authentication attempt as a header or POST variable. The site stackoverflow.com does this to great effect, if you create posts quicker than a certain threshold which a person would not exceed then they pop a captcha up, in the normal use of the site you will never see one; I've only hit two captchas in the last in the last 8 months using the site. Josh Dewald Pretorius wrote: Jim raised a huge weakness with the authentication rate limiting that could essentially break third-party apps. Anybody can try to add anybody else's Twitter account to a third-party app using an invalid password. If they do that 15 times with a Twitter account, the real owner of that Twitter account, who may have added his account a long time ago with the correct password, is locked out from using that app for an hour. I believe you will absolutely have to reset / remove the lock as soon as the Twitter account uses the correct password. On Jul 22, 4:58 pm, jim.renkel james.ren...@gmail.com wrote: My concern with this proposal is that it opens up denials of service, not to twitter.com, but to associated sites such as twitpic, or my site twxlate, among others For example, Lance Armstrong is a heavy user of twitpic. It is very easy for anyone to find Lance's twitter ID (@lancearmstrong), view his status updates, and see that he is a frequent user of twitpic. Now, someone that is unhappy with Lance, say one of George Hincapie's ardent fans that really believes that Lance was a significant contributor to George not winning the maillot jeune last Sunday, could go to twitpic, fail to login as Lance the requisite number of times, and deny Lance access to twitpic. Not only celebrities would or could be subject to such denials of service. I notice that @dougw occasionally uses twitpic! :-) One solution to this problem is to add to each twitter account another private ID. By default this private ID would be equal to the existing (public) ID (If not equal to the account's public ID, it would have to be unique among all twitter IDs, both public and private.). The public ID would be used just as the existing twitter ID is now: others would use it to follow, mention, DM, etc., the user. But the user MUST use their private ID for authenticated requests through the API, and CAN also use it for non-authenticated requests. In either case, twitter would treat a request from a private ID as if it came from the corresponding public ID. Blocking the
[twitter-dev] id field is missing in status from streaming API frequently
twitter streaming api has lots of statuses missing id?
the following exception appears almost continuously in my log. it indicates
the id field is missing in status from streaming API.
twitter4j.TwitterException: JSONObject[id] not
found.:{delete:{status:{id:2813410502,user_id:47157439}}}
twitter4j.TwitterException: JSONObject[id] not
found.:{delete:{status:{id:2812385903,user_id:54420955}}}
thanks,
-aj
--
AJ Chen, PhD
Co-Chair, Semantic Web SIG, sdforum.org
http://web2express.org
Palo Alto, CA
[twitter-dev] Re: id field is missing in status from streaming API frequently
It appears that you are treating status deletions as statuses.
-John Kalucki
http://twitter.com/jkalucki
Services, Twitter Inc.
On Jul 24, 3:18 pm, AJ Chen cano...@gmail.com wrote:
twitter streaming api has lots of statuses missing id?
the following exception appears almost continuously in my log. it indicates
the id field is missing in status from streaming API.
twitter4j.TwitterException: JSONObject[id] not
found.:{delete:{status:{id:2813410502,user_id:47157439}}}
twitter4j.TwitterException: JSONObject[id] not
found.:{delete:{status:{id:2812385903,user_id:54420955}}}
thanks,
-aj
--
AJ Chen, PhD
Co-Chair, Semantic Web SIG, sdforum.orghttp://web2express.org
Palo Alto, CA
[twitter-dev] Re: Updating the APIs authentication limiting policy
I would much rather have Twitter lock me out of my account for an hour then let some script kiddie brute force my password. Thanks Twitter. Abraham On Fri, Jul 24, 2009 at 14:51, Marco Kaiser kaiser.ma...@gmail.com wrote: I think Dewald's concern is very valid - and even though OAuth might solve it, the reality is that most (if not all) desktop and mobile apps are using Basic Auth today for various reasons, so if you implement this policy as described, there's a pretty high risk that many users can be locked out of twitter from their usual ways to access it. Also, again a reminder that AFAIK the last official status re: OAuth from Twitter was that it is still in beta, and therefore not recommended for production use - or has there been another announcement that I missed? Marco 2009/7/24 Doug Williams d...@twitter.com Well said Joshua. Dewald, you have identified the risk of using basic authentication. If your users being locked out due to malicious behavior, you should either implement further user-level rate limiting on your side or adopt OAuth. Are there any other glaring omissions in our thinking or should we proceed with this as our solution? Thanks, Doug On Fri, Jul 24, 2009 at 11:08 AM, Joshua Perryj...@6bit.com wrote: Jim's concern is valid, fortunately OAuth is immune to brute-force attacks once the access key has been issued to an application. For this reason alone I would urge people to switch to OAuth if at all possible. I would hope (and assume) that if login attempts for an account are locked out that a user would still be able to successfully use an already authorized OAuth driven application. Unfortunately allowing a successful un/pw login while an account is locked out even when the correct password is presented effectively bypasses the whole reason for a lockout in the first place, preventing brute-force password attempts. If an attacker used a dictionary or brute-force attack and the account was locked out after 15 attempts, then they could continue trying even though the system replied locked out; if they eventually sent the correct password it would just bypass the lockout and they would then know the correct password. Perhaps Twitter could implement a selective captcha, I know they are annoying but if executed properly it could be effective protection against brute-force and dictionary attacks. Say after 3 or 4 failed attempts without a captch the API would then include a captcha image URL in it's response that the application would then need to show to the person and include the user's response with the next authentication attempt as a header or POST variable. The site stackoverflow.com does this to great effect, if you create posts quicker than a certain threshold which a person would not exceed then they pop a captcha up, in the normal use of the site you will never see one; I've only hit two captchas in the last in the last 8 months using the site. Josh Dewald Pretorius wrote: Jim raised a huge weakness with the authentication rate limiting that could essentially break third-party apps. Anybody can try to add anybody else's Twitter account to a third-party app using an invalid password. If they do that 15 times with a Twitter account, the real owner of that Twitter account, who may have added his account a long time ago with the correct password, is locked out from using that app for an hour. I believe you will absolutely have to reset / remove the lock as soon as the Twitter account uses the correct password. On Jul 22, 4:58 pm, jim.renkel james.ren...@gmail.com wrote: My concern with this proposal is that it opens up denials of service, not to twitter.com, but to associated sites such as twitpic, or my site twxlate, among others For example, Lance Armstrong is a heavy user of twitpic. It is very easy for anyone to find Lance's twitter ID (@lancearmstrong), view his status updates, and see that he is a frequent user of twitpic. Now, someone that is unhappy with Lance, say one of George Hincapie's ardent fans that really believes that Lance was a significant contributor to George not winning the maillot jeune last Sunday, could go to twitpic, fail to login as Lance the requisite number of times, and deny Lance access to twitpic. Not only celebrities would or could be subject to such denials of service. I notice that @dougw occasionally uses twitpic! :-) One solution to this problem is to add to each twitter account another private ID. By default this private ID would be equal to the existing (public) ID (If not equal to the account's public ID, it would have to be unique among all twitter IDs, both public and private.). The public ID would be used just as the existing twitter ID is now: others would use it to follow, mention, DM, etc., the user. But the user MUST use their private ID
[twitter-dev] Re: id field is missing in status from streaming API frequently
To clarify what John said. Those reference existing statuses that users are
deleting and you are being notified to also delete them from you database.
On Fri, Jul 24, 2009 at 15:20, John Kalucki jkalu...@gmail.com wrote:
It appears that you are treating status deletions as statuses.
-John Kalucki
http://twitter.com/jkalucki
Services, Twitter Inc.
On Jul 24, 3:18 pm, AJ Chen cano...@gmail.com wrote:
twitter streaming api has lots of statuses missing id?
the following exception appears almost continuously in my log. it
indicates
the id field is missing in status from streaming API.
twitter4j.TwitterException: JSONObject[id] not
found.:{delete:{status:{id:2813410502,user_id:47157439}}}
twitter4j.TwitterException: JSONObject[id] not
found.:{delete:{status:{id:2812385903,user_id:54420955}}}
thanks,
-aj
--
AJ Chen, PhD
Co-Chair, Semantic Web SIG, sdforum.orghttp://web2express.org
Palo Alto, CA
--
Abraham Williams | Community Evangelist | http://web608.org
Hacker | http://abrah.am | http://twitter.com/abraham
Project | http://fireeagle.labs.poseurtech.com
This email is: [ ] blogable [x] ask first [ ] private.
[twitter-dev] Re: need help with OAuth library usage in cocoa app
Magic! Now it works.. thanks a ton Isaiah :) On Jul 25, 4:40 am, Isaiah supp...@yourhead.com wrote: Thank you for your prompt reply Isaiah, No worries. Glad to help out. Thank you also for the offer of emailing you directly, but I think your code is also precious for other newbie like me who want to know more about this OAuth thing :) It wasn't so easy to figure out a nice way to do this stuff. I figured it would be a good thing to share. Glad it's helped. About changing application type to be Browser, then there is Callback URL: box that I have to fill, which I have no idea what do I have to fill in there. But I tried it anyway, and same result. The callback URL can be any URL you like. After a successful authorization this is the URL that's loaded. You should specify the URL both in the twitter app setup and also as a constant in the YHTwitter.m file on line 25: #define kYHOAuthTwitterCallbackSuccessURL @http://twitter.com/; Through the browser delegate methods the client will know that the callback URL was reached and the authorization was granted by the user. This triggers the example app to close the browser window and save the OAuth info to the keychain. Although I would imagine you would want to change it to your own web site for practical reasons, leaving it ashttp://twitter.com/should *work* at least for testing purposes. Isaiah YourHead Software supp...@yourhead.comhttp://www.yourhead.com On Jul 24, 2009, at 1:06 PM, Fares Farhan wrote: Thank you for your prompt reply Isaiah, Thank you also for the offer of emailing you directly, but I think your code is also precious for other newbie like me who want to know more about this OAuth thing :) About changing application type to be Browser, then there is Callback URL: box that I have to fill, which I have no idea what do I have to fill in there. But I tried it anyway, and same result. Fares On Jul 24, 10:58 pm, Isaiah supp...@yourhead.com wrote: My example was built right as the pin code method was invented/ implemented in the API. So my example still uses the Browser method that doesn't require a pin code. If you go to your application settings page in twitter and set your Application Type to be Browser you should be good to go. As I understand it the PIN code was invented to help clients that couldn't detect if the browser had been sent to the success callback URL. However, my example doesn't have this issue. My example embeds the browser and communicates directly with it to determine when the callback URL is sent. This technique obviates the need for the pin code. I like to think of my example as a hybrid app -- neither purely a desktop client nor really a web app -- but a little bit of both in the right places. ;-) I've considered adding the pin code, but it seemed to further complicate an already challenging UI without adding any value. If you have any other issues with the example code, please feel free to email me directly. I'd be happy to help out. Isaiah YourHead Software supp...@yourhead.comhttp://www.yourhead.com On Jul 24, 2009, at 12:04 AM, Fares Farhan wrote: Dear Twitter developers, First, I apologize if I misplace the question. I've cloned Isaiah's git repository of his AOuth implementation from http://github.com/yourhead/OAuth_ObjC_Test_App/tree/master but I experienced an issue that after the web sheet closed, there is no place that I can put the PIN retrieved from the authentication result, or anywhere in the code that I need to pass the oauth_verifier parameter along with other params. the debugger said that ther is Operation could not be completed. (NSURLErrorDomain error -1012.) Thank you in advance for any response, Cheers, Fares
[twitter-dev] Re: Search / track term
No. If you are tracking 3 things, for example, the only way to determine which of those 3 terms matched, would be for you to search within the tweet for your terms and determine it yourself. On Fri, Jul 24, 2009 at 5:30 PM, Joseph northwest...@gmail.com wrote: If I'm tracking a hash tag (using the streaming API), will that hash tag (or search term), be returned as part of the JSON stream I'm receiving?
[twitter-dev] Re: id field is missing in status from streaming API frequently
John, thanks.
Yusuke, it may be a good idea for twitter4j library to exclude the deleted
statuses as they are received. currently, twitter4j throws an exception for
them, which is less informative. thanks.
-aj
On Fri, Jul 24, 2009 at 3:20 PM, John Kalucki jkalu...@gmail.com wrote:
It appears that you are treating status deletions as statuses.
-John Kalucki
http://twitter.com/jkalucki
Services, Twitter Inc.
On Jul 24, 3:18 pm, AJ Chen cano...@gmail.com wrote:
twitter streaming api has lots of statuses missing id?
the following exception appears almost continuously in my log. it
indicates
the id field is missing in status from streaming API.
twitter4j.TwitterException: JSONObject[id] not
found.:{delete:{status:{id:2813410502,user_id:47157439}}}
twitter4j.TwitterException: JSONObject[id] not
found.:{delete:{status:{id:2812385903,user_id:54420955}}}
thanks,
-aj
--
AJ Chen, PhD
Co-Chair, Semantic Web SIG, sdforum.orghttp://web2express.org
Palo Alto, CA
--
AJ Chen, PhD
Co-Chair, Semantic Web SIG, sdforum.org
http://web2express.org
Palo Alto, CA
[twitter-dev] e-book on internet Retailer
Hai, In this link containing tips for internet marketing http://netelixir.com/semfreshideas.html?2http://netelixir.com/semfreshideas.html?1 There is a direct url to the ebook also, http://netelixir.com/freshideas.html?2http://netelixir.com/freshideas.html Regards, Praveen Kumar .N
