[twitter-dev] Re: search.twitter.com JSONP random problem
I'm getting this suddenly, too, where it wasn't happening as late as
late yesterday evening (US Pacific time).
I'm using JSONP through jQuery, Firefox 5.0 as a client. Same problem
also with IE 8.0.
The following query, for example, just returned a JSON response *not*
wrapped in the callback function:
http://search.twitter.com/search.json?q=
%23scalarpp=100page=1callback=jQuery16105326658435454098_1310741955350
I changed the page number to 2, I then received a response with the
callback wrapper included once -- I tried the same query for page 2
agin and received a response *without* the callback wrapper.
This is intermittent problem is effectively disabling my software.
Brian Maso
On Jul 15, 3:38 am, Stefa acid...@gmail.com wrote:
Hello,
I've noticed a random problem in the latest days,
sometimes the jsonp data is returned without the callback function
for instance if you try this for 10
times,http://search.twitter.com/search.json?since_id=91770779645124609q=te...
it should always returns
a (
{
...
results: [ … ]
...}
)
instead sometimes it only returns the content but with NO callback
name
{
...
results: [ … ]
...
}
--
Have you visited the Developer Discussions feature on
https://dev.twitter.com/discussions yet?
Twitter developer links:
Documentation and resources: https://dev.twitter.com/docs
API updates via Twitter: https://twitter.com/twitterapi
Unsubscribe or change your group membership settings:
http://groups.google.com/group/twitter-development-talk/subscribe
[twitter-dev] Re: twitter search json api not working
I'm willing to be it is the same no callback wrapper problem reported in the thread titled search.json.com JSONP random problem. Brian Maso On Jul 14, 1:58 pm, Taylor Singletary taylorsinglet...@twitter.com wrote: What's not working about it? On Thu, Jul 14, 2011 at 1:40 PM, sahu sahu.vi...@gmail.com wrote: twitter search json api not working #twitter #search #api http://search.twitter.com/search.json?q=twitter -- Have you visited the Developer Discussions feature on https://dev.twitter.com/discussionsyet? Twitter developer links: Documentation and resources:https://dev.twitter.com/docs API updates via Twitter:https://twitter.com/twitterapi Unsubscribe or change your group membership settings: http://groups.google.com/group/twitter-development-talk/subscribe -- Have you visited the Developer Discussions feature on https://dev.twitter.com/discussions yet? Twitter developer links: Documentation and resources: https://dev.twitter.com/docs API updates via Twitter: https://twitter.com/twitterapi Unsubscribe or change your group membership settings: http://groups.google.com/group/twitter-development-talk/subscribe
[twitter-dev] Re: The New Twitter Developer Site
Congratulations on the new site! It looks great and really seems to be a big improvement in terms of topic organization. Brian Maso On Jul 11, 11:12 am, Arnaud Meunier arn...@twitter.com wrote: Today we're launching the new version of our Twitter Developer Site. With over 1,000,000 registered applications and 750,000 developers building on the platform today, we needed a new home to support the Twitter community better. So, we listened to everyone and gathered your ideas. The new site enhances communication channels, offers improved reference material and documentation, and will foster better interaction for everyone who visits it. As you dive in, you’ll notice a number of changes. We focused on tightly structuring the content so it is more easily discoverable and understandable. Because the site will have to change over time to serve the needs of our evolving community, we relaunched it using Drupal, which provides more flexibility, better tools than we had, and a large developer network with whom we can work more closely. The new dev.twitter.com also includes the following highlights: * Discussions - We need a place to talk with each other that gives us more functionality than we have now. So, it’s time to move away from the mailing list. In the new discussions section, you’ll find “Hot Topics” for the most popular conversations. And you can subscribe to the categories and threads that interest you most. Lastly, we created a “Dev Teatime” section to focus on a more social side of the community. We hope to get to know more of you through this feature, so please check it out.https://dev.twitter.com/discussions * Developer Blog - The new blog provides a place to learn about important API announcements, events, tips and how-tos, case studies on great apps, product insights, and more. We’ll be featuring content from a variety of different teams here at Twitter, as well as highlighting great applications built on the platform. We’ll also be featuring guest blog posts from community members.https://dev.twitter.com/blog * Better Documentation - The docs have better structure and searchability and should feel more intuitive. They are easier to update, so you’ll know you are always reading the most up to date information when you’re looking for what you need.https://dev.twitter.com/docs * Improved Apps Management - The new app manager has a streamlined design that provides more comprehensive information for your app.https://dev.twitter.com/apps * Enhanced Search - Powered by Apache Solr, searching the new dev.twitter.com also got a boost. We have a unified search engine with filters and expect results to be more relevant. Also, we can search the archive from our Groups mailing list.https://dev.twitter.com/search Most importantly, this is just a starting point and we really want to hear from everyone. We will continue to add new features based on what we need and what we learn. The site will be an ongoing effort and we’d love to hear your thoughts in the Feedback section of our new Discussions area. We hope everyone enjoys the new site and look forward to making this the best developer site it can be. Arnaud / @rno -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
Re: [twitter-dev] Secure handling of the consumer key and consumer secret
Thanks for your reply, Tom. Something I'm not certain of is why it matters if people know your app's key and secret. What's the worst that can happen? They would still need to carry out operations as a valid Twitter user. Presumably if someone were to use Tweetdeck to spam, it would be that user that would be blocked, not all Tweetdeck users in the world? On 21 June 2011 17:07, Tom van der Woerdt i...@tvdw.eu wrote: Twitter's own handling of keys varies. For example: Twitter for iOS has the keys plaintext in the code. The iOS5 library actually doesn't store them as plaintext, but encrypts the consumer key the same way as the consumer secret, which still means it's easy because once you have the consumer key (packet sniffing?) you'll know how to get the secret. (Believe me, it's almost easier than that.) Those are two examples of how NOT to do it. I can give you these tips: * Don't encode your consumer key or don't use the same algorithm as the secret. The consumer key is supposed to be non-secret information as it is also transmitted on the request. If you can get the decoded version of the consumer key and the encoded version of the consumer key, it's often easy to reverse-engineer the algorithm. * Write your encryption in a safe language. For example, Objective-C is *very* easy to use with a debugger. C++ however, is not. Write your hashing code in C++ (hashing code: getting the secret all the way up to doing the HMAC hash). Also try to avoid using system libraries for the HMAC: preferably implement it yourself. This will make it harder as the attacker won't know what to target. Of course, these won't really work with opensource applications, as everyone can get the keys. If you distribute your application under a GPL license, there isn't much you can do, as you're forced to share the code (which include the keys). There are currently two options for OS projects I can think of : * Route all your requests via an application on your server (the TweetDeck way: just redirect api.tweetdeck.com to api.twitter.com but sign the requests with your key. However: this *will* cause issues with POST requests, so you'll have to handle those on your sever which may cause some heavy load) * Have your users register an application on dev.twitter.com/apps. Tom On 6/21/11 5:08 PM, Brian Remmington wrote: What techniques are people using to keep their Twitter app's consumer key and consumer secret, um, secret? What lengths are you going to to make sure nasty people can't get at this information? I have a particular problem in that I want my app to be open source - does anyone have any experience of building open source apps that interact with Twitter (or other services that use OAuth)? Thoughts? Suggestions? Brian -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Re: Twitter search question
Hey Casey, Not all Tweets make it into our search index. For more information, check out this article on our help center: https://support.twitter.com/articles/42646. If you think your account has been affected, please fill out the form linked from the bottom of that page while logged in as the account, and we'll get back to you. Brian Sutorius Twitter API Policy On Jun 14, 9:06 am, Casey Wilson cas...@downlifesroad.com wrote: Hey all, I understand this is probably the wrong forum for this, but if you could point me in the right direction I'd be appreciative. We have a question for the search side of things. We've had some niche related sites using twitter for a long time now. Here within the last month or two we've noticed some of our accounts getting flagged as not being able to be emailed (Mail server went down) and since then accounts have been going completely missing from search. We were wondering who to talk to about this, as I've gotten a few emails on the issue and I don't know what to tell our users. Thanks! -Casey -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Re: Clarification on Geolocation TOS
Hi Johnathan, Sorry for any confusion. This policy item requires that if you cache Twitter geo data, it must be stored with the rest of the tweet from where it came (including tweet text). Hope that helps, Brian Sutorius Twitter API Policy On May 16, 9:41 am, Johnathan Rush rus...@gmail.com wrote: I'm working with a research group at the Ohio State University that is interested in using tweets to study communication. Our project is made up of sociologists and geographers, and we are particularly interested in looking at social networks and the space-time context of discussions. We want to be sure not to violate the terms of service, specifically: 4. You will not attempt or encourage others to: E. use or access the Twitter API to aggregate, cache (except as part of a Tweet), or store place and other geographic location information contained in Twitter Content. We want to use locations, and would like to know what steps can we take to avoid violating the TOS. Would any of these measures below or some combination of them satisfy the requirements? - Not storing tweet ID - Not storing user ID - Not storing full 140-character status, only whether our topics of interest were mentioned - Generalize precise geolocations to a coarser level (Census tract/ neighborhood/county) Hopefully I haven't overlooked an answer to this question elsewhere. I found another post here asking for clarification (http://goo.gl/ hArk9), so it looks like clarification could benefit others, as well. If we need to ask for an exception to the TOS, where should we direct our application? Thanks, Johnathan Rush @rushgeo PhD student in Geography -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Re: ToS and redistribution of aggregate analysis results
Hey Amaç, Since the dataset you plan to distribute does not include Twitter content directly from the API, you can totally post it for public consumption. We allow tweet IDs to be shared in datasets like these, so if it would help fellow researchers to compare your results to the original corpus, you can also attach a list of the tweet IDs from your data set (just not their full tweet text or the tweet objects). Thanks, Brian Sutorius Twitter API Policy On May 16, 12:27 pm, amacinho a...@herdagdelen.com wrote: Hello, I have been using Twitter API for research purposes and created an ngram dataset of a tweet corpus that I have collected over the time. I want to make this dataset public for research purposes so other researchers may carry out their own studies without having to create a similar corpus. I read the ToS and didn't see any explicit statement that forbids such an action. I just want to be sure that my interpretation is correct. Could anyone tell me more about this? The dataset I plan to share is a collection of frequently-used ngram phrases and their frequencies in my corpus. I don't plan to keep phrases longer than 5 words. For instance, a sample of the file I plan to make public is below: drinking a glass of wine 233 drinking a cup of coffee 398 drinking poison and waiting for 10 drinking a tea without sugar 98 In this case the phrases are 5-grams (they all consist of 5 words/ tokens) and the number bext to them is the number of times they are observed in my corpus. As far as I can tell I am not redistributing the content of tweets because these samples contain common phrases that are already used commonly in daily language and I am merely releasing their frequency in a sample of tweets. Thanks in advance for your thoughts. Amaç Herdağdelen -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Re: App got switched to 'read-only' access, now can't switch back to 'read-write'
If you're not editing your application at http://dev.twitter.com/apps try there also. This issue can happen on the older app-edit pages on the main twitter.com domain. Brian Twitter API Policy On May 11, 6:26 am, Damon Parker cartmet...@gmail.com wrote: Try with a different browser to see if it is caching the old page for some reason. On Wednesday, May 11, 2011 at 2:11 AM, Jason Ling wrote: I edited my app, added a picture and saved, and now it's changed to 'read only' access. I edit again and select 'read write' and save - and it still says 'read only' on the next screen! (application details) I try to create a new app, select 'read write' and save, and it still says 'read only'! Any ideas anyone? -- Twitter developer documentation and resources:https://dev.twitter.com/doc API updates via Twitter:https://twitter.com/twitterapi Issues/Enhancements Tracker:https://code.google.com/p/twitter-api/issues/list Change your membership to this group:https://groups.google.com/forum/#!forum/twitter-development-talk -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Re: Is This Possible: People who mention Term A, also frequently mention Term B
The term you should Google on is Bayesian Analysis, which is basically a fancy term assigned to the task of figuring out the probability of event X when you know that a related event Y took place. If you set up your DB correctly, you can use normal SQL to perform ad hoc Bayesian analysis of Twitter data, but it takes some work to set it all up well. For what you describe, it seems you would be able to get the stats you want by: - Collecting tweets mentioning Term A - Count # of that population which mentions Term B, divide by total number of tweets mentioning Term A, multiply by 100 If you need to be able to perform ad hoc queries, then you need to cache the stream of all tweets you might be interested in (using the firehose, or Streaming API), indexing it on all terms you might possibly be interested in. Brian Maso On May 10, 9:06 am, Stephen Corby corby.step...@gmail.com wrote: Hey There, My first post, so go easy on me! I do social media strategy and monitoring freelance work, so I've recently dipped my toes into programming with Python and the Twitter API to build some simple applications for clients to perform some basic custom functions queries without having to dish out tons of money for some of the expensive social media monitoring tools out there. Gotta say, I'm loving this so far and I'm able to build little one-off applications for clients. Anyways, I have a client that is interesting in figuring out the following: Of twitter users who mention Term A, what are the additional terms that they frequently mention. Sort of like eCommerce systems that show users who bought A, also frequently purchased B, C and D. the output I'm hoping for is something along these lines: Of users who mentioned apples, 50% mentioned oranges, 25% mentioned lemons, etc... I know I'll eventually have to write some code to filter out certain words that are irrelevant, but just trying to figure out the framework of getting the right data from the API right now. This has reached the limit of my understanding of the twitter API thus far, and just want to see if anyone knew if this is possible. No need to explain fully, but anything to point me in the right direction would be greatly appreciated-- I'm sure I can eventually figure it out (with enough time), but wanted to make sure I wasn't wasting my time trying to do something that is beyond the capabilities of the API. Much appreciated! Steve -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Re: Someone can help me in this situation ?
Hello, please send an email to a...@twitter.com -- from the email address used with your old Twitter account if possible -- and include as much information about these apps and accounts as you can. We'll work with you from there. Brian Sutorius Twitter API Policy On May 6, 6:27 pm, anirudha gupta anir...@gmail.com wrote: I have twitter account who i used for create a new application twitter. suddenly i revoke access of all application i used to access twitter so i deleted the account on twitter. now i don't know what happen with the first application i register in twitter. i try to access them in twitter and i am able to use the key of application that's means twitter not deleted them with the delete the account last. i recreate the account on twitter and now how i can access the old registered application i make last time. -- Twitter developer documentation and resources: https://dev.twitter.com/doc API updates via Twitter: https://twitter.com/twitterapi Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list Change your membership to this group: https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Re: What's the best practices when creating a mobile app as an extension to a web app in regards to oauth?
Bess, we do recommend that you register a separate application for each platform to avoid user confusion (for example, if a user revokes an app's access from their account settings, even with the intent to just revoke access to an Android version and not a Windows version or web client, they still revoke access across all apps). If you choose to keep the same application for all versions, that's up to you. Since you can't register more than one application with the same name, you'll need to use a clarifier with each registration name (like Twitter for iPhone, Twitter for Android, and so on). Brian On May 5, 5:21 pm, Bess bess...@gmail.com wrote: Hi Brian, Could you explain and clarify the policy on web/mobile? Use case: Build iOS app first. Depot to Android. Add web app after both iOS and Android app are released How many Twitter apps do I have to create? Can I keep them the same name for the same startup? On May 5, 11:45 am, Brian Sutorius bsutor...@twitter.com wrote: We recommend separate application registrations for each platform (http://support.twitter.com/articles/79901) and this is the approach we take (web, Twitter for iPhone, Twitter for Android, and so on). You may not use the exact same name across multiple applications, however. Brian Sutorius Twitter API Policy On May 5, 9:01 am, YCBM youcannotb...@gmail.com wrote: Hi All, When an existing web app that is setup and registered on Twitter decides to launch a mobile extension, what are the best practices involved here with oauth? Are there may be benefits of registering a new app on Twitter with all new API key Consumer Secret/Key than what you are using for the web app? Does it provide any more security in any way to have both the web app and mobile app using separate keys? If so, can you register a mobile app with the same name as one that exists (assuming you own both)? We'd like the source name of status updates to come from the same name as our web app/brand if possible. That may not be possible, not sure. Any guidance is greatly appreciated. Best, YCB -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: What's the best practices when creating a mobile app as an extension to a web app in regards to oauth?
We recommend separate application registrations for each platform (http://support.twitter.com/articles/79901) and this is the approach we take (web, Twitter for iPhone, Twitter for Android, and so on). You may not use the exact same name across multiple applications, however. Brian Sutorius Twitter API Policy On May 5, 9:01 am, YCBM youcannotb...@gmail.com wrote: Hi All, When an existing web app that is setup and registered on Twitter decides to launch a mobile extension, what are the best practices involved here with oauth? Are there may be benefits of registering a new app on Twitter with all new API key Consumer Secret/Key than what you are using for the web app? Does it provide any more security in any way to have both the web app and mobile app using separate keys? If so, can you register a mobile app with the same name as one that exists (assuming you own both)? We'd like the source name of status updates to come from the same name as our web app/brand if possible. That may not be possible, not sure. Any guidance is greatly appreciated. Best, YCB -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: App Changed to Inactive - What To Do?
Send an email to a...@twitter.com from the address listed on your Twitter account and we'll be happy to help you. Brian Sutorius Twitter API Policy On Apr 23, 9:39 am, yolandapadilla yolandapadi...@gmail.com wrote: My app stopped working this week after humming along for over a year. Trying to figure out what's up. Have not changed anything on my end. Just discovered in my registered apps page that my app is tagged inactive. It's a low use app. I have no emails or contact from Twitter saying I have done anything wrong or what I should do. Just bam no longer active. Don't know who to contact and am wondering what to do next because I would like to fix whatever issue is amiss and fly right (so to speak). Thanks. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Search API bug? (strange results)
I wanted to get some counts for a number of domains, so I wrote a
simple script to perform a search with a domain name and loop through
all of the pages, counting up the number of results. This works fine
usually, but I have discovered that occasionally, the API returns
rather strange results:
?rpp=100q=disney.com
?page=2max_id=61422857900670976rpp=100q=disney.com
?page=3max_id=61422857900670976rpp=100q=disney.com
?page=4max_id=61422857900670976rpp=100q=disney.com
?page=5max_id=61422857900670976rpp=100q=disney.com
content:
{results:[],max_id:-1,since_id:0,results_per_page:15,page:
1,completed_in:
1.532267,since_id_str:0,max_id_str:-1,query:disney.com}
You'll notice that the results array is empty, the max_id is set to
-1, the results_per_page doesn't match what I specified, and the page
is set to 1 (and in that example it should have been page 5).
If I retry the URL, it usually works, so I made a work-around to
detect a max_id=-1 and retry the API call, but this seems like an
actual bug in the API (and the work-around will inflate the rate-
limit).
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Twitter Search API - Questions Regarding Scaling Out
On Apr 13, 10:28 am, Corey Ballou ball...@gmail.com wrote: I'm still looking for a community leader answer on this one. On Apr 11, 5:50 pm, Corey Ballou ball...@gmail.com wrote: Thanks for the reply, I appreciate it. I have concerns regarding the streaming APIs, which mainly concern the following: * usage of logical OR when using locations * firehose limitations * the user’s location field is not used to filter tweets * increased application complexity for parsing the resulting stream of data back out into individual searches I know that the Search API is not Twitter's preferred choice, but it's currently returning the best applicable results for my application. It's also worth noting that the API recently received a drastic improvement to speed which should theoretically relax the strain on the API: http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faste... I guess I'm mainly interested in knowing whether @twitterapi will allow me to use the Search API in the manner I indicated above? Essentially I would be willing to guarantee the application worker nodes handles 420 rate limiting errors accordingly while still supporting multiple twitter accounts and searches. On Apr 11, 1:05 pm, M. Edward (Ed) Borasky zn...@borasky- research.net wrote: I don't see an answer here, but I'll tell you how *I* would go about implementing this: 1. Switch to the Streaming API. Using Search in an application puts a strain on Twitter's servers and makes it difficult to Twitter to manage capacity. That's why it's rate-limited and why the rate limits aren't publicly disclosed. 2. If your application is a desktop application, use User Streams. If it is a server, use User Streams on a desktop or the low-frequency free access to Streaming on a server to prototype and develop. Your target for a server will be Site Streams, but that's in closed beta at the moment IIRC. 3. *Concurrently with development*, your business development / sales / marketing / planning people, or yourself, if it's a one-person shop, should be negotiating with Twitter for access to Site Streams, I'm assuming an agile development methodology - customer-in-the-loop - and one of the parties that needs to be in the loop is Twitter for Site Streams. You simply *can't* build an at-scale Twitter application without direct business discussions with Twitter! On Mon, Apr 11, 2011 at 8:14 AM, Corey Ballou ball...@gmail.com wrote: I tried speaking with Ryan Sarver directly, but he's forwarding me here to the community advocates to answer. I believe this answer will need to come top down from Twitter, as it's your rate limiting that I'm most worried about. I have a technical question for all of you in regards to the Search API as I want to maintain full compliancy. Currently, the old Search API implementation (albeit slower) provides a fuller result set and allows for more flexibility in the types and combinations of searches allowed. The manner I have developed my application would allow for a number of daemonized worker instances running on different IP addresses to make calls to the search API on behalf of the stored OAuth credentials to avoid rate limiting issues. I had a conversation with the Pluggio developer in which he stated Twitter had threatened to shutdown his application if he didn't switch to a different implementation of the Search API. The problem indicated was that he was performing searches for multiple Twitter accounts, which is exactly my use case. Site streams does not make as much sense for my application given the search queries I wish to perform and the necessity for logical AND operations on geo-location. Do you foresee any problems with my current method of using different IP addresses to stay under the rate limit? I'm trying to stay in full compliance with Twitter's TOS and would love to find the most applicable and API friendly solution. I know headway is being made with Twitter's new search implementation so I would like to stay ahead of the curve and not get myself stuck in a box. I still need a method for polling for new search results (say, every 30 minutes, dependent upon the pricing plan) for non-logged in users. Below is a scaled down representation of how I'm currently handling searches to help you decide the best plan of action: 1) Searches are performed on a rolling queue basis, say one search every thirty minutes. There can be a finite number of searches per Twitter user (say 5 searches per Twitter account). There can be any number of Twitter accounts. 2) Search results are stored locally for retrieval by a javascript AJAX long-poller every minute to check for frequent changes. 3) When a user visits the search results page and filters
[twitter-dev] Re: Twitter Search API - Questions Regarding Scaling Out
While the Streaming API may not provide processed results to you in the way that search queries can (logical ORs, etc.), it's a more scalable solution for returning a lot of Tweets. Our search system can rate limit queries if they become too computationally expensive (in addition to the normal query limit), so continuing to add parameters to the query up front rather than doing this processing yourself may cause you to keep running into limits. Ultimately, circumventing the limits put in place by our APIs is not allowed by our API ToS, and building your architecture this way just to get around the defaults is something we strongly discourage. If you keep being rate limited, you should think about re-factoring your prioritization strategy. Can you go into a little more detail about what your application does? We might be able to guide you towards a mix of Streaming API and search queries that gets you what you need but stays within the rate limits. Brian Sutorius Twitter API Policy On Apr 13, 10:28 am, Corey Ballou ball...@gmail.com wrote: I'm still looking for a community leader answer on this one. On Apr 11, 5:50 pm, Corey Ballou ball...@gmail.com wrote: Thanks for the reply, I appreciate it. I have concerns regarding the streaming APIs, which mainly concern the following: * usage of logical OR when using locations * firehose limitations * the user’s location field is not used to filter tweets * increased application complexity for parsing the resulting stream of data back out into individual searches I know that the Search API is not Twitter's preferred choice, but it's currently returning the best applicable results for my application. It's also worth noting that the API recently received a drastic improvement to speed which should theoretically relax the strain on the API: http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faste... I guess I'm mainly interested in knowing whether @twitterapi will allow me to use the Search API in the manner I indicated above? Essentially I would be willing to guarantee the application worker nodes handles 420 rate limiting errors accordingly while still supporting multiple twitter accounts and searches. On Apr 11, 1:05 pm, M. Edward (Ed) Borasky zn...@borasky- research.net wrote: I don't see an answer here, but I'll tell you how *I* would go about implementing this: 1. Switch to the Streaming API. Using Search in an application puts a strain on Twitter's servers and makes it difficult to Twitter to manage capacity. That's why it's rate-limited and why the rate limits aren't publicly disclosed. 2. If your application is a desktop application, use User Streams. If it is a server, use User Streams on a desktop or the low-frequency free access to Streaming on a server to prototype and develop. Your target for a server will be Site Streams, but that's in closed beta at the moment IIRC. 3. *Concurrently with development*, your business development / sales / marketing / planning people, or yourself, if it's a one-person shop, should be negotiating with Twitter for access to Site Streams, I'm assuming an agile development methodology - customer-in-the-loop - and one of the parties that needs to be in the loop is Twitter for Site Streams. You simply *can't* build an at-scale Twitter application without direct business discussions with Twitter! On Mon, Apr 11, 2011 at 8:14 AM, Corey Ballou ball...@gmail.com wrote: I tried speaking with Ryan Sarver directly, but he's forwarding me here to the community advocates to answer. I believe this answer will need to come top down from Twitter, as it's your rate limiting that I'm most worried about. I have a technical question for all of you in regards to the Search API as I want to maintain full compliancy. Currently, the old Search API implementation (albeit slower) provides a fuller result set and allows for more flexibility in the types and combinations of searches allowed. The manner I have developed my application would allow for a number of daemonized worker instances running on different IP addresses to make calls to the search API on behalf of the stored OAuth credentials to avoid rate limiting issues. I had a conversation with the Pluggio developer in which he stated Twitter had threatened to shutdown his application if he didn't switch to a different implementation of the Search API. The problem indicated was that he was performing searches for multiple Twitter accounts, which is exactly my use case. Site streams does not make as much sense for my application given the search queries I wish to perform and the necessity for logical AND operations on geo-location. Do you foresee any problems with my current method of using different IP addresses to stay under the rate limit? I'm trying to stay
[twitter-dev] Re: The thinking behind not drawing attention to Unfollows?
For a little clarification, this policy item was added to our API
Terms of Service with the release of our User Streams and Site Streams
products. Both streams deliver negative events such as unfollows and
unfavorites as distinct objects in the streams, so apps can adjust in
real-time. This policy is intended to prevent the broadcast of these
events to the end-user as notifications that the event happened.
In general, reporting who has unfollowed User A back to User A through
derivative methods (such as comparing their current follower list to a
cached version) is discouraged, but not prohibited. That same user
could come to the same conclusions through a similar method. Following
the spirit of this policy item, though, you may not broadcast these
kinds of events to other users (i.e. you may not show User C that User
B unfollowed User A).
As always, if you have any questions about our policies, you can email
a...@twitter.com.
Brian Sutorius
Twitter API Policy
On Apr 11, 4:47 pm, Whonew haag.j...@gmail.com wrote:
I just wanted to make clear that I was in no way questioning the rule.
I was just curious about the reasoning behind it, from Twitter's POV.
I, of course, came to the same logical conclusion that you did, Nick.
That it was simply to maintain a positive atmosphere and avoid
contention.
Thanks for your thoughtful replies.
- John
On Apr 9, 8:51 pm, nickmilon nickmi...@gmail.com wrote:
The intentions behind the rule is good, but what about the following
list of applications (and many more) that do not respect the TOS ?
http://mashable.com/2010/08/09/track-twitter-unfollowers/
happy coding :-)
Nick
On Apr 9, 5:05 am, Nicholas Chase nch...@earthlink.net wrote:
From a user perspective, I think it's good to know that you can
unfollow someone without them noticing, so you don't hurt their
feelings. The last thing that Twitter wants is to be linked to hard
feelings between people.
But that's just my opinion. YMMV, but I wouldn't be surprised if that
were the reason.
Nick
On 4/8/2011 9:57 PM, Whonew wrote:
Could someone from the Twitter staff go into some detail about why the
Terms of Service stress not drawing attention to user's Unfollows?
I have no particular interest in doing so; but I have been struggling
to figure out why as I'm certain that many users would like to know
without jumping through hoops.
Thanks a lot!
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] FireFix JSON parsing error? id != id_str in FireFox/jQuery?
Just noticed that on a whole lot of tweets I am retrieving through the
Twitter Search API that the tweet id numeric value is not equal to
the id_str value when JSON is turned in to JavaScript objects.
For example, the following JSON was just recently retrieved from
Twitter Search API using search term #scala -- note that both is
and id_str values evaluate to 54960878524710913 (numeric and string,
respectively):
{
from_user_id_str: 65831,
profile_image_url: http://a3.twimg.com/profile_images/
1257814978/eightbit-0e7493d6-6ebd-45ae-a5a9-145b181f1971_normal.png,
created_at: Mon, 04 Apr 2011 17:37:49 +,
from_user: torbjornvatn,
id_str: 54960878524710913,
metadata: {
result_type: recent
},
to_user_id: null,
text: RT @skskytteren: Had my first day as a professional
#scala #lift developer today. This is going to be so #great:-D,
id: 54960878524710913,
from_user_id: 65831,
geo: null,
iso_language_code: en,
to_user_id_str: null,
source: lt;a href=quot;http://twitter.com/quot;
rel=quot;nofollowquot;gt;Twitter for iPhonelt;/agt;
}
When I parse this JSON in FireFox, however, I get different values of
id and id_str in the resultant JavaScript object:
val jsobj = //...the JSON above
console.log(jsobj.id); // prints incorrect value 54960878524710910
console.log(jsobj.is_str); // prints correct value 54960878524710913
Anyone got an idea what's going on here? Recently I've noticed a few
different web pages that with links to individual tweets failing,
including my own web application. I'm almost positive the source of
the problem is this incorrect parsing/interpretation of numeric
values.
Anyone seen anything similar happen?
And can anyone confirm/deny the issue on their local system using the
JSON above?
Brian Maso
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Introducing Web Intents
Developers, users, and journalists are finding more creative ways to use Tweets on the web to leverage the power of the network to spread news. In the past it’s been difficult to make these Tweets interactive, requiring you to write an OAuth app simply to attach Reply, Retweet, and Favorite actions to Tweets. Today we’re releasing a simple new addition to the API called Web Intents that makes it possible to make Tweets that you display on the web interactive. Web Intents provide popup optimized flows for all the ways you interact with Tweets and users on Twitter: Tweet, Reply, Retweet, Favorite, and Follow. The new tool makes it possible for users to interact with Twitter content in the context of your site, without leaving the page or having to authorize a new app just for the interaction. Web intents are mobile friendly and easy to implement. For example, here’s how you add Reply, Retweet, and Favorite links to a specific Tweet: script type=text/javascript src=http://platform.twitter.com/widgets.js /script pa href=http://twitter.com/intent/tweet?in_reply_to=51113028241989632 Reply/a/p pa href=http://twitter.com/intent/retweet?tweet_id=51113028241989632 Retweet/a/p pa href=http://twitter.com/intent/favorite?tweet_id=51113028241989632 Favorite/a/p Detailed documentation is available at http://dev.twitter.com/pages/intents To see Web Intents in action check out Wordpress.com’s great tool for quoting Tweets in blog posts: Twitter Blackbird Piehttp://en.support.wordpress.com/twitter-blackbird-pie/. Here's a post that uses their tool to quote @jack's Tweets about our 5 year anniversary http://techcrunch.com/2011/03/13/twitters-beginning/. We’ve also added these standard Tweet actions to our timeline widgetshttps://twitter.com/about/resources/widgetsthat are used all over the web. We’ve also updated the display guidelineshttp://dev.twitter.com/pages/display_guidelines with some suggestions on how to make your Tweets actionable, and made the standard Reply, Retweet and Favorite icons available for downloadhttps://dev.twitter.com/pages/image-resources . Cheers, Brian Ellin Product Manager, Platform http://twitter.com/brianellin -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] beginner help needed with oAuth and xAuth
http://jaanus.com/post/1451098316/understanding-the-guts-of-twit On Sat, Mar 5, 2011 at 2:33 AM, Amrit bunkde...@gmail.com wrote: Hello everyone here in talks. I want to create signing url without using any library or api. I tried a lot according to http://dev.twitter.com/pages/auth website but I couldnot success. Can I get any nice tutorial regarding signing url for twitter? Thanks -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Re: Confused
well i was trying to get commets without just retweeting From: L. Mohan Arun mar...@gmail.com To: Twitter Development Talk twitter-development-talk@googlegroups.com Sent: Sat, February 12, 2011 7:00:16 PM Subject: [twitter-dev] Re: Confused Can you tell us exactly what it is that you are looking for? This is the Twitter API development group where we discuss issues faced by developers. - Mohan -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Re: Is there going to be another Chirp?
I have one question? what is DM? From: M. Edward (Ed) Borasky zn...@borasky-research.net To: twitter-development-talk@googlegroups.com Sent: Sat, February 12, 2011 12:25:27 PM Subject: Re: [twitter-dev] Re: Is there going to be another Chirp? On Sat, 12 Feb 2011 11:29:09 -0500, Brainewave Consulting i...@brainewave.com wrote: On Feb 7, 2011, at 5:25 PM, M. Edward (Ed) Borasky wrote: On Mon, 07 Feb 2011 15:25:59 +0100, Tom van der Woerdt wrote: I'd prefer London or some other West-European city. I'm guessing it will be in SFO, given how closely the Twitter team worked with the developers last year. They can't fly a few hundred folks to NYC or London. The question is *where* in SFO - Fort Mason wasn't particularly well suited for a barcamp-style conference with breakout sessions, wifi, etc. The noise level / acoustics were unacceptable. True enough, but Twitter is a global application - the developer conference should go global too! Rio! Twitter's huge in Brasil, and Facebook is conspicuously absent there! -- http://twitter.com/znmeb http://borasky-research.net A mathematician is a device for turning coffee into theorems. -- Paul Erdős -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk The fish are biting. Get more visitors on your site using Yahoo! Search Marketing. http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Where to send questions re: terms of service?
Hi Miles, I work on the API Policy team and monitor the messages at a...@twitter.com. We'd be happy to answer any questions you have about our policies or a specific feature you're thinking of. Brian Sutorius On Feb 9, 11:56 am, Taylor Singletary taylorsinglet...@twitter.com wrote: Hi Miles, While this public forum is great if you want to discuss the terms themselves with others, if you want to privately discuss API terms with Twitter, it's best to send a message to a...@twitter.com -- it might take a bit for you to get a response but the policy team will get your inquiry. That said, it's best to steer clear of anything explicitly prohibited in the terms and to follow the shoulds as closely as possible. Taylor @episod http://twitter.com/episod - Taylor Singletary - Twitter Developer Advocate On Wed, Feb 9, 2011 at 11:39 AM, Miles Parker milespar...@gmail.com wrote: Hi, I've got a case where I'm not sure whether a potential use would conflict with terms of service. I'd rather not get into details on public forum ;) but I can if this is the only place for it. But I'm wondering if there is someone or somewhere to ask questions? i.e. re: If you are doing something prohibited by the Rules, talk to us about whether we should make a change or give you an exception. -- Who is us? thanks, Miles -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: ~25% loss rate Streaming API vs. Search API
Hi Matt, Thanks for the explanation. I will file the bug report. I'd like to hear more about the sample size. I've read through the Streaming API docs a lot, and I haven't come across anything specific about the rate limits. Where can I read more? Brian Maso On Jan 10, 5:24 pm, Matt Harris thematthar...@twitter.com wrote: Hey Brian, When you use the Streaming API filter method we will stream to you all the Tweets which match your track terms - up to your allowed sample size. What this means is over the course of a sampling window we apply your track terms to the full firehose, and then return as many results as your sample rate allows. If you exceed your allowed sample size we will return a 'rate_limited' response containing the total number of matched Tweets missed. When matching track terms we apply the 'track' keywords to the raw Tweet text. This is different to the Search API which applies the track terms to the raw Tweet text plus the expanded URL. (The Streaming API doesn't expand URLs because it would delay the delivery of the Tweet). The issue you are describing is not caused by sampling limits or reduced subsets, but is instead due to a retweet parsing issue our engineers are looking into. What appears to be happening is the Streaming API is trying to match against the truncated RT version of the Tweet instead of the original Tweet text. If you file this in our issue tracker we can let you know when the issue is resolved. The issue tracker can be found here: http://code.google.com/p/twitter-api/issues/list Best, @themattharris Developer Advocate, Twitterhttp://twitter.com/themattharris On Mon, Jan 10, 2011 at 4:48 PM, Brian Maso br...@blumenfeld-maso.comwrote: Sounds consistent with what I've been seeing. Where did you get your impression of how the streaming API is optimized? I am having a hard time finding any authoritative documentation describing what the powers that be at Twitter *intend* to be included in the stream (as opposed to what they actually *implemented*, which may differ from intentions for a variety of reasons). If what you say is true, it kind of limits to use-cases of the streaming API to a far narrower set than what one would think by reading the Streaming API documentation. There's one section of the documentation that attempts to describe how to implement a system that utilizes the streaming API and avoids missing any tweets. Obviously if the stream of tweets is already a reduced subset, then it doesn't matter very much if you miss a few. Brian Maso On Jan 9, 4:06 pm, Bess bess...@gmail.com wrote: Streaming API is build by Twitter while Search API is build by Startup Summize acquired by Twitter. Search API is rate-limited. If you just use Twitter search feature, you may see everything. Using Search API to display API returned results is limited by your developer API. Streaming API may not show everything b/c it is optimized on the content based on its logarithm. On Jan 9, 2:29 pm, Brian Maso br...@blumenfeld-maso.com wrote: What I did is opened up three separate normal browser tabs in Firefox, each using the Twitter search web interface to search for three different hashtags (#ces, ces11, and nfl -- examples of three tags that should have decent ongoing traffic). At the same time I have an application capturing tweets from the same three hashtags using the streaming API (filter.json? q=#ces,#ces11,#nfl, with appropriate URL encoding). Irregardless of the amount of time, the streaming application captured about 25% fewer tweets. Detailed analysis of the tweet IDs captured by the browsers vs. those captured by the standalone application retrieving tweets via the streaming API verified that there were tweets delivered through the browsers that did not appear through the streaming API. There were no tweets delivered through the streaming API that did not also appear in the set of tweets delivewred through the browsers. I would love it if anyone else would try a similar experiment and report back results. Maybe I'm doing something wrong, or maybe this is an anomaly, or maybe the streaming API just doesn't capture as much -- impossible for me to say. I note that the streaming API documentation doesn't claim an intent to match accuracy with the search API (nor vice versa). At this point I'm thinking to get the greatest accuracy I should be collecting tweets from *both* APIs. Brian Maso On Jan 7, 5:08 pm, Bess bess...@gmail.com wrote: This is hard to believe. Streaming API is an approved API that should not have any limit. It should give you everything without any limit. On the other hand Search API has rate-limitation. Did you use any filter? On Jan 6, 9:42 pm, Brian Maso br...@blumenfeld-maso.com wrote: Hi All, Using the Streaming API
[twitter-dev] Re: ~25% loss rate Streaming API vs. Search API
Sounds consistent with what I've been seeing. Where did you get your impression of how the streaming API is optimized? I am having a hard time finding any authoritative documentation describing what the powers that be at Twitter *intend* to be included in the stream (as opposed to what they actually *implemented*, which may differ from intentions for a variety of reasons). If what you say is true, it kind of limits to use-cases of the streaming API to a far narrower set than what one would think by reading the Streaming API documentation. There's one section of the documentation that attempts to describe how to implement a system that utilizes the streaming API and avoids missing any tweets. Obviously if the stream of tweets is already a reduced subset, then it doesn't matter very much if you miss a few. Brian Maso On Jan 9, 4:06 pm, Bess bess...@gmail.com wrote: Streaming API is build by Twitter while Search API is build by Startup Summize acquired by Twitter. Search API is rate-limited. If you just use Twitter search feature, you may see everything. Using Search API to display API returned results is limited by your developer API. Streaming API may not show everything b/c it is optimized on the content based on its logarithm. On Jan 9, 2:29 pm, Brian Maso br...@blumenfeld-maso.com wrote: What I did is opened up three separate normal browser tabs in Firefox, each using the Twitter search web interface to search for three different hashtags (#ces, ces11, and nfl -- examples of three tags that should have decent ongoing traffic). At the same time I have an application capturing tweets from the same three hashtags using the streaming API (filter.json? q=#ces,#ces11,#nfl, with appropriate URL encoding). Irregardless of the amount of time, the streaming application captured about 25% fewer tweets. Detailed analysis of the tweet IDs captured by the browsers vs. those captured by the standalone application retrieving tweets via the streaming API verified that there were tweets delivered through the browsers that did not appear through the streaming API. There were no tweets delivered through the streaming API that did not also appear in the set of tweets delivewred through the browsers. I would love it if anyone else would try a similar experiment and report back results. Maybe I'm doing something wrong, or maybe this is an anomaly, or maybe the streaming API just doesn't capture as much -- impossible for me to say. I note that the streaming API documentation doesn't claim an intent to match accuracy with the search API (nor vice versa). At this point I'm thinking to get the greatest accuracy I should be collecting tweets from *both* APIs. Brian Maso On Jan 7, 5:08 pm, Bess bess...@gmail.com wrote: This is hard to believe. Streaming API is an approved API that should not have any limit. It should give you everything without any limit. On the other hand Search API has rate-limitation. Did you use any filter? On Jan 6, 9:42 pm, Brian Maso br...@blumenfeld-maso.com wrote: Hi All, Using the Streaming API, I'm noticing about a 25% loss rate when tracking multiple hashtags vs. using the good old Search API. I'm fouind it hard to believe this is true, so I tested over and over, but I keep getting the same results. The Streaming API just seems to not provide a fair number of tweets. Note that I have the lowest rate limit with the Streaming API -- perhaps highest rate limits have lower loss rates. Has anyone else noticed the rate loss Streaming vs. Search API? Or am I on crack? Does the loss rate get lower with the higher Streaming API account limits? Brian Maso -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: ~25% loss rate Streaming API vs. Search API
What I did is opened up three separate normal browser tabs in Firefox, each using the Twitter search web interface to search for three different hashtags (#ces, ces11, and nfl -- examples of three tags that should have decent ongoing traffic). At the same time I have an application capturing tweets from the same three hashtags using the streaming API (filter.json? q=#ces,#ces11,#nfl, with appropriate URL encoding). Irregardless of the amount of time, the streaming application captured about 25% fewer tweets. Detailed analysis of the tweet IDs captured by the browsers vs. those captured by the standalone application retrieving tweets via the streaming API verified that there were tweets delivered through the browsers that did not appear through the streaming API. There were no tweets delivered through the streaming API that did not also appear in the set of tweets delivewred through the browsers. I would love it if anyone else would try a similar experiment and report back results. Maybe I'm doing something wrong, or maybe this is an anomaly, or maybe the streaming API just doesn't capture as much -- impossible for me to say. I note that the streaming API documentation doesn't claim an intent to match accuracy with the search API (nor vice versa). At this point I'm thinking to get the greatest accuracy I should be collecting tweets from *both* APIs. Brian Maso On Jan 7, 5:08 pm, Bess bess...@gmail.com wrote: This is hard to believe. Streaming API is an approved API that should not have any limit. It should give you everything without any limit. On the other hand Search API has rate-limitation. Did you use any filter? On Jan 6, 9:42 pm, Brian Maso br...@blumenfeld-maso.com wrote: Hi All, Using the Streaming API, I'm noticing about a 25% loss rate when tracking multiple hashtags vs. using the good old Search API. I'm fouind it hard to believe this is true, so I tested over and over, but I keep getting the same results. The Streaming API just seems to not provide a fair number of tweets. Note that I have the lowest rate limit with the Streaming API -- perhaps highest rate limits have lower loss rates. Has anyone else noticed the rate loss Streaming vs. Search API? Or am I on crack? Does the loss rate get lower with the higher Streaming API account limits? Brian Maso -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Names already taken
This error message means that the application name has indeed already been registered. The Twitter username and application name spaces are separate. We don't have a public directory of all registered applications, but if you own the registered trademark for the application you're trying to register, we can help you out. Check out http://support.twitter.com/articles/328848 for more information. Brian Sutorius Twitter API Policy On Jan 7, 10:18 am, Jim jimk...@gmail.com wrote: When you try to give your app a name at dev.twitter.com and get the this name is already taken message does this mean that there's already an app with that name? Or a Twitter user with that name? And does the app name have to be unique across the sets of both usernames and app names? Finally, is there any way to find out if a taken app name is actually being used? I tried Googling for my taken app name and can't find a Twitter app by that name. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] ~25% loss rate Streaming API vs. Search API
Hi All, Using the Streaming API, I'm noticing about a 25% loss rate when tracking multiple hashtags vs. using the good old Search API. I'm fouind it hard to believe this is true, so I tested over and over, but I keep getting the same results. The Streaming API just seems to not provide a fair number of tweets. Note that I have the lowest rate limit with the Streaming API -- perhaps highest rate limits have lower loss rates. Has anyone else noticed the rate loss Streaming vs. Search API? Or am I on crack? Does the loss rate get lower with the higher Streaming API account limits? Brian Maso -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Unable to tweet with leading *
I suspect you might have stumbled on to some control signal mechanism within Twitter's backend. There might be special words that you can send that somehow control aspect's of twitter's message routing and buffering backend when sent this way. Alternatively there's some pattern matcher inside Twitter's backend (looking for special patterns in tweet text, such as retweet patterns or @mention patterns) that hiccups on this pattern. I like the first option more because it excites the imagination, but the second may actually be more likely. Brian Maso On Dec 30, 9:04 am, Adam Green 140...@gmail.com wrote: Interesting. I did some more tests here. I can only get this error with a single asterisk followed by a single word. A single asterisk followed by two or more words is accepted. Also, when I enter a tweet with just a single asterisk and no other characters, I don't get an error, but the tweet is not put into my account. It is just ignored. This set of errors is reproduced when I try it with multiple accounts. On Thu, Dec 30, 2010 at 11:36 AM, Adam Green 140...@gmail.com wrote: When I try entering a tweet through Twitter.com that starts with an asterisk, I get an error of Sorry! We did something wrong, and the tweet does not get sent. The same thing happens if I use Tweetdeck. To try this, send the following tweet: * test There is no problem using a single asterisk anywhere else in a tweet that I can find. -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker:http://code.google.com/p/twitter-api/issues/list Change your membership to this group:http://groups.google.com/group/twitter-development-talk -- Adam Green Twitter API Consultant and Trainerhttp://140dev.com @140dev -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] OAuth 401 Unauthorized failure
Hi all.
I am working on a web app for Twitter using the Tweepy Python API. My
code has been working for a month, but at some point this week, it
stopped working. Now when I request the access token from the
verification code, I get a 401: Unauthorized failure.
I did see from Matt's post yesterday that Tweepy was using the wrong
URLs. (It was using http://twitter.com/oauth instead of
http://api.twitter.com/oauth.) I fixed that problem and I'm still
seeing the problem.
I see from everyone else who is reporting this that you want to see
the basestring and Authorization header, so here's what I'm using:
URL: http://api.twitter.com/oauth/access_token
Headers: {'Authorization': 'OAuth realm=, oauth_nonce=58011146,
oauth_timestamp=1292624570, oauth_signature_method=HMAC-SHA1,
oauth_consumer_key=bSf2ywUggWN1w7Yx5FtvJg,
oauth_verifier=8bKG5j8sbzKIkycG61MxtpznR9dOx4L1WQe7rxZ0,
oauth_version=1.0,
oauth_token=8bKG5j8sbzKIkycG61MxtpznR9dOx4L1WQe7rxZ0,
oauth_signature=tOqtyU6E%2FulE8xiYBQRzuv2RMog%3D'}
Does anyone see a problem here?
Thank you!
Brian
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: OAuth 401 Unauthorized failure
I was hoping that URL is the basestring. Apparently not.
What do you mean by basestring? I'm using the Tweepy Python library
to hide the details of this from me, but I'm digging through there to
instrument it so I can send you what you want.
Brian
On Dec 17, 2:37 pm, Matt Harris thematthar...@twitter.com wrote:
Hi Brian,
A couple of questions so we can investigate:
1. What was the response body content we returned with the 401?
2. What was the basestring you used?
Best
@themattharris
Developer Advocate, Twitterhttp://twitter.com/themattharris
On Fri, Dec 17, 2010 at 2:30 PM, Brian Gunn gun...@gmail.com wrote:
Hi all.
I am working on a web app for Twitter using the Tweepy Python API. My
code has been working for a month, but at some point this week, it
stopped working. Now when I request the access token from the
verification code, I get a 401: Unauthorized failure.
I did see from Matt's post yesterday that Tweepy was using the wrong
URLs. (It was usinghttp://twitter.com/oauthinstead of
http://api.twitter.com/oauth.) I fixed that problem and I'm still
seeing the problem.
I see from everyone else who is reporting this that you want to see
the basestring and Authorization header, so here's what I'm using:
URL:http://api.twitter.com/oauth/access_token
Headers: {'Authorization': 'OAuth realm=, oauth_nonce=58011146,
oauth_timestamp=1292624570, oauth_signature_method=HMAC-SHA1,
oauth_consumer_key=bSf2ywUggWN1w7Yx5FtvJg,
oauth_verifier=8bKG5j8sbzKIkycG61MxtpznR9dOx4L1WQe7rxZ0,
oauth_version=1.0,
oauth_token=8bKG5j8sbzKIkycG61MxtpznR9dOx4L1WQe7rxZ0,
oauth_signature=tOqtyU6E%2FulE8xiYBQRzuv2RMog%3D'}
Does anyone see a problem here?
Thank you!
Brian
--
Twitter developer documentation and resources:http://dev.twitter.com/doc
API updates via Twitter:http://twitter.com/twitterapi
Issues/Enhancements Tracker:
http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: OAuth 401 Unauthorized failure
Okay--how about this for a basestring:
http://api.twitter.com/oauth/access_token?oauth_nonce=62391593oauth_timestamp=1292628636oauth_signature_method=HMAC-SHA1oauth_consumer_key=bSf2ywUggWN1w7Yx5FtvJgoauth_verifier=zzp3G0k6zoVBI11iCMMGtcpxfhks92vlsXxxZOwAqhsoauth_version=1.0oauth_token=zzp3G0k6zoVBI11iCMMGtcpxfhks92vlsXxxZOwAqhsoauth_signature=LuZt%2FwyR1OAYxO%2F%2BLNRouhP%2Fg%2Fc%3D
I'm still trying to get the text of the response. At the moment, it
looks like it's buried in another Python library somewhere because the
401 error causes an exception. Hopefully I'll be able to get that
shortly.
Brian
On Dec 17, 2:40 pm, Brian Gunn gun...@gmail.com wrote:
I was hoping that URL is the basestring. Apparently not.
What do you mean by basestring? I'm using the Tweepy Python library
to hide the details of this from me, but I'm digging through there to
instrument it so I can send you what you want.
Brian
On Dec 17, 2:37 pm, Matt Harris thematthar...@twitter.com wrote:
Hi Brian,
A couple of questions so we can investigate:
1. What was the response body content we returned with the 401?
2. What was the basestring you used?
Best
@themattharris
Developer Advocate, Twitterhttp://twitter.com/themattharris
On Fri, Dec 17, 2010 at 2:30 PM, Brian Gunn gun...@gmail.com wrote:
Hi all.
I am working on a web app for Twitter using the Tweepy Python API. My
code has been working for a month, but at some point this week, it
stopped working. Now when I request the access token from the
verification code, I get a 401: Unauthorized failure.
I did see from Matt's post yesterday that Tweepy was using the wrong
URLs. (It was usinghttp://twitter.com/oauthinsteadof
http://api.twitter.com/oauth.) I fixed that problem and I'm still
seeing the problem.
I see from everyone else who is reporting this that you want to see
the basestring and Authorization header, so here's what I'm using:
URL:http://api.twitter.com/oauth/access_token
Headers: {'Authorization': 'OAuth realm=, oauth_nonce=58011146,
oauth_timestamp=1292624570, oauth_signature_method=HMAC-SHA1,
oauth_consumer_key=bSf2ywUggWN1w7Yx5FtvJg,
oauth_verifier=8bKG5j8sbzKIkycG61MxtpznR9dOx4L1WQe7rxZ0,
oauth_version=1.0,
oauth_token=8bKG5j8sbzKIkycG61MxtpznR9dOx4L1WQe7rxZ0,
oauth_signature=tOqtyU6E%2FulE8xiYBQRzuv2RMog%3D'}
Does anyone see a problem here?
Thank you!
Brian
--
Twitter developer documentation and resources:http://dev.twitter.com/doc
API updates via Twitter:http://twitter.com/twitterapi
Issues/Enhancements Tracker:
http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: OAuth 401 Unauthorized failure
Okay. Very interesting! I'll take a look at that.
Thank you!
Brian
On Dec 17, 3:32 pm, Matt Harris thematthar...@twitter.com wrote:
OK. One other thing I've just noticed looking over your Authorization header
is that your oauth_verifer and oauth_token are set to the same value. This
is unusual and not something I would expect.
Double check your code used the oauth_verifier we return to you in the
callback URL and the token is the one you received in the request_token
request.
Best
@themattharris
Developer Advocate, Twitterhttp://twitter.com/themattharris
On Fri, Dec 17, 2010 at 2:40 PM, Brian Gunn gun...@gmail.com wrote:
I was hoping that URL is the basestring. Apparently not.
What do you mean by basestring? I'm using the Tweepy Python library
to hide the details of this from me, but I'm digging through there to
instrument it so I can send you what you want.
Brian
On Dec 17, 2:37 pm, Matt Harris thematthar...@twitter.com wrote:
Hi Brian,
A couple of questions so we can investigate:
1. What was the response body content we returned with the 401?
2. What was the basestring you used?
Best
@themattharris
Developer Advocate, Twitterhttp://twitter.com/themattharris
On Fri, Dec 17, 2010 at 2:30 PM, Brian Gunn gun...@gmail.com wrote:
Hi all.
I am working on a web app for Twitter using the Tweepy Python API. My
code has been working for a month, but at some point this week, it
stopped working. Now when I request the access token from the
verification code, I get a 401: Unauthorized failure.
I did see from Matt's post yesterday that Tweepy was using the wrong
URLs. (It was usinghttp://twitter.com/oauthinsteadof
http://api.twitter.com/oauth.) I fixed that problem and I'm still
seeing the problem.
I see from everyone else who is reporting this that you want to see
the basestring and Authorization header, so here's what I'm using:
URL:http://api.twitter.com/oauth/access_token
Headers: {'Authorization': 'OAuth realm=, oauth_nonce=58011146,
oauth_timestamp=1292624570, oauth_signature_method=HMAC-SHA1,
oauth_consumer_key=bSf2ywUggWN1w7Yx5FtvJg,
oauth_verifier=8bKG5j8sbzKIkycG61MxtpznR9dOx4L1WQe7rxZ0,
oauth_version=1.0,
oauth_token=8bKG5j8sbzKIkycG61MxtpznR9dOx4L1WQe7rxZ0,
oauth_signature=tOqtyU6E%2FulE8xiYBQRzuv2RMog%3D'}
Does anyone see a problem here?
Thank you!
Brian
--
Twitter developer documentation and resources:
http://dev.twitter.com/doc
API updates via Twitter:http://twitter.com/twitterapi
Issues/Enhancements Tracker:
http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
--
Twitter developer documentation and resources:http://dev.twitter.com/doc
API updates via Twitter:http://twitter.com/twitterapi
Issues/Enhancements Tracker:
http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] List Statuses: 401 - Incorrect signature
Hey there, I've had a site up and running for quite some time now (July 2010) since Dec 14, and a little after 2:30 (Mountain) we stopped receiving tweets from that service call. After checking in on it today I discovered I was receiving a 401: Incorrect signature error. Here is my request /1/mdcpartnerssite/lists/executives/statuses.json? oauth_consumer_key=KQSwCfwSi2QEW9l6Yu8bgoauth_nonce=caf8db043f02f2612675cfc2b64755c1oauth_signature=hA6%2FA0VtOSyFAxk3Qa4InU %2FrAdo%3Doauth_signature_method=HMAC- SHA1oauth_timestamp=1292540218oauth_token=154351009-8yk6Cl0bqA9VzomDPrFsS2JmfnGgox1UHo6eZAOdoauth_version=1.0page=1per_page=200since_id=9996331253 Here is my base string GEThttp%3A%2F%2Fapi.twitter.com%2F%2F1%2Fmdcpartnerssite%2Flists %2Fexecutives%2Fstatuses.jsonoauth_consumer_key %3DKQSwCfwSi2QEW9l6Yu8bg%26oauth_nonce %3Dcaf8db043f02f2612675cfc2b64755c1%26oauth_signature_method%3DHMAC- SHA1%26oauth_timestamp%3D1292540218%26oauth_token %3D154351009-8yk6Cl0bqA9VzomDPrFsS2JmfnGgox1UHo6eZAOd%26oauth_version %3D1.0%26page%3D1%26per_page%3D200%26since_id%3D9996331253 Please let me know if there is any other information you need that would help in this manner. Did something change on twitter's end that required some more strict checking, that I might not be adhering to? I've also noticed a few other's mailing in about 401 issues. Any help would be greatly appreciated. Thank you! Brian Wigginton Interactive Developer Crispin Porter + Bogusky -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: How're Special Characters Treated in the Search API?
I'm not sure what are considered special characters either, but from what I can tell this particular case seems to be a combination of factors. For some reason, by using the +OR+ it seems to be matching anything that has a period character in it. However, if you do the queries separately, they seem to work (albeit the second one doesn't return any data). http://search.twitter.com/search.atom?q=Battle+for+Bean+Streetpage=1rpp=11 http://search.twitter.com/search.atom?q=Battle+for+Bean+St.page=1rpp=11 I don't know if doing separate queries will work for you, but maybe that's another option. On Dec 14, 2:52 pm, Howard h0w4...@gmail.com wrote: Hi Everyone, The following query, which includes a period, returns nonsensical results:http://search.twitter.com/search.atom?q=%22Battle+for+Bean+Street%22+... Removing the period fixes it. I've looked athttp://apiwiki.twitter.com/w/page/22554756/Twitter-Search-API-Method:... and can't find anything about how special characters are treated. Which characters are considered special and what're the rules regarding them? Thanks all! -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Search API cURL strangeness
So it must be based on the IP Address and the UserAgent...
I have changed the UserAgent, so it works now, but I don't
particularly like this solution. It would be nice to know what
happened, and what caused it, so I can try to prevent it from
happening in the future.
On Dec 14, 11:24 am, Tom van der Woerdt i...@tvdw.eu wrote:
Tested it myself with :
tom-mbp:~ tom$ curl --user-agent PivotalVeracity/0.4
http://search.twitter.com/search.json?q=batteryoperatedcandles.netrp...;
Result :
{results:[],max_id:14696108638863361,since_id:9431322892177408,refresh_url:?since_id=14696108638863361q=batteryoperatedcandles.net,results_per_page:100,page:1,completed_in:0.006856,since_id_str:9431322892177408,max_id_str:14696108638863361,query:batteryoperatedcandles.net}
Seems to work fine... Getting exactly the same results when using the
default User Agent.
Tom
On 12/14/10 5:15 PM, Brian Medendorp wrote:
UserAgent is 'PivotalVeracity/0.4'
Here's the test script that helped me track down the problem:
[code]
?php
$timeout = 30;
$useragent = 'PivotalVeracity/0.4';
#$useragent = 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:
1.9.2.13) Gecko/20101203 Firefox/3.6.13';
$url = 'http://search.twitter.com/search.json?
q=batteryoperatedcandles.netrpp=100since_id=9431322892177408since=until=';
#$url = 'http://search.twitter.com/search.json?
q=carnationbreakfastessentials.comrpp=100since_id=since=2010-12-14until=';
#$url = 'http://search.twitter.com/search.json?
q=apple.comrpp=100since_id=since=2010-12-14until=';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_TIMEOUT, $timeout);
$content = curl_exec($ch);
if(curl_errno($ch))
{
print curl error: .curl_error($ch).\n;
print_r(curl_getinfo($ch));
}
print_r($content);
[/code]
On Dec 14, 11:03 am, Tom van der Woerdti...@tvdw.eu wrote:
And your UserAgent is?
Tom
On 12/14/10 5:02 PM, Brian Medendorp wrote:
I'm building an application that uses the search API to check for data
related to particular domains, and suddenly (within the last week or
so), I have started to experience a strange problem. Some of my
requests are coming back with a cURL error Empty reply from server,
but only when I am searching for a specific set of domains (all of the
other domains work fine).
I wrote a small test script to try and track down the problem, and it
seems that the UserAgent I am setting with cURL seems to be causing
the problem (or part of the problem). If I change the UserAgent to
anything else, I get a normal response.
I remember reading in the documentation that Twitter expects a unique
UserAgent for the application, so that's what I did, but that seems to
be causing problems. This seems like it's likely some sort of
blacklist problem, but I can't figure out why it would work in this
manner (only blocking a small subset of my queries, and not IP-based).
Here are some sample queries I am trying to cURL:
http://search.twitter.com/search.json?q=batteryoperatedcandles.netrp...
http://search.twitter.com/search.json?q=carnationbreakfastessentials
http://search.twitter.com/search.json?q=apple.comrpp=100since_id=s...
The first two don't work unless I change my UserAgent to something
else, but the last one works no matter what.
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: api@twitter not responding?
Hi Shachar, Sorry for the delay. I've located your ticket and will follow up with you shortly. Brian On Dec 13, 10:53 pm, Shachar shach...@gmail.com wrote: Hi, I'm a web app developer and I'm looking to get my track access level raised beyond the default of 400 keywords. I've tried to contact a...@twitter.com for over a week now, and I'm getting zero response from them. On their support pages, Twitter state that they answer their email within 72 hours... I'm looking for advice on how to get in touch with Twitter dev support. Best, Shachar -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Search API cURL strangeness
I'm building an application that uses the search API to check for data related to particular domains, and suddenly (within the last week or so), I have started to experience a strange problem. Some of my requests are coming back with a cURL error Empty reply from server, but only when I am searching for a specific set of domains (all of the other domains work fine). I wrote a small test script to try and track down the problem, and it seems that the UserAgent I am setting with cURL seems to be causing the problem (or part of the problem). If I change the UserAgent to anything else, I get a normal response. I remember reading in the documentation that Twitter expects a unique UserAgent for the application, so that's what I did, but that seems to be causing problems. This seems like it's likely some sort of blacklist problem, but I can't figure out why it would work in this manner (only blocking a small subset of my queries, and not IP-based). Here are some sample queries I am trying to cURL: http://search.twitter.com/search.json?q=batteryoperatedcandles.netrpp=100since_id=9431322892177408since=until= http://search.twitter.com/search.json?q=carnationbreakfastessentials.comrpp=100since_id=since=2010-12-14until= http://search.twitter.com/search.json?q=apple.comrpp=100since_id=since=2010-12-14until= The first two don't work unless I change my UserAgent to something else, but the last one works no matter what. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Search API cURL strangeness
UserAgent is 'PivotalVeracity/0.4'
Here's the test script that helped me track down the problem:
[code]
?php
$timeout = 30;
$useragent = 'PivotalVeracity/0.4';
#$useragent = 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:
1.9.2.13) Gecko/20101203 Firefox/3.6.13';
$url = 'http://search.twitter.com/search.json?
q=batteryoperatedcandles.netrpp=100since_id=9431322892177408since=until=';
#$url = 'http://search.twitter.com/search.json?
q=carnationbreakfastessentials.comrpp=100since_id=since=2010-12-14until=';
#$url = 'http://search.twitter.com/search.json?
q=apple.comrpp=100since_id=since=2010-12-14until=';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_TIMEOUT, $timeout);
$content = curl_exec($ch);
if(curl_errno($ch))
{
print curl error: .curl_error($ch).\n;
print_r(curl_getinfo($ch));
}
print_r($content);
[/code]
On Dec 14, 11:03 am, Tom van der Woerdt i...@tvdw.eu wrote:
And your UserAgent is?
Tom
On 12/14/10 5:02 PM, Brian Medendorp wrote:
I'm building an application that uses the search API to check for data
related to particular domains, and suddenly (within the last week or
so), I have started to experience a strange problem. Some of my
requests are coming back with a cURL error Empty reply from server,
but only when I am searching for a specific set of domains (all of the
other domains work fine).
I wrote a small test script to try and track down the problem, and it
seems that the UserAgent I am setting with cURL seems to be causing
the problem (or part of the problem). If I change the UserAgent to
anything else, I get a normal response.
I remember reading in the documentation that Twitter expects a unique
UserAgent for the application, so that's what I did, but that seems to
be causing problems. This seems like it's likely some sort of
blacklist problem, but I can't figure out why it would work in this
manner (only blocking a small subset of my queries, and not IP-based).
Here are some sample queries I am trying to cURL:
http://search.twitter.com/search.json?q=batteryoperatedcandles.netrp...
http://search.twitter.com/search.json?q=carnationbreakfastessentials
http://search.twitter.com/search.json?q=apple.comrpp=100since_id=s...
The first two don't work unless I change my UserAgent to something
else, but the last one works no matter what.
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Twitter talking with max/MSP
Hey Everyone, Right now I am trying to get twitter to talk with max/MSP and I am having trouble getting the API's for the specific keyword I plan on needing for the patch. Do I need to have a specific web app before I can get access to retrieve specific key word tweets. All it would do is just increment a number and not even copy or retweet anything. It is meant to just adjust a numeric value that in turn will adjust an arduino serial value I hope this makes sense. If not I would be will to try to explain it further. Please feel free to contact via email or on here. Cheers Brian Putz -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Question about TT's
Hi Emerson,
Please contact our communications team for more information about
this. You may reach them by filling out the form at
http://twitter.com/help/contact/make_press_inquiry .
Brian Sutorius
Twitter API Policy
On Oct 12, 6:44 pm, Emerson Damasceno emerson...@gmail.com wrote:
Hello there. Obviously this is not the proper place, but since it's being a
lot of talking in Brazil, have you heard anything about Twitter monitoring
the TT's for political reasons?
In Brazil some are saying the Hashtag #dilma13 was somehow pulled off the
TT's (it's a brazilian Candidate to Presidential Pools).
Also as a Journalist I wonder if that is somehow possible (since Twitter is
able to accept a promoted TT) but as far as I know, not Stop a Trending (if
really trending).
Anyway, thank you all again
Emerson
2010/10/12 D. Smith emai...@sharedlog.com
I noticed that the value of source field looks somewhat strange:
source:a href=\http://www.echofon.com/\; rel=\nofollow\Echofon
\/a,
Why in the world would you have an html string as a value and on top
of than why do you include the rel=nofollow tag?
This just looks wrong, not structured.
The right way whould have been to represent the source as an object
with fileds: name, url, like this:
source:{name : Echofon, url:http://www.echofon.com},
Usually you try to pre-parse everying for us, but in the case or
source, we have to do extra parsing to extract values of title and url
Will you fix this soon?
--
Twitter developer documentation and resources:http://dev.twitter.com/doc
API updates via Twitter:http://twitter.com/twitterapi
Issues/Enhancements Tracker:
http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
--
*Emerson
Damascenohttp://www.twitter.com/emersonanomiahttp://www.blog.opovo.com.br/bloganomia/*
*http://www.opovo.com.br/colunas/tecnologia/listagemmidiaesocial/*
*Tel: +55 85-8697 3224/ 85-3458 1977/ 11- 7356 9693*
*Nextel: 55*86*28199*
*Damasceno Associados*
*Shopping Aldeota 1620/1621*
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Ultimately send my twitter followers direct messages from my application
Our Automation Rules ( http://support.twitter.com/articles/76915 ) include guidelines for automating DMs when a user follows you. Specifically, we do not prohibit this behavior but we do not recommend it. The 250 DMs per day limit that Thomas mentioned is correct. Brian Sutorius On Oct 2, 11:25 am, Thomas Mango tsma...@gmail.com wrote: Yes, there's a limit of 250 direct messages per day according to:http://support.twitter.com/forums/10711/entries/15364 I'm not sure if there are any policies against automatically direct messaging someone when they follow you, but a 250/day would certainly prevent that at some point. I don't know the details of your application, but if you were only planning to send new followers a direct message, perhaps you can avoid asking them to follow you and sending them a direct message by just showing them what you wanted to message them when they come back from the OAuth authorization. -- Thomas Mango On Oct 2, 2010, at 1:12 PM, Dean Collins d...@cognation.net wrote: Thomas are there restrictions on what/how many direct messages can be sent? I haven't been paying attention with twitter for a while but I thought twitter banned automatic direct messages. Thanks in advance, Dean I think what you described is exactly right. You're looking for an app that users can authorize with using OAuth. Once they're redirected back to your site (part of the OAuth process), you can create a user account for them locally and ask them to follow your Twitter account. Because they've authorized your application, when they agree to follow you, you can use the /friendships/create API method on their behalf. Relevant API documentation: http://dev.twitter.com/pages/auth http://dev.twitter.com/doc/post/friendships/create Dialflow wrote: Hi: I was wondering if any one could suggest an elegant approach to ultimately sending direct messages to my Twitter followers from my application. I'd like people that join web site to do the following: From their member page on my site, I'd like for them to click a Twitter follow button, go to Twitter, follow me, then return to their member page on my site. After they do this, I want capture their twitter ID and associate it with their user account on my site so I can send them direct messages from my application. I'd really appreciate an elegant approach to solving this. I guess I'm looking for an answer like: Use oAuth to have the user authorize your app on Twitter, then redirect redirect back to your app, click a twittter follow button, and extract their Twitter ID from x_file and then My days of programming are way behind me so I hope that makes some sense. Thanks so much. Curtis -- Thomas Mango tsma...@gmail.com -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker:http://code.google.com/p/twitter-api/issues/list Change your membership to this group:http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker:http://code.google.com/p/twitter-api/issues/list Change your membership to this group:http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Desktop vs web apps
You *can* but we strongly recommend that you register a separate application on Twitter for each platform you operate on. http://support.twitter.com/articles/79901 Brian Sutorius Twitter API Policy On Sep 30, 6:25 am, Tom van der Woerdt i...@tvdw.eu wrote: Yes, you can. Tom On 9/30/10 3:21 PM, John Meyer wrote: Can I use the same tokens that I generated with a desktop application for a web application, or vice versa? -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] FYI: Undocumented response code from users/show
Grabbing some data from users/show, have received a unexpected
response code. Just thought I'd memorialize it here:
If the requested user account is suspended, the response code is
403. Twitter API docs only mention 403 being used for update rate
limits (like limiting how much an app updates users' statuses).
The full response I get actually is 403 with the following JSON
content:
{request:/1/users/show.json?screen_name=PapaRocks6,error:User
has been suspended}
Brian Maso
--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: #newtwitter and the API
include_entities Methods which return statuses support an include_entities parameter. When set to either true, t or 1, each tweet will include a node called entities,. This node offers a variety of metadata about the tweet in a discreet structure, including: user_mentions, urls, and hashtags. Entities is currently opt-in but will become defaulted to on. Is there any chance that this could be included in the search API? -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Privacy Policy Question
As a developer of an application open for public use, it is a best practice to offer your own privacy policy on your website or within your application. At the very least, you must be clear about how you will use your users' account data and/or take actions on their behalf. This is mirrored in our Developer Principles within the API Terms of Service [1]: Don't Surprise Users. Of course, there are applications that interface with Twitter's API without being open to public usage. Twitter's privacy policy [2] states that public account data is distributed through the service via the API. Brian Sutorius Twitter API Policy [1] http://dev.twitter.com/pages/api_terms [2] http://twitter.com/privacy On Sep 11, 2:54 am, Hermi aherma...@gmail.com wrote: The Twitter Privacy Policy says that 'developers must clearly disclose what they will be doing with data collected from users. Does anyone know how this works in practice with Twitter data? Do I need to include a privacy policy on my website telling saying that I use personal data from Twitter users? What if the Twitter users don't even know I am using their data and they never look at my website? If anyone knows the answer thanks! -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en
[twitter-dev] users/lookup POST read-only applications cannot post
I just started getting this error message within the past hour for my application. My application is read-only in the sense that it never updates a user profile or posts any tweets or anything. My application does, however, use the users/lookup.json API, and I use POST because I am requesting many profiles at once when I use the API. Does the Read-only indication on my application's profile mean I can never use POST? Anyone else seeing anything similar? Brian Maso -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en
[twitter-dev] Update on Twifficiency
Hi all, Over the past 24 hours, we've received some questions about the Twifficiency app, so we thought we'd use this as an opportunity to quickly share some information around our Developer Principles. For background, the Twifficiency app computes a Twifficiency score based on different aspects of your Twitter account and posts the score as a Tweet. While the developer included a disclaimer that these Tweets would be posted to Twitter, user feedback indicated that the text was too far down on the page to be noticed before proceeding. As a result, many users were surprised that their scores were being tweeted automatically. Which brings us to our Developer Principles, one of which is Don't surprise users. Specifically, we require developers to get users' permission before sending Tweets or other messages on their behalf. Allowing an application to access your account does not constitute consent for actions to automatically be taken on your behalf. Twifficiency violated this principle, so we suspended the app yesterday afternoon while we worked with the developer to make sure users were better informed about the application's actions and could control whether or not a Tweet would be posted. With these changes --which include a more prominent warning and a checkbox on the main page-- the application has been re-enabled. Our developer principles can be found in our API Terms of Service: http://dev.twitter.com/pages/api_terms Brian Sutorius API Policy
[twitter-dev] Re: A total novice to both Twitter web development
It is possible to use only HTTP POST to do this, but it's going to require several requests because you will need to authenticate the user (in this case, it sounds like that user is you, so you may only need to do this once and just keep your tokens) with OAuth first. Once you have the access_token you will be able to make a HTTP POST request to send your tweet, but it's not going to be pretty because you will need to send all of your credentials and sign the request manually (more on that here: http://hueniverse.com/2008/10/beginners-guide-to-oauth-part-iv-signing-requests/). You'll probably be a whole lot better off trying to use one of the OAuth libraries listed here: http://dev.twitter.com/pages/oauth_libraries Hope that helps! On Aug 9, 4:43 pm, Sashkoff sanya...@gmail.com wrote: Folks, as a total novice to Twitter web development altogether, I am relying on you goodwill and help: - I want to use HTTP only to post new (only) tweets from my personal webpage. After I have spent a fortune of time reading thru Twitter API and Wiki, I couldn't come to any conlcusion if this is all possible. So, may any one help me anwering the following: - is it possible to use only HTTP POST to send a new tweet to my twitter account? And, - how should such HTTP POST request look? Thanks in advance, Sashkoff :-)
[twitter-dev] Re: OAuth and a readonly app
I had to do the same thing for something I am working on, it's not very ideal, but it seems to be the only way to get the job done. There is supposed to be a second type of OAuth that allows that sort of one-way communication (basically the same thing but without the user's tokens), but it seems that no one has implemented that (and I can't remember the name they gave it). On Aug 9, 10:39 pm, russ.au russell.say...@gmail.com wrote: Hi all, Let's say I'm writing a read only app - you come to my website enter someones twitter name, and I give you some statistics about them. I can get all the stats I need by making anon calls to the REST api from my webserver. The API docs say Anonymous calls are based on the IP of the host and are permitted 150 requests per hour, where as OAuth calls are permitted 350 requests per hour. If my app gets popular enough I'd like to make as many calls as I can. What is the protocol here? Should I create a twitter account just for my app, take this account through the OAuth process, and use this account's access token for all my requests? Thanks, Russ
[twitter-dev] Re: 101 HELP: How to update status using Javascript/jSON?
Without any further information, I am going to guess that it's probably an OAuth issue. Unfortunately, I don't think that you will be able to use jQuery to do this. I'm betting that the API methods you are calling with jQuery don't require any OAuth authentication. I suppose it's possible that you could do all of the OAuth negotiation and signing with straight javascript (because you'll have to build the HTTP request yourself, instead of the very handy jQuery AJAX functions), but it's not going to be easy. You'll probably have much better luck using a server side scripting language (such as PHP, Perl, Phython, Ruby, etc) There is a list of OAuth libraries available here: http://dev.twitter.com/pages/oauth_libraries Apparently there is actually a Javascript OAuth library, but as they mention on that page you probably shouldn't use it (because among other things you'll likely end up exposing your Consumer token and secret to anyone looking at your source code). On Aug 8, 3:18 pm, Claudia cbern...@gmail.com wrote: HI all Just getting started with the Twitter API and simultaneously getting familiar with jSON. Does someone have an example of the best practice for updating a user status using Javascript/JQuery/jSON? I've successfully retrieved statuses and user information, but am struggling with the post side of things. Any help would be greatly appreciated! Thanks, Claudia
[twitter-dev] Good morning
I am having a hard time reading it all. Is there someone to teach me what to do next?
RE: [twitter-dev] Re: Will you guys help me here?
Omg. Its all so clear now! J'son. No, being an ass. -Original Message- From: Tom allerleiga...@gmail.com Sent: Friday, August 06, 2010 10:23 AM To: Twitter Development Talk twitter-development-talk@googlegroups.com Subject: [twitter-dev] Re: Will you guys help me here? On Aug 6, 11:06 am, Rushikesh Bhanage rishibhan...@gmail.com wrote: Hi guys, with a hope to get some sort of help here, I wanted to ask you guys, I am getting error while loading xml formatted data into object like: * Warning*: simplexml_load_string() [function.simplexml-load-stringhttp://www.honestfollowers.com/function.simplexml-load-string]: Entity: line 50: parser error : Entity 'rsaquo' not defined in * /home/ajit/public_html/honestfollowers.com/TwitterAPILibrary/mytwitter1.php*on line *486 Simplexml can't handle all (any?) entities. You can try using the full PHP XML parser (http://php.net/xml) or write your own. * Line 486 on mytwitter1.php is : $data = simplexml_load_string($xml); * *Due to this error I am not getting whole data of a user. * *Is it twitter overcapacity error? because this error comes usually when even my twitter homepage comes in a damaged state. It is possible - make sure to write a few lines of code to make sure you have some error handling. if this is not from twitter-side , what can be the problem(means like internet speed or anything)? PHP can be the problem. * *I am using rest API method and My Twitter Class PHP by Andres Artux Scheffer, Mozilla browser (v3.6.8). I will really appreciate your help guys. Thank you in advance. You're welcome. One more note before ending this message: You should consider using JSON instead of XML. It's easier to parse and requires less bandwidth. Tom
[twitter-dev] Get resolved URLs?
I can see that twitter itself must be resolving any shortened URLs somewhere, because if you search for a domain name (such as amazon.com), you get a bunch of results that don't seem to match until you resolve the shortened URL in the tweet and see that it points to the domain you searched for, which is fantastic! However, I am wondering if there is any way to get those resolved URLs from the API, or (better yet) if there is anyway that those URLs could be exposed in the search results themselves. Currently, I am resolving the URLs myself by requesting the URL and saving the resulting location, but that starts to take a while when there are a lot of results returned.
RE: [twitter-dev] Callback URL not authorized issue
I was able to click right in to the page from my black jack, which is good. Half the links in cyberspace don't usually come mobile windows based! -Original Message- From: Steve atif-nas...@hotmail.com Sent: Friday, August 06, 2010 1:56 PM To: Twitter Development Talk twitter-development-talk@googlegroups.com Subject: [twitter-dev] Callback URL not authorized issue I have implemented @Anywhere on a web-page. I have registered an app with Twitter and inserted the API key in the head section and both modules in the body section of my page. I am getting the following error: The provided callback url http://mymobileplanet.co.uk/iPhone.html is not authorized for the client registered to 'http:// www.mymobileplanet.co.uk'. I have tried several methods to somehow make it work but all in vain. I would appreciate if someone can visit my pages and see what the problem is or guide me how I can make it work.
[twitter-dev] Re: User protected account privacy - API terms
Hi Furkan, Public information is public. If someone without a Twitter account can view that information on a user's Twitter profile, or if the same information can be returned from an unauthenticated API call, it's considered public information and you may display it. Twitter does not require certain display conventions to indicate that the information comes from a protected account, but as you may notice, we use a lock icon on protected accounts. Brian Sutorius Twitter API Policy On Jul 11, 3:02 pm, Furkan Kuru furkank...@gmail.com wrote: I have read the terms of service (https://twitter.com/tos) and api rules. But it is not clear whether we can publish a protected account's profile information as shown in their profile page. (only screen_name, name, website, bio, follower, friends count) with a proper way as twitter specifies (i.e twitter icon, screen name) We will add a filter for protected accounts if we do not have right to display basic user information for protected users. -- Furkan Kuru
[twitter-dev] OAuth with Streaming API causes 500
We are working on adding OAuth support to the twitter stream gem (http://github.com/voloko/twitter-stream) using roauth (http:// github.com/maccman/roauth) to generate the oauth header. We have the library working fine with basic auth but as soon as we turn on OAuth we get 500 errors from the server. Here is the request we're getting the error with: GET /1/statuses/filter.json?track=miami HTTP/1.1 Host: stream.twitter.com User-Agent: TwitterStream Authorization: OAuth oauth_nonce=V60jgkOhLoKWvZyu7tmrM6LCSSiWfCxCcAQxUeGHREc, oauth_timestamp=1279049530, oauth_signature_method=HMAC-SHA1, oauth_consumer_key=[redacted], oauth_token=[redacted, oauth_signature=RGz8UfkGHPQ8shsQopOSFD3r3Qg%3D, oauth_version=1.0 Any thoughts? -Brian
[twitter-dev] Re: What's the approx. timescale for xAuth approval ?
There *is* currently a backlog with xAuth approvals, but we are working through them as quickly as we can. I know this access is crucial to development, but our response time may be as long as a week for the next couple days. Bear with us. :) If the email address you initially filed your request from is different from the email address associated with your Twitter account, your ticket may not show up in the http://support.twitter.com help center. Reply to me directly and I can look for it. Brian Sutorius On Jul 8, 8:50 am, DW dara...@gmail.com wrote: Hi folks, just wondering if there's a big backlog of xAuth approvals right now or how long the approval process is taking ? I submitted an app for approval early last week and haven't heard anything back other than the initial automated message. Noticing today that there appears to be two different support forums (help.twitter.com and support.twitter.com), I resubmitted the request to support. thinking that maybe help. was obsolete, but the new ticket is linking to a page with a Request not found error. Bottom line is I'm well behind now on testing my app and the client is wondering what the story is, but I can't even give them a date since I've heard nothing from Twitter. Any help or pointers would be great ! -DW. P.S. This forum seems to require a Gmail address, but our Twitter account is @Axonista
[twitter-dev] Re: Unfollows
Whether through automated software or by hand, aggressive follower churn is prohibited by our Twitter Rules: http://support.twitter.com/articles/18311 . See some of the bullet points under Spam for more detail. Thanks, Brian Sutorius Twitter API Policy On Jun 18, 5:44 pm, cdrecordings crownsdownrecordi...@gmail.com wrote: Using SOFTWARE to constantly churn followers in a repeated pattern of following and unfollowing will however risk suspension. What about following a certain number by hand, then unfollowing those who don't follow back by hand the next month or so?
[twitter-dev] Dev Portal Login
Is anyone else having problems logging into the dev portal? I keep getting directed to https://twitter.com/sessions with the message Sorry, that page doesn’t exist! -Brian Wigginton
[twitter-dev] Re: Dev Portal Login
Logging in via twitter.com then going to the portal site worked. Thanks Taylor! -Brian On Jun 15, 2:55 pm, Taylor Singletary taylorsinglet...@twitter.com wrote: Sorry for all the issues around this login -- I really want to get this login functioning correctly but we've had some system-wide changes recently that have made some elements of fixing this for reals though difficult. It's an incredibly basic issue that's overcomplicated by the particularities of our production environment, the interaction of SSL, and subdomains. I hope to have it fixed by the end of the week. For now -- login to twitter.com, then go to the portal. Taylor Singletary Developer Advocate, Twitterhttp://twitter.com/episod On Tue, Jun 15, 2010 at 12:41 PM, Brian Wigginton brianwiggin...@gmail.comwrote: Is anyone else having problems logging into the dev portal? I keep getting directed tohttps://twitter.com/sessionswith the message Sorry, that page doesn’t exist! -Brian Wigginton
RE: [twitter-dev] Re: link wrapping on the API
Dewald Pretorius wrote: StatusNet, blog, etc.). If the privacy argument carried any water, then bit.ly would be a far greater privacy threat than Twitter. I agree, and that's why I (and others) were working on solutions to get around bit.ly and other link shorteners. And, Twitter has already developed the ultimate solution to that privacy issue, but they just are refusing to let API developers use it. That is what is so incredibly frustrating. Regards, Brian
RE: [twitter-dev] Re: link wrapping on the API
If you do this, you will literally be forcing app developers to waste users time and money, especially over metered GRPS/3G connections. If the user can see the full URL, then why do they need to be protected any more than they are when they use any other service? If anything, you should be cutting through any and all redirects (shorteners) so that the application can show the final URL to the user and avoid multiple useless, latency-inducing redirects that reduce reliability and increase costs for end-users and network operators. Cutting through all the redirects would improve security AND improve on the users' privacy, instead of hurting it. And, what about the user's right to privacy? You're basically forcing every Twitter app to become spyware. Who wants to create spyware? No developers with a conscience-and I'm sure that includes you guys at Twitter. Please ask whoever's making these horrible decisions lately to reconsider at least this one. Sincerely, Brian Smith @BRIAN_ two things to note: the text of the returned status object doesn't have the original URL and instead it has a t.co URL, and the entities block now has a display_url attribute associated with it. what we're hoping is that with this data, it should be relatively easy to create a UI where you replace the http://t.co/s9gfk2d4in the text with the equivalent of a href=http://t.co/s9gfk2d4;http://dev.twitter.com/a this means the user would not see the t.co link, but we all can still provide the protection and gather data from the wrapped link. for the applications that don't choose to update, the t.co link will be shown (and the goal to protect users will be met). i just want to emphasize -- we really do hope that you all render the original URL, but please send the user through the t.co link. if you do choose to prefetch all the URLs on a timeline, then, when a user actually clicks on one of the links, please still send him or her through t.co. We will be updating the TOS to require you to check t.co and register the click.
RE: [twitter-dev] Re: link wrapping on the API
Sami wrote: I don't see how this feature could impact user privacy more than what it is right now. Today Twitter stores all links for all users and they can spy on them and the t.co shortner is not changing that :) Right now, Twitter can see all the links that users *post*, but they don't see which links users *click*. In order to implement this feature, Twitter has already built the framework that does all the hard work that applications need to protect users' privacy against (link-shortener) click-tracking. Twitter will be withholding that final URL from applications, preventing us (through the ToS) from implementing our own anti-click-tracking privacy measures. If, instead, they gave the final URL to the application and let the application use that URL, then applications could implement anti-click-tracking privacy measures with an even greater degree of privacy than they could by using a third-party service. In other words, instead of Twitter using their existing link-unshortening technology to let applications tell *fewer* companies what links your users are clicking on, they are using it to force applications to tell *more* companies what your users are clicking on. Only advertisers could build a privacy-improving technology and using it for the exact opposite purpose. :( http://mashable.com/2010/06/03/alex-payne-twitter-interview/ Regards, Brian Smith @BRIAN_
RE: [twitter-dev] Re: link wrapping on the API
I was basing my statement on the blog post, which indicated that at least some display URLs will be truncated: http://blog.twitter.com/2010/06/links-and-twitter-length-shouldnt.html A really long link such as http://www.amazon.com/Delivering-Happiness-Profits-Passion-Purpose/dp/044656 3048 might be wrapped as http://t.co/DRo0trj for display on SMS, but it could be displayed to web or application users as amazon.com/Delivering-. Will the application be doing the truncation from the full URL to the truncated one (amazon.com/Delivering-), or will the API? And, if the application really will get the full destination URL, then it is even more ridiculous to prevent them (through the ToS) from using it to improve the user's privacy, don't you think? Regards, Brian Right now, Twitter can see all the links that users *post*, but they don't see which links users *click*. In order to implement this feature, Twitter has already built the framework that does all the hard work that applications need to protect users' privacy against (link-shortener) click-tracking. Twitter will be withholding that final URL from applications, preventing us (through the ToS) from implementing our own anti-click-tracking privacy measures. If, instead, they gave the final URL to the application and let the application use that URL, then applications could implement anti-click-tracking privacy measures with an even greater degree of privacy than they could by using a third-party service. hey brian - just wanted to point out - the Twitter will be withholding that final URL from applications is NOT true at all. we are providing, as part of the entities the original URL back to the developers. stated another way - we are giving all the data back and we are not withholding the data. -- Raffi Krikorian Twitter Platform Team http://twitter.com/raffi
[twitter-dev] Saerch API and Twitter Live Search Results disparity
Last night I collected tweets through the search API for the hashtag #glossgreen, and got a sizeable number of tweets. This morning I did the equivalent thing through the search box on my Twitter homepage (the URL that appears in my browser is http:// twitter.com/#search?q=%23glossgreen), and got different results. More specifically, I found that there were a few users who's tweets appeared when doing the search through the search box in the browser who do not appear at all through the search API results. For example, the user @gloss had many tweets using the #glossgreen hashtag in the time period around 6-8 pm PDT 6/2 -- none of these appear in the twitter search results, but many appear in results through the twitter search box on my personal twitter homepage. I just re-performed both searches this morning to make sure this isn't a temporary issue, but got the same disparity, What expectation should I have about search API accuracy? Shold I expect the search API results to eventually repair, or is are the @gloss tweets permanently missing from the search API's database? I don't want to have to use multiple different APIs/screen-scrapes/ streams just to make sure I get accurate search results, but if that's what I have to do then please let me know. Brian Maso
RE: [twitter-dev] Problem with the API Console
Daniel wrote: Hello everyone, For example, I try to retrive the last tweets of a particular user I choose GET statuses/user_timeline with json protocol and in the parameters and values fields I set id and radiohead as the user screen name. The result I get is my own timeline. That is the default result when you set no parameters at all and the parameters don't show up in the generated request either. It seems that the console ignores the parameters, at least if the method is GET. I recommend using twurl instead. Twurl was easier to install than I expected and it seems to work great, even on Windows. Regards, Brian
RE: [twitter-dev] New opt-in API features available today, May 26th: entities, retweets in timelines, custom oauth_callback schemes
How are characters indexed in the indices values of entities? My guess would be that they are indexed as Unicode code points--not bytes—and that the indexes refer to the text before entity expansion (“amp;” - “”) is done. Is that correct? Like I mentioned in the previous thread, it would be very useful if we could use the entities construct when posting. First, it would help to mitigate problems when replying to somebody that has changed their username since the tweet you are replying to. Also, it would allow an application to clarify what is exactly in a URL. For example, “Check out http://example.org/foo.” should have the URL parsed without the trailing period, but currently Twitter considers the period to be part of the URL. From other observations it seems like Twitter has already built a short-link expansion service that is used internally. Will expanded links be exposed via this entities mechanism anytime soon? Regards, Brian From: Taylor Singletary Entities Raffi's already introduced the concept of entities to you in a previous post: http://bit.ly/boHXYv You can now retrieve entities for tweets by specifying a include_entities=true parameter to statuses/home_timeline, statuses/user_timeline, statuses/friends_timeline, and statuses/mentions API calls to receive additional per-tweet payloads dissecting parse-able elements from the tweet body like @mentions, links, and hashtags. It's really cool! Some examples of how entities are represented can be found here: http://dev.twitter.com/pages/tweet_entities
[twitter-dev] Why is UNIQLO LUCKY LINEに行列 tr ending?
Uniqlo created a (web?) app that allows people to enter a contest or get a discount by tweeting a message from their account; see [1]. (It also seems like they’re not using OAuth as the tweets are “via API,” but that’s not why I’m writing.) So many people are entering the contest that “UNIQLO LUCKY LINEに行列” has been the top trending topic for the last couple of days. I am curious about the future of this kind of marketing campaign. I have had people contact me about creating these kinds of apps-not so much “tweet to win” but “tweet to get a discount.” Are they going to be allowed indefinitely? Is it acceptable for stores to set up a kiosk at the cash register for users to sign into, in order to send a (pre-written) from their account? Are there any guidelines at all? Personally, I find these kinds of campaigns extremely annoying because they turn my friends into spammers. But, it seems like they are not excluded from even the new ToS. Regards, Brian [1] http://www.independent.co.uk/life-style/fashion/uniqlo-twitters-top-trending -topic-thanks-to-virtual-line-campaign-1982399.html
RE: [twitter-dev] Twitter OAuth Timestamps
This is known and expected behavior. There have been other threads about it in the last couple of weeks. If you get a 401 response, you should compare the Date header of Twitter's response to the current system time. If it is significantly off then you should warn the user so they can fix it and/or calculate the difference and add that offset to all your timestamps. More details are available in the mailing list archive. Regards, Brian -Original Message- From: twitter-development-talk@googlegroups.com [mailto:twitter- development-t...@googlegroups.com] On Behalf Of Eric Woodward Sent: Tuesday, May 25, 2010 7:40 PM To: Twitter Development Talk Subject: [twitter-dev] Twitter OAuth Timestamps I have confirmed a problem with xAuth/OAUth that I believe resides within Twitter OAuth implementation that has been a thorn in our side for a while. I say *believe* because I do not claim to know for sure, thus this post. I assume no one at Twitter will be inclined to do me any favours, but please answer for the sake of the users in general, and other developers in here that do a better job of not publicly expressing their opinions of what Twitter has been doing to its ecosystem. If a user's desktop time is off by a significant margin, say 30m, we have confirmed that a valid username/password via an xAuth request will fail. This has been very painful to track down since those working on Nambu tend to have the desktop time set correctly, and only a handful users complain legitimately, with credibility. This tweet started us on to a solution: http://twitter.com/imhassan/status/14639986090. It is not affecting just Nambu. I cant find anything in the OAuth specs to suggest this comparison to the actual time should take place, so I assume Twitter is just going ahead and comparing the submitted timestamp to the actual time, and rejecting the request (for perhaps a good reason), or it is a bug. We are getting a 401 on a valid request with an inaccurate timestamp. This issue is hinted at here: http://weblog.bluedonkey.org/?p=959. Anyway, we are putting a workaround in place, so if no one at Twitter responds, no worries, Nambu will work going forward. Other developers, be aware that this issue exists. This is very annoying to me because users with inaccurate time settings have tried to verify their accounts in Nambu, failed, and then use the official Twitter application for OSX (aka Tweetie), which works because it is still on HTTP Basic authentication, and declared Nambu to be broken. Twitter, please clarify which part of the process is indeed broken, and what you expect to see regarding timestamps on your end. I assume that by the time Twitter for OSX is updated to use xAuth you will have put a solution in place for this, or will at some point soon afterward as users complain. It would be nice if you outlined that solution for the rest of us when the time comes, so perhaps we can improve on what we have come up with. I apologize in advance if I missed something obvious in the docs somewhere. I am not an expert on OAuth by any means, and have not studied this issue per se. I have only been trying to resolve the issue for us to move on to something more important. Our OAuth implementation works fine otherwise. Well, as well as the rest of the Twitter API works, anyway. Cheers. --ejw Eric Woodward Email: e...@nambu.com
[twitter-dev] DHE ciphersuites disabled?
I noticed about a week ago that my application stopped working. Now I have tested it and it appears that api.twitter.com is now blocking DHE cipher suites such as TLS_DHE_RSA_WITH_AES_128_CBC_SHA, whereas previously these DHE cipher suites were working perfectly. The DHE cipher suites have a distinct security advantage over the non-DHE cipher suites like TLS_RSA_WITH_AES_128_CBC_SHA: in the event that Twitter's RSA private key is compromised (via hacking, a warrant, a court order, or other means), all the previous traffic encrypted with the non-DHE cipher suites can be decrypted, but the previous traffic encrypted with DHE cipher suites cannot be decrypted after the fact. My questions: 1. Is there any chance the DHE cipher suites can be re-enabled? 2. Can you provide any guarantees about which cipher suites will always be available? openssl s_client -host api.twitter.com -port 443 -tls1 -cipher DHE-RSA- AES128-SHA openssl s_client -host api.twitter.com -port 443 -tls1 -cipher DHE-RSA- AES256-SHA openssl s_client -host api.twitter.com -port 443 -tls1 -cipher AES128-SHA openssl s_client -host api.twitter.com -port 443 -tls1 -cipher AES256-SHA Thanks, Brian
RE: [twitter-dev] DHE ciphersuites disabled?
Thanks for the response, Bob. I know a fair amount about this kind of thing. Feel free to contact me you'd like any more information or suggestions on a good ephemeral Diffie-Hellman parameters and/or ways of mitigating the TLS handshake performance costs. Regards, Brian From: twitter-development-talk@googlegroups.com [mailto:twitter-development-t...@googlegroups.com] On Behalf Of Bob Lord Sent: Monday, May 24, 2010 7:43 PM To: twitter-development-talk@googlegroups.com Subject: Re: [twitter-dev] DHE ciphersuites disabled? We're talking about this internally. I'm cautiously optimistic. /B On Mon, May 24, 2010 at 12:43 AM, Brian Smith br...@briansmith.org wrote: I noticed about a week ago that my application stopped working. Now I have tested it and it appears that api.twitter.com is now blocking DHE cipher suites such as TLS_DHE_RSA_WITH_AES_128_CBC_SHA, whereas previously these DHE cipher suites were working perfectly. The DHE cipher suites have a distinct security advantage over the non-DHE cipher suites like TLS_RSA_WITH_AES_128_CBC_SHA: in the event that Twitter's RSA private key is compromised (via hacking, a warrant, a court order, or other means), all the previous traffic encrypted with the non-DHE cipher suites can be decrypted, but the previous traffic encrypted with DHE cipher suites cannot be decrypted after the fact. My questions: 1. Is there any chance the DHE cipher suites can be re-enabled? 2. Can you provide any guarantees about which cipher suites will always be available? openssl s_client -host api.twitter.com -port 443 -tls1 -cipher DHE-RSA- AES128-SHA openssl s_client -host api.twitter.com -port 443 -tls1 -cipher DHE-RSA- AES256-SHA openssl s_client -host api.twitter.com -port 443 -tls1 -cipher AES128-SHA openssl s_client -host api.twitter.com -port 443 -tls1 -cipher AES256-SHA Thanks, Brian
[twitter-dev] Re: How to register current Basic Auth application as OAuth application
Registering a basic-auth source parameter was not the same action as registering an application. It is possible that between the time you registered the SimplyTweet source parameter and now, someone else registered an application under the same name. If you own a trademark on SimplyTweet, you can follow the process at the lower half of http://twitter.zendesk.com/entries/18367 and our Policy team will be happy to help you with this. If not, please follow up on your ticket (for privacy reasons) and we'll look into it further. Thanks! Brian Sutorius On May 15, 10:39 pm, Hwee-Boon Yar hweeb...@gmail.com wrote: My Twitter app runs on iPhone (and has a server side component that user doesn't directly interact with). It has been running on Basic Auth for more than a year. I would like to register it as OAuth and migrated users over, i.e. running both in parallel under end June since not everyone will update their their version immediately. When I register an app with the same name, it says the name is already taken. I presume that's referring to the previous Basic Auth app? (or someone registered my app name - SimplyTweet). How should I proceed with this? I sent a support ticket, but one of us isn't understanding the other. Thanks.
RE: [twitter-dev] Re: oauth and embedded microcontrollers
Mr Blog wrote: For example, the current 'tweet' code binary is 18K bytes. If you can add oAuth in 100K bytes or less, that might work, but that one function would then still be bigger than the entire rest of the application. In fact, the entire file system ROM image, with all the binaries and data is 114K bytes. How large is your TLS stack and root CA certificate database? Regards, Brian
RE: [twitter-dev] parsing out entities from tweets (a.k.a. parsing out hashtags is hard!)
Raffi,
I have noticed that the API sometimes returns user ID's that are out of sync
with username. I think one case is where a Alice retweets Bob's tweet, and
then Bob changes his name to Charlie. When I try to reply to it, it doesn't
show up as in reply to to original tweet because the reply contains @Bob
instead of @Charlie. It would actually get confusing because a new user
could sign up as Bob and kind of take over Charlie's old @mentions that
contain @bob. Is this change an attempt to address that, by fixing the
screen_name-userid mapping at the time a tweet is created?
When we post tweets that include @mentions, can we include our own
entities/user_mentions in our request body, so that Twitter can notify us if
one of the mentioned screen names has a different userid than what we were
expecting and/or one of the mentioned screen names is not a valid screen
name anymore? That would be extremely helpful in dealing with this edge
case-even if it was subject to some race conditions over a narrow period of
time.
entities/user_mentions/screen_name and entities/user_mentions/text are
redundant. I would rather just pick the text out of the tweet using original
tweet text indexed by the indices property, to save bandwidth.
Regards,
Brian
Raffi Krikorian wrote:
As part of our JSON and XML payloads, we are going to start supporting an
entities attribute that will contain this parsed and structured data.
you'll see it like so:
{
text : hey @raffi tell @noradio to check out http://dev.twitter.com
#hot,
...
entities : {
user_mentions : [
{
id : 8285392,
screen_name : raffi,
indices : [4, 9]
},
{
id : 3191321,
screen_name : noradio,
indices : [16, 23]
}
],
urls : [
{
url : http://dev.twitter.com;,
indices : [38, 64]
},
],
hashtags : [
{
text : #hot,
indices : [66, 69]
url : http://search.twitter.com/search?q=%23hot;
}
]
}
...
}
as shown above, we'll be parsing out all mentioned users, all lists, all
included URLs, and all hashtags. in the case of users, we'll provide you
their user ID, and for hashtags we'll provide you the query you can run
against the search API. and, for all of them, we'll also tell you at what
character count the entity starts and stops -- that should really take the
burden off you guys to parse the text properly.
this entities block will probably be extended later, and these entities are
just the start. have we missed anything? is there anything else you would
like to see? as always - just drop us a note, and look for these entities
to start slowly rolling out.
RE: [twitter-dev] Re: parsing out entities from tweets (a.k.a. parsing out hashtags is hard!)
Glenn Gillen wrote: Without looking at how twitter.com would currently handle that example, I would have expected the url to be http://dev.twitter.com/ #hot and for the tweet to contain no hashtag. If the hashtag always takes precedence I'd have no way to link to the following without using a URL shortener: http://oauth.net/core/1.0a/#anchor41 I think you are overlooking the space between the last slash and #hot. URLs cannot contain (un-encoded) spaces. Regards, Brian
[twitter-dev] Re: Help whitelist my ip
Sorry for the delay. Your whitelist request has been processed and you should receive an email from us soon. Brian Sutorius On May 12, 5:34 am, a...@topyapps.info a...@topyapps.info wrote: Hi, I'm waiting for a response regarding ip/account whitelisting for about a week now. I've first filled in the required form, then after several days emailed to a...@twitter.com, got a reply suggesting to fill the form again, did it 2 days ago. I runhttp://topytalk.com- a talk-oriented timeline and considering expanding my offering but am not able to do so without prior elevated access to the api. My account is @topytalk. Many thanks
[twitter-dev] Re: did they lift the limits on direct messages?
Hi all, Sorry for the confusion. We have a semi-comprehensive help page on whitelisting [1] and I'll relay the relevant points here. As Taylor said, there are per-account limits on tweets and DMs: 1000 per day and 250 per day, respectively. The daily tweet limit cannot be raised by any whitelist, and is further broken up into sub-limits throughout the day (to avoid users from blowing through all 1000 in a short time). We do not reveal the specifics of these sub-limits to prevent users from operating right at them. Twitter accounts that are on the REST API whitelist are allowed to send up to 10,000 DMs a day; this has likely changed since Doug's email. This increased limit only applies to accounts, not IPs, and the normal requirements for REST API whitelisting apply (notably, it is restricted to developers with demonstrable special needs). Hope this clears everything up! Brian Sutorius [1] http://help.twitter.com/entries/160385 On May 12, 9:52 am, Mo maur...@moluv.com wrote: Does that mean if @account has a whitelisted app, 5000 messages/day can be sent through that app, but each app user (say @user_of_account) only gets 250/day? If so, is the 100 DM/hour limit the same for both @account and @user_of_account, or is there a different hourly limit for @account? -Mo On May 12, 9:25 am, Abraham Williams 4bra...@gmail.com wrote: I read Doug's email as any account that is specifically whitelisted has 5k DM and that DMs are not effected by IP whitelisting. Abraham On Wed, May 12, 2010 at 09:21, Mo maur...@moluv.com wrote: Hi Taylor, This is different than what Doug Williams stated in this post - http://bit.ly/cLVv1Q Whitelisted users have a direct messaging limit of 5K messages per day. What I'm still not clear on, though, is how user is being defined. Is the user the app owner or the someone using the app? Also, is 5K DMs a day stated by Doug correct or is it 250 DMs? Apparently Alex and I posted essentially the same request 5 minutes apart. Answering to either this message or to my other post would be much appreciated. -Mo http://www.pay4tweet.com On May 12, 8:39 am, Taylor Singletary taylorsinglet...@twitter.com wrote: Hi Alex, Whitelisting only effects API call rate limiting -- so the answer to your question is no. T On Wed, May 12, 2010 at 8:35 AM, alex urdea alex.urdea.fi...@gmail.com wrote: Thanks for your answer. One more: is the 250 MD limit increased if the application is whitelisted? Or does the whitelist concernt the rates only? Thanks On Wed, May 12, 2010 at 5:15 PM, Taylor Singletary taylorsinglet...@twitter.com wrote: Rate limits and limits on particular actions are different. We could do better in providing a X-FeatureRateLimit header on tweets and DMs and the such that have their own issuance limit -- but I can imagine potential performance issues with that. Rate limits provide a ceiling on the amount of API calls you can make. Their main purpose is to keep the entire platform running smoothly and to not allow any one application to spoil the resource pool for its peers. Twitter, aside from the API itself, has limits on how many status updates and DMs can be sent -- the API just respects the rules of Twitter here. If you're concerned you might be hitting the upper limit, for now the best thing to do would be to implement a counter in your application and queue updates when your counter is full. A user may issue 1000 tweets per day and 250 DMs. Taylor Singletary Developer Advocate, Twitter http://twitter.com/episod On Wed, May 12, 2010 at 4:47 AM, alex alex.urdea.fi...@gmail.com wrote: I'm confused: - here it says that there's a limit on direct messages URL:http://help.twitter.com/entries/15364 In the documentation page for this method you have : API rate limited false: URL: http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-direct_messages new Here it says that API methods that use HTTP POST to submit data to Twitter, such as statuses/update do not affect rate limits. I guess that this is a POST method that submits data and is not subject to limits? URL:http://apiwiki.twitter.com/Rate-limiting Which one is true? Thank you! -- Abraham Williams | Developer for hire |http://abrah.am @abraham |http://projects.abrah.am|http://blog.abrah.am This email is: [ ] shareable [x] ask first [ ] private.
[twitter-dev] Re: Whitelist Limits for Direct Messaging
As I posted in another thread [1], here is information from our help center [2] to hopefully clarify this: - By default, Twitter accounts can send 250 DMs per day. - Accounts (not IPs and not apps) that are on the REST whitelist can send up to 10,000 DMs per day Taylor's point about the limit being account-based and not application- based is important to note. Brian Sutorius [1] http://bit.ly/9DyGDB [2] http://help.twitter.com/entries/160385 On May 12, 9:08 am, Taylor Singletary taylorsinglet...@twitter.com wrote: To my knowledge (and I might be wrong, but this is what I understand to be true): - there is a limit of 250 DMs per day for a user account, blanketly applied. Whitelisting for an application has no effect on this limit. This isn't an API limit. It's a limit for a Twitter user. A twitter user could contribute to their allocation by using the website or an API client. Taylor Singletary Developer Advocate, Twitterhttp://twitter.com/episod On Wed, May 12, 2010 at 8:43 AM, Mo maur...@moluv.com wrote: I'm trying to find a reliable source for whitelist limits for Direct Messaging. I looked through the direct messaging limits and best practices for individual services? thread -http://bit.ly/cLVv1Qbut there weren't any authoritative descriptions of whitelist limits. What I'm looking for is: 1. DMs allowed per user per hour, and per day - (Where user is defined as someone using an app). 2. DMs allowed per app per hour, and per day I saw that Doug Williams had said that whitelisted users get 5000 DMs per day, but didn't specify whether that was an app total or a total for a random user using an app for DMs. The hourly limit for whitelisted apps wasn't specified at all. -Mo http://www.pay4tweet.com
[twitter-dev] Re: Whitelist Limits for Direct Messaging
I'm not sure what you mean - our REST whitelist only accepts usernames and IP addresses as whitelistable entities. Applications don't send direct messages, users do; the DM limit is on a per-user basis. Brian On May 12, 1:27 pm, Mo maur...@moluv.com wrote: Thanks Brian and Taylor. This definitely adds some clarification. There is one last thing, though. Brian, you mentioned that the limits you specified were NOT for IPs and apps. What would be the DM limit for a whitelisted app? I can't find that explicitly stated in any of the references. On May 12, 12:31 pm, Brian Sutorius bsutor...@twitter.com wrote: As I posted in another thread [1], here is information from our help center [2] to hopefully clarify this: - By default, Twitter accounts can send 250 DMs per day. - Accounts (not IPs and not apps) that are on the REST whitelist can send up to 10,000 DMs per day Taylor's point about the limit being account-based and not application- based is important to note. Brian Sutorius [1]http://bit.ly/9DyGDB [2]http://help.twitter.com/entries/160385 On May 12, 9:08 am, Taylor Singletary taylorsinglet...@twitter.com wrote: To my knowledge (and I might be wrong, but this is what I understand to be true): - there is a limit of 250 DMs per day for a user account, blanketly applied. Whitelisting for an application has no effect on this limit. This isn't an API limit. It's a limit for a Twitter user. A twitter user could contribute to their allocation by using the website or an API client. Taylor Singletary Developer Advocate, Twitterhttp://twitter.com/episod On Wed, May 12, 2010 at 8:43 AM, Mo maur...@moluv.com wrote: I'm trying to find a reliable source for whitelist limits for Direct Messaging. I looked through the direct messaging limits and best practices for individual services? thread -http://bit.ly/cLVv1Qbut there weren't any authoritative descriptions of whitelist limits. What I'm looking for is: 1. DMs allowed per user per hour, and per day - (Where user is defined as someone using an app). 2. DMs allowed per app per hour, and per day I saw that Doug Williams had said that whitelisted users get 5000 DMs per day, but didn't specify whether that was an app total or a total for a random user using an app for DMs. The hourly limit for whitelisted apps wasn't specified at all. -Mo http://www.pay4tweet.com
[twitter-dev] Re: help about whitelisting request form
Sorry for the delay. I just reviewed your whitelisting request and responded - you should receive an email shortly. Brian Sutorius On May 11, 2:30 am, tao yametei@gmail.com wrote: dear sir last friday i Filling in whitelisting request form on twitter and submit my request but now i cant get any Reply. please tell me When Will I be able to get Reply? my twitter user is yametei thank you
[twitter-dev] Re: Source param in Lists timelines
Hi Ron, I can't find a ticket requesting xAuth under your email address. Can you please follow up with me directly? Thanks Brian Sutorius On May 11, 12:05 pm, Ron B rbther...@gmail.com wrote: Hi Taylor, Thanks for getting back to me so quickly. You're right, this was cockpit error on my part. I apologize for taking up your time unnecessarily. On another note, I've have an xAuth access request in for over a week now. Can you help expedite it. It's for registration ID: 135852. I'm pretty much dead-in-the-water with further testing on my app until this approval goes through. Thanks again! Ron On May 11, 12:48 pm, Taylor Singletary taylorsinglet...@twitter.com wrote: Hi Ron, I'm not able to reproduce this problem -- when I fetch a group of statuses from a list, I'm seeing unique source tags corresponding to the origin of the updates. Is it possible that the tweets you're evaluating in your list were all, in fact, posted via web? Taylor Singletary Developer Advocate, Twitterhttp://twitter.com/episod On Tue, May 11, 2010 at 9:23 AM, Ron B rbther...@gmail.com wrote: The source param in Lists timelines seems to be hardwired to web. Is this on purpose, or is something wrong? i.e.http://api.twitter.com/1/user/lists/list_id/statuses.format, yields sourceweb/source.
[twitter-dev] Re: TweetDeck and xAuth
I can't find a ticket under your email address requesting xAuth. Could you please follow up with me directly? I'll be happy to review your request. Brian Sutorius On May 10, 3:28 am, Steve Loft kettletoft@googlemail.com wrote: Does anyone know how long it should take to get xAuth privilege? It's just that I applied nearly a week ago for access for my desktop app, and time is running out. It looks like I am going to have an app which doesn't work with Twitter come the end of June.
[twitter-dev] Re: About update limits
To clarify, statuses/update is not affected by rate-limit whitelisting as it's a POST call and we don't maintain a separate whitelist for boosting the daily tweet limit above 1000. While we do not give out the specifics around the sub-limits, they *are* administered on a per-account basis and if you stay around your approximation of 20 tweets per half-hour you should be fine. Brian Sutorius On Apr 29, 6:07 am, Raffi Krikorian ra...@twitter.com wrote: the numbers are roughly broken up over the day. and the limit applies to an account. and yes - there is a whitelisting for status/updates -- please e-mail a...@twitter to ask for it. On Thu, Apr 29, 2010 at 5:26 AM, akaii chibiak...@gmail.com wrote: This is what the FAQ has to say about status update limits: Updates: 1,000 per day. The daily update limit is further broken down into smaller limits for semi-hourly intervals. Retweets are counted as updates. I'm a little unclear as to what exactly is meant by further broken down into smaller limits for semi-hourly intervals. Is the 1000 per day limit divided evenly between the 48 half hours each day (around 20 or so tweets per half an hour?). Also, I'm assuming this limit applies to each unique account? Is this limit absolutely fixed? Or is there some equivalent to whitelisting for status/update limits as well? Thanks... -- Raffi Krikorian Twitter Platform Teamhttp://twitter.com/raffi
[twitter-dev] Re: My applications were Suspended
My name is Brian Truebe and I am on the API Policy team, when apps are suspended they are sent a notice as to how to contest the suspension, however this may have gotten lost in the tubes. Please email a...@twitter.com and let us know the app name and we'll see if we can sort this out. Sorry for the inconvenience. Regards, Brian On Apr 22, 12:51 pm, Revabad lookst...@gmail.com wrote: My applications were suspended and none from twitter has given me a reason as to why. Can someone help me out. -- Subscription settings:http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
[twitter-dev] Re: My applications were Suspended
Yes, the email that is sent out after an application is suspended does explain possible rule violations. This email is sent to the account that registered the application, so if you've registered an app with an auxiliary account not tied to an email address you check regularly then an app suspension may come as a rather unfortunate surprise. While there is no sandbox, we're very open to discussing any concerns an app developer may have while they develop their app. The best course of action is to read the rules first while developing. If you're still worried a feature you're developing may result in your users being suspended our your entire app being suspended then you can always email us at a...@twitter.com and we'll be happy to work with you to ensure the longevity of your application. I hope this helps. -Brian On Apr 23, 11:37 am, John Meyer john.l.me...@gmail.com wrote: On 4/23/2010 10:58 AM, Brian Truebe wrote: My name is Brian Truebe and I am on the API Policy team, when apps are suspended they are sent a notice as to how to contest the suspension, however this may have gotten lost in the tubes. Please email a...@twitter.com and let us know the app name and we'll see if we can sort this out. Sorry for the inconvenience. Regards, Brian One question: does the e-mail have an explanation about why the application was suspended in the first place (you mention how to contest the suspension but nothing about what the suspension is about). And is there some way to create a sandbox for suspended apps where they can re-test to see if they are in compliance with the rules before going out into the real world Twitterverse? -- Subscription settings:http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
RE: [twitter-dev] Re: Privacy issues with the proposed annotations feature
Alexro wrote: not to ignore privacy issues but just to simplify the situation a bit ... What currently protects a user from a malicious (desktop) application stealing all kinds of user data via submitting tweets through it's proxy? And even by submitting such information directly to it's website? That is a very good point. It is an unsolved problem that affects nearly all installable software, and that is a problem that needs to be solved at the operating system level. iPhone OS and the upcoming Windows 7 Phone do have measures to protect against that kind of data theft and inadvertent information disclosure; in fact, basically all of the API limitations in Windows 7 Phone (no background apps, no access to the user's personal information, warnings about GPS usage) can be traced back to privacy protection. Similarly, even in desktop Windows, access to PIM information (email and contacts in particular) is severely restricted with the official APIs. Location is a different because Twitter has special privacy protections for its geo feature, including technical limitations that control whether applications may use it, and a way to remove the location information from tweets after the fact. I don't think it makes sense to have a lock for the built-in location feature and at the same time allow applications to use annotations to disclose the same (or even more precise) location information regardless of that lock. If you can trust applications not to disclose location information via annotations without the user's consent, why can't you trust those same applications to not use the built-in geo feature without the user's consent? If an application isn't trusted enough to be able to flip the geo switch, why should it be trusted to not disclose the user's location in a different way without the user's consent? At least with the built-in geo feature, there is a built-in mechanism for the user to remove the geo annotations, unlike other annotations. It is the same deal with Twitter XAuth and website-only functionality. A malicious Twitter XAuth application has full access to everything in the user's account because it has the user's password. Meanwhile, the API doesn't expose useful actions (sign up, approve followers, change password, read email address) in an effort to prevent malicious applications from using that functionality. But, of course, malicious applications don't *need* any official API at all to perform those actions. My conclusion is that the API restrictions punish well-behaved applications due to fear that they may be malicious, but they don't actually prevent any unwanted behavior by applications that actually are malicious. This is something that Raffi touched on when he blogged about how he thinks that every OAuth approval should be going through the Twitter website (i.e. no Twitter XAuth). The annotations feature would compound that problem in a way that can't be solved by disabling Twitter XAuth access to applications. (Note that Twitter XAuth is different from, xauth.org XAuth.) Regards, Brian -- Subscription settings: http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
[twitter-dev] Privacy issues with the proposed annotations feature
Right now the web UI exposes every piece of metadata in a tweet to end-users. That is, an end-user can use twitter.com to check the complete contents of tweet sent by an application. I didn't see anything in the proposals regarding the annotation feature that says that users will be able to see all the annotations through the web UI. And, even if they could see them, chances are they couldn't understand them. And, even if end-users could understand them, applications will be able to use encryption and other obfuscation to make them impossible to interpret. This reduces the amount of control users have over their tweets. Right now an application cannot disclose the user's location in a tweet, except by putting the location information in the tweet text (which the user can see very clearly), or by putting the location information in the built-in geo feature. The ability for applications to expose the user's information is controlled by a preference that can be controlled only by the official web interface on twitter.com. However, with the annotations feature, applications will be able to expose the user's location-again, possibly encrypted or otherwise obfuscated-even when application access to the location feature is disabled. It doesn't make sense to disable an applications' access to the geo feature and then let it silently and undetectably disclose the user's location-perhaps in even more detail than the built-in geo feature allows. I think there must be some kind of control mechanism in place for annotations, or the web UI must present all the annotations of a user's tweets to that user, or both, in order to prevent the annotations feature from becoming a side channel for applications to communicate users' private information without users' knowledge or consent. I would like to know more about how this is going to be done. Thanks, Brian -- Subscription settings: http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
RE: [twitter-dev] Re: Basic Auth Deprecation
If an app is trusted enough to have xAuth access, it should have the ability to do things like accept and reject followers for protected accounts via the API. An malicious xAuth application would already have full access to the user's account so it could already accept/reject followers through other means. It's the same with creating an account-there's no security justification for disallowing creating accounts for xAuth applications when the account settings are already available to xAuth applications that choose not to follow the rules and restrictions. Perhaps xAuth access should be granted only to people that have been authenticated with a high level of assurance-the same method you use for verified accounts and/or through some high-assurance certificate provider. For example, you could require xAuth developers to sign a token using their iPhone appstore/Symbian Signed/WinMo/Java Verified/High Assurance SSL certificate in order to prove their identities. And/or, charge a token amount of money for xAuth access so that you can verify identity somewhat via the credit card used. The problem with socializing security like you suggested is that it can only detect what end-users can detect. Chances are that all the users of a mythical Bieber Ponies application are going to be relatively unsophisticated people that couldn't care less about security. And, also a malicious application can keep its bad behavior dormant for a long time until it builds up a large userbase and/or until some software update activates the bad behavior. -Brian From: twitter-development-talk@googlegroups.com [mailto:twitter-development-t...@googlegroups.com] On Behalf Of Raffi Krikorian Sent: Wednesday, April 14, 2010 9:09 AM To: twitter-development-talk@googlegroups.com Subject: Re: [twitter-dev] Re: Basic Auth Deprecation we developed xauth specifically for that - mobile and desktop clients were complaining about usability problems when they have to bounce their users to a web browser. i'm well aware of the implications about xauth, and have blogged about it here: http://mehack.com/xauth-and-perhaps-the-need-for-socializing-ap would love to hear more comments as i formalize my thoughts around this! On Wed, Apr 14, 2010 at 6:15 AM, Jaanus jaa...@gmail.com wrote: Why are you Twitter guys pushing xAuth so hard? Even for new desktop clients? Instead of recommending a proper oAuth flow with PIN or such? I understood its main purpose is to help legacy clients with transition, and new clients should do proper oAuth. One argument I have seen is that oAuth has usability problems. I would like to see more substance around this statement beyond just developers thinking out loud. I have implemented oAuth in @cremeapp (ok, it uses in-app browser instead of separate, but otherwise it is the proper PIN flow) and not a single person has complained. I see from usage numbers that people breeze through the oAuth authentication just fine. I was expecting worse, but it's fine. It comes down to proper UI design and clarity of instructions. J On Apr 14, 1:15 am, Taylor Singletary taylorsinglet...@twitter.com wrote: Basic auto being turned off means just that.. Desktop clients can implement xAuth as an alternative, where you do a one-time exchange of login and password for an OAuth access token and continue from there signing your requests and doing things in the OAuth way. You'd no longer, as a best practice and one that I would stress in the upmost even on a desktop client, store the login and password beyond the xAuth access token negotiation step. If the token were revoked you would then query for the login and password again and so on and so on and also and also. Obtaining permission to use xAuth for desktop clients is as easy as sending a well-identified and verbose note to a...@twitter.com. Basic auth had a good run. It's nearly time to say goodnight. Taylor On Tuesday, April 13, 2010, Dean Collins d...@cognation.net wrote: Just so I understand this, applications running on the desktop will still work correct? Basic functionality is only being turned off for web apps correct? It's not like desktop apps will have to start using oauth. Cheers, Dean -Original Message- From: twitter-development-talk@googlegroups.com [mailto:twitter-development-t...@googlegroups.com] On Behalf Of Dewald Pretorius Sent: Tuesday, April 13, 2010 7:31 PM To: Twitter Development Talk Subject: [twitter-dev] Re: Basic Auth Deprecation Could you please announce the hard turn off date somewhere on one of your Twitter blogs about a month ahead of time, so that we all have an official source to point our users to when we explain to them why we're converting everything over to OAuth? On Apr 13, 8:19 pm, Raffi Krikorian ra...@twitter.com wrote: we have announced deprecation, and will hard turn off basic authentication in june. the exact date has not been set, but i
[twitter-dev] Re: chirp questions for non-attendee's
I'm one of the Brians on Twitter's API Policy team, and we're going to be participating in an API Policy panel with @delbius on the second day of Chirp. We have our own Google Moderator page set up for this (accessible from the link Abraham sent out, but here also for good measure): http://bit.ly/chirppolicy . Please add your questions here and we'll answer them at the panel, as well as post a recap for you here. Thanks, Brian Sutorius On Apr 14, 12:59 am, Abraham Williams 4bra...@gmail.com wrote: http://www.google.com/moderator/#16/e=5c0f -- To unsubscribe, reply using remove me as the subject.
RE: [twitter-dev] Some thoughts leading up to Chirp
Ryan Sarver wrote: [...] However when we dug in a little bit we realized that it was causing massive confusion among user's who had an iPhone and were looking to use Twitter for the first time. They would head to the App Store, search for Twitter and would see results that included a lot of apps that had nothing to do with Twitter and a few that did, but a new user wouldn't find what they were looking for and give up. That is a lost user for all of us. [...] We will also admit our mistakes when they are made and the Blackberry client should never have been labeled official. It has since been changed and you won't see that language used with Twitter clients in the future. The officialness of the Blackberry app wasn't much of a problem. The problem was/is the name and the logo. It is confusing to users to have the app named Twitter and it is confusing to see the app branded solely with the t logo. The t mark is something that should definitely be protected, but I think it has a lot of *functional* uses for it as an indicator or badge that make it inappropriate as the logo for a single application on any platform. IMO, it would be much better for Twitter in the long run to have the t logo used as a badge to indicate that an app has met some quality/security criteria--like the Compatible with Windows 7 logo program, the Made for iPod, the Visa logo, etc. That would be something that would allow you to start a process of ensuring a wide variety of high-quality applications that are closely aligned with your business goals, without setting the bar too high or the terms too strict for simpler applications with a more casual connection to Twitter. Many mobile operators and phone manufacturers have very bad policies about supporting their products once they've been replaced with newer models. It is very likely that, if you let mobile operators and mobile manufacturers have exclusive uses of the trademarks on their platforms, that those trademarks will be attached to software that becomes stale, obsolete, or even totally non-functional on otherwise serviceable devices that aren't even that old. It would be a big mistake to reserve Twitter's branding for applications which don't even end up staying in the top tier of Twitter apps on their platform in terms of quality. Anyway, I think that everybody will soon see that officialness of competing applications is a very small problem compared to issues like degradation of UI w/ advertising or strict restrictions on how spam-ish Twitter-provided content is filtered. I really hope that you guys have something extremely clever planned for monetization. I have been unable to think of many ways to make money with Twitter that didn't involve annoying end-users with ads or encouraging end-users to annoy each other (RT to win...), and AFAICT nothing is going to work unless it keeps users' home and @mentions timelines clean with less advertising/spam than there already is now, instead of more. I am genuinely curious to see what you guys have come up with. Regards, Brian @BRIAN_ -- To unsubscribe, reply using remove me as the subject.
RE: [twitter-dev] Re: Upcoming changes to the way status IDs are sequenced
John, Thank you. That was one of the most informative emails on the Twitter API I have seen on the list. Basically, even now, an application should not use an ID of a tweet for since_id if the tweet is less than 10 seconds old, ignoring service abnormalities. Probably a larger threshold (30 seconds or even a minute) would be better, especially when you take into consideration the likelihood of clock skew between the servers that generate the timestamps. I think this is information that would be useful to have added to the API documentation, as I know many applications are taking a much more naive approach to pagination. Thanks again, Brian From: twitter-development-talk@googlegroups.com On Behalf Of John Kalucki Sent: Friday, April 09, 2010 1:20 PM To: twitter-development-talk@googlegroups.com Subject: Re: [twitter-dev] Re: Upcoming changes to the way status IDs are sequenced Folks are making a lot of incorrect assumptions about the Twitter architecture, especially around how we materialize and present timeline vectors and just what QoS we're really offering. This new scheme does not significantly, or perhaps even observably, make the existing issues around since_id any better or any worse. And I'm being very precise here. The since_id situation is such that the few milliseconds skew possible in Snowflake are practically irrelevant and lost in the noise of a 4 to 6 orders-of-magnitude misconception. (That's a very big misconception.) If you do not know the rough ordering of our event stream as it applied to the materialized timeline vectors and also the expected rate of change on the timeline in question, you cannot make good choices about making since_id perfect. But, neither you should you try to make it perfect, nor should you have to worry about this. If you insist upon worrying about this, here's my slight salting of Mark's advice: In the existing continuously increasing id generation scheme on the Twitter.com API, I'd subtract about 5000 ids from since_id to ensure sufficient overlap in nearly all cases, but even this could be lossy in the face of severe operational issues -- issues of a type that we haven't seen in many many months. The search API has a different K in its rough ordering, so you might need more like 10,000 ids. In the new Snowflake scheme, I'd overlap by about 5000 milliseconds for twitter.com APIs and 10,000 ms for search APIs. Despite all this, things still could go wrong. An engineer here is known for pointing out that even things that almost never ever happen, happen all the time on the Twitter system. Now, just because they are happening, to someone, all the time, doesn't mean that they'll ever ever happen to you or your users in a thousand years -- but some's getting hit with it, somewhere, a few times a day. The above schemes no longer treat the id as an opaque unique ordered identifier. And woe lies in wait for you as changes are made to these ids. Woe. You also need to deduplicate. Be very careful and understand fully what you summon by breaking this semantic contract. In the end, since_id issues go away on the Streaming API, and other than around various start-up discontinuities, you can ignore this issue. I'll be talking about Rough Ordering, among other things Streaming, at the Chirp conference. Come geek out. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Fri, Apr 9, 2010 at 1:58 AM, Dave Sherohman d...@fishtwits.com wrote: On Thu, Apr 08, 2010 at 05:03:29PM -0700, Naveen wrote: However, I wanted to be clear and feel it should be made obvious that with this change, there is a possibility that a tweet may not be delivered to client if the implementation of how since_id is currently used is not updated to cover the case. I still envision the situation as more likely than you seem to believe and figure as tweet velocity increases, the likelihood will also increase; But I am assuming have better data to support your viewpoint than I and shall defer. Maybe I'm just missing something here, but it seems trivial to fix on Twitter's side (enough so that I assume it's what they've been planning from the start to do): Only return tweets from closed buckets. We are guaranteed that the buckets will be properly ordered. The order will only be randomized within a bucket. Therefore, by only returning tweets from buckets which are no longer receiving new tweets, since_id works and will never miss a tweet. And, yes, this does mean a slight delay in getting the tweets out because they have to wait a few milliseconds for their bucket to close before being exposed to calls which can use since_id, plus maybe a little longer for the contents of that bucket to be distributed to multiple servers. That's still going to only take time comparable to round-trip times for an HTTP request to fetch the data for display to a user and be far, far less than the average refresh delay required by those clients which fall under
RE: [twitter-dev] Re: Upcoming changes to the way status IDs are sequenced
John, I am not polling. I am simply trying to implement a basic refresh feature like every desktop/mobile Twitter app has. Basically, I just want to let users scroll through their timelines, and be reasonably sure that I am presenting them with an accurate complete view of the timeline, while using as little bandwidth as possible. When I said 10 seconds old/30 seconds old/etc. I was referring to I was referring to the age at the time the page of tweets was generated. So, basically, if the tweet's timestamp - the response's Last-Modified time more than 10,000 ms (from what you said below), you are almost definitely getting At Least Once behavior if Twitter is operating normally, and you can use that information to get At Least Once behavior that emulates Exactly Once behavior with little (usually no) overhead. Is that a correct interpretation of what you were saying? Thanks, Brian From: twitter-development-talk@googlegroups.com [mailto:twitter-development-t...@googlegroups.com] On Behalf Of John Kalucki Sent: Friday, April 09, 2010 3:31 PM To: twitter-development-talk@googlegroups.com Subject: Re: [twitter-dev] Re: Upcoming changes to the way status IDs are sequenced Your second paragraph doesn't quite make sense. The period between your next poll and the timestamp of the last status is irrelevant. The issue is solely the magnitude of K on the roughly sorted stream of events that are applied to the materialized timeline vector. As K varies, so do the odds, however infinitesimally small, that you will miss a tweet using the last status id returned. The period between your polls of the API does not affect this K. My recommendation is to ignore this issue in nearly every use case. If you are, however, polling high velocity timelines (including search queries) and attempting to approximate an Exactly Once QoS, you should, basically, stop doing that. You are probably wasting resources and you'll probably never get Exactly Once behavior anyway. Use the Streaming API instead. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Fri, Apr 9, 2010 at 12:20 PM, Brian Smith br...@briansmith.org wrote: John, Thank you. That was one of the most informative emails on the Twitter API I have seen on the list. Basically, even now, an application should not use an ID of a tweet for since_id if the tweet is less than 10 seconds old, ignoring service abnormalities. Probably a larger threshold (30 seconds or even a minute) would be better, especially when you take into consideration the likelihood of clock skew between the servers that generate the timestamps. I think this is information that would be useful to have added to the API documentation, as I know many applications are taking a much more naive approach to pagination. Thanks again, Brian From: twitter-development-talk@googlegroups.com On Behalf Of John Kalucki Sent: Friday, April 09, 2010 1:20 PM To: twitter-development-talk@googlegroups.com Subject: Re: [twitter-dev] Re: Upcoming changes to the way status IDs are sequenced Folks are making a lot of incorrect assumptions about the Twitter architecture, especially around how we materialize and present timeline vectors and just what QoS we're really offering. This new scheme does not significantly, or perhaps even observably, make the existing issues around since_id any better or any worse. And I'm being very precise here. The since_id situation is such that the few milliseconds skew possible in Snowflake are practically irrelevant and lost in the noise of a 4 to 6 orders-of-magnitude misconception. (That's a very big misconception.) If you do not know the rough ordering of our event stream as it applied to the materialized timeline vectors and also the expected rate of change on the timeline in question, you cannot make good choices about making since_id perfect. But, neither you should you try to make it perfect, nor should you have to worry about this. If you insist upon worrying about this, here's my slight salting of Mark's advice: In the existing continuously increasing id generation scheme on the Twitter.com API, I'd subtract about 5000 ids from since_id to ensure sufficient overlap in nearly all cases, but even this could be lossy in the face of severe operational issues -- issues of a type that we haven't seen in many many months. The search API has a different K in its rough ordering, so you might need more like 10,000 ids. In the new Snowflake scheme, I'd overlap by about 5000 milliseconds for twitter.com APIs and 10,000 ms for search APIs. Despite all this, things still could go wrong. An engineer here is known for pointing out that even things that almost never ever happen, happen all the time on the Twitter system. Now, just because they are happening, to someone, all the time, doesn't mean that they'll ever ever happen to you or your users in a thousand years -- but some's getting hit with it, somewhere, a few times a day
RE: [twitter-dev] Re: Upcoming changes to the way status IDs are sequenced
What does “within the caveats given above” mean? Either since_id will work or
it won’t. It seems to me that if IDs are only in a “rough” order, since_id
won’t work—in particular, there is a possibility that paging through tweets
using since_id will completely skip over some tweets.
My concern is that, since tweets will not be serialized at the time they are
written, there will be a race condition between me making a request and users
posting new statuses. That is, I could get a response with the largest id in
the response being X that gets evaluated just before a tweet (X-1) has been
saved in the database; If so, when I issue a request with since_id=X, my
program will never see the newer tweet (X-1).
Are you going to change the implementation of the timeline methods so that they
never return a tweet with ID X until all nodes in the cluster guarantee that
they won’t create a new tweet with an ID less than X?
I implement the following logic:
1. Let LATEST start out as the earliest tweet available in the user’s
timeline.
2. Make a request with since_id={LATEST}, which returns a set of tweets T.
3. If T is empty then stop.
4. Let LATEST= max({ id(t), for all t in T}).
5. Goto 2.
Will I be guaranteed not to skip over any tweets in the timeline using this
logic? If not, what do I need to do to ensure I get them all?
Thanks,
Brian
From: twitter-development-talk@googlegroups.com
[mailto:twitter-development-t...@googlegroups.com] On Behalf Of Mark McBride
Sent: Thursday, April 08, 2010 5:10 PM
To: twitter-development-talk@googlegroups.com
Subject: Re: [twitter-dev] Re: Upcoming changes to the way status IDs are
sequenced
Thank you for the feedback. It's great to hear about the variety of use cases
people have for the API, and in particular all the different ways people are
using IDs. To alleviate some of the concerns raised in this thread we thought
it would be useful to give more details about how we plan to generate IDs
1) IDs are still 64-bit integers. This should minimize any migration pains.
2) You can still sort on ID. Within a few millieconds you may get out of order
results, but for most use cases this shouldn't be an issue.
3) since_id will still work (within the caveats given above).
4) We will provide a way to backfill from the streaming API.
5) You cannot use the generated ID to reverse engineer tweet velocity. Note
that you can still use the streaming API to determine the rate of public
statuses.
Additional items of interest
1) At some point we will likely start using this as an ID for direct messages
too
2) We will almost certainly open source the ID generation code, probably before
we actually cut over to using it.
3) We STRONGLY suggest that you treat IDs as roughly sorted (roughly being
within a few ms buckets), opaque 64-bit integers. We may need to change the
scheme again at some point in the future, and want to minimize migration pains
should we need to do this.
Hopefully this puts you more at ease with the changes we're making. If it
raises new concerns, please let us know!
---Mark
http://twitter.com/mccv http://twitter.com/mccv
On Mon, Apr 5, 2010 at 4:18 PM, M. Edward (Ed) Borasky zn...@comcast.net
wrote:
On 04/05/2010 12:55 AM, Tim Haines wrote:
This made me laugh. Hard.
On Fri, Apr 2, 2010 at 6:47 AM, Dewald Pretorius dpr...@gmail.com wrote:
Mark,
It's extremely important where you have two bots that reply to each
others' tweets. With incorrectly sorted tweets, you get conversations
that look completely unnatural.
On Apr 1, 1:39 pm, Mark McBride mmcbr...@twitter.com wrote:
Just out of curiosity, what applications are you building that require
sub-second sorting resolution for tweets?
Yeah - my bot laughed too ;-)
--
M. Edward (Ed) Borasky
borasky-research.net/m-edward-ed-borasky
A mathematician is a device for turning coffee into theorems. ~ Paul Erdős
--
To unsubscribe, reply using remove me as the subject.
RE: [twitter-dev] Re: Upcoming changes to the way status IDs are sequenced
Mark, thank you for taking the time to respond.
What is the smallest “comfort threshold” that will guarantee that we will see
all the tweets, with none skipped over and the fewest tweets returned multiple
times?
Let’s say the comfort threshold was 2 seconds. It seems to me like there could
realistically be dozens or hundreds of tweets within those two seconds in a
single timeline, and a request that used the logic you mentioned would return
an entire page (200 tweets) consisting of tweets that the application already
has; the application would be making a relatively large download, receiving
nothing useful for it, and not be able to make any progress because its
since_id would get “stuck”. This is at odds with many (most?) applications goal
in using since_id, which is to transfer as little data as possible.
It seems like a better alternative would a new parameter that says “don’t give
me any tweets that are less than X seconds old,” where X seconds is the
comfort threshold. That way, the application may lag behind by a few of
seconds, but at least it would be able to confidently page through the timeline
without excessive data transfer. Without such a mechanism, it looks like this
change will be a significant degradation of service that result in
applications’ “refresh” features becoming either unreliable or very wasteful.
But, is it realistic for applications to expect the Twitter cluster to be in
sync within 2 seconds? 10 seconds? 30 seconds? That is the part that is unclear
to me.
Thanks again,
Brian
From: twitter-development-talk@googlegroups.com
[mailto:twitter-development-t...@googlegroups.com] On Behalf Of Mark McBride
Sent: Thursday, April 08, 2010 6:38 PM
To: twitter-development-talk@googlegroups.com
Subject: Re: [twitter-dev] Re: Upcoming changes to the way status IDs are
sequenced
It's a possibility, but by no means a probability. Note that you can mitigate
this by using the newest tweet that is outside your danger zone. For example
in a sequence of tweets t1, t2 ... ti ... tn with creation times c1, c2 ... ci
... cn and a comfort threshold e you could use since_id from the latest ti such
that c1 - ci e.
---Mark
http://twitter.com/mccv
On Thu, Apr 8, 2010 at 4:27 PM, Naveen knig...@gmail.com wrote:
This was my initial concern with the randomly generated ids that I
brought up, though I think Brian described it better than I.
It simply seems very likely that when using since_id to populate newer
tweets for the user, that some tweets will never be seen, because the
since_id of the last message received will be larger than one
generated 1ms later.
With the random generation of ids, I can see two way guarantee
delivery of all tweets in a users timeline
1. Page forwards and backwards to ensure no tweets generated at or
near the same time as the newest one did not receive a lower id. This
will be very expensive for a mobile client not to mention complicate
any refresh algorithms significantly.
2. Given that we know how IDs are generated (i.e. which bits represent
the time) we can simply over request by decrementing the since_id time
bits, by a second or two and filter out duplicates. (again, not really
ideal for mobile clients where battery life is an issue, plus it then
makes the implementation very dependent on twitters id format
remaining stable)
Please anyone explain if Brian and I are misinterpreting this as a
very real possibility of never displaying some tweets in a time line,
without changing how we request data from twitter (i.e. since_id
doesn't break)
--Naveen Ayyagari
@knight9
@SocialScope
On Apr 8, 7:01 pm, Brian Smith br...@briansmith.org wrote:
What does “within the caveats given above” mean? Either since_id will work or
it won’t. It seems to me that if IDs are only in a “rough” order, since_id
won’t work—in particular, there is a possibility that paging through tweets
using since_id will completely skip over some tweets.
My concern is that, since tweets will not be serialized at the time they are
written, there will be a race condition between me making a request and users
posting new statuses. That is, I could get a response with the largest id in
the response being X that gets evaluated just before a tweet (X-1) has been
saved in the database; If so, when I issue a request with since_id=X, my
program will never see the newer tweet (X-1).
Are you going to change the implementation of the timeline methods so that
they never return a tweet with ID X until all nodes in the cluster guarantee
that they won’t create a new tweet with an ID less than X?
I implement the following logic:
1. Let LATEST start out as the earliest tweet available in the user’s
timeline.
2. Make a request with since_id={LATEST}, which returns a set of tweets
T.
3. If T is empty then stop.
4. Let LATEST= max({ id(t), for all t in T}).
5. Goto 2.
Will I be guaranteed
RE: [twitter-dev] Upcoming changes to the way status IDs are sequenced
Any app that pages through timelines uses since_id or max_id depends responses being ordered by tweet ID. What will be the replacement for since_id and max_id? Taylor Singletary wrote: We are planning to replace our current sequential tweet ID generation routine with a simple, more scalable solution. IDs will still be 64-bit unsigned integers. However, this new solution is no longer guaranteed to generate sequential IDs. Instead IDs will be derived based on time: the most significant bits being sourced from a timestamp and the least significant bits will be effectively random. For the majority of applications we think this scheme switch will be a non-event. Before implementing these changes, we'd like to know if your applications currently depend on the sequential nature of IDs. Do you depend on the density of the tweet sequence being constant? Are you trying to analyze the IDs as anything other than opaque, ordered identifiers? Aside for guaranteed sequential tweet ID ordering, what APIs can we provide you to accomplish your goals? To unsubscribe from this group, send email to twitter-development-talk+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
[twitter-dev] Re: Question about xAuth.
Hey John - I see you've written into a...@twitter.com and I'm following up with you there. :) On Mar 23, 4:14 pm, John Meyer john.l.me...@gmail.com wrote: On 3/23/2010 3:45 PM, Brian Sutorius wrote: I just refreshed your application's xAuth access. Can you try again? You may reply to me directly if you're still having issues. Brian While we're on the topic, Brian. I'm going to start implementing xAuth support into TwitterVB. To do that I'm probably going to need a dummy app (TwitterVB isn't an application unto itself but rather a library). How would I go about doing that? To unsubscribe from this group, send email to twitter-development-talk+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
[twitter-dev] Re: xAuth approvals?
Hey Cameron, Could you reply to me directly? I can help get you set up. Brian On Mar 23, 11:00 am, Cameron Kaiser spec...@floodgap.com wrote: I'm still (somewhat ;-) patiently waiting for xAuth approval so I can work on an implementation in TTYtter. Any news on the timeline? Will these be done in time for Chirp so that we can pillory you guys with questions? ;-) -- personal:http://www.cameronkaiser.com/-- Cameron Kaiser * Floodgap Systems *www.floodgap.com* ckai...@floodgap.com -- When you don't know what you're doing, do it neatly. --- To unsubscribe from this group, send email to twitter-development-talk+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
[twitter-dev] Re: Question about xAuth.
I just refreshed your application's xAuth access. Can you try again? You may reply to me directly if you're still having issues. Brian On Mar 23, 9:10 am, IoriAYANE iori.ay...@gmail.com wrote: I have trouble for xAuth. I applied by sending an email to a...@twitter.com. And I received the email of the following contents. received mail -- Thanks for your interest in XAuth. Your application now has the ability to use XAuth, and you can read the documentation here:http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-oauth-access_to... . received mail -- I'm testing xAuth on my application. However, I cannot certify it. My application received HTTP 401 error. I had the developer of my friend who test xAuth in the following applications. In that case, it was OK. However, I fail with my key. Test application Linkhttp://relog.xii.jp/download/test/xAuthTest.LZH Please help me. To unsubscribe from this group, send email to twitter-development-talk+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
RE: [twitter-dev] New Resources: Who has a tweet been retweeted by?
It is neat (mostly for the original tweeter) to know how popular a tweet is globally, but when reading your timeline it is much more useful to know which of the people you follow have retweeted something. Such a friend-centric view of rewteets and/or favoriates is what would make the retweet functionality a compelling replacement for communities like Reddit and Digg. I think these resources will not work well once retweeting becomes popular-especially when it is common for tweets to get retweeted more than 100 times. Let's a tweet gets retweeted 100 times from people in the UK before people even wake up in the US; then any American users that retweet it will be basically drowned out. It also seems vulnerable to intentional DoS (to prevent others from showing up), spam (retweeting stuff just to get linked to in non-followers' timelines), and other manipulation. And, it also encourages the old-style RTs to be used by people that want to make sure their retweets show up. Accordingly, it would be really great to have at least the /statuses/:status_id/retweeted_by/ids maintain enough information to let applications retrieve a list of all the user's friends that have retweeted the tweet. (I know that is easier said than done.) -Brian From: Marcel Molina We've just deployed two new resources for the retweet API: * /statuses/:status_id/retweeted_by * /statuses/:status_id/retweeted_by/ids The first will return up to the first 100 user representations of those who have retweeted the tweet specified in the url by :status_id. The second will return just the ids of those retweeters for the cases where that's all you care about. Full docs here: http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-GET-statuses-id-retwee ted_by http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-GET-statuses-id-retwee ted_by-ids In conjunction with these additions we also want to provide a retweet_count parameter in the status payload for convenience. The code for this is mostly done but there are some details around cache invalidation that will likely push out the availability of the retweet_count parameter for a few weeks while we work on an infrastructure change to obviate these cache invalidation issues. But we just wanted to give you a heads up about that too! To unsubscribe from this group, send email to twitter-development-talk+unsubscribegooglegroups.com or reply to this email with the words REMOVE ME as the subject.
