[twitter-dev] User Streaming API and use of OAuth from web browser
Hi, We are building an application client that is browser based. We're very comfortable with using OAuth from our server side code and are using it fine with the REST API (users sign in, authenticate with Twitter, we store their access tokens and reuse as requested - at the moment we mimic the required Twitter API on our server and when a user does something like a POST, we call our stub, use their token to then make the call via OAuth to Twitter). So far so good, but we'd like to implement User Streaming directly into the client side application. I've been browsing the Twitter Development documentation and there's a couple of points I'd like clarification on: * http://dev.twitter.com/pages/auth_overview says Streaming supports Basic and OAuth. * http://dev.twitter.com/pages/user_streams says that the user streams supports OAuth only HTTPS, OAuth and JSON only. No problems here, I just raise it to point out the auth_overview doco is slightly out of date. * http://dev.twitter.com/pages/oauth_libraries talks about a JS library but says Javascript really shouldn't be used for OAuth 1.0A with respect to websites in web browsers. Ideally, you'll only use Javascript to perform OAuth operations when using server-side. The points I'd like some clarification on: 1. Given user_streams API is the intended way for clients to access Twitter going forwards, I presume it's intended not just for desktop, but also web clients too? 2. If 1 is correct, then is it OK to use JavaScript for the OAuth? If it's not, what is the recommended approach for a client side web application to connect and authenticate to the user_stream? Thanks, Tim -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] User Streaming API and use of OAuth from web browser
OAuth 1.0a was made for server to server authentication. It uses secrets for the client, which would make your application vulnerable. So, no, you should not use JavaScript for OAuth. Not before Twitter starts using OAuth 2.0. About the document you are referring to: there are three types of streams: normal streams, user streams and site streams. In most cases when the documentation refers to Streaming, it refers to the Normal Streams (= filter, sample, firehose, etc). There would be one more issue which requires mentioning: JavaScript's Same-origin policy. You can't make a request directly to the Twitter API via JavaScript: you *will* need a proxy on your own server. Tom On 10/6/10 10:54 PM, Tim Bull wrote: Hi, We are building an application client that is browser based. We're very comfortable with using OAuth from our server side code and are using it fine with the REST API (users sign in, authenticate with Twitter, we store their access tokens and reuse as requested - at the moment we mimic the required Twitter API on our server and when a user does something like a POST, we call our stub, use their token to then make the call via OAuth to Twitter). So far so good, but we'd like to implement User Streaming directly into the client side application. I've been browsing the Twitter Development documentation and there's a couple of points I'd like clarification on: * http://dev.twitter.com/pages/auth_overview says Streaming supports Basic and OAuth. * http://dev.twitter.com/pages/user_streams says that the user streams supports OAuth only HTTPS, OAuth and JSON only. No problems here, I just raise it to point out the auth_overview doco is slightly out of date. * http://dev.twitter.com/pages/oauth_libraries talks about a JS library but says Javascript really shouldn't be used for OAuth 1.0A with respect to websites in web browsers. Ideally, you'll only use Javascript to perform OAuth operations when using server-side. The points I'd like some clarification on: 1. Given user_streams API is the intended way for clients to access Twitter going forwards, I presume it's intended not just for desktop, but also web clients too? 2. If 1 is correct, then is it OK to use JavaScript for the OAuth? If it's not, what is the recommended approach for a client side web application to connect and authenticate to the user_stream? Thanks, Tim -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] User Streaming API and use of OAuth from web browser
There would be one more issue which requires mentioning: JavaScript's Same-origin policy. You can't make a request directly to the Twitter API via JavaScript: you *will* need a proxy on your own server. Which seems to put web developers at a sever disadvantage for search and streaming APIs since rate limits are based on IP addresses. Meaning all my web users count as one whereas the rate limiting is spread out among all the users a given desktop client. I asked a while back about this and didn't get a response. It just don't seem fair. Seems impossible to build a web app of anything more than a couple hundred users if those users want to use search and or streaming. Or correct me. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] User Streaming API and use of OAuth from web browser
I will indeed correct you: rate limits are based on account when using oauth. Tom On Oct 6, 2010, at 11:39 PM, Matthew Terenzio mteren...@gmail.com wrote: There would be one more issue which requires mentioning: JavaScript's Same-origin policy. You can't make a request directly to the Twitter API via JavaScript: you *will* need a proxy on your own server. Which seems to put web developers at a sever disadvantage for search and streaming APIs since rate limits are based on IP addresses. Meaning all my web users count as one whereas the rate limiting is spread out among all the users a given desktop client. I asked a while back about this and didn't get a response. It just don't seem fair. Seems impossible to build a web app of anything more than a couple hundred users if those users want to use search and or streaming. Or correct me. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] User Streaming API and use of OAuth from web browser
On Wed, Oct 6, 2010 at 5:48 PM, Tom van der Woerdt i...@tvdw.eu wrote: I will indeed correct you: rate limits are based on account when using oauth. Really? Can someone second that. I re-read the documentation and it doesn't look like it to me. Are the IP limits ignored when you log in as a user. I know that is the case for the REST api in most cases but I'm talking about streaming and search. Tom On Oct 6, 2010, at 11:39 PM, Matthew Terenzio mteren...@gmail.com wrote: There would be one more issue which requires mentioning: JavaScript's Same-origin policy. You can't make a request directly to the Twitter API via JavaScript: you *will* need a proxy on your own server. Which seems to put web developers at a sever disadvantage for search and streaming APIs since rate limits are based on IP addresses. Meaning all my web users count as one whereas the rate limiting is spread out among all the users a given desktop client. I asked a while back about this and didn't get a response. It just don't seem fair. Seems impossible to build a web app of anything more than a couple hundred users if those users want to use search and or streaming. Or correct me. -- Twitter developer documentation and resources: http://dev.twitter.com/dochttp://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] User Streaming API and use of OAuth from web browser
All the information about rate limits can be found on our developer site: http://dev.twitter.com/pages/rate-limiting When talking about rate limits it is important to be clear about the API being used, as each has their own. For the REST API (requests to api.twitter.com) the limit is 150 requests per hour unauthenticated and 350 request per hour for an authenticated user. When you make an authenticated request the users rate limit is affected, not the IPs. The Search API has it's own rate limit based on the IP the request comes from. There is no authenticating for Search so all requests are IP rate limited. The Streaming APIs do not have rate limits in the same way. For the Streaming API the rate limit is controlled by the predicate limits (5,000 user ids etc) and the allowed sampling rate (1% etc). I hope that clarifies how the rate limits apply. Best @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Wed, Oct 6, 2010 at 3:28 PM, Matthew Terenzio mteren...@gmail.com wrote: On Wed, Oct 6, 2010 at 5:48 PM, Tom van der Woerdt i...@tvdw.eu wrote: I will indeed correct you: rate limits are based on account when using oauth. Really? Can someone second that. I re-read the documentation and it doesn't look like it to me. Are the IP limits ignored when you log in as a user. I know that is the case for the REST api in most cases but I'm talking about streaming and search. Tom On Oct 6, 2010, at 11:39 PM, Matthew Terenzio mteren...@gmail.com wrote: There would be one more issue which requires mentioning: JavaScript's Same-origin policy. You can't make a request directly to the Twitter API via JavaScript: you *will* need a proxy on your own server. Which seems to put web developers at a sever disadvantage for search and streaming APIs since rate limits are based on IP addresses. Meaning all my web users count as one whereas the rate limiting is spread out among all the users a given desktop client. I asked a while back about this and didn't get a response. It just don't seem fair. Seems impossible to build a web app of anything more than a couple hundred users if those users want to use search and or streaming. Or correct me. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] User Streaming API and use of OAuth from web browser
So yes, I was correct (at least with search) that a web based solution is severely limited compared to a desktop. It will share usage among all it's users while a desktop client can spread the load amongst its users IPs. That stinks in my opinion. (I'm a web developer.) On Wed, Oct 6, 2010 at 7:44 PM, Matt Harris thematthar...@twitter.comwrote: All the information about rate limits can be found on our developer site: http://dev.twitter.com/pages/rate-limiting When talking about rate limits it is important to be clear about the API being used, as each has their own. For the REST API (requests to api.twitter.com) the limit is 150 requests per hour unauthenticated and 350 request per hour for an authenticated user. When you make an authenticated request the users rate limit is affected, not the IPs. The Search API has it's own rate limit based on the IP the request comes from. There is no authenticating for Search so all requests are IP rate limited. The Streaming APIs do not have rate limits in the same way. For the Streaming API the rate limit is controlled by the predicate limits (5,000 user ids etc) and the allowed sampling rate (1% etc). I hope that clarifies how the rate limits apply. Best @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Wed, Oct 6, 2010 at 3:28 PM, Matthew Terenzio mteren...@gmail.com wrote: On Wed, Oct 6, 2010 at 5:48 PM, Tom van der Woerdt i...@tvdw.eu wrote: I will indeed correct you: rate limits are based on account when using oauth. Really? Can someone second that. I re-read the documentation and it doesn't look like it to me. Are the IP limits ignored when you log in as a user. I know that is the case for the REST api in most cases but I'm talking about streaming and search. Tom On Oct 6, 2010, at 11:39 PM, Matthew Terenzio mteren...@gmail.com wrote: There would be one more issue which requires mentioning: JavaScript's Same-origin policy. You can't make a request directly to the Twitter API via JavaScript: you *will* need a proxy on your own server. Which seems to put web developers at a sever disadvantage for search and streaming APIs since rate limits are based on IP addresses. Meaning all my web users count as one whereas the rate limiting is spread out among all the users a given desktop client. I asked a while back about this and didn't get a response. It just don't seem fair. Seems impossible to build a web app of anything more than a couple hundred users if those users want to use search and or streaming. Or correct me. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] User Streaming API and use of OAuth from web browser
Well remember with Search you don't need to proxy from your server - instead the Search API supports JSONP so you can run it directly from the website. Regarding Toms proxy comment. I think Tom was suggesting it for the userstreams functionality. As userstreams require a long poll connection there are various other obstacles to overcome if it were to be run from within the browser directly. In addition, userstreams are for single user use and not suitable for web applications where multiple users interact. Instead the something like the Site Streams service (currently in beta) could be better suited. On Wed, Oct 6, 2010 at 5:18 PM, Matthew Terenzio mteren...@gmail.com wrote: So yes, I was correct (at least with search) that a web based solution is severely limited compared to a desktop. It will share usage among all it's users while a desktop client can spread the load amongst its users IPs. That stinks in my opinion. (I'm a web developer.) On Wed, Oct 6, 2010 at 7:44 PM, Matt Harris thematthar...@twitter.com wrote: All the information about rate limits can be found on our developer site: http://dev.twitter.com/pages/rate-limiting When talking about rate limits it is important to be clear about the API being used, as each has their own. For the REST API (requests to api.twitter.com) the limit is 150 requests per hour unauthenticated and 350 request per hour for an authenticated user. When you make an authenticated request the users rate limit is affected, not the IPs. The Search API has it's own rate limit based on the IP the request comes from. There is no authenticating for Search so all requests are IP rate limited. The Streaming APIs do not have rate limits in the same way. For the Streaming API the rate limit is controlled by the predicate limits (5,000 user ids etc) and the allowed sampling rate (1% etc). I hope that clarifies how the rate limits apply. Best @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Wed, Oct 6, 2010 at 3:28 PM, Matthew Terenzio mteren...@gmail.com wrote: On Wed, Oct 6, 2010 at 5:48 PM, Tom van der Woerdt i...@tvdw.eu wrote: I will indeed correct you: rate limits are based on account when using oauth. Really? Can someone second that. I re-read the documentation and it doesn't look like it to me. Are the IP limits ignored when you log in as a user. I know that is the case for the REST api in most cases but I'm talking about streaming and search. Tom On Oct 6, 2010, at 11:39 PM, Matthew Terenzio mteren...@gmail.com wrote: There would be one more issue which requires mentioning: JavaScript's Same-origin policy. You can't make a request directly to the Twitter API via JavaScript: you *will* need a proxy on your own server. Which seems to put web developers at a sever disadvantage for search and streaming APIs since rate limits are based on IP addresses. Meaning all my web users count as one whereas the rate limiting is spread out among all the users a given desktop client. I asked a while back about this and didn't get a response. It just don't seem fair. Seems impossible to build a web app of anything more than a couple hundred users if those users want to use search and or streaming. Or correct me. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources:
Re: [twitter-dev] User Streaming API and use of OAuth from web browser
Fair enough. On Wed, Oct 6, 2010 at 8:39 PM, Matt Harris thematthar...@twitter.comwrote: Well remember with Search you don't need to proxy from your server - instead the Search API supports JSONP so you can run it directly from the website. Regarding Toms proxy comment. I think Tom was suggesting it for the userstreams functionality. As userstreams require a long poll connection there are various other obstacles to overcome if it were to be run from within the browser directly. In addition, userstreams are for single user use and not suitable for web applications where multiple users interact. Instead the something like the Site Streams service (currently in beta) could be better suited. On Wed, Oct 6, 2010 at 5:18 PM, Matthew Terenzio mteren...@gmail.com wrote: So yes, I was correct (at least with search) that a web based solution is severely limited compared to a desktop. It will share usage among all it's users while a desktop client can spread the load amongst its users IPs. That stinks in my opinion. (I'm a web developer.) On Wed, Oct 6, 2010 at 7:44 PM, Matt Harris thematthar...@twitter.com wrote: All the information about rate limits can be found on our developer site: http://dev.twitter.com/pages/rate-limiting When talking about rate limits it is important to be clear about the API being used, as each has their own. For the REST API (requests to api.twitter.com) the limit is 150 requests per hour unauthenticated and 350 request per hour for an authenticated user. When you make an authenticated request the users rate limit is affected, not the IPs. The Search API has it's own rate limit based on the IP the request comes from. There is no authenticating for Search so all requests are IP rate limited. The Streaming APIs do not have rate limits in the same way. For the Streaming API the rate limit is controlled by the predicate limits (5,000 user ids etc) and the allowed sampling rate (1% etc). I hope that clarifies how the rate limits apply. Best @themattharris Developer Advocate, Twitter http://twitter.com/themattharris On Wed, Oct 6, 2010 at 3:28 PM, Matthew Terenzio mteren...@gmail.com wrote: On Wed, Oct 6, 2010 at 5:48 PM, Tom van der Woerdt i...@tvdw.eu wrote: I will indeed correct you: rate limits are based on account when using oauth. Really? Can someone second that. I re-read the documentation and it doesn't look like it to me. Are the IP limits ignored when you log in as a user. I know that is the case for the REST api in most cases but I'm talking about streaming and search. Tom On Oct 6, 2010, at 11:39 PM, Matthew Terenzio mteren...@gmail.com wrote: There would be one more issue which requires mentioning: JavaScript's Same-origin policy. You can't make a request directly to the Twitter API via JavaScript: you *will* need a proxy on your own server. Which seems to put web developers at a sever disadvantage for search and streaming APIs since rate limits are based on IP addresses. Meaning all my web users count as one whereas the rate limiting is spread out among all the users a given desktop client. I asked a while back about this and didn't get a response. It just don't seem fair. Seems impossible to build a web app of anything more than a couple hundred users if those users want to use search and or streaming. Or correct me. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi
