Re: [twitter-dev] Re: Requesting increased access levels for Streaming API
Thanks Taylor. So just to clarify, the 350 requests is per user account, not per server/ip address? We are creating a web application (not a desktop/mobile client) that will need to query account multiple times per hour. If the rate limits are per user account then we have no problem. If the rate limits are per server or ip address, and we even have a few dozen users then we would quickly be over the rate limit. Happy to use the REST API if that will work, though as we scale it likely means we will send many tens and then hundreds of thousands of requests per hour. The use case is that we are allowing people to backup their tweets (and other data types) and search them. Ultimately we will want to use site streams because we will waste a lot of processing power polling, but as long as the rate limits are per user account we are fine for now. Regards, Hank On Thu, Mar 17, 2011 at 6:36 PM, Taylor Singletary wrote: > Hi Hank, > We believe it to be entirely possible to build a web-based Twitter client > using only the REST API without whitelisting. Where are you thinking that > you would require it? Site Streams makes it easier in some ways, though the > implementation can be more complicated and intensive. > By requiring that your end-users authenticate a Twitter account, you can > execute ~350 authenticated GET requests per hour on behalf of that user from > your server's IP address. There are 24 hours in a day. That's 8,400 > authenticated GET requests you can make on their behalf per day, in which > you're fetching timelines for them, user profile metadata, and so on. > If there are specific actions you can't perform for a certain user within > 350 requests in a given hour, you queue the rest of the activity and ask the > user to wait until you can process the data for them. > Interested to know where a whitelisting requirement fits in with your use > case. > @episod - Taylor Singletary - Twitter Developer Advocate > > > On Thu, Mar 17, 2011 at 3:22 PM, hank williams wrote: >> >> Ryan, >> >> I have asked this a few times, (every time you mention using site >> streams) and I realize everyone at twitter is really busy, but it >> would be really helpful to know whether it is possible to write >> twitter web based apps right now given that there is no whitelisting, >> and site streams seems to be in closed beta. It would seem without >> site streams, creating webapps that use twitter would be impossible. >> If there is some workaround that I don't know about, please let me >> know. >> >> Thanks, >> Hank >> >> On Thu, Mar 17, 2011 at 6:09 PM, Ryan Sarver wrote: >> > Ed, I'm not sure what you mean by: "You need to get *all* your users to >> > *explicitly* authorize the application's *exact* usage of their data!" >> > Of course! that is exactly what we are saying and I'm not sure if you're >> > really saying you shouldn't get the user's authorization as that doesn't >> > make sense. >> > I don't expect everyone to be able to use User Streams or Site Streams, >> > but >> > that is why the REST API exists. >> > >> > -- >> > Ryan Sarver >> > @rsarver >> > >> > >> > On Wed, Mar 16, 2011 at 8:52 PM, M. Edward (Ed) Borasky >> > wrote: >> >> >> >> On Wed, 16 Mar 2011 09:10:13 -0700 (PDT), "Ryan Sarver (@rsarver)" >> >> wrote: >> >>> >> >>> Also as we stated before, you can use User Streams or Site Streams and >> >>> get more data by getting more users to authorize your application. >> >> >> >> Ryan, it's not as simple as "getting more users to authorize your >> >> application." You need to get *all* your users to *explicitly* >> >> authorize the >> >> application's *exact* usage of their data! Users tend not to "read the >> >> fine >> >> print". I'd hate to see some data collection / analytics application >> >> make >> >> some assumptions based on the implicit openness of the tweet stream and >> >> then >> >> get nailed by a bunch of angry users. Angry users tend to write to >> >> their >> >> Congressmen and Senators. ;-) >> >> >> >> Managing a *single* user's User Streams feed is a relatively >> >> straightforward coding task - I've got a smallish Perl script that can >> >> do it >> >> for my own account. Managing multiple users' Site Streams is a much >> >> more >
Re: [twitter-dev] Re: Requesting increased access levels for Streaming API
Ryan, I have asked this a few times, (every time you mention using site streams) and I realize everyone at twitter is really busy, but it would be really helpful to know whether it is possible to write twitter web based apps right now given that there is no whitelisting, and site streams seems to be in closed beta. It would seem without site streams, creating webapps that use twitter would be impossible. If there is some workaround that I don't know about, please let me know. Thanks, Hank On Thu, Mar 17, 2011 at 6:09 PM, Ryan Sarver wrote: > Ed, I'm not sure what you mean by: "You need to get *all* your users to > *explicitly* authorize the application's *exact* usage of their data!" > Of course! that is exactly what we are saying and I'm not sure if you're > really saying you shouldn't get the user's authorization as that doesn't > make sense. > I don't expect everyone to be able to use User Streams or Site Streams, but > that is why the REST API exists. > > -- > Ryan Sarver > @rsarver > > > On Wed, Mar 16, 2011 at 8:52 PM, M. Edward (Ed) Borasky > wrote: >> >> On Wed, 16 Mar 2011 09:10:13 -0700 (PDT), "Ryan Sarver (@rsarver)" >> wrote: >>> >>> Also as we stated before, you can use User Streams or Site Streams and >>> get more data by getting more users to authorize your application. >> >> Ryan, it's not as simple as "getting more users to authorize your >> application." You need to get *all* your users to *explicitly* authorize the >> application's *exact* usage of their data! Users tend not to "read the fine >> print". I'd hate to see some data collection / analytics application make >> some assumptions based on the implicit openness of the tweet stream and then >> get nailed by a bunch of angry users. Angry users tend to write to their >> Congressmen and Senators. ;-) >> >> Managing a *single* user's User Streams feed is a relatively >> straightforward coding task - I've got a smallish Perl script that can do it >> for my own account. Managing multiple users' Site Streams is a much more >> complex endeavor, and to use that mechanism for a data collection / >> analytics application is ludicrous IMHO. Somehow, the notion of "the right >> tool for the job" seems to have been ignored. ;-) >> >> -- >> http://twitter.com/znmeb http://borasky-research.net >> >> "A mathematician is a device for turning coffee into theorems." -- Paul >> Erdős >> >> -- >> Twitter developer documentation and resources: http://dev.twitter.com/doc >> API updates via Twitter: http://twitter.com/twitterapi >> Issues/Enhancements Tracker: >> http://code.google.com/p/twitter-api/issues/list >> Change your membership to this group: >> http://groups.google.com/group/twitter-development-talk > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk > -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Re: Requesting increased access levels for Streaming API
Ryan, Are site streams actually available? As far as I can tell whitelisting is gone and site streams are in private beta meaning that it is now impossible to launch a website that needs access to twitter data. if I have misread the situation, please advise. Thanks, Hank On Mar 16, 2011, at 12:10 PM, "Ryan Sarver (@rsarver)" wrote: > We should have been more clear, but elevated levels of streaming was > included in the previous statement about ending the whitelisting > program. There are open levels for each stream or you can contact Gnip > if you are looking for elevated access for the purposes of data > analysis. > > Also as we stated before, you can use User Streams or Site Streams and > get more data by getting more users to authorize your application. > > Hope that helps clarify. > > Best, Ryan > > On Mar 16, 1:47 am, Scott Wilcox wrote: >> Highly unlikely. At the present time it's either the Streaming API or using >> GNIP. >> >> I don't believe there are any use cases where they would provide you with >> elevated Streaming API access to the level you desire. >> >> Sent from my iPhone >> >> On 16 Mar 2011, at 04:23, "manusis" wrote: >> >> >> >> >> >> >> >>> Yeah I went through gnip in detail but their pricing is excessively >>> expensive especially when I care only about twitter data and not the >>> hundred other sources that they provide. I was hoping that if not >>> "partner track", twitter might be open to give at least "restricted >>> track" access to developers. >> >>> On Mar 15, 8:10 pm, hax0rsteve wrote: From that same post :http://groups.google.com/group/twitter-development-talk/browse_thread... >> > Developers > interested in elevated access to the Twitter stream for the purpose of > research or analytics can contact our partner Gnip for more > information. >> Fromhttp://gnip.com/ >> > Gnip and Twitter have partnered to bring more Twitter feeds to Gnip > customers. Check out Power Track for 100% guaranteed coverage firehose > filtering and all commercial Twitter data, only from Gnip. >> Fromhttp://gnip.com/twitter/power-track >> >• The only feed of its kind: Twitter firehose filtering with 100% > coverage guaranteed >• Boolean operators, unwound URLs, and matching within unwound URLs > supported >• Keyword, username, and location filtering supported >• Unlimited capacity: no restrictions on filter parameters or results > volume - Premium Feed >• Pay for what you get - pricing depends on Tweet volume delivered - > Premium Feed >• Contact i...@gnip.com for more information - Premium Feed >> HTH >> On 15 Mar 2011, at 15:04, manusis wrote: >> > Thanks Augusto. >> > But the same thread indicates that tools like Streaming API will > replace whitelisting. So it does not make sense for me for Streaming > API to put under the same umbrella as whitelisting. >> > "Since then, we've added new, more efficient tools for developers, > including lookups, ID lists, authentication and the Streaming API. > Instead of whitelisting, developers can use these tools to create > applications and integrate with the Twitter platform." >> > On Mar 15, 7:41 pm, Augusto Santos wrote: >> I think the answer is you never will. >> This kind of benefit might follow the same rules that whitelist, that >> will >> no longer be supported just as the thread below >> said.http://groups.google.com/group/twitter-development-talk/browse_thread... >> >> On Tue, Mar 15, 2011 at 6:58 AM, manusis wrote: >>> The streaming API mentions about different access roles but does not >>> indicate how one could apply for them. >> >>> "The default access level allows up to 400 track keywords, 5,000 >>> follow userids and 25 0.1-360 degree location boxes. Increased access >>> levels allow 100,000 follow userids (“shadow” role), 400,000 follow >>> userids (“birddog” role), 10,000 track keywords (“restricted track” >>> role), 200,000 track keywords (“partner track” role), and 200 0.1-360 >>> degree location boxes (“locRestricted” role). Increased track access >>> levels also pass a higher proportion of statuses before limiting the >>> stream." >> >>> For our product, we need "shadow" and "partner track" access roles. >>> Could somebody shed any light on how one could apply for the increased >>> access levels? >> >>> Thanks, >>> Rajiv >> >>> -- >>> Twitter developer documentation and resources:http://dev.twitter.com/doc >>> API updates via Twitter:http://twitter.com/twitterapi >>> Issues/Enhancements Tracker: >>> http://code.google.com/p/twitter-api/issues/list >>> Change your membership to this group: >>> http://groups.google.com/group/twitter-development-talk >> >> -- >> 氣 >> > -- > Twitter developer documentation
Re: [twitter-dev] Re: wait time for Site Streams whitelist ?
I understand the APIs from a technical perspective. The issue is about twitter *policy* around rate limits and connection limits. As a non whitelisted app we will quickly run into rate limits so I am just trying to understand what the current rules are. On Mar 2, 2011, at 10:52 AM, Adam Green <140...@gmail.com> wrote: > There are lots of ways to create website apps without using site > streams. If you describe the type of things you want to do here, you > will get plenty of advice. You don't have to reveal any secret plans. > Just describe the type of data you need from Twitter and the type of > changes you need to make to user accounts. > > On Wed, Mar 2, 2011 at 10:37 AM, hank777 wrote: >> I just registered for site streams this morning because I did not >> realize that this was the only authorized way of building a website >> that uses twitter. We are trying to demo something at SXSW, and launch >> a few weeks after, and I am wondering if I cant use the other APIs >> because of rate limits or connection limits, and I cant use site >> streams because you are not accepting new developers, how do we move >> forward, or does this really mean no one should be developing new web >> site based applications right now if they are not in the beta program? >> >> Thanks, >> Hank >> >> On Feb 1, 10:36 am, Taylor Singletary >> wrote: >>> We still have a number of tasks to accomplish before we can move to the next >>> stage of the beta. Pending applications will be reviewed once they are >>> actionable. >>> >>> Thanks for your patience. >>> >>> Taylor >>> >>> >>> >>> >>> >>> >>> >>> On Mon, Jan 31, 2011 at 7:20 PM, Jan Paricka wrote: Weeks upon weeks upon weeks. No joking. >>> Jan >>> On Tue, Feb 1, 2011 at 4:14 AM, paloalto wrote: >>> > How long does it take to get whitelisted for Site Streams API? > I submitted a form to be whitelisted and have not received any > confirmation e-mail. > I am in the dark with no clue as to how long I should expect to wait. >>> > Sungho >>> > -- > Twitter developer documentation and resources:http://dev.twitter.com/doc > API updates via Twitter:http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk >>> -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk >> >> -- >> Twitter developer documentation and resources: http://dev.twitter.com/doc >> API updates via Twitter: http://twitter.com/twitterapi >> Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list >> Change your membership to this group: >> http://groups.google.com/group/twitter-development-talk >> > > > > -- > Adam Green > Twitter API Consultant and Trainer > http://140dev.com > @140dev > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
