[twitter-dev] Re: Twitter Search API - Questions Regarding Scaling Out
Thanks for the feedback Brian. Late response here, but I'd be more than willing to provide you with more details regarding our application in a private email. You should be receiving said email shortly. Regards, Corey On Apr 14, 1:12 pm, Brian Sutorius bsutor...@twitter.com wrote: While the Streaming API may not provide processed results to you in the way that search queries can (logical ORs, etc.), it's a more scalable solution for returning a lot of Tweets. Our search system can rate limit queries if they become too computationally expensive (in addition to the normal query limit), so continuing to add parameters to the query up front rather than doing this processing yourself may cause you to keep running into limits. Ultimately, circumventing the limits put in place by our APIs is not allowed by our API ToS, and building your architecture this way just to get around the defaults is something we strongly discourage. If you keep being rate limited, you should think about re-factoring your prioritization strategy. Can you go into a little more detail about what your application does? We might be able to guide you towards a mix of Streaming API and search queries that gets you what you need but stays within the rate limits. Brian Sutorius Twitter API Policy On Apr 13, 10:28 am, Corey Ballou ball...@gmail.com wrote: I'm still looking for a community leader answer on this one. On Apr 11, 5:50 pm, Corey Ballou ball...@gmail.com wrote: Thanks for the reply, I appreciate it. I have concerns regarding the streaming APIs, which mainly concern the following: * usage of logical OR when using locations * firehose limitations * the user’s location field is not used to filter tweets * increased application complexity for parsing the resulting stream of data back out into individual searches I know that the Search API is not Twitter's preferred choice, but it's currently returning the best applicable results for my application. It's also worth noting that the API recently received a drastic improvement to speed which should theoretically relax the strain on the API: http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faste... I guess I'm mainly interested in knowing whether @twitterapi will allow me to use the Search API in the manner I indicated above? Essentially I would be willing to guarantee the application worker nodes handles 420 rate limiting errors accordingly while still supporting multiple twitter accounts and searches. On Apr 11, 1:05 pm, M. Edward (Ed) Borasky zn...@borasky- research.net wrote: I don't see an answer here, but I'll tell you how *I* would go about implementing this: 1. Switch to the Streaming API. Using Search in an application puts a strain on Twitter's servers and makes it difficult to Twitter to manage capacity. That's why it's rate-limited and why the rate limits aren't publicly disclosed. 2. If your application is a desktop application, use User Streams. If it is a server, use User Streams on a desktop or the low-frequency free access to Streaming on a server to prototype and develop. Your target for a server will be Site Streams, but that's in closed beta at the moment IIRC. 3. *Concurrently with development*, your business development / sales / marketing / planning people, or yourself, if it's a one-person shop, should be negotiating with Twitter for access to Site Streams, I'm assuming an agile development methodology - customer-in-the-loop - and one of the parties that needs to be in the loop is Twitter for Site Streams. You simply *can't* build an at-scale Twitter application without direct business discussions with Twitter! On Mon, Apr 11, 2011 at 8:14 AM, Corey Ballou ball...@gmail.com wrote: I tried speaking with Ryan Sarver directly, but he's forwarding me here to the community advocates to answer. I believe this answer will need to come top down from Twitter, as it's your rate limiting that I'm most worried about. I have a technical question for all of you in regards to the Search API as I want to maintain full compliancy. Currently, the old Search API implementation (albeit slower) provides a fuller result set and allows for more flexibility in the types and combinations of searches allowed. The manner I have developed my application would allow for a number of daemonized worker instances running on different IP addresses to make calls to the search API on behalf of the stored OAuth credentials to avoid rate limiting issues. I had a conversation with the Pluggio developer in which he stated Twitter had threatened to shutdown his application if he didn't switch to a different implementation of the Search API. The problem indicated was that he was performing searches
[twitter-dev] window.open() and OAuth
Your new OAuth authentication handler does a check to determine if the window has been opened in a new window and triggers a resize. I'll preface this message by saying that I have a high res monitor at 1920x1600. I currently have handling to center the window. Your new JavaScript is essentially resizing the window outside of the viewport, giving no consideration to the end user's window height or current window position. Is this something that can be resolved? It's kind of a nuisance. I'll repost in the tracker. - Corey -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Announcing Twitter Devnest - May 12th (sign up now)
Hey Jason, We'd love to help you guys host a #devnest out here in Charlotte, NC if you ever swing by our way. Regards, Corey On Apr 27, 6:50 pm, Jason Costa jasonco...@twitter.com wrote: Hi everyone, On May 12th from 6:30pm to 8:30pm, we're going to be hosting a developer event, Twitter #devnestSF, here at our headquarters in San Francisco. Please RSVP to join us for the evening. Due to space constraints, we'll only be able to accommodate the first 200 sign-ups, and each RSVP is limited to one person. If you can't attend this time around, don't worry - this is going to be the first of many events to come. And for those not based in San Francisco, we're planning to do a string of dates in several cities around the US, and in other cities around the world. Plus, we're planning to record the event and release it on YouTube. More details will be released on that soon. Rather than create a new event format, we're choosing to embrace and extend what certain developers in the community are already doing - specifically the Devnest UK guys. Thanks to Angus Fox and Jonathan Markwell for letting us build on their work. Please RSVP here, and we look forward to seeing you on May 12th: http://twtvite.com/twitterdevnest Thanks, --Jason -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Twitter Search API - Questions Regarding Scaling Out
I'm still looking for a community leader answer on this one. On Apr 11, 5:50 pm, Corey Ballou ball...@gmail.com wrote: Thanks for the reply, I appreciate it. I have concerns regarding the streaming APIs, which mainly concern the following: * usage of logical OR when using locations * firehose limitations * the user’s location field is not used to filter tweets * increased application complexity for parsing the resulting stream of data back out into individual searches I know that the Search API is not Twitter's preferred choice, but it's currently returning the best applicable results for my application. It's also worth noting that the API recently received a drastic improvement to speed which should theoretically relax the strain on the API: http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faste... I guess I'm mainly interested in knowing whether @twitterapi will allow me to use the Search API in the manner I indicated above? Essentially I would be willing to guarantee the application worker nodes handles 420 rate limiting errors accordingly while still supporting multiple twitter accounts and searches. On Apr 11, 1:05 pm, M. Edward (Ed) Borasky zn...@borasky- research.net wrote: I don't see an answer here, but I'll tell you how *I* would go about implementing this: 1. Switch to the Streaming API. Using Search in an application puts a strain on Twitter's servers and makes it difficult to Twitter to manage capacity. That's why it's rate-limited and why the rate limits aren't publicly disclosed. 2. If your application is a desktop application, use User Streams. If it is a server, use User Streams on a desktop or the low-frequency free access to Streaming on a server to prototype and develop. Your target for a server will be Site Streams, but that's in closed beta at the moment IIRC. 3. *Concurrently with development*, your business development / sales / marketing / planning people, or yourself, if it's a one-person shop, should be negotiating with Twitter for access to Site Streams, I'm assuming an agile development methodology - customer-in-the-loop - and one of the parties that needs to be in the loop is Twitter for Site Streams. You simply *can't* build an at-scale Twitter application without direct business discussions with Twitter! On Mon, Apr 11, 2011 at 8:14 AM, Corey Ballou ball...@gmail.com wrote: I tried speaking with Ryan Sarver directly, but he's forwarding me here to the community advocates to answer. I believe this answer will need to come top down from Twitter, as it's your rate limiting that I'm most worried about. I have a technical question for all of you in regards to the Search API as I want to maintain full compliancy. Currently, the old Search API implementation (albeit slower) provides a fuller result set and allows for more flexibility in the types and combinations of searches allowed. The manner I have developed my application would allow for a number of daemonized worker instances running on different IP addresses to make calls to the search API on behalf of the stored OAuth credentials to avoid rate limiting issues. I had a conversation with the Pluggio developer in which he stated Twitter had threatened to shutdown his application if he didn't switch to a different implementation of the Search API. The problem indicated was that he was performing searches for multiple Twitter accounts, which is exactly my use case. Site streams does not make as much sense for my application given the search queries I wish to perform and the necessity for logical AND operations on geo-location. Do you foresee any problems with my current method of using different IP addresses to stay under the rate limit? I'm trying to stay in full compliance with Twitter's TOS and would love to find the most applicable and API friendly solution. I know headway is being made with Twitter's new search implementation so I would like to stay ahead of the curve and not get myself stuck in a box. I still need a method for polling for new search results (say, every 30 minutes, dependent upon the pricing plan) for non-logged in users. Below is a scaled down representation of how I'm currently handling searches to help you decide the best plan of action: 1) Searches are performed on a rolling queue basis, say one search every thirty minutes. There can be a finite number of searches per Twitter user (say 5 searches per Twitter account). There can be any number of Twitter accounts. 2) Search results are stored locally for retrieval by a javascript AJAX long-poller every minute to check for frequent changes. 3) When a user visits the search results page and filters results, no API calls to Twitter are made, only a local query is required Due to this process, the queue is constantly searching for the next searches
[twitter-dev] Twitter Search API - Questions Regarding Scaling Out
I tried speaking with Ryan Sarver directly, but he's forwarding me here to the community advocates to answer. I believe this answer will need to come top down from Twitter, as it's your rate limiting that I'm most worried about. I have a technical question for all of you in regards to the Search API as I want to maintain full compliancy. Currently, the old Search API implementation (albeit slower) provides a fuller result set and allows for more flexibility in the types and combinations of searches allowed. The manner I have developed my application would allow for a number of daemonized worker instances running on different IP addresses to make calls to the search API on behalf of the stored OAuth credentials to avoid rate limiting issues. I had a conversation with the Pluggio developer in which he stated Twitter had threatened to shutdown his application if he didn't switch to a different implementation of the Search API. The problem indicated was that he was performing searches for multiple Twitter accounts, which is exactly my use case. Site streams does not make as much sense for my application given the search queries I wish to perform and the necessity for logical AND operations on geo-location. Do you foresee any problems with my current method of using different IP addresses to stay under the rate limit? I'm trying to stay in full compliance with Twitter's TOS and would love to find the most applicable and API friendly solution. I know headway is being made with Twitter's new search implementation so I would like to stay ahead of the curve and not get myself stuck in a box. I still need a method for polling for new search results (say, every 30 minutes, dependent upon the pricing plan) for non-logged in users. Below is a scaled down representation of how I'm currently handling searches to help you decide the best plan of action: 1) Searches are performed on a rolling queue basis, say one search every thirty minutes. There can be a finite number of searches per Twitter user (say 5 searches per Twitter account). There can be any number of Twitter accounts. 2) Search results are stored locally for retrieval by a javascript AJAX long-poller every minute to check for frequent changes. 3) When a user visits the search results page and filters results, no API calls to Twitter are made, only a local query is required Due to this process, the queue is constantly searching for the next searches and mentions to perform. I foresee rate limiting concerns cropping up with searches being performed for any number of users. Can you steer me in the right direction to avoid shutdown notices or access revocation? Regards, Corey @cballou -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Twitter Search API - Questions Regarding Scaling Out
Thanks for the reply, I appreciate it. I have concerns regarding the streaming APIs, which mainly concern the following: * usage of logical OR when using locations * firehose limitations * the user’s location field is not used to filter tweets * increased application complexity for parsing the resulting stream of data back out into individual searches I know that the Search API is not Twitter's preferred choice, but it's currently returning the best applicable results for my application. It's also worth noting that the API recently received a drastic improvement to speed which should theoretically relax the strain on the API: http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html I guess I'm mainly interested in knowing whether @twitterapi will allow me to use the Search API in the manner I indicated above? Essentially I would be willing to guarantee the application worker nodes handles 420 rate limiting errors accordingly while still supporting multiple twitter accounts and searches. On Apr 11, 1:05 pm, M. Edward (Ed) Borasky zn...@borasky- research.net wrote: I don't see an answer here, but I'll tell you how *I* would go about implementing this: 1. Switch to the Streaming API. Using Search in an application puts a strain on Twitter's servers and makes it difficult to Twitter to manage capacity. That's why it's rate-limited and why the rate limits aren't publicly disclosed. 2. If your application is a desktop application, use User Streams. If it is a server, use User Streams on a desktop or the low-frequency free access to Streaming on a server to prototype and develop. Your target for a server will be Site Streams, but that's in closed beta at the moment IIRC. 3. *Concurrently with development*, your business development / sales / marketing / planning people, or yourself, if it's a one-person shop, should be negotiating with Twitter for access to Site Streams, I'm assuming an agile development methodology - customer-in-the-loop - and one of the parties that needs to be in the loop is Twitter for Site Streams. You simply *can't* build an at-scale Twitter application without direct business discussions with Twitter! On Mon, Apr 11, 2011 at 8:14 AM, Corey Ballou ball...@gmail.com wrote: I tried speaking with Ryan Sarver directly, but he's forwarding me here to the community advocates to answer. I believe this answer will need to come top down from Twitter, as it's your rate limiting that I'm most worried about. I have a technical question for all of you in regards to the Search API as I want to maintain full compliancy. Currently, the old Search API implementation (albeit slower) provides a fuller result set and allows for more flexibility in the types and combinations of searches allowed. The manner I have developed my application would allow for a number of daemonized worker instances running on different IP addresses to make calls to the search API on behalf of the stored OAuth credentials to avoid rate limiting issues. I had a conversation with the Pluggio developer in which he stated Twitter had threatened to shutdown his application if he didn't switch to a different implementation of the Search API. The problem indicated was that he was performing searches for multiple Twitter accounts, which is exactly my use case. Site streams does not make as much sense for my application given the search queries I wish to perform and the necessity for logical AND operations on geo-location. Do you foresee any problems with my current method of using different IP addresses to stay under the rate limit? I'm trying to stay in full compliance with Twitter's TOS and would love to find the most applicable and API friendly solution. I know headway is being made with Twitter's new search implementation so I would like to stay ahead of the curve and not get myself stuck in a box. I still need a method for polling for new search results (say, every 30 minutes, dependent upon the pricing plan) for non-logged in users. Below is a scaled down representation of how I'm currently handling searches to help you decide the best plan of action: 1) Searches are performed on a rolling queue basis, say one search every thirty minutes. There can be a finite number of searches per Twitter user (say 5 searches per Twitter account). There can be any number of Twitter accounts. 2) Search results are stored locally for retrieval by a javascript AJAX long-poller every minute to check for frequent changes. 3) When a user visits the search results page and filters results, no API calls to Twitter are made, only a local query is required Due to this process, the queue is constantly searching for the next searches and mentions to perform. I foresee rate limiting concerns cropping up with searches being performed for any number of users. Can you steer me in the right direction to avoid shutdown
[twitter-dev] Re: Access Tokens Changed?
I concur with David on this one. I didn't take the time to verify this scenario myself, but it does seem like it's a problem. Consider the following scenario: 1. A user has whitelisted 10+ web applications using their credentials. 2. The end user has no knowledge of what an access token is or what it entails. 3. The end user is forced to login using force_login to my application. 4. The end user hits Cancel during the authentication process. 5. The user's access token changes, revoking their access for all 10+ web applications. I guess the kicker is whether or not this is reproducible. If it is, this would seem to be a problem. Perhaps there is a workaround? On Dec 23, 11:58 am, David dtran...@gmail.com wrote: I feel like this isn't the expected behavior if a user hits Cancel when you authenticate with force_login=True - if start typing in another username, then hit cancel, it shouldn't revoke the access token for the currently authenticated user. -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Twitter Search API - Possible Proximity Bug
bump On Dec 22, 9:31 am, Corey Ballou ball...@gmail.com wrote: So... did this make it's way to the bug tracker? Is there any clarification if not? On Dec 21, 9:37 am, Corey Ballou ball...@gmail.com wrote: API call: http://search.twitter.com/search.json?q=ands=keggeocode=35.22708600... On Dec 20, 7:31 pm, Corey Ballou ball...@gmail.com wrote: It's also worth noting that while the above example may not be pulling a place, the API call I'll be including tomorrow uses the latitude/ longitude pair pulled from the Google Maps API V3 geocoder with the same outcome. On Dec 20, 7:22 pm, Corey Ballou ball...@gmail.com wrote: Here's my non API call searches from search.twitter.com with a 5mi range: keg near:121 W Trade St, Charlotte NC within:5mihttp://search.twitter.com/search?q=+keg+near%3A%22121+W+Trade+St%2C+C.. If you extend the radius to 10mi, you'll get significantly more results. My office is in the heart of Charlotte at a major intersection as well (Trade St and Tryon St). http://search.twitter.com/search?q=+keg+near%3A%22121+W+Trade+St%2C+C.. Based on the results returned from the 10mi radius, it seems to indicate that even though I tweeted from a Place, the tweet is being generalized to the parent city bounding box of Charlotte, NC. It just doesn't feel right; does it have to do with an optimization to avoid the heavy cost of performing calculations for sorting by distance? I'll update this post with the API call tomorrow morning when I get in the office. I've got it tucked away in the error logs with no root password in my keychain. In the meantime, any clarification or light you can shed on how you guys are calculating proximity would be much appreciated. Regards, Corey On Dec 20, 5:36 pm, Matt Harris thematthar...@twitter.com wrote: Im not entirely clear on how to reconstruct the query you are trying to make. Can you share the full Search URL request you are making so we can take a look. Thanks, @themattharris Developer Advocate, Twitterhttp://twitter.com/themattharris On Mon, Dec 20, 2010 at 1:16 PM, Corey Ballou ball...@gmail.com wrote: While running some tests on the search API I noticed a potential issue with the search API's proximity handling when filtering by places. If I specify a radius of 5 miles with a search term of keg and the address of my office building, I would expect to retrieve a previous tweet of mine: Tweet in question: http://twitter.com/cballou/statuses/15770007518584832 The status update in question was created with an associated place_id of my office, which properly maps to the right address. The place in question (Skookum, Charlotte, NC): https://search.twitter.com/search?q=place%3A5c9b53e1da87e502 Theoretically, I should be able to search within a one mile radius with the address. In this case, the only way I'm able to retrieve my tweet is to bump the radius up to 10 miles. I've tested this issue using both lat/lon coordinates with my application as well as using your advanced search (search.twitter.com/advanced) with the full address. Am I missing something here? Are you associating places to the overall city and not the exact lat/lon marker of an address? When I view the place directly, the marker placement is indicative of having the proper lat/lon coordinates. Let me know if you need any clarification or additional details. Regards, Corey -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Search API from_user_id doesn't match up with the proper Twitter user_id
I just wanted to bring group-wide awareness to the fact that search results from Twitter do not return an actual user_id. This has been a known defect (and yes, I do believe it's a *very large* defect) going on over 2 years now. This is a call to arms to get this shit fixed. I can't believe it's marked as an enhancement. There's nobody else to blame for providing a return param of from_user that doesn't actually map to an actual user. For those of us storing relational data, you're costing precious API calls for those users who are still utilizing the search API. The streaming API is not sufficient for all use cases, so that's not a valid answer. Below is the direct link to the issue tracker. https://code.google.com/p/twitter-api/issues/detail?id=214 -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Search API from_user_id doesn't match up with the proper Twitter user_id
Also, while it would be possible to use screen names for relations (i.e. from_user), this would have a very negative side effect. Mainly, if a user were to change their Twitter account name, previous relations would be lost. On Dec 22, 9:44 am, Corey Ballou ball...@gmail.com wrote: For clarification, I had intended to say from_user_id, as the username is returned properly. On Dec 22, 9:42 am, Corey Ballou ball...@gmail.com wrote: I just wanted to bring group-wide awareness to the fact that search results from Twitter do not return an actual user_id. This has been a known defect (and yes, I do believe it's a *very large* defect) going on over 2 years now. This is a call to arms to get this shit fixed. I can't believe it's marked as an enhancement. There's nobody else to blame for providing a return param of from_user that doesn't actually map to an actual user. For those of us storing relational data, you're costing precious API calls for those users who are still utilizing the search API. The streaming API is not sufficient for all use cases, so that's not a valid answer. Below is the direct link to the issue tracker. https://code.google.com/p/twitter-api/issues/detail?id=214 -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Search API from_user_id doesn't match up with the proper Twitter user_id
I'm sure I came off a little strong in the initial post; unfortunately for me google groups doesn't supply an edit button. I think there is still a grain of merit to the request to fix the issue, regardless of the API being free. I'm interest in knowing the trade-offs of Twitter essentially requiring third party apps to make subsequent calls to users/lookup for each unique user in a batch of results. The current problem I see, from Twitter's end, is that the subsequent call returns far more data than necessary. It's doubling the RTTs on both ends and creating an excessive amount of bandwidth for a trivial amount of data. I've got to imagine there's a number of cache misses going on due to the frequency of user updates and pulling the latest tweet, so it would seem rather costly. On Dec 22, 4:33 pm, Robbie Coleman rob...@robnrob.com wrote: I think twitter's response to this call to arms should be the HTTP Status Code: 420 - Chill ;-} -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Twitter Search API - Possible Proximity Bug
API call: http://search.twitter.com/search.json?q=ands=keggeocode=35.22708600%2C-80.84312600%2C5milang=allresult_type=recentrpp=50 On Dec 20, 7:31 pm, Corey Ballou ball...@gmail.com wrote: It's also worth noting that while the above example may not be pulling a place, the API call I'll be including tomorrow uses the latitude/ longitude pair pulled from the Google Maps API V3 geocoder with the same outcome. On Dec 20, 7:22 pm, Corey Ballou ball...@gmail.com wrote: Here's my non API call searches from search.twitter.com with a 5mi range: keg near:121 W Trade St, Charlotte NC within:5mihttp://search.twitter.com/search?q=+keg+near%3A%22121+W+Trade+St%2C+C.. If you extend the radius to 10mi, you'll get significantly more results. My office is in the heart of Charlotte at a major intersection as well (Trade St and Tryon St). http://search.twitter.com/search?q=+keg+near%3A%22121+W+Trade+St%2C+C.. Based on the results returned from the 10mi radius, it seems to indicate that even though I tweeted from a Place, the tweet is being generalized to the parent city bounding box of Charlotte, NC. It just doesn't feel right; does it have to do with an optimization to avoid the heavy cost of performing calculations for sorting by distance? I'll update this post with the API call tomorrow morning when I get in the office. I've got it tucked away in the error logs with no root password in my keychain. In the meantime, any clarification or light you can shed on how you guys are calculating proximity would be much appreciated. Regards, Corey On Dec 20, 5:36 pm, Matt Harris thematthar...@twitter.com wrote: Im not entirely clear on how to reconstruct the query you are trying to make. Can you share the full Search URL request you are making so we can take a look. Thanks, @themattharris Developer Advocate, Twitterhttp://twitter.com/themattharris On Mon, Dec 20, 2010 at 1:16 PM, Corey Ballou ball...@gmail.com wrote: While running some tests on the search API I noticed a potential issue with the search API's proximity handling when filtering by places. If I specify a radius of 5 miles with a search term of keg and the address of my office building, I would expect to retrieve a previous tweet of mine: Tweet in question: http://twitter.com/cballou/statuses/15770007518584832 The status update in question was created with an associated place_id of my office, which properly maps to the right address. The place in question (Skookum, Charlotte, NC): https://search.twitter.com/search?q=place%3A5c9b53e1da87e502 Theoretically, I should be able to search within a one mile radius with the address. In this case, the only way I'm able to retrieve my tweet is to bump the radius up to 10 miles. I've tested this issue using both lat/lon coordinates with my application as well as using your advanced search (search.twitter.com/advanced) with the full address. Am I missing something here? Are you associating places to the overall city and not the exact lat/lon marker of an address? When I view the place directly, the marker placement is indicative of having the proper lat/lon coordinates. Let me know if you need any clarification or additional details. Regards, Corey -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Twitter Search API - Possible Proximity Bug
While running some tests on the search API I noticed a potential issue with the search API's proximity handling when filtering by places. If I specify a radius of 5 miles with a search term of keg and the address of my office building, I would expect to retrieve a previous tweet of mine: Tweet in question: http://twitter.com/cballou/statuses/15770007518584832 The status update in question was created with an associated place_id of my office, which properly maps to the right address. The place in question (Skookum, Charlotte, NC): https://search.twitter.com/search?q=place%3A5c9b53e1da87e502 Theoretically, I should be able to search within a one mile radius with the address. In this case, the only way I'm able to retrieve my tweet is to bump the radius up to 10 miles. I've tested this issue using both lat/lon coordinates with my application as well as using your advanced search (search.twitter.com/advanced) with the full address. Am I missing something here? Are you associating places to the overall city and not the exact lat/lon marker of an address? When I view the place directly, the marker placement is indicative of having the proper lat/lon coordinates. Let me know if you need any clarification or additional details. Regards, Corey -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Twitter Search API - Possible Proximity Bug
Here's my non API call searches from search.twitter.com with a 5mi range: keg near:121 W Trade St, Charlotte NC within:5mi http://search.twitter.com/search?q=+keg+near%3A%22121+W+Trade+St%2C+Charlotte+NC%22+within%3A5mi http://search.twitter.com/search?q=ands=kegphrase=ors=nots=tag=lang=allfrom=to=ref=near=121+West+Trade+St%2C+Charlotte%2C+NCwithin=5units=misince=until=rpp=50 If you extend the radius to 10mi, you'll get significantly more results. My office is in the heart of Charlotte at a major intersection as well (Trade St and Tryon St). http://search.twitter.com/search?q=+keg+near%3A%22121+W+Trade+St%2C+Charlotte+NC%22+within%3A10mi http://search.twitter.com/search?q=ands=kegphrase=ors=nots=tag=lang=allfrom=to=ref=near=121+West+Trade+St%2C+Charlotte%2C+NCwithin=5units=misince=until=rpp=50 Based on the results returned from the 10mi radius, it seems to indicate that even though I tweeted from a Place, the tweet is being generalized to the parent city bounding box of Charlotte, NC. It just doesn't feel right; does it have to do with an optimization to avoid the heavy cost of performing calculations for sorting by distance? I'll update this post with the API call tomorrow morning when I get in the office. I've got it tucked away in the error logs with no root password in my keychain. In the meantime, any clarification or light you can shed on how you guys are calculating proximity would be much appreciated. Regards, Corey On Dec 20, 5:36 pm, Matt Harris thematthar...@twitter.com wrote: Im not entirely clear on how to reconstruct the query you are trying to make. Can you share the full Search URL request you are making so we can take a look. Thanks, @themattharris Developer Advocate, Twitterhttp://twitter.com/themattharris On Mon, Dec 20, 2010 at 1:16 PM, Corey Ballou ball...@gmail.com wrote: While running some tests on the search API I noticed a potential issue with the search API's proximity handling when filtering by places. If I specify a radius of 5 miles with a search term of keg and the address of my office building, I would expect to retrieve a previous tweet of mine: Tweet in question: http://twitter.com/cballou/statuses/15770007518584832 The status update in question was created with an associated place_id of my office, which properly maps to the right address. The place in question (Skookum, Charlotte, NC): https://search.twitter.com/search?q=place%3A5c9b53e1da87e502 Theoretically, I should be able to search within a one mile radius with the address. In this case, the only way I'm able to retrieve my tweet is to bump the radius up to 10 miles. I've tested this issue using both lat/lon coordinates with my application as well as using your advanced search (search.twitter.com/advanced) with the full address. Am I missing something here? Are you associating places to the overall city and not the exact lat/lon marker of an address? When I view the place directly, the marker placement is indicative of having the proper lat/lon coordinates. Let me know if you need any clarification or additional details. Regards, Corey -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Re: Twitter Search API - Possible Proximity Bug
It's also worth noting that while the above example may not be pulling a place, the API call I'll be including tomorrow uses the latitude/ longitude pair pulled from the Google Maps API V3 geocoder with the same outcome. On Dec 20, 7:22 pm, Corey Ballou ball...@gmail.com wrote: Here's my non API call searches from search.twitter.com with a 5mi range: keg near:121 W Trade St, Charlotte NC within:5mihttp://search.twitter.com/search?q=+keg+near%3A%22121+W+Trade+St%2C+C...http://search.twitter.com/search?q=ands=kegphrase=ors=nots=tag=;... If you extend the radius to 10mi, you'll get significantly more results. My office is in the heart of Charlotte at a major intersection as well (Trade St and Tryon St). http://search.twitter.com/search?q=+keg+near%3A%22121+W+Trade+St%2C+C...http://search.twitter.com/search?q=ands=kegphrase=ors=nots=tag=;... Based on the results returned from the 10mi radius, it seems to indicate that even though I tweeted from a Place, the tweet is being generalized to the parent city bounding box of Charlotte, NC. It just doesn't feel right; does it have to do with an optimization to avoid the heavy cost of performing calculations for sorting by distance? I'll update this post with the API call tomorrow morning when I get in the office. I've got it tucked away in the error logs with no root password in my keychain. In the meantime, any clarification or light you can shed on how you guys are calculating proximity would be much appreciated. Regards, Corey On Dec 20, 5:36 pm, Matt Harris thematthar...@twitter.com wrote: Im not entirely clear on how to reconstruct the query you are trying to make. Can you share the full Search URL request you are making so we can take a look. Thanks, @themattharris Developer Advocate, Twitterhttp://twitter.com/themattharris On Mon, Dec 20, 2010 at 1:16 PM, Corey Ballou ball...@gmail.com wrote: While running some tests on the search API I noticed a potential issue with the search API's proximity handling when filtering by places. If I specify a radius of 5 miles with a search term of keg and the address of my office building, I would expect to retrieve a previous tweet of mine: Tweet in question: http://twitter.com/cballou/statuses/15770007518584832 The status update in question was created with an associated place_id of my office, which properly maps to the right address. The place in question (Skookum, Charlotte, NC): https://search.twitter.com/search?q=place%3A5c9b53e1da87e502 Theoretically, I should be able to search within a one mile radius with the address. In this case, the only way I'm able to retrieve my tweet is to bump the radius up to 10 miles. I've tested this issue using both lat/lon coordinates with my application as well as using your advanced search (search.twitter.com/advanced) with the full address. Am I missing something here? Are you associating places to the overall city and not the exact lat/lon marker of an address? When I view the place directly, the marker placement is indicative of having the proper lat/lon coordinates. Let me know if you need any clarification or additional details. Regards, Corey -- Twitter developer documentation and resources:http://dev.twitter.com/doc API updates via Twitter:http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
