[twitter-dev] Location Data From Stream API
OK my app basically provides a way for users to come to the site, and look at local tweets by city/state combo (I have to include state because a lot of states have identical city names). I WAS using the search API feature with geocodes to get local tweets and it worked PERFECTLY minus of course the limited data set problem -- but I obviously can't do that due to API call limits and having (hopefully :)) thousands of users per day searching for local tweets repeatedly. Now according to Raffi Krikorian search, however, attempts to use other signals to determine where the tweet is, and will attempt to return more tweets when you use its search parameter. it does not, however, expose those signals in the search results. Well, not having knowledge of those other signals... leaves me with pretty much nothing but the Location field to parse for location information. Right now I'm working on a DB search scheme to match likely city, state combos but other than that do you guys see any other methodology I may be overlooking?? The location field, unless it contains lon/lat coordinates, is a mess of garbage, nonsense, mispelled names, and a host of other useless noise. The ones that have lon/lat information in the tweet location field are perfect because then I can do my own radius calculations locally. But, for example, out of a 1.5 million tweet sample only 100,200 of those had lon/lat coordinates :(
Re: [twitter-dev] Location Data From Stream API
Parsing the location field is probably your best bet, but I'd say you have a challenging road ahead. It is indeed a mess, but there are geocoding solutions available to try and sort this stuff out. ---Mark http://twitter.com/mccv On Fri, Mar 5, 2010 at 1:04 PM, GeorgeMedia georgeme...@gmail.com wrote: OK my app basically provides a way for users to come to the site, and look at local tweets by city/state combo (I have to include state because a lot of states have identical city names). I WAS using the search API feature with geocodes to get local tweets and it worked PERFECTLY minus of course the limited data set problem -- but I obviously can't do that due to API call limits and having (hopefully :)) thousands of users per day searching for local tweets repeatedly. Now according to Raffi Krikorian search, however, attempts to use other signals to determine where the tweet is, and will attempt to return more tweets when you use its search parameter. it does not, however, expose those signals in the search results. Well, not having knowledge of those other signals... leaves me with pretty much nothing but the Location field to parse for location information. Right now I'm working on a DB search scheme to match likely city, state combos but other than that do you guys see any other methodology I may be overlooking?? The location field, unless it contains lon/lat coordinates, is a mess of garbage, nonsense, mispelled names, and a host of other useless noise. The ones that have lon/lat information in the tweet location field are perfect because then I can do my own radius calculations locally. But, for example, out of a 1.5 million tweet sample only 100,200 of those had lon/lat coordinates :(
Re: [twitter-dev] Location Data From Stream API
Quoting Mark McBride mmcbr...@twitter.com: Parsing the location field is probably your best bet, but I'd say you have a challenging road ahead. It is indeed a mess, but there are geocoding solutions available to try and sort this stuff out. Be *very* careful with geocoding solutions, especially taking note of the terms of service and licensing constraints. Google, Yahoo and Microsoft all have restrictions on what you can do with their tools. There are some open source / free as in freedom tools too, but they may be more limited. I've spent a number of hours recently working with various open source projects associated with mapping earthquake and other disaster zones, and this is a constant source of frustration. I'm guessing it would be even more a source of frustration if you're building marketing / sales tools rather than non-profit ones. People trapped in the rubble of a collapsed build usually *want* to be found; people sitting in a restaurant having a glass of wine with some friends might not. ;-) -- M. Edward (Ed) Borasky borasky-research.net/m-edward-ed-borasky/ A mathematician is a device for turning coffee into theorems. ~ Paul Erdos
