[twitter-dev] Re: Do any of the libraries implement a method for returning statuses_count?
Ok. So then, in Python, which library do I need to inmport to parse the returned XML? I have simplejson (needed it for python-twitter) so maybe that would be easier? On Apr 13, 1:56 am, Peter Denton petermden...@gmail.com wrote: I dont know which specific libraries, but its a very common part of the API. For example, if you look at:http://twitter.com/users/show/khyron4eva.xml you will get xml back and are able to extract: $status_count = $xml - statuses_count; On Sun, Apr 12, 2009 at 12:57 PM, Khyron khyron4...@gmail.com wrote: If so, which one? If not, what is the best way to extract this piece of info? I'm a beginning programmer here, but I need this bit to accomplish my goal. -- You can choose your friends, you can choose the deals. - Equity Private AlphaGuy -http://alphaguy.blogspot.com On Twitter - @khyron4eva -- Peter M. Dentonwww.twibs.com i...@twibs.com Twibs makes Top 20 apps on Twitter -http://tinyurl.com/bopu6c
[twitter-dev] Re: Do any of the libraries implement a method for returning statuses_count?
On Mon, Apr 13, 2009 at 2:00 AM, Khyron khyron4...@gmail.com wrote:
Ok.
So then, in Python, which library do I need to inmport to parse the
returned
XML? I have simplejson (needed it for python-twitter) so maybe that
would
be easier?
Yes. If you decode the content you get from a status request, like this:
data = simplejson.loads(content)
The number of statuses you have retrieved will be the length of the
resulting object -- len(data)
However, you'll need to page through the results (make additional http
requests) to get more than 200 statuses. You can concatenate the received
content before parsing it. That would look like the following, in a loop:
if response.status == 200:
if
response['content-type'].startswith('application/json'):
try:
data = json.loads(content)
except:
print json decode failed: %s % (content)
return False
if 'next_page' in data:
next_page = data['next_page']
if 'results' in data:
data = data['results']
if not all:
return data
elif len(data) and i self.max_pages:
concat_content += (data)
[twitter-dev] Re: Do any of the libraries implement a method for returning statuses_count?
On Mon, Apr 13, 2009 at 7:43 AM, Nick Arnett nick.arn...@gmail.com wrote:
Ack chrome sent the reply before I was ready... trying to fix the white
space to make it proper Python, I hit tab and return, which sent the
message!
Anyway, you wouldn't need the lines that have to do with next_page or
results, those are for other kinds of requests. The idea is to increment a
page (page=1)parameter in your request URL in the loop, continuing until
you get no data (or a result code other than 200), which indicates that
you've gotten all the statuses Twitter is storing.
Let me try again to show example code here... There's some stuff here that
lets me other url parameters, which you might not need. self.http is an
instance of httplib2.Http(), self. max_pages is a variable for an upper
limit on the number of pages I get and it sets self.last_status_id to the id
of the last status it received. There's some simple error handling and
retries when the error seems to be something temporary.
I haven't tested this code, but I adapted it from something that is working
fairly reliably. The white space is screwed up
i = 0
concat_content = []
url_parameters = [count=200]
trys = 0
paged = True
page_url = http://twitter.com/statuses/user_timeline/%s.json; %
(twitter_id)
while 1:
i += 1
params = .join(url_parameters)
if paged:
if len(params):
page_url = url + ?%spage=%s % (params, i)
else:
page_url = url + ?page=%s % (i)
elif len(params):
page_url = url + ?%s % (params)
else:
page_url = url
response, content = self.http.request(page_url, method)
if response.status == 200:
if response['content-type'].startswith('application/json'):
try:
data = json.loads(content)
except:
print json decode failed: %s % (content)
return False
if len(data) and i self.max_pages:
concat_content += (data)
else:
self.last_status_id = concat_content[-1]['id']
return concat_content
if response.status in (500, 520, 503):
time.sleep(10)
trys += 1
if trys = 3:
print %s failed with internal errors % (status_type)
return False
else:
print %s failed: %s; Reason %s % (status_type, twitter_id,
response.reason)
return False
