I've been making my headers very much like a proxy would, even sending
X-forwarded-for headers with the original client ip and such. Sending
very close to the same thing that would be sent if the user really was
using a web proxy. Seems to be keeping google happy... so far.
Jonathan
Michael Beccaria wrote:
Not everyone will care, but I will put it in here for posterity sake and
probably for my own reference when I forget in the future.
I was having trouble getting the new google books dynamic link api to
work right with python
(http://code.google.com/apis/books/docs/dynamic-links.html). I was using
the basic urllib python library with a non-working code base that looks
like this:
import urllib,urllib2
gparams = urllib.urlencode({'bibkeys': 'ISBN:061837943',
'jscmd':'viewapi','callback':'mycallback'})
g=urllib2.urlopen(url=http://books.google.com/books?%s; % gparams)
print g.read()
I was getting an http 401 error, Unauthorized. Code4lib IRC folks told
me it was probably the headers urllib was sending, and they were right.
I wrote code to modify the headers to make google believe I was
requesting from firefox. The working code is below. I know most of you
can write this stuff in your sleep, but I thought this might save
someone like me some time in the end.
Hope it helps,
Mike Beccaria
Systems Librarian
Head of Digital Initiatives
Paul Smith's College
518.327.6376
[EMAIL PROTECTED]
import urllib,urllib2
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
params = urllib.urlencode({'bibkeys': 'ISBN:061837943',
'jscmd':'viewapi','callback':'mycallback'})
request =
urllib2.Request('http://books.google.com/books?bibkeys=0618379436jscmd=
viewapicallback=mycallback')
opener.addheaders = [('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT
5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')]
data = opener.open(request).read()
print data
--
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886
rochkind (at) jhu.edu
