I would like to use bibTex data in a perl script from google scholar. These data are controlled in google scholar with the preference panel wich use cookies to remember the user's settings. bibTex are not displayed with the default settings.
Using the perl script I receive an empty page with the following url http://scholar.google.com/scholar.bib?num=100&hl=en&lr=&q=info:9Y4Rq3zllPUJ:scholar.google.com/&output=citation&oe=ASCII&oi=citation despite I try to set the same cookie as the cookie sets in my browser when "display bibTex links" is on. The same url give what I want if I send it in my browser with the correct preferences. My code goes like this: my $url1="http://scholar.google.com/scholar.bib? num=100&hl=en&lr=&q=info:9Y4Rq3zllPUJ:scholar.google.com/ &output=citation&oe=ASCII&oi=citation"; my $ua = LWP::UserAgent->new; $ua->agent('Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9'); my $c = HTTP::Cookies::Netscape->new(file=>"cookies.txt", autosave=>"1"); my $h = HTTP::Headers->new( Accept => "text/xml,application/xml,application/xhtml+xml,text/ html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5", Host => "www.unifr.ch", ); my $req =HTTP::Request->new(GET=>$url1,$h); $c->add_cookie_header($req); $ua->cookie_jar($c); my $res =$ua->request($req); die ($res->status_line) unless ($res->is_success); print $res->as_string; The output is HTTP/1.1 200 OK Cache-Control: private Connection: Close Date: Fri, 09 Nov 2007 16:04:42 GMT Server: GWS/2.1 Content-Length: 0 Content-Type: text/plain; charset=ISO-8859-1 Client-Date: Fri, 09 Nov 2007 16:04:44 GMT Client-Peer: 66.102.1.99:80 Client-Response-Num: 1 Set-Cookie: PREF=ID=8d4ee7c002f1dc72:TM=1194624282:LM=1194624282:S=vBeATmmCJPdAHCQy; expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.com Set-Cookie: GSP=ID=8d4ee7c002f1dc72; expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.scholar.unifr.ch The relevant line in my cookies file use by the script is .scholar.google.com TRUE / FALSE 2147368448 GSP ID=c77566a14ed11a16:IN=7e6cc990821af63+b8acc395c41ea61f:CF=4 The cookie data from my browser are name: GSP Value: ID=c77566a14ed11a16:IN=7e6cc990821af63+b8acc395c41ea61f:CF=4 Domain: .scholar.google.com Path: / ... Well the cookie from the script output seems to be different form the cookie I want to send. This is maybe the reason of my empty page but I can't understand why that cookie is changed ... Thanks for any help ! Francois
