#!/usr/bin/env perl
use LWP::UserAgent;
my $ua = LWP::UserAgent->new(
env_proxy => 1,
keep_alive => 1,
timeout => 30,
agent => "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) "
);
#doing a search for keywords 'perl' and 'qt'
my
$response=$ua->get("http://www.google.com/search?q=qt+perl&ie=UTF-8&oe=UTF-8&hl=no&lr=";);
print $response->content;
#
# The Useragent header is what does the trick here. The others
# arent really necessary
#
Regards,
Torbj�rn Lindahl
On Tue, 1 Oct 2002, John Von Essen wrote:
> I am not sure if this has been covered already, but here goes:
>
> I am using LWP::UserAgent to perform a GET request (search query) on
> google.com.
>
> I guess google.com got tired of people running autonomous queries through
> their portal because they apparently check-out the client now to see of it
> is a real web browser or just a Perl script in disguise. My Perl script
> get the following 403 Error from google.com when I attempt a query:
>
> .....
> Forbidden
>
> Your client does not have permission to get URL xxx (Client IP address:
> xxx)
> Please see Google's Terms of Service posted at
> http://www.google.com/terms_of_service.html
> .....
>
> I though I could fool google.com by just setting my UA to Mozilla/4.0 but
> that didn't help. What's the secret? What other things do I have to forge
> to get google.com to believe that I am a browser?
>
>
> Thanks.
>
> -John
>
