Re: simple spider in python

2007-09-01 Thread Lawrence D'Oliveiro
In message [EMAIL PROTECTED], Michael Bentley wrote: First thing to know is that google doesn't like the User-agent header urllib2 uses by default -- you'll have to masquerade as a browser (google throws a 403 error if you connect as 'User-Agent: Python- urllib/2.5': look into

Re: simple spider in python

2007-08-25 Thread samushack
On Aug 23, 4:49 pm, [EMAIL PROTECTED] wrote: well, i think that they deserve people moving toward yahoo's API... check this out:http://pysearch.sourceforge.net/ it's basically the same thing as pygoogle BUT working with google's competitor. it seems like internet has its own built-in

simple spider in python

2007-08-23 Thread gmcalendar
Hi everybody, i'm new to the forum so: hello everybody (should I say world?) ^_^ I'm trying to do a simple spider in python which: 1) ask google a query 2) parse the data I'm a python newbie so *any* help would be very, very welcommed. Thanks in advice! cheers! -- http://mail.python.org

Re: simple spider in python

2007-08-23 Thread Frederick Polgardy
On Aug 23, 8:33 am, [EMAIL PROTECTED] wrote: Hi everybody, i'm new to the forum so: hello everybody (should I say world?) ^_^ I'm trying to do a simple spider in python which: 1) ask google a query 2) parse the data I'm a python newbie so *any* help would be very, very welcommed. Thanks

Re: simple spider in python

2007-08-23 Thread samushack
I'm trying to do a simple spider in python which: 1) ask google a query 2) parse the data While you could use urllib2.urlopen() as Frederick mentioned, there is actually a Python module built JUST for getting info from Google queries! So check out PyGoogle: http://pygoogle.sourceforge.net

Re: simple spider in python

2007-08-23 Thread gmcalendar
thanks everybody, s kind. I'll take a look at booth. have a nice day/night (depending on your latitude!) ^_^ ciao! -- http://mail.python.org/mailman/listinfo/python-list

Re: simple spider in python

2007-08-23 Thread gmcalendar
Well, it turned out that google since Dec 2006 is not giving out SOAP api keys anymore. What a shame! any tip? ;-) -- http://mail.python.org/mailman/listinfo/python-list

Re: simple spider in python

2007-08-23 Thread Cameron Laird
In article [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: thanks everybody, s kind. I'll take a look at booth. have a nice day/night (depending on your latitude!) ^_^ ciao! Somewhere in the middle between the two suggestions you've already received is URL:

Re: simple spider in python

2007-08-23 Thread samushack
Somewhere in the middle between the two suggestions you've already received is URL:http://www.unixreview.com/documents/s=10133/ur0706e/. I followed that link, and got an error page... As to the Google API key issue, I was unaware of that. Very annoying of them to stop that service. PyGoogle

Re: simple spider in python

2007-08-23 Thread Michael Bentley
On Aug 23, 2007, at 6:33 AM, [EMAIL PROTECTED] wrote: Hi everybody, i'm new to the forum so: hello everybody (should I say world?) ^_^ I'm trying to do a simple spider in python which: 1) ask google a query 2) parse the data I'm a python newbie so *any* help would be very, very welcommed

Re: simple spider in python

2007-08-23 Thread gmcalendar
As to the Google API key issue, I was unaware of that. Very annoying of them to stop that service. PyGoogle will basically be useless. well, i think that they deserve people moving toward yahoo's API... check this out: http://pysearch.sourceforge.net/ it's basically the same thing as pygoogle