On Jun 15, 2008, at 5:48 PM, Sarabjeet wrote: > > > I am working on a project where I need to crawl google search to > download the first 50 results for a hundred queries and store them on > my computer. I do not need a webpage or something. What is the best > API for me to use because I found out that google does not allow > client programs to crawl their websites directly. Also, I am not at > all familiar with Javascipt but I know Python. The Python gdata > library, unfortunately, does not have the search APIs.
There is a restful interface to Google search: http://code.google.com/apis/ajaxsearch/documentation/reference.html#_intro_fonje You will need a valid referrer page, perhaps something describing your project and how to contact you. There is some limitation on the number of results that can be obtained, but I don't see it documented. In my brief testing, the current limitation is 100 results, ie. http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Google&start=24 returns results, buthttp://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=Google&start=25 doesn't. The results are json. The query ignores any alt=xml parameter. Ray --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google Data Protocol" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-help-dataapi?hl=en -~----------~----~----~----~------~----~------~--~---
