Re: doing a Google search from Clojure?

2013-03-22 Thread Armando Blancas 
Thanks, Anthony; will use that one.

On Friday, March 22, 2013 11:37:44 AM UTC-7, Anthony Grimes wrote:
>
> clojure-http-client is more or less unmaintained. 
> https://github.com/dakrone/clj-http is the canonical http client these 
> days.
>
> Lazybot has a plugin for doing this with the google ajax api, if that's 
> helpful. No API key needed. 
> https://github.com/flatland/lazybot/blob/develop/src/lazybot/plugins/google.clj
>
> On Friday, March 22, 2013 10:54:37 AM UTC-7, Armando Blancas wrote:
>>
>> Rich, you may want to check out clojure-http-client.
>> https://github.com/technomancy/clojure-http-client
>>
>> (require '[clj-http.client :as client])
>> (spit "result.html" (client/get "http://www.google.com/search?q=clojure
>> "))
>>
>> On Friday, March 22, 2013 12:09:07 AM UTC-7, Rich Morin wrote:
>>>
>>> I've been successfully using slurp and laser to harvest and pull 
>>> apart some web pages.  However, I can't figure out how to use 
>>> Google Search from my code. 
>>>
>>> My first thought was to use the Google Search API, but after 
>>> a lot of frustration in trying to get and use an API key, I 
>>> gave up on that. 
>>>
>>> My next thought was to slurp in a page from the interactive 
>>> Google Search facility, using the URL from Advanced Search: 
>>>
>>>   "http://www.google.com/search?hl=en&as_q=..."; 
>>>
>>> However, this gives me a 403 nastygram: 
>>>
>>>   IOException Server returned HTTP response code: 403 for URL: 
>>>   https://www.google.com/search?hl=en&as_q=&as_epq=... 
>>>   sun.net.www.protocol.http.HttpURLConnection.getInputStream 
>>>   (HttpURLConnection.java:1436) 
>>>
>>> Has anyone here, by chance, been able to do this sort of thing? 
>>>
>>> -r 
>>>
>>>  -- 
>>> http://www.cfcl.com/rdmRich Morin 
>>> http://www.cfcl.com/rdm/resume r...@cfcl.com 
>>> http://www.cfcl.com/rdm/weblog +1 650-873-7841 
>>>
>>> Software system design, development, and documentation 
>>>
>>>
>>>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread Anthony Grimes 
clojure-http-client is more or less unmaintained. 
https://github.com/dakrone/clj-http is the canonical http client these days.

Lazybot has a plugin for doing this with the google ajax api, if that's 
helpful. No API key 
needed. 
https://github.com/flatland/lazybot/blob/develop/src/lazybot/plugins/google.clj

On Friday, March 22, 2013 10:54:37 AM UTC-7, Armando Blancas wrote:
>
> Rich, you may want to check out clojure-http-client.
> https://github.com/technomancy/clojure-http-client
>
> (require '[clj-http.client :as client])
> (spit "result.html" (client/get "http://www.google.com/search?q=clojure";))
>
> On Friday, March 22, 2013 12:09:07 AM UTC-7, Rich Morin wrote:
>>
>> I've been successfully using slurp and laser to harvest and pull 
>> apart some web pages.  However, I can't figure out how to use 
>> Google Search from my code. 
>>
>> My first thought was to use the Google Search API, but after 
>> a lot of frustration in trying to get and use an API key, I 
>> gave up on that. 
>>
>> My next thought was to slurp in a page from the interactive 
>> Google Search facility, using the URL from Advanced Search: 
>>
>>   "http://www.google.com/search?hl=en&as_q=..."; 
>>
>> However, this gives me a 403 nastygram: 
>>
>>   IOException Server returned HTTP response code: 403 for URL: 
>>   https://www.google.com/search?hl=en&as_q=&as_epq=... 
>>   sun.net.www.protocol.http.HttpURLConnection.getInputStream 
>>   (HttpURLConnection.java:1436) 
>>
>> Has anyone here, by chance, been able to do this sort of thing? 
>>
>> -r 
>>
>>  -- 
>> http://www.cfcl.com/rdmRich Morin 
>> http://www.cfcl.com/rdm/resume r...@cfcl.com 
>> http://www.cfcl.com/rdm/weblog +1 650-873-7841 
>>
>> Software system design, development, and documentation 
>>
>>
>>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread Armando Blancas 
Rich, you may want to check out clojure-http-client.
https://github.com/technomancy/clojure-http-client

(require '[clj-http.client :as client])
(spit "result.html" (client/get "http://www.google.com/search?q=clojure";))

On Friday, March 22, 2013 12:09:07 AM UTC-7, Rich Morin wrote:
>
> I've been successfully using slurp and laser to harvest and pull 
> apart some web pages.  However, I can't figure out how to use 
> Google Search from my code. 
>
> My first thought was to use the Google Search API, but after 
> a lot of frustration in trying to get and use an API key, I 
> gave up on that. 
>
> My next thought was to slurp in a page from the interactive 
> Google Search facility, using the URL from Advanced Search: 
>
>   "http://www.google.com/search?hl=en&as_q=..."; 
>
> However, this gives me a 403 nastygram: 
>
>   IOException Server returned HTTP response code: 403 for URL: 
>   https://www.google.com/search?hl=en&as_q=&as_epq=... 
>   sun.net.www.protocol.http.HttpURLConnection.getInputStream 
>   (HttpURLConnection.java:1436) 
>
> Has anyone here, by chance, been able to do this sort of thing? 
>
> -r 
>
>  -- 
> http://www.cfcl.com/rdmRich Morin 
> http://www.cfcl.com/rdm/resume r...@cfcl.com  
> http://www.cfcl.com/rdm/weblog +1 650-873-7841 
>
> Software system design, development, and documentation 
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread juan.facorro 
I gave the code another look and remembered that *slurp* can actually 
handle a bunch of types as input, so I just passed the *InputStream *from 
the connection and got the same results. Additionaly in the code I posted 
before, the *get-response* function was never closing the stream, which *
slurp* does.

(ns google-search
  (:import [java.net URL URLEncoder]))
 (def google-search-url "http://www.google.com/search?q=";)
(def user-agent "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, 
like Gecko) Chrome/25.0.1364.172")
 (defn open-connection [url]
  (doto (.openConnection url)
(.setRequestProperty "User-Agent" user-agent)))
 (defn get-response [url]
  (let [conn (open-connection url)
sb   (StringBuilder.)]
(slurp (.getInputStream conn
 (defn search [query]
  (let [url (java.net.URL. (str google-search-url (URLEncoder/encode query)))]
(get-response url)))
(spit "response.html" (search "clojure google"))


J
On Friday, March 22, 2013 1:02:53 PM UTC-3, Rich Morin wrote:
>
> On Mar 22, 2013, at 08:00, juan.facorro wrote: 
> > Setting the user agent did the trick, at least in my case. 
>
> Thanks!  Using your code, I was able to bring in a page and 
> write it to a file.  I was then able to confirm that it had 
> the expected content.  FTW! 
>
> I still think this should be easier, but now I'm back on a 
> productive path. 
>
> -r 
>
>  -- 
> http://www.cfcl.com/rdmRich Morin 
> http://www.cfcl.com/rdm/resume r...@cfcl.com  
> http://www.cfcl.com/rdm/weblog +1 650-873-7841 
>
> Software system design, development, and documentation 
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread Rich Morin 
On Mar 22, 2013, at 08:00, juan.facorro wrote:
> Setting the user agent did the trick, at least in my case.

Thanks!  Using your code, I was able to bring in a page and
write it to a file.  I was then able to confirm that it had
the expected content.  FTW!

I still think this should be easier, but now I'm back on a
productive path.

-r

 -- 
http://www.cfcl.com/rdmRich Morin
http://www.cfcl.com/rdm/resume r...@cfcl.com
http://www.cfcl.com/rdm/weblog +1 650-873-7841

Software system design, development, and documentation


-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread Jim - FooBar(); 

On 22/03/13 15:20, Jim - FooBar(); wrote:

On 22/03/13 15:00, juan.facorro wrote:

(do
(.append sb (char c))


do you really need the 'do'?

Jim



ooops! I'm really sorry! my bad!

JIm

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread Juan Martín 
Yes, the *do *is necessary since the character needs to be appended to
the *StringBuilder
*and *recur *needs to be called after doing that.

I actually took the code from the
clojure.core/slurpfunction
:).

Cheers,

Juan

On Fri, Mar 22, 2013 at 12:20 PM, Jim - FooBar(); wrote:

>  On 22/03/13 15:00, juan.facorro wrote:
>
> (do
>   (.append sb (char c))
>
>
> do you really need the 'do'?
>
> Jim
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/QwKmsLwLhjE/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>



-- 
Juan Facorro

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread Jim - FooBar(); 

On 22/03/13 15:00, juan.facorro wrote:

(do
(.append sb (char c))


do you really need the 'do'?

Jim

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread juan.facorro 
Setting the user agent did the trick, at least in my case.

(ns google-search
  (:import [java.net URL URLEncoder]))
 
(def google-search-url "http://www.google.com/search?q=";)
(def user-agent "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, 
like Gecko) Chrome/25.0.1364.172")
 
(defn open-connection [url]
  (doto (.openConnection url)
(.setRequestProperty "User-Agent" user-agent)))
 
(defn get-response [url]
  (let [conn (open-connection url)
in   (.getInputStream conn)
sb   (StringBuilder.)]
(loop [c (.read in)]
  (if (neg? c)
(str sb)
(do 
  (.append sb (char c))
  (recur (.read in)))
 
(defn search [query]
  (let [url (URL. (str google-search-url (URLEncoder/encode query)))]
(get-response url)))
(spit "response.html" (search "URLEncoder java 7"))


HIH,


Juan

On Friday, March 22, 2013 4:32:33 AM UTC-3, Cedric Greevey wrote:
>
> Change your code to it spoofs a common browser user-agent, change your 
> DHCP-assigned IP address, and try again. They're probably trying to 
> obstruct bots from making overwhelming numbers of requests or something. As 
> long as you don't flood them with requests at a higher rate than a human 
> would generate by clicking, I don't see any ethical issue with 
> circumventing their countermeasures, especially not if the search will be 
> triggered by a user input to your application anyway.
>
>
> On Fri, Mar 22, 2013 at 3:09 AM, Rich Morin  >wrote:
>
>> I've been successfully using slurp and laser to harvest and pull
>> apart some web pages.  However, I can't figure out how to use
>> Google Search from my code.
>>
>> My first thought was to use the Google Search API, but after
>> a lot of frustration in trying to get and use an API key, I
>> gave up on that.
>>
>> My next thought was to slurp in a page from the interactive
>> Google Search facility, using the URL from Advanced Search:
>>
>>   "http://www.google.com/search?hl=en&as_q=...";
>>
>> However, this gives me a 403 nastygram:
>>
>>   IOException Server returned HTTP response code: 403 for URL:
>>   https://www.google.com/search?hl=en&as_q=&as_epq=...
>>   sun.net.www.protocol.http.HttpURLConnection.getInputStream
>>   (HttpURLConnection.java:1436)
>>
>> Has anyone here, by chance, been able to do this sort of thing?
>>
>> -r
>>
>>  --
>> http://www.cfcl.com/rdmRich Morin
>> http://www.cfcl.com/rdm/resume r...@cfcl.com 
>> http://www.cfcl.com/rdm/weblog +1 650-873-7841
>>
>> Software system design, development, and documentation
>>
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread Jonathan Fischer Friberg 
Found some info here:

http://stackoverflow.com/questions/3727662/how-can-you-search-google-programmatically-java-api

Jonathan


On Fri, Mar 22, 2013 at 8:32 AM, Cedric Greevey  wrote:

> Change your code to it spoofs a common browser user-agent, change your
> DHCP-assigned IP address, and try again. They're probably trying to
> obstruct bots from making overwhelming numbers of requests or something. As
> long as you don't flood them with requests at a higher rate than a human
> would generate by clicking, I don't see any ethical issue with
> circumventing their countermeasures, especially not if the search will be
> triggered by a user input to your application anyway.
>
>
> On Fri, Mar 22, 2013 at 3:09 AM, Rich Morin  wrote:
>
>> I've been successfully using slurp and laser to harvest and pull
>> apart some web pages.  However, I can't figure out how to use
>> Google Search from my code.
>>
>> My first thought was to use the Google Search API, but after
>> a lot of frustration in trying to get and use an API key, I
>> gave up on that.
>>
>> My next thought was to slurp in a page from the interactive
>> Google Search facility, using the URL from Advanced Search:
>>
>>   "http://www.google.com/search?hl=en&as_q=...";
>>
>> However, this gives me a 403 nastygram:
>>
>>   IOException Server returned HTTP response code: 403 for URL:
>>   https://www.google.com/search?hl=en&as_q=&as_epq=...
>>   sun.net.www.protocol.http.HttpURLConnection.getInputStream
>>   (HttpURLConnection.java:1436)
>>
>> Has anyone here, by chance, been able to do this sort of thing?
>>
>> -r
>>
>>  --
>> http://www.cfcl.com/rdmRich Morin
>> http://www.cfcl.com/rdm/resume r...@cfcl.com
>> http://www.cfcl.com/rdm/weblog +1 650-873-7841
>>
>> Software system design, development, and documentation
>>
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: doing a Google search from Clojure?

2013-03-22 Thread Cedric Greevey 
Change your code to it spoofs a common browser user-agent, change your
DHCP-assigned IP address, and try again. They're probably trying to
obstruct bots from making overwhelming numbers of requests or something. As
long as you don't flood them with requests at a higher rate than a human
would generate by clicking, I don't see any ethical issue with
circumventing their countermeasures, especially not if the search will be
triggered by a user input to your application anyway.


On Fri, Mar 22, 2013 at 3:09 AM, Rich Morin  wrote:

> I've been successfully using slurp and laser to harvest and pull
> apart some web pages.  However, I can't figure out how to use
> Google Search from my code.
>
> My first thought was to use the Google Search API, but after
> a lot of frustration in trying to get and use an API key, I
> gave up on that.
>
> My next thought was to slurp in a page from the interactive
> Google Search facility, using the URL from Advanced Search:
>
>   "http://www.google.com/search?hl=en&as_q=...";
>
> However, this gives me a 403 nastygram:
>
>   IOException Server returned HTTP response code: 403 for URL:
>   https://www.google.com/search?hl=en&as_q=&as_epq=...
>   sun.net.www.protocol.http.HttpURLConnection.getInputStream
>   (HttpURLConnection.java:1436)
>
> Has anyone here, by chance, been able to do this sort of thing?
>
> -r
>
>  --
> http://www.cfcl.com/rdmRich Morin
> http://www.cfcl.com/rdm/resume r...@cfcl.com
> http://www.cfcl.com/rdm/weblog +1 650-873-7841
>
> Software system design, development, and documentation
>
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

doing a Google search from Clojure?

2013-03-22 Thread Rich Morin 
I've been successfully using slurp and laser to harvest and pull
apart some web pages.  However, I can't figure out how to use
Google Search from my code.

My first thought was to use the Google Search API, but after
a lot of frustration in trying to get and use an API key, I
gave up on that.

My next thought was to slurp in a page from the interactive
Google Search facility, using the URL from Advanced Search:

  "http://www.google.com/search?hl=en&as_q=...";

However, this gives me a 403 nastygram:

  IOException Server returned HTTP response code: 403 for URL:
  https://www.google.com/search?hl=en&as_q=&as_epq=...
  sun.net.www.protocol.http.HttpURLConnection.getInputStream
  (HttpURLConnection.java:1436)

Has anyone here, by chance, been able to do this sort of thing?

-r

 -- 
http://www.cfcl.com/rdmRich Morin
http://www.cfcl.com/rdm/resume r...@cfcl.com
http://www.cfcl.com/rdm/weblog +1 650-873-7841

Software system design, development, and documentation


-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.