I believe so.
I have this in my pom..
> <dependency>
> <groupId>commons-httpclient</groupId>
> <artifactId>commons-httpclient</artifactId>
> <version>3.1</version>
> </dependency>
And my code looks like this:
Note: I override getResponseCharSeta and plugin the Mozilla charset
detector.
Also, I currently replace https:// with http:// because https is
throwing an exception.
Which is what I'm not trying to solve.
> public static GetMethod newGetMethod(String url2, int timeOut)
> throws IOException, HttpException, Exception {
> GetMethod getMethod;
> HttpClient httpClient = new HttpClient();
>
>
> String url = url2.replaceFirst("(?i)https://";, "http://";);
> System.out.println(url);
>
> httpClient.getHttpConnectionManager().getParams
> ().setConnectionTimeout(timeOut);
> httpClient.getHttpConnectionManager().getParams().setSoTimeout
> (timeOut);
>
> getMethod = new GetMethod(url) {
>
> String charset = null;
>
> @Override
> public String getResponseCharSet() {
> if (charset == null) {
> charset = super.getResponseCharSet();
> if (charset == null ||
> CHARSET_NOT_FOUND.equals(charset)) {
> try {
> charset =
> detectCharSet(getResponseBodyAsStream());
> } catch (Exception e) {
> charset = UTF_8;
> }
> }
> }
> return charset;
> }
> };
> getMethod.getParams().setUriCharset(UTF_8);
> getMethod.getParams().setContentCharset(CHARSET_NOT_FOUND);
>
> getMethod.setRequestHeader("User-Agent", "Mozilla/3.0 (Win95;
> I)");
> getMethod.addRequestHeader("Accept-Charset", UTF_8);
>
> getMethod.setFollowRedirects(true);
>
> int statusCode = httpClient.executeMethod(getMethod);
> if (statusCode != HttpStatus.SC_OK) {
> throw new Exception("GetMethod failed: " + statusCode +
> ": " +
> getMethod.getStatusLine());
> }
>
> return getMethod;
> }
On Sep 27, 2009, at 2:12 PM, Ken Krugler wrote:
> Hi Douglas,
>
> I'm assuming "commons version HttpClient" means version 3.1.
>
> I haven't directly set up https with 3.1, but I've used the Nutch
> protocol-httpclient plugin, which does.
>
> I'd suggest looking at the code in the Http class, e.g.
> configureClient() which does:
>
> // Set up an HTTPS socket factory that accepts self-signed certs.
> Protocol https = new Protocol("https",
> new DummySSLProtocolSocketFactory(), 443);
> Protocol.registerProtocol("https", https);
>
> where DummySSLProtocolSocketFactory is another class in the same
> package.
>
> All of this code is in the Nutch project, at
>
> src/plugin/protocol-httpclient/src/java/org/apache/nutch/protocol/
> httpclient
>
> -- Ken
>
> On Sep 27, 2009, at 11:25am, Douglas Ferguson wrote:
>
>> I did read that page, but I ran into a couple of issues.
>>
>> 1) This is deprecated: Protocol.registerProtocol("myhttps",
>> new Protocol("https", new MySSLSocketFactory(), 9443));
>>
>> 2) I wasn't sure which factory I should be using.
>>
>> 3) Is the contribution package avaible through a maven repository?
>>
>> I'm using commons version HttpClient.
>>
>>
>> On Sep 27, 2009, at 1:18 PM, Tobias N. Sasse wrote:
>>
>>> Douglas Ferguson schrieb:
>>>> I have been unsuccessful in finding information on how to setup
>>>> HttpClient to retrieve random https:// pages.
>>>>
>>>> Does anybody know how to do this
>>>
>>> You should look at the following pages:
>>>
>>> http://hc.apache.org/httpclient-3.x/sslguide.html (version 3)
>>> http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484
>>>
>>> HTH,
>>> Tobi
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
