Re: Fully Android compatible HttpClient 4.1.x; was Re: HttpClient-Cache release that works against HttpClient 4.0?

[Oleg Kalnichevski]

On Sun, 2012-07-08 at 06:39 +, Mayur Rustagi wrote:
> Hi Guys,
> Any progress on this? This would work wonders with my application.
> Regards
> Mayur
> 


Not much progress, unfortunately. I have my hands full with the next
release of HttpAsyncClient and a major refactoring of HttpClient in the
4.3 branch. It might well happen that 4.3 release would technically be
the best opportunity for an Android port. Pretty much all high level
impl classes shipped with Android will be deprecated by then and
replaced with new implementations, making it much easier to provide
backward compatibility with previous releases.  

Anyway, help would be welcome.

Oleg


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Fully Android compatible HttpClient 4.1.x; was Re: HttpClient-Cache release that works against HttpClient 4.0?

[Mayur Rustagi]

Hi Guys,
Any progress on this? This would work wonders with my application.
Regards
Mayur


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Fully Android compatible HttpClient 4.1.x; was Re: HttpClient-Cache release that works against HttpClient 4.0?

[Oleg Kalnichevski]

candrews  wrote:

>
>That idea sounds awesome to me - I may even do it. How would you like
>such a
>thing implemented? As a script? A diff against "regular" httpclient?
>

Ideally a script that svn copies files off a release tag, say 4.1.2 and a diff 
containing adjustments. That would be the ideal case.


>Interestingly, for the case of HttpClient-Cache, my original problem
>would
>not be solved by this approach. HttpClient-Cache uses
>EntityUtils.consume
>statically, and there's no way to fix that. Perhaps a simple change to
>not
>use EntityUtils.consume and thus conform to the HttpClient 4.0 API
>would be
>alright? Also, I confirmed with that change, HttpClient-Cache works in
>Android (the jar, that is) and the compiled jar only uses HttpClient
>4.0
>APIs. The unit tests, however, use methods from beyond the 4.0 API and
>thus
>do not run against HttpClient 4.0.
>

There is absolutely nothing that prevents you from shading that class and 
making the cache module use that shaded class instead of the old one. Only 
public interfaces and public classes referenced by those interfaces cannot be 
shaded.

Oleg


>
>olegk wrote:
>> 
>> On Thu, Nov 10, 2011 at 11:49:09AM -0800, candrews wrote:
>>> 
>>> I thought of that, but it won't work when an Android app has
>dependencies
>>> that use HttpClient.
>>> 
>>> I use Spring Android Rest Template. My goal is to use Rest Template
>with
>>> caching via HttpClient-Cache. If I shade HttpClient (and
>>> HttpClient-Cache),
>>> and include the shaded jar in my project, then Rest Template will
>not use
>>> it
>>> (as it's compiled to use the "real" HttpClient, not my private copy
>in a
>>> new
>>> namespace). So now I have to process Rest Template using Maven
>Shade. I'd
>>> also like osmdroid to use the same HttpClient, so I have to shade it
>>> too...
>>> Now I'm using at least 4 custom created jar's outside of my Android
>app's
>>> ant build system, causing a serious maintenance problem :-)
>>> 
>>> Is there an ant analogue to Maven Shade? Is there a different
>approach to
>>> solving these problems?
>>> 
>> 
>> There is a way to build a reasonably up-to-date version of HttpClient
>> fully compatible with Android: shade impl classes and new interfaces
>only
>> and make them compileable against 4.0 API. That would involve a bit
>more
>> work but result in a library fully compatible with Android and 3rd
>party
>> libraries at the same time.
>> 
>> If someone were to contribute such a port to the project, we probably
>> could even ship it as an official ASF release.
>> 
>> Oleg
>> 
>> 
>> 
>>> (I apologize if I'm hijacking this list inappropriately - IMHO this
>seems
>>> relevant to httpclient-user, and I'm not sure where else to ask.
>I'll
>>> happily shut up if the list maintainer believes this thread is
>>> off-topic).
>>> 
>>> Thanks again!
>>> ~Craig
>>> 
>>> 
>>> sebb-2-2 wrote:
>>> > 
>>> > On 9 November 2011 04:05, candrews 
>wrote:
>>> >>
>>> >> Maven Shade isn't compatible with maven-android-plugin:
>>> >>
>https://code.google.com/p/maven-android-plugin/issues/detail?id=170
>>> >> Perhaps
>>> >> when that is resolved, Shade will be a solution for Maven users.
>But
>>> what
>>> >> about Ant users (which I think are what most of the Android apps
>are
>>> >> built
>>> >> with)?
>>> > 
>>> > It would also be possible to use the Maven Shade plugin to create
>a
>>> > version of the HC jars with a different package name.
>>> > This would be done independently of any Android build, whether
>using
>>> > Maven or Ant.
>>> > 
>>> > You then build the Android stuff against the shaded jar, using the
>new
>>> > package names.
>>> > 
>>> > This is not an ideal long-term solution, but should be sufficient
>for
>>> > prototyping purposes.
>>> > 
>>> >
>-
>>> > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>>> > For additional commands, e-mail:
>httpclient-users-h...@hc.apache.org
>>> > 
>>> > 
>>> > 
>>> 
>>> -- 
>>> View this message i

Re: Fully Android compatible HttpClient 4.1.x; was Re: HttpClient-Cache release that works against HttpClient 4.0?

[candrews]

That idea sounds awesome to me - I may even do it. How would you like such a
thing implemented? As a script? A diff against "regular" httpclient?

Interestingly, for the case of HttpClient-Cache, my original problem would
not be solved by this approach. HttpClient-Cache uses EntityUtils.consume
statically, and there's no way to fix that. Perhaps a simple change to not
use EntityUtils.consume and thus conform to the HttpClient 4.0 API would be
alright? Also, I confirmed with that change, HttpClient-Cache works in
Android (the jar, that is) and the compiled jar only uses HttpClient 4.0
APIs. The unit tests, however, use methods from beyond the 4.0 API and thus
do not run against HttpClient 4.0.


olegk wrote:
> 
> On Thu, Nov 10, 2011 at 11:49:09AM -0800, candrews wrote:
>> 
>> I thought of that, but it won't work when an Android app has dependencies
>> that use HttpClient.
>> 
>> I use Spring Android Rest Template. My goal is to use Rest Template with
>> caching via HttpClient-Cache. If I shade HttpClient (and
>> HttpClient-Cache),
>> and include the shaded jar in my project, then Rest Template will not use
>> it
>> (as it's compiled to use the "real" HttpClient, not my private copy in a
>> new
>> namespace). So now I have to process Rest Template using Maven Shade. I'd
>> also like osmdroid to use the same HttpClient, so I have to shade it
>> too...
>> Now I'm using at least 4 custom created jar's outside of my Android app's
>> ant build system, causing a serious maintenance problem :-)
>> 
>> Is there an ant analogue to Maven Shade? Is there a different approach to
>> solving these problems?
>> 
> 
> There is a way to build a reasonably up-to-date version of HttpClient
> fully compatible with Android: shade impl classes and new interfaces only
> and make them compileable against 4.0 API. That would involve a bit more
> work but result in a library fully compatible with Android and 3rd party
> libraries at the same time.
> 
> If someone were to contribute such a port to the project, we probably
> could even ship it as an official ASF release.
> 
> Oleg
> 
> 
> 
>> (I apologize if I'm hijacking this list inappropriately - IMHO this seems
>> relevant to httpclient-user, and I'm not sure where else to ask. I'll
>> happily shut up if the list maintainer believes this thread is
>> off-topic).
>> 
>> Thanks again!
>> ~Craig
>> 
>> 
>> sebb-2-2 wrote:
>> > 
>> > On 9 November 2011 04:05, candrews  wrote:
>> >>
>> >> Maven Shade isn't compatible with maven-android-plugin:
>> >> https://code.google.com/p/maven-android-plugin/issues/detail?id=170
>> >> Perhaps
>> >> when that is resolved, Shade will be a solution for Maven users. But
>> what
>> >> about Ant users (which I think are what most of the Android apps are
>> >> built
>> >> with)?
>> > 
>> > It would also be possible to use the Maven Shade plugin to create a
>> > version of the HC jars with a different package name.
>> > This would be done independently of any Android build, whether using
>> > Maven or Ant.
>> > 
>> > You then build the Android stuff against the shaded jar, using the new
>> > package names.
>> > 
>> > This is not an ideal long-term solution, but should be sufficient for
>> > prototyping purposes.
>> > 
>> > -
>> > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>> > For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>> > 
>> > 
>> > 
>> 
>> -- 
>> View this message in context:
>> http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClient-4.0--tp32775131p32821369.html
>> Sent from the HttpClient-User mailing list archive at Nabble.com.
>> 
>> 
>> -
>> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>> 
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 
> 
> 

-- 
View this message in context: 
http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClient-4.0--tp32775131p32826896.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Fully Android compatible HttpClient 4.1.x; was Re: HttpClient-Cache release that works against HttpClient 4.0?

[Oleg Kalnichevski]

On Thu, Nov 10, 2011 at 11:49:09AM -0800, candrews wrote:
> 
> I thought of that, but it won't work when an Android app has dependencies
> that use HttpClient.
> 
> I use Spring Android Rest Template. My goal is to use Rest Template with
> caching via HttpClient-Cache. If I shade HttpClient (and HttpClient-Cache),
> and include the shaded jar in my project, then Rest Template will not use it
> (as it's compiled to use the "real" HttpClient, not my private copy in a new
> namespace). So now I have to process Rest Template using Maven Shade. I'd
> also like osmdroid to use the same HttpClient, so I have to shade it too...
> Now I'm using at least 4 custom created jar's outside of my Android app's
> ant build system, causing a serious maintenance problem :-)
> 
> Is there an ant analogue to Maven Shade? Is there a different approach to
> solving these problems?
> 

There is a way to build a reasonably up-to-date version of HttpClient fully 
compatible with Android: shade impl classes and new interfaces only and make 
them compileable against 4.0 API. That would involve a bit more work but result 
in a library fully compatible with Android and 3rd party libraries at the same 
time.

If someone were to contribute such a port to the project, we probably could 
even ship it as an official ASF release.

Oleg



> (I apologize if I'm hijacking this list inappropriately - IMHO this seems
> relevant to httpclient-user, and I'm not sure where else to ask. I'll
> happily shut up if the list maintainer believes this thread is off-topic).
> 
> Thanks again!
> ~Craig
> 
> 
> sebb-2-2 wrote:
> > 
> > On 9 November 2011 04:05, candrews  wrote:
> >>
> >> Maven Shade isn't compatible with maven-android-plugin:
> >> https://code.google.com/p/maven-android-plugin/issues/detail?id=170
> >> Perhaps
> >> when that is resolved, Shade will be a solution for Maven users. But what
> >> about Ant users (which I think are what most of the Android apps are
> >> built
> >> with)?
> > 
> > It would also be possible to use the Maven Shade plugin to create a
> > version of the HC jars with a different package name.
> > This would be done independently of any Android build, whether using
> > Maven or Ant.
> > 
> > You then build the Android stuff against the shaded jar, using the new
> > package names.
> > 
> > This is not an ideal long-term solution, but should be sufficient for
> > prototyping purposes.
> > 
> > ---------
> > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> > For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> > 
> > 
> > 
> 
> -- 
> View this message in context: 
> http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClient-4.0--tp32775131p32821369.html
> Sent from the HttpClient-User mailing list archive at Nabble.com.
> 
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient-Cache release that works against HttpClient 4.0?

[candrews]

I thought of that, but it won't work when an Android app has dependencies
that use HttpClient.

I use Spring Android Rest Template. My goal is to use Rest Template with
caching via HttpClient-Cache. If I shade HttpClient (and HttpClient-Cache),
and include the shaded jar in my project, then Rest Template will not use it
(as it's compiled to use the "real" HttpClient, not my private copy in a new
namespace). So now I have to process Rest Template using Maven Shade. I'd
also like osmdroid to use the same HttpClient, so I have to shade it too...
Now I'm using at least 4 custom created jar's outside of my Android app's
ant build system, causing a serious maintenance problem :-)

Is there an ant analogue to Maven Shade? Is there a different approach to
solving these problems?

(I apologize if I'm hijacking this list inappropriately - IMHO this seems
relevant to httpclient-user, and I'm not sure where else to ask. I'll
happily shut up if the list maintainer believes this thread is off-topic).

Thanks again!
~Craig


sebb-2-2 wrote:
> 
> On 9 November 2011 04:05, candrews  wrote:
>>
>> Maven Shade isn't compatible with maven-android-plugin:
>> https://code.google.com/p/maven-android-plugin/issues/detail?id=170
>> Perhaps
>> when that is resolved, Shade will be a solution for Maven users. But what
>> about Ant users (which I think are what most of the Android apps are
>> built
>> with)?
> 
> It would also be possible to use the Maven Shade plugin to create a
> version of the HC jars with a different package name.
> This would be done independently of any Android build, whether using
> Maven or Ant.
> 
> You then build the Android stuff against the shaded jar, using the new
> package names.
> 
> This is not an ideal long-term solution, but should be sufficient for
> prototyping purposes.
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 
> 
> 

-- 
View this message in context: 
http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClient-4.0--tp32775131p32821369.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient-Cache release that works against HttpClient 4.0?

[sebb]

On 9 November 2011 04:05, candrews  wrote:
>
> Maven Shade isn't compatible with maven-android-plugin:
> https://code.google.com/p/maven-android-plugin/issues/detail?id=170 Perhaps
> when that is resolved, Shade will be a solution for Maven users. But what
> about Ant users (which I think are what most of the Android apps are built
> with)?

It would also be possible to use the Maven Shade plugin to create a
version of the HC jars with a different package name.
This would be done independently of any Android build, whether using
Maven or Ant.

You then build the Android stuff against the shaded jar, using the new
package names.

This is not an ideal long-term solution, but should be sufficient for
prototyping purposes.

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient-Cache release that works against HttpClient 4.0?

[Oleg Kalnichevski]

o the latest release of HttpClient nonetheless.
> >>>
> >>> I'm the
> >>> maintainer<http://android-developers.blogspot.com/2011/09/androids-http-clients.html>of
> >>> Android's HTTP clients. Android's non-adoption of new versions of
> >>> Apache
> >>> HTTP client is my decision, but it hasn't been an easy one.
> >>>
> >>> We made a mistake early on by exposing the full Apache HTTP client
> >>> API—including
> >>> impl<http://developer.android.com/reference/org/apache/http/impl/cookie/package-summary.html>packages
> >>> —in Android's API. This makes upgrading Apache HTTP client risky as I
> >>> have
> >>> to avoid breaking applications. It also adds a burden on application
> >>> developers who would need to test their applications against multiple
> >>> releases of Apache HTTP client to support multiple releases of Android.
> >>> The
> >>> rich API means there's many moving parts to test.
> >>>
> >>> If I could put the genie back in the bottle I would not have exposed
> >>> Apache
> >>> HTTP as a public API in Android. It's a great API, but our including it
> >>> the
> >>> platform locks everyone into an old build that we grabbed back in 2008.
> >>> It's much simpler for developers to pick a single release, test it, and
> >>> bundle it. For this reason I recommend that developers looking for new
> >>> Apache HTTP features follow Oleg's advice and repackage Apache HTTP in
> >>> their application.
> >>>
> >>> Or use HttpURLConnection, which I've spent a lot of time on lately.
> >>> There's
> >>> even an HTTP response cache in Ice Cream Sandwich.
> >>>
> >>> Thanks Oleg & Apache HTTP client contributors. It's powerful software
> >>> that
> >>> gets lots of exercise on Android.
> >>>
> >>> Thanks,
> >>> Jesse
> >>>
> >>>
> >>
> >> --
> >> View this message in context:
> >> http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClient-4.0--tp32775131p32805637.html
> >> Sent from the HttpClient-User mailing list archive at Nabble.com.
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> >> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> >>
> >>
> > 
> > -
> > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> > For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> > 
> > 
> > 
> 



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient-Cache release that works against HttpClient 4.0?

[candrews]

> releases of Apache HTTP client to support multiple releases of Android.
>>> The
>>> rich API means there's many moving parts to test.
>>>
>>> If I could put the genie back in the bottle I would not have exposed
>>> Apache
>>> HTTP as a public API in Android. It's a great API, but our including it
>>> the
>>> platform locks everyone into an old build that we grabbed back in 2008.
>>> It's much simpler for developers to pick a single release, test it, and
>>> bundle it. For this reason I recommend that developers looking for new
>>> Apache HTTP features follow Oleg's advice and repackage Apache HTTP in
>>> their application.
>>>
>>> Or use HttpURLConnection, which I've spent a lot of time on lately.
>>> There's
>>> even an HTTP response cache in Ice Cream Sandwich.
>>>
>>> Thanks Oleg & Apache HTTP client contributors. It's powerful software
>>> that
>>> gets lots of exercise on Android.
>>>
>>> Thanks,
>>> Jesse
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClient-4.0--tp32775131p32805637.html
>> Sent from the HttpClient-User mailing list archive at Nabble.com.
>>
>>
>> -
>> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>>
>>
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 
> 
> 

-- 
View this message in context: 
http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClient-4.0--tp32775131p32808593.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient-Cache release that works against HttpClient 4.0?

[sebb]

On 8 November 2011 20:14, candrews  wrote:
>
> There are a number of issues with Java's HttpUrlConnection that make it a
> less than ideal option compared to HttpClient. For example,
> HttpUrlConnection has no way to set default connection/socket timeouts - you
> have to call
> http://developer.android.com/reference/java/net/URLConnection.html#setConnectTimeout%28int%29
> and
> http://developer.android.com/reference/java/net/URLConnection.html#setReadTimeout%28int%29
> for every connection. This particular issue is a killer for me - and I only
> just started evaluating switching my apps to HttpUrlConnection.
>
> There is precedent for Android "breaking" apps with upgrades - for example,
> https://code.google.com/p/android/issues/detail?id=15022 broke quite a few
> apps and annoyed more than a couple developers.
>
> In any case, I don't want to hijack this thread into a request for Android
> to upgrade HttpClient - I really just want to use HttpClient (and
> HttpClient-Cache) on Android.
>
> Since the org.apache.httpclient namespace is (probably) permanently taken
> with an outdated implementation, what can be done to allow Android
> developers to use HttpClient?
>
> Could there be releases, concurrent with the "normal" releases, of the
> org.apache.httpclient.* projects renamed to an alternative namespace? Of
> course, that means that non-httpclient projects would be broken, as they
> wouldn't change to use that alternative namespace, so developers wouldn't
> simply be able to drop in dependencies originally developed for non-Android
> targets. And if they did, they'd be in for a surprise when the dependency
> doesn't work due to VerifyError's or simply encountering runtime bugs that
> they "know" were fixed as they included a newer HttpClient dependency, but
> it's not actually used.
>
> IMHO, it seems that the Android decision to never change their bundled
> httpclient torpedos httpclient on Android forever - at least until Apache
> releases HttpClient 5 that uses a new namespace. Are there any other ideas
> on what can be done to remedy this situation?

Have you considered the Maven Shade plugin?

AIUI, that allows one to rename packages within a jar.
So you could change HC to use a different package name and thus use HC
4.1 on Android without clashing with the native classes.

I've only used it to allow Clirr to be used across a package rename,
but I think it is quite flexible.

I think you can develop everything using standard package names, then
use Shade to rename whatever packages would clash with Android.

Certainly worth trying if you have access to an Android development system.

> Thanks,
> ~Craig
>
>
> Jesse Wilson-4 wrote:
>>
>> Apache HTTP folks,
>>
>>> I have no problem with making httpclient-cache compatible with Android
>>> but at the same time I think users we should be encouraging Android
>>> users to migrate to the latest release of HttpClient nonetheless.
>>
>> I'm the
>> maintainer<http://android-developers.blogspot.com/2011/09/androids-http-clients.html>of
>> Android's HTTP clients. Android's non-adoption of new versions of
>> Apache
>> HTTP client is my decision, but it hasn't been an easy one.
>>
>> We made a mistake early on by exposing the full Apache HTTP client
>> API—including
>> impl<http://developer.android.com/reference/org/apache/http/impl/cookie/package-summary.html>packages
>> —in Android's API. This makes upgrading Apache HTTP client risky as I have
>> to avoid breaking applications. It also adds a burden on application
>> developers who would need to test their applications against multiple
>> releases of Apache HTTP client to support multiple releases of Android.
>> The
>> rich API means there's many moving parts to test.
>>
>> If I could put the genie back in the bottle I would not have exposed
>> Apache
>> HTTP as a public API in Android. It's a great API, but our including it
>> the
>> platform locks everyone into an old build that we grabbed back in 2008.
>> It's much simpler for developers to pick a single release, test it, and
>> bundle it. For this reason I recommend that developers looking for new
>> Apache HTTP features follow Oleg's advice and repackage Apache HTTP in
>> their application.
>>
>> Or use HttpURLConnection, which I've spent a lot of time on lately.
>> There's
>> even an HTTP response cache in Ice Cream Sandwich.
>>
>> Thanks Oleg & Apache HTTP client contributors. It's powerfu

Re: HttpClient-Cache release that works against HttpClient 4.0?

[candrews]

There are a number of issues with Java's HttpUrlConnection that make it a
less than ideal option compared to HttpClient. For example,
HttpUrlConnection has no way to set default connection/socket timeouts - you
have to call
http://developer.android.com/reference/java/net/URLConnection.html#setConnectTimeout%28int%29
and
http://developer.android.com/reference/java/net/URLConnection.html#setReadTimeout%28int%29
for every connection. This particular issue is a killer for me - and I only
just started evaluating switching my apps to HttpUrlConnection.

There is precedent for Android "breaking" apps with upgrades - for example,
https://code.google.com/p/android/issues/detail?id=15022 broke quite a few
apps and annoyed more than a couple developers.

In any case, I don't want to hijack this thread into a request for Android
to upgrade HttpClient - I really just want to use HttpClient (and
HttpClient-Cache) on Android.

Since the org.apache.httpclient namespace is (probably) permanently taken
with an outdated implementation, what can be done to allow Android
developers to use HttpClient?

Could there be releases, concurrent with the "normal" releases, of the
org.apache.httpclient.* projects renamed to an alternative namespace? Of
course, that means that non-httpclient projects would be broken, as they
wouldn't change to use that alternative namespace, so developers wouldn't
simply be able to drop in dependencies originally developed for non-Android
targets. And if they did, they'd be in for a surprise when the dependency
doesn't work due to VerifyError's or simply encountering runtime bugs that
they "know" were fixed as they included a newer HttpClient dependency, but
it's not actually used.

IMHO, it seems that the Android decision to never change their bundled
httpclient torpedos httpclient on Android forever - at least until Apache
releases HttpClient 5 that uses a new namespace. Are there any other ideas
on what can be done to remedy this situation?

Thanks,
~Craig


Jesse Wilson-4 wrote:
> 
> Apache HTTP folks,
> 
>> I have no problem with making httpclient-cache compatible with Android
>> but at the same time I think users we should be encouraging Android
>> users to migrate to the latest release of HttpClient nonetheless.
> 
> I'm the
> maintainer<http://android-developers.blogspot.com/2011/09/androids-http-clients.html>of
> Android's HTTP clients. Android's non-adoption of new versions of
> Apache
> HTTP client is my decision, but it hasn't been an easy one.
> 
> We made a mistake early on by exposing the full Apache HTTP client
> API—including
> impl<http://developer.android.com/reference/org/apache/http/impl/cookie/package-summary.html>packages
> —in Android's API. This makes upgrading Apache HTTP client risky as I have
> to avoid breaking applications. It also adds a burden on application
> developers who would need to test their applications against multiple
> releases of Apache HTTP client to support multiple releases of Android.
> The
> rich API means there's many moving parts to test.
> 
> If I could put the genie back in the bottle I would not have exposed
> Apache
> HTTP as a public API in Android. It's a great API, but our including it
> the
> platform locks everyone into an old build that we grabbed back in 2008.
> It's much simpler for developers to pick a single release, test it, and
> bundle it. For this reason I recommend that developers looking for new
> Apache HTTP features follow Oleg's advice and repackage Apache HTTP in
> their application.
> 
> Or use HttpURLConnection, which I've spent a lot of time on lately.
> There's
> even an HTTP response cache in Ice Cream Sandwich.
> 
> Thanks Oleg & Apache HTTP client contributors. It's powerful software that
> gets lots of exercise on Android.
> 
> Thanks,
> Jesse
> 
> 

-- 
View this message in context: 
http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClient-4.0--tp32775131p32805637.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient-Cache release that works against HttpClient 4.0?

[Jesse Wilson]

Apache HTTP folks,

> I have no problem with making httpclient-cache compatible with Android
> but at the same time I think users we should be encouraging Android
> users to migrate to the latest release of HttpClient nonetheless.

I'm the 
maintainerof
Android's HTTP clients. Android's non-adoption of new versions of
Apache
HTTP client is my decision, but it hasn't been an easy one.

We made a mistake early on by exposing the full Apache HTTP client
API—including
implpackages
—in Android's API. This makes upgrading Apache HTTP client risky as I have
to avoid breaking applications. It also adds a burden on application
developers who would need to test their applications against multiple
releases of Apache HTTP client to support multiple releases of Android. The
rich API means there's many moving parts to test.

If I could put the genie back in the bottle I would not have exposed Apache
HTTP as a public API in Android. It's a great API, but our including it the
platform locks everyone into an old build that we grabbed back in 2008.
It's much simpler for developers to pick a single release, test it, and
bundle it. For this reason I recommend that developers looking for new
Apache HTTP features follow Oleg's advice and repackage Apache HTTP in
their application.

Or use HttpURLConnection, which I've spent a lot of time on lately. There's
even an HTTP response cache in Ice Cream Sandwich.

Thanks Oleg & Apache HTTP client contributors. It's powerful software that
gets lots of exercise on Android.

Thanks,
Jesse

Re: HttpClient-Cache release that works against HttpClient 4.0?

[Oleg Kalnichevski]

On Mon, 2011-11-07 at 14:25 +, Moore, Jonathan (CIM) wrote:
> Is that really the only change that would be necessary to work with 4.0? I
> think it might be worth double-checking that all the tests pass with this
> change and when compiled against 4.0.
> 
> While I agree that it would be better for Google to be upgrading the
> version of HttpClient they're using, at the same time, there are a lot of
> Android developers and users that would benefit from having the caching
> module. I'm willing to spend a little time to investigate further and
> perhaps make a change like this if it's not going to compromise the
> codebase going forwards.
> 
> At the same time, Craig: couldn't you just apply that diff yourself and
> run with a patched version of httpclient-cache in your apps, then wait
> until Google upgrades the HttpClient that's embedded in Android?
> 

Jon

The trouble is that Google made it quite clear they were not going to
upgrade HttpClient in Android. To make matters worse they will continue
shipping their outdated pre-BETA version with Android for releases to
come. 

I have no problem with making httpclient-cache compatible with Android
but at the same time I think users we should be encouraging Android
users to migrate to the latest release of HttpClient nonetheless. 

Oleg




-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient-Cache release that works against HttpClient 4.0?

[Moore, Jonathan (CIM)]

Is that really the only change that would be necessary to work with 4.0? I
think it might be worth double-checking that all the tests pass with this
change and when compiled against 4.0.

While I agree that it would be better for Google to be upgrading the
version of HttpClient they're using, at the same time, there are a lot of
Android developers and users that would benefit from having the caching
module. I'm willing to spend a little time to investigate further and
perhaps make a change like this if it's not going to compromise the
codebase going forwards.

At the same time, Craig: couldn't you just apply that diff yourself and
run with a patched version of httpclient-cache in your apps, then wait
until Google upgrades the HttpClient that's embedded in Android?

Jon

Jon Moore
Comcast Interactive Media






On 11/3/11 2:32 PM, "candrews"  wrote:

>
>HttpClient-Cache doesn't work against HttpClient 4.0 because it uses
>org.apache.http.util.EntityUtils.consume(org.apache.http.HttpEntity) which
>wasn't introduced until HttpClient 4.1.
>
>Since Android uses HttpClient 4.0 (yes, it sucks that they won't upgrade,
>and I don't understand their reasons for not upgrading, but anyways), this
>means you can't use HttpClient-Cache on Android - and IMHO, it would be
>really useful.
>
>Could HttpClient-Cache simply not use
>org.apache.http.util.EntityUtils.consume and therefore set the minimum
>HttpClient version to 4.0? The change seems quite simple - in
>ResponseProtocolCompliance, use this method body instead of the existing
>one:
>
>private void consumeBody(HttpResponse response) throws IOException {
>HttpEntity body = response.getEntity();
>if (body != null){
> if (body.isStreaming()) {
> InputStream instream = body.getContent();
> if (instream != null) {
> instream.close();
> }
> }
>}
>
>Thank you,
>~Craig
>-- 
>View this message in context:
>http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClie
>nt-4.0--tp32775131p32775131.html
>Sent from the HttpClient-User mailing list archive at Nabble.com.
>
>
>-
>To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient-Cache release that works against HttpClient 4.0?

[Oleg Kalnichevski]

On Thu, 2011-11-03 at 11:32 -0700, candrews wrote:
> HttpClient-Cache doesn't work against HttpClient 4.0 because it uses
> org.apache.http.util.EntityUtils.consume(org.apache.http.HttpEntity) which
> wasn't introduced until HttpClient 4.1.
> 
> Since Android uses HttpClient 4.0 (yes, it sucks that they won't upgrade,
> and I don't understand their reasons for not upgrading, but anyways), this
> means you can't use HttpClient-Cache on Android - and IMHO, it would be
> really useful.
> 
> Could HttpClient-Cache simply not use
> org.apache.http.util.EntityUtils.consume and therefore set the minimum
> HttpClient version to 4.0? The change seems quite simple - in
> ResponseProtocolCompliance, use this method body instead of the existing
> one:
> 
> private void consumeBody(HttpResponse response) throws IOException {
> HttpEntity body = response.getEntity();
> if (body != null){
>  if (body.isStreaming()) {
>  InputStream instream = body.getContent();
>  if (instream != null) {
>  instream.close();
>  }
>  }
> }
> 
> Thank you,
> ~Craig

Crag,

The problem is that what Google ships with Android platform is an
extremely outdated, pre-BETA version of HttpClient. There have been
literally hundreds of changes and bug fixes since that version that
Google never bothered to backport. I personally see no point in
maintaining compatibility with Adnroid's version of HttpClient.  

You will be much better off using the latest version of HttpClient
repackaged to a different name space such as
'screw.you.google.org.apache.http'. If you are a Maven user you could
use Maven Shade plugin to create a package of HttpClient with a custom
package namespace.  

Hope this helps

Oleg


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

HttpClient-Cache release that works against HttpClient 4.0?

[candrews]

HttpClient-Cache doesn't work against HttpClient 4.0 because it uses
org.apache.http.util.EntityUtils.consume(org.apache.http.HttpEntity) which
wasn't introduced until HttpClient 4.1.

Since Android uses HttpClient 4.0 (yes, it sucks that they won't upgrade,
and I don't understand their reasons for not upgrading, but anyways), this
means you can't use HttpClient-Cache on Android - and IMHO, it would be
really useful.

Could HttpClient-Cache simply not use
org.apache.http.util.EntityUtils.consume and therefore set the minimum
HttpClient version to 4.0? The change seems quite simple - in
ResponseProtocolCompliance, use this method body instead of the existing
one:

private void consumeBody(HttpResponse response) throws IOException {
HttpEntity body = response.getEntity();
if (body != null){
 if (body.isStreaming()) {
 InputStream instream = body.getContent();
 if (instream != null) {
 instream.close();
 }
 }
}

Thank you,
~Craig
-- 
View this message in context: 
http://old.nabble.com/HttpClient-Cache-release-that-works-against-HttpClient-4.0--tp32775131p32775131.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: How to release the connection in HttpClient 4.0

[Oleg Kalnichevski]

On Sun, 2010-09-19 at 11:47 +0530, arpita k wrote:
> Hi,
> 
> I am using HttpClient 4.0 for client and server communication.
> 
> Here is my code to open a connection:
> 
> HttpParams params = *new* BasicHttpParams();
> 
> // Increase max total connection to 200
> 
> ConnManagerParams.*setMaxTotalConnections*(params, 5);
> 
> // Increase default max connection per route to 20
> 
> ConnPerRouteBean connPerRoute = *new* ConnPerRouteBean(5);
> 
> ConnManagerParams.*setMaxConnectionsPerRoute*(params, connPerRoute);
> 
> Scheme httpsScheme = *new* Scheme("https", socketFactory, 443);
> 
> SchemeRegistry schemeRegistry = *new* SchemeRegistry();
> 
> schemeRegistry.register(httpsScheme);
> 
> ClientConnectionManager cm = *new* ThreadSafeClientConnManager(params,
> schemeRegistry);
> 
> signClient = *new* DefaultHttpClient(cm, params);
> 
> But I am using around 5 connection in the connection pool.but after using
> one connection it is not returning back the connection in the connection
> pool.It was kept alive.I like  to know how to release the connection so
> that  same connection could be used further.
> 
> Your help will be appreciated.
> 

http://hc.apache.org/httpcomponents-client-ga/tutorial/html/fundamentals.html#d4e143

Oleg



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

How to release the connection in HttpClient 4.0

[arpita k]

Hi,

I am using HttpClient 4.0 for client and server communication.

Here is my code to open a connection:

HttpParams params = *new* BasicHttpParams();

// Increase max total connection to 200

ConnManagerParams.*setMaxTotalConnections*(params, 5);

// Increase default max connection per route to 20

ConnPerRouteBean connPerRoute = *new* ConnPerRouteBean(5);

ConnManagerParams.*setMaxConnectionsPerRoute*(params, connPerRoute);

Scheme httpsScheme = *new* Scheme("https", socketFactory, 443);

SchemeRegistry schemeRegistry = *new* SchemeRegistry();

schemeRegistry.register(httpsScheme);

ClientConnectionManager cm = *new* ThreadSafeClientConnManager(params,
schemeRegistry);

signClient = *new* DefaultHttpClient(cm, params);

But I am using around 5 connection in the connection pool.but after using
one connection it is not returning back the connection in the connection
pool.It was kept alive.I like  to know how to release the connection so
that  same connection could be used further.

Your help will be appreciated.

Thanks,

Arpita

Re: How to upgrade httpclient 3.0 to httpclient 4.0?

[arpita k]

Hi Oleg,

Thanks for your reply.
But I am using the the *CRL check* feature of the not-yet-common-ssl library
and that's why I am using the *AuthSSLProtocolSocketFactory *factory.Is the
class *AuthSSLProtocolSocketFactory* compatible with* *the *Httpclient
4.0*library
*?*

~Arpita

On Thu, Jul 22, 2010 at 1:01 AM, Oleg Kalnichevski  wrote:

> On Wed, 2010-07-21 at 21:34 +0530, arpita k wrote:
> > Hi,
> >
> > I am using common-httpclient 3.0 and common-ssl 0.3.11 library to
> establish
> > SSL communication between client and the server.Here is the code:
> >
> >
> > *AuthSSLProtocolSocketFactory myAuthFactory = null;
> > try
> > {
> > myAuthFactory = new AuthSSLProtocolSocketFactory(new
> URL("file:"
> > + KEYSTORE_LOCATION), KEYSTORE_PASSWORD, new
> URL("file:"
> > + TRUSTSTORE_LOCATION), TRUSTSTORE_PASSWORD);
> >
> >
> > Protocol myhttps = new Protocol("https", myAuthFactory,
> > serverPort);
> > Protocol.registerProtocol("https", myhttps);
> > signClient.getHostConfiguration().setHost(serverName,
> > serverPort,
> > myhttps);
> >
> signClient.getParams().setCookiePolicy(CookiePolicy.RFC_2109);
> >
> > *
> > The AuthSSLProtocolSocketFactory  is part of common-ssl library.
> > Now I need to upgrade the httpclient from 3.0 to 4.0.But I want to use
> the
> > AuthSSLProtocolSocketFactory  factory as it is doin gCRL check
> > automatically.
> >
> > I know that the schem is replaced by the Protocol of httpclient 4.0.But I
> > don't think it could take AuthSSLProtocolSocketFactory   as a
> parameter.So,
> > which way I should proceed?
> >
>
> You should create an instance of SSLContext using not-yet-commons SSL
> and pass it to the constructor of the SSLSocketFactory as described
> here:
>
>
> http://hc.apache.org/httpcomponents-client-4.0.1/tutorial/html/connmgmt.html#d4e497
>
> Oleg
>
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>

Re: How to upgrade httpclient 3.0 to httpclient 4.0?

[Oleg Kalnichevski]

On Wed, 2010-07-21 at 21:34 +0530, arpita k wrote:
> Hi,
> 
> I am using common-httpclient 3.0 and common-ssl 0.3.11 library to establish
> SSL communication between client and the server.Here is the code:
> 
> 
> *AuthSSLProtocolSocketFactory myAuthFactory = null;
> try
> {
> myAuthFactory = new AuthSSLProtocolSocketFactory(new URL("file:"
> + KEYSTORE_LOCATION), KEYSTORE_PASSWORD, new URL("file:"
> + TRUSTSTORE_LOCATION), TRUSTSTORE_PASSWORD);
> 
> 
> Protocol myhttps = new Protocol("https", myAuthFactory,
> serverPort);
> Protocol.registerProtocol("https", myhttps);
> signClient.getHostConfiguration().setHost(serverName,
> serverPort,
> myhttps);
> signClient.getParams().setCookiePolicy(CookiePolicy.RFC_2109);
> 
> *
> The AuthSSLProtocolSocketFactory  is part of common-ssl library.
> Now I need to upgrade the httpclient from 3.0 to 4.0.But I want to use the
> AuthSSLProtocolSocketFactory  factory as it is doin gCRL check
> automatically.
> 
> I know that the schem is replaced by the Protocol of httpclient 4.0.But I
> don't think it could take AuthSSLProtocolSocketFactory   as a parameter.So,
> which way I should proceed?
> 

You should create an instance of SSLContext using not-yet-commons SSL
and pass it to the constructor of the SSLSocketFactory as described
here:

http://hc.apache.org/httpcomponents-client-4.0.1/tutorial/html/connmgmt.html#d4e497
  

Oleg


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: How to upgrade httpclient 3.0 to httpclient 4.0?

[Ryan Smith]

Id like to add on, is there an overall guide to upgrade from 3.x to 4.x?

Thanks,
-Ryan

On Wed, Jul 21, 2010 at 12:04 PM, arpita k  wrote:

> Hi,
>
> I am using common-httpclient 3.0 and common-ssl 0.3.11 library to establish
> SSL communication between client and the server.Here is the code:
>
>
> *AuthSSLProtocolSocketFactory myAuthFactory = null;
>try
>{
>myAuthFactory = new AuthSSLProtocolSocketFactory(new URL("file:"
>+ KEYSTORE_LOCATION), KEYSTORE_PASSWORD, new URL("file:"
>+ TRUSTSTORE_LOCATION), TRUSTSTORE_PASSWORD);
>
>
>Protocol myhttps = new Protocol("https", myAuthFactory,
> serverPort);
>Protocol.registerProtocol("https", myhttps);
>signClient.getHostConfiguration().setHost(serverName,
> serverPort,
>myhttps);
>signClient.getParams().setCookiePolicy(CookiePolicy.RFC_2109);
>
> *
> The AuthSSLProtocolSocketFactory  is part of common-ssl library.
> Now I need to upgrade the httpclient from 3.0 to 4.0.But I want to use the
> AuthSSLProtocolSocketFactory  factory as it is doin gCRL check
> automatically.
>
> I know that the schem is replaced by the Protocol of httpclient 4.0.But I
> don't think it could take AuthSSLProtocolSocketFactory   as a parameter.So,
> which way I should proceed?
>
> Please help me to upgrade the httpclient.
>
> Thanks in advance!!
>
> Arpita
>

How to upgrade httpclient 3.0 to httpclient 4.0?

[arpita k]

Hi,

I am using common-httpclient 3.0 and common-ssl 0.3.11 library to establish
SSL communication between client and the server.Here is the code:


*AuthSSLProtocolSocketFactory myAuthFactory = null;
try
{
myAuthFactory = new AuthSSLProtocolSocketFactory(new URL("file:"
+ KEYSTORE_LOCATION), KEYSTORE_PASSWORD, new URL("file:"
+ TRUSTSTORE_LOCATION), TRUSTSTORE_PASSWORD);


Protocol myhttps = new Protocol("https", myAuthFactory,
serverPort);
Protocol.registerProtocol("https", myhttps);
signClient.getHostConfiguration().setHost(serverName,
serverPort,
myhttps);
signClient.getParams().setCookiePolicy(CookiePolicy.RFC_2109);

*
The AuthSSLProtocolSocketFactory  is part of common-ssl library.
Now I need to upgrade the httpclient from 3.0 to 4.0.But I want to use the
AuthSSLProtocolSocketFactory  factory as it is doin gCRL check
automatically.

I know that the schem is replaced by the Protocol of httpclient 4.0.But I
don't think it could take AuthSSLProtocolSocketFactory   as a parameter.So,
which way I should proceed?

Please help me to upgrade the httpclient.

Thanks in advance!!

Arpita

Re: How to initiate CRL check or OCSP check with httpclient 4.0

[Oleg Kalnichevski]

On Thu, 2010-07-01 at 23:43 +0530, arpita k wrote:
> Hi,
> 
> I am using Httplcient 4.0 to communicate between client and the server. The
> client is written in Java and the server is IIS.SO, I am doing mutual
> authentication to establish the SSL connection between these two. I like to
> initiate the CRL check for the server certificate. I am using normal
> Httpclient 4.0 and httpcore  4.0 .So, I believe it is not initiating any CRL
> checking by its own.
> 
> Please let me know how to initiate the CRL check or OCSP check when I am
> using httpclient 4.0??
> 
> 

HttpClient relies on the standard JSSE mechanisms for SSL transport
encryption. Please refer to the JSSE documentation for instructions on
activating CRL / OCSP checks.

Oleg


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

How to initiate CRL check or OCSP check with httpclient 4.0

[arpita k]

Hi,

I am using Httplcient 4.0 to communicate between client and the server. The
client is written in Java and the server is IIS.SO, I am doing mutual
authentication to establish the SSL connection between these two. I like to
initiate the CRL check for the server certificate. I am using normal
Httpclient 4.0 and httpcore  4.0 .So, I believe it is not initiating any CRL
checking by its own.

Please let me know how to initiate the CRL check or OCSP check when I am
using httpclient 4.0??





Thanks,

Arpita Kundu

Problem Post page using HTTPClient 4.0

[piHuv]

Hi,

I get the 400 Bad Request  when I do a Post to a website that has the
"hostname/script?querystring" path in the form action field.  Below is the
link to the website that I try to do a post, a partial html code and the
Java code for the form post.  Hope you can help me with this problem. 
Thanks in advance.

http://www.research.gov/rgov/anonymous.portal?_nfpb=true&_pageLabel=page_research_funding_search&_nfls=false

http://www.research.gov:80/rgov/anonymous.portal?_nfpb=true&_windowLabel=awardInfo_1_4&awardInfo_1_4_actionOverride=%2Fgov%2Fresearch%2Fservices%2FawardInfo%2FbasicSearchCommand&_pageLabel=page_research_funding_search";
method="post">


url =
"http://www.research.gov/rgov/anonymous.portal?_nfpb=true&_windowLabel=awardInfo_1_4&awardInfo_1_4_actionOverride=%2Fgov%2Fresearch%2Fservices%2FawardInfo%2FbasicSearchCommand&_pageLabel=page_research_funding_search";;
data =
"awardInfo_1_4wlw-checkbox_key%3A%7BactionForm.recoveryAct%7DOldValue=false&awardInfo_1_4%7BactionForm.awardDateEndStr%7D=03%2F04%2F2009&awardInfo_1_4%7BactionForm.awardDateStartStr%7D=03%2F02%2F2009&awardInfo_1_4%7BactionForm.awardeeProjectSearch%7D=&awardInfo_1_4%7BactionForm.maxAwardAmountStr%7D=&awardInfo_1_4%7BactionForm.minAwardAmountStr%7D=&awardInfo_1_4%7BactionForm.personnelSearch%7D=";
ret = null;
response = null;
StringEntity tmp = null;

httpPost = new HttpPost(url);

httpPost.setHeader("User-Agent", "Mozilla/5.0 (X11; U; Linux " +
"i686; en-US; rv:1.8.1.6) Gecko/20061201 Firefox/2.0.0.6 (Ubuntu-feisty)");
httpPost.setHeader("Accept", "text/html,application/xml," +
"application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5");
httpPost.setHeader("Content-Type",
"application/x-www-form-urlencoded");

List nameValues = createNameValues(data);
tmp = new UrlEncodedFormEntity(nameValues, "UTF-8");
tmp.setChunked(true);
httpPost.setEntity(tmp);
        response = httpClient.execute(httpPost);
ret = response.getStatusLine().toString();






-- 
View this message in context: 
http://old.nabble.com/Problem-Post-page-using-HTTPClient-4.0-tp28032942p28032942.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient 4.0 encoding madness

[sebb]

On 30/01/2010, amoldavsky  wrote:
>
>  Hi,
>
>  This solution worked out very well:

Glad you finally got there.

>  byte[] tmp = new byte[bufferSize];
>
> int bytesRead;
> try
> {
>   if((bytesRead = chunkedIns.read(tmp)) != -1)
>
>   {
> return new String(tmp, 0, bytesRead);
>   }
>
>   else
>   {
> finish();
> return null;
>   }
> }
> catch(IOException e)
> {
>   HTTPClientException e2 = new HTTPClientException(e.getMessage());
>   e2.setStackTrace(e.getStackTrace());
>   throw e2;
> }
>
>
>
> If it's not too much of a trouble would anybody please explain to me why is
>  it possible that the buffer maybe not be 100% full when I read it? I think
>  it's all depends on how the implementation was done (in this case by Sun),
>  and if Sun decided to implement buffering this way I don't understand the
>  logic behind it.

In that case you had better ask the Oracle.

>
>  Thank you very much Oleg, Ken and Seb-2-2 for your earlier inputs!
>
>
>
>
>  sebb-2-2 wrote:
>  >
>  > On 29/01/2010, amoldavsky  wrote:
>  >>
>  >>  Hi Oleg,
>  >>
>  >>  Let me rephrase the question in better terms:
>  >>  If the server document is Y and buffer size is X, let's even assume that
>  >> Y =
>  >>  kX where X < Y, is it possible that any buffer 0 < x < (k-1) will not be
>  >>  fully filled?
>  >
>  > Remember that HTTP packets may be broken up in transit.
>  >
>  > However, even without that, it's never safe to assume that a buffer is
>  > filled.
>  >
>  > That's what the return value from read(buffer) is for - it tells you
>  > how many bytes are available.
>  >
>  >>  Thanks!
>  >>  -Assaf
>  >>
>  >>
>  >>
>  >>  Ken Krugler wrote:
>  >>  >
>  >>  >
>  >>  > On Jan 28, 2010, at 10:09pm, amoldavsky wrote:
>  >>  >
>  >>  >>
>  >>  >> Hi Oleg,
>  >>  >> Thank you for the quick reply.
>  >>  >>
>  >>  >> So if there is a possibility that not the whole buffer is filled how
>  >>  >> can I
>  >>  >> insure or force HttpClient to fill the whole buffer? Should I maybe
>  >>  >> avoid
>  >>  >> Stream Readers all together?
>  >>  >
>  >>  > If bufferSize is X, and the server document you're fetching has Y
>  >>  > bytes, then what do you mean by "force HttpClient to fill the whole
>  >>  > buffer"?
>  >>  >
>  >>  > At a minimum, you'd want
>  >>  >
>  >>  > int bytesRead = chunkedIns.read(tmp);
>  >>  > if (bytesRead != -1) {
>  >>  > return new String(tmp, 0, bytesRead);
>  >>  > }
>  >>  >
>  >>  > But that also uses the platform default encoding for the character
>  >>  > set, which often won't be correct.
>  >>  >
>  >>  > -- Ken
>  >>  >
>  >>  >>
>  >>  >> olegk wrote:
>  >>  >>>
>  >>  >>> On Wed, 2010-01-27 at 20:24 -0800, amoldavsky wrote:
>  >>  >>>> Hi
>  >>  >>>>
>  >>  >>>> I have coded a simple file downloader using HttpClient 4.0.
>  >>  >>>> It works fine but there is something wrong with the String
>  >>  >>>> encoding or
>  >>  >>>> the
>  >>  >>>> buffer stream. The problem is that there are long sequences of
>  >>  >>>> "NULL"
>  >>  >>>> (ANSI
>  >>  >>>> code 00) through out the final file, like this:
>  >>  >>>> http://old.nabble.com/file/p27350930/httpclient_error01.jpg
>  >>  >>>> http://old.nabble.com/file/p27350930/httpclient_error02.jpg
>  >>  >>>>
>  >>  >>>> Here is the main code:
>  >>  >>>>
>  >>  >>>> public String getChunk(String url, int bufferSize) throws
>  >>  >>>> HTTPClientException
>  >>  >>>>  {
>  >>  >>>>if(!chunkedStarted)
>  >>  >>>>{
>  >>  >>>>  chunkedIns = getInputStream(url);
>  >>  >>>>  chunkedStarted = true;
>  >>  >>>>}
>  >>  >>

Re: HttpClient 4.0 encoding madness

[amoldavsky]

Hi,

This solution worked out very well:
 byte[] tmp = new byte[bufferSize];
int bytesRead;
try
{
  if((bytesRead = chunkedIns.read(tmp)) != -1)
  {
return new String(tmp, 0, bytesRead);
  }
  else
  {
finish();
return null;
  }
}
catch(IOException e)
{
  HTTPClientException e2 = new HTTPClientException(e.getMessage());
  e2.setStackTrace(e.getStackTrace());
  throw e2;
}


If it's not too much of a trouble would anybody please explain to me why is
it possible that the buffer maybe not be 100% full when I read it? I think
it's all depends on how the implementation was done (in this case by Sun),
and if Sun decided to implement buffering this way I don't understand the
logic behind it.


Thank you very much Oleg, Ken and Seb-2-2 for your earlier inputs!



sebb-2-2 wrote:
> 
> On 29/01/2010, amoldavsky  wrote:
>>
>>  Hi Oleg,
>>
>>  Let me rephrase the question in better terms:
>>  If the server document is Y and buffer size is X, let's even assume that
>> Y =
>>  kX where X < Y, is it possible that any buffer 0 < x < (k-1) will not be
>>  fully filled?
> 
> Remember that HTTP packets may be broken up in transit.
> 
> However, even without that, it's never safe to assume that a buffer is
> filled.
> 
> That's what the return value from read(buffer) is for - it tells you
> how many bytes are available.
> 
>>  Thanks!
>>  -Assaf
>>
>>
>>
>>  Ken Krugler wrote:
>>  >
>>  >
>>  > On Jan 28, 2010, at 10:09pm, amoldavsky wrote:
>>  >
>>  >>
>>  >> Hi Oleg,
>>  >> Thank you for the quick reply.
>>  >>
>>  >> So if there is a possibility that not the whole buffer is filled how
>>  >> can I
>>  >> insure or force HttpClient to fill the whole buffer? Should I maybe
>>  >> avoid
>>  >> Stream Readers all together?
>>  >
>>  > If bufferSize is X, and the server document you're fetching has Y
>>  > bytes, then what do you mean by "force HttpClient to fill the whole
>>  > buffer"?
>>  >
>>  > At a minimum, you'd want
>>  >
>>  > int bytesRead = chunkedIns.read(tmp);
>>  > if (bytesRead != -1) {
>>  > return new String(tmp, 0, bytesRead);
>>  > }
>>  >
>>  > But that also uses the platform default encoding for the character
>>  > set, which often won't be correct.
>>  >
>>  > -- Ken
>>  >
>>  >>
>>  >> olegk wrote:
>>  >>>
>>  >>> On Wed, 2010-01-27 at 20:24 -0800, amoldavsky wrote:
>>  >>>> Hi
>>  >>>>
>>  >>>> I have coded a simple file downloader using HttpClient 4.0.
>>  >>>> It works fine but there is something wrong with the String
>>  >>>> encoding or
>>  >>>> the
>>  >>>> buffer stream. The problem is that there are long sequences of
>>  >>>> "NULL"
>>  >>>> (ANSI
>>  >>>> code 00) through out the final file, like this:
>>  >>>> http://old.nabble.com/file/p27350930/httpclient_error01.jpg
>>  >>>> http://old.nabble.com/file/p27350930/httpclient_error02.jpg
>>  >>>>
>>  >>>> Here is the main code:
>>  >>>>
>>  >>>> public String getChunk(String url, int bufferSize) throws
>>  >>>> HTTPClientException
>>  >>>>  {
>>  >>>>if(!chunkedStarted)
>>  >>>>{
>>  >>>>  chunkedIns = getInputStream(url);
>>  >>>>  chunkedStarted = true;
>>  >>>>}
>>  >>>>
>>  >>>>byte[] tmp = new byte[bufferSize];
>>  >>>>try
>>  >>>>{
>>  >>>>  if(chunkedIns.read(tmp) != -1)
>>  >>>>  {
>>  >>>
>>  >>> What makes you think that the entire buffer will be filled with
>> data?
>>  >>>
>>  >>> Oleg
>>  >>>
>>  >>>
>>  >>>>return new String(tmp);
>>  >>>>  }
>>  >>>>  else
>>  >>>>  {
>>  >>>>finish();
>>  >>>>return null;
>>  >>>>  }
>>  >>>>}
>>  >>>>catch(IOExce

Re: HttpClient 4.0 encoding madness

[sebb]

On 29/01/2010, amoldavsky  wrote:
>
>  Hi Oleg,
>
>  Let me rephrase the question in better terms:
>  If the server document is Y and buffer size is X, let's even assume that Y =
>  kX where X < Y, is it possible that any buffer 0 < x < (k-1) will not be
>  fully filled?

Remember that HTTP packets may be broken up in transit.

However, even without that, it's never safe to assume that a buffer is filled.

That's what the return value from read(buffer) is for - it tells you
how many bytes are available.

>  Thanks!
>  -Assaf
>
>
>
>  Ken Krugler wrote:
>  >
>  >
>  > On Jan 28, 2010, at 10:09pm, amoldavsky wrote:
>  >
>  >>
>  >> Hi Oleg,
>  >> Thank you for the quick reply.
>  >>
>  >> So if there is a possibility that not the whole buffer is filled how
>  >> can I
>  >> insure or force HttpClient to fill the whole buffer? Should I maybe
>  >> avoid
>  >> Stream Readers all together?
>  >
>  > If bufferSize is X, and the server document you're fetching has Y
>  > bytes, then what do you mean by "force HttpClient to fill the whole
>  > buffer"?
>  >
>  > At a minimum, you'd want
>  >
>  > int bytesRead = chunkedIns.read(tmp);
>  > if (bytesRead != -1) {
>  > return new String(tmp, 0, bytesRead);
>  > }
>  >
>  > But that also uses the platform default encoding for the character
>  > set, which often won't be correct.
>  >
>  > -- Ken
>  >
>  >>
>  >> olegk wrote:
>  >>>
>  >>> On Wed, 2010-01-27 at 20:24 -0800, amoldavsky wrote:
>  >>>> Hi
>  >>>>
>  >>>> I have coded a simple file downloader using HttpClient 4.0.
>  >>>> It works fine but there is something wrong with the String
>  >>>> encoding or
>  >>>> the
>  >>>> buffer stream. The problem is that there are long sequences of
>  >>>> "NULL"
>  >>>> (ANSI
>  >>>> code 00) through out the final file, like this:
>  >>>> http://old.nabble.com/file/p27350930/httpclient_error01.jpg
>  >>>> http://old.nabble.com/file/p27350930/httpclient_error02.jpg
>  >>>>
>  >>>> Here is the main code:
>  >>>>
>  >>>> public String getChunk(String url, int bufferSize) throws
>  >>>> HTTPClientException
>  >>>>  {
>  >>>>if(!chunkedStarted)
>  >>>>{
>  >>>>  chunkedIns = getInputStream(url);
>  >>>>  chunkedStarted = true;
>  >>>>}
>  >>>>
>  >>>>byte[] tmp = new byte[bufferSize];
>  >>>>try
>  >>>>{
>  >>>>  if(chunkedIns.read(tmp) != -1)
>  >>>>  {
>  >>>
>  >>> What makes you think that the entire buffer will be filled with data?
>  >>>
>  >>> Oleg
>  >>>
>  >>>
>  >>>>return new String(tmp);
>  >>>>  }
>  >>>>  else
>  >>>>  {
>  >>>>finish();
>  >>>>return null;
>  >>>>  }
>  >>>>}
>  >>>>catch(IOException e)
>  >>>>{
>  >>>>  HTTPClientException e2 = new
>  >>>> HTTPClientException(e.getMessage());
>  >>>>  e2.setStackTrace(e.getStackTrace());
>  >>>>  throw e2;
>  >>>>}
>  >>>>  }
>  >>>>
>  >>>>  public void finish()
>  >>>>  {
>  >>>>// do some cleaning
>  >>>>  }
>  >>>>
>  >>>>   private InputStream getInputStream(String url) throws
>  >>>> HTTPClientException
>  >>>>  {
>  >>>>InputStream instream = null;
>  >>>>
>  >>>>httpClient = new DefaultHttpClient();
>  >>>>httpClient.getParams().setParameter("http.useragent",
>  >>>> AGENT_NAME);
>  >>>>
>  >>>>HttpGet httpGet = new HttpGet(url);
>  >>>>HttpResponse response = null;
>  >>>>
>  >>>>try
>  >>>>{
>  >>>>  response = httpClient.execute(httpGet);
>  >>>>  HttpEntity entity = response.getEntity();
>  >>>>
>  >>>>  i

Re: HttpClient 4.0 encoding madness

[amoldavsky]

Hi Oleg,

Let me rephrase the question in better terms: 
If the server document is Y and buffer size is X, let's even assume that Y =
kX where X < Y, is it possible that any buffer 0 < x < (k-1) will not be
fully filled?

Thanks!
-Assaf


Ken Krugler wrote:
> 
> 
> On Jan 28, 2010, at 10:09pm, amoldavsky wrote:
> 
>>
>> Hi Oleg,
>> Thank you for the quick reply.
>>
>> So if there is a possibility that not the whole buffer is filled how  
>> can I
>> insure or force HttpClient to fill the whole buffer? Should I maybe  
>> avoid
>> Stream Readers all together?
> 
> If bufferSize is X, and the server document you're fetching has Y  
> bytes, then what do you mean by "force HttpClient to fill the whole  
> buffer"?
> 
> At a minimum, you'd want
> 
> int bytesRead = chunkedIns.read(tmp);
> if (bytesRead != -1) {
> return new String(tmp, 0, bytesRead);
> }
> 
> But that also uses the platform default encoding for the character  
> set, which often won't be correct.
> 
> -- Ken
> 
>>
>> olegk wrote:
>>>
>>> On Wed, 2010-01-27 at 20:24 -0800, amoldavsky wrote:
>>>> Hi
>>>>
>>>> I have coded a simple file downloader using HttpClient 4.0.
>>>> It works fine but there is something wrong with the String  
>>>> encoding or
>>>> the
>>>> buffer stream. The problem is that there are long sequences of  
>>>> "NULL"
>>>> (ANSI
>>>> code 00) through out the final file, like this:
>>>> http://old.nabble.com/file/p27350930/httpclient_error01.jpg
>>>> http://old.nabble.com/file/p27350930/httpclient_error02.jpg
>>>>
>>>> Here is the main code:
>>>>
>>>> public String getChunk(String url, int bufferSize) throws
>>>> HTTPClientException
>>>>  {
>>>>if(!chunkedStarted)
>>>>{
>>>>  chunkedIns = getInputStream(url);
>>>>  chunkedStarted = true;
>>>>}
>>>>
>>>>byte[] tmp = new byte[bufferSize];
>>>>try
>>>>{
>>>>  if(chunkedIns.read(tmp) != -1)
>>>>  {
>>>
>>> What makes you think that the entire buffer will be filled with data?
>>>
>>> Oleg
>>>
>>>
>>>>return new String(tmp);
>>>>  }
>>>>  else
>>>>  {
>>>>finish();
>>>>return null;
>>>>  }
>>>>}
>>>>catch(IOException e)
>>>>{
>>>>  HTTPClientException e2 = new  
>>>> HTTPClientException(e.getMessage());
>>>>  e2.setStackTrace(e.getStackTrace());
>>>>  throw e2;
>>>>}
>>>>  }
>>>>
>>>>  public void finish()
>>>>  {
>>>>// do some cleaning
>>>>  }
>>>>
>>>>   private InputStream getInputStream(String url) throws
>>>> HTTPClientException
>>>>  {
>>>>InputStream instream = null;
>>>>
>>>>httpClient = new DefaultHttpClient();
>>>>httpClient.getParams().setParameter("http.useragent",  
>>>> AGENT_NAME);
>>>>
>>>>HttpGet httpGet = new HttpGet(url);
>>>>HttpResponse response = null;
>>>>
>>>>try
>>>>{
>>>>  response = httpClient.execute(httpGet);
>>>>  HttpEntity entity = response.getEntity();
>>>>
>>>>  if(entity != null)
>>>>  {
>>>>instream = entity.getContent();
>>>>  }
>>>>}
>>>>catch(ClientProtocolException e)
>>>>{
>>>>  HTTPClientException e2 = new  
>>>> HTTPClientException(e.getMessage());
>>>>  e2.setStackTrace(e.getStackTrace());
>>>>  throw e2;
>>>>}
>>>>catch(IOException e)
>>>>{
>>>>  HTTPClientException e2 = new  
>>>> HTTPClientException(e.getMessage());
>>>>  e2.setStackTrace(e.getStackTrace());
>>>>  throw e2;
>>>>}
>>>>
>>>>return instream;
>>>>  }
>>>>
>>>> getChuck and getInputStream can basically be one method but I just  
>>>> have
>>>> th

Re: HttpClient 4.0 encoding madness

[Ken Krugler]

On Jan 29, 2010, at 3:35am, sebb wrote:


On 29/01/2010, Ken Krugler  wrote:


On Jan 28, 2010, at 10:09pm, amoldavsky wrote:




Hi Oleg,
Thank you for the quick reply.

So if there is a possibility that not the whole buffer is filled  
how can I
insure or force HttpClient to fill the whole buffer? Should I  
maybe avoid

Stream Readers all together?



If bufferSize is X, and the server document you're fetching has Y  
bytes,

then what do you mean by "force HttpClient to fill the whole buffer"?

At a minimum, you'd want

int bytesRead = chunkedIns.read(tmp);
if (bytesRead != -1) {
  return new String(tmp, 0, bytesRead);
}

But that also uses the platform default encoding for the character  
set,

which often won't be correct.


However, if the user just wants to create a file with the contents of
the response, then surely there is no need to mess with encodings?
Just write the bytes to a file output stream without any conversion.


Good point that streaming to disk avoids dealing with this issue up- 
front.


Note though that you lose the context from the response headers that  
is often used to determine the correct encoding, for when you actually  
do decide to process the data. That's one reason why the arc/warc file  
formats (used for storing web crawl data) include the headers.


--Ken



On Wed, 2010-01-27 at 20:24 -0800, amoldavsky wrote:


Hi

I have coded a simple file downloader using HttpClient 4.0.
It works fine but there is something wrong with the String  
encoding or

the
buffer stream. The problem is that there are long sequences of  
"NULL"

(ANSI
code 00) through out the final file, like this:


http://old.nabble.com/file/p27350930/httpclient_error01.jpg



http://old.nabble.com/file/p27350930/httpclient_error02.jpg


Here is the main code:

public String getChunk(String url, int bufferSize) throws
HTTPClientException
{
 if(!chunkedStarted)
 {
   chunkedIns = getInputStream(url);
   chunkedStarted = true;
 }

 byte[] tmp = new byte[bufferSize];
 try
 {
   if(chunkedIns.read(tmp) != -1)
   {



What makes you think that the entire buffer will be filled with  
data?


Oleg




 return new String(tmp);
   }
   else
   {
 finish();
 return null;
   }
 }
 catch(IOException e)
 {
   HTTPClientException e2 = new

HTTPClientException(e.getMessage());

   e2.setStackTrace(e.getStackTrace());
   throw e2;
 }
}

public void finish()
{
 // do some cleaning
}

private InputStream getInputStream(String url) throws
HTTPClientException
{
 InputStream instream = null;

 httpClient = new DefaultHttpClient();


httpClient.getParams().setParameter("http.useragent",
AGENT_NAME);


 HttpGet httpGet = new HttpGet(url);
 HttpResponse response = null;

 try
 {
   response = httpClient.execute(httpGet);
   HttpEntity entity = response.getEntity();

   if(entity != null)
   {
 instream = entity.getContent();
   }
 }
 catch(ClientProtocolException e)
 {
   HTTPClientException e2 = new

HTTPClientException(e.getMessage());

   e2.setStackTrace(e.getStackTrace());
   throw e2;
 }
 catch(IOException e)
 {
   HTTPClientException e2 = new

HTTPClientException(e.getMessage());

   e2.setStackTrace(e.getStackTrace());
   throw e2;
 }

 return instream;
}

getChuck and getInputStream can basically be one method but I just

have

the
need to split them for internal conveniece, that does not change  
the

funtionality as a whole.

It seems like either the conversion from bytes to string is a  
problem:

return new String(tmp);

or that the buffer is not getting filled to the end. The latter  
could

not

be
possible because the files are ~30MB each and the buffer size is  
2Kb.


I have attached the file, it's a CSV (shortened to ~6KB), note  
that

long
white space between some of the URLs, if you just remove it, the  
URL

makes
sense.
http://old.nabble.com/file/p27350930/datafeed.csv

datafeed.csv


Where can this white space come (null) from??

thank!







-

To unsubscribe, e-mail:

httpclient-users-unsubscr...@hc.apache.org

For additional commands, e-mail:

httpclient-users-h...@hc.apache.org







--
View this message in context:

http://old.nabble.com/HttpClient-4.0-encoding-madness-tp27350930p27366928.html

Sent from the HttpClient-User mailing list archive at Nabble.com.




-

To unsubscribe, e-mail:

httpclient-users-unsubscr...@hc.apache.org

For additional commands, e-mail:

httpclient-users-h...@hc.apache.org






Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c   w e b   m i n i n g







-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient 4.0 encoding madness

[sebb]

On 29/01/2010, Ken Krugler  wrote:
>
>  On Jan 28, 2010, at 10:09pm, amoldavsky wrote:
>
>
> >
> > Hi Oleg,
> > Thank you for the quick reply.
> >
> > So if there is a possibility that not the whole buffer is filled how can I
> > insure or force HttpClient to fill the whole buffer? Should I maybe avoid
> > Stream Readers all together?
> >
>
>  If bufferSize is X, and the server document you're fetching has Y bytes,
> then what do you mean by "force HttpClient to fill the whole buffer"?
>
>  At a minimum, you'd want
>
>  int bytesRead = chunkedIns.read(tmp);
>  if (bytesRead != -1) {
>return new String(tmp, 0, bytesRead);
>  }
>
>  But that also uses the platform default encoding for the character set,
> which often won't be correct.

However, if the user just wants to create a file with the contents of
the response, then surely there is no need to mess with encodings?
Just write the bytes to a file output stream without any conversion.

>  -- Ken
>
>
>
> >
> > olegk wrote:
> >
> > >
> > > On Wed, 2010-01-27 at 20:24 -0800, amoldavsky wrote:
> > >
> > > > Hi
> > > >
> > > > I have coded a simple file downloader using HttpClient 4.0.
> > > > It works fine but there is something wrong with the String encoding or
> > > > the
> > > > buffer stream. The problem is that there are long sequences of "NULL"
> > > > (ANSI
> > > > code 00) through out the final file, like this:
> > > >
> http://old.nabble.com/file/p27350930/httpclient_error01.jpg
> > > >
> http://old.nabble.com/file/p27350930/httpclient_error02.jpg
> > > >
> > > > Here is the main code:
> > > >
> > > > public String getChunk(String url, int bufferSize) throws
> > > > HTTPClientException
> > > >  {
> > > >   if(!chunkedStarted)
> > > >   {
> > > > chunkedIns = getInputStream(url);
> > > > chunkedStarted = true;
> > > >   }
> > > >
> > > >   byte[] tmp = new byte[bufferSize];
> > > >   try
> > > >   {
> > > > if(chunkedIns.read(tmp) != -1)
> > > > {
> > > >
> > >
> > > What makes you think that the entire buffer will be filled with data?
> > >
> > > Oleg
> > >
> > >
> > >
> > > >   return new String(tmp);
> > > > }
> > > > else
> > > > {
> > > >   finish();
> > > >   return null;
> > > > }
> > > >   }
> > > >   catch(IOException e)
> > > >   {
> > > > HTTPClientException e2 = new
> HTTPClientException(e.getMessage());
> > > > e2.setStackTrace(e.getStackTrace());
> > > > throw e2;
> > > >   }
> > > >  }
> > > >
> > > >  public void finish()
> > > >  {
> > > >   // do some cleaning
> > > >  }
> > > >
> > > >  private InputStream getInputStream(String url) throws
> > > > HTTPClientException
> > > >  {
> > > >   InputStream instream = null;
> > > >
> > > >   httpClient = new DefaultHttpClient();
> > > >
> httpClient.getParams().setParameter("http.useragent",
> AGENT_NAME);
> > > >
> > > >   HttpGet httpGet = new HttpGet(url);
> > > >   HttpResponse response = null;
> > > >
> > > >   try
> > > >   {
> > > > response = httpClient.execute(httpGet);
> > > > HttpEntity entity = response.getEntity();
> > > >
> > > > if(entity != null)
> > > > {
> > > >   instream = entity.getContent();
> > > > }
> > > >   }
> > > >   catch(ClientProtocolException e)
> > > >   {
> > > > HTTPClientException e2 = new
> HTTPClientException(e.getMessage());
> > > > e2.setStackTrace(e.getStackTrace());
> > > > throw e2;
> > > >   }
> > > >   catch(IOException e)
> > > >   {
> > > > HTTPClientException e2 = new
> HTTPClientException(e.getMessage());
> > > > e2.setStackTrace(e.getStackTrace());
> > > > throw e2;
> > > >   }
> > > >
> > > >   return instream;
> > > >  }
> > > >
> 

Re: HttpClient 4.0 encoding madness

[Ken Krugler]

On Jan 28, 2010, at 10:09pm, amoldavsky wrote:



Hi Oleg,
Thank you for the quick reply.

So if there is a possibility that not the whole buffer is filled how  
can I
insure or force HttpClient to fill the whole buffer? Should I maybe  
avoid

Stream Readers all together?


If bufferSize is X, and the server document you're fetching has Y  
bytes, then what do you mean by "force HttpClient to fill the whole  
buffer"?


At a minimum, you'd want

int bytesRead = chunkedIns.read(tmp);
if (bytesRead != -1) {
   return new String(tmp, 0, bytesRead);
}

But that also uses the platform default encoding for the character  
set, which often won't be correct.


-- Ken



olegk wrote:


On Wed, 2010-01-27 at 20:24 -0800, amoldavsky wrote:

Hi

I have coded a simple file downloader using HttpClient 4.0.
It works fine but there is something wrong with the String  
encoding or

the
buffer stream. The problem is that there are long sequences of  
"NULL"

(ANSI
code 00) through out the final file, like this:
http://old.nabble.com/file/p27350930/httpclient_error01.jpg
http://old.nabble.com/file/p27350930/httpclient_error02.jpg

Here is the main code:

public String getChunk(String url, int bufferSize) throws
HTTPClientException
 {
   if(!chunkedStarted)
   {
 chunkedIns = getInputStream(url);
 chunkedStarted = true;
   }

   byte[] tmp = new byte[bufferSize];
   try
   {
 if(chunkedIns.read(tmp) != -1)
 {


What makes you think that the entire buffer will be filled with data?

Oleg



   return new String(tmp);
 }
 else
 {
   finish();
   return null;
 }
   }
   catch(IOException e)
   {
 HTTPClientException e2 = new  
HTTPClientException(e.getMessage());

 e2.setStackTrace(e.getStackTrace());
 throw e2;
   }
 }

 public void finish()
 {
   // do some cleaning
 }

  private InputStream getInputStream(String url) throws
HTTPClientException
 {
   InputStream instream = null;

   httpClient = new DefaultHttpClient();
   httpClient.getParams().setParameter("http.useragent",  
AGENT_NAME);


   HttpGet httpGet = new HttpGet(url);
   HttpResponse response = null;

   try
   {
 response = httpClient.execute(httpGet);
 HttpEntity entity = response.getEntity();

 if(entity != null)
 {
   instream = entity.getContent();
 }
   }
   catch(ClientProtocolException e)
   {
 HTTPClientException e2 = new  
HTTPClientException(e.getMessage());

 e2.setStackTrace(e.getStackTrace());
 throw e2;
   }
   catch(IOException e)
   {
 HTTPClientException e2 = new  
HTTPClientException(e.getMessage());

 e2.setStackTrace(e.getStackTrace());
 throw e2;
   }

   return instream;
 }

getChuck and getInputStream can basically be one method but I just  
have

the
need to split them for internal conveniece, that does not change the
funtionality as a whole.

It seems like either the conversion from bytes to string is a  
problem:

return new String(tmp);

or that the buffer is not getting filled to the end. The latter  
could not

be
possible because the files are ~30MB each and the buffer size is  
2Kb.


I have attached the file, it's a CSV (shortened to ~6KB), note  
that long

white space between some of the URLs, if you just remove it, the URL
makes
sense.
http://old.nabble.com/file/p27350930/datafeed.csv datafeed.csv

Where can this white space come (null) from??

thank!




-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org





--
View this message in context: 
http://old.nabble.com/HttpClient-4.0-encoding-madness-tp27350930p27366928.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org




Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c   w e b   m i n i n g

Re: HttpClient 4.0 encoding madness

[amoldavsky]

Hi Oleg,
Thank you for the quick reply.

So if there is a possibility that not the whole buffer is filled how can I
insure or force HttpClient to fill the whole buffer? Should I maybe avoid
Stream Readers all together?

What do you think will be the best route to go?

Thank you!
-Assaf


olegk wrote:
> 
> On Wed, 2010-01-27 at 20:24 -0800, amoldavsky wrote:
>> Hi
>> 
>> I have coded a simple file downloader using HttpClient 4.0.
>> It works fine but there is something wrong with the String encoding or
>> the
>> buffer stream. The problem is that there are long sequences of "NULL"
>> (ANSI
>> code 00) through out the final file, like this:
>> http://old.nabble.com/file/p27350930/httpclient_error01.jpg 
>> http://old.nabble.com/file/p27350930/httpclient_error02.jpg 
>> 
>> Here is the main code:
>> 
>> public String getChunk(String url, int bufferSize) throws
>> HTTPClientException
>>   {
>> if(!chunkedStarted)
>> {
>>   chunkedIns = getInputStream(url);
>>   chunkedStarted = true;
>> }
>> 
>> byte[] tmp = new byte[bufferSize];
>> try
>> {
>>   if(chunkedIns.read(tmp) != -1)
>>   {
> 
> What makes you think that the entire buffer will be filled with data?
> 
> Oleg 
> 
> 
>> return new String(tmp);
>>   }
>>   else
>>   {
>> finish();
>> return null;
>>   }
>> }
>> catch(IOException e)
>> {
>>   HTTPClientException e2 = new HTTPClientException(e.getMessage());
>>   e2.setStackTrace(e.getStackTrace());
>>   throw e2;
>> }
>>   }
>>   
>>   public void finish()
>>   {
>> // do some cleaning
>>   }
>> 
>>private InputStream getInputStream(String url) throws
>> HTTPClientException
>>   {
>> InputStream instream = null;
>> 
>> httpClient = new DefaultHttpClient();
>> httpClient.getParams().setParameter("http.useragent", AGENT_NAME);
>> 
>> HttpGet httpGet = new HttpGet(url);
>> HttpResponse response = null;
>> 
>> try
>> {
>>   response = httpClient.execute(httpGet);
>>   HttpEntity entity = response.getEntity();
>> 
>>   if(entity != null) 
>>   {
>> instream = entity.getContent();
>>   }
>> }
>> catch(ClientProtocolException e)
>> {
>>   HTTPClientException e2 = new HTTPClientException(e.getMessage());
>>   e2.setStackTrace(e.getStackTrace());
>>   throw e2;
>> }
>> catch(IOException e)
>> {
>>   HTTPClientException e2 = new HTTPClientException(e.getMessage());
>>   e2.setStackTrace(e.getStackTrace());
>>   throw e2;
>> }
>> 
>> return instream;
>>   }
>> 
>> getChuck and getInputStream can basically be one method but I just have
>> the
>> need to split them for internal conveniece, that does not change the
>> funtionality as a whole.
>> 
>> It seems like either the conversion from bytes to string is a problem:
>> return new String(tmp);
>> 
>> or that the buffer is not getting filled to the end. The latter could not
>> be
>> possible because the files are ~30MB each and the buffer size is 2Kb.
>> 
>> I have attached the file, it's a CSV (shortened to ~6KB), note that long
>> white space between some of the URLs, if you just remove it, the URL
>> makes
>> sense.
>> http://old.nabble.com/file/p27350930/datafeed.csv datafeed.csv 
>> 
>> Where can this white space come (null) from??
>> 
>> thank!
> 
> 
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 
> 
> 

-- 
View this message in context: 
http://old.nabble.com/HttpClient-4.0-encoding-madness-tp27350930p27366928.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient 4.0 encoding madness

[Oleg Kalnichevski]

On Wed, 2010-01-27 at 20:24 -0800, amoldavsky wrote:
> Hi
> 
> I have coded a simple file downloader using HttpClient 4.0.
> It works fine but there is something wrong with the String encoding or the
> buffer stream. The problem is that there are long sequences of "NULL" (ANSI
> code 00) through out the final file, like this:
> http://old.nabble.com/file/p27350930/httpclient_error01.jpg 
> http://old.nabble.com/file/p27350930/httpclient_error02.jpg 
> 
> Here is the main code:
> 
> public String getChunk(String url, int bufferSize) throws
> HTTPClientException
>   {
> if(!chunkedStarted)
> {
>   chunkedIns = getInputStream(url);
>   chunkedStarted = true;
> }
> 
> byte[] tmp = new byte[bufferSize];
> try
> {
>   if(chunkedIns.read(tmp) != -1)
>   {

What makes you think that the entire buffer will be filled with data?

Oleg 


> return new String(tmp);
>   }
>   else
>   {
> finish();
> return null;
>   }
> }
> catch(IOException e)
> {
>   HTTPClientException e2 = new HTTPClientException(e.getMessage());
>   e2.setStackTrace(e.getStackTrace());
>   throw e2;
> }
>   }
>   
>   public void finish()
>   {
> // do some cleaning
>   }
> 
>private InputStream getInputStream(String url) throws HTTPClientException
>   {
> InputStream instream = null;
> 
> httpClient = new DefaultHttpClient();
> httpClient.getParams().setParameter("http.useragent", AGENT_NAME);
> 
> HttpGet httpGet = new HttpGet(url);
> HttpResponse response = null;
> 
> try
> {
>   response = httpClient.execute(httpGet);
>   HttpEntity entity = response.getEntity();
> 
>   if(entity != null) 
>   {
> instream = entity.getContent();
>   }
> }
> catch(ClientProtocolException e)
> {
>   HTTPClientException e2 = new HTTPClientException(e.getMessage());
>   e2.setStackTrace(e.getStackTrace());
>   throw e2;
> }
> catch(IOException e)
> {
>   HTTPClientException e2 = new HTTPClientException(e.getMessage());
>   e2.setStackTrace(e.getStackTrace());
>   throw e2;
> }
> 
> return instream;
>   }
> 
> getChuck and getInputStream can basically be one method but I just have the
> need to split them for internal conveniece, that does not change the
> funtionality as a whole.
> 
> It seems like either the conversion from bytes to string is a problem:
> return new String(tmp);
> 
> or that the buffer is not getting filled to the end. The latter could not be
> possible because the files are ~30MB each and the buffer size is 2Kb.
> 
> I have attached the file, it's a CSV (shortened to ~6KB), note that long
> white space between some of the URLs, if you just remove it, the URL makes
> sense.
> http://old.nabble.com/file/p27350930/datafeed.csv datafeed.csv 
> 
> Where can this white space come (null) from??
> 
> thank!



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

HttpClient 4.0 encoding madness

[amoldavsky]

Hi

I have coded a simple file downloader using HttpClient 4.0.
It works fine but there is something wrong with the String encoding or the
buffer stream. The problem is that there are long sequences of "NULL" (ANSI
code 00) through out the final file, like this:
http://old.nabble.com/file/p27350930/httpclient_error01.jpg 
http://old.nabble.com/file/p27350930/httpclient_error02.jpg 

Here is the main code:

public String getChunk(String url, int bufferSize) throws
HTTPClientException
  {
if(!chunkedStarted)
{
  chunkedIns = getInputStream(url);
  chunkedStarted = true;
}

byte[] tmp = new byte[bufferSize];
try
{
  if(chunkedIns.read(tmp) != -1)
  {
return new String(tmp);
  }
  else
  {
finish();
return null;
  }
}
catch(IOException e)
{
  HTTPClientException e2 = new HTTPClientException(e.getMessage());
  e2.setStackTrace(e.getStackTrace());
  throw e2;
}
  }
  
  public void finish()
  {
// do some cleaning
  }

   private InputStream getInputStream(String url) throws HTTPClientException
  {
InputStream instream = null;

httpClient = new DefaultHttpClient();
httpClient.getParams().setParameter("http.useragent", AGENT_NAME);

HttpGet httpGet = new HttpGet(url);
HttpResponse response = null;

try
{
  response = httpClient.execute(httpGet);
  HttpEntity entity = response.getEntity();

  if(entity != null) 
  {
instream = entity.getContent();
  }
}
catch(ClientProtocolException e)
{
  HTTPClientException e2 = new HTTPClientException(e.getMessage());
  e2.setStackTrace(e.getStackTrace());
  throw e2;
}
catch(IOException e)
{
  HTTPClientException e2 = new HTTPClientException(e.getMessage());
  e2.setStackTrace(e.getStackTrace());
  throw e2;
}

return instream;
  }

getChuck and getInputStream can basically be one method but I just have the
need to split them for internal conveniece, that does not change the
funtionality as a whole.

It seems like either the conversion from bytes to string is a problem:
return new String(tmp);

or that the buffer is not getting filled to the end. The latter could not be
possible because the files are ~30MB each and the buffer size is 2Kb.

I have attached the file, it's a CSV (shortened to ~6KB), note that long
white space between some of the URLs, if you just remove it, the URL makes
sense.
http://old.nabble.com/file/p27350930/datafeed.csv datafeed.csv 

Where can this white space come (null) from??

thank!
-- 
View this message in context: 
http://old.nabble.com/HttpClient-4.0-encoding-madness-tp27350930p27350930.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: webrec questions httpclient 4.0 stable

[Stefan Parvu]

Hi,




You may also want to search the archives for questions on performance
tuning.


We will do that. I expect if I have 5 or 40sec sleep time in that
sample from your site the R be almost similar. We will look for
stale connection check.



Thanks for all comments and help. We been able to obtain
consisten numbers with HTTPCLient 4.0.1 and we will base our
webrec on this library.

We did:

 a). remove the STALE check connection
 b). handle manually the idle/closed connections.

It was a bit of pain to get deeper into the persistance
connection mantra but we sorted out. Soon you will be able
to see our webrec in source code at our site.

Thanks again,
Stefan

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Sending SOAP request using HTTPPost (HttpClient 4.0)

[sac_sachin]

Hi Arpita,

Hope, you have fixed this problem.  I am new to SOAP, i am writing new
application based on SOAP, can you tell me whtat the below soap body does ?

   + ""
+ "http://videoos.net/2/XProtectCSServerCommand\";>"
+
"6a716afc-708e-479f-8b42-f97338d794f9"
+ "" + "" + ""

I have understood that your are login to the server "http://videoos.net";
with the instance ID 6a716afc-708e-479f-8b42-f97338d794f9. 

what kind of service should be running on the server to communicate with the
soap service?

Thanks in advance.





arpita dhundia wrote:
> 
> Hello Folks,
> 
> I have run into a very odd problem.
> I have to send a SOAP (xml data over HTTP Post) request to another server
> behind NTLM authentication. I am using HttpClient 4.0 for that.
> The NTLM authentication works fine but when I try to send the xml data
> using StringEntity and set that Entity to the HTTPPpost Object, I am
> returned 400 error code, meaning "bad request".
> I tried to see the exact request that HTTPClient is sending using
> WireShark and indeed the format of the request is not correct. The headers
> are repeated twice and the body too is not correctly placed (unfortunately
> I cant point out the exact mistake, but comparing it with a request that a
> soap client sends its different and seeming incorrect).
> 
> Is anybody aware of any problems related to StringEntity? Or maybe any
> other way I can send xml data using HttpPost method in HttpClient 4.0 API.
> 
> Any help would be highly appreciated, I am behind my deadline on this task
> :(.
> 
> Thanks!!
> 
> PS:- For better understanding, here is a snippet of the code:-
> 
> DefaultHttpClient client = new DefaultHttpClient();
>         client.getAuthSchemes().register("ntlm", new NTLMSchemeFactory());
>         client.getCredentialsProvider().setCredentials(
>                 new AuthScope("host", port, "domain"),
>                 new NTCredentials("username", "password", "domain",
>                         "domain2"));
> 
>         HttpPost httpget = new HttpPost(
>                 "http://ip/ServerAPI/ServerCommandService.asmx";);
> 
>         try {
>             String soapRequest = " version=\"1.0\"encoding=\"utf-8\"?>"
>                     + "                     +
> "xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"; "
>                     + "xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"; "
>                     +
> "xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\"";
>                     + ""
>                     + " xmlns=\"http://videoos.net/2/XProtectCSServerCommand\";>"
>                     +
> "6a716afc-708e-479f-8b42-f97338d794f9"
>                     + "" + "" + ""
>                     + "";
> 
> 
>             httpget.addHeader("SOAPAction",
>                    
> "\"http://videoos.net/2/XProtectCSServerCommand/Login\"";);
> 
>             StringEntity strent= new StringEntity(soapRequest);
>             strent.setContentType("text/xml; charset=utf-8");
>             httpget.setEntity(strent);
> 
>             ResponseHandler response = new BasicResponseHandler();
>             String responseBody = client.execute(httpget, response);
>             System.out.println(responseBody);
> }catch(IOException ex)
> {
> ex.printStackTrace();
> }
> 
> 
> 
>   Now, send attachments up to 25MB with Yahoo! India Mail. Learn how.
> http://in.overview.mail.yahoo.com/photos
> 

-- 
View this message in context: 
http://old.nabble.com/Sending-SOAP-request-using-HTTPPost-%28HttpClient-4.0%29-tp26127809p26943790.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: webrec questions httpclient 4.0 stable

[Stefan Parvu]

what on earth is Solaris HTTP Server?


That was a simple explanation how we ran the test:

 client java application, Linux box
 server side, Solaris box with HTTP server


That is because wget and the likes usually do not re-use connections.
Some web servers can not handle persistent connections well, so opening
a new connection may turn out to be faster than re-using a persistent
connection.


Im trying to compare the use of our java code. wget, curl were given as
example.



Turn off the stale connection check and use context / wire logging to
see where exactly Httpclient gets blocked.

http://hc.apache.org/httpcomponents-client/logging.html



are you suggesting we turn off in our code the stale connection check ?

java client:
 * sleep time  5sec, R=10 sec
 * sleep time 35sec, R=3  sec



You may also want to search the archives for questions on performance
tuning.


We will do that. I expect if I have 5 or 40sec sleep time in that
sample from your site the R be almost similar. We will look for
stale connection check.

Thanks for advice and tips,
Stefan

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: webrec questions httpclient 4.0 stable

[Oleg Kalnichevski]

On Mon, 2009-12-28 at 14:44 +0200, Stefan Parvu wrote:
> Hello Oleg,
> 
> > Makes perfect sense. HttpClient allows only two concurrent connections
> > to the same host per default as required by the HTTP spec. Therefore,
> > those two threads that manage to lease a connection from the pool first
> > execute faster than those that stay blocked waiting for a connection to
> > free up.
> 
> True. There is a RFC stating than no more than 2 concurrent
> connections are allowed. That's fine.
> 
> However our problem is this: if we change the sleep time
> in the example:
> http://www.systemdatarecorder.org/ClientMultiThreadedExecution.java
> 
> 
>   /* ** */
>   long timeLapse = cTime - pTime;
>   System.out.println(id + " - get executed. Response time:" + timeLapse );
> 
>   // get the response body as an array of bytes
>   HttpEntity entity = response.getEntity();
> 
>   if (entity != null) {
>  byte[] bytes = EntityUtils.toByteArray(entity);
>  System.out.println(id + " - " + bytes.length + " bytes read");
>   }
> 
>   sleep(3);
>   ^
> 
>   /* ** */
> 
> more than 30sec, the response times is 5 smaller than in
> other cases.
> 
> Example: running the previous code example with only 2
> HTTP requests to avoid the RFC confusion.
> 
> 
> 1). Sleep time 5s
> 
> 1 - get executed. Response time:173
> 2 - get executed. Response time:170
> 2 - 8005 bytes read
> 1 - 8005 bytes read
> 
>  the methods are not yet compiled>
> 
> 1 - get executed. Response time:11
> 2 - get executed. Response time:11
> 1 - 8005 bytes read
> 2 - 8005 bytes read
> 
> 
> 2 - get executed. Response time:11
> 1 - get executed. Response time:11
> 2 - 8005 bytes read
> 1 - 8005 bytes read
> 
> So our response time is 11ms. Seems consistent 10-11ms.
> 
> 
> 2). Sleep time 35s
> 
> 2 - get executed. Response time:130
> 1 - get executed. Response time:131
> 2 - 8005 bytes read
> 1 - 8005 bytes read
> 
> 
> 2 - get executed. Response time:3
> 1 - get executed. Response time:3
> 2 - 8005 bytes read
> 1 - 8005 bytes read
> 
> 2 - get executed. Response time:2
> 1 - get executed. Response time:2
> 2 - 8005 bytes read
> 1 - 8005 bytes read
> 
> Now we started to see a different thing. Response times went
> down 5 times since previous run. Why is that ?
> 
> So our problem seems to be dependant on sleep time between runs.
> If it is > 30s the response times seem to align with the reality.
> 
> We tested the code from this example:
> 
> 
>   Client
>   Linux  --- Solaris HTTP Server
> 

what on earth is Solaris HTTP Server?

> Usually it takes 1 or 2ms for httping or a simple wget or curl.
> 
> Any ideas ?
> 

That is because wget and the likes usually do not re-use connections.
Some web servers can not handle persistent connections well, so opening
a new connection may turn out to be faster than re-using a persistent
connection.

Turn off the stale connection check and use context / wire logging to
see where exactly Httpclient gets blocked.

http://hc.apache.org/httpcomponents-client/logging.html

You may also want to search the archives for questions on performance
tuning.

Oleg

> Many thanks for help.
> 
> Stefan
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: webrec questions httpclient 4.0 stable

[Stefan Parvu]

Hello Oleg,


Makes perfect sense. HttpClient allows only two concurrent connections
to the same host per default as required by the HTTP spec. Therefore,
those two threads that manage to lease a connection from the pool first
execute faster than those that stay blocked waiting for a connection to
free up.


True. There is a RFC stating than no more than 2 concurrent
connections are allowed. That's fine.

However our problem is this: if we change the sleep time
in the example:
http://www.systemdatarecorder.org/ClientMultiThreadedExecution.java


 /* ** */
 long timeLapse = cTime - pTime;
 System.out.println(id + " - get executed. Response time:" + timeLapse );

 // get the response body as an array of bytes
 HttpEntity entity = response.getEntity();

 if (entity != null) {
byte[] bytes = EntityUtils.toByteArray(entity);
System.out.println(id + " - " + bytes.length + " bytes read");
 }

 sleep(3);
 ^

 /* ** */

more than 30sec, the response times is 5 smaller than in
other cases.

Example: running the previous code example with only 2
HTTP requests to avoid the RFC confusion.


1). Sleep time 5s

1 - get executed. Response time:173
2 - get executed. Response time:170
2 - 8005 bytes read
1 - 8005 bytes read



1 - get executed. Response time:11
2 - get executed. Response time:11
1 - 8005 bytes read
2 - 8005 bytes read


2 - get executed. Response time:11
1 - get executed. Response time:11
2 - 8005 bytes read
1 - 8005 bytes read

So our response time is 11ms. Seems consistent 10-11ms.


2). Sleep time 35s

2 - get executed. Response time:130
1 - get executed. Response time:131
2 - 8005 bytes read
1 - 8005 bytes read


2 - get executed. Response time:3
1 - get executed. Response time:3
2 - 8005 bytes read
1 - 8005 bytes read

2 - get executed. Response time:2
1 - get executed. Response time:2
2 - 8005 bytes read
1 - 8005 bytes read

Now we started to see a different thing. Response times went
down 5 times since previous run. Why is that ?

So our problem seems to be dependant on sleep time between runs.
If it is > 30s the response times seem to align with the reality.

We tested the code from this example:


 Client
 Linux  --- Solaris HTTP Server

Usually it takes 1 or 2ms for httping or a simple wget or curl.

Any ideas ?

Many thanks for help.

Stefan

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: webrec questions httpclient 4.0 stable

[Oleg Kalnichevski]

On Mon, 2009-12-28 at 00:55 +0200, Stefan Parvu wrote:
> > We have seen that after the first run (usually 1min)
> > the first response time recorded for the first request
> > is 2milliseconds followed by the next requests with R=10ms or 11ms.
> 
> Here maybe a much simpler example:
> http://www.systemdatarecorder.org/ClientMultiThreadedExecution.java
> which can compile easily.
> 
> This code loops over 4 similar HTTP requests obtaining different
> Response times. We have a sleep after hitting all 4 requests
> set to 35secs.
> 
> When we lower this value, the sleep, we obtain the response times
> similar 10, 11 ms. When we increase the sleep time the first request
> and second are ok, the others are bogus.
> 
> Why this variation ?
> 
> Makes sense ?
> 

Makes perfect sense. HttpClient allows only two concurrent connections
to the same host per default as required by the HTTP spec. Therefore,
those two threads that manage to lease a connection from the pool first
execute faster than those that stay blocked waiting for a connection to
free up.

Oleg


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: webrec questions httpclient 4.0 stable

[Stefan Parvu]

We have seen that after the first run (usually 1min)
the first response time recorded for the first request
is 2milliseconds followed by the next requests with R=10ms or 11ms.


Here maybe a much simpler example:
http://www.systemdatarecorder.org/ClientMultiThreadedExecution.java
which can compile easily.

This code loops over 4 similar HTTP requests obtaining different
Response times. We have a sleep after hitting all 4 requests
set to 35secs.

When we lower this value, the sleep, we obtain the response times
similar 10, 11 ms. When we increase the sleep time the first request
and second are ok, the others are bogus.

Why this variation ?

Makes sense ?

Stefan

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: webrec questions httpclient 4.0 stable

[Stefan Parvu]

Evening,

Many thanks for help.

Yes, they do, but it is usually significantly lower than that of establishing 
a new connection.


Cheers. I have still something unclear.

Our application is recording response time for a sequence
of URLs with a delay of 5s between them, like an example.
Then main execution thread is sleeping xx seconds
before next round of measurements.

We have seen that after the first run (usually 1min)
the first response time recorded for the first request
is 2milliseconds followed by the next requests with R=10ms or 11ms.

They all should have the same response time, because we used
the same url in all the sequence.

Below is extract of the execution of the webrec running
many minutes, running on the same server as the HTTP
server to avoid first the network delay etc ...

1261847585:2:11:10:10:11:10:9
1261847675:2:10:10:11:10:10:8
1261847765:2:11:10:10:11:10:9
1261847855:2:10:11:11:10:11:9
1261847946:2:11:10:11:10:10:9
1261848036:2:11:11:10:10:11:9
1261848126:2:10:10:11:10:10:8
1261848216:1:11:10:10:11:10:8
1261848306:2:10:11:11:10:10:9
1261848396:2:11:11:10:11:10:9
1261848486:2:10:11:10:10:11:9
1261848576:2:10:10:11:10:10:8

where each field is defined:
timestamp:
response time for 1st HTTP request:
...
response time for 6th HTTP request:
average response time

Any ideas why always the 1st request has least response time
(1-2ms) than other request? We see an overhead on all the
other request but not the first one.

The real response time should actually be like 2ms,
because we have measured it with other tools (wget, curl).
Below the code responsible for doing this:
http://www.systemdatarecorder.org/webrec.snippet.java

Many thanks,
Stefan

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: webrec questions httpclient 4.0 stable

[Oleg Kalnichevski]

Stefan Parvu wrote:


Hi,

We are a small group of folks working on a measurement toolkit:
www.systemdatarecorder.org. One of the tools, webrec is supposed
to be a HTTP client measurement tool, recording the response
times for certain HTTP methods.
http://systemdatarecorder.org:9009/bugzilla/show_bug.cgi?id=21

The utility uses HTTPClient in a multithreaded way, see the specs for 
webrec, one thread per workload.


I'm using ThreadSafeClientConnManager in my application. Now, I have 
noticed in this mailing list is recommeded to use interceptors for 
measuring metrics with HTTPClient.



Questions:

1) Do I have to use interceptor also measuring response time?
Can't I just use:

long pTime = System.currentTimeMillis();
long cTime = 0;
response = httpclient.execute(httpget);
if (response.getStatusLine().getStatusCode() == 
HttpURLConnection.HTTP_OK) {

cTime = System.currentTimeMillis();
}
long timeLapse = cTime - pTime;



You do not have to.



2). Does connection managers add some overhead to requests? 


Yes, they do, but it is usually significantly lower than that of 
establishing a new connection.


Why there is

a need to use interceptors?



Interceptors provide you with a finer control over the request execution 
process, but you do not have to use them if you do not want to.


Oleg



Many thanks,
Stefan

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org




-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

webrec questions httpclient 4.0 stable

[Stefan Parvu]

Hi,

We are a small group of folks working on a measurement toolkit:
www.systemdatarecorder.org. One of the tools, webrec is supposed
to be a HTTP client measurement tool, recording the response
times for certain HTTP methods.
http://systemdatarecorder.org:9009/bugzilla/show_bug.cgi?id=21

The utility uses HTTPClient in a multithreaded way, see the specs for 
webrec, one thread per workload.


I'm using ThreadSafeClientConnManager in my application. Now, I have 
noticed in this mailing list is recommeded to use interceptors 
for measuring metrics with HTTPClient.



Questions:

1) Do I have to use interceptor also measuring response time?
Can't I just use:

long pTime = System.currentTimeMillis();
long cTime = 0;
response = httpclient.execute(httpget);
if (response.getStatusLine().getStatusCode() == HttpURLConnection.HTTP_OK) 
{

cTime = System.currentTimeMillis();
}
long timeLapse = cTime - pTime;


2). Does connection managers add some overhead to requests? Why there is a 
need to use interceptors?


Many thanks,
Stefan

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

RE: httpClient 4.0 and JSON

[Juanjo]

Thank you
Juanjo 

-Mensaje original-
De: Oleg Kalnichevski [mailto:ol...@apache.org] 
Enviado el: viernes, 11 de diciembre de 2009 12:31
Para: HttpClient User Discussion
Asunto: Re: httpClient 4.0 and JSON

Juanjo wrote:
> Hi,
>  
> In my application, I need to send a post request using JSON
parameters.
>  
> In PostData, the format is somthing like this:  
> {"aaa":"a","bbb":"b"}
>  
> How can I use the setEntity method of HttpPost?
>  
> Thanks in advance
>  
> Juanjo
> 

http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html#d
4e164

Oleg

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: httpClient 4.0 and JSON

[Oleg Kalnichevski]

Juanjo wrote:

Hi,
 
In my application, I need to send a post request using JSON parameters.
 
In PostData, the format is somthing like this:  {"aaa":"a","bbb":"b"}
 
How can I use the setEntity method of HttpPost?
 
Thanks in advance
 
Juanjo




http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html#d4e164

Oleg

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

httpClient 4.0 and JSON

[Juanjo]

Hi,
 
In my application, I need to send a post request using JSON parameters.
 
In PostData, the format is somthing like this:  {"aaa":"a","bbb":"b"}
 
How can I use the setEntity method of HttpPost?
 
Thanks in advance
 
Juanjo

Re: Calling servlet help (HttpClient 4.0)

[scook]

Thanks, looking into it, so far no success but the battle continues.

Thanks
steve

> Well, the idea is to try to set the "Accept" header of the request, so
> that the media type requested is compatible with what the servlets are
> configured to serve:
>
> HttpGet httpget = new HttpGet("...");
> httpget.setRequestHeader("Accept", "text/xml");
>
> If it still doesn't work, try to tweak the media type, i.e.
> "application/xml", ...
>
> You can learn more about the Accept header in RFC 2616, section 14.1.
>
> Hope this helps.
>
> On 4 déc. 2009, at 03:59, sc...@seanet.com wrote:
>
>> Can you expand on your answer a bit?  The accept header in the servlet
>> or
>> some API call (setRequest...() - what value?) in the http client?  The
>> servlets set to return text/xml BTW.
>>
>> I've also tried it using straight Java HttpUrlConnection, get same
>> result.
>>
>> Thanks for the help
>> Steve
>>
>>> Have you tried to set the "Accept" header of your request to the proper
>>> media type ? Without it, your server might consider that you expect the
>>> default, e.g. text/html, and won't serve you the xml. That's my guess.
>>>
>>> On 2 déc. 2009, at 04:25, sc...@seanet.com wrote:
>>>
 Hi,

 Please excuse a newbie question but how do I get back the XML content
 back
 from a servlet.  All I seem to get is the index.html web page..

 Basic call is:

 HttpClient httpclient = new DefaultHttpClient();
 HttpGet httpget = new
  HttpGet("http://localhost:8080/wc_server?task=get_results";);
 HttpResponse response = httpclient.execute(httpget);

 HttpEntity entity = response.getEntity();

 if (entity != null)
 {
   long len = entity.getContentLength();
   if (len != -1 ) // && len < 8192)
   {
   System.out.println(EntityUtils.toString(entity));
   }
 }


 Whats returned is the index.html page in that servlets top directory,
 not
 the XML data for that method call.  I get the same results with
 either:

 HttpGet("http://localhost:8080/wc_server?task=get_results";);

 or

 HttpGet("http://localhost:8080/wc_server";);

 It seems like it's not getting/processing the '?task=get_results'

 If I remove the index.html file from the servlet directory it just
 errors
 back...

 Thanks
 Steve Cook
 sc...@seanet.com
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Calling servlet help (HttpClient 4.0)

[Oleg Kalnichevski]

sc...@seanet.com wrote:

Can you expand on your answer a bit?  The accept header in the servlet or
some API call (setRequest...() - what value?) in the http client?  The
servlets set to return text/xml BTW.

I've also tried it using straight Java HttpUrlConnection, get same result.

Thanks for the help
Steve



Steve,

You need to know _exactly_ what kind of conversation session your web 
server expects. The usual way of approaching the problem would be to 
capture an HTTP session generated with a browser and then to simulate 
the same session with HttpClient


Oleg



Have you tried to set the "Accept" header of your request to the proper
media type ? Without it, your server might consider that you expect the
default, e.g. text/html, and won't serve you the xml. That's my guess.

On 2 déc. 2009, at 04:25, sc...@seanet.com wrote:


Hi,

Please excuse a newbie question but how do I get back the XML content
back
from a servlet.  All I seem to get is the index.html web page..

Basic call is:

HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new
  HttpGet("http://localhost:8080/wc_server?task=get_results";);
HttpResponse response = httpclient.execute(httpget);

HttpEntity entity = response.getEntity();

if (entity != null)
{
   long len = entity.getContentLength();
   if (len != -1 ) // && len < 8192)
   {
   System.out.println(EntityUtils.toString(entity));
   }
}


Whats returned is the index.html page in that servlets top directory,
not
the XML data for that method call.  I get the same results with either:

HttpGet("http://localhost:8080/wc_server?task=get_results";);

or

HttpGet("http://localhost:8080/wc_server";);

It seems like it's not getting/processing the '?task=get_results'

If I remove the index.html file from the servlet directory it just
errors
back...

Thanks
Steve Cook
sc...@seanet.com

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org






-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org




-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Setting cookie policy with HttpClient 4.0

[Oleg Kalnichevski]

Ken Krugler wrote:

Hi Oleg,

On Dec 3, 2009, at 2:40am, Oleg Kalnichevski wrote:


On Wed, 2009-12-02 at 19:15 -0800, Ken Krugler wrote:

Below is an email from August 7th, which I'm reviving due to this
becoming a bigger issue over in Bixo-land.

I've continued to run into this issue with my crawls, but so far I'm
not doing anything with cookies, so it hasn't been a priority to track
down.

However another Bixo user also runs into it, and he noticed that by
switching back to HttpClient 4.0-beta3, the warnings went away.

I believe he just opened HTTPCLIENT-896 as a clone of HTTPCLIENT-773,
which seemed to be this exact same bug (fixed by Oleg around 17/May/08).

I'm wondering if the bug crept back into the code sometime between
then and the final release.

Thanks,

-- Ken



Hi Ken

The cookie in question violates the format of 'expires' attribute
expected by the Netscape policy. One can configure the policy to be more
lenient about the format of 'expires' attribute by using a special HTTP
parameter. For details see HTTPCLIENT-896.

It is not really a regression. I think the Netscape cookie policy was
made stricter at some point of time post 4.0-beta1

Hope this clarifies the situation.


Thanks for the clarification, and the example code you added in a 
comment to HTTPCLIENT-896.


Given the number of invalid cookies w/this issue that I see during a 
crawl, would it make sense for the "best match" policy to select a more 
lenient Netscape format?


Or maybe add a "best match-lenient" policy that does this?

I haven't had to do much in the way of cookie processing in the past, so 
I'll confess up front that I'm ignorant about the potential issues that 
could arise from using a more lenient policy.


Thanks again,

-- Ken




Ken

I am somewhat reluctant to optimize HttpClient for just one particular 
use case, such as web crawling. Not only does the cookie in question 
violate the HTTP state management standards, it also violates the 
Netscape Draft spec. I do not think HttpClient should accept such 
cookies as valid per default. At the same time it is really easy to 
override the default behavior with just one parameter.


Cheers

Oleg


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Calling servlet help (HttpClient 4.0)

[Charles François Rey]

Well, the idea is to try to set the "Accept" header of the request, so that the 
media type requested is compatible with what the servlets are configured to 
serve:

HttpGet httpget = new HttpGet("...");
httpget.setRequestHeader("Accept", "text/xml");

If it still doesn't work, try to tweak the media type, i.e. "application/xml", 
...

You can learn more about the Accept header in RFC 2616, section 14.1.

Hope this helps.

On 4 déc. 2009, at 03:59, sc...@seanet.com wrote:

> Can you expand on your answer a bit?  The accept header in the servlet or
> some API call (setRequest...() - what value?) in the http client?  The
> servlets set to return text/xml BTW.
> 
> I've also tried it using straight Java HttpUrlConnection, get same result.
> 
> Thanks for the help
> Steve
> 
>> Have you tried to set the "Accept" header of your request to the proper
>> media type ? Without it, your server might consider that you expect the
>> default, e.g. text/html, and won't serve you the xml. That's my guess.
>> 
>> On 2 déc. 2009, at 04:25, sc...@seanet.com wrote:
>> 
>>> Hi,
>>> 
>>> Please excuse a newbie question but how do I get back the XML content
>>> back
>>> from a servlet.  All I seem to get is the index.html web page..
>>> 
>>> Basic call is:
>>> 
>>> HttpClient httpclient = new DefaultHttpClient();
>>> HttpGet httpget = new
>>>  HttpGet("http://localhost:8080/wc_server?task=get_results";);
>>> HttpResponse response = httpclient.execute(httpget);
>>> 
>>> HttpEntity entity = response.getEntity();
>>> 
>>> if (entity != null)
>>> {
>>>   long len = entity.getContentLength();
>>>   if (len != -1 ) // && len < 8192)
>>>   {
>>>   System.out.println(EntityUtils.toString(entity));
>>>   }
>>> }
>>> 
>>> 
>>> Whats returned is the index.html page in that servlets top directory,
>>> not
>>> the XML data for that method call.  I get the same results with either:
>>> 
>>> HttpGet("http://localhost:8080/wc_server?task=get_results";);
>>> 
>>> or
>>> 
>>> HttpGet("http://localhost:8080/wc_server";);
>>> 
>>> It seems like it's not getting/processing the '?task=get_results'
>>> 
>>> If I remove the index.html file from the servlet directory it just
>>> errors
>>> back...
>>> 
>>> Thanks
>>> Steve Cook
>>> sc...@seanet.com

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Calling servlet help (HttpClient 4.0)

[scook]

Can you expand on your answer a bit?  The accept header in the servlet or
some API call (setRequest...() - what value?) in the http client?  The
servlets set to return text/xml BTW.

I've also tried it using straight Java HttpUrlConnection, get same result.

Thanks for the help
Steve

> Have you tried to set the "Accept" header of your request to the proper
> media type ? Without it, your server might consider that you expect the
> default, e.g. text/html, and won't serve you the xml. That's my guess.
>
> On 2 déc. 2009, at 04:25, sc...@seanet.com wrote:
>
>> Hi,
>>
>> Please excuse a newbie question but how do I get back the XML content
>> back
>> from a servlet.  All I seem to get is the index.html web page..
>>
>> Basic call is:
>>
>> HttpClient httpclient = new DefaultHttpClient();
>> HttpGet httpget = new
>>   HttpGet("http://localhost:8080/wc_server?task=get_results";);
>> HttpResponse response = httpclient.execute(httpget);
>>
>> HttpEntity entity = response.getEntity();
>>
>> if (entity != null)
>> {
>>long len = entity.getContentLength();
>>if (len != -1 ) // && len < 8192)
>>{
>>System.out.println(EntityUtils.toString(entity));
>>}
>> }
>>
>>
>> Whats returned is the index.html page in that servlets top directory,
>> not
>> the XML data for that method call.  I get the same results with either:
>>
>> HttpGet("http://localhost:8080/wc_server?task=get_results";);
>>
>> or
>>
>> HttpGet("http://localhost:8080/wc_server";);
>>
>> It seems like it's not getting/processing the '?task=get_results'
>>
>> If I remove the index.html file from the servlet directory it just
>> errors
>> back...
>>
>> Thanks
>> Steve Cook
>> sc...@seanet.com
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Setting cookie policy with HttpClient 4.0

[Ken Krugler]

Hi Oleg,

On Dec 3, 2009, at 2:40am, Oleg Kalnichevski wrote:


On Wed, 2009-12-02 at 19:15 -0800, Ken Krugler wrote:

Below is an email from August 7th, which I'm reviving due to this
becoming a bigger issue over in Bixo-land.

I've continued to run into this issue with my crawls, but so far I'm
not doing anything with cookies, so it hasn't been a priority to  
track

down.

However another Bixo user also runs into it, and he noticed that by
switching back to HttpClient 4.0-beta3, the warnings went away.

I believe he just opened HTTPCLIENT-896 as a clone of HTTPCLIENT-773,
which seemed to be this exact same bug (fixed by Oleg around 17/May/ 
08).


I'm wondering if the bug crept back into the code sometime between
then and the final release.

Thanks,

-- Ken



Hi Ken

The cookie in question violates the format of 'expires' attribute
expected by the Netscape policy. One can configure the policy to be  
more
lenient about the format of 'expires' attribute by using a special  
HTTP

parameter. For details see HTTPCLIENT-896.

It is not really a regression. I think the Netscape cookie policy was
made stricter at some point of time post 4.0-beta1

Hope this clarifies the situation.


Thanks for the clarification, and the example code you added in a  
comment to HTTPCLIENT-896.


Given the number of invalid cookies w/this issue that I see during a  
crawl, would it make sense for the "best match" policy to select a  
more lenient Netscape format?


Or maybe add a "best match-lenient" policy that does this?

I haven't had to do much in the way of cookie processing in the past,  
so I'll confess up front that I'm ignorant about the potential issues  
that could arise from using a more lenient policy.


Thanks again,

-- Ken


Ken Krugler
+1 530-210-6378
http://bixolabs.com
e l a s t i c   w e b   m i n i n g





-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Setting cookie policy with HttpClient 4.0

[Oleg Kalnichevski]

On Wed, 2009-12-02 at 19:15 -0800, Ken Krugler wrote:
> Below is an email from August 7th, which I'm reviving due to this  
> becoming a bigger issue over in Bixo-land.
> 
> I've continued to run into this issue with my crawls, but so far I'm  
> not doing anything with cookies, so it hasn't been a priority to track  
> down.
> 
> However another Bixo user also runs into it, and he noticed that by  
> switching back to HttpClient 4.0-beta3, the warnings went away.
> 
> I believe he just opened HTTPCLIENT-896 as a clone of HTTPCLIENT-773,  
> which seemed to be this exact same bug (fixed by Oleg around 17/May/08).
> 
> I'm wondering if the bug crept back into the code sometime between  
> then and the final release.
> 
> Thanks,
> 
> -- Ken
> 

Hi Ken

The cookie in question violates the format of 'expires' attribute
expected by the Netscape policy. One can configure the policy to be more
lenient about the format of 'expires' attribute by using a special HTTP
parameter. For details see HTTPCLIENT-896.

It is not really a regression. I think the Netscape cookie policy was
made stricter at some point of time post 4.0-beta1   

Hope this clarifies the situation.

Cheers,

Oleg


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Setting cookie policy with HttpClient 4.0

[Ken Krugler]

Below is an email from August 7th, which I'm reviving due to this  
becoming a bigger issue over in Bixo-land.


I've continued to run into this issue with my crawls, but so far I'm  
not doing anything with cookies, so it hasn't been a priority to track  
down.


However another Bixo user also runs into it, and he noticed that by  
switching back to HttpClient 4.0-beta3, the warnings went away.


I believe he just opened HTTPCLIENT-896 as a clone of HTTPCLIENT-773,  
which seemed to be this exact same bug (fixed by Oleg around 17/May/08).


I'm wondering if the bug crept back into the code sometime between  
then and the final release.


Thanks,

-- Ken

= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 
= 



Hi all,

I'm seeing errors in my logs relating to parsing the expires date  
value in a cookie:


09/08/07 10:26:46 WARN protocol.ResponseProcessCookies:137 - Invalid  
cookie header: "Set-Cookie: IU=deleted; expires=Thu, 07 Aug 2008  
17:26:45 GMT; path=/; domain=.yahoo.com". Unable to parse expires  
attribute: Thu, 07 Aug 2008 17:26:45 GMT


I looked through the HtttpClient source, and it seems like if I've set  
up my cookie policy properly, the above date would be parsed properly.


Here's now I'm setting up the cookie policy:

=
HttpParams params = new BasicHttpParams();
...
CookieSpecParamBean cookieParams = new CookieSpecParamBean(params);
cookieParams.setSingleHeader(true);
...
ClientConnectionManager cm = new ThreadSafeClientConnManager(params,  
schemeRegistry);

DefaultHttpClient httpClient = new DefaultHttpClient(cm, params);
...
params = httpClient.getParams();
HttpClientParams.setCookiePolicy(params, CookiePolicy.BEST_MATCH);
=

But the above code was assembled from a few different snippets online,  
IIRC. So maybe this isn't correct.


For example, in the "Choosing Cookie Policy" section of the tutorial  
docs, it uses the setParameter() API to set the policy:


httpclient.getParams().setParameter(ClientPNames.COOKIE_POLICY,  
CookiePolicy.RFC_2965);


I assume the two are equivalent, but any input would be appreciated.

Thanks,

-- Ken


<http://ken-blog.krugler.org>
+1 530-265-2225




--
Ken Krugler
TransPac Software, Inc.
<http://www.transpac.com>
+1 530-210-6378


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Calling servlet help (HttpClient 4.0)

[Charles François Rey]

Have you tried to set the "Accept" header of your request to the proper media 
type ? Without it, your server might consider that you expect the default, e.g. 
text/html, and won't serve you the xml. That's my guess.

On 2 déc. 2009, at 04:25, sc...@seanet.com wrote:

> Hi,
> 
> Please excuse a newbie question but how do I get back the XML content back
> from a servlet.  All I seem to get is the index.html web page..
> 
> Basic call is:
> 
> HttpClient httpclient = new DefaultHttpClient();
> HttpGet httpget = new
>   HttpGet("http://localhost:8080/wc_server?task=get_results";);
> HttpResponse response = httpclient.execute(httpget);
> 
> HttpEntity entity = response.getEntity();
> 
> if (entity != null)
> {
>long len = entity.getContentLength();
>if (len != -1 ) // && len < 8192)
>{
>System.out.println(EntityUtils.toString(entity));
>}
> }
> 
> 
> Whats returned is the index.html page in that servlets top directory, not
> the XML data for that method call.  I get the same results with either:
> 
> HttpGet("http://localhost:8080/wc_server?task=get_results";);
> 
> or
> 
> HttpGet("http://localhost:8080/wc_server";);
> 
> It seems like it's not getting/processing the '?task=get_results'
> 
> If I remove the index.html file from the servlet directory it just errors
> back...
> 
> Thanks
> Steve Cook
> sc...@seanet.com

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Calling servlet help (HttpClient 4.0)

[scook]

Hi,

Please excuse a newbie question but how do I get back the XML content back
from a servlet.  All I seem to get is the index.html web page..

Basic call is:

HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new
   HttpGet("http://localhost:8080/wc_server?task=get_results";);
HttpResponse response = httpclient.execute(httpget);

HttpEntity entity = response.getEntity();

if (entity != null)
{
long len = entity.getContentLength();
if (len != -1 ) // && len < 8192)
{
System.out.println(EntityUtils.toString(entity));
}
}


Whats returned is the index.html page in that servlets top directory, not
the XML data for that method call.  I get the same results with either:

HttpGet("http://localhost:8080/wc_server?task=get_results";);

or

HttpGet("http://localhost:8080/wc_server";);

It seems like it's not getting/processing the '?task=get_results'

If I remove the index.html file from the servlet directory it just errors
back...

Thanks
Steve Cook
sc...@seanet.com


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient 4.0. SocketTimeoutException and RST signal problem

[tompra]

Thank you for your help. I increse timeouts and connector is running fine
without problems. 

Best Regards,

Tom
-- 
View this message in context: 
http://old.nabble.com/HttpClient-4.0.-SocketTimeoutException-and-RST-signal-problem-tp25911599p26432212.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

RE: Sending SOAP request using HTTPPost (HttpClient 4.0)

[Brijesh Deo]

The best way to do this would be to try and simulate the steps using
HttpClient in the same way as a browser does.

If you have Mozilla Firefox, use the HttpFox plugin and see the sequence
of GET and POST request and responses when you interact with your web
server using the browser. Then try to follow the same/similar sequence
of GET and POSTs using the HttpClient and it should work.

I had a similar problem, where after the authentication (login) to the
server, there was an implicit POST from the browser using a javascript
code which happened without the user having to POST anything explicitly.
When I was trying to login using the HttpClient and then trying to do a
GET on something else, it was failing. Then I did a similar POST (the
implicit POST done by javascript) using HttpClient after the login POST
and then I was able to proceed successfully.

So, the problem is not necessarily with the HttpClient. It could be how
your server likes to be interacted withfind that out and do the same
using HttpClient and I am sure it will work. I have been using
HttpClient to perform quite complicated series of POSTs, GETs and
Redirects in my projectand it's been working fine and as expected.
But I have had to find the proper sequence by observing the browser
behavior first.

Hope this helps...


Thanks,
Brijesh


-Original Message-
From: Oleg Kalnichevski [mailto:ol...@apache.org] 
Sent: Tuesday, November 10, 2009 3:10 AM
To: HttpClient User Discussion
Subject: Re: Sending SOAP request using HTTPPost (HttpClient 4.0)

arpita dhundia wrote:
> I tried setting the expect 100 continue param false but the problem
still persists..
 >
> I am performimg very simple stpes, using NTLm authentication, creating
an HTTPPost object and attaching a StringEntity object to it, on which I
have written the xml data. 
> Wireshark shows that the post data is not in the right format, hence
the recieving server sends back an HTTP 400 error.
> I am not sure what I am doing wrong, or maybe what can I do to avoid
this error.
>

Doe not use Wireshark. Use wire log of HttpClient instead

http://hc.apache.org/httpcomponents-client/logging.html

Oleg


> Please help.
> 
> Thanks,
> Arpita
> 
> 
> 
> --- On Sat, 7/11/09, Thierry Boileau  wrote:
> 
> From: Thierry Boileau 
> Subject: Re: Sending SOAP request using HTTPPost (HttpClient 4.0)
> To: "HttpClient User Discussion" 
> Date: Saturday, 7 November, 2009, 5:03 PM
> 
> Hello,
> 
> you can find documentation here
>
http://hc.apache.org/httpcomponents-client/tutorial/pdf/httpclient-tutor
ial.pdf
> 
> best regards,
> Thierry Boileau
> 
>> Thanks for the reply Oleg, could you guide a little how can I do
that?
>>
>> Regards,
>> Arpita
>>
>> --- On Sat, 31/10/09, Oleg Kalnichevski  wrote:
>>
>> From: Oleg Kalnichevski 
>> Subject: Re: Sending SOAP request using HTTPPost (HttpClient 4.0)
>> To: "HttpClient User Discussion" 
>> Date: Saturday, 31 October, 2009, 4:40 AM
>>
>> arpita dhundia wrote:
>>> Hello Folks,
>>>
>>> I have run into a very odd problem.
>>> I have to send a SOAP (xml data over HTTP Post) request to another
server behind NTLM authentication. I am using HttpClient 4.0 for that.
>>> The NTLM authentication works fine but when I try to send the xml
data using StringEntity and set that Entity to the HTTPPpost Object, I
am returned 400 error code, meaning "bad request".
>>> I tried to see the exact request that HTTPClient is sending using
WireShark and indeed the format of the request is not correct. The
headers are repeated twice and the body too is not correctly placed
(unfortunately I cant point out the exact mistake, but comparing it with
a request that a soap client sends its different and seeming incorrect).
>>>
>>> Is anybody aware of any problems related to StringEntity? Or maybe
any other way I can send xml data using HttpPost method in HttpClient
4.0 API.
>>>
>>> Any help would be highly appreciated, I am behind my deadline on
this task :(.
>>>
>>> Thanks!!
>>>
>> Turn off 'expect-continue' handshaking.
>>
>> Oleg
>>
>> -
>> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>>
>>
>>
>>
>>  Add whatever you love to the Yahoo! India homepage. Try now!
http://in.yahoo.com/trynew
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 
> 
>

Re: Sending SOAP request using HTTPPost (HttpClient 4.0)

[Oleg Kalnichevski]

arpita dhundia wrote:

I tried setting the expect 100 continue param false but the problem still 
persists..

>
I am performimg very simple stpes, using NTLm authentication, creating an HTTPPost object and attaching a StringEntity object to it, on which I have written the xml data. 
Wireshark shows that the post data is not in the right format, hence the recieving server sends back an HTTP 400 error.

I am not sure what I am doing wrong, or maybe what can I do to avoid this error.



Doe not use Wireshark. Use wire log of HttpClient instead

http://hc.apache.org/httpcomponents-client/logging.html

Oleg



Please help.

Thanks,
Arpita



--- On Sat, 7/11/09, Thierry Boileau  wrote:

From: Thierry Boileau 
Subject: Re: Sending SOAP request using HTTPPost (HttpClient 4.0)
To: "HttpClient User Discussion" 
Date: Saturday, 7 November, 2009, 5:03 PM

Hello,

you can find documentation here
http://hc.apache.org/httpcomponents-client/tutorial/pdf/httpclient-tutorial.pdf

best regards,
Thierry Boileau


Thanks for the reply Oleg, could you guide a little how can I do that?

Regards,
Arpita

--- On Sat, 31/10/09, Oleg Kalnichevski  wrote:

From: Oleg Kalnichevski 
Subject: Re: Sending SOAP request using HTTPPost (HttpClient 4.0)
To: "HttpClient User Discussion" 
Date: Saturday, 31 October, 2009, 4:40 AM

arpita dhundia wrote:

Hello Folks,

I have run into a very odd problem.
I have to send a SOAP (xml data over HTTP Post) request to another server 
behind NTLM authentication. I am using HttpClient 4.0 for that.
The NTLM authentication works fine but when I try to send the xml data using StringEntity 
and set that Entity to the HTTPPpost Object, I am returned 400 error code, meaning 
"bad request".
I tried to see the exact request that HTTPClient is sending using WireShark and 
indeed the format of the request is not correct. The headers are repeated twice 
and the body too is not correctly placed (unfortunately I cant point out the 
exact mistake, but comparing it with a request that a soap client sends its 
different and seeming incorrect).

Is anybody aware of any problems related to StringEntity? Or maybe any other 
way I can send xml data using HttpPost method in HttpClient 4.0 API.

Any help would be highly appreciated, I am behind my deadline on this task :(.

Thanks!!


Turn off 'expect-continue' handshaking.

Oleg

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org




 Add whatever you love to the Yahoo! India homepage. Try now! 
http://in.yahoo.com/trynew


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org




  The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. 
http://in.yahoo.com/



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Sending SOAP request using HTTPPost (HttpClient 4.0)

[arpita dhundia]

I tried setting the expect 100 continue param false but the problem still 
persists..
I am performimg very simple stpes, using NTLm authentication, creating an 
HTTPPost object and attaching a StringEntity object to it, on which I have 
written the xml data. 
Wireshark shows that the post data is not in the right format, hence the 
recieving server sends back an HTTP 400 error.
I am not sure what I am doing wrong, or maybe what can I do to avoid this error.

Please help.

Thanks,
Arpita



--- On Sat, 7/11/09, Thierry Boileau  wrote:

From: Thierry Boileau 
Subject: Re: Sending SOAP request using HTTPPost (HttpClient 4.0)
To: "HttpClient User Discussion" 
Date: Saturday, 7 November, 2009, 5:03 PM

Hello,

you can find documentation here
http://hc.apache.org/httpcomponents-client/tutorial/pdf/httpclient-tutorial.pdf

best regards,
Thierry Boileau

> Thanks for the reply Oleg, could you guide a little how can I do that?
>
> Regards,
> Arpita
>
> --- On Sat, 31/10/09, Oleg Kalnichevski  wrote:
>
> From: Oleg Kalnichevski 
> Subject: Re: Sending SOAP request using HTTPPost (HttpClient 4.0)
> To: "HttpClient User Discussion" 
> Date: Saturday, 31 October, 2009, 4:40 AM
>
> arpita dhundia wrote:
>> Hello Folks,
>>
>> I have run into a very odd problem.
>> I have to send a SOAP (xml data over HTTP Post) request to another server 
>> behind NTLM authentication. I am using HttpClient 4.0 for that.
>> The NTLM authentication works fine but when I try to send the xml data using 
>> StringEntity and set that Entity to the HTTPPpost Object, I am returned 400 
>> error code, meaning "bad request".
>> I tried to see the exact request that HTTPClient is sending using WireShark 
>> and indeed the format of the request is not correct. The headers are 
>> repeated twice and the body too is not correctly placed (unfortunately I 
>> cant point out the exact mistake, but comparing it with a request that a 
>> soap client sends its different and seeming incorrect).
>>
>> Is anybody aware of any problems related to StringEntity? Or maybe any other 
>> way I can send xml data using HttpPost method in HttpClient 4.0 API.
>>
>> Any help would be highly appreciated, I am behind my deadline on this task 
>> :(.
>>
>> Thanks!!
>>
>
> Turn off 'expect-continue' handshaking.
>
> Oleg
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>
>
>
>      Add whatever you love to the Yahoo! India homepage. Try now! 
> http://in.yahoo.com/trynew

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org




  The INTERNET now has a personality. YOURS! See your Yahoo! Homepage. 
http://in.yahoo.com/

Re: Sending SOAP request using HTTPPost (HttpClient 4.0)

[Oleg Kalnichevski]

arpita dhundia wrote:

Thanks for the reply Oleg, could you guide a little how can I do that?

Regards,
Arpita



DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setBooleanParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, 
false);


Oleg


--- On Sat, 31/10/09, Oleg Kalnichevski  wrote:

From: Oleg Kalnichevski 
Subject: Re: Sending SOAP request using HTTPPost (HttpClient 4.0)
To: "HttpClient User Discussion" 
Date: Saturday, 31 October, 2009, 4:40 AM

arpita dhundia wrote:

Hello Folks,

I have run into a very odd problem.
I have to send a SOAP (xml data over HTTP Post) request to another server 
behind NTLM authentication. I am using HttpClient 4.0 for that.
The NTLM authentication works fine but when I try to send the xml data using StringEntity 
and set that Entity to the HTTPPpost Object, I am returned 400 error code, meaning 
"bad request".
I tried to see the exact request that HTTPClient is sending using WireShark and 
indeed the format of the request is not correct. The headers are repeated twice 
and the body too is not correctly placed (unfortunately I cant point out the 
exact mistake, but comparing it with a request that a soap client sends its 
different and seeming incorrect).

Is anybody aware of any problems related to StringEntity? Or maybe any other 
way I can send xml data using HttpPost method in HttpClient 4.0 API.

Any help would be highly appreciated, I am behind my deadline on this task :(.

Thanks!!



Turn off 'expect-continue' handshaking.

Oleg

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org




  Add whatever you love to the Yahoo! India homepage. Try now! 
http://in.yahoo.com/trynew



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Sending SOAP request using HTTPPost (HttpClient 4.0)

[Thierry Boileau]

Hello,

you can find documentation here
http://hc.apache.org/httpcomponents-client/tutorial/pdf/httpclient-tutorial.pdf

best regards,
Thierry Boileau

> Thanks for the reply Oleg, could you guide a little how can I do that?
>
> Regards,
> Arpita
>
> --- On Sat, 31/10/09, Oleg Kalnichevski  wrote:
>
> From: Oleg Kalnichevski 
> Subject: Re: Sending SOAP request using HTTPPost (HttpClient 4.0)
> To: "HttpClient User Discussion" 
> Date: Saturday, 31 October, 2009, 4:40 AM
>
> arpita dhundia wrote:
>> Hello Folks,
>>
>> I have run into a very odd problem.
>> I have to send a SOAP (xml data over HTTP Post) request to another server 
>> behind NTLM authentication. I am using HttpClient 4.0 for that.
>> The NTLM authentication works fine but when I try to send the xml data using 
>> StringEntity and set that Entity to the HTTPPpost Object, I am returned 400 
>> error code, meaning "bad request".
>> I tried to see the exact request that HTTPClient is sending using WireShark 
>> and indeed the format of the request is not correct. The headers are 
>> repeated twice and the body too is not correctly placed (unfortunately I 
>> cant point out the exact mistake, but comparing it with a request that a 
>> soap client sends its different and seeming incorrect).
>>
>> Is anybody aware of any problems related to StringEntity? Or maybe any other 
>> way I can send xml data using HttpPost method in HttpClient 4.0 API.
>>
>> Any help would be highly appreciated, I am behind my deadline on this task 
>> :(.
>>
>> Thanks!!
>>
>
> Turn off 'expect-continue' handshaking.
>
> Oleg
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>
>
>
>      Add whatever you love to the Yahoo! India homepage. Try now! 
> http://in.yahoo.com/trynew

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Sending SOAP request using HTTPPost (HttpClient 4.0)

[Oleg Kalnichevski]

arpita dhundia wrote:

Hello Folks,

I have run into a very odd problem.
I have to send a SOAP (xml data over HTTP Post) request to another server 
behind NTLM authentication. I am using HttpClient 4.0 for that.
The NTLM authentication works fine but when I try to send the xml data using StringEntity 
and set that Entity to the HTTPPpost Object, I am returned 400 error code, meaning 
"bad request".
I tried to see the exact request that HTTPClient is sending using WireShark and 
indeed the format of the request is not correct. The headers are repeated twice 
and the body too is not correctly placed (unfortunately I cant point out the 
exact mistake, but comparing it with a request that a soap client sends its 
different and seeming incorrect).

Is anybody aware of any problems related to StringEntity? Or maybe any other 
way I can send xml data using HttpPost method in HttpClient 4.0 API.

Any help would be highly appreciated, I am behind my deadline on this task :(.

Thanks!!



Turn off 'expect-continue' handshaking.

Oleg

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Sending SOAP request using HTTPPost (HttpClient 4.0)

[arpita dhundia]

Hello Folks,

I have run into a very odd problem.
I have to send a SOAP (xml data over HTTP Post) request to another server 
behind NTLM authentication. I am using HttpClient 4.0 for that.
The NTLM authentication works fine but when I try to send the xml data using 
StringEntity and set that Entity to the HTTPPpost Object, I am returned 400 
error code, meaning "bad request".
I tried to see the exact request that HTTPClient is sending using WireShark and 
indeed the format of the request is not correct. The headers are repeated twice 
and the body too is not correctly placed (unfortunately I cant point out the 
exact mistake, but comparing it with a request that a soap client sends its 
different and seeming incorrect).

Is anybody aware of any problems related to StringEntity? Or maybe any other 
way I can send xml data using HttpPost method in HttpClient 4.0 API.

Any help would be highly appreciated, I am behind my deadline on this task :(.

Thanks!!

PS:- For better understanding, here is a snippet of the code:-

DefaultHttpClient client = new DefaultHttpClient();
        client.getAuthSchemes().register("ntlm", new NTLMSchemeFactory());
        client.getCredentialsProvider().setCredentials(
                new AuthScope("host", port, "domain"),
                new NTCredentials("username", "password", "domain",
                        "domain2"));

        HttpPost httpget = new HttpPost(
                "http://ip/ServerAPI/ServerCommandService.asmx";);

        try {
            String soapRequest = ""
                    + "http://www.w3.org/2001/XMLSchema-instance\"; "
                    + "xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"; "
                    + "xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\"";
                    + ""
                    + "http://videoos.net/2/XProtectCSServerCommand\";>"
                    + 
"6a716afc-708e-479f-8b42-f97338d794f9"
                    + "" + "" + ""
                    + "";


            httpget.addHeader("SOAPAction",
                    "\"http://videoos.net/2/XProtectCSServerCommand/Login\"";);

            StringEntity strent= new StringEntity(soapRequest);
            strent.setContentType("text/xml; charset=utf-8");
            httpget.setEntity(strent);

            ResponseHandler response = new BasicResponseHandler();
            String responseBody = client.execute(httpget, response);
            System.out.println(responseBody);
}catch(IOException ex)
{
ex.printStackTrace();
}



  Now, send attachments up to 25MB with Yahoo! India Mail. Learn how. 
http://in.overview.mail.yahoo.com/photos

Re: HttpClient 4.0 httppost with parameters

[Oleg Kalnichevski]

Tobias Frei wrote:

Hello together,
i am a new subsciber to this mailing list and i hope that anyone can help.

I am trying to simulate a login to a webpage via a httppost-request. When i 
send the post, i cannot see the pair parameter/value in the post-data section 
of my http sniffer. When i try to send the information via a webbrowser, i can 
see the right data in the sniffer. Strangely in the header of the request, i 
see a content-length of 23. Where are the parameters, what is wrong with my 
request?

Here is the relevant part of my code:

public void sendLoginInformation(){

HttpPost httppost=new HttpPost(ggserverurl+phpexec);
HttpResponse response;
HttpEntity entity;
BufferedReader bufferedreader;
List  parameters = new ArrayList 
();
parameters.add(new BasicNameValuePair("a", "a"));
parameters.add(new BasicNameValuePair("b", "b"));
parameters.add(new BasicNameValuePair("c", "c"));
parameters.add(new BasicNameValuePair("d", "e"));
parameters.add(new BasicNameValuePair("e", "f"));
parameters.add(new BasicNameValuePair("f", "g"));
try {
UrlEncodedFormEntity sendentity = new 
UrlEncodedFormEntity(parameters, HTTP.UTF_8);
httppost.setEntity(sendentity);
response=httpclient.execute(httppost);
entity=response.getEntity();
/**if (entity!=null){
bufferedreader=new BufferedReader(new 
InputStreamReader (new BufferedInputStream(entity.getContent(;
int i;

for(i=0;i<1000;i++)System.out.println(bufferedreader.readLine());
}**/
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

Help would be appreciated.



You can use wire logging to see what gets sent across the wire

http://hc.apache.org/httpcomponents-client/logging.html

Oleg



Thanks

Tobias



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient 4.0 httppost with parameters

[Charles François Rey]

I've tried your example and I do see the parameters in the analyzer (I  
use WireShark). Keep in mind that the actual http request and the  
content can be split across multiple frames, so be sure to examine the  
reassembled data, not below the TCP level.


The content-length of 23 corresponds effectively to your 6 key+values  
with '=' and 5 '&' in between.


On 27 oct. 09, at 13:17, Tobias Frei wrote:


Hello together,
i am a new subsciber to this mailing list and i hope that anyone can  
help.


I am trying to simulate a login to a webpage via a httppost-request.  
When i send the post, i cannot see the pair parameter/value in the  
post-data section of my http sniffer. When i try to send the  
information via a webbrowser, i can see the right data in the  
sniffer. Strangely in the header of the request, i see a content- 
length of 23. Where are the parameters, what is wrong with my request?


Here is the relevant part of my code:

public void sendLoginInformation(){

HttpPost httppost=new HttpPost(ggserverurl+phpexec);
HttpResponse response;
HttpEntity entity;
BufferedReader bufferedreader;
List  parameters = new ArrayList 
();
parameters.add(new BasicNameValuePair("a", "a"));
parameters.add(new BasicNameValuePair("b", "b"));
parameters.add(new BasicNameValuePair("c", "c"));
parameters.add(new BasicNameValuePair("d", "e"));
parameters.add(new BasicNameValuePair("e", "f"));
parameters.add(new BasicNameValuePair("f", "g"));
try {
			UrlEncodedFormEntity sendentity = new  
UrlEncodedFormEntity(parameters, HTTP.UTF_8);

httppost.setEntity(sendentity);
response=httpclient.execute(httppost);
entity=response.getEntity();
/**if (entity!=null){
bufferedreader=new BufferedReader(new InputStreamReader (new  
BufferedInputStream(entity.getContent(;

int i;

for(i=0;i<1000;i++)System.out.println(bufferedreader.readLine());
}**/
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

Help would be appreciated.

Thanks

Tobias


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

HttpClient 4.0 httppost with parameters

[Tobias Frei]

Hello together,
i am a new subsciber to this mailing list and i hope that anyone can help.

I am trying to simulate a login to a webpage via a httppost-request. When i 
send the post, i cannot see the pair parameter/value in the post-data section 
of my http sniffer. When i try to send the information via a webbrowser, i can 
see the right data in the sniffer. Strangely in the header of the request, i 
see a content-length of 23. Where are the parameters, what is wrong with my 
request?

Here is the relevant part of my code:

public void sendLoginInformation(){

HttpPost httppost=new HttpPost(ggserverurl+phpexec);
HttpResponse response;
HttpEntity entity;
BufferedReader bufferedreader;
List  parameters = new ArrayList 
();
parameters.add(new BasicNameValuePair("a", "a"));
parameters.add(new BasicNameValuePair("b", "b"));
parameters.add(new BasicNameValuePair("c", "c"));
parameters.add(new BasicNameValuePair("d", "e"));
parameters.add(new BasicNameValuePair("e", "f"));
parameters.add(new BasicNameValuePair("f", "g"));
try {
UrlEncodedFormEntity sendentity = new 
UrlEncodedFormEntity(parameters, HTTP.UTF_8);
httppost.setEntity(sendentity);
response=httpclient.execute(httppost);
entity=response.getEntity();
/**if (entity!=null){
bufferedreader=new BufferedReader(new 
InputStreamReader (new BufferedInputStream(entity.getContent(;
int i;

for(i=0;i<1000;i++)System.out.println(bufferedreader.readLine());
}**/
} catch (ClientProtocolException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

Help would be appreciated.

Thanks

Tobias
-- 
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

-- 
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: BasicAuthentication using HttpClient 4.0

[jamesrusso]

Sorry for the noise. I was using "HttpClient" as my client (holding an
instance of DefaultHttpClient), but I now realize that I need
DefaultHttpClient to expose the setCredentialProvider method.

thanks,

-jr


jamesrusso wrote:
> 
> Hello, 
> 
> This does not seem to be a correct example for 4.0? Can anyone provide an
> example of the proper way to do basic authentication in 4.0?
> 
> thanks,
> 
> -jr
> 
> 
> olegk wrote:
>> 
>> 
>> While this certainly does the trick for BASIC authentication, more 
>> complex schemes will not work that way.
>> 
>> This is the recommended way of handling authentication with HttpClient
>> 4.0:
>> 
>> http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientAuthentication.java
>> http://hc.apache.org/httpcomponents-client/tutorial/html/authentication.html
>> 
>> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/BasicAuthentication-using-HttpClient-4.0-tp25578027p25982392.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: BasicAuthentication using HttpClient 4.0

[jamesrusso]

Hello, 

This does not seem to be a correct example for 4.0? Can anyone provide an
example of the proper way to do basic authentication in 4.0?

thanks,

-jr


olegk wrote:
> 
> 
> While this certainly does the trick for BASIC authentication, more 
> complex schemes will not work that way.
> 
> This is the recommended way of handling authentication with HttpClient
> 4.0:
> 
> http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientAuthentication.java
> http://hc.apache.org/httpcomponents-client/tutorial/html/authentication.html
> 
> 

-- 
View this message in context: 
http://www.nabble.com/BasicAuthentication-using-HttpClient-4.0-tp25578027p25982351.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient 4.0. SocketTimeoutException and RST signal problem

[Oleg Kalnichevski]

tompra wrote:

Hi,

I have problem with RST signal from instance of HttpClient that invokes a
SocketTimeoutException: Read Timeout.



This is because HttpClient automatically closes connections that time 
out on an I/O operation.



In an attachement there is an image of network communication between my
client and server, where client invokes RST. 


http://www.nabble.com/file/p25911599/SocketTimeoutException1.jpg
SocketTimeoutException1.jpg 


My client makes a lot of correct requests but sometimes there is a problem
with RST. Sometimes comes many SocketTimeoutException in the same time and
than we haven't problem several next hours. Can the server be a reason of
this problem? 


Yes, absolutely. Most likely this is a server related issue.



The exception is:

class java.net.SocketTimeoutException:
 Read timed out
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.read(SocketInputStream.java:129)
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:130)
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:127)
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:233)
org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:210)
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:271)
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:227)
org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:229)
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:292)
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:126)
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)

My code is:

HttpParams params = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(params, 3);
HttpConnectionParams.setSoTimeout(params, 3);

HttpRequestRetryHandler retryHandler = new DefaultHttpRequestRetryHandler(3,
true);

DefaultHttpClient httpClient = new DefaultHttpClient(params);
httpClient.setHttpRequestRetryHandler(retryHandler);

HttpGet httpGet = new HttpGet(URIUtil.encodeQuery(targetUrl));

HttpResponse response = httpClient.execute(httpGet);


HttpEntity entity = response.getEntity();
if (entity == null) {
throw new 
}

String content = EntityUtils.toString(entity);

Can you help me and show me the way? Thank you.



Maybe 30 second timeout is just too small. Try setting it to a greater 
number (3 minutes, for instance).


Oleg



Best regards,

Tom





-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient 4.0. SocketTimeoutException and RST signal problem

[tompra]

Anyone?

Do you have any idea? Where I made a mistake?

Tom


tompra wrote:
> 
> Hi,
> 
> I have problem with RST signal from instance of HttpClient that invokes a
> SocketTimeoutException: Read Timeout.
> 
> 

-- 
View this message in context: 
http://www.nabble.com/HttpClient-4.0.-SocketTimeoutException-and-RST-signal-problem-tp25911599p25925311.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

HttpClient 4.0. SocketTimeoutException and RST signal problem

[tompra]

Hi,

I have problem with RST signal from instance of HttpClient that invokes a
SocketTimeoutException: Read Timeout.

In an attachement there is an image of network communication between my
client and server, where client invokes RST. 

http://www.nabble.com/file/p25911599/SocketTimeoutException1.jpg
SocketTimeoutException1.jpg 

My client makes a lot of correct requests but sometimes there is a problem
with RST. Sometimes comes many SocketTimeoutException in the same time and
than we haven't problem several next hours. Can the server be a reason of
this problem? 

The exception is:

class java.net.SocketTimeoutException:
 Read timed out
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.read(SocketInputStream.java:129)
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:130)
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:127)
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:233)
org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:210)
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:271)
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:227)
org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:229)
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:292)
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:126)
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:576)
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:554)

My code is:

HttpParams params = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(params, 3);
HttpConnectionParams.setSoTimeout(params, 3);

HttpRequestRetryHandler retryHandler = new DefaultHttpRequestRetryHandler(3,
true);

DefaultHttpClient httpClient = new DefaultHttpClient(params);
httpClient.setHttpRequestRetryHandler(retryHandler);

HttpGet httpGet = new HttpGet(URIUtil.encodeQuery(targetUrl));

HttpResponse response = httpClient.execute(httpGet);

HttpEntity entity = response.getEntity();
if (entity == null) {
throw new 
}

String content = EntityUtils.toString(entity);

Can you help me and show me the way? Thank you.

Best regards,

Tom
-- 
View this message in context: 
http://www.nabble.com/HttpClient-4.0.-SocketTimeoutException-and-RST-signal-problem-tp25911599p25911599.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: HttpClient 4.0

[Joan Balagueró Valls]

--Mensaje original--
De: visualize
Para:httpclient-users@hc.apache.org
Responder a:HttpClient User Discussion
Asunto: Re: HttpClient 4.0
Enviado: 28 Sep, 2009 11:06



olegk wrote:
> 
> visualize wrote:
>> Hello,
>> 
>> I'm trying to port my application to use HttpClient 4.0 instead of
>> version
>> 3.0 I make a substantial number of HTTP-gets and used to use
>> aMultiThreadedHttpConnectionManager to setup a pool and retrieve clients. 
>> 
>> Let's say I want to get some stuff from
>> "http://my.server.com:8080/my-rest-service/data"; and
>> "http://my.server.com:8080/my-rest-service/data2";.
>> 
>> In 3.0 you are able to set host and port via
>> getHostConfiguration().setHost(host, port) which was kind of 
>> convenient since I then could just create the pool, and when getting
>> clients
>> just set the host and port to "http://my.server.com"; and 8080. In each
>> request, I just had to set the URL to "/my-rest-service/data" or
>> "/my-rest-service/data2" and didn't have to care about the host and port.
>> 
>> Is there a way to accomplish the same with a client of version 4.0? Like
>> set
>> a default host/route (or whatever you may call it), so when doing new
>> HttpGet("/my-rest-service/data") it "falls back" to the default one?
>> 
>> / K
>> 
>> 
> 
> 
> Use "http.default-host" parameter.
> 
> -
> DefaultHttpClient httpclient = new DefaultHttpClient();
> 
> httpclient.getParams().setParameter(ClientPNames.DEFAULT_HOST,
>  new HttpHost("www.google.com", 80, "http"));
> 
> HttpGet req = new HttpGet("/");
> 
> HttpResponse rsp = httpclient.execute(req);
> HttpEntity entity = rsp.getEntity();
> 
> System.out.println("");
> System.out.println(rsp.getStatusLine());
> System.out.println("");
> 
> if (entity != null) {
>  System.out.println(EntityUtils.toString(entity));
> }
> -
> 
> Hope this helps
> 
> Oleg
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 
> 
> 

Fantastic, thanks a lot!
-- 
View this message in context: 
http://www.nabble.com/HttpClient-4.0-tp25531034p25641698.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org



Enviado desde mi dispositivo BlackBerry® de Orange.

Re: HttpClient 4.0

[visualize]

olegk wrote:
> 
> visualize wrote:
>> Hello,
>> 
>> I'm trying to port my application to use HttpClient 4.0 instead of
>> version
>> 3.0 I make a substantial number of HTTP-gets and used to use
>> aMultiThreadedHttpConnectionManager to setup a pool and retrieve clients. 
>> 
>> Let's say I want to get some stuff from
>> "http://my.server.com:8080/my-rest-service/data"; and
>> "http://my.server.com:8080/my-rest-service/data2";.
>> 
>> In 3.0 you are able to set host and port via
>> getHostConfiguration().setHost(host, port) which was kind of 
>> convenient since I then could just create the pool, and when getting
>> clients
>> just set the host and port to "http://my.server.com"; and 8080. In each
>> request, I just had to set the URL to "/my-rest-service/data" or
>> "/my-rest-service/data2" and didn't have to care about the host and port.
>> 
>> Is there a way to accomplish the same with a client of version 4.0? Like
>> set
>> a default host/route (or whatever you may call it), so when doing new
>> HttpGet("/my-rest-service/data") it "falls back" to the default one?
>> 
>> / K
>> 
>> 
> 
> 
> Use "http.default-host" parameter.
> 
> -
> DefaultHttpClient httpclient = new DefaultHttpClient();
> 
> httpclient.getParams().setParameter(ClientPNames.DEFAULT_HOST,
>  new HttpHost("www.google.com", 80, "http"));
> 
> HttpGet req = new HttpGet("/");
> 
> HttpResponse rsp = httpclient.execute(req);
> HttpEntity entity = rsp.getEntity();
> 
> System.out.println("");
> System.out.println(rsp.getStatusLine());
> System.out.println("");
> 
> if (entity != null) {
>  System.out.println(EntityUtils.toString(entity));
> }
> -
> 
> Hope this helps
> 
> Oleg
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 
> 
> 

Fantastic, thanks a lot!
-- 
View this message in context: 
http://www.nabble.com/HttpClient-4.0-tp25531034p25641698.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: BasicAuthentication using HttpClient 4.0

[Oleg Kalnichevski]

kash_meu wrote:

Hi Tobi,

Thanks for your help. The link you told is useful for HttpClient 3.1 but
there is a lot of difference between 4.0 and 3.1.

I solved my problem. This is the solution.

HttpGet httpGet = new HttpGet(url);
UsernamePasswordCredentials credentials = new
UsernamePasswordCredentials(username, password);
BasicScheme scheme = new BasicScheme();
Header authorizationHeader = scheme.authenticate(credentials,
httpRequest);

httpGet.addHeader(authorizationHeader);



Regards,
Kashif




While this certainly does the trick for BASIC authentication, more 
complex schemes will not work that way.


This is the recommended way of handling authentication with HttpClient 4.0:

http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientAuthentication.java
http://hc.apache.org/httpcomponents-client/tutorial/html/authentication.html

Oleg



Tobias N. Sasse wrote:

kash_meu schrieb:

Thanks. I am not getting any error messages. But the output is not right.
When I paste http://abcdef.4y3...@hostname.com/abc.php in the browser, I
get
the right output. But on running the client, the output is not what I am
expectin
Hi, I have no idea of authentication myself, but you should probably 
have a look at this tutorial:


http://hc.apache.org/httpclient-3.x/authentication.html

I know this is for version-3 but perhaps you get an idea how it could 
work. I guess the problem is that providing the username and password in 
the URL string is not the way httpclient is triggert to provide 
authentication to the server?


Regards,
Tobi


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org








-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: BasicAuthentication using HttpClient 4.0

[kash_meu]

Hi Tobi,

Thanks for your help. The link you told is useful for HttpClient 3.1 but
there is a lot of difference between 4.0 and 3.1.

I solved my problem. This is the solution.

HttpGet httpGet = new HttpGet(url);
UsernamePasswordCredentials credentials = new
UsernamePasswordCredentials(username, password);
BasicScheme scheme = new BasicScheme();
Header authorizationHeader = scheme.authenticate(credentials,
httpRequest);

httpGet.addHeader(authorizationHeader);


Regards,
Kashif



Tobias N. Sasse wrote:
> 
> kash_meu schrieb:
>> Thanks. I am not getting any error messages. But the output is not right.
>> When I paste http://abcdef.4y3...@hostname.com/abc.php in the browser, I
>> get
>> the right output. But on running the client, the output is not what I am
>> expectin
> 
> Hi, I have no idea of authentication myself, but you should probably 
> have a look at this tutorial:
> 
> http://hc.apache.org/httpclient-3.x/authentication.html
> 
> I know this is for version-3 but perhaps you get an idea how it could 
> work. I guess the problem is that providing the username and password in 
> the URL string is not the way httpclient is triggert to provide 
> authentication to the server?
> 
> Regards,
> Tobi
> 
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/BasicAuthentication-using-HttpClient-4.0-tp25578027p25624006.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: BasicAuthentication using HttpClient 4.0

[Tobias N. Sasse]

kash_meu schrieb:

Thanks. I am not getting any error messages. But the output is not right.
When I paste http://abcdef.4y3...@hostname.com/abc.php in the browser, I get
the right output. But on running the client, the output is not what I am
expectin


Hi, I have no idea of authentication myself, but you should probably 
have a look at this tutorial:


http://hc.apache.org/httpclient-3.x/authentication.html

I know this is for version-3 but perhaps you get an idea how it could 
work. I guess the problem is that providing the username and password in 
the URL string is not the way httpclient is triggert to provide 
authentication to the server?


Regards,
   Tobi


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: BasicAuthentication using HttpClient 4.0

[kash_meu]

Tobias N. Sasse wrote:
> 
> You should provide an error message..
> 

Thanks. I am not getting any error messages. But the output is not right.
When I paste http://abcdef.4y3...@hostname.com/abc.php in the browser, I get
the right output. But on running the client, the output is not what I am
expecting


-- 
View this message in context: 
http://www.nabble.com/BasicAuthentication-using-HttpClient-4.0-tp25578027p25605235.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: BasicAuthentication using HttpClient 4.0

[Tobias N. Sasse]

kash_meu wrote:

Hi,

I am trying to send a GET request to
http://abcdef.4y3...@hostname.com/abc.php I am not sure how to send request
to this url. I am trying like this.

HttpGet httpGet = new HttpGet("http://abcdef.4y3...@hostname.com/abc.php";);

But this does not seem to be working. Any ideas?

Regards,
Kashif
  

You should provide an error message..

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

BasicAuthentication using HttpClient 4.0

[kash_meu]

Hi,

I am trying to send a GET request to
http://abcdef.4y3...@hostname.com/abc.php I am not sure how to send request
to this url. I am trying like this.

HttpGet httpGet = new HttpGet("http://abcdef.4y3...@hostname.com/abc.php";);

But this does not seem to be working. Any ideas?

Regards,
Kashif
-- 
View this message in context: 
http://www.nabble.com/BasicAuthentication-using-HttpClient-4.0-tp25578027p25578027.html
Sent from the HttpClient-User mailing list archive at Nabble.com.

Re: HttpClient 4.0

[Oleg Kalnichevski]

visualize wrote:

Hello,

I'm trying to port my application to use HttpClient 4.0 instead of version
3.0 I make a substantial number of HTTP-gets and used to use
aMultiThreadedHttpConnectionManager to setup a pool and retrieve clients. 


Let's say I want to get some stuff from
"http://my.server.com:8080/my-rest-service/data"; and
"http://my.server.com:8080/my-rest-service/data2";.

In 3.0 you are able to set host and port via
getHostConfiguration().setHost(host, port) which was kind of 
convenient since I then could just create the pool, and when getting clients

just set the host and port to "http://my.server.com"; and 8080. In each
request, I just had to set the URL to "/my-rest-service/data" or
"/my-rest-service/data2" and didn't have to care about the host and port.

Is there a way to accomplish the same with a client of version 4.0? Like set
a default host/route (or whatever you may call it), so when doing new
HttpGet("/my-rest-service/data") it "falls back" to the default one?

/ K





Use "http.default-host" parameter.

-
DefaultHttpClient httpclient = new DefaultHttpClient();

httpclient.getParams().setParameter(ClientPNames.DEFAULT_HOST,
new HttpHost("www.google.com", 80, "http"));

HttpGet req = new HttpGet("/");

HttpResponse rsp = httpclient.execute(req);
HttpEntity entity = rsp.getEntity();

System.out.println("");
System.out.println(rsp.getStatusLine());
System.out.println("");

if (entity != null) {
System.out.println(EntityUtils.toString(entity));
}
-

Hope this helps

Oleg

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

HttpClient 4.0

[visualize]

Hello,

I'm trying to port my application to use HttpClient 4.0 instead of version
3.0 I make a substantial number of HTTP-gets and used to use
aMultiThreadedHttpConnectionManager to setup a pool and retrieve clients. 

Let's say I want to get some stuff from
"http://my.server.com:8080/my-rest-service/data"; and
"http://my.server.com:8080/my-rest-service/data2";.

In 3.0 you are able to set host and port via
getHostConfiguration().setHost(host, port) which was kind of 
convenient since I then could just create the pool, and when getting clients
just set the host and port to "http://my.server.com"; and 8080. In each
request, I just had to set the URL to "/my-rest-service/data" or
"/my-rest-service/data2" and didn't have to care about the host and port.

Is there a way to accomplish the same with a client of version 4.0? Like set
a default host/route (or whatever you may call it), so when doing new
HttpGet("/my-rest-service/data") it "falls back" to the default one?

/ K


-- 
View this message in context: 
http://www.nabble.com/HttpClient-4.0-tp25531034p25531034.html
Sent from the HttpClient-User mailing list archive at Nabble.com.


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Httpclient 4.0 vs 3.1

[Roger Studner]

Oleg:

When I did the new Scheme("https", new Easy etc etc);

I got some error about the constructor being bad

But, I imagine that has something to do with httpclient 3.1 and 4.0  
both being on the classpath


Roger

On Sep 6, 2009, at 12:12 PM, Oleg Kalnichevski wrote:


Roger Studner wrote:
I was wondering if anyone has ever discovered how to "redo" the 3.1  
SSL example that went (very tersely) as follows:
   Protocol easyhttps = new Protocol("https", new  
EasySSLProtocolSocketFactory(), 443);

   Protocol.registerProtocol("https", easyhttps);
I tried this:
//SchemeRegistry schemeRegistry = new SchemeRegistry();
//// http scheme
//schemeRegistry.register( new Scheme( "http",  
PlainSocketFactory.getSocketFactory(), 80 ) );

//// https scheme
//schemeRegistry.register( new Scheme( "https", new  
EasySSLProtocolSocketFactory(), 443 ) );


Try this:

httpclient.getConnectionManager().getSchemeRegistry().register(
 new Scheme("https",
   new EasySSLProtocolSocketFactory(), 443 ));


But that example failed to take note that you can't even use that  
constructor with Scheme :)


Constructor of what?

Oleg

I'm currently using 2 libraries.. one of which depends on  
httpclient 3.1 and the other on httpclient 4.0 hah.
So if I can find a way to remove the use of 3.1 it'd be great!  my  
current major sticking point is just telling httpclient 4.0 "go  
ahead and accept any SSL from anything, I dont care :)"

Thanks,
Roger
-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org




-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Httpclient 4.0 vs 3.1

[Oleg Kalnichevski]

Roger Studner wrote:
I was wondering if anyone has ever discovered how to "redo" the 3.1 SSL 
example that went (very tersely) as follows:


Protocol easyhttps = new Protocol("https", new 
EasySSLProtocolSocketFactory(), 443);

Protocol.registerProtocol("https", easyhttps);

I tried this:
//SchemeRegistry schemeRegistry = new SchemeRegistry();
//// http scheme
//schemeRegistry.register( new Scheme( "http", 
PlainSocketFactory.getSocketFactory(), 80 ) );

//// https scheme
//schemeRegistry.register( new Scheme( "https", new 
EasySSLProtocolSocketFactory(), 443 ) );




Try this:

httpclient.getConnectionManager().getSchemeRegistry().register(
  new Scheme("https",
new EasySSLProtocolSocketFactory(), 443 ));


But that example failed to take note that you can't even use that 
constructor with Scheme :)




Constructor of what?

Oleg

I'm currently using 2 libraries.. one of which depends on httpclient 3.1 
and the other on httpclient 4.0 hah.


So if I can find a way to remove the use of 3.1 it'd be great!  my 
current major sticking point is just telling httpclient 4.0 "go ahead 
and accept any SSL from anything, I dont care :)"


Thanks,
Roger


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org




-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Httpclient 4.0 vs 3.1

[Roger Studner]

I was wondering if anyone has ever discovered how to "redo" the 3.1  
SSL example that went (very tersely) as follows:


Protocol easyhttps = new Protocol("https", new  
EasySSLProtocolSocketFactory(), 443);

Protocol.registerProtocol("https", easyhttps);

I tried this:
//SchemeRegistry schemeRegistry = new SchemeRegistry();
//// http scheme
//schemeRegistry.register( new Scheme( "http",  
PlainSocketFactory.getSocketFactory(), 80 ) );

//// https scheme
//schemeRegistry.register( new Scheme( "https", new  
EasySSLProtocolSocketFactory(), 443 ) );


But that example failed to take note that you can't even use that  
constructor with Scheme :)


I'm currently using 2 libraries.. one of which depends on httpclient  
3.1 and the other on httpclient 4.0 hah.


So if I can find a way to remove the use of 3.1 it'd be great!  my  
current major sticking point is just telling httpclient 4.0 "go ahead  
and accept any SSL from anything, I dont care :)"


Thanks,
Roger


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: [ANNOUNCEMENT] HttpComponents HttpClient 4.0 (GA) release

[Ken Krugler]

I should have responded earlier, but...

Congratulations to Oleg for getting this out the door.

And thanks again for all of the hard work. I've been using HttpClient  
4.0 for heavy-duty crawling, and it's been great!


-- Ken


On Aug 13, 2009, at 5:26am, Oleg Kalnichevski wrote:

The HttpComponents project is pleased to announce the first stable  
(GA)

release of Apache HttpClient 4.0

HttpClient 4.0 represents a complete, ground-up redesign and almost a
complete rewrite of the HttpClient 3.x codeline. This release finally
addresses several design flaws that existed since the 1.0 release and
could not be fixed without a major code overhaul and breaking API
compatibility.


Architectural changes
-

* Redesign of the HttpClient internals addressing all known major
architectural shortcomings of the 3.x codeline.

* Cleaner, more flexible and expressive API.

* More modular structure.

* Better performance and smaller memory footprint due to a more
efficient HTTP transport based on HttpCore.

* Implementation of cross-cutting HTTP protocol aspects through  
protocol

interceptors.

* Improved connection management, better handling of persistent
connections, support for stateful connections

* Pluggable redirect and authentication handlers.

* Improved support for sending requests via a proxy or a chain of
proxies

* More flexible SSL context customization

* Reduced intermediate garbage in the process of generating HTTP
requests and parsing HTTP responses

All upstream projects are strongly encouraged to upgrade.


Important notes
---

* Future releases of HttpMime module may be binary incompatible with
this release due to possible API changes in Apache Mime4J. Apache  
Mime4J

is still being actively developed and its API is considered unstable.

* HttpClient 4.0 is not fully binary compatible with 4.0 BETA1  
release.

Some protected variables in connection management class have been made
final in order to help ensure their thread safety:

org.apache.http.conn.BasicEofSensorWatcher#attemptReuse
org.apache.http.conn.BasicEofSensorWatcher#managedConn
org 
.apache.http.impl.conn.DefaultClientConnectionOperator#schemeRegistry

org.apache.http.impl.conn.DefaultHttpRoutePlanner#schemeRegistry
org.apache.http.impl.conn.ProxySelectorRoutePlanner#schemeRegistry
org.apache.http.impl.conn.SingleClientConnManager#alwaysShutDown
org.apache.http.impl.conn.SingleClientConnManager#connOperator
org.apache.http.impl.conn.SingleClientConnManager#schemeRegistry
org 
.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager#connOperator
org 
.apache 
.http.impl.conn.tsccm.ThreadSafeClientConnManager#schemeRegistry


---
Download -
<http://hc.apache.org/downloads.cgi>

Release notes -
<http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt 
>


HttpComponents site -
<http://hc.apache.org/>

Please note HttpClient 4.0 currently provides only limited support for
NTLM authentication. For details please refer to
<http://hc.apache.org/httpcomponents-client/ntlm.html>
---

About Apache HttpClient

Although the java.net package provides basic functionality for  
accessing

resources via HTTP, it doesn't provide the full flexibility or
functionality needed by many applications. HttpClient seeks to fill  
this

void by providing an efficient, up-to-date, and feature-rich package
implementing the client side of the most recent HTTP standards and
recommendations.

Designed for extension while providing robust support for the base  
HTTP

protocol, HttpClient may be of interest to anyone building HTTP-aware
client applications such as web browsers, web service clients, or
systems that leverage or extend the HTTP protocol for distributed
communication.

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org



--
Ken Krugler
TransPac Software, Inc.
<http://www.transpac.com>
+1 530-210-6378


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

[ANNOUNCEMENT] HttpComponents HttpClient 4.0 (GA) release

[Oleg Kalnichevski]

The HttpComponents project is pleased to announce the first stable (GA)
release of Apache HttpClient 4.0

HttpClient 4.0 represents a complete, ground-up redesign and almost a
complete rewrite of the HttpClient 3.x codeline. This release finally
addresses several design flaws that existed since the 1.0 release and
could not be fixed without a major code overhaul and breaking API
compatibility.


Architectural changes
-

* Redesign of the HttpClient internals addressing all known major
architectural shortcomings of the 3.x codeline.

* Cleaner, more flexible and expressive API.

* More modular structure.

* Better performance and smaller memory footprint due to a more
efficient HTTP transport based on HttpCore. 

* Implementation of cross-cutting HTTP protocol aspects through protocol
interceptors.

* Improved connection management, better handling of persistent
connections, support for stateful connections

* Pluggable redirect and authentication handlers.   

* Improved support for sending requests via a proxy or a chain of
proxies 

* More flexible SSL context customization 

* Reduced intermediate garbage in the process of generating HTTP
requests and parsing HTTP responses 

All upstream projects are strongly encouraged to upgrade.


Important notes
---

* Future releases of HttpMime module may be binary incompatible with
this release due to possible API changes in Apache Mime4J. Apache Mime4J
is still being actively developed and its API is considered unstable.
  
* HttpClient 4.0 is not fully binary compatible with 4.0 BETA1 release.
Some protected variables in connection management class have been made
final in order to help ensure their thread safety: 

org.apache.http.conn.BasicEofSensorWatcher#attemptReuse
org.apache.http.conn.BasicEofSensorWatcher#managedConn
org.apache.http.impl.conn.DefaultClientConnectionOperator#schemeRegistry
org.apache.http.impl.conn.DefaultHttpRoutePlanner#schemeRegistry
org.apache.http.impl.conn.ProxySelectorRoutePlanner#schemeRegistry
org.apache.http.impl.conn.SingleClientConnManager#alwaysShutDown
org.apache.http.impl.conn.SingleClientConnManager#connOperator
org.apache.http.impl.conn.SingleClientConnManager#schemeRegistry
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager#connOperator
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager#schemeRegistry

---
Download -
<http://hc.apache.org/downloads.cgi>

Release notes -
<http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt>

HttpComponents site -
<http://hc.apache.org/>

Please note HttpClient 4.0 currently provides only limited support for
NTLM authentication. For details please refer to
<http://hc.apache.org/httpcomponents-client/ntlm.html>
---

About Apache HttpClient

Although the java.net package provides basic functionality for accessing
resources via HTTP, it doesn't provide the full flexibility or
functionality needed by many applications. HttpClient seeks to fill this
void by providing an efficient, up-to-date, and feature-rich package
implementing the client side of the most recent HTTP standards and
recommendations.

Designed for extension while providing robust support for the base HTTP
protocol, HttpClient may be of interest to anyone building HTTP-aware
client applications such as web browsers, web service clients, or
systems that leverage or extend the HTTP protocol for distributed
communication.

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: Setting cookie policy with HttpClient 4.0

[Oleg Kalnichevski]

On Fri, Aug 07, 2009 at 10:32:32AM -0700, Ken Krugler wrote:
> Hi all,
>
> I'm seeing errors in my logs relating to parsing the expires date value 
> in a cookie:
>
> 09/08/07 10:26:46 WARN protocol.ResponseProcessCookies:137 - Invalid  
> cookie header: "Set-Cookie: IU=deleted; expires=Thu, 07 Aug 2008  
> 17:26:45 GMT; path=/; domain=.yahoo.com". Unable to parse expires  
> attribute: Thu, 07 Aug 2008 17:26:45 GMT
>
> I looked through the HtttpClient source, and it seems like if I've set  
> up my cookie policy properly, the above date would be parsed properly.
>
> Here's now I'm setting up the cookie policy:
>
> =
> HttpParams params = new BasicHttpParams();
> ...
> CookieSpecParamBean cookieParams = new CookieSpecParamBean(params);
> cookieParams.setSingleHeader(true);
> ...
> ClientConnectionManager cm = new ThreadSafeClientConnManager(params,  
> schemeRegistry);
> DefaultHttpClient httpClient = new DefaultHttpClient(cm, params);
> ...
> params = httpClient.getParams();
> HttpClientParams.setCookiePolicy(params, CookiePolicy.BEST_MATCH);
> =
>
> But the above code was assembled from a few different snippets online,  
> IIRC. So maybe this isn't correct.
>

Hi Ken

The code looks ok to me. In this particular case, though, you should simply be
using the 'http.protocol.cookie-datepatterns' parameter to override the default
expiry format.

http://hc.apache.org/httpcomponents-client/httpclient/apidocs/org/apache/http/cookie/params/CookieSpecPNames.html#DATE_PATTERNS


> For example, in the "Choosing Cookie Policy" section of the tutorial  
> docs, it uses the setParameter() API to set the policy:
>
> httpclient.getParams().setParameter(ClientPNames.COOKIE_POLICY,  
> CookiePolicy.RFC_2965);
>
> I assume the two are equivalent, but any input would be appreciated.
>

Yes, they are.

Hope this helps

Oleg

> Thanks,
>
> -- Ken
>
> 
> 
> +1 530-265-2225
>
>
>
>
> --
> Ken Krugler
> TransPac Software, Inc.
> 
> +1 530-210-6378
>

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Setting cookie policy with HttpClient 4.0

[Ken Krugler]

Hi all,

I'm seeing errors in my logs relating to parsing the expires date  
value in a cookie:


09/08/07 10:26:46 WARN protocol.ResponseProcessCookies:137 - Invalid  
cookie header: "Set-Cookie: IU=deleted; expires=Thu, 07 Aug 2008  
17:26:45 GMT; path=/; domain=.yahoo.com". Unable to parse expires  
attribute: Thu, 07 Aug 2008 17:26:45 GMT


I looked through the HtttpClient source, and it seems like if I've set  
up my cookie policy properly, the above date would be parsed properly.


Here's now I'm setting up the cookie policy:

=
HttpParams params = new BasicHttpParams();
...
CookieSpecParamBean cookieParams = new CookieSpecParamBean(params);
cookieParams.setSingleHeader(true);
...
ClientConnectionManager cm = new ThreadSafeClientConnManager(params,  
schemeRegistry);

DefaultHttpClient httpClient = new DefaultHttpClient(cm, params);
...
params = httpClient.getParams();
HttpClientParams.setCookiePolicy(params, CookiePolicy.BEST_MATCH);
=

But the above code was assembled from a few different snippets online,  
IIRC. So maybe this isn't correct.


For example, in the "Choosing Cookie Policy" section of the tutorial  
docs, it uses the setParameter() API to set the policy:


httpclient.getParams().setParameter(ClientPNames.COOKIE_POLICY,  
CookiePolicy.RFC_2965);


I assume the two are equivalent, but any input would be appreciated.

Thanks,

-- Ken



+1 530-265-2225




--
Ken Krugler
TransPac Software, Inc.

+1 530-210-6378

Re: [HttpClient] HttpClient 4.0-rc2

[Oleg Kalnichevski]

sebb wrote:

On 24/07/2009, Oleg Kalnichevski  wrote:

On Fri, Jul 24, 2009 at 03:01:34PM +0100, sebb wrote:
 > On 24/07/2009, Oleg Kalnichevski  wrote:
 > > On Fri, Jul 24, 2009 at 01:59:23PM +0100, sebb wrote:
 > >  > On 24/07/2009, Oleg Kalnichevski  wrote:
 > >  > > On Fri, Jul 24, 2009 at 01:07:35AM +0100, sebb wrote:
 > >  > >  > Pressed send too soon:
 > >  > >  >
 > >  > >  > SSLSocketFactory is not inherently thread-safe, because of the
 > >  > >  > [gs]etHostNameVerifier() methods.
 > >  > >  > Is there a need to change the HostNameVerifier after construction, 
or
 > >  > >  > could the verifier be provided to the ctor? Alternatively, perhaps 
the
 > >  > >  > field could be made volatile?
 > >  > >  >
 > >  > >  >
 > >  > >
 > >  > >
 > >  > > I see no good reason for changing hostname verifier after 
construction. Feel free to make the variable final and deprecate the setter.
 > >  > >
 > >  >
 > >  > Huh? There cannot be a setter if the field is final ...
 > >  >
 > >
 > >
 > > The setter does not necessarily have to mutate any variable. It can simply 
be
 > >  left empty. Feel free to throw a runtime exception
 > >  (UnsupportedOperationException for instance) if that makes you more
 > >  comfortable.
 > >
 >
 > OK, I see. Keep the setter method for API compatibility, but disable
 > (and deprecate) it.
 >
 > However, that would mean that existing code would only discover the
 > problem at run-time, whereas removing the method would cause the
 > change to be detected at compile-time.
 >
 > In both cases, client code will need to be changed.
 >
 > I would rather find out the problem at compile-time...
 >


Same here. In my opinion full binary compatibility without full behavioral
 compatibility is completely and utterly pointless. I rather have things break
 on me at compile time than at runtime. But for some reason behavioral
 incompatibility is often seen as less severe as binary incompatibility. There
 is no point in trying to be a better catholic than Pope of Rome.


OK, but what is the way forward?



Feel free to make any changes you deem necessary. Anything that helps us 
get the release rather sooner than later is OK with me.




Is it OK to break binary compatibility here?



I _personally_ would consider making the field final OK (given the fact 
we have already made a few similar changes) but removing the public 
method less so.


Oleg



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: [HttpClient] HttpClient 4.0-rc2

[sebb]

On 24/07/2009, Oleg Kalnichevski  wrote:
> On Fri, Jul 24, 2009 at 03:01:34PM +0100, sebb wrote:
>  > On 24/07/2009, Oleg Kalnichevski  wrote:
>  > > On Fri, Jul 24, 2009 at 01:59:23PM +0100, sebb wrote:
>  > >  > On 24/07/2009, Oleg Kalnichevski  wrote:
>  > >  > > On Fri, Jul 24, 2009 at 01:07:35AM +0100, sebb wrote:
>  > >  > >  > Pressed send too soon:
>  > >  > >  >
>  > >  > >  > SSLSocketFactory is not inherently thread-safe, because of the
>  > >  > >  > [gs]etHostNameVerifier() methods.
>  > >  > >  > Is there a need to change the HostNameVerifier after 
> construction, or
>  > >  > >  > could the verifier be provided to the ctor? Alternatively, 
> perhaps the
>  > >  > >  > field could be made volatile?
>  > >  > >  >
>  > >  > >  >
>  > >  > >
>  > >  > >
>  > >  > > I see no good reason for changing hostname verifier after 
> construction. Feel free to make the variable final and deprecate the setter.
>  > >  > >
>  > >  >
>  > >  > Huh? There cannot be a setter if the field is final ...
>  > >  >
>  > >
>  > >
>  > > The setter does not necessarily have to mutate any variable. It can 
> simply be
>  > >  left empty. Feel free to throw a runtime exception
>  > >  (UnsupportedOperationException for instance) if that makes you more
>  > >  comfortable.
>  > >
>  >
>  > OK, I see. Keep the setter method for API compatibility, but disable
>  > (and deprecate) it.
>  >
>  > However, that would mean that existing code would only discover the
>  > problem at run-time, whereas removing the method would cause the
>  > change to be detected at compile-time.
>  >
>  > In both cases, client code will need to be changed.
>  >
>  > I would rather find out the problem at compile-time...
>  >
>
>
> Same here. In my opinion full binary compatibility without full behavioral
>  compatibility is completely and utterly pointless. I rather have things break
>  on me at compile time than at runtime. But for some reason behavioral
>  incompatibility is often seen as less severe as binary incompatibility. There
>  is no point in trying to be a better catholic than Pope of Rome.

OK, but what is the way forward?

Is it OK to break binary compatibility here?

>
>  Oleg
>
>
>
>  > >  Oleg
>  > >
>  > >
>  > >
>  > >  > If the field is made final, the only solution is to add the verifier
>  > >  > to all the constructors.
>  > >  > This means adding some new constructors with the extra field.
>  > >  >
>  > >  > Which I am happy to do, but is a bit messy now
>  > >  >
>  > >  > ==
>  > >  >
>  > >  > If the class is constructed and the setHNV() method is called just
>  > >  > once, then the instance can be safely passed to a newly created
>  > >  > thread, because Thread.start() acts as a common synch. lock between
>  > >  > the two threads.
>  > >  >
>  > >  > However, if the instance is passed to an existing thread, there may be
>  > >  > no common synch. lock involved, and the value of any non-final fields
>  > >  > may not be published correctly.
>  > >  >
>  > >  > I'm not sure how the class is intended to be used, so I'm not sure if
>  > >  > this is a reasonable approach for the class.
>  > >  >
>  > >  > >  Oleg
>  > >  > >
>  > >  > >
>  > >  > >
>  > >  > >  >
>  > >  > >  > On 23/07/2009, sebb  wrote:
>  > >  > >  > > On 15/07/2009, Oleg Kalnichevski  wrote:
>  > >  > >  > >  > Folks
>  > >  > >  > >  >
>  > >  > >  > >  >  Please test your applications against 4.0-rc2 and report 
> bugs if found.
>  > >  > >  > >  >
>  > >  > >  > >  >  There have been three fixes since 4.0-rc1
>  > >  > >  > >  >
>  > >  > >  > >  >  * [HTTPCLIENT-860] HttpClient no longer converts redirects 
> of PUT/POST
>  > >  > >  > >  >   to GET for status codes 301, 302, 307, as required by the 
> HTTP spec.
>  > >  > >  > >  >
>  > >  > >  > >  >  * [HTTPCLIENT-859] CookieIdentityComparator now takes path 

Re: [HttpClient] HttpClient 4.0-rc2

[Oleg Kalnichevski]

On Fri, Jul 24, 2009 at 03:01:34PM +0100, sebb wrote:
> On 24/07/2009, Oleg Kalnichevski  wrote:
> > On Fri, Jul 24, 2009 at 01:59:23PM +0100, sebb wrote:
> >  > On 24/07/2009, Oleg Kalnichevski  wrote:
> >  > > On Fri, Jul 24, 2009 at 01:07:35AM +0100, sebb wrote:
> >  > >  > Pressed send too soon:
> >  > >  >
> >  > >  > SSLSocketFactory is not inherently thread-safe, because of the
> >  > >  > [gs]etHostNameVerifier() methods.
> >  > >  > Is there a need to change the HostNameVerifier after construction, 
> > or
> >  > >  > could the verifier be provided to the ctor? Alternatively, perhaps 
> > the
> >  > >  > field could be made volatile?
> >  > >  >
> >  > >  >
> >  > >
> >  > >
> >  > > I see no good reason for changing hostname verifier after 
> > construction. Feel free to make the variable final and deprecate the setter.
> >  > >
> >  >
> >  > Huh? There cannot be a setter if the field is final ...
> >  >
> >
> >
> > The setter does not necessarily have to mutate any variable. It can simply 
> > be
> >  left empty. Feel free to throw a runtime exception
> >  (UnsupportedOperationException for instance) if that makes you more
> >  comfortable.
> >
> 
> OK, I see. Keep the setter method for API compatibility, but disable
> (and deprecate) it.
> 
> However, that would mean that existing code would only discover the
> problem at run-time, whereas removing the method would cause the
> change to be detected at compile-time.
> 
> In both cases, client code will need to be changed.
> 
> I would rather find out the problem at compile-time...
> 

Same here. In my opinion full binary compatibility without full behavioral
compatibility is completely and utterly pointless. I rather have things break
on me at compile time than at runtime. But for some reason behavioral
incompatibility is often seen as less severe as binary incompatibility. There
is no point in trying to be a better catholic than Pope of Rome.

Oleg


> >  Oleg
> >
> >
> >
> >  > If the field is made final, the only solution is to add the verifier
> >  > to all the constructors.
> >  > This means adding some new constructors with the extra field.
> >  >
> >  > Which I am happy to do, but is a bit messy now
> >  >
> >  > ==
> >  >
> >  > If the class is constructed and the setHNV() method is called just
> >  > once, then the instance can be safely passed to a newly created
> >  > thread, because Thread.start() acts as a common synch. lock between
> >  > the two threads.
> >  >
> >  > However, if the instance is passed to an existing thread, there may be
> >  > no common synch. lock involved, and the value of any non-final fields
> >  > may not be published correctly.
> >  >
> >  > I'm not sure how the class is intended to be used, so I'm not sure if
> >  > this is a reasonable approach for the class.
> >  >
> >  > >  Oleg
> >  > >
> >  > >
> >  > >
> >  > >  >
> >  > >  > On 23/07/2009, sebb  wrote:
> >  > >  > > On 15/07/2009, Oleg Kalnichevski  wrote:
> >  > >  > >  > Folks
> >  > >  > >  >
> >  > >  > >  >  Please test your applications against 4.0-rc2 and report bugs 
> > if found.
> >  > >  > >  >
> >  > >  > >  >  There have been three fixes since 4.0-rc1
> >  > >  > >  >
> >  > >  > >  >  * [HTTPCLIENT-860] HttpClient no longer converts redirects of 
> > PUT/POST
> >  > >  > >  >   to GET for status codes 301, 302, 307, as required by the 
> > HTTP spec.
> >  > >  > >  >
> >  > >  > >  >  * [HTTPCLIENT-859] CookieIdentityComparator now takes path 
> > attribute
> >  > >  > >  >   into consideration when comparing cookies.
> >  > >  > >  >
> >  > >  > >  >  * HttpClient will no longer send expired cookies back to the 
> > origin
> >  > >  > >  >   server.
> >  > >  > >  >
> >  > >  > >  >  Please also find a few minutes to review the release packages.
> >  > >  > >  >
> >  > >  > >  >  Packages:
> >  > >  > >  >  http://people.apache.org/~olegk/httpclient-4.0-rc2/
> >  >

Re: [HttpClient] HttpClient 4.0-rc2

[sebb]

On 24/07/2009, Oleg Kalnichevski  wrote:
> On Fri, Jul 24, 2009 at 01:59:23PM +0100, sebb wrote:
>  > On 24/07/2009, Oleg Kalnichevski  wrote:
>  > > On Fri, Jul 24, 2009 at 01:07:35AM +0100, sebb wrote:
>  > >  > Pressed send too soon:
>  > >  >
>  > >  > SSLSocketFactory is not inherently thread-safe, because of the
>  > >  > [gs]etHostNameVerifier() methods.
>  > >  > Is there a need to change the HostNameVerifier after construction, or
>  > >  > could the verifier be provided to the ctor? Alternatively, perhaps the
>  > >  > field could be made volatile?
>  > >  >
>  > >  >
>  > >
>  > >
>  > > I see no good reason for changing hostname verifier after construction. 
> Feel free to make the variable final and deprecate the setter.
>  > >
>  >
>  > Huh? There cannot be a setter if the field is final ...
>  >
>
>
> The setter does not necessarily have to mutate any variable. It can simply be
>  left empty. Feel free to throw a runtime exception
>  (UnsupportedOperationException for instance) if that makes you more
>  comfortable.
>

OK, I see. Keep the setter method for API compatibility, but disable
(and deprecate) it.

However, that would mean that existing code would only discover the
problem at run-time, whereas removing the method would cause the
change to be detected at compile-time.

In both cases, client code will need to be changed.

I would rather find out the problem at compile-time...

>  Oleg
>
>
>
>  > If the field is made final, the only solution is to add the verifier
>  > to all the constructors.
>  > This means adding some new constructors with the extra field.
>  >
>  > Which I am happy to do, but is a bit messy now
>  >
>  > ==
>  >
>  > If the class is constructed and the setHNV() method is called just
>  > once, then the instance can be safely passed to a newly created
>  > thread, because Thread.start() acts as a common synch. lock between
>  > the two threads.
>  >
>  > However, if the instance is passed to an existing thread, there may be
>  > no common synch. lock involved, and the value of any non-final fields
>  > may not be published correctly.
>  >
>  > I'm not sure how the class is intended to be used, so I'm not sure if
>  > this is a reasonable approach for the class.
>  >
>  > >  Oleg
>  > >
>  > >
>  > >
>  > >  >
>  > >  > On 23/07/2009, sebb  wrote:
>  > >  > > On 15/07/2009, Oleg Kalnichevski  wrote:
>  > >  > >  > Folks
>  > >  > >  >
>  > >  > >  >  Please test your applications against 4.0-rc2 and report bugs 
> if found.
>  > >  > >  >
>  > >  > >  >  There have been three fixes since 4.0-rc1
>  > >  > >  >
>  > >  > >  >  * [HTTPCLIENT-860] HttpClient no longer converts redirects of 
> PUT/POST
>  > >  > >  >   to GET for status codes 301, 302, 307, as required by the HTTP 
> spec.
>  > >  > >  >
>  > >  > >  >  * [HTTPCLIENT-859] CookieIdentityComparator now takes path 
> attribute
>  > >  > >  >   into consideration when comparing cookies.
>  > >  > >  >
>  > >  > >  >  * HttpClient will no longer send expired cookies back to the 
> origin
>  > >  > >  >   server.
>  > >  > >  >
>  > >  > >  >  Please also find a few minutes to review the release packages.
>  > >  > >  >
>  > >  > >  >  Packages:
>  > >  > >  >  http://people.apache.org/~olegk/httpclient-4.0-rc2/
>  > >  > >
>  > >  > >
>  > >  > > I've started looking at thread-safety, beginning with
>  > >  > >  ThreadSafeClientConnManager.
>  > >  > >  That sems to be thread-safe, however one of the fields contains 
> ConnPoolByRoute.
>  > >  > >  This has non-final protected fields:
>  > >  > >  - freeConnections
>  > >  > >  - waitingThreads
>  > >  > >  which are created in the constructor; I think these should be made 
> final.
>  > >  > >
>  > >  > >
>  > >  > >  The Interface RefQueueHandler is marked as deprecated, and only 
> seems
>  > >  > >  to be used in AbstractConnPool and RefQueueWorker - could these be
>  > >  > >  deleted?
>  > >  > &g

Re: [HttpClient] HttpClient 4.0-rc2

[Oleg Kalnichevski]

On Fri, Jul 24, 2009 at 01:59:23PM +0100, sebb wrote:
> On 24/07/2009, Oleg Kalnichevski  wrote:
> > On Fri, Jul 24, 2009 at 01:07:35AM +0100, sebb wrote:
> >  > Pressed send too soon:
> >  >
> >  > SSLSocketFactory is not inherently thread-safe, because of the
> >  > [gs]etHostNameVerifier() methods.
> >  > Is there a need to change the HostNameVerifier after construction, or
> >  > could the verifier be provided to the ctor? Alternatively, perhaps the
> >  > field could be made volatile?
> >  >
> >  >
> >
> >
> > I see no good reason for changing hostname verifier after construction. 
> > Feel free to make the variable final and deprecate the setter.
> >
> 
> Huh? There cannot be a setter if the field is final ...
> 

The setter does not necessarily have to mutate any variable. It can simply be
left empty. Feel free to throw a runtime exception
(UnsupportedOperationException for instance) if that makes you more
comfortable.

Oleg


> If the field is made final, the only solution is to add the verifier
> to all the constructors.
> This means adding some new constructors with the extra field.
> 
> Which I am happy to do, but is a bit messy now
> 
> ==
> 
> If the class is constructed and the setHNV() method is called just
> once, then the instance can be safely passed to a newly created
> thread, because Thread.start() acts as a common synch. lock between
> the two threads.
> 
> However, if the instance is passed to an existing thread, there may be
> no common synch. lock involved, and the value of any non-final fields
> may not be published correctly.
> 
> I'm not sure how the class is intended to be used, so I'm not sure if
> this is a reasonable approach for the class.
> 
> >  Oleg
> >
> >
> >
> >  >
> >  > On 23/07/2009, sebb  wrote:
> >  > > On 15/07/2009, Oleg Kalnichevski  wrote:
> >  > >  > Folks
> >  > >  >
> >  > >  >  Please test your applications against 4.0-rc2 and report bugs if 
> > found.
> >  > >  >
> >  > >  >  There have been three fixes since 4.0-rc1
> >  > >  >
> >  > >  >  * [HTTPCLIENT-860] HttpClient no longer converts redirects of 
> > PUT/POST
> >  > >  >   to GET for status codes 301, 302, 307, as required by the HTTP 
> > spec.
> >  > >  >
> >  > >  >  * [HTTPCLIENT-859] CookieIdentityComparator now takes path 
> > attribute
> >  > >  >   into consideration when comparing cookies.
> >  > >  >
> >  > >  >  * HttpClient will no longer send expired cookies back to the origin
> >  > >  >   server.
> >  > >  >
> >  > >  >  Please also find a few minutes to review the release packages.
> >  > >  >
> >  > >  >  Packages:
> >  > >  >  http://people.apache.org/~olegk/httpclient-4.0-rc2/
> >  > >
> >  > >
> >  > > I've started looking at thread-safety, beginning with
> >  > >  ThreadSafeClientConnManager.
> >  > >  That sems to be thread-safe, however one of the fields contains 
> > ConnPoolByRoute.
> >  > >  This has non-final protected fields:
> >  > >  - freeConnections
> >  > >  - waitingThreads
> >  > >  which are created in the constructor; I think these should be made 
> > final.
> >  > >
> >  > >
> >  > >  The Interface RefQueueHandler is marked as deprecated, and only seems
> >  > >  to be used in AbstractConnPool and RefQueueWorker - could these be
> >  > >  deleted?
> >  > >
> >  > >
> >  > >  >  Release notes:
> >  > >  > 
> > http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/RELEASE_NOTES.txt
> >  > >  >
> >  > >  >  Oleg
> >  > >  >
> >  > >  > 
> > -
> >  > >  >  To unsubscribe, e-mail:
> >  > >  > httpclient-users-unsubscr...@hc.apache.org
> >  > >  >  For additional commands, e-mail:
> >  > >  > httpclient-users-h...@hc.apache.org
> >  > >  >
> >  > >  >
> >  > >
> >  >
> >  > -
> >  > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> >  > For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> >  >
> >
> >  -
> >  To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> >  For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> >
> >
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: [HttpClient] HttpClient 4.0-rc2

[sebb]

On 24/07/2009, Oleg Kalnichevski  wrote:
> On Fri, Jul 24, 2009 at 01:07:35AM +0100, sebb wrote:
>  > Pressed send too soon:
>  >
>  > SSLSocketFactory is not inherently thread-safe, because of the
>  > [gs]etHostNameVerifier() methods.
>  > Is there a need to change the HostNameVerifier after construction, or
>  > could the verifier be provided to the ctor? Alternatively, perhaps the
>  > field could be made volatile?
>  >
>  >
>
>
> I see no good reason for changing hostname verifier after construction. Feel 
> free to make the variable final and deprecate the setter.
>

Huh? There cannot be a setter if the field is final ...

If the field is made final, the only solution is to add the verifier
to all the constructors.
This means adding some new constructors with the extra field.

Which I am happy to do, but is a bit messy now

==

If the class is constructed and the setHNV() method is called just
once, then the instance can be safely passed to a newly created
thread, because Thread.start() acts as a common synch. lock between
the two threads.

However, if the instance is passed to an existing thread, there may be
no common synch. lock involved, and the value of any non-final fields
may not be published correctly.

I'm not sure how the class is intended to be used, so I'm not sure if
this is a reasonable approach for the class.

>  Oleg
>
>
>
>  >
>  > On 23/07/2009, sebb  wrote:
>  > > On 15/07/2009, Oleg Kalnichevski  wrote:
>  > >  > Folks
>  > >  >
>  > >  >  Please test your applications against 4.0-rc2 and report bugs if 
> found.
>  > >  >
>  > >  >  There have been three fixes since 4.0-rc1
>  > >  >
>  > >  >  * [HTTPCLIENT-860] HttpClient no longer converts redirects of 
> PUT/POST
>  > >  >   to GET for status codes 301, 302, 307, as required by the HTTP spec.
>  > >  >
>  > >  >  * [HTTPCLIENT-859] CookieIdentityComparator now takes path attribute
>  > >  >   into consideration when comparing cookies.
>  > >  >
>  > >  >  * HttpClient will no longer send expired cookies back to the origin
>  > >  >   server.
>  > >  >
>  > >  >  Please also find a few minutes to review the release packages.
>  > >  >
>  > >  >  Packages:
>  > >  >  http://people.apache.org/~olegk/httpclient-4.0-rc2/
>  > >
>  > >
>  > > I've started looking at thread-safety, beginning with
>  > >  ThreadSafeClientConnManager.
>  > >  That sems to be thread-safe, however one of the fields contains 
> ConnPoolByRoute.
>  > >  This has non-final protected fields:
>  > >  - freeConnections
>  > >  - waitingThreads
>  > >  which are created in the constructor; I think these should be made 
> final.
>  > >
>  > >
>  > >  The Interface RefQueueHandler is marked as deprecated, and only seems
>  > >  to be used in AbstractConnPool and RefQueueWorker - could these be
>  > >  deleted?
>  > >
>  > >
>  > >  >  Release notes:
>  > >  > 
> http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/RELEASE_NOTES.txt
>  > >  >
>  > >  >  Oleg
>  > >  >
>  > >  > -
>  > >  >  To unsubscribe, e-mail:
>  > >  > httpclient-users-unsubscr...@hc.apache.org
>  > >  >  For additional commands, e-mail:
>  > >  > httpclient-users-h...@hc.apache.org
>  > >  >
>  > >  >
>  > >
>  >
>  > -
>  > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>  > For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>  >
>
>  -
>  To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>  For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: [HttpClient] HttpClient 4.0-rc2

[Oleg Kalnichevski]

On Fri, Jul 24, 2009 at 01:07:35AM +0100, sebb wrote:
> Pressed send too soon:
> 
> SSLSocketFactory is not inherently thread-safe, because of the
> [gs]etHostNameVerifier() methods.
> Is there a need to change the HostNameVerifier after construction, or
> could the verifier be provided to the ctor? Alternatively, perhaps the
> field could be made volatile?
> 
> 

I see no good reason for changing hostname verifier after construction. Feel 
free to make the variable final and deprecate the setter.

Oleg


> 
> On 23/07/2009, sebb  wrote:
> > On 15/07/2009, Oleg Kalnichevski  wrote:
> >  > Folks
> >  >
> >  >  Please test your applications against 4.0-rc2 and report bugs if found.
> >  >
> >  >  There have been three fixes since 4.0-rc1
> >  >
> >  >  * [HTTPCLIENT-860] HttpClient no longer converts redirects of PUT/POST
> >  >   to GET for status codes 301, 302, 307, as required by the HTTP spec.
> >  >
> >  >  * [HTTPCLIENT-859] CookieIdentityComparator now takes path attribute
> >  >   into consideration when comparing cookies.
> >  >
> >  >  * HttpClient will no longer send expired cookies back to the origin
> >  >   server.
> >  >
> >  >  Please also find a few minutes to review the release packages.
> >  >
> >  >  Packages:
> >  >  http://people.apache.org/~olegk/httpclient-4.0-rc2/
> >
> >
> > I've started looking at thread-safety, beginning with
> >  ThreadSafeClientConnManager.
> >  That sems to be thread-safe, however one of the fields contains 
> > ConnPoolByRoute.
> >  This has non-final protected fields:
> >  - freeConnections
> >  - waitingThreads
> >  which are created in the constructor; I think these should be made final.
> >
> >
> >  The Interface RefQueueHandler is marked as deprecated, and only seems
> >  to be used in AbstractConnPool and RefQueueWorker - could these be
> >  deleted?
> >
> >
> >  >  Release notes:
> >  > 
> > http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/RELEASE_NOTES.txt
> >  >
> >  >  Oleg
> >  >
> >  > -
> >  >  To unsubscribe, e-mail:
> >  > httpclient-users-unsubscr...@hc.apache.org
> >  >  For additional commands, e-mail:
> >  > httpclient-users-h...@hc.apache.org
> >  >
> >  >
> >
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: [HttpClient] HttpClient 4.0-rc2

[Oleg Kalnichevski]

On Thu, Jul 23, 2009 at 11:53:36PM +0100, sebb wrote:
> On 15/07/2009, Oleg Kalnichevski  wrote:
> > Folks
> >
> >  Please test your applications against 4.0-rc2 and report bugs if found.
> >
> >  There have been three fixes since 4.0-rc1
> >
> >  * [HTTPCLIENT-860] HttpClient no longer converts redirects of PUT/POST
> >   to GET for status codes 301, 302, 307, as required by the HTTP spec.
> >
> >  * [HTTPCLIENT-859] CookieIdentityComparator now takes path attribute
> >   into consideration when comparing cookies.
> >
> >  * HttpClient will no longer send expired cookies back to the origin
> >   server.
> >
> >  Please also find a few minutes to review the release packages.
> >
> >  Packages:
> >  http://people.apache.org/~olegk/httpclient-4.0-rc2/
> 
> I've started looking at thread-safety, beginning with
> ThreadSafeClientConnManager.
> That sems to be thread-safe, however one of the fields contains 
> ConnPoolByRoute.
> This has non-final protected fields:
> - freeConnections
> - waitingThreads
> which are created in the constructor; I think these should be made final.
> 

+1

> 
> The Interface RefQueueHandler is marked as deprecated, and only seems
> to be used in AbstractConnPool and RefQueueWorker - could these be
> deleted?
> 

Let's keep them deprecated for a release or two. There is still a possibility
someone might want to look at fixing GC of unused connections.

Oleg

> >  Release notes:
> > http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/RELEASE_NOTES.txt
> >
> >  Oleg
> >
> > -
> >  To unsubscribe, e-mail:
> > httpclient-users-unsubscr...@hc.apache.org
> >  For additional commands, e-mail:
> > httpclient-users-h...@hc.apache.org
> >
> >
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: [HttpClient] HttpClient 4.0-rc2

[sebb]

Pressed send too soon:

SSLSocketFactory is not inherently thread-safe, because of the
[gs]etHostNameVerifier() methods.
Is there a need to change the HostNameVerifier after construction, or
could the verifier be provided to the ctor? Alternatively, perhaps the
field could be made volatile?



On 23/07/2009, sebb  wrote:
> On 15/07/2009, Oleg Kalnichevski  wrote:
>  > Folks
>  >
>  >  Please test your applications against 4.0-rc2 and report bugs if found.
>  >
>  >  There have been three fixes since 4.0-rc1
>  >
>  >  * [HTTPCLIENT-860] HttpClient no longer converts redirects of PUT/POST
>  >   to GET for status codes 301, 302, 307, as required by the HTTP spec.
>  >
>  >  * [HTTPCLIENT-859] CookieIdentityComparator now takes path attribute
>  >   into consideration when comparing cookies.
>  >
>  >  * HttpClient will no longer send expired cookies back to the origin
>  >   server.
>  >
>  >  Please also find a few minutes to review the release packages.
>  >
>  >  Packages:
>  >  http://people.apache.org/~olegk/httpclient-4.0-rc2/
>
>
> I've started looking at thread-safety, beginning with
>  ThreadSafeClientConnManager.
>  That sems to be thread-safe, however one of the fields contains 
> ConnPoolByRoute.
>  This has non-final protected fields:
>  - freeConnections
>  - waitingThreads
>  which are created in the constructor; I think these should be made final.
>
>
>  The Interface RefQueueHandler is marked as deprecated, and only seems
>  to be used in AbstractConnPool and RefQueueWorker - could these be
>  deleted?
>
>
>  >  Release notes:
>  > 
> http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/RELEASE_NOTES.txt
>  >
>  >  Oleg
>  >
>  > -
>  >  To unsubscribe, e-mail:
>  > httpclient-users-unsubscr...@hc.apache.org
>  >  For additional commands, e-mail:
>  > httpclient-users-h...@hc.apache.org
>  >
>  >
>

-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org