Re: Disabling retries
On Mon, Mar 11, 2013 at 01:59:03PM -0400, Karl Wright wrote: > So, in your opinion, the following is a valid HTTP response? And, > sorry, it's not Resin, it's Apache Web Server: > > >> > HTTP/1.1 100 Continue > HTTP/1.1 401 Unauthorized > Date: Mon, 11 Mar 2013 17:16:49 GMT > Server: Apache/2.2.22 (Unix) > WWW-Authenticate: Basic realm="resin" > Content-Length: 159 > Content-Type: text/html; charset=utf-8 > X-UA-Compatible: IE=EmulateIE7 > << > > FWIW, it also appears that this problem is acknowledged by the Apache > webserver team as being real, since at least 2007: > > http://www.gossamer-threads.com/lists/apache/users/340406 > > Karl > Assuming these messages are sent in response to an entity enclosing request containing 'Expect: 100-continue' header, yes, they are. The first response is utterly useless and misleading but correct from the protocol standpoint. Oleg > On Mon, Mar 11, 2013 at 1:48 PM, Oleg Kalnichevski wrote: > > On Mon, Mar 11, 2013 at 01:44:06PM -0400, Karl Wright wrote: > >> To finish up this thread, it seems that Resin is the problem. It > >> sends TWO status lines - a "100 Continue" AND a "401 Unauthorized" in > >> the same response in this situation. An obvious garden-variety bug, > >> which we'll report to the Resin people ASAP. > >> > >> Karl > >> > > > > Karl > > > > From the HTTP protocol standpoint this behavior is perfectly legal. > > Basically Resin chooses to not validate incoming entity enclosing requests > > early. This is not nice but not a bug. > > > > Oleg > > > > > >> On Mon, Mar 11, 2013 at 11:26 AM, Karl Wright wrote: > >> > Turns out that HttpClient is doing just fine, submitting the right > >> > headers etc. It's the server in this case that is saying "100 > >> > Continue" instead of "401 Unauthorized". And then when data is > >> > actually transmitted it responds with 401. Brilliant. > >> > > >> > The server is running on Resin, so it is possible there's a > >> > configuration problem, or improperly coded webapp. But in any case > >> > the fix looks correct. > >> > > >> > Thanks again! > >> > Karl > >> > > >> > On Mon, Mar 11, 2013 at 10:07 AM, Karl Wright wrote: > >> >> We tried enabling expect-continue but we're still getting the same > >> >> behavior. This is a bit of a surprise given the discussion so far. I > >> >> will try to get a full debugging dump to see what is going on. > >> >> > >> >> Karl > >> >> > >> >> On Fri, Mar 8, 2013 at 12:18 PM, Oleg Kalnichevski > >> >> wrote: > >> >>> On Fri, 2013-03-08 at 11:50 -0500, Karl Wright wrote: > >> Trying again on a reply - google seems to have deleted my previous > >> attempt. > >> > >> > This is what the 'expect-continue' handshake is for. It enables the > >> > client to verify server expectations prior to sending the request > >> > body. > >> > >> Is there a reason this isn't getting used? Is it gated by server > >> behavior, or is there a setting in HttpClient that allows it to work? > >> > >> >>> > >> >>> It is turned off by default due to compatibility issues with older > >> >>> (HTTP/1.0) proxies. > >> >>> > >> > HttpClient comes with a number of HttpEntity implementations > >> > including > >> > those backed by a byte array, a string or a file. Probably all you > >> > have > >> > to do is to use the right implementation. > >> > >> Problem is that ManifoldCF output connectors get an inputstream handed > >> to them, not a file. But I was asking this question to see if anyone > >> knew why a resettable input stream wouldn't work. Because, it doesn't > >> seem to. > >> > >> >>> > >> >>> I am not sure I understand how resettable input stream could help here. > >> >>> One would effectively need to buffer the entire content of the entity > >> >>> in > >> >>> memory in order to be able to reset from the very end of the stream to > >> >>> the very beginning. > >> >>> > >> >>> Oleg > >> >>> > >> >>> > >> >>> > >> >>> - > >> >>> To unsubscribe, e-mail: [email protected] > >> >>> For additional commands, e-mail: [email protected] > >> >>> > >> > >> - > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > > > > - > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > - > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: dev-h...@hc
Re: Disabling retries
So, in your opinion, the following is a valid HTTP response? And, sorry, it's not Resin, it's Apache Web Server: >> HTTP/1.1 100 Continue HTTP/1.1 401 Unauthorized Date: Mon, 11 Mar 2013 17:16:49 GMT Server: Apache/2.2.22 (Unix) WWW-Authenticate: Basic realm="resin" Content-Length: 159 Content-Type: text/html; charset=utf-8 X-UA-Compatible: IE=EmulateIE7 << FWIW, it also appears that this problem is acknowledged by the Apache webserver team as being real, since at least 2007: http://www.gossamer-threads.com/lists/apache/users/340406 Karl On Mon, Mar 11, 2013 at 1:48 PM, Oleg Kalnichevski wrote: > On Mon, Mar 11, 2013 at 01:44:06PM -0400, Karl Wright wrote: >> To finish up this thread, it seems that Resin is the problem. It >> sends TWO status lines - a "100 Continue" AND a "401 Unauthorized" in >> the same response in this situation. An obvious garden-variety bug, >> which we'll report to the Resin people ASAP. >> >> Karl >> > > Karl > > From the HTTP protocol standpoint this behavior is perfectly legal. Basically > Resin chooses to not validate incoming entity enclosing requests early. This > is not nice but not a bug. > > Oleg > > >> On Mon, Mar 11, 2013 at 11:26 AM, Karl Wright wrote: >> > Turns out that HttpClient is doing just fine, submitting the right >> > headers etc. It's the server in this case that is saying "100 >> > Continue" instead of "401 Unauthorized". And then when data is >> > actually transmitted it responds with 401. Brilliant. >> > >> > The server is running on Resin, so it is possible there's a >> > configuration problem, or improperly coded webapp. But in any case >> > the fix looks correct. >> > >> > Thanks again! >> > Karl >> > >> > On Mon, Mar 11, 2013 at 10:07 AM, Karl Wright wrote: >> >> We tried enabling expect-continue but we're still getting the same >> >> behavior. This is a bit of a surprise given the discussion so far. I >> >> will try to get a full debugging dump to see what is going on. >> >> >> >> Karl >> >> >> >> On Fri, Mar 8, 2013 at 12:18 PM, Oleg Kalnichevski >> >> wrote: >> >>> On Fri, 2013-03-08 at 11:50 -0500, Karl Wright wrote: >> Trying again on a reply - google seems to have deleted my previous >> attempt. >> >> > This is what the 'expect-continue' handshake is for. It enables the >> > client to verify server expectations prior to sending the request >> > body. >> >> Is there a reason this isn't getting used? Is it gated by server >> behavior, or is there a setting in HttpClient that allows it to work? >> >> >>> >> >>> It is turned off by default due to compatibility issues with older >> >>> (HTTP/1.0) proxies. >> >>> >> > HttpClient comes with a number of HttpEntity implementations including >> > those backed by a byte array, a string or a file. Probably all you >> > have >> > to do is to use the right implementation. >> >> Problem is that ManifoldCF output connectors get an inputstream handed >> to them, not a file. But I was asking this question to see if anyone >> knew why a resettable input stream wouldn't work. Because, it doesn't >> seem to. >> >> >>> >> >>> I am not sure I understand how resettable input stream could help here. >> >>> One would effectively need to buffer the entire content of the entity in >> >>> memory in order to be able to reset from the very end of the stream to >> >>> the very beginning. >> >>> >> >>> Oleg >> >>> >> >>> >> >>> >> >>> - >> >>> To unsubscribe, e-mail: [email protected] >> >>> For additional commands, e-mail: [email protected] >> >>> >> >> - >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > - > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: Disabling retries
On Mon, Mar 11, 2013 at 01:44:06PM -0400, Karl Wright wrote: > To finish up this thread, it seems that Resin is the problem. It > sends TWO status lines - a "100 Continue" AND a "401 Unauthorized" in > the same response in this situation. An obvious garden-variety bug, > which we'll report to the Resin people ASAP. > > Karl > Karl >From the HTTP protocol standpoint this behavior is perfectly legal. Basically >Resin chooses to not validate incoming entity enclosing requests early. This >is not nice but not a bug. Oleg > On Mon, Mar 11, 2013 at 11:26 AM, Karl Wright wrote: > > Turns out that HttpClient is doing just fine, submitting the right > > headers etc. It's the server in this case that is saying "100 > > Continue" instead of "401 Unauthorized". And then when data is > > actually transmitted it responds with 401. Brilliant. > > > > The server is running on Resin, so it is possible there's a > > configuration problem, or improperly coded webapp. But in any case > > the fix looks correct. > > > > Thanks again! > > Karl > > > > On Mon, Mar 11, 2013 at 10:07 AM, Karl Wright wrote: > >> We tried enabling expect-continue but we're still getting the same > >> behavior. This is a bit of a surprise given the discussion so far. I > >> will try to get a full debugging dump to see what is going on. > >> > >> Karl > >> > >> On Fri, Mar 8, 2013 at 12:18 PM, Oleg Kalnichevski > >> wrote: > >>> On Fri, 2013-03-08 at 11:50 -0500, Karl Wright wrote: > Trying again on a reply - google seems to have deleted my previous > attempt. > > > This is what the 'expect-continue' handshake is for. It enables the > > client to verify server expectations prior to sending the request body. > > Is there a reason this isn't getting used? Is it gated by server > behavior, or is there a setting in HttpClient that allows it to work? > > >>> > >>> It is turned off by default due to compatibility issues with older > >>> (HTTP/1.0) proxies. > >>> > > HttpClient comes with a number of HttpEntity implementations including > > those backed by a byte array, a string or a file. Probably all you have > > to do is to use the right implementation. > > Problem is that ManifoldCF output connectors get an inputstream handed > to them, not a file. But I was asking this question to see if anyone > knew why a resettable input stream wouldn't work. Because, it doesn't > seem to. > > >>> > >>> I am not sure I understand how resettable input stream could help here. > >>> One would effectively need to buffer the entire content of the entity in > >>> memory in order to be able to reset from the very end of the stream to > >>> the very beginning. > >>> > >>> Oleg > >>> > >>> > >>> > >>> - > >>> To unsubscribe, e-mail: [email protected] > >>> For additional commands, e-mail: [email protected] > >>> > > - > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: Disabling retries
To finish up this thread, it seems that Resin is the problem. It sends TWO status lines - a "100 Continue" AND a "401 Unauthorized" in the same response in this situation. An obvious garden-variety bug, which we'll report to the Resin people ASAP. Karl On Mon, Mar 11, 2013 at 11:26 AM, Karl Wright wrote: > Turns out that HttpClient is doing just fine, submitting the right > headers etc. It's the server in this case that is saying "100 > Continue" instead of "401 Unauthorized". And then when data is > actually transmitted it responds with 401. Brilliant. > > The server is running on Resin, so it is possible there's a > configuration problem, or improperly coded webapp. But in any case > the fix looks correct. > > Thanks again! > Karl > > On Mon, Mar 11, 2013 at 10:07 AM, Karl Wright wrote: >> We tried enabling expect-continue but we're still getting the same >> behavior. This is a bit of a surprise given the discussion so far. I >> will try to get a full debugging dump to see what is going on. >> >> Karl >> >> On Fri, Mar 8, 2013 at 12:18 PM, Oleg Kalnichevski wrote: >>> On Fri, 2013-03-08 at 11:50 -0500, Karl Wright wrote: Trying again on a reply - google seems to have deleted my previous attempt. > This is what the 'expect-continue' handshake is for. It enables the > client to verify server expectations prior to sending the request body. Is there a reason this isn't getting used? Is it gated by server behavior, or is there a setting in HttpClient that allows it to work? >>> >>> It is turned off by default due to compatibility issues with older >>> (HTTP/1.0) proxies. >>> > HttpClient comes with a number of HttpEntity implementations including > those backed by a byte array, a string or a file. Probably all you have > to do is to use the right implementation. Problem is that ManifoldCF output connectors get an inputstream handed to them, not a file. But I was asking this question to see if anyone knew why a resettable input stream wouldn't work. Because, it doesn't seem to. >>> >>> I am not sure I understand how resettable input stream could help here. >>> One would effectively need to buffer the entire content of the entity in >>> memory in order to be able to reset from the very end of the stream to >>> the very beginning. >>> >>> Oleg >>> >>> >>> >>> - >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: Disabling retries
Turns out that HttpClient is doing just fine, submitting the right headers etc. It's the server in this case that is saying "100 Continue" instead of "401 Unauthorized". And then when data is actually transmitted it responds with 401. Brilliant. The server is running on Resin, so it is possible there's a configuration problem, or improperly coded webapp. But in any case the fix looks correct. Thanks again! Karl On Mon, Mar 11, 2013 at 10:07 AM, Karl Wright wrote: > We tried enabling expect-continue but we're still getting the same > behavior. This is a bit of a surprise given the discussion so far. I > will try to get a full debugging dump to see what is going on. > > Karl > > On Fri, Mar 8, 2013 at 12:18 PM, Oleg Kalnichevski wrote: >> On Fri, 2013-03-08 at 11:50 -0500, Karl Wright wrote: >>> Trying again on a reply - google seems to have deleted my previous attempt. >>> >>> > This is what the 'expect-continue' handshake is for. It enables the >>> > client to verify server expectations prior to sending the request body. >>> >>> Is there a reason this isn't getting used? Is it gated by server >>> behavior, or is there a setting in HttpClient that allows it to work? >>> >> >> It is turned off by default due to compatibility issues with older >> (HTTP/1.0) proxies. >> >>> > HttpClient comes with a number of HttpEntity implementations including >>> > those backed by a byte array, a string or a file. Probably all you have >>> > to do is to use the right implementation. >>> >>> Problem is that ManifoldCF output connectors get an inputstream handed >>> to them, not a file. But I was asking this question to see if anyone >>> knew why a resettable input stream wouldn't work. Because, it doesn't >>> seem to. >>> >> >> I am not sure I understand how resettable input stream could help here. >> One would effectively need to buffer the entire content of the entity in >> memory in order to be able to reset from the very end of the stream to >> the very beginning. >> >> Oleg >> >> >> >> - >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: Disabling retries
We tried enabling expect-continue but we're still getting the same behavior. This is a bit of a surprise given the discussion so far. I will try to get a full debugging dump to see what is going on. Karl On Fri, Mar 8, 2013 at 12:18 PM, Oleg Kalnichevski wrote: > On Fri, 2013-03-08 at 11:50 -0500, Karl Wright wrote: >> Trying again on a reply - google seems to have deleted my previous attempt. >> >> > This is what the 'expect-continue' handshake is for. It enables the >> > client to verify server expectations prior to sending the request body. >> >> Is there a reason this isn't getting used? Is it gated by server >> behavior, or is there a setting in HttpClient that allows it to work? >> > > It is turned off by default due to compatibility issues with older > (HTTP/1.0) proxies. > >> > HttpClient comes with a number of HttpEntity implementations including >> > those backed by a byte array, a string or a file. Probably all you have >> > to do is to use the right implementation. >> >> Problem is that ManifoldCF output connectors get an inputstream handed >> to them, not a file. But I was asking this question to see if anyone >> knew why a resettable input stream wouldn't work. Because, it doesn't >> seem to. >> > > I am not sure I understand how resettable input stream could help here. > One would effectively need to buffer the entire content of the entity in > memory in order to be able to reset from the very end of the stream to > the very beginning. > > Oleg > > > > - > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: Disabling retries
On Fri, 2013-03-08 at 11:50 -0500, Karl Wright wrote: > Trying again on a reply - google seems to have deleted my previous attempt. > > > This is what the 'expect-continue' handshake is for. It enables the > > client to verify server expectations prior to sending the request body. > > Is there a reason this isn't getting used? Is it gated by server > behavior, or is there a setting in HttpClient that allows it to work? > It is turned off by default due to compatibility issues with older (HTTP/1.0) proxies. > > HttpClient comes with a number of HttpEntity implementations including > > those backed by a byte array, a string or a file. Probably all you have > > to do is to use the right implementation. > > Problem is that ManifoldCF output connectors get an inputstream handed > to them, not a file. But I was asking this question to see if anyone > knew why a resettable input stream wouldn't work. Because, it doesn't > seem to. > I am not sure I understand how resettable input stream could help here. One would effectively need to buffer the entire content of the entity in memory in order to be able to reset from the very end of the stream to the very beginning. Oleg - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: Disabling retries
Trying again on a reply - google seems to have deleted my previous attempt. > This is what the 'expect-continue' handshake is for. It enables the > client to verify server expectations prior to sending the request body. Is there a reason this isn't getting used? Is it gated by server behavior, or is there a setting in HttpClient that allows it to work? > HttpClient comes with a number of HttpEntity implementations including > those backed by a byte array, a string or a file. Probably all you have > to do is to use the right implementation. Problem is that ManifoldCF output connectors get an inputstream handed to them, not a file. But I was asking this question to see if anyone knew why a resettable input stream wouldn't work. Because, it doesn't seem to. Thanks, Karl On Fri, Mar 8, 2013 at 9:48 AM, Oleg Kalnichevski wrote: > On Fri, 2013-03-08 at 09:07 -0500, Karl Wright wrote: >> Ok, what is happening can be seen here: >> >> >> >> (Thread-20661) - << "HTTP/1.1 401 Unauthorized[\r][\n]" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Date: Fri, 08 Mar >> 2013 13:32:51 GMT[\r][\n]" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Server: >> Apache/2.2.22 (Unix)[\r][\n]" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "WWW-Authenticate: >> Basic realm="resin"[\r][\n]" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Content-Length: >> 159[\r][\n]" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Content-Type: >> text/html; charset=utf-8[\r][\n]" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "X-UA-Compatible: >> IE=EmulateIE7[\r][\n]" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Connection: close[\r][\n]" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "[\r][\n]" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Receiving response: >> HTTP/1.1 401 Unauthorized >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << HTTP/1.1 401 Unauthorized >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Date: Fri, 08 Mar >> 2013 13:32:51 GMT >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Server: Apache/2.2.22 >> (Unix) >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << WWW-Authenticate: >> Basic realm="resin" >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Content-Length: 159 >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Content-Type: >> text/html; charset=utf-8 >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << X-UA-Compatible: >> IE=EmulateIE7 >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Connection: close >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Authentication required >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - solr-prod01.uio.no:443 >> requested authentication >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Authentication schemes >> in the order of preference: [negotiate, Kerberos, NTLM, Digest, Basic] >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for negotiate >> authentication scheme not available >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for Kerberos >> authentication scheme not available >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for NTLM >> authentication scheme not available >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for Digest >> authentication scheme not available >> DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Selected authentication >> options: [BASIC] >> DEBUG 2013-03-08 14:32:51,447 (Thread-20661) - Connection >> 0.0.0.0:35590<->129.240.120.16:443 closed >> DEBUG 2013-03-08 14:32:51,447 (Thread-20661) - Connecting to >> solr-prod01.uio.no:443 >> DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - CookieSpec selected: >> best-match >> DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Auth cache not set in the >> context >> DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Target auth state: CHALLENGED >> DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Generating response to >> an authentication challenge using basic scheme >> DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Proxy auth state: UNCHALLENGED >> DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Cannot retry >> non-repeatable request >> DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Connection >> 0.0.0.0:35591<->129.240.120.16:443 shut down >> DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Connection >> 0.0.0.0:35591<->129.240.120.16:443 closed >> << >> >> The problem is that HttpClient is discovering that it needs to use >> Basic Auth after the fact, as a result of the response from the >> server. >> > > This is what the 'expect-continue' handshake is for. It enables the > client to verify server expectations prior to sending the request body. > >> For basic auth, at least, this is an inefficient interaction, because >> the user often knows in advance that Basic Auth will be required at >> the outset. While it's not an option for more involved authentication >> methods such as NTLM, it seems likely to be an important case for >> Basic Auth. Is there any way to signal HttpClient th
Re: Disabling retries
On Fri, 2013-03-08 at 09:07 -0500, Karl Wright wrote: > Ok, what is happening can be seen here: > > >> > (Thread-20661) - << "HTTP/1.1 401 Unauthorized[\r][\n]" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Date: Fri, 08 Mar > 2013 13:32:51 GMT[\r][\n]" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Server: > Apache/2.2.22 (Unix)[\r][\n]" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "WWW-Authenticate: > Basic realm="resin"[\r][\n]" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Content-Length: > 159[\r][\n]" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Content-Type: > text/html; charset=utf-8[\r][\n]" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "X-UA-Compatible: > IE=EmulateIE7[\r][\n]" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Connection: close[\r][\n]" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "[\r][\n]" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Receiving response: > HTTP/1.1 401 Unauthorized > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << HTTP/1.1 401 Unauthorized > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Date: Fri, 08 Mar > 2013 13:32:51 GMT > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Server: Apache/2.2.22 (Unix) > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << WWW-Authenticate: > Basic realm="resin" > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Content-Length: 159 > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Content-Type: > text/html; charset=utf-8 > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << X-UA-Compatible: > IE=EmulateIE7 > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Connection: close > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Authentication required > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - solr-prod01.uio.no:443 > requested authentication > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Authentication schemes > in the order of preference: [negotiate, Kerberos, NTLM, Digest, Basic] > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for negotiate > authentication scheme not available > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for Kerberos > authentication scheme not available > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for NTLM > authentication scheme not available > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for Digest > authentication scheme not available > DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Selected authentication > options: [BASIC] > DEBUG 2013-03-08 14:32:51,447 (Thread-20661) - Connection > 0.0.0.0:35590<->129.240.120.16:443 closed > DEBUG 2013-03-08 14:32:51,447 (Thread-20661) - Connecting to > solr-prod01.uio.no:443 > DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - CookieSpec selected: best-match > DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Auth cache not set in the > context > DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Target auth state: CHALLENGED > DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Generating response to > an authentication challenge using basic scheme > DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Proxy auth state: UNCHALLENGED > DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Cannot retry > non-repeatable request > DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Connection > 0.0.0.0:35591<->129.240.120.16:443 shut down > DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Connection > 0.0.0.0:35591<->129.240.120.16:443 closed > << > > The problem is that HttpClient is discovering that it needs to use > Basic Auth after the fact, as a result of the response from the > server. > This is what the 'expect-continue' handshake is for. It enables the client to verify server expectations prior to sending the request body. > For basic auth, at least, this is an inefficient interaction, because > the user often knows in advance that Basic Auth will be required at > the outset. While it's not an option for more involved authentication > methods such as NTLM, it seems likely to be an important case for > Basic Auth. Is there any way to signal HttpClient that I want to > provide pre-emptive authentication headers in that case? > Generally preemptive auth can pose security issues if used carelessly. One, however, one can pre-populate local auth cache to make HttpClient authenticate without being explicitly challenged. http://hc.apache.org/httpcomponents-client-ga/httpclient/examples/org/apache/http/examples/client/ClientPreemptiveBasicAuthentication.java > If not, the next step is figuring out how to hand HttpClient a stream > that it is happy to replay. The particular stream in question *ought* > to be resettable, since it comes from a disk-based file in this case, > and yet it isn't showing up that way to HttpClient. I can do more > research, but how is this typically done? Should I create my own > HttpEntity extension that has the right characteristics? HttpClient comes with a number of HttpEntity implementations including those backed by a byte array, a str
Re: Disabling retries
Ok, what is happening can be seen here: >> (Thread-20661) - << "HTTP/1.1 401 Unauthorized[\r][\n]" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Date: Fri, 08 Mar 2013 13:32:51 GMT[\r][\n]" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Server: Apache/2.2.22 (Unix)[\r][\n]" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "WWW-Authenticate: Basic realm="resin"[\r][\n]" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Content-Length: 159[\r][\n]" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Content-Type: text/html; charset=utf-8[\r][\n]" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "X-UA-Compatible: IE=EmulateIE7[\r][\n]" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "Connection: close[\r][\n]" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << "[\r][\n]" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Receiving response: HTTP/1.1 401 Unauthorized DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << HTTP/1.1 401 Unauthorized DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Date: Fri, 08 Mar 2013 13:32:51 GMT DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Server: Apache/2.2.22 (Unix) DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << WWW-Authenticate: Basic realm="resin" DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Content-Length: 159 DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Content-Type: text/html; charset=utf-8 DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << X-UA-Compatible: IE=EmulateIE7 DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - << Connection: close DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Authentication required DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - solr-prod01.uio.no:443 requested authentication DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Authentication schemes in the order of preference: [negotiate, Kerberos, NTLM, Digest, Basic] DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for negotiate authentication scheme not available DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for Kerberos authentication scheme not available DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for NTLM authentication scheme not available DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Challenge for Digest authentication scheme not available DEBUG 2013-03-08 14:32:51,446 (Thread-20661) - Selected authentication options: [BASIC] DEBUG 2013-03-08 14:32:51,447 (Thread-20661) - Connection 0.0.0.0:35590<->129.240.120.16:443 closed DEBUG 2013-03-08 14:32:51,447 (Thread-20661) - Connecting to solr-prod01.uio.no:443 DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - CookieSpec selected: best-match DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Auth cache not set in the context DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Target auth state: CHALLENGED DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Generating response to an authentication challenge using basic scheme DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Proxy auth state: UNCHALLENGED DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Cannot retry non-repeatable request DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Connection 0.0.0.0:35591<->129.240.120.16:443 shut down DEBUG 2013-03-08 14:32:51,448 (Thread-20661) - Connection 0.0.0.0:35591<->129.240.120.16:443 closed << The problem is that HttpClient is discovering that it needs to use Basic Auth after the fact, as a result of the response from the server. For basic auth, at least, this is an inefficient interaction, because the user often knows in advance that Basic Auth will be required at the outset. While it's not an option for more involved authentication methods such as NTLM, it seems likely to be an important case for Basic Auth. Is there any way to signal HttpClient that I want to provide pre-emptive authentication headers in that case? If not, the next step is figuring out how to hand HttpClient a stream that it is happy to replay. The particular stream in question *ought* to be resettable, since it comes from a disk-based file in this case, and yet it isn't showing up that way to HttpClient. I can do more research, but how is this typically done? Should I create my own HttpEntity extension that has the right characteristics? Karl On Fri, Mar 8, 2013 at 7:30 AM, Oleg Kalnichevski wrote: > On Fri, 2013-03-08 at 07:27 -0500, Karl Wright wrote: >> Just to be as specific as possible, here is a portion of the trace: >> >> Caused by: org.apache.http.client.ClientProtocolException >> at >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:909) >> at >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) >> at >> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) >> at >> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353) >> at >> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181) >> at >> org.apache.solr.client.solrj.
Re: Disabling retries
On Fri, 2013-03-08 at 07:27 -0500, Karl Wright wrote: > Just to be as specific as possible, here is a portion of the trace: > > Caused by: org.apache.http.client.ClientProtocolException > at > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:909) > at > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) > at > org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) > at > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353) > at > org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181) > at > org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117) > at > org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:833) > Caused by: org.apache.http.client.NonRepeatableRequestException: > Cannot retry request with a non-repeatable request entity. > at > org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:695) > at > org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522) > > I am hoping this is not the expected result of authentication being > required, etc. Please clarify if I am incorrect about that. > > Karl > I could only tell if I had a context log of the entire HTTP session. Oleg - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: Disabling retries
Just to be as specific as possible, here is a portion of the trace:
Caused by: org.apache.http.client.ClientProtocolException
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:909)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
at
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at
org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at
org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:833)
Caused by: org.apache.http.client.NonRepeatableRequestException:
Cannot retry request with a non-repeatable request entity.
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:695)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
I am hoping this is not the expected result of authentication being
required, etc. Please clarify if I am incorrect about that.
Karl
On Fri, Mar 8, 2013 at 7:21 AM, Karl Wright wrote:
> After looking at the code, I've asked the client to turn on an
> appropriate logger to see what is going on in his case. The symptom
> is a "non-resettable stream" exception being thrown. The concern now
> is that, for input which includes a non-resettable stream, there is a
> non-error code path that causes this exception, perhaps erroneously.
> I'll provide further details, and if warranted a ticket and a fix,
> when I have them.
>
> Karl
>
> On Fri, Mar 8, 2013 at 7:13 AM, Oleg Kalnichevski wrote:
>> On Thu, 2013-03-07 at 08:39 -0500, Karl Wright wrote:
>>> Hi all,
>>>
>>> We have code that creates a DefaultHttpClient instance for use with
>>> Solr. The HttpEntity that is created when sending data is not
>>> reusable, so we've disabled retries (we thought) using the following
>>> code:
>>>
>>> DefaultHttpClient localClient = new
>>> DefaultHttpClient(connectionManager,params);
>>>
>>> // No retries
>>> localClient.setHttpRequestRetryHandler(new HttpRequestRetryHandler()
>>> {
>>> public boolean retryRequest(
>>> IOException exception,
>>> int executionCount,
>>> HttpContext context)
>>> {
>>> return false;
>>> }
>>>
>>> });
>>>
>>>
>>> Unfortunately it does not seem to have actually worked; we are still
>>> seeing "non-reusable stream" retry errors in some cases. Has anybody
>>> seen this before, and what
>>> are we doing wrong?
>>>
>>> Karl
>>>
>>
>> Karl
>>
>> Requests can also be retried automatically in case of an authentication
>> challenge and in rate cases due to a redirect. I/O errors is the most
>> common but not the only possible cause of automatic retries.
>>
>> Oleg
>>
>>
>>
>> -
>> To unsubscribe, e-mail: [email protected]
>> For additional commands, e-mail: [email protected]
>>
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: Disabling retries
After looking at the code, I've asked the client to turn on an
appropriate logger to see what is going on in his case. The symptom
is a "non-resettable stream" exception being thrown. The concern now
is that, for input which includes a non-resettable stream, there is a
non-error code path that causes this exception, perhaps erroneously.
I'll provide further details, and if warranted a ticket and a fix,
when I have them.
Karl
On Fri, Mar 8, 2013 at 7:13 AM, Oleg Kalnichevski wrote:
> On Thu, 2013-03-07 at 08:39 -0500, Karl Wright wrote:
>> Hi all,
>>
>> We have code that creates a DefaultHttpClient instance for use with
>> Solr. The HttpEntity that is created when sending data is not
>> reusable, so we've disabled retries (we thought) using the following
>> code:
>>
>> DefaultHttpClient localClient = new
>> DefaultHttpClient(connectionManager,params);
>>
>> // No retries
>> localClient.setHttpRequestRetryHandler(new HttpRequestRetryHandler()
>> {
>> public boolean retryRequest(
>> IOException exception,
>> int executionCount,
>> HttpContext context)
>> {
>> return false;
>> }
>>
>> });
>>
>>
>> Unfortunately it does not seem to have actually worked; we are still
>> seeing "non-reusable stream" retry errors in some cases. Has anybody
>> seen this before, and what
>> are we doing wrong?
>>
>> Karl
>>
>
> Karl
>
> Requests can also be retried automatically in case of an authentication
> challenge and in rate cases due to a redirect. I/O errors is the most
> common but not the only possible cause of automatic retries.
>
> Oleg
>
>
>
> -
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: Disabling retries
On Thu, 2013-03-07 at 08:39 -0500, Karl Wright wrote:
> Hi all,
>
> We have code that creates a DefaultHttpClient instance for use with
> Solr. The HttpEntity that is created when sending data is not
> reusable, so we've disabled retries (we thought) using the following
> code:
>
> DefaultHttpClient localClient = new
> DefaultHttpClient(connectionManager,params);
>
> // No retries
> localClient.setHttpRequestRetryHandler(new HttpRequestRetryHandler()
> {
> public boolean retryRequest(
> IOException exception,
> int executionCount,
> HttpContext context)
> {
> return false;
> }
>
> });
>
>
> Unfortunately it does not seem to have actually worked; we are still
> seeing "non-reusable stream" retry errors in some cases. Has anybody
> seen this before, and what
> are we doing wrong?
>
> Karl
>
Karl
Requests can also be retried automatically in case of an authentication
challenge and in rate cases due to a redirect. I/O errors is the most
common but not the only possible cause of automatic retries.
Oleg
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
