Re: HttpClient 4.0
--Mensaje original-- De: visualize Para:httpclient-users@hc.apache.org Responder a:HttpClient User Discussion Asunto: Re: HttpClient 4.0 Enviado: 28 Sep, 2009 11:06 olegk wrote: visualize wrote: Hello, I'm trying to port my application to use HttpClient 4.0 instead of version 3.0 I make a substantial number of HTTP-gets and used to use aMultiThreadedHttpConnectionManager to setup a pool and retrieve clients. Let's say I want to get some stuff from http://my.server.com:8080/my-rest-service/data; and http://my.server.com:8080/my-rest-service/data2;. In 3.0 you are able to set host and port via getHostConfiguration().setHost(host, port) which was kind of convenient since I then could just create the pool, and when getting clients just set the host and port to http://my.server.com; and 8080. In each request, I just had to set the URL to /my-rest-service/data or /my-rest-service/data2 and didn't have to care about the host and port. Is there a way to accomplish the same with a client of version 4.0? Like set a default host/route (or whatever you may call it), so when doing new HttpGet(/my-rest-service/data) it falls back to the default one? / K Use http.default-host parameter. - DefaultHttpClient httpclient = new DefaultHttpClient(); httpclient.getParams().setParameter(ClientPNames.DEFAULT_HOST, new HttpHost(www.google.com, 80, http)); HttpGet req = new HttpGet(/); HttpResponse rsp = httpclient.execute(req); HttpEntity entity = rsp.getEntity(); System.out.println(); System.out.println(rsp.getStatusLine()); System.out.println(); if (entity != null) { System.out.println(EntityUtils.toString(entity)); } - Hope this helps Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org Fantastic, thanks a lot! -- View this message in context: http://www.nabble.com/HttpClient-4.0-tp25531034p25641698.html Sent from the HttpClient-User mailing list archive at Nabble.com. - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org Enviado desde mi dispositivo BlackBerry® de Orange.
RE: RV: NO_HTTP_RESPONSE_EXCEPTION
Hello Oleg, I attach the http log. I've been taking a look to this trace, and I've observed the following. I don't know if it's important or not. In line 53315, we have a request that is correctly processed. When HttpClient tries to get a connection from pool, the information is: Total connections kept alive: 2 Total issued connections: 0 Total allocated connection: 2 out of 100 Getting free connection [HttpRoute[{}-http://192.168.38.87:80]][null] The following request fails with noHttpResponse (starts at line 53362). In this request, when HttpClient tries to get a connection from pool, the information is: Total connections kept alive: 11 Total issued connections: 0 Total allocated connection: 11 out of 100 Getting free connection [HttpRoute[{}-http://192.168.46.152:11003]][null] Is it normal that this request shows a pool with 11 allocated connections when the previous request only shows 2? Here, httpNoResponseException starts until the pool is empty. You can see 11 consecutive erroneous requests with Connection closed and Released connection is not reusable UNTIL total allocated connection is 0. When total allocated connection is 0, HttpClient has to create a new connection (line 53713), and then everything works again. Well, exactly the same situation is reproduced starting at line 66610. In this line, the request goes OK. The information is: Total connections kept alive: 2 Total issued connections: 0 Total allocated connection: 2 out of 100 Getting free connection [HttpRoute[{}-http://10.12.112.180:12003]][null] The following request again fails, and the information is: Total connections kept alive: 10 Total issued connections: 0 Total allocated connection: 10 out of 100 Getting free connection [HttpRoute[{}-http://192.168.46.152:11003]][null] Again, we can see 10 consecutive erroneous requests, until allocated connections comes 0 again. In this point, everything starts to work again when Httpclient has to create a new connection. I try to attach the log file as a zip file. Last time it was impossible to send it to you. If you don't receive it, tell how I can send it. Thanks for your time, Oleg. Joan. -Mensaje original- De: Oleg Kalnichevski [mailto:ol...@apache.org] Enviado el: miércoles, 22 de julio de 2009 14:41 Para: HttpClient User Discussion Asunto: Re: RV: NO_HTTP_RESPONSE_EXCEPTION On Wed, Jul 22, 2009 at 02:17:17PM +0200, Joan Balaguer? Valls wrote: Hello Oleg, Five months ago, I sent you the emails below because I was experiencing a lot of NoHttpResponseException errors when sending requests to a couple of servers. Yesterday night, I installed my app with H4 in one of my clients. This morning I've had to move back to H3 because of these errors. The app consists of a proxy that is sending requests to 3 different webservices. The app has a different connection pool for each webservice, and the target servers are also different for each one. App - requests to Webservice1 -- uses pool1 -- send to serversA,B - requests to Webservice2 -- uses pool2 -- send to serversC,D - requests to Webservice3 -- uses pool3 -- send to serversE,F Joan, Please post a context / wire log of the HTTP session that exhibits the problem http://hc.apache.org/httpcomponents-client/logging.html Please also consider using the latest HttpClient 4.0 RC build: http://people.apache.org/~olegk/httpclient-4.0-rc2/ Mind you it is very unlikely this has anything to do with HttpClient, so expect no miracles. Oleg The results for webservice1 are: Tuesday 22/07:HttpClient 3.1. Total Requests sent : 175.003 From 00:00 to 23:59 OK Requests : 174.936 Error Requests: : 67 Wednes. 23/07:HttpClient 4.0 Total Requests sent : 54.851 From 00:00 to 13:30 OK Requests : 51.183 Error Requests: : 3.668 From these 3.668 errors, 3.647 corresponding to NoHttpResponseException. The error percentages are similar in ws2 and ws3. The target servers for the ws1 and ws2 are in the same LAN than my app, while target servers for w3 are on the Internet. After moving back to H3, again 0 errors in the last 2 hours. Please, just tell me what you need to get more information about this problem (traces, configurations, ...). Anything you need. Thanks in advance, Joan. -Mensaje original- De: Oleg Kalnichevski [mailto:ol...@apache.org] Enviado el: s?bado, 07 de febrero de 2009 14:08 Para: HttpClient User Discussion Asunto: Re: NO_HTTP_RESPONSE_EXCEPTION sebb wrote: On 06/02/2009, Oleg Kalnichevski ol...@apache.org wrote: Joan Balaguer? wrote: Thanks Oleg. It seems strange to me because HttpClient3 (sending requests to the same server) never fails. I'll take a look at the code again and I'll comment you
RE: RV: NO_HTTP_RESPONSE_EXCEPTION
Hello Oleg, Ok, then, in your opinion, it seems that some connections were closed on the server side, but for H4 they are OK, and when H4 tries to use them, then a NoHttpResponseException is thrown. In fact, I've implemented an IdleConnectionsHandler that closes expired connections and idle connections (with a timeout). Now, this process is executed every 5 minutes, and closes expired connections and connections idle for more than 60 seconds. I suppose that this is too much wait time. What values do you recommend? Another question: the same app works perfectly with Http3. What does it mean? Is H4 much more sensible to server closed connections than H3? I've never used the stale check (H3 or H4). Thanks, Joan. -Mensaje original- De: Oleg Kalnichevski [mailto:ol...@apache.org] Enviado el: miércoles, 22 de julio de 2009 18:42 Para: HttpClient User Discussion Asunto: Re: RV: NO_HTTP_RESPONSE_EXCEPTION On Wed, Jul 22, 2009 at 06:06:25PM +0200, Joan Balaguer? Valls wrote: Hello Oleg, I attach the http log. Hi Joan, Unfortunately the log did not get through I've been taking a look to this trace, and I've observed the following. I don't know if it's important or not. In line 53315, we have a request that is correctly processed. When HttpClient tries to get a connection from pool, the information is: Total connections kept alive: 2 Total issued connections: 0 Total allocated connection: 2 out of 100 Getting free connection [HttpRoute[{}-http://192.168.38.87:80]][null] The following request fails with noHttpResponse (starts at line 53362). In this request, when HttpClient tries to get a connection from pool, the information is: Total connections kept alive: 11 Total issued connections: 0 Total allocated connection: 11 out of 100 Getting free connection [HttpRoute[{}-http://192.168.46.152:11003]][null] Is it normal that this request shows a pool with 11 allocated connections when the previous request only shows 2? It is normal. Those two requests have _completely_ different routes. Here, httpNoResponseException starts until the pool is empty. You can see 11 consecutive erroneous requests with Connection closed and Released connection is not reusable UNTIL total allocated connection is 0. When total allocated connection is 0, HttpClient has to create a new connection (line 53713), and then everything works again. It is all very simple. Those 11 connections have most likely been idle for two long and therefore got dropped on the server side. You should either use the stale connection check or implement some sort of a stale connection eviction policy to make sure HttpClient does not try to re-use them. For details see http://hc.apache.org/httpcomponents-client/tutorial/html/ch02.html#d4e638 Hope this helps Oleg Well, exactly the same situation is reproduced starting at line 66610. In this line, the request goes OK. The information is: Total connections kept alive: 2 Total issued connections: 0 Total allocated connection: 2 out of 100 Getting free connection [HttpRoute[{}-http://10.12.112.180:12003]][null] The following request again fails, and the information is: Total connections kept alive: 10 Total issued connections: 0 Total allocated connection: 10 out of 100 Getting free connection [HttpRoute[{}-http://192.168.46.152:11003]][null] Again, we can see 10 consecutive erroneous requests, until allocated connections comes 0 again. In this point, everything starts to work again when Httpclient has to create a new connection. I try to attach the log file as a zip file. Last time it was impossible to send it to you. If you don't receive it, tell how I can send it. Thanks for your time, Oleg. Joan. -Mensaje original- De: Oleg Kalnichevski [mailto:ol...@apache.org] Enviado el: mi?rcoles, 22 de julio de 2009 14:41 Para: HttpClient User Discussion Asunto: Re: RV: NO_HTTP_RESPONSE_EXCEPTION On Wed, Jul 22, 2009 at 02:17:17PM +0200, Joan Balaguer? Valls wrote: Hello Oleg, Five months ago, I sent you the emails below because I was experiencing a lot of NoHttpResponseException errors when sending requests to a couple of servers. Yesterday night, I installed my app with H4 in one of my clients. This morning I've had to move back to H3 because of these errors. The app consists of a proxy that is sending requests to 3 different webservices. The app has a different connection pool for each webservice, and the target servers are also different for each one. App - requests to Webservice1 -- uses pool1 -- send to serversA,B - requests to Webservice2 -- uses pool2 -- send to serversC,D - requests to Webservice3 -- uses pool3 -- send to serversE,F Joan, Please post a context / wire log of the HTTP session that exhibits the problem http://hc.apache.org/httpcomponents-client/logging.html Please also consider using the latest HttpClient 4.0
RE: Http trace for cookies
Hello Oleg, Then, how is it possible that I always receive -1 in maxAge attribute? Am I forgetting something, or doing something wrong? If I try to set the MAX_AGE_ATTR and the EXPIRES_ATTR, I get -1 again. How is the correct way to propagate the cookie expiry date? Thanks in advance, Joan. Date expires = new Date(System.currentTimeMillis() + 6); BasicClientCookie bcc = new BasicClientCookie(testName, testValue); bcc.setVersion(1); bcc.setDomain(wstest.rhodasol.es); bcc.setPath(/wsserhs/rhodasol); bcc.setSecure(false); bcc.setExpiryDate(expires); bcc.setAttribute(ClientCookie.VERSION_ATTR, 1); bcc.setAttribute(ClientCookie.DOMAIN_ATTR, wstest.rhodasol.es); bcc.setAttribute(ClientCookie.PATH_ATTR, /wsserhs/rhodasol); bcc.setAttribute(ClientCookie.MAX_AGE_ATTR, 60); bcc.setAttribute(ClientCookie.EXPIRES_ATTR, DateUtils.formatDate(expires)); -Mensaje original- De: Oleg Kalnichevski [mailto:ol...@apache.org] Enviado el: miércoles, 08 de julio de 2009 23:54 Para: HttpClient User Discussion Asunto: Re: Http trace for cookies Joan Balagueró Valls wrote: Hello Oleg, Im trying to send you the trace, but your mail server does not accept it. I paste here a couple of lines of this trace: This is the interesting part of the trace. The expiry date is correct in the line 7 (60 seconds from now). But it seems this expiry date does not appear in Cookie header. Joan. CookieSpec selected: best-match Cookie [version: 1][name: testName][value: testValue][domain: wstest.rhodasol.es][path: /wsserhs/rhodasol][expiry: Wed Jul 08 16:22:39 CEST 2009] match [wstest.rhodasol.es:81/wsserhs/rhodasol] Sending request: POST /wsserhs/rhodasol HTTP/1.1 POST /wsserhs/rhodasol HTTP/1.1 Accept-Encoding: gzip Content-Length: 444 Content-Type: text/xml Host: wstest.rhodasol.es:81 Connection: Keep-Alive Cookie: $Version=1; testName=testValue; $Path=/wsserhs/rhodasol; $Domain=wstest.rhodasol.es Cookie2: $Version=1 Receiving response: HTTP/1.1 200 OK HTTP/1.1 200 OK I am sorry, Joan, but I see nothing wrong with the cookie. The cookie looks perfectly valid to me. Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
EXPECT-CONTINUE HANDSHAKE
Hello Oleg, Can I set the expect-continue handshake at httpclient level? HttpParams objHttpParams = new BasicHttpParams(); HttpProtocolParams.setVersion(objHttpParams, HttpVersion.HTTP_1_1); ClientConnectionManager cm = new ThreadSafeClientConnManager(this.objHttpParams, HttpUtils.createDefaultSchemeRegistry()); this.objHttp = new DefaultHttpClient(cm, this.objHttpParams); -- this.objHttpParams.setParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, Boolean.FALSE); Or should I set at request level? Thanks, Joan. - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
QUESTION ABOUT COOKIES
Again the attach zipped. Joan. - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
TRYING AGAIN THE ATTACH FOR COOKIES QUESTION
Hello, Here the attach (I hope) Joan. - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Http trace for cookies
Hello Oleg, Im trying to send you the trace, but your mail server does not accept it. I paste here a couple of lines of this trace: This is the interesting part of the trace. The expiry date is correct in the line 7 (60 seconds from now). But it seems this expiry date does not appear in Cookie header. Joan. CookieSpec selected: best-match Cookie [version: 1][name: testName][value: testValue][domain: wstest.rhodasol.es][path: /wsserhs/rhodasol][expiry: Wed Jul 08 16:22:39 CEST 2009] match [wstest.rhodasol.es:81/wsserhs/rhodasol] Sending request: POST /wsserhs/rhodasol HTTP/1.1 POST /wsserhs/rhodasol HTTP/1.1 Accept-Encoding: gzip Content-Length: 444 Content-Type: text/xml Host: wstest.rhodasol.es:81 Connection: Keep-Alive Cookie: $Version=1; testName=testValue; $Path=/wsserhs/rhodasol; $Domain=wstest.rhodasol.es Cookie2: $Version=1 Receiving response: HTTP/1.1 200 OK HTTP/1.1 200 OK
RE: QUESTION ABOUT COOKIES
Hello Oleg, Thanks, after some tests this is exactly what it happens. When I send this cookie to my servlet, I receive all its data correctly except the max-age attribute, which is always -1. I'm trying to set MAX_AGE with your API, and I'm getting crazy... I've tried stdCookie.setExpiryDate(new java.util.Date(System.currentTimeMillis() + 6)); // Expires after 60 seconds But my servlet gets -1 (when I get cookies from HttpServletRequest with request.getCookies()). I supposed that I had to set the same value for MAX_AGE attribute. Then I tried: stdCookie.setExpiryDate(new java.util.Date(System.currentTimeMillis() + 6)); stdCookie.setAttribute(ClientCookie.MAX_AGE_ATTR, 60); // 60 seconds But my servlet still receives -1 in MAX_AGE. Finally, I'm trying to set the ClientCookie.EXPIRES_ATTR with the value of java.util.Date(System.currentTimeMillis() + 6)), but the value passed to setAttribute is expected to be a String, and I have a java.util.Date. How can I make this conversion? Thanks in advance, Joan. -Mensaje original- De: Oleg Kalnichevski [mailto:ol...@apache.org] Enviado el: martes, 07 de julio de 2009 16:18 Para: HttpClient User Discussion Asunto: Re: QUESTION ABOUT COOKIES On Mon, Jul 06, 2009 at 08:14:48PM +0200, Joan Balaguer? Valls wrote: Hello Oleg, I?m trying to send cookies to a servlet with a simple app. Following the tutorial: HttpContext localContext = new BasicHttpContext(); CookieStore cookieStore = new BasicCookieStore(); BasicClientCookie stdCookie = new BasicClientCookie(name, value); stdCookie.setVersion(1); stdCookie.setDomain(.mycompany.com); stdCookie.setPath(/); stdCookie.setSecure(true); // Set attributes EXACTLY as sent by the server stdCookie.setAttribute(ClientCookie.VERSION_ATTR, 1); stdCookie.setAttribute(ClientCookie.DOMAIN_ATTR, .mycompany.com); cookieStore.addCookie(stdCookie); localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore); HttpEntity entity = objHttp.execute(objPost, localContext).getEntity(); This does not work (at least for me). To work, you need to add: stdCookie.setAttribute(ClientCookie.PATH, /); If I forget any of the ?set? statement, or any of the ?setAttribute? statement, it does not work. The question is: Why have we to set twice the components of the ?BasicClientCookie?, the first using ?stdCookie.set? and the second using ?stdCookie.setAttribute?? And should I set ?ClientCookie.SECURE_ATTR? and ?ClientCookie.MAX_AGE_ATTR? ? Joan This is because some cookies set domain / path / port attributes explicitly, while some do not, in which case values of those attributes are derived from the properties of the origin server. Consider the following example: Set-Cookie: stuff=very important; path=/; domain=myhost.mydomain.com; version=1 Set-Cookie: stuff=very important; version=1 These two cookies are obviously different but they essentially represent the same piece of state information if sent in response to a request for http://myhost.mydomain.com/index.html; And the second part: when the servlet receives this cookie, it is resent to another servlet using the same sequence of operations. But debugging, one can see that the cookie is not added to the cookieStore (the sentence ?cookieStore.addCookie(stdCookie);? does not add anything to ?cookieStore?). A cookie does not get added to the cookie store only if it has expired. Hope this helps Oleg Can you help me? Thanks in advance, Joan. - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
RE: QUESTION ABOUT COOKIES
Hello, Here the attach with the trace. It's a post request from my local app to my servlet. In all tests I've always used: bcc.setVersion(1); bcc.setAttribute(ClientCookie.VERSION_ATTR, 1); Thanks, Joan. -Mensaje original- De: Oleg Kalnichevski [mailto:ol...@apache.org] Enviado el: martes, 07 de julio de 2009 17:18 Para: HttpClient User Discussion Asunto: Re: QUESTION ABOUT COOKIES On Tue, Jul 07, 2009 at 05:04:59PM +0200, Joan Balaguer? Valls wrote: Hello Oleg, Thanks, after some tests this is exactly what it happens. When I send this cookie to my servlet, I receive all its data correctly except the max-age attribute, which is always -1. I'm trying to set MAX_AGE with your API, and I'm getting crazy... I've tried stdCookie.setExpiryDate(new java.util.Date(System.currentTimeMillis() + 6)); // Expires after 60 seconds But my servlet gets -1 (when I get cookies from HttpServletRequest with request.getCookies()). I supposed that I had to set the same value for MAX_AGE attribute. Then I tried: stdCookie.setExpiryDate(new java.util.Date(System.currentTimeMillis() + 6)); stdCookie.setAttribute(ClientCookie.MAX_AGE_ATTR, 60); // 60 seconds But my servlet still receives -1 in MAX_AGE. Finally, I'm trying to set the ClientCookie.EXPIRES_ATTR with the value of java.util.Date(System.currentTimeMillis() + 6)), but the value passed to setAttribute is expected to be a String, and I have a java.util.Date. How can I make this conversion? Thanks in advance, Post a wire log of the session: http://hc.apache.org/httpcomponents-client/logging.html Also, try setting cookie version to version 1 to force the use of a RFC compliant cookie spec and see if that makes any difference. Oleg Joan. -Mensaje original- De: Oleg Kalnichevski [mailto:ol...@apache.org] Enviado el: martes, 07 de julio de 2009 16:18 Para: HttpClient User Discussion Asunto: Re: QUESTION ABOUT COOKIES On Mon, Jul 06, 2009 at 08:14:48PM +0200, Joan Balaguer? Valls wrote: Hello Oleg, I?m trying to send cookies to a servlet with a simple app. Following the tutorial: HttpContext localContext = new BasicHttpContext(); CookieStore cookieStore = new BasicCookieStore(); BasicClientCookie stdCookie = new BasicClientCookie(name, value); stdCookie.setVersion(1); stdCookie.setDomain(.mycompany.com); stdCookie.setPath(/); stdCookie.setSecure(true); // Set attributes EXACTLY as sent by the server stdCookie.setAttribute(ClientCookie.VERSION_ATTR, 1); stdCookie.setAttribute(ClientCookie.DOMAIN_ATTR, .mycompany.com); cookieStore.addCookie(stdCookie); localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore); HttpEntity entity = objHttp.execute(objPost, localContext).getEntity(); This does not work (at least for me). To work, you need to add: stdCookie.setAttribute(ClientCookie.PATH, /); If I forget any of the ?set? statement, or any of the ?setAttribute? statement, it does not work. The question is: Why have we to set twice the components of the ?BasicClientCookie?, the first using ?stdCookie.set? and the second using ?stdCookie.setAttribute?? And should I set ?ClientCookie.SECURE_ATTR? and ?ClientCookie.MAX_AGE_ATTR? ? Joan This is because some cookies set domain / path / port attributes explicitly, while some do not, in which case values of those attributes are derived from the properties of the origin server. Consider the following example: Set-Cookie: stuff=very important; path=/; domain=myhost.mydomain.com; version=1 Set-Cookie: stuff=very important; version=1 These two cookies are obviously different but they essentially represent the same piece of state information if sent in response to a request for http://myhost.mydomain.com/index.html; And the second part: when the servlet receives this cookie, it is resent to another servlet using the same sequence of operations. But debugging, one can see that the cookie is not added to the cookieStore (the sentence ?cookieStore.addCookie(stdCookie);? does not add anything to ?cookieStore?). A cookie does not get added to the cookie store only if it has expired. Hope this helps Oleg Can you help me? Thanks in advance, Joan. - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org - To unsubscribe, e-mail:
H4 - B3
Hello Oleg, Do you know any idea about when you plan to release H4-beta3? Many thanks, Joan.