On 2 August 2012 19:36, Jean-Marc Spaggiari <[email protected]> wrote: > I'm using 1.7.0_05 from Sun. > > I have enabled the RequestAddCookies debug logs and will see when I > will get the next NPE. > > One the line 193 (I don't have the source) is version an int? Or an Integer?
int, see: http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/impl/cookie/BestMatchSpec.html#193 and http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/cookie/Cookie.html#126 > I agree with you, cookie can't be null, else we will have got an NPE > at "if (!matchedCookies.isEmpty())" > > I will restart my tool with the debug mode enabled and keep you posted. > > 2012/8/2, sebb <[email protected]>: >> On 2 August 2012 19:03, Jean-Marc Spaggiari <[email protected]> >> wrote: >>> I'm using version 4.2.1 binary distribution. >> >> In which case the NPE is very odd. >> >> The stack dump shows: >> >> java.lang.NullPointerException >> at >> org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193) >> at >> org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196) >> >> Now the code in question is: >> >> BestMatchSpec.java:193: if (cookie.getVersion() < version) { >> >> AFAICT, this can only cause NPE if "cookie" is null. >> >> The cookie variable is derived from the list created in >> RequestAddCookies.java: >> >> List<Cookie> cookies = new >> ArrayList<Cookie>(cookieStore.getCookies()); >> // Find cookies matching the given origin >> List<Cookie> matchedCookies = new ArrayList<Cookie>(); >> Date now = new Date(); >> for (Cookie cookie : cookies) { >> if (!cookie.isExpired(now)) { // <== cookie cannot be null here >> if (cookieSpec.match(cookie, cookieOrigin)) { >> if (this.log.isDebugEnabled()) { >> this.log.debug("Cookie " + cookie + " match " >> + cookieOrigin); >> } >> matchedCookies.add(cookie); // <== so cannot be null >> here >> } >> } else { >> if (this.log.isDebugEnabled()) { >> this.log.debug("Cookie " + cookie + " expired"); >> } >> } >> } >> // Generate Cookie request headers >> if (!matchedCookies.isEmpty()) { >> List<Header> headers = >> cookieSpec.formatCookies(matchedCookies); // <== 196 >> for (Header header : headers) { >> request.addHeader(header); >> } >> } >> >> I cannot see how a null cookie can be added to the list, nor how a >> list entry could be set to null later. >> >> It would be worth enabling Debug logging for the RequestAddCookies class. >> >> What JVM are you using? >> >>> 2012/8/2, sebb <[email protected]>: >>>> On 1 August 2012 12:08, Jean-Marc Spaggiari <[email protected]> >>>> wrote: >>>>> Hi Oleg, >>>>> >>>>> My application is reading many other URLs before this one. So maybe >>>>> it's a combination of multiple loads. >>>>> >>>>> I will try to dump more when I will get the issue again. >>>>> >>>>> In the meantime, here is how I'm initializing the HttpClient: >>>>> >>>>> // Creation of the HTTP Client >>>>> SchemeRegistry schemeRegistry = new SchemeRegistry(); >>>>> schemeRegistry.register(new Scheme("http", 80, >>>>> PlainSocketFactory.getSocketFactory())); >>>>> schemeRegistry.register(new Scheme("https", 443, >>>>> SSLSocketFactory.getSocketFactory())); >>>>> PoolingClientConnectionManager cm = new >>>>> PoolingClientConnectionManager(schemeRegistry); >>>>> cm.setMaxTotal(200); >>>>> cm.setDefaultMaxPerRoute(20); >>>>> >>>>> HttpParams params = new BasicHttpParams(); >>>>> if (useProxy) >>>>> params.setParameter(ConnRoutePNames.DEFAULT_PROXY, new >>>>> HttpHost("proxy", 80)); >>>>> client = new DefaultHttpClient(cm, params); >>>>> client.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, new >>>>> Integer(30000)); >>>>> client.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, >>>>> new Integer(30000)); >>>>> client.getParams().setParameter(CoreConnectionPNames.TCP_NODELAY, >>>>> false); >>>>> >>>>> It's not the first time I'm getting this exception, so I might be able >>>>> to reproduce that in a near futur. >>>> >>>> What version of HttpClient are you using? >>>> >>>>> JM >>>>> >>>>> 2012/8/1, Oleg Kalnichevski <[email protected]>: >>>>>> On Tue, 2012-07-31 at 17:37 -0400, Jean-Marc Spaggiari wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I'm getting the exception below when I'm trying to load >>>>>>> http://www.expedia.ca/Sunset-District-Hotels.d163838.Travel-Guide-Hotels >>>>>>> ... It's not critical since I'm catching everything, but I'm >>>>>>> wondering >>>>>>> if this is normal or if this is something which need to be fixed... >>>>>>> >>>>>>> java.lang.NullPointerException >>>>>>> at >>>>>>> org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193) >>>>>>> at >>>>>>> org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196) >>>>>>> at >>>>>>> org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:109) >>>>>>> at >>>>>>> org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:176) >>>>>>> at >>>>>>> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515) >>>>>>> at >>>>>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) >>>>>>> at >>>>>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) >>>>>>> at >>>>>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) >>>>>>> at >>>>>>> org.spaggiari.distparser.CrawlerThread$1.run(CrawlerThread.java:267) >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> JM >>>>>>> >>>>>> >>>>>> Jean-Marc >>>>>> >>>>>> I am not able to reproduce the problem just by executing HTTP GET to >>>>>> the >>>>>> URL in question. All looks perfectly normal. >>>>>> >>>>>> [DEBUG] BasicClientConnectionManager - Get connection for route >>>>>> {}->http://www.expedia.ca >>>>>> [DEBUG] DefaultClientConnectionOperator - Connecting to >>>>>> www.expedia.ca:80 >>>>>> [DEBUG] RequestAddCookies - CookieSpec selected: best-match >>>>>> [DEBUG] RequestAuthCache - Auth cache not set in the context >>>>>> [DEBUG] RequestTargetAuthentication - Target auth state: UNCHALLENGED >>>>>> [DEBUG] RequestProxyAuthentication - Proxy auth state: UNCHALLENGED >>>>>> [DEBUG] DefaultHttpClient - Attempt 1 to execute request >>>>>> [DEBUG] DefaultClientConnection - Sending request: >>>>>> GET /Sunset-District-Hotels.d163838.Travel-Guide-Hotels HTTP/1.1 >>>>>> [DEBUG] headers - >> >>>>>> GET /Sunset-District-Hotels.d163838.Travel-Guide-Hotels HTTP/1.1 >>>>>> [DEBUG] headers - >> Host: www.expedia.ca >>>>>> [DEBUG] headers - >> Connection: Keep-Alive >>>>>> [DEBUG] headers - >> User-Agent: Apache-HttpClient/4.2.1 (java 1.5) >>>>>> [DEBUG] DefaultClientConnection - Receiving response: HTTP/1.1 200 OK >>>>>> [DEBUG] headers - << HTTP/1.1 200 OK >>>>>> [DEBUG] headers - << Content-Type: text/html;charset=UTF-8 >>>>>> [DEBUG] headers - << Content-Language: en-CA >>>>>> [DEBUG] headers - << P3P: policyref="/w3c/p3p.xml", CP="CAO DSP IND >>>>>> COR >>>>>> ADM CONo CUR CUSi DEV PSA PSD DELi OUR COM NAV PHY ONL PUR UNI" >>>>>> [DEBUG] headers - << RTSS: 1 >>>>>> [DEBUG] headers - << Date: Wed, 01 Aug 2012 10:49:28 GMT >>>>>> [DEBUG] headers - << Transfer-Encoding: chunked >>>>>> [DEBUG] headers - << Connection: keep-alive >>>>>> [DEBUG] headers - << Connection: Transfer-Encoding >>>>>> [DEBUG] headers - << Set-Cookie: SSLB=1; path=/; domain=.expedia.ca >>>>>> [DEBUG] headers - << Set-Cookie: >>>>>> MC1=GUID=c55cfa1393ce4bfca1ac5b6532085717; Domain=.expedia.ca; >>>>>> Expires=Fri, 04-Aug-2017 03:44:09 GMT; Path=/ >>>>>> [DEBUG] headers - << Set-Cookie: iEAPID=000000,; Domain=.expedia.ca; >>>>>> Path=/ >>>>>> [DEBUG] headers - << Set-Cookie: >>>>>> JSESSION=13201604-6f17-4f6e-8b03-570fd433ad4e; Domain=.expedia.ca; >>>>>> Path=/ >>>>>> [DEBUG] headers - << Set-Cookie: s1=`0; Domain=.expedia.ca; Path=/ >>>>>> [DEBUG] headers - << Set-Cookie: p1=`tpid=v.1,4`linfo=v.4,|0|0|255|1| >>>>>> 0||||||||4105|0|0||0|0|0|-1|-1`63; Domain=.expedia.ca; Expires=Tue, >>>>>> 01-Aug-2017 15:54:40 GMT; Path=/ >>>>>> [DEBUG] ResponseProcessCookies - Cookie accepted: "[version: 0][name: >>>>>> SSLB][value: 1][domain: .expedia.ca][path: /][expiry: null]". >>>>>> [DEBUG] ResponseProcessCookies - Cookie accepted: "[version: 0][name: >>>>>> MC1][value: >>>>>> GUID=c55cfa1393ce4bfca1ac5b6532085717][domain: .expedia.ca][path: >>>>>> /][expiry: >>>>>> Fri Aug 04 04:44:09 GMT+01:00 2017]". >>>>>> [DEBUG] ResponseProcessCookies - Cookie accepted: "[version: 0][name: >>>>>> iEAPID][value: 000000,][domain: .expedia.ca][path: /][expiry: null]". >>>>>> [DEBUG] ResponseProcessCookies - Cookie accepted: "[version: 0][name: >>>>>> JSESSION][value: >>>>>> 13201604-6f17-4f6e-8b03-570fd433ad4e][domain: .expedia.ca][path: >>>>>> /][expiry: >>>>>> null]". >>>>>> [DEBUG] ResponseProcessCookies - Cookie accepted: "[version: 0][name: >>>>>> s1][value: `0][domain: .expedia.ca][path: /][expiry: null]". >>>>>> [DEBUG] ResponseProcessCookies - Cookie accepted: "[version: 0][name: >>>>>> p1][value: `tpid=v.1,4`linfo=v.4,|0|0|255|1|0||||||||4105|0|0||0|0| >>>>>> 0|-1|-1`63][domain: .expedia.ca][path: /][expiry: Tue Aug 01 16:54:40 >>>>>> GMT+01:00 2017]". >>>>>> [DEBUG] DefaultHttpClient - Connection can be kept alive indefinitely >>>>>> >>>>>> Can you put together a small test app that could be used to reproduce >>>>>> the NPE? >>>>>> >>>>>> Oleg >>>>>> >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>>> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: [email protected] >>>> For additional commands, e-mail: [email protected] >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
