Re: NullPointerException in BestMatchSpec.formatCookies
On 1 August 2012 12:08, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote:
Hi Oleg,
My application is reading many other URLs before this one. So maybe
it's a combination of multiple loads.
I will try to dump more when I will get the issue again.
In the meantime, here is how I'm initializing the HttpClient:
// Creation of the HTTP Client
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme(http, 80,
PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(new Scheme(https, 443,
SSLSocketFactory.getSocketFactory()));
PoolingClientConnectionManager cm = new
PoolingClientConnectionManager(schemeRegistry);
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(20);
HttpParams params = new BasicHttpParams();
if (useProxy)
params.setParameter(ConnRoutePNames.DEFAULT_PROXY, new
HttpHost(proxy, 80));
client = new DefaultHttpClient(cm, params);
client.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, new
Integer(3));
client.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
new Integer(3));
client.getParams().setParameter(CoreConnectionPNames.TCP_NODELAY, false);
It's not the first time I'm getting this exception, so I might be able
to reproduce that in a near futur.
What version of HttpClient are you using?
JM
2012/8/1, Oleg Kalnichevski ol...@apache.org:
On Tue, 2012-07-31 at 17:37 -0400, Jean-Marc Spaggiari wrote:
Hi,
I'm getting the exception below when I'm trying to load
http://www.expedia.ca/Sunset-District-Hotels.d163838.Travel-Guide-Hotels
... It's not critical since I'm catching everything, but I'm wondering
if this is normal or if this is something which need to be fixed...
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196)
at
org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:109)
at
org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:176)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.spaggiari.distparser.CrawlerThread$1.run(CrawlerThread.java:267)
--
JM
Jean-Marc
I am not able to reproduce the problem just by executing HTTP GET to the
URL in question. All looks perfectly normal.
[DEBUG] BasicClientConnectionManager - Get connection for route
{}-http://www.expedia.ca
[DEBUG] DefaultClientConnectionOperator - Connecting to
www.expedia.ca:80
[DEBUG] RequestAddCookies - CookieSpec selected: best-match
[DEBUG] RequestAuthCache - Auth cache not set in the context
[DEBUG] RequestTargetAuthentication - Target auth state: UNCHALLENGED
[DEBUG] RequestProxyAuthentication - Proxy auth state: UNCHALLENGED
[DEBUG] DefaultHttpClient - Attempt 1 to execute request
[DEBUG] DefaultClientConnection - Sending request:
GET /Sunset-District-Hotels.d163838.Travel-Guide-Hotels HTTP/1.1
[DEBUG] headers -
GET /Sunset-District-Hotels.d163838.Travel-Guide-Hotels HTTP/1.1
[DEBUG] headers - Host: www.expedia.ca
[DEBUG] headers - Connection: Keep-Alive
[DEBUG] headers - User-Agent: Apache-HttpClient/4.2.1 (java 1.5)
[DEBUG] DefaultClientConnection - Receiving response: HTTP/1.1 200 OK
[DEBUG] headers - HTTP/1.1 200 OK
[DEBUG] headers - Content-Type: text/html;charset=UTF-8
[DEBUG] headers - Content-Language: en-CA
[DEBUG] headers - P3P: policyref=/w3c/p3p.xml, CP=CAO DSP IND COR
ADM CONo CUR CUSi DEV PSA PSD DELi OUR COM NAV PHY ONL PUR UNI
[DEBUG] headers - RTSS: 1
[DEBUG] headers - Date: Wed, 01 Aug 2012 10:49:28 GMT
[DEBUG] headers - Transfer-Encoding: chunked
[DEBUG] headers - Connection: keep-alive
[DEBUG] headers - Connection: Transfer-Encoding
[DEBUG] headers - Set-Cookie: SSLB=1; path=/; domain=.expedia.ca
[DEBUG] headers - Set-Cookie:
MC1=GUID=c55cfa1393ce4bfca1ac5b6532085717; Domain=.expedia.ca;
Expires=Fri, 04-Aug-2017 03:44:09 GMT; Path=/
[DEBUG] headers - Set-Cookie: iEAPID=00,; Domain=.expedia.ca;
Path=/
[DEBUG] headers - Set-Cookie:
JSESSION=13201604-6f17-4f6e-8b03-570fd433ad4e; Domain=.expedia.ca;
Path=/
[DEBUG] headers - Set-Cookie: s1=`0; Domain=.expedia.ca; Path=/
[DEBUG] headers - Set-Cookie: p1=`tpid=v.1,4`linfo=v.4,|0|0|255|1|
04105|0|0||0|0|0|-1|-1`63; Domain=.expedia.ca; Expires=Tue,
01-Aug-2017 15:54:40 GMT; Path=/
[DEBUG] ResponseProcessCookies - Cookie accepted: [version: 0][name:
SSLB][value: 1][domain: .expedia.ca][path: /][expiry: null].
[DEBUG] ResponseProcessCookies - Cookie accepted: [version: 0][name:
MC1][value:
Re: NullPointerException in BestMatchSpec.formatCookies
I'm using version 4.2.1 binary distribution.
2012/8/2, sebb seb...@gmail.com:
On 1 August 2012 12:08, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
Hi Oleg,
My application is reading many other URLs before this one. So maybe
it's a combination of multiple loads.
I will try to dump more when I will get the issue again.
In the meantime, here is how I'm initializing the HttpClient:
// Creation of the HTTP Client
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme(http, 80,
PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(new Scheme(https, 443,
SSLSocketFactory.getSocketFactory()));
PoolingClientConnectionManager cm = new
PoolingClientConnectionManager(schemeRegistry);
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(20);
HttpParams params = new BasicHttpParams();
if (useProxy)
params.setParameter(ConnRoutePNames.DEFAULT_PROXY, new
HttpHost(proxy, 80));
client = new DefaultHttpClient(cm, params);
client.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, new
Integer(3));
client.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
new Integer(3));
client.getParams().setParameter(CoreConnectionPNames.TCP_NODELAY, false);
It's not the first time I'm getting this exception, so I might be able
to reproduce that in a near futur.
What version of HttpClient are you using?
JM
2012/8/1, Oleg Kalnichevski ol...@apache.org:
On Tue, 2012-07-31 at 17:37 -0400, Jean-Marc Spaggiari wrote:
Hi,
I'm getting the exception below when I'm trying to load
http://www.expedia.ca/Sunset-District-Hotels.d163838.Travel-Guide-Hotels
... It's not critical since I'm catching everything, but I'm wondering
if this is normal or if this is something which need to be fixed...
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196)
at
org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:109)
at
org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:176)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.spaggiari.distparser.CrawlerThread$1.run(CrawlerThread.java:267)
--
JM
Jean-Marc
I am not able to reproduce the problem just by executing HTTP GET to the
URL in question. All looks perfectly normal.
[DEBUG] BasicClientConnectionManager - Get connection for route
{}-http://www.expedia.ca
[DEBUG] DefaultClientConnectionOperator - Connecting to
www.expedia.ca:80
[DEBUG] RequestAddCookies - CookieSpec selected: best-match
[DEBUG] RequestAuthCache - Auth cache not set in the context
[DEBUG] RequestTargetAuthentication - Target auth state: UNCHALLENGED
[DEBUG] RequestProxyAuthentication - Proxy auth state: UNCHALLENGED
[DEBUG] DefaultHttpClient - Attempt 1 to execute request
[DEBUG] DefaultClientConnection - Sending request:
GET /Sunset-District-Hotels.d163838.Travel-Guide-Hotels HTTP/1.1
[DEBUG] headers -
GET /Sunset-District-Hotels.d163838.Travel-Guide-Hotels HTTP/1.1
[DEBUG] headers - Host: www.expedia.ca
[DEBUG] headers - Connection: Keep-Alive
[DEBUG] headers - User-Agent: Apache-HttpClient/4.2.1 (java 1.5)
[DEBUG] DefaultClientConnection - Receiving response: HTTP/1.1 200 OK
[DEBUG] headers - HTTP/1.1 200 OK
[DEBUG] headers - Content-Type: text/html;charset=UTF-8
[DEBUG] headers - Content-Language: en-CA
[DEBUG] headers - P3P: policyref=/w3c/p3p.xml, CP=CAO DSP IND COR
ADM CONo CUR CUSi DEV PSA PSD DELi OUR COM NAV PHY ONL PUR UNI
[DEBUG] headers - RTSS: 1
[DEBUG] headers - Date: Wed, 01 Aug 2012 10:49:28 GMT
[DEBUG] headers - Transfer-Encoding: chunked
[DEBUG] headers - Connection: keep-alive
[DEBUG] headers - Connection: Transfer-Encoding
[DEBUG] headers - Set-Cookie: SSLB=1; path=/; domain=.expedia.ca
[DEBUG] headers - Set-Cookie:
MC1=GUID=c55cfa1393ce4bfca1ac5b6532085717; Domain=.expedia.ca;
Expires=Fri, 04-Aug-2017 03:44:09 GMT; Path=/
[DEBUG] headers - Set-Cookie: iEAPID=00,; Domain=.expedia.ca;
Path=/
[DEBUG] headers - Set-Cookie:
JSESSION=13201604-6f17-4f6e-8b03-570fd433ad4e; Domain=.expedia.ca;
Path=/
[DEBUG] headers - Set-Cookie: s1=`0; Domain=.expedia.ca; Path=/
[DEBUG] headers - Set-Cookie: p1=`tpid=v.1,4`linfo=v.4,|0|0|255|1|
04105|0|0||0|0|0|-1|-1`63; Domain=.expedia.ca; Expires=Tue,
01-Aug-2017 15:54:40 GMT; Path=/
[DEBUG] ResponseProcessCookies - Cookie accepted: [version: 0][name:
SSLB][value: 1][domain: .expedia.ca][path: /][expiry: null].
Re: NullPointerException in BestMatchSpec.formatCookies
On 2 August 2012 19:03, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote:
I'm using version 4.2.1 binary distribution.
In which case the NPE is very odd.
The stack dump shows:
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196)
Now the code in question is:
BestMatchSpec.java:193: if (cookie.getVersion() version) {
AFAICT, this can only cause NPE if cookie is null.
The cookie variable is derived from the list created in RequestAddCookies.java:
ListCookie cookies = new ArrayListCookie(cookieStore.getCookies());
// Find cookies matching the given origin
ListCookie matchedCookies = new ArrayListCookie();
Date now = new Date();
for (Cookie cookie : cookies) {
if (!cookie.isExpired(now)) { // == cookie cannot be null here
if (cookieSpec.match(cookie, cookieOrigin)) {
if (this.log.isDebugEnabled()) {
this.log.debug(Cookie + cookie + match
+ cookieOrigin);
}
matchedCookies.add(cookie); // == so cannot be null here
}
} else {
if (this.log.isDebugEnabled()) {
this.log.debug(Cookie + cookie + expired);
}
}
}
// Generate Cookie request headers
if (!matchedCookies.isEmpty()) {
ListHeader headers =
cookieSpec.formatCookies(matchedCookies); // == 196
for (Header header : headers) {
request.addHeader(header);
}
}
I cannot see how a null cookie can be added to the list, nor how a
list entry could be set to null later.
It would be worth enabling Debug logging for the RequestAddCookies class.
What JVM are you using?
2012/8/2, sebb seb...@gmail.com:
On 1 August 2012 12:08, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
Hi Oleg,
My application is reading many other URLs before this one. So maybe
it's a combination of multiple loads.
I will try to dump more when I will get the issue again.
In the meantime, here is how I'm initializing the HttpClient:
// Creation of the HTTP Client
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme(http, 80,
PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(new Scheme(https, 443,
SSLSocketFactory.getSocketFactory()));
PoolingClientConnectionManager cm = new
PoolingClientConnectionManager(schemeRegistry);
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(20);
HttpParams params = new BasicHttpParams();
if (useProxy)
params.setParameter(ConnRoutePNames.DEFAULT_PROXY, new
HttpHost(proxy, 80));
client = new DefaultHttpClient(cm, params);
client.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, new
Integer(3));
client.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
new Integer(3));
client.getParams().setParameter(CoreConnectionPNames.TCP_NODELAY, false);
It's not the first time I'm getting this exception, so I might be able
to reproduce that in a near futur.
What version of HttpClient are you using?
JM
2012/8/1, Oleg Kalnichevski ol...@apache.org:
On Tue, 2012-07-31 at 17:37 -0400, Jean-Marc Spaggiari wrote:
Hi,
I'm getting the exception below when I'm trying to load
http://www.expedia.ca/Sunset-District-Hotels.d163838.Travel-Guide-Hotels
... It's not critical since I'm catching everything, but I'm wondering
if this is normal or if this is something which need to be fixed...
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196)
at
org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:109)
at
org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:176)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
org.spaggiari.distparser.CrawlerThread$1.run(CrawlerThread.java:267)
--
JM
Jean-Marc
I am not able to reproduce the problem just by executing HTTP GET to the
URL in question. All looks perfectly normal.
[DEBUG] BasicClientConnectionManager - Get connection for route
{}-http://www.expedia.ca
[DEBUG] DefaultClientConnectionOperator - Connecting to
www.expedia.ca:80
[DEBUG] RequestAddCookies - CookieSpec selected: best-match
[DEBUG] RequestAuthCache - Auth cache not set in
Re: NullPointerException in BestMatchSpec.formatCookies
I'm using 1.7.0_05 from Sun.
I have enabled the RequestAddCookies debug logs and will see when I
will get the next NPE.
One the line 193 (I don't have the source) is version an int? Or an Integer?
I agree with you, cookie can't be null, else we will have got an NPE
at if (!matchedCookies.isEmpty())
I will restart my tool with the debug mode enabled and keep you posted.
2012/8/2, sebb seb...@gmail.com:
On 2 August 2012 19:03, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
I'm using version 4.2.1 binary distribution.
In which case the NPE is very odd.
The stack dump shows:
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196)
Now the code in question is:
BestMatchSpec.java:193: if (cookie.getVersion() version) {
AFAICT, this can only cause NPE if cookie is null.
The cookie variable is derived from the list created in
RequestAddCookies.java:
ListCookie cookies = new
ArrayListCookie(cookieStore.getCookies());
// Find cookies matching the given origin
ListCookie matchedCookies = new ArrayListCookie();
Date now = new Date();
for (Cookie cookie : cookies) {
if (!cookie.isExpired(now)) { // == cookie cannot be null here
if (cookieSpec.match(cookie, cookieOrigin)) {
if (this.log.isDebugEnabled()) {
this.log.debug(Cookie + cookie + match
+ cookieOrigin);
}
matchedCookies.add(cookie); // == so cannot be null
here
}
} else {
if (this.log.isDebugEnabled()) {
this.log.debug(Cookie + cookie + expired);
}
}
}
// Generate Cookie request headers
if (!matchedCookies.isEmpty()) {
ListHeader headers =
cookieSpec.formatCookies(matchedCookies); // == 196
for (Header header : headers) {
request.addHeader(header);
}
}
I cannot see how a null cookie can be added to the list, nor how a
list entry could be set to null later.
It would be worth enabling Debug logging for the RequestAddCookies class.
What JVM are you using?
2012/8/2, sebb seb...@gmail.com:
On 1 August 2012 12:08, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
Hi Oleg,
My application is reading many other URLs before this one. So maybe
it's a combination of multiple loads.
I will try to dump more when I will get the issue again.
In the meantime, here is how I'm initializing the HttpClient:
// Creation of the HTTP Client
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme(http, 80,
PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(new Scheme(https, 443,
SSLSocketFactory.getSocketFactory()));
PoolingClientConnectionManager cm = new
PoolingClientConnectionManager(schemeRegistry);
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(20);
HttpParams params = new BasicHttpParams();
if (useProxy)
params.setParameter(ConnRoutePNames.DEFAULT_PROXY, new
HttpHost(proxy, 80));
client = new DefaultHttpClient(cm, params);
client.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, new
Integer(3));
client.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
new Integer(3));
client.getParams().setParameter(CoreConnectionPNames.TCP_NODELAY,
false);
It's not the first time I'm getting this exception, so I might be able
to reproduce that in a near futur.
What version of HttpClient are you using?
JM
2012/8/1, Oleg Kalnichevski ol...@apache.org:
On Tue, 2012-07-31 at 17:37 -0400, Jean-Marc Spaggiari wrote:
Hi,
I'm getting the exception below when I'm trying to load
http://www.expedia.ca/Sunset-District-Hotels.d163838.Travel-Guide-Hotels
... It's not critical since I'm catching everything, but I'm
wondering
if this is normal or if this is something which need to be fixed...
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196)
at
org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:109)
at
org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:176)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at
Re: NullPointerException in BestMatchSpec.formatCookies
On 2 August 2012 19:36, Jean-Marc Spaggiari jean-m...@spaggiari.org wrote:
I'm using 1.7.0_05 from Sun.
I have enabled the RequestAddCookies debug logs and will see when I
will get the next NPE.
One the line 193 (I don't have the source) is version an int? Or an Integer?
int, see:
http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/impl/cookie/BestMatchSpec.html#193
and
http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/cookie/Cookie.html#126
I agree with you, cookie can't be null, else we will have got an NPE
at if (!matchedCookies.isEmpty())
I will restart my tool with the debug mode enabled and keep you posted.
2012/8/2, sebb seb...@gmail.com:
On 2 August 2012 19:03, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
I'm using version 4.2.1 binary distribution.
In which case the NPE is very odd.
The stack dump shows:
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196)
Now the code in question is:
BestMatchSpec.java:193: if (cookie.getVersion() version) {
AFAICT, this can only cause NPE if cookie is null.
The cookie variable is derived from the list created in
RequestAddCookies.java:
ListCookie cookies = new
ArrayListCookie(cookieStore.getCookies());
// Find cookies matching the given origin
ListCookie matchedCookies = new ArrayListCookie();
Date now = new Date();
for (Cookie cookie : cookies) {
if (!cookie.isExpired(now)) { // == cookie cannot be null here
if (cookieSpec.match(cookie, cookieOrigin)) {
if (this.log.isDebugEnabled()) {
this.log.debug(Cookie + cookie + match
+ cookieOrigin);
}
matchedCookies.add(cookie); // == so cannot be null
here
}
} else {
if (this.log.isDebugEnabled()) {
this.log.debug(Cookie + cookie + expired);
}
}
}
// Generate Cookie request headers
if (!matchedCookies.isEmpty()) {
ListHeader headers =
cookieSpec.formatCookies(matchedCookies); // == 196
for (Header header : headers) {
request.addHeader(header);
}
}
I cannot see how a null cookie can be added to the list, nor how a
list entry could be set to null later.
It would be worth enabling Debug logging for the RequestAddCookies class.
What JVM are you using?
2012/8/2, sebb seb...@gmail.com:
On 1 August 2012 12:08, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
Hi Oleg,
My application is reading many other URLs before this one. So maybe
it's a combination of multiple loads.
I will try to dump more when I will get the issue again.
In the meantime, here is how I'm initializing the HttpClient:
// Creation of the HTTP Client
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme(http, 80,
PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(new Scheme(https, 443,
SSLSocketFactory.getSocketFactory()));
PoolingClientConnectionManager cm = new
PoolingClientConnectionManager(schemeRegistry);
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(20);
HttpParams params = new BasicHttpParams();
if (useProxy)
params.setParameter(ConnRoutePNames.DEFAULT_PROXY, new
HttpHost(proxy, 80));
client = new DefaultHttpClient(cm, params);
client.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, new
Integer(3));
client.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
new Integer(3));
client.getParams().setParameter(CoreConnectionPNames.TCP_NODELAY,
false);
It's not the first time I'm getting this exception, so I might be able
to reproduce that in a near futur.
What version of HttpClient are you using?
JM
2012/8/1, Oleg Kalnichevski ol...@apache.org:
On Tue, 2012-07-31 at 17:37 -0400, Jean-Marc Spaggiari wrote:
Hi,
I'm getting the exception below when I'm trying to load
http://www.expedia.ca/Sunset-District-Hotels.d163838.Travel-Guide-Hotels
... It's not critical since I'm catching everything, but I'm
wondering
if this is normal or if this is something which need to be fixed...
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196)
at
org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:109)
at
org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:176)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515)
at
Re: NullPointerException in BestMatchSpec.formatCookies
So it's can be that neither... Thanks for the links.
I have restarted the application. I have added the debug level for
RequestAddCookies and asked the application to stop as soon as there
is an exception. I just need to let it run for some time. Last time I
saw the exception it was running for few days. So it might take time
before it happens again.
2012/8/2, sebb seb...@gmail.com:
On 2 August 2012 19:36, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
I'm using 1.7.0_05 from Sun.
I have enabled the RequestAddCookies debug logs and will see when I
will get the next NPE.
One the line 193 (I don't have the source) is version an int? Or an
Integer?
int, see:
http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/impl/cookie/BestMatchSpec.html#193
and
http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/cookie/Cookie.html#126
I agree with you, cookie can't be null, else we will have got an NPE
at if (!matchedCookies.isEmpty())
I will restart my tool with the debug mode enabled and keep you posted.
2012/8/2, sebb seb...@gmail.com:
On 2 August 2012 19:03, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
I'm using version 4.2.1 binary distribution.
In which case the NPE is very odd.
The stack dump shows:
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
org.apache.http.client.protocol.RequestAddCookies.process(RequestAddCookies.java:196)
Now the code in question is:
BestMatchSpec.java:193: if (cookie.getVersion() version) {
AFAICT, this can only cause NPE if cookie is null.
The cookie variable is derived from the list created in
RequestAddCookies.java:
ListCookie cookies = new
ArrayListCookie(cookieStore.getCookies());
// Find cookies matching the given origin
ListCookie matchedCookies = new ArrayListCookie();
Date now = new Date();
for (Cookie cookie : cookies) {
if (!cookie.isExpired(now)) { // == cookie cannot be null
here
if (cookieSpec.match(cookie, cookieOrigin)) {
if (this.log.isDebugEnabled()) {
this.log.debug(Cookie + cookie + match
+ cookieOrigin);
}
matchedCookies.add(cookie); // == so cannot be null
here
}
} else {
if (this.log.isDebugEnabled()) {
this.log.debug(Cookie + cookie + expired);
}
}
}
// Generate Cookie request headers
if (!matchedCookies.isEmpty()) {
ListHeader headers =
cookieSpec.formatCookies(matchedCookies); // == 196
for (Header header : headers) {
request.addHeader(header);
}
}
I cannot see how a null cookie can be added to the list, nor how a
list entry could be set to null later.
It would be worth enabling Debug logging for the RequestAddCookies
class.
What JVM are you using?
2012/8/2, sebb seb...@gmail.com:
On 1 August 2012 12:08, Jean-Marc Spaggiari jean-m...@spaggiari.org
wrote:
Hi Oleg,
My application is reading many other URLs before this one. So maybe
it's a combination of multiple loads.
I will try to dump more when I will get the issue again.
In the meantime, here is how I'm initializing the HttpClient:
// Creation of the HTTP Client
SchemeRegistry schemeRegistry = new SchemeRegistry();
schemeRegistry.register(new Scheme(http, 80,
PlainSocketFactory.getSocketFactory()));
schemeRegistry.register(new Scheme(https, 443,
SSLSocketFactory.getSocketFactory()));
PoolingClientConnectionManager cm = new
PoolingClientConnectionManager(schemeRegistry);
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(20);
HttpParams params = new BasicHttpParams();
if (useProxy)
params.setParameter(ConnRoutePNames.DEFAULT_PROXY, new
HttpHost(proxy, 80));
client = new DefaultHttpClient(cm, params);
client.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, new
Integer(3));
client.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT,
new Integer(3));
client.getParams().setParameter(CoreConnectionPNames.TCP_NODELAY,
false);
It's not the first time I'm getting this exception, so I might be
able
to reproduce that in a near futur.
What version of HttpClient are you using?
JM
2012/8/1, Oleg Kalnichevski ol...@apache.org:
On Tue, 2012-07-31 at 17:37 -0400, Jean-Marc Spaggiari wrote:
Hi,
I'm getting the exception below when I'm trying to load
http://www.expedia.ca/Sunset-District-Hotels.d163838.Travel-Guide-Hotels
... It's not critical since I'm catching everything, but I'm
wondering
if this is normal or if this is something which need to be fixed...
java.lang.NullPointerException
at
org.apache.http.impl.cookie.BestMatchSpec.formatCookies(BestMatchSpec.java:193)
at
