Re: clearThreadLocalMap and NullPointerException
On 24/11/2010 01:28, Blair Zajac wrote: On Nov 23, 2010, at 4:49 PM, Blair Zajac wrote: On Nov 23, 2010, at 2:24 AM, Mark Thomas wrote: Do let us know what the problem was when you find it. Does JNA provide its own clean up methods that I could run at shutdown? I haven't looked yet to see. I found the cause, it's an issue with JNA, they should be providing a real iterator() implementation, not one that just returns null. Not that they expected anyone to call toString() on it. http://article.gmane.org/gmane.comp.java.jna.user/4347 I guess it's up to you if you want to handle a case like this. I could see other projects not bothering to provide real implementations of things if they don't expect anything to go looking into their internals. Thanks for getting back with details of the problem. I agree with you that there are likely to be other libraries doing similar things. I think the general handling now in place is the right way to go. Hopefully, the issues it highlights in third party libraries they will get reported and fixed. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: clearThreadLocalMap and NullPointerException
On Nov 24, 2010, at 12:41 AM, Mark Thomas wrote: On 24/11/2010 01:28, Blair Zajac wrote: On Nov 23, 2010, at 4:49 PM, Blair Zajac wrote: On Nov 23, 2010, at 2:24 AM, Mark Thomas wrote: Do let us know what the problem was when you find it. Does JNA provide its own clean up methods that I could run at shutdown? I haven't looked yet to see. I found the cause, it's an issue with JNA, they should be providing a real iterator() implementation, not one that just returns null. Not that they expected anyone to call toString() on it. http://article.gmane.org/gmane.comp.java.jna.user/4347 I guess it's up to you if you want to handle a case like this. I could see other projects not bothering to provide real implementations of things if they don't expect anything to go looking into their internals. Thanks for getting back with details of the problem. I agree with you that there are likely to be other libraries doing similar things. I think the general handling now in place is the right way to go. Hopefully, the issues it highlights in third party libraries they will get reported and fixed. JNA just committed a fix for this and with the Tomcat improvement, we're all good. Thanks for the help! Blair - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: clearThreadLocalMap and NullPointerException
On 23/11/2010 02:19, Blair Zajac wrote: Any suggestions in tracking this down? Would a custom Tomcat build that gets each key and value in a try/catch block be useful? Almost certainly. As far as I can tell there is a collections object in a thread local that is non-null yet returns null for a call to iterator(). That seems wrong to me. The root cause could be a bug in the collection class or it might be a side-effect of multiple threads accessing a non-thread-safe collection. Putting the value.toString() calls inside try-catches should make it more robust. I'll do that for 7.0.x and propose it for 6.0.x. Do let us know what the problem was when you find it. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: clearThreadLocalMap and NullPointerException
On Nov 23, 2010, at 2:24 AM, Mark Thomas wrote:
On 23/11/2010 02:19, Blair Zajac wrote:
Any suggestions in tracking this down? Would a custom Tomcat build that
gets each key and value in a try/catch block be useful?
Almost certainly. As far as I can tell there is a collections object in
a thread local that is non-null yet returns null for a call to
iterator(). That seems wrong to me.
The root cause could be a bug in the collection class or it might be a
side-effect of multiple threads accessing a non-thread-safe collection.
Putting the value.toString() calls inside try-catches should make it
more robust. I'll do that for 7.0.x and propose it for 6.0.x.
Do let us know what the problem was when you find it.
I got the type of object by logging the class of the object, it's a
com.sun.jna.Structure$2.StructureSet. I'm using JNA 3.2.7 in my app.
Nov 23, 2010 4:42:51 PM org.apache.catalina.loader.WebappClassLoader
clearThreadLocalMap
SEVERE: Calling toString on a com.sun.jna.Structure$2.StructureSet threw a
NullPointerException.
java.lang.NullPointerException
at java.util.AbstractCollection.toString(AbstractCollection.java:415)
at
org.apache.catalina.loader.WebappClassLoader.clearThreadLocalMap(WebappClassLoader.java:2380)
at
org.apache.catalina.loader.WebappClassLoader.clearReferencesThreadLocals(WebappClassLoader.java:2304)
at
org.apache.catalina.loader.WebappClassLoader.clearReferences(WebappClassLoader.java:1886)
at
org.apache.catalina.loader.WebappClassLoader.stop(WebappClassLoader.java:1798)
at org.apache.catalina.loader.WebappLoader.stop(WebappLoader.java:738)
at
org.apache.catalina.core.StandardContext.stop(StandardContext.java:4812)
at
org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:924)
at
org.apache.catalina.startup.HostConfig.undeployApps(HostConfig.java:1319)
at org.apache.catalina.startup.HostConfig.stop(HostConfig.java:1290)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:323)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.stop(ContainerBase.java:1086)
at org.apache.catalina.core.ContainerBase.stop(ContainerBase.java:1098)
at org.apache.catalina.core.StandardEngine.stop(StandardEngine.java:450)
at
org.apache.catalina.core.StandardService.stop(StandardService.java:587)
at org.apache.catalina.core.StandardServer.stop(StandardServer.java:744)
at org.apache.catalina.startup.Catalina.stop(Catalina.java:648)
at org.apache.catalina.startup.Catalina.start(Catalina.java:615)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Nov 23, 2010 4:42:51 PM org.apache.catalina.loader.WebappClassLoader
clearThreadLocalMap
SEVERE: The web application [/foobar] created a ThreadLocal with key of type
[com.sun.jna.Structure$2] (value [com.sun.jna.structur...@2e4f7bc2]) and a
value of type [com.sun.jna.Structure$2.StructureSet] (value [null]) but failed
to remove it when the web application was stopped. This is very likely to
create a memory leak.
Nov 23, 2010 4:42:51 PM org.apache.catalina.loader.WebappClassLoader
clearThreadLocalMap
SEVERE: The web application [/foobar] created a ThreadLocal with key of type
[com.sun.jna.Native$3] (value [com.sun.jna.nativ...@6446154e]) and a value of
type [java.lang.Integer] (value [2]) but failed to remove it when the web
application was stopped. This is very likely to create a memory leak.
Does JNA provide its own clean up methods that I could run at shutdown? I
haven't looked yet to see.
BTW, there's code in this section:
if (value != null) {
args[3] = value.getClass().getCanonicalName();
args[4] = value.toString();
}
if (value == null) {
if (log.isDebugEnabled()) {
log.debug(sm.getString(
webappClassLoader.clearThreadLocalDebug,
args));
if (clearReferencesThreadLocals) {
log.debug(sm.getString(
webappClassLoader.clearThreadLocalDebugClear
Re: clearThreadLocalMap and NullPointerException
On Nov 23, 2010, at 4:49 PM, Blair Zajac wrote: On Nov 23, 2010, at 2:24 AM, Mark Thomas wrote: On 23/11/2010 02:19, Blair Zajac wrote: Any suggestions in tracking this down? Would a custom Tomcat build that gets each key and value in a try/catch block be useful? Almost certainly. As far as I can tell there is a collections object in a thread local that is non-null yet returns null for a call to iterator(). That seems wrong to me. The root cause could be a bug in the collection class or it might be a side-effect of multiple threads accessing a non-thread-safe collection. Putting the value.toString() calls inside try-catches should make it more robust. I'll do that for 7.0.x and propose it for 6.0.x. Do let us know what the problem was when you find it. Does JNA provide its own clean up methods that I could run at shutdown? I haven't looked yet to see. I found the cause, it's an issue with JNA, they should be providing a real iterator() implementation, not one that just returns null. Not that they expected anyone to call toString() on it. http://article.gmane.org/gmane.comp.java.jna.user/4347 I guess it's up to you if you want to handle a case like this. I could see other projects not bothering to provide real implementations of things if they don't expect anything to go looking into their internals. Blair - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
clearThreadLocalMap and NullPointerException
With Tomcat 6.0.29 and Java 6u22, when I'm stopping Tomcat I'm getting
the attached log, which at the end is a NullPointerException thrown from
AbstractCollection#toString() which is called from clearThreadLocalMap().
java.lang.NullPointerException
at java.util.AbstractCollection.toString(AbstractCollection.java:415)
at
org.apache.catalina.loader.WebappClassLoader.clearThreadLocalMap(WebappClassLoader.java:2379)
at
org.apache.catalina.loader.WebappClassLoader.clearReferencesThreadLocals(WebappClassLoader.java:2304)
at
org.apache.catalina.loader.WebappClassLoader.clearReferences(WebappClassLoader.java:1886)
at
org.apache.catalina.loader.WebappClassLoader.stop(WebappClassLoader.java:1798)
Unfortunately, I can't tell what the key or value is due to this.
The web application does a GC when it unloads ZeroC's Ice runtime to
ensure that all objects that implement #finialize() are properly
finalized, otherwise I get even more exceptions. This is seen in Java's
GC stats in the log.
Any suggestions in tracking this down? Would a custom Tomcat build that
gets each key and value in a try/catch block be useful?
Regards,
Blair
Nov 22, 2010 5:49:06 PM org.apache.coyote.http11.Http11Protocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8080
Nov 22, 2010 5:49:07 PM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
Nov 22, 2010 5:49:07 PM org.apache.catalina.core.ApplicationContext log
INFO: Closing Spring root WebApplicationContext
37.480: [GC [PSYoungGen: 1057265K-9094K(7340032K)]
1058825K-10654K(15728640K), 0.0128710 secs] [Times: user=0.06 sys=0.00,
real=0.01 secs]
37.493: [Full GC (System) [PSYoungGen: 9094K-0K(7340032K)] [ParOldGen:
1560K-10200K(8388608K)] 10654K-10200K(15728640K) [PSPermGen:
45121K-45016K(85184K)], 0.2334650 secs] [Times: user=0.50 sys=0.01, real=0.23
secs]
Nov 22, 2010 5:49:12 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads
SEVERE: The web application [/foobar] appears to have started a thread named
[com.google.common.base.internal.Finalizer] but has failed to stop it. This is
very likely to create a memory leak.
Nov 22, 2010 5:49:12 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads
SEVERE: The web application [/foobar] appears to have started a thread named
[com.google.common.base.internal.Finalizer] but has failed to stop it. This is
very likely to create a memory leak.
Nov 22, 2010 5:49:12 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads
SEVERE: The web application [/foobar] appears to have started a thread named
[Timer-1] but has failed to stop it. This is very likely to create a memory
leak.
Nov 22, 2010 5:49:12 PM org.apache.catalina.loader.WebappClassLoader
clearReferencesThreads
SEVERE: The web application [/foobar] appears to have started a thread named
[Thread-21] but has failed to stop it. This is very likely to create a memory
leak.
Nov 22, 2010 5:49:12 PM org.apache.catalina.loader.WebappClassLoader
clearThreadLocalMap
SEVERE: The web application [/foobar] created a ThreadLocal with key of type
[com.sun.jna.Native$3] (value [com.sun.jna.nativ...@6197cc]) and a value of
type [java.lang.Integer] (value [2]) but failed to remove it when the web
application was stopped. This is very likely to create a memory leak.
Nov 22, 2010 5:49:12 PM org.apache.catalina.loader.WebappClassLoader
clearThreadLocalMap
SEVERE: The web application [/foobar] created a ThreadLocal with key of type
[com.sun.jna.Structure$1] (value [com.sun.jna.structur...@734d246]) and a value
of type [java.util.HashMap] (value [{}]) but failed to remove it when the web
application was stopped. This is very likely to create a memory leak.
Nov 22, 2010 5:49:12 PM org.apache.catalina.startup.HostConfig undeployApps
WARNING: Error while removing context [/foobar]
java.lang.NullPointerException
at java.util.AbstractCollection.toString(AbstractCollection.java:415)
at
org.apache.catalina.loader.WebappClassLoader.clearThreadLocalMap(WebappClassLoader.java:2379)
at
org.apache.catalina.loader.WebappClassLoader.clearReferencesThreadLocals(WebappClassLoader.java:2304)
at
org.apache.catalina.loader.WebappClassLoader.clearReferences(WebappClassLoader.java:1886)
at
org.apache.catalina.loader.WebappClassLoader.stop(WebappClassLoader.java:1798)
at org.apache.catalina.loader.WebappLoader.stop(WebappLoader.java:738)
at
org.apache.catalina.core.StandardContext.stop(StandardContext.java:4812)
at
org.apache.catalina.core.ContainerBase.removeChild(ContainerBase.java:924)
at
org.apache.catalina.startup.HostConfig.undeployApps(HostConfig.java:1319)
at org.apache.catalina.startup.HostConfig.stop(HostConfig.java:1290)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:323)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119
clearThreadLocalMap
These show up in my Catalina logs, not very often - maybe half a dozen per day. Does this mean I didn't clean up my variables correctly in my webapp? May 18, 2010 7:51:01 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [null] (value [javax.faces.context.facescontex...@724356]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Using CATALINA_BASE: C:\apache-tomcat-6.0.24 Using CATALINA_HOME: C:\apache-tomcat-6.0.24 Using CATALINA_TMPDIR: C:\apache-tomcat-6.0.24\temp Using JRE_HOME:C:\Program Files\Java\jdk1.6.0_14 Using CLASSPATH: C:\apache-tomcat-6.0.24\bin\bootstrap.jar Server version: Apache Tomcat/6.0.24 Server built: January 19 2010 1439 Server number: 6.0.0.0 OS Name:Windows 2003 OS Version: 5.2 Architecture: x86 JVM Version:1.6.0_14-b08 JVM Vendor: Sun Microsystems Inc. Leo
Re: clearThreadLocalMap
On 19/05/2010 23:46, Leo Donahue - PLANDEVX wrote: These show up in my Catalina logs, not very often - maybe half a dozen per day. Does this mean I didn't clean up my variables correctly in my webapp? Looks like a false positive. It is fixed in 6.0.26. Mark May 18, 2010 7:51:01 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: A web application created a ThreadLocal with key of type [null] (value [javax.faces.context.facescontex...@724356]) and a value of type [null] (value [null]) but failed to remove it when the web application was stopped. To prevent a memory leak, the ThreadLocal has been forcibly removed. Using CATALINA_BASE: C:\apache-tomcat-6.0.24 Using CATALINA_HOME: C:\apache-tomcat-6.0.24 Using CATALINA_TMPDIR: C:\apache-tomcat-6.0.24\temp Using JRE_HOME:C:\Program Files\Java\jdk1.6.0_14 Using CLASSPATH: C:\apache-tomcat-6.0.24\bin\bootstrap.jar Server version: Apache Tomcat/6.0.24 Server built: January 19 2010 1439 Server number: 6.0.0.0 OS Name:Windows 2003 OS Version: 5.2 Architecture: x86 JVM Version:1.6.0_14-b08 JVM Vendor: Sun Microsystems Inc. Leo - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
