[ https://issues.apache.org/jira/browse/CASSANDRA-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517038#comment-15517038 ]
Simon Zhou commented on CASSANDRA-12462: ---------------------------------------- I don't see a schema mismatch, at least from the output of command "nodetool describecluster". There should be new or removed keyspaces/tables recently because it's a shared dev cluster that's being actively used. But I don't know how to get the keyspace/table creation/deletion time. > NullPointerException in CompactionInfo.getId(CompactionInfo.java:65) > -------------------------------------------------------------------- > > Key: CASSANDRA-12462 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12462 > Project: Cassandra > Issue Type: Bug > Components: Compaction > Reporter: Jonathan DePrizio > Attachments: 0001-Fix-NPE-when-running-nodetool-compactionstats.patch > > > Note: The same trace is cited in the last comment of > https://issues.apache.org/jira/browse/CASSANDRA-11961 > I've noticed that some of my nodes in my 2.1 cluster have fallen way behind > on compactions, and have huge numbers (thousands) of uncompacted, tiny > SSTables (~30MB or so). > In diagnosing the issue, I've found that "nodetool compactionstats" returns > the exception below. Restarting cassandra on the node here causes the > pending tasks count to jump to ~2000. Compactions run properly for about an > hour, until this exception occurs again. Once it occurs, I see the pending > tasks value rapidly drop towards zero, but without any compactions actually > running (the logs show no compactions finishing). It would seem that this is > causing compactions to fail on this node, which is leading to it running out > of space, etc. > [redacted]# nodetool compactionstats > xss = -ea -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar > -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms12G -Xmx12G > -Xmn1000M -Xss255k > pending tasks: 5 > error: null > -- StackTrace -- > java.lang.NullPointerException > at > org.apache.cassandra.db.compaction.CompactionInfo.getId(CompactionInfo.java:65) > at > org.apache.cassandra.db.compaction.CompactionInfo.asMap(CompactionInfo.java:118) > at > org.apache.cassandra.db.compaction.CompactionManager.getCompactions(CompactionManager.java:1405) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at sun.reflect.misc.Trampoline.invoke(Unknown Source) > at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at sun.reflect.misc.MethodUtil.invoke(Unknown Source) > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown > Source) > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown > Source) > at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source) > at com.sun.jmx.mbeanserver.PerInterface.getAttribute(Unknown Source) > at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(Unknown Source) > at > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown > Source) > at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source) > at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown > Source) > at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown > Source) > at > javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown > Source) > at > javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown > Source) > at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown > Source) > at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source) > at sun.rmi.transport.Transport$1.run(Unknown Source) > at sun.rmi.transport.Transport$1.run(Unknown Source) > at java.security.AccessController.doPrivileged(Native Method) > at sun.rmi.transport.Transport.serviceCall(Unknown Source) > at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source) > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown > Source) > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown > Source) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) -- This message was sent by Atlassian JIRA (v6.3.4#6332)