[
https://issues.apache.org/jira/browse/CASSANDRA-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-9019:
--------------------------------------
Attachment: 9019.txt
In this case, I think it's better to return string "N/A" (not available) rather
than throwing RuntimeException when Stages are not initialized and JMX
instances are not ready, so that we still can display it in StatusLogger output.
I cannot reproduce with GC, but I used {{nodetool tpstats}} because it uses the
same method in ThreadPoolMetrics. If you do {{nodetool tpstats}} right after
the node starts, you get similar error as follows:
{code}
$ bin/nodetool -p 7100 tpstats
Pool Name Active Pending Completed Blocked All
time blocked
error:
org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
-- StackTrace --
javax.management.InstanceNotFoundException:
org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:643)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1464)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
-- snip --
at
org.apache.cassandra.metrics.ThreadPoolMetrics.getJmxMetric(ThreadPoolMetrics.java:123)
at
org.apache.cassandra.tools.NodeProbe.getThreadPoolMetric(NodeProbe.java:1000)
at
org.apache.cassandra.tools.NodeTool$TpStats.execute(NodeTool.java:2630)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:256)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:170)
{code}
After the patch, it shows "N/A" for not yet ready thread pools.
> GCInspector detected GC before ThreadPools are initialized
> ----------------------------------------------------------
>
> Key: CASSANDRA-9019
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9019
> Project: Cassandra
> Issue Type: Bug
> Reporter: Philip Thompson
> Assignee: Yuki Morishita
> Fix For: 3.0
>
> Attachments: 9019.txt
>
>
> While running the dtest {{one_all_test (consistency_test.TestConsistency)}},
> I ran into the following exception:
> {code}
> java.lang.RuntimeException: Error reading:
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
> at
> org.apache.cassandra.metrics.ThreadPoolMetrics.getJmxMetric(ThreadPoolMetrics.java:134)
> at org.apache.cassandra.utils.StatusLogger.log(StatusLogger.java:55)
> at
> org.apache.cassandra.service.GCInspector.handleNotification(GCInspector.java:147)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor$ListenerWrapper.handleNotification(DefaultMBeanServerInterceptor.java:1754)
> at
> sun.management.NotificationEmitterSupport.sendNotification(NotificationEmitterSupport.java:156)
> at
> sun.management.GarbageCollectorImpl.createGCNotification(GarbageCollectorImpl.java:150)
> Caused by: java.lang.reflect.UndeclaredThrowableException
> at com.sun.proxy.$Proxy3.getValue(Unknown Source)
> at
> org.apache.cassandra.metrics.ThreadPoolMetrics.getJmxMetric(ThreadPoolMetrics.java:123)
> ... 5 more
> Caused by: javax.management.InstanceNotFoundException:
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:643)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
> at
> javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:267)
> ... 7 more
>
> {code}
> Dtest didn't preserve the logs, which implies that this wasn't in the
> system.log, but printed to stderr somehow, it's unclear with all the piping
> dtest and ccm do. I have yet to reproduce the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
