[
https://issues.apache.org/jira/browse/CASSANDRA-16941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412894#comment-17412894
]
Mohsen Zohrevandi commented on CASSANDRA-16941:
-----------------------------------------------
Here are the last few lines of logs before the pod was restarted (nodetool
status was used in a Kubernetes livenessProbe that failed due to NPE):
{code:java}
INFO [GossipTasks:1] 2021-09-09 23:59:32,486 Gossiper.java:894 - FatClient
/10.244.47.73 has been silent for 30000ms, removing from gossip
INFO [Service Thread] 2021-09-10 00:00:11,050 GCInspector.java:285 - ParNew GC
in 373ms. CMS Old Gen: 2433961072 -> 2495651632; Par Eden Space: 335544320 ->
0; Par Survivor Space: 35273304 -> 41943040
INFO [Service Thread] 2021-09-10 00:00:11,975 GCInspector.java:285 - ParNew GC
in 426ms. CMS Old Gen: 2495652488 -> 2571204104; Par Eden Space: 335544320 ->
0;
INFO [Service Thread] 2021-09-10 00:00:12,880 GCInspector.java:285 - ParNew GC
in 389ms. CMS Old Gen: 2571204264 -> 2643651528; Par Eden Space: 335544320 ->
0;
INFO [Service Thread] 2021-09-10 00:00:13,649 GCInspector.java:285 - ParNew GC
in 339ms. CMS Old Gen: 2643651528 -> 2710161032; Par Eden Space: 333721088 ->
0;
INFO [HintsDispatcher:129] 2021-09-10 00:03:16,564 HintsStore.java:133 -
Deleted hint file 770229d7-1412-4d46-8a2f-50fdf51df82c-1631216119494-1.hints
INFO [HintsDispatcher:129] 2021-09-10 00:03:16,564
HintsDispatchExecutor.java:282 - Finished hinted handoff of file
770229d7-1412-4d46-8a2f-50fdf51df82c-1631216119494-1.hints to endpoint
/10.244.38.27: 770229d7-1412-4d46-8a2f-50fdf51df82c
INFO [HintsDispatcher:128] 2021-09-10 00:04:32,423 HintsStore.java:133 -
Deleted hint file c67cdbd7-194c-48df-b638-7b33f88c173b-1631217661409-1.hints
INFO [HintsDispatcher:128] 2021-09-10 00:04:32,423
HintsDispatchExecutor.java:282 - Finished hinted handoff of file
c67cdbd7-194c-48df-b638-7b33f88c173b-1631217661409-1.hints to endpoint
/10.244.46.145: c67cdbd7-194c-48df-b638-7b33f88c173b
{code}
> NullPointerException on nodetool status
> ---------------------------------------
>
> Key: CASSANDRA-16941
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16941
> Project: Cassandra
> Issue Type: Bug
> Reporter: Mohsen Zohrevandi
> Priority: Normal
>
> Sometimes {{nodetool status}} fails with the following error:
> {code:java}
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:4871)
> at
> org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The cluster is running Cassandra 3.11.10 and nothing in Cassandra logs
> indicates any errors. The particular node where the error happened was
> running for 7 days. I believe the same NPE happens randomly on other nodes as
> well.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
