[jira] [Comment Edited] (CASSANDRA-15263) LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null
[ https://issues.apache.org/jira/browse/CASSANDRA-15263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903727#comment-16903727 ] Benedict edited comment on CASSANDRA-15263 at 8/9/19 9:06 AM: -- Thanks. The sample system log appears to have had the message truncated though? I'm not sure if the ASF has a secure system to upload to - perhaps somebody in the community knows better? [~ferozshaik...@gmail.com]: if you have somewhere you can upload it, and provide me access, that might be easier though. Otherwise I'll see if I can find some other mechanism. was (Author: benedict): Thanks [~ferozshaik...@gmail.com]. The sample system log appears to have had the message truncated? > LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null > --- > > Key: CASSANDRA-15263 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15263 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: feroz shaik >Assignee: Benedict >Priority: Normal > Labels: 2.1.16, 3.11.4 > Attachments: sample.system.log, schema.txt, stack_trace.txt > > > We have hit a problem today while upgrading from 2.1.16 to 3.11.4. > we encountered this as soon as the first node started up with 3.11.4 > The full error stack is attached - [^stack_trace.txt] > > The below errors continued in the log file as long as the process was up. > ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454 > > The nodetool version says 3.11.4 and the no of connections on native por t- > 9042 was similar to other nodes. The exceptions were scary that we had to > call off the change. Any help and insights to this problem from the community > is appreciated. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15263) LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null
[ https://issues.apache.org/jira/browse/CASSANDRA-15263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901760#comment-16901760 ] feroz shaik edited comment on CASSANDRA-15263 at 8/7/19 6:23 AM: - Schema is attached - [^schema.txt] ^We current use CQL . Legacy mode was thrift.^ This cluster is one of the oldest and very first Cassandra implementations. so pretty legacy data in there.We also witness that it has tombstones for certain tables . We see them logged in the system.log. Could it be this specific behavior of converting this legacy data combined with tombstones thats causing the problem? Currently, we have removed that node from serving traffic. Thrift, Native tramsport is disabled. We are running upgradesstables on this node to see whether the newer version can re-write all the sstables and somehow get us out of this problem. was (Author: ferozshaik...@gmail.com): Schema is attached - [^schema.txt] ^We current use CQL . Legacy mode was thrift.^ This cluster is one of the oldest and very first Cassandra implementations. so pretty legacy data in there.We also witness that it has tombstones for certain tables . We see them logged in the system.log. Could it be this specific behavior of this legacy data combined with tombstones that causing the problem? Currently, we have removed that node from serving traffic. Thrift, Native tramsport is disabled. We are running upgradesstables on this node to see whether the newer version can re-write all the sstables and somehow get us out of this problem. > LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null > --- > > Key: CASSANDRA-15263 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15263 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: feroz shaik >Priority: Normal > Labels: 2.1.16, 3.11.4 > Attachments: schema.txt, stack_trace.txt > > > We have also hit a problem today while upgrading from 2.1.16 to 3.11.4. > we encountered this as soon as the first node started up with 3.11.4 > The full error stack is attached - [^stack_trace.txt] > > The below errors continued in the log file as long as the process was up. > ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454 > > The nodetool version says 3.11.4 and the no of connections on native por t- > 9042 was similar to other nodes. The exceptions were scary that we had to > call off the change. Any help and insights to this problem from the community > is appreciated. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15263) LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null
[ https://issues.apache.org/jira/browse/CASSANDRA-15263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901760#comment-16901760 ] feroz shaik edited comment on CASSANDRA-15263 at 8/7/19 6:22 AM: - Schema is attached - [^schema.txt] ^We current use CQL . Legacy mode was thrift.^ This cluster is one of the oldest and very first Cassandra implementations. so pretty legacy data in there.We also witness that it has tombstones for certain tables . We see them logged in the system.log. Could it be this specific behavior of this legacy data combined with tombstones that causing the problem? Currently, we have removed that node from serving traffic. Thrift, Native tramsport is disabled. We are running upgradesstables on this node to see whether the newer version can re-write all the sstables and somehow get us out of this problem. was (Author: ferozshaik...@gmail.com): Schema is attached - [^schema.txt] ^We current use CQL . Legacy mode was thrift.^ > LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null > --- > > Key: CASSANDRA-15263 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15263 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: feroz shaik >Priority: Normal > Labels: 2.1.16, 3.11.4 > Attachments: schema.txt, stack_trace.txt > > > We have also hit a problem today while upgrading from 2.1.16 to 3.11.4. > we encountered this as soon as the first node started up with 3.11.4 > The full error stack is attached - [^stack_trace.txt] > > The below errors continued in the log file as long as the process was up. > ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454 > > The nodetool version says 3.11.4 and the no of connections on native por t- > 9042 was similar to other nodes. The exceptions were scary that we had to > call off the change. Any help and insights to this problem from the community > is appreciated. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15263) LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null
[ https://issues.apache.org/jira/browse/CASSANDRA-15263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901760#comment-16901760 ] feroz shaik edited comment on CASSANDRA-15263 at 8/7/19 6:18 AM: - Schema is attached - [^schema.txt] ^We current use CQL . Legacy mode was thrift.^ was (Author: ferozshaik...@gmail.com): Schema is attached - [^schema.txt] > LegacyLayout RangeTombstoneList throws java.lang.NullPointerException: null > --- > > Key: CASSANDRA-15263 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15263 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: feroz shaik >Priority: Normal > Labels: 2.1.16, 3.11.4 > Attachments: schema.txt, stack_trace.txt > > > We have also hit a problem today while upgrading from 2.1.16 to 3.11.4. > we encountered this as soon as the first node started up with 3.11.4 > The full error stack is attached - [^stack_trace.txt] > > The below errors continued in the log file as long as the process was up. > ERROR [Native-Transport-Requests-12] 2019-08-06 03:00:47,135 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-8] 2019-08-06 03:00:48,778 > ErrorMessage.java:384 - Unexpected exception during request > java.lang.NullPointerException: null > ERROR [Native-Transport-Requests-13] 2019-08-06 03:00:57,454 > > The nodetool version says 3.11.4 and the no of connections on native por t- > 9042 was similar to other nodes. The exceptions were scary that we had to > call off the change. Any help and insights to this problem from the community > is appreciated. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
