[jira] [Commented] (CASSANDRA-11489) DynamicCompositeType failures during 2.1 to 3.0 upgrade.
[
https://issues.apache.org/jira/browse/CASSANDRA-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308470#comment-15308470
]
Aleksey Yeschenko commented on CASSANDRA-11489:
---
So, the ticket had nothing to do with DCT in the end, and instead with two
separate upgrade issues - CASSANDRA-11930 and CASSANDRA-11932. I'm closing this
ticket in favour of the mentioned two, them being more specific and easily
reproducible individually.
> DynamicCompositeType failures during 2.1 to 3.0 upgrade.
>
>
> Key: CASSANDRA-11489
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11489
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.x
>
>
> When upgrading from 2.1.13 to 3.0.4+some (hash
> 70eab633f289eb1e4fbe47b3e17ff3203337f233) we are seeing the following
> exceptions on 2.1 nodes after other nodes have been upgraded. With tables
> using DynamicCompositeType in use. The workload runs fine once everything is
> upgraded.
> {code}
> ERROR [MessagingService-Incoming-/10.200.182.2] 2016-04-03 21:49:10,531
> CassandraDaemon.java:229 - Exception in thread
> Thread[MessagingService-Incoming-/10.200.182.2,5,main]
> java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input
> length = 1
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:181)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:200)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:54)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.SimpleSparseCellNameType.fromByteBuffer(SimpleSparseCellNameType.java:83)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.RangeTombstoneList$Serializer.deserialize(RangeTombstoneList.java:843)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.DeletionInfo$Serializer.deserialize(DeletionInfo.java:407)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:105)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:89)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.db.Row$RowSerializer.deserialize(Row.java:73)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:116)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:195)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:172)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
> at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
> ~[na:1.8.0_40]
> at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
> ~[na:1.8.0_40]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:152)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:109)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:169)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> ... 16 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11489) DynamicCompositeType failures during 2.1 to 3.0 upgrade.
[
https://issues.apache.org/jira/browse/CASSANDRA-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308448#comment-15308448
]
Aleksey Yeschenko commented on CASSANDRA-11489:
---
The second issue is about mixed and dynamic Thrift CFs deletions of the dynamic
component being translated into range tombstones in 3.0 -> 2.1 direction. On
3.0 side they are whole row deletions, and {{LegacyLayout::fromRow(CFMetaData,
Row)}} turns them into range tombstones even for non-composite, dynamic and
mixed Thrift CFs. They should be converted into single-cell deletions instead.
This is not a correctness problem, strictly speaking, and on its own 'only'
causes digest mismatches on reads (and thus a lot of extra traffic w/ CL > ONE)
and less efficient representation on wire.
Otherwise it's not visible on its own, without bugs like CASSANDRA-11930
revealing it.
> DynamicCompositeType failures during 2.1 to 3.0 upgrade.
>
>
> Key: CASSANDRA-11489
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11489
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.x
>
>
> When upgrading from 2.1.13 to 3.0.4+some (hash
> 70eab633f289eb1e4fbe47b3e17ff3203337f233) we are seeing the following
> exceptions on 2.1 nodes after other nodes have been upgraded. With tables
> using DynamicCompositeType in use. The workload runs fine once everything is
> upgraded.
> {code}
> ERROR [MessagingService-Incoming-/10.200.182.2] 2016-04-03 21:49:10,531
> CassandraDaemon.java:229 - Exception in thread
> Thread[MessagingService-Incoming-/10.200.182.2,5,main]
> java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input
> length = 1
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:181)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:200)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:54)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.SimpleSparseCellNameType.fromByteBuffer(SimpleSparseCellNameType.java:83)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.RangeTombstoneList$Serializer.deserialize(RangeTombstoneList.java:843)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.DeletionInfo$Serializer.deserialize(DeletionInfo.java:407)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:105)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:89)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.db.Row$RowSerializer.deserialize(Row.java:73)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:116)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:195)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:172)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
> at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
> ~[na:1.8.0_40]
> at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
> ~[na:1.8.0_40]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:152)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil
[jira] [Commented] (CASSANDRA-11489) DynamicCompositeType failures during 2.1 to 3.0 upgrade.
[
https://issues.apache.org/jira/browse/CASSANDRA-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307909#comment-15307909
]
Aleksey Yeschenko commented on CASSANDRA-11489:
---
At least two bugs here.
1. {{LegacyLayout.LegacyRangeTombstoneList::serialize}} has a broken assumption
that a range tombstone implies {{CompositeType}}. This is incorrect, as you can
have non-{{CompositeType}} range tombstones created via Thrift, and as such
wrapping {{clusteringComparator.subtypes()}} in a {{CompositeType}} is
incorrect. On 2.1/2.2 side, when decoding the range tombstone list,
{{RangeTombstoneList::deserialize()}} will use the raw type to decode start and
end bounds, which will end up being confused by the extra 2 bytes in the
beginning (short length) plus end of component header in the end.
In this particular case, DCT is failing to decode {{00 09 80 62 00 04 64 61 74
61 00 00}}, interpreting {{00 09}} as length, instead of {{80 62}} which would
be an alias for {{d => BytesType}} in CFS.
2. CFS doesn't really use range tombstones, however, so issue (1) shouldn't
even manifest itself in the first place. Still looking into how that happens,
but a few other bugs - including in {{sstabledump}} - are making this whole
process frustrating/annoying.
(1) is however legit on its own and can be reproduced easily outside of CFS/DCT
context.
> DynamicCompositeType failures during 2.1 to 3.0 upgrade.
>
>
> Key: CASSANDRA-11489
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11489
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.x
>
>
> When upgrading from 2.1.13 to 3.0.4+some (hash
> 70eab633f289eb1e4fbe47b3e17ff3203337f233) we are seeing the following
> exceptions on 2.1 nodes after other nodes have been upgraded. With tables
> using DynamicCompositeType in use. The workload runs fine once everything is
> upgraded.
> {code}
> ERROR [MessagingService-Incoming-/10.200.182.2] 2016-04-03 21:49:10,531
> CassandraDaemon.java:229 - Exception in thread
> Thread[MessagingService-Incoming-/10.200.182.2,5,main]
> java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input
> length = 1
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:181)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:200)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:54)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.SimpleSparseCellNameType.fromByteBuffer(SimpleSparseCellNameType.java:83)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.RangeTombstoneList$Serializer.deserialize(RangeTombstoneList.java:843)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.DeletionInfo$Serializer.deserialize(DeletionInfo.java:407)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:105)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:89)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.db.Row$RowSerializer.deserialize(Row.java:73)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:116)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:195)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:172)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:88)
> ~[cassandra-all-2.1
[jira] [Commented] (CASSANDRA-11489) DynamicCompositeType failures during 2.1 to 3.0 upgrade.
[
https://issues.apache.org/jira/browse/CASSANDRA-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15291396#comment-15291396
]
Tyler Hobbs commented on CASSANDRA-11489:
-
No, unfortunately nothing obvious comes to mind. The code for serializing
range tombstones in the "legacy" format is fairly complex, so there's
definitely a possibility that there's a bug there. I would put a bunch of
debug statements in {{LegacyLayout.fromUnfilteredRowIterator()}} to check what
kinds of deletions are present in 3.0 and what the 3.0 node _thinks_ it's
serializing.
> DynamicCompositeType failures during 2.1 to 3.0 upgrade.
>
>
> Key: CASSANDRA-11489
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11489
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.x
>
>
> When upgrading from 2.1.13 to 3.0.4+some (hash
> 70eab633f289eb1e4fbe47b3e17ff3203337f233) we are seeing the following
> exceptions on 2.1 nodes after other nodes have been upgraded. With tables
> using DynamicCompositeType in use. The workload runs fine once everything is
> upgraded.
> {code}
> ERROR [MessagingService-Incoming-/10.200.182.2] 2016-04-03 21:49:10,531
> CassandraDaemon.java:229 - Exception in thread
> Thread[MessagingService-Incoming-/10.200.182.2,5,main]
> java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input
> length = 1
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:181)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:200)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:54)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.SimpleSparseCellNameType.fromByteBuffer(SimpleSparseCellNameType.java:83)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.RangeTombstoneList$Serializer.deserialize(RangeTombstoneList.java:843)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.DeletionInfo$Serializer.deserialize(DeletionInfo.java:407)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:105)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:89)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.db.Row$RowSerializer.deserialize(Row.java:73)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:116)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:195)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:172)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
> at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
> ~[na:1.8.0_40]
> at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
> ~[na:1.8.0_40]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:152)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:109)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:169)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> ... 16 common frames omitted
> {code}
--
This message was sent by At
[jira] [Commented] (CASSANDRA-11489) DynamicCompositeType failures during 2.1 to 3.0 upgrade.
[
https://issues.apache.org/jira/browse/CASSANDRA-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15291188#comment-15291188
]
Aleksey Yeschenko commented on CASSANDRA-11489:
---
[~thobbs] I initially assumed that the problem here was with DCT, but
apparently a 2.1 node, when decoding a read response from a 3.0 node, is trying
to deserialise some range tombstones that just cannot be there (this trace is
from reading a CFS table, and those use only cell level tombstones and whole
partition deletions, exclusively).
Having written 3.0-2.1 upgrade compat code, anything obvious comes to mind re:
how this got here?
> DynamicCompositeType failures during 2.1 to 3.0 upgrade.
>
>
> Key: CASSANDRA-11489
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11489
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.x
>
>
> When upgrading from 2.1.13 to 3.0.4+some (hash
> 70eab633f289eb1e4fbe47b3e17ff3203337f233) we are seeing the following
> exceptions on 2.1 nodes after other nodes have been upgraded. With tables
> using DynamicCompositeType in use. The workload runs fine once everything is
> upgraded.
> {code}
> ERROR [MessagingService-Incoming-/10.200.182.2] 2016-04-03 21:49:10,531
> CassandraDaemon.java:229 - Exception in thread
> Thread[MessagingService-Incoming-/10.200.182.2,5,main]
> java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input
> length = 1
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:181)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:200)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:54)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.SimpleSparseCellNameType.fromByteBuffer(SimpleSparseCellNameType.java:83)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.RangeTombstoneList$Serializer.deserialize(RangeTombstoneList.java:843)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.DeletionInfo$Serializer.deserialize(DeletionInfo.java:407)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:105)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:89)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.db.Row$RowSerializer.deserialize(Row.java:73)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:116)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:195)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:172)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
> at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
> ~[na:1.8.0_40]
> at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
> ~[na:1.8.0_40]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:152)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:109)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:169)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> ... 16
[jira] [Commented] (CASSANDRA-11489) DynamicCompositeType failures during 2.1 to 3.0 upgrade.
[
https://issues.apache.org/jira/browse/CASSANDRA-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257899#comment-15257899
]
Jeremiah Jordan commented on CASSANDRA-11489:
-
[~iamaleksey] yes, still seeing it during upgrade tests from 2.1.13 to
cassandra-3.0@a4e1182816909761c98355b1079b2f9de8efc4bd
> DynamicCompositeType failures during 2.1 to 3.0 upgrade.
>
>
> Key: CASSANDRA-11489
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11489
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.x
>
>
> When upgrading from 2.1.13 to 3.0.4+some (hash
> 70eab633f289eb1e4fbe47b3e17ff3203337f233) we are seeing the following
> exceptions on 2.1 nodes after other nodes have been upgraded. With tables
> using DynamicCompositeType in use. The workload runs fine once everything is
> upgraded.
> {code}
> ERROR [MessagingService-Incoming-/10.200.182.2] 2016-04-03 21:49:10,531
> CassandraDaemon.java:229 - Exception in thread
> Thread[MessagingService-Incoming-/10.200.182.2,5,main]
> java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input
> length = 1
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:181)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:200)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:54)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.SimpleSparseCellNameType.fromByteBuffer(SimpleSparseCellNameType.java:83)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.RangeTombstoneList$Serializer.deserialize(RangeTombstoneList.java:843)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.DeletionInfo$Serializer.deserialize(DeletionInfo.java:407)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:105)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:89)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.db.Row$RowSerializer.deserialize(Row.java:73)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:116)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:195)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:172)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
> at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
> ~[na:1.8.0_40]
> at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
> ~[na:1.8.0_40]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:152)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:109)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:169)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> ... 16 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11489) DynamicCompositeType failures during 2.1 to 3.0 upgrade.
[
https://issues.apache.org/jira/browse/CASSANDRA-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256902#comment-15256902
]
Aleksey Yeschenko commented on CASSANDRA-11489:
---
Can you still reproduce this, [~ht...@datastax.com]?
> DynamicCompositeType failures during 2.1 to 3.0 upgrade.
>
>
> Key: CASSANDRA-11489
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11489
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Aleksey Yeschenko
> Fix For: 3.0.x, 3.x
>
>
> When upgrading from 2.1.13 to 3.0.4+some (hash
> 70eab633f289eb1e4fbe47b3e17ff3203337f233) we are seeing the following
> exceptions on 2.1 nodes after other nodes have been upgraded. With tables
> using DynamicCompositeType in use. The workload runs fine once everything is
> upgraded.
> {code}
> ERROR [MessagingService-Incoming-/10.200.182.2] 2016-04-03 21:49:10,531
> CassandraDaemon.java:229 - Exception in thread
> Thread[MessagingService-Incoming-/10.200.182.2,5,main]
> java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input
> length = 1
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:181)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getString(AbstractCompositeType.java:200)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.cql3.ColumnIdentifier.(ColumnIdentifier.java:54)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.SimpleSparseCellNameType.fromByteBuffer(SimpleSparseCellNameType.java:83)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:398)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:382)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.RangeTombstoneList$Serializer.deserialize(RangeTombstoneList.java:843)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.DeletionInfo$Serializer.deserialize(DeletionInfo.java:407)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:105)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:89)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.db.Row$RowSerializer.deserialize(Row.java:73)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:116)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:99)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:195)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:172)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:88)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> Caused by: java.nio.charset.MalformedInputException: Input length = 1
> at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
> ~[na:1.8.0_40]
> at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
> ~[na:1.8.0_40]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:152)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:109)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> at
> org.apache.cassandra.db.marshal.DynamicCompositeType.getAndAppendComparator(DynamicCompositeType.java:169)
> ~[cassandra-all-2.1.13.1131.jar:2.1.13.1131]
> ... 16 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
