Yongle Zhang created CASSANDRA-16268:
----------------------------------------
Summary: Rolling upgrade from 1.1.0 to 1.2.0 runs into
IllegalArgumentException because of Invalid UUID
Key: CASSANDRA-16268
URL: https://issues.apache.org/jira/browse/CASSANDRA-16268
Project: Cassandra
Issue Type: Bug
Reporter: Yongle Zhang
Steps to reproduce:
# setup a 2-node C* 1.1.0 cluster
# upgrade the non-seed node to 1.2.0
# run stress testing tool on the 1.1.0 node, e.g.,
`/cassandra/tools/bin/stress -d 250.16.238.1 -r`
Error logs:
{code:java}
DEBUG [WRITE-/250.16.238.3] 2020-06-18 02:10:51,087 OutboundTcpConnection.java
(line 165) error writing to /250.16.238.3
java.net.SocketException: Broken pipe (Write failed)
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:156)
at
org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:126)
DEBUG [Thrift:1] 2020-07-11 22:32:56,017 CassandraServer.java (line 962)
add_keyspace
DEBUG [WRITE-/250.16.238.2] 2020-07-11 22:32:56,024 OutboundTcpConnection.java
(line 229) attempting to connect to /250.16.238.2
DEBUG [MigrationStage:1] 2020-07-11 22:32:56,039 SchemaCheckVerbHandler.java
(line 36) Received schema check request.
DEBUG [InternalResponseStage:9] 2020-07-11 22:32:56,047
ResponseVerbHandler.java (line 44) Processing response on a callback from
380@/250.16.238.1
DEBUG [InternalResponseStage:9] 2020-07-11 22:32:56,048 StorageProxy.java (line
955) Received schema check response from 250.16.238.1
DEBUG [InternalResponseStage:10] 2020-07-11 22:32:56,049
ResponseVerbHandler.java (line 44) Processing response on a callback from
381@/250.16.238.2
DEBUG [InternalResponseStage:10] 2020-07-11 22:32:56,050 StorageProxy.java
(line 955) Received schema check response from 250.16.238.2
ERROR [InternalResponseStage:10] 2020-07-11 22:32:56,053
AbstractCassandraDaemon.java (line 134) Exception in thread
Thread[InternalResponseStage:10,5,main]
java.lang.IllegalArgumentException: Invalid UUID string: Y��N��>^B��[9X'E?
at java.util.UUID.fromString(UUID.java:194)
at org.apache.cassandra.service.StorageProxy$8.response(StorageProxy.java:956)
at
org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:45)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
Root cause analysis: data format incompatibility
In 1.2.0, when the payload of `MessageOut` is `UUID`, it will write two long
integers to serialize the payload, representing the most significant 8 bytes
and the least significant 8 bytes of this `UUID`. See
[https://github.com/apache/cassandra/blob/69337a43670f71ae1fc55e23d6a9031230423900/src/java/org/apache/cassandra/utils/UUIDSerializer.java#L36].
In 1.1.0, it will read a String from the payload, and call `UUID.fromString` to
obtain the `UUID`. Since the string representation is different from the msb +
lsb 16 bytes representation, here we get the format mismatch. See
[https://github.com/apache/cassandra/blob/c671532825b9e05bd2b50e3aa2d1c5f5156d1c9f/src/java/org/apache/cassandra/service/StorageProxy.java#L956].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
