[
https://issues.apache.org/jira/browse/CASSANDRA-6732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103203#comment-14103203
]
Shawn Smith commented on CASSANDRA-6732:
This week I ran into this problem testing upgrade from 1.1.12 to 1.2.18. I
want to document for others that, as far as I can tell, the attached patch
fixes one case but not all. When the local coordinator wants to relay cross-DC
writes through a remote relay, there are 4 cases to consider (based on my tests
and reading the code):
# Local coordinator is Cassandra 1.1, remote relay is Cassandra 1.1: \\
Everything works. The old FORWARD header is used as expected.
# Local coordinator is Cassandra 1.2, remote relay is Cassandra 1.1: \\
Everything works due to this patch. The Cassandra 1.2 coordinator will send
individual messages to all the remote DC servers.
# Local coordinator is Cassandra 1.2, remote relay is Cassandra 1.2: \\ All
Cassandra 1.1 servers in the remote DC receive the write but ack to the remote
relay instead of the coordinator. This is because the Cassandra 1.1 servers
ignore the new FWD_FRM header and, starting with Cassandra 1.2, the relay
sets message.from to itself on the forwarded messages. The net effect:
writes get applied, but EACH_QUORUM, ALL writes fail due to lost acks.
# Local coordinator is Cassandra 1.1, remote relay is Cassandra 1.2: \\ The
remote relay ignores the FORWARD header and does not forward messages. The
net effect: only the remote relay applies the write, other servers in the
remote DC drop the write. EACH_QUORUM, ALL writes fail and data is left
inconsistent, repair is required if hints are lost.
Apologies if I have any mistakes here. It's hard to test each combination
independently.
Cross DC writes not compatible 1.2-1.1 during rolling upgrade
--
Key: CASSANDRA-6732
URL: https://issues.apache.org/jira/browse/CASSANDRA-6732
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Jeremiah Jordan
Assignee: Jeremiah Jordan
Fix For: 1.2.16
Attachments:
0001-Don-t-try-to-do-Cross-DC-forwarding-to-old-nodes.patch
During a rolling upgrade from 1.1.12 to 1.2.15 one DC at a time only 1/3 of
the writes to the first DC to be upgraded actually make it to the other DC,
and LOTS of hints attempt to get made.
Looks like the header for forwarded writes changed from 1.1-1.2 so the 1.1
nodes can't read it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
