[
https://issues.apache.org/jira/browse/CASSANDRA-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399941#comment-15399941
]
Paulo Motta commented on CASSANDRA-8523:
----------------------------------------
Thanks for taking a look! Created CASSANDRA-12344 to follow-up with support for
this when the replacement node has the same address as the original node.
Rebased patch and dtests as well as merged up to 3.0+. All patches and CI
results available below:
||2.2||3.0||3.9||trunk||dtest||
|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-8523]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-8523]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.9...pauloricardomg:3.9-8523]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-8523]|[branch|https://github.com/riptano/cassandra-dtest/compare/master...pauloricardomg:8523]|
|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-8523-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-8523-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.9-8523-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-8523-testall/lastCompletedBuild/testReport/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-8523-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-8523-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.9-8523-dtest/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-8523-dtest/lastCompletedBuild/testReport/]|
There were some minor merge conflicts on 3.0, and a slightly larger conflict on
3.9 due to CASSANDRA-10134, so I did some refactoring in the 3.9+ version to
move most of the logic to {{prepareForReplacement}}. Can you take another look
[~rlow]?
Dtest PR created [here|https://github.com/riptano/cassandra-dtest/pull/1155].
While this is marked an improvement and would theoretically only go to trunk,
this limitation is pretty counter-intuitive and probably hurts many users in
the wild, and since the changeset is relatively small and self-contained, I
think it could be interpreted as a bugfix and perhaps go on 2.2+ or maybe 3.0+.
WDYT [~brandon.williams] [~jkni] ?
> Writes should be sent to a replacement node while it is streaming in data
> -------------------------------------------------------------------------
>
> Key: CASSANDRA-8523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8523
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Richard Wagner
> Assignee: Paulo Motta
> Fix For: 2.1.x
>
>
> In our operations, we make heavy use of replace_address (or
> replace_address_first_boot) in order to replace broken nodes. We now realize
> that writes are not sent to the replacement nodes while they are in hibernate
> state and streaming in data. This runs counter to what our expectations were,
> especially since we know that writes ARE sent to nodes when they are
> bootstrapped into the ring.
> It seems like cassandra should arrange to send writes to a node that is in
> the process of replacing another node, just like it does for a nodes that are
> bootstraping. I hesitate to phrase this as "we should send writes to a node
> in hibernate" because the concept of hibernate may be useful in other
> contexts, as per CASSANDRA-8336. Maybe a new state is needed here?
> Among other things, the fact that we don't get writes during this period
> makes subsequent repairs more expensive, proportional to the number of writes
> that we miss (and depending on the amount of data that needs to be streamed
> during replacement and the time it may take to rebuild secondary indexes, we
> could miss many many hours worth of writes). It also leaves us more exposed
> to consistency violations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
