[
https://issues.apache.org/jira/browse/CASSANDRA-10727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sharvanath Pathak updated CASSANDRA-10727:
------------------------------------------
Description:
There have been proposals for getting rid of the GC grace seconds, and
automating the GC of tombstones by waiting for acks from all the nodes about
the receipt of the tombstone.
1. CASSANDRA-3620
2. CASSANDRA-6192
This mechanism has two major benefits in my opinion:
* Since the GC of tomstones can be much more agressive, it minimizes the number
of tombstones in the system. Thereby, increasing the performance of read
operations.
* Eliminates the possibility of resurrection of keys in case a node is comes up
after being down for more than GC grace seconds.
As per CASSANDRA-3620, the main issue with the proposal seems to be its
potential race with the hinted handoff. Seems like we can have a good solution
to that race.
The solution is essentially to record the hint locations. So we before writing
any hints, we write a record on the alive replicas saying a hint was written at
so and so node. Now the GC will wait for an ack from all the nodes, and also
for all the related hints to be replayed and purged before it clears the
corresponding tombstone.
On potential problem with this scheme is that if the hints are written on the
coordinator node the same way they are being done right now, this process will
have to wait for a large number of nodes to be up before the GC could be
performed. However, this can be easily solve by writing the hints to a node
which is determined based on the key token. For instance, write the hint to the
node that comes up next to the replicas in the token ring.
Writing the hints in the way described in the last paragraph actually seems
like agood idea anyway, because it minimizes the number of nodes that have to
replay hints when a node comes up. The Dynamo paper actually describes this
pattern for hinted handoffs as well.
Lastly, it might also have a race with any concurrent read repairs. However, it
can be solved the same way, by writing the repairs in progress for a key and
then aborting them before the GC is performed.
was:
There have been proposals for getting rid of the GC grace seconds, and
automating the GC of tombstones by waiting for acks from all the nodes about
the receipt of the tombstone.
1. CASSANDRA-3620
2. CASSANDRA-6192
This mechanism has two major benefits in my opinion:
* Since the GC of tomstones can be much more agressive, it minimizes the number
of tombstones in the system. Thereby, increasing the performance of read
operations.
* Eliminates the possibility of resurrection of keys in case a node is comes up
after being down for more than GC grace seconds.
As per CASSANDRA-3620, the main issue with the proposal seems to be its
potential race with the hinted handoff. Seems like we can have a good solution
to that race.
The solution is essentially to record the hint locations. So we before writing
any hints, we write a record saying a hint was written at so and so node. Now
the GC will wait for an ack from all the nodes, and also for all the related
hints to be replayed and purged before it clears the corresponding tombstone.
On potential problem with this scheme is that if the hints are written on the
coordinator node the same way they are being done right now, this process will
have to wait for a large number of nodes to be up before the GC could be
performed. However, this can be easily solve by writing the hints to a node
which is determined based on the key token. For instance, write the hint to the
node that comes up next to the replicas in the token ring.
Writing the hints in the way described in the last paragraph actually seems
like agood idea anyway, because it minimizes the number of nodes that have to
replay hints when a node comes up. The Dynamo paper actually describes this
pattern for hinted handoffs as well.
Lastly, it might also have a race with any concurrent read repairs. However, it
can be solved the same way, by writing the repairs in progress for a key and
then aborting them before the GC is performed.
> Solution for getting rid of GC grace seconds
> --------------------------------------------
>
> Key: CASSANDRA-10727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10727
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Sharvanath Pathak
>
> There have been proposals for getting rid of the GC grace seconds, and
> automating the GC of tombstones by waiting for acks from all the nodes about
> the receipt of the tombstone.
> 1. CASSANDRA-3620
> 2. CASSANDRA-6192
> This mechanism has two major benefits in my opinion:
> * Since the GC of tomstones can be much more agressive, it minimizes the
> number of tombstones in the system. Thereby, increasing the performance of
> read operations.
> * Eliminates the possibility of resurrection of keys in case a node is comes
> up after being down for more than GC grace seconds.
> As per CASSANDRA-3620, the main issue with the proposal seems to be its
> potential race with the hinted handoff. Seems like we can have a good
> solution to that race.
> The solution is essentially to record the hint locations. So we before
> writing any hints, we write a record on the alive replicas saying a hint was
> written at so and so node. Now the GC will wait for an ack from all the
> nodes, and also for all the related hints to be replayed and purged before it
> clears the corresponding tombstone.
> On potential problem with this scheme is that if the hints are written on the
> coordinator node the same way they are being done right now, this process
> will have to wait for a large number of nodes to be up before the GC could be
> performed. However, this can be easily solve by writing the hints to a node
> which is determined based on the key token. For instance, write the hint to
> the node that comes up next to the replicas in the token ring.
> Writing the hints in the way described in the last paragraph actually seems
> like agood idea anyway, because it minimizes the number of nodes that have to
> replay hints when a node comes up. The Dynamo paper actually describes this
> pattern for hinted handoffs as well.
> Lastly, it might also have a race with any concurrent read repairs. However,
> it can be solved the same way, by writing the repairs in progress for a key
> and then aborting them before the GC is performed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
