[jira] [Updated] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-15327:
-
Resolution: Won't Fix
Status: Resolved (was: Open)
> Deleted data can re-appear if range movement streaming time exceeds
> gc_grace_seconds
>
>
> Key: CASSANDRA-15327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Fix For: 2.2.15, 2.1.x
>
> Attachments: CASSANDRA-15327-2.1.txt, CASSANDRA-15327-2.2.txt
>
>
> Hey,
> We've come across a scenario in production (noticed on Cassandra 2.2.14)
> where data that is deleted from Cassandra at consistency {{ALL}} can be
> resurrected. I've added a reproduction in a comment.
> If a {{delete}} is issued during a range movement (i.e. bootstrap,
> decommission, move), and {{gc_grace_seconds}} is surpassed before the stream
> is finished, then the tombstones from the {{delete}} can be purged from the
> recipient node before the data is streamed. Once the move is complete, the
> data now exists on the recipient node without a tombstone.
> We noticed this because our bootstrapping time occasionally exceeds our
> configured gc_grace_seconds, so we lose the consistency guarantee. As an
> operator, it would be great to not have to worry about this edge case.
> I've attached a patch that we have tested and successfully used in
> production, and haven't noticed any ill effects. Happy to submit patches for
> more recent versions, I'm not sure how cleanly this will actually merge since
> there was some refactoring to this logic in 3.x.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leon Zaruvinsky updated CASSANDRA-15327:
Discovered By: (was: User Report)
> Deleted data can re-appear if range movement streaming time exceeds
> gc_grace_seconds
>
>
> Key: CASSANDRA-15327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Fix For: 2.2.15, 2.1.x
>
> Attachments: CASSANDRA-15327-2.1.txt, CASSANDRA-15327-2.2.txt
>
>
> Hey,
> We've come across a scenario in production (noticed on Cassandra 2.2.14)
> where data that is deleted from Cassandra at consistency {{ALL}} can be
> resurrected. I've added a reproduction in a comment.
> If a {{delete}} is issued during a range movement (i.e. bootstrap,
> decommission, move), and {{gc_grace_seconds}} is surpassed before the stream
> is finished, then the tombstones from the {{delete}} can be purged from the
> recipient node before the data is streamed. Once the move is complete, the
> data now exists on the recipient node without a tombstone.
> We noticed this because our bootstrapping time occasionally exceeds our
> configured gc_grace_seconds, so we lose the consistency guarantee. As an
> operator, it would be great to not have to worry about this edge case.
> I've attached a patch that we have tested and successfully used in
> production, and haven't noticed any ill effects. Happy to submit patches for
> more recent versions, I'm not sure how cleanly this will actually merge since
> there was some refactoring to this logic in 3.x.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leon Zaruvinsky updated CASSANDRA-15327:
Description:
Hey,
We've come across a scenario in production (noticed on Cassandra 2.2.14) where
data that is deleted from Cassandra at consistency {{ALL}} can be resurrected.
I've added a reproduction in a comment.
If a {{delete}} is issued during a range movement (i.e. bootstrap,
decommission, move), and {{gc_grace_seconds}} is surpassed before the stream is
finished, then the tombstones from the {{delete}} can be purged from the
recipient node before the data is streamed. Once the move is complete, the data
now exists on the recipient node without a tombstone.
We noticed this because our bootstrapping time occasionally exceeds our
configured gc_grace_seconds, so we lose the consistency guarantee. As an
operator, it would be great to not have to worry about this edge case.
I've attached a patch that we have tested and successfully used in production,
and haven't noticed any ill effects. Happy to submit patches for more recent
versions, I'm not sure how cleanly this will actually merge since there was
some refactoring to this logic in 3.x.
was:
Hey,
We've come across a scenario in production (noticed on Cassandra 2.2.14) where
data that is deleted from Cassandra at consistency {{ALL}} can be resurrected.
I've added a reproduction in a comment.
If a {{delete}} is issued during a range movement (i.e. bootstrap,
decommission, move), and {{gc_grace_seconds}} is surpassed before the stream is
finished, then the tombstones from the {{delete}} can be purged from the
recipient node before the data is streamed. Once the move is complete, the data
now exists on the recipient node without a tombstone.
We noticed this because our bootstrapping time occasionally exceeds our
configured gc_grace_seconds, so we lose the consistency guarantee. As an
operator, it would be great to not have to worry about this edge case.
I've attached a patch that we have tested and successfully used in production,
and haven't noticed any ill effects.
> Deleted data can re-appear if range movement streaming time exceeds
> gc_grace_seconds
>
>
> Key: CASSANDRA-15327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Fix For: 2.2.15, 2.1.x
>
> Attachments: CASSANDRA-15327-2.1.txt, CASSANDRA-15327-2.2.txt
>
>
> Hey,
> We've come across a scenario in production (noticed on Cassandra 2.2.14)
> where data that is deleted from Cassandra at consistency {{ALL}} can be
> resurrected. I've added a reproduction in a comment.
> If a {{delete}} is issued during a range movement (i.e. bootstrap,
> decommission, move), and {{gc_grace_seconds}} is surpassed before the stream
> is finished, then the tombstones from the {{delete}} can be purged from the
> recipient node before the data is streamed. Once the move is complete, the
> data now exists on the recipient node without a tombstone.
> We noticed this because our bootstrapping time occasionally exceeds our
> configured gc_grace_seconds, so we lose the consistency guarantee. As an
> operator, it would be great to not have to worry about this edge case.
> I've attached a patch that we have tested and successfully used in
> production, and haven't noticed any ill effects. Happy to submit patches for
> more recent versions, I'm not sure how cleanly this will actually merge since
> there was some refactoring to this logic in 3.x.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leon Zaruvinsky updated CASSANDRA-15327:
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Discovered By: User Report
Fix Version/s: 2.1.x
2.2.15
Platform: (was: All)
Severity: Normal
Status: Open (was: Triage Needed)
> Deleted data can re-appear if range movement streaming time exceeds
> gc_grace_seconds
>
>
> Key: CASSANDRA-15327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Fix For: 2.2.15, 2.1.x
>
> Attachments: CASSANDRA-15327-2.1.txt, CASSANDRA-15327-2.2.txt
>
>
> Hey,
> We've come across a scenario in production (noticed on Cassandra 2.2.14)
> where data that is deleted from Cassandra at consistency {{ALL}} can be
> resurrected. I've added a reproduction in a comment.
> If a {{delete}} is issued during a range movement (i.e. bootstrap,
> decommission, move), and {{gc_grace_seconds}} is surpassed before the stream
> is finished, then the tombstones from the {{delete}} can be purged from the
> recipient node before the data is streamed. Once the move is complete, the
> data now exists on the recipient node without a tombstone.
> We noticed this because our bootstrapping time occasionally exceeds our
> configured gc_grace_seconds, so we lose the consistency guarantee. As an
> operator, it would be great to not have to worry about this edge case.
> I've attached a patch that we have tested and successfully used in
> production, and haven't noticed any ill effects.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leon Zaruvinsky updated CASSANDRA-15327:
Attachment: CASSANDRA-15327-2.2.txt
> Deleted data can re-appear if range movement streaming time exceeds
> gc_grace_seconds
>
>
> Key: CASSANDRA-15327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Attachments: CASSANDRA-15327-2.1.txt, CASSANDRA-15327-2.2.txt
>
>
> Hey,
> We've come across a scenario in production (noticed on Cassandra 2.2.14)
> where data that is deleted from Cassandra at consistency {{ALL}} can be
> resurrected. I've added a reproduction in a comment.
> If a {{delete}} is issued during a range movement (i.e. bootstrap,
> decommission, move), and {{gc_grace_seconds}} is surpassed before the stream
> is finished, then the tombstones from the {{delete}} can be purged from the
> recipient node before the data is streamed. Once the move is complete, the
> data now exists on the recipient node without a tombstone.
> We noticed this because our bootstrapping time occasionally exceeds our
> configured gc_grace_seconds, so we lose the consistency guarantee. As an
> operator, it would be great to not have to worry about this edge case.
> I've attached a patch that we have tested and successfully used in
> production, and haven't noticed any ill effects.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leon Zaruvinsky updated CASSANDRA-15327:
Attachment: CASSANDRA-15327-2.2.txt
> Deleted data can re-appear if range movement streaming time exceeds
> gc_grace_seconds
>
>
> Key: CASSANDRA-15327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Attachments: CASSANDRA-15327-2.1.txt
>
>
> Hey,
> We've come across a scenario in production (noticed on Cassandra 2.2.14)
> where data that is deleted from Cassandra at consistency {{ALL}} can be
> resurrected. I've added a reproduction in a comment.
> If a {{delete}} is issued during a range movement (i.e. bootstrap,
> decommission, move), and {{gc_grace_seconds}} is surpassed before the stream
> is finished, then the tombstones from the {{delete}} can be purged from the
> recipient node before the data is streamed. Once the move is complete, the
> data now exists on the recipient node without a tombstone.
> We noticed this because our bootstrapping time occasionally exceeds our
> configured gc_grace_seconds, so we lose the consistency guarantee. As an
> operator, it would be great to not have to worry about this edge case.
> I've attached a patch that we have tested and successfully used in
> production, and haven't noticed any ill effects.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leon Zaruvinsky updated CASSANDRA-15327:
Attachment: (was: CASSANDRA-15327-2.2.txt)
> Deleted data can re-appear if range movement streaming time exceeds
> gc_grace_seconds
>
>
> Key: CASSANDRA-15327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Attachments: CASSANDRA-15327-2.1.txt
>
>
> Hey,
> We've come across a scenario in production (noticed on Cassandra 2.2.14)
> where data that is deleted from Cassandra at consistency {{ALL}} can be
> resurrected. I've added a reproduction in a comment.
> If a {{delete}} is issued during a range movement (i.e. bootstrap,
> decommission, move), and {{gc_grace_seconds}} is surpassed before the stream
> is finished, then the tombstones from the {{delete}} can be purged from the
> recipient node before the data is streamed. Once the move is complete, the
> data now exists on the recipient node without a tombstone.
> We noticed this because our bootstrapping time occasionally exceeds our
> configured gc_grace_seconds, so we lose the consistency guarantee. As an
> operator, it would be great to not have to worry about this edge case.
> I've attached a patch that we have tested and successfully used in
> production, and haven't noticed any ill effects.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leon Zaruvinsky updated CASSANDRA-15327:
Description:
Hey,
We've come across a scenario in production (noticed on Cassandra 2.2.14) where
data that is deleted from Cassandra at consistency {{ALL}} can be resurrected.
I've added a reproduction in a comment.
If a {{delete}} is issued during a range movement (i.e. bootstrap,
decommission, move), and {{gc_grace_seconds}} is surpassed before the stream is
finished, then the tombstones from the {{delete}} can be purged from the
recipient node before the data is streamed. Once the move is complete, the data
now exists on the recipient node without a tombstone.
We noticed this because our bootstrapping time occasionally exceeds our
configured gc_grace_seconds, so we lose the consistency guarantee. As an
operator, it would be great to not have to worry about this edge case.
I've attached a patch that we have tested and successfully used in production,
and haven't noticed any ill effects.
was:
Hey,
We've come across a scenario in production (noticed on Cassandra 2.2.14) where
data that is deleted from Cassandra at consistency {{ALL}} can be resurrected.
I've added a reproduction in a comment.
If a {{delete}} is issued during a range movement (i.e. bootstrap,
decommission, move), and {{gc_grace_seconds}} is surpassed before the stream is
finished, then the tombstones from the {{delete}} can be purged from the
recipient node before the data is streamed. Once the move is complete, the data
now exists on the recipient node without a tombstone.
We noticed this because our bootstrapping time occasionally exceeds our
configured gc_grace_seconds, so we lose the consistency guarantee. As an
operator, it would be great to not have to worry about this edge case.
I've attached a patch that we have tested and successfully used in production,
and haven't noticed any ill effects.
> Deleted data can re-appear if range movement streaming time exceeds
> gc_grace_seconds
>
>
> Key: CASSANDRA-15327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Attachments: CASSANDRA-15327-2.1.txt
>
>
> Hey,
> We've come across a scenario in production (noticed on Cassandra 2.2.14)
> where data that is deleted from Cassandra at consistency {{ALL}} can be
> resurrected. I've added a reproduction in a comment.
> If a {{delete}} is issued during a range movement (i.e. bootstrap,
> decommission, move), and {{gc_grace_seconds}} is surpassed before the stream
> is finished, then the tombstones from the {{delete}} can be purged from the
> recipient node before the data is streamed. Once the move is complete, the
> data now exists on the recipient node without a tombstone.
> We noticed this because our bootstrapping time occasionally exceeds our
> configured gc_grace_seconds, so we lose the consistency guarantee. As an
> operator, it would be great to not have to worry about this edge case.
> I've attached a patch that we have tested and successfully used in
> production, and haven't noticed any ill effects.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15327) Deleted data can re-appear if range movement streaming time exceeds gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leon Zaruvinsky updated CASSANDRA-15327:
Attachment: CASSANDRA-15327-2.1.txt
> Deleted data can re-appear if range movement streaming time exceeds
> gc_grace_seconds
>
>
> Key: CASSANDRA-15327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15327
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Bootstrap and Decommission
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Attachments: CASSANDRA-15327-2.1.txt
>
>
> Hey,
> We've come across a scenario in production (noticed on Cassandra 2.2.14)
> where data that is deleted from Cassandra at consistency {{ALL}} can be
> resurrected. I've added a reproduction in a comment.
> If a {{delete}} is issued during a range movement (i.e. bootstrap,
> decommission, move), and {{gc_grace_seconds}} is surpassed before the stream
> is finished, then the tombstones from the {{delete}} can be purged from the
> recipient node before the data is streamed. Once the move is complete, the
> data now exists on the recipient node without a tombstone.
>
> We noticed this because our bootstrapping time occasionally exceeds our
> configured gc_grace_seconds, so we lose the consistency guarantee. As an
> operator, it would be great to not have to worry about this edge case.
> I've attached a patch that we have tested and successfully used in
> production, and haven't noticed any ill effects.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
