[jira] [Created] (CASSANDRA-14751) Can't truncate counter CFs via Thrift
Erik Swanson created CASSANDRA-14751:
Summary: Can't truncate counter CFs via Thrift
Key: CASSANDRA-14751
URL: https://issues.apache.org/jira/browse/CASSANDRA-14751
Project: Cassandra
Issue Type: Bug
Environment: Cassandra 3.11.2
Reporter: Erik Swanson
When connecting to Cassandra 3.11.2 using Thrift, attempting to truncate a CF
with counters results in "invalid operation for commutative table [table name]".
It appears that there was no checking re this prior to the materialized views
work. The materialized views work added:
{{CFMetaData metadata = ThriftValidation.validateColumnFamily(keyspace, cfname,
true);}}
This was changed to `..., false);` in
3be99eeaa3546fecaf474d848eecca6aa0254a22, "ninja fix commutative op flag for
truncate validation", but that fix (allowing truncation of CFs that do not
include counters) breaks truncation of CFs that do contain counters.
It's possible to truncate counter CFs via CQL, but that's no help re trying to
get an application that uses Thrift to work, which needs to be able to truncate
as part of its test suite.
Because truncation via CQL works, or at least appears to work, would the
appropriate fix be to change CassandraServer.truncate to use this function on
ThriftValidation instead?:
{{// To be used when the operation should be authorized whether this is a
counter CF or not}}
{{public static CFMetaData validateColumnFamily(String keyspaceName, String
cfName) throws org.apache.cassandra.exceptions.InvalidRequestException}}
{{{}}
{{ return validateColumnFamilyWithCompactMode(keyspaceName, cfName, false);}}
{{}}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14703) Document Apache Cassandra Backup Operations
[ https://issues.apache.org/jira/browse/CASSANDRA-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa reassigned CASSANDRA-14703: -- Assignee: pedro vidigal > Document Apache Cassandra Backup Operations > --- > > Key: CASSANDRA-14703 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14703 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: pedro vidigal >Assignee: pedro vidigal >Priority: Major > Labels: documentation > Fix For: 4.x > > Attachments: backups.rst, backups.rst > > > The documentation for the Backup Operations is missing in on the > documentation site > ([https://github.com/apache/cassandra/blob/trunk/doc/source/operating/backups.rst)] > > Branch with the changes: [https://github.com/vidigalp/cassandra/tree/14703] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14289) Document sstable tools
[ https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-14289: --- Reviewer: (was: Jeff Jirsa) > Document sstable tools > -- > > Key: CASSANDRA-14289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14289 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Hannu Kröger >Assignee: Valerie Parham-Thompson >Priority: Major > Fix For: 4.0 > > Attachments: gen-sstable-docs.py, sstabledocs.tar.gz > > > Following tools are missing in the documentation of cassandra tools on the > documentation site (http://cassandra.apache.org/doc/latest/tools/index.html): > * sstabledump > * sstableexpiredblockers > * sstablelevelreset > * sstableloader > * sstablemetadata > * sstableofflinerelevel > * sstablerepairedset > * sstablescrub > * sstablesplit > * sstableupgrade > * sstableutil > * sstableverify -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14289) Document sstable tools
[ https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-14289: --- Status: Ready to Commit (was: Patch Available) > Document sstable tools > -- > > Key: CASSANDRA-14289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14289 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Hannu Kröger >Assignee: Valerie Parham-Thompson >Priority: Major > Fix For: 4.0 > > Attachments: gen-sstable-docs.py, sstabledocs.tar.gz > > > Following tools are missing in the documentation of cassandra tools on the > documentation site (http://cassandra.apache.org/doc/latest/tools/index.html): > * sstabledump > * sstableexpiredblockers > * sstablelevelreset > * sstableloader > * sstablemetadata > * sstableofflinerelevel > * sstablerepairedset > * sstablescrub > * sstablesplit > * sstableupgrade > * sstableutil > * sstableverify -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14289) Document sstable tools
[
https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-14289:
---
Resolution: Fixed
Fix Version/s: 4.0
Reviewers: Jeff Jirsa, pedro vidigal
Status: Resolved (was: Ready to Commit)
Committed to trunk only as {{c2c43f814994fe0be6964b3ab0d15ea838aa1000}}, added
[~vidigalp] as reviewer as indicated. Thanks so much for the patch! It'll take
a bit for the site to be updated, but it's in git now.
> Document sstable tools
> --
>
> Key: CASSANDRA-14289
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14289
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation and Website
>Reporter: Hannu Kröger
>Assignee: Valerie Parham-Thompson
>Priority: Major
> Fix For: 4.0
>
> Attachments: gen-sstable-docs.py, sstabledocs.tar.gz
>
>
> Following tools are missing in the documentation of cassandra tools on the
> documentation site (http://cassandra.apache.org/doc/latest/tools/index.html):
> * sstabledump
> * sstableexpiredblockers
> * sstablelevelreset
> * sstableloader
> * sstablemetadata
> * sstableofflinerelevel
> * sstablerepairedset
> * sstablescrub
> * sstablesplit
> * sstableupgrade
> * sstableutil
> * sstableverify
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[2/2] cassandra git commit: Adding documentation for sstable tools
Adding documentation for sstable tools Patch by Valerie Parham-Thompson; reviewed by pedro vidigal and Jeff Jirsa for CASSANDRA-14289 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c2c43f81 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c2c43f81 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c2c43f81 Branch: refs/heads/trunk Commit: c2c43f814994fe0be6964b3ab0d15ea838aa1000 Parents: 2046c30 Author: Valerie Parham-Thompson Authored: Thu Sep 13 15:42:06 2018 -0400 Committer: Jeff Jirsa Committed: Thu Sep 13 15:36:11 2018 -0700 -- doc/source/tools/index.rst | 3 +- doc/source/tools/sstable/index.rst | 39 +++ doc/source/tools/sstable/sstabledump.rst| 294 ++ .../tools/sstable/sstableexpiredblockers.rst| 48 +++ doc/source/tools/sstable/sstablelevelreset.rst | 82 + doc/source/tools/sstable/sstableloader.rst | 273 + doc/source/tools/sstable/sstablemetadata.rst| 300 +++ .../tools/sstable/sstableofflinerelevel.rst | 95 ++ doc/source/tools/sstable/sstablerepairedset.rst | 79 + doc/source/tools/sstable/sstablescrub.rst | 93 ++ doc/source/tools/sstable/sstablesplit.rst | 93 ++ doc/source/tools/sstable/sstableupgrade.rst | 137 + doc/source/tools/sstable/sstableutil.rst| 91 ++ doc/source/tools/sstable/sstableverify.rst | 91 ++ 14 files changed, 1717 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2c43f81/doc/source/tools/index.rst -- diff --git a/doc/source/tools/index.rst b/doc/source/tools/index.rst index 20a5383..d28929c 100644 --- a/doc/source/tools/index.rst +++ b/doc/source/tools/index.rst @@ -20,8 +20,9 @@ Cassandra Tools This section describes the command line tools provided with Apache Cassandra. .. toctree:: - :maxdepth: 1 + :maxdepth: 3 cqlsh nodetool/nodetool + sstable/index cassandra_stress http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2c43f81/doc/source/tools/sstable/index.rst -- diff --git a/doc/source/tools/sstable/index.rst b/doc/source/tools/sstable/index.rst new file mode 100644 index 000..b9e483f --- /dev/null +++ b/doc/source/tools/sstable/index.rst @@ -0,0 +1,39 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +SSTable Tools += + +This section describes the functionality of the various sstable tools. + +Cassandra must be stopped before these tools are executed, or unexpected results will occur. Note: the scripts do not verify that Cassandra is stopped. + +.. toctree:: + :maxdepth: 2 + + sstabledump + sstableexpiredblockers + sstablelevelreset + sstableloader + sstablemetadata + sstableofflinerelevel + sstablerepairedset + sstablescrub + sstablesplit + sstableupgrade + sstableutil + sstableverify + http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2c43f81/doc/source/tools/sstable/sstabledump.rst -- diff --git a/doc/source/tools/sstable/sstabledump.rst b/doc/source/tools/sstable/sstabledump.rst new file mode 100644 index 000..8f38afa --- /dev/null +++ b/doc/source/tools/sstable/sstabledump.rst @@ -0,0 +1,294 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed
[1/2] cassandra git commit: Adding documentation for sstable tools
Repository: cassandra Updated Branches: refs/heads/trunk 2046c30ad -> c2c43f814 http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2c43f81/doc/source/tools/sstable/sstableverify.rst -- diff --git a/doc/source/tools/sstable/sstableverify.rst b/doc/source/tools/sstable/sstableverify.rst new file mode 100644 index 000..dad3f44 --- /dev/null +++ b/doc/source/tools/sstable/sstableverify.rst @@ -0,0 +1,91 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, software +.. distributed under the License is distributed on an "AS IS" BASIS, +.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +.. See the License for the specific language governing permissions and +.. limitations under the License. + +sstableverify +- + +Check sstable(s) for errors or corruption, for the provided table. + +ref: https://issues.apache.org/jira/browse/CASSANDRA-5791 + +Cassandra must be stopped before this tool is executed, or unexpected results will occur. Note: the script does not verify that Cassandra is stopped. + +Usage +^ +sstableverify + +=== +--debug display stack traces +-e, --extendedextended verification +-h, --helpdisplay this help message +-v, --verbose verbose output +=== + +Basic Verification +^^ + +This is the basic verification. It is not a very quick process, and uses memory. You might need to increase your memory settings if you have many sstables. + +Example:: + +sstableverify keyspace eventlog +Verifying BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-32-big-Data.db') (7.353MiB) +Deserializing sstable metadata for BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-32-big-Data.db') +Checking computed hash of BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-32-big-Data.db') +Verifying BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-37-big-Data.db') (3.775MiB) +Deserializing sstable metadata for BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-37-big-Data.db') +Checking computed hash of BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-37-big-Data.db') + +Extended Verification +^ + +During an extended verification, the individual values will be validated for errors or corruption. This of course takes more time. + +Example:: + +root@DC1C1:/# sstableverify -e keyspace eventlog +WARN 14:08:06,255 Only 33.096GiB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots +Verifying BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-32-big-Data.db') (7.353MiB) +Deserializing sstable metadata for BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-32-big-Data.db') +Checking computed hash of BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-32-big-Data.db') +Extended Verify requested, proceeding to inspect values +Verify of BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-32-big-Data.db') succeeded. All 33211 rows read successfully +Verifying BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-37-big-Data.db') (3.775MiB) +Deserializing sstable metadata for BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-37-big-Data.db') +Checking computed hash of BigTableReader(path='/var/lib/cassandra/data/keyspace/eventlog-6365332094dd11e88f324f9c503e4753/mc-37-big-Data.db') +Extended Verify requested, proceeding to inspect values +Verify of
[jira] [Commented] (CASSANDRA-14289) Document sstable tools
[ https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613976#comment-16613976 ] Valerie Parham-Thompson commented on CASSANDRA-14289: - [~jjirsa] Here are all the changes (12 sstable files, one new sstable index file, one changed index file a level higher) in one commit. Let me know if this will work. Thank you! https://github.com/dataindataout/cassandra/tree/docs_sstable_manually/doc/source/tools/sstable > Document sstable tools > -- > > Key: CASSANDRA-14289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14289 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Hannu Kröger >Assignee: Valerie Parham-Thompson >Priority: Major > Attachments: gen-sstable-docs.py, sstabledocs.tar.gz > > > Following tools are missing in the documentation of cassandra tools on the > documentation site (http://cassandra.apache.org/doc/latest/tools/index.html): > * sstabledump > * sstableexpiredblockers > * sstablelevelreset > * sstableloader > * sstablemetadata > * sstableofflinerelevel > * sstablerepairedset > * sstablescrub > * sstablesplit > * sstableupgrade > * sstableutil > * sstableverify -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14750) Missing check for receiving digest response from transient node
Alex Petrov created CASSANDRA-14750: --- Summary: Missing check for receiving digest response from transient node Key: CASSANDRA-14750 URL: https://issues.apache.org/jira/browse/CASSANDRA-14750 Project: Cassandra Issue Type: Bug Reporter: Alex Petrov Assignee: Alex Petrov Range reads do not check for transient nodes returning a request. Read Command also currently allows a combination of transient node and digest query. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14289) Document sstable tools
[ https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613953#comment-16613953 ] Valerie Parham-Thompson commented on CASSANDRA-14289: - I don't know how to do a squash easily, but I'll just re-fork and add the proposed docs in one commit. I'll do that right now. > Document sstable tools > -- > > Key: CASSANDRA-14289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14289 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Hannu Kröger >Assignee: Valerie Parham-Thompson >Priority: Major > Attachments: gen-sstable-docs.py, sstabledocs.tar.gz > > > Following tools are missing in the documentation of cassandra tools on the > documentation site (http://cassandra.apache.org/doc/latest/tools/index.html): > * sstabledump > * sstableexpiredblockers > * sstablelevelreset > * sstableloader > * sstablemetadata > * sstableofflinerelevel > * sstablerepairedset > * sstablescrub > * sstablesplit > * sstableupgrade > * sstableutil > * sstableverify -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14703) Document Apache Cassandra Backup Operations
[ https://issues.apache.org/jira/browse/CASSANDRA-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613919#comment-16613919 ] pedro vidigal commented on CASSANDRA-14703: --- Hi [~spo...@gmail.com], I've started creating an additional page, with the examples of how to use the described strategies, including the nodetool examples and recovery considerations. This page was meant as an overview of the options. I agree with you that the "datacenter backup" isn't exactly a backup, since it does not provide protection from application or user error. The reason I included it, was because I see so many companies using this without realizing the disadvantages. I can remove it if you think it does not fit into this section though. > Document Apache Cassandra Backup Operations > --- > > Key: CASSANDRA-14703 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14703 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: pedro vidigal >Priority: Major > Labels: documentation > Fix For: 4.x > > Attachments: backups.rst, backups.rst > > > The documentation for the Backup Operations is missing in on the > documentation site > ([https://github.com/apache/cassandra/blob/trunk/doc/source/operating/backups.rst)] > > Branch with the changes: [https://github.com/vidigalp/cassandra/tree/14703] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14727) Transient Replication: EACH_QUORUM not implemented
[
https://issues.apache.org/jira/browse/CASSANDRA-14727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613916#comment-16613916
]
Benedict commented on CASSANDRA-14727:
--
Pretty much. I already have a patch that builds on top of CASSANDRA-14705
ready, and moves {{ConsistencyLevel.filterForQuery}} into {{ReplicaPlans}} so
that the related logic is in one place.
I just had to draw a line somewhere on things I needed you guys to review for
14705.
> Transient Replication: EACH_QUORUM not implemented
> --
>
> Key: CASSANDRA-14727
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14727
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Priority: Major
>
> Transient replication cannot presently handle EACH_QUORUM consistency; reads
> and writes should currently fail, though without good error messages. Not
> clear if this is acceptable for GA, since we cannot impose this limitation at
> Keyspace declaration time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14727) Transient Replication: EACH_QUORUM not implemented
[ https://issues.apache.org/jira/browse/CASSANDRA-14727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613904#comment-16613904 ] Ariel Weisberg commented on CASSANDRA-14727: Is this a relatively simple change to the selector in ReplicaPlans? All we have to do is assemble a quorum inside each DC and return that list as the head? Speculation writes to all transient replicas so we don't need to fix speculation right now. > Transient Replication: EACH_QUORUM not implemented > -- > > Key: CASSANDRA-14727 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14727 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Priority: Major > > Transient replication cannot presently handle EACH_QUORUM consistency; reads > and writes should currently fail, though without good error messages. Not > clear if this is acceptable for GA, since we cannot impose this limitation at > Keyspace declaration time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
[ https://issues.apache.org/jira/browse/CASSANDRA-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14749: -- Status: Ready to Commit (was: Patch Available) > Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows > -- > > Key: CASSANDRA-14749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14749 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Benedict >Priority: Major > Fix For: 3.0.18 > > > Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node > containing a deletion for a dropped collection column, instead of deleting > the collection, we will delete the row containing the collection. > > This is an admittedly unlikely cluster state but, during such a state, a > great deal of data loss could happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
[ https://issues.apache.org/jira/browse/CASSANDRA-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14749: -- Status: Patch Available (was: Open) > Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows > -- > > Key: CASSANDRA-14749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14749 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Benedict >Priority: Major > Fix For: 3.0.18 > > > Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node > containing a deletion for a dropped collection column, instead of deleting > the collection, we will delete the row containing the collection. > > This is an admittedly unlikely cluster state but, during such a state, a > great deal of data loss could happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
[
https://issues.apache.org/jira/browse/CASSANDRA-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613870#comment-16613870
]
Benedict commented on CASSANDRA-14749:
--
FWIW, I've been planning to file a ticket to audit the non-use of
{{getDroppedColumn}} also, off the back of this, and perhaps we should combine
the two efforts. I'm sure most uses of {{getColumn}} should have a
corresponding use of {{getDroppedColumn}} and I'm sure there have been other
oversights than this. We could keep two separate endeavours though.
> Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
> --
>
> Key: CASSANDRA-14749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14749
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
> Fix For: 3.0.18
>
>
> Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node
> containing a deletion for a dropped collection column, instead of deleting
> the collection, we will delete the row containing the collection.
>
> This is an admittedly unlikely cluster state but, during such a state, a
> great deal of data loss could happen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
[ https://issues.apache.org/jira/browse/CASSANDRA-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613865#comment-16613865 ] Aleksey Yeschenko commented on CASSANDRA-14749: --- LGTM. I'll file a JIRA to follow-up on the defaulting variant audit and/or removal. > Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows > -- > > Key: CASSANDRA-14749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14749 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Benedict >Priority: Major > Fix For: 3.0.18 > > > Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node > containing a deletion for a dropped collection column, instead of deleting > the collection, we will delete the row containing the collection. > > This is an admittedly unlikely cluster state but, during such a state, a > great deal of data loss could happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14744) TR transient -> full and full -> transient enters pending state reducing availability
[ https://issues.apache.org/jira/browse/CASSANDRA-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg resolved CASSANDRA-14744. Resolution: Not A Problem OK. I am convinced that right now we are handling it as best we can. > TR transient -> full and full -> transient enters pending state reducing > availability > - > > Key: CASSANDRA-14744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14744 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Priority: Major > Fix For: 4.x > > > A transient replica transitioning to full and vice versa doesn't cease to be > the kind of replica it used to be before. I think this could cause > availability issues if a node goes down and it could even cause unavailable > with some ring movements with all nodes up. I haven't convinced myself 100% > this is true, but I think it should be possible to construct an example. > We can't read from pending replicas so if a ring movement were to make 2/3 > out of a group of 3 pending we would have this problem. > The key difference here is that the existing replica should be available > during this period as the kind of replica it was before. A full replica > remains full until nodetool cleanup is run. A transient replica remains > transient the entire time. > Obviously this doesn't hold for transitions to/from not replicated at all. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14666) Race condition in AbstractReplicationStrategy.getNaturalReplicas
[
https://issues.apache.org/jira/browse/CASSANDRA-14666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613851#comment-16613851
]
Benedict commented on CASSANDRA-14666:
--
Ideally, we would cache a {{ReplicaLayout}} or even a {{ReplicaPlan}} for each
range.
> Race condition in AbstractReplicationStrategy.getNaturalReplicas
>
>
> Key: CASSANDRA-14666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14666
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
> Labels: correctness
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> There is a very narrow and infrequent race window, in which two ring updates
> occur in a short space of time (or during an interval of no queries):
> - thread A invalidates the cache after the first ring change, snapshots this
> version of the ring, and begins to calculate its natural endpoints
> - thread B sees the second ring change, and invalidates the cache before
> thread A completes
> - thread A writes its value to the cache, based on the old ring layout
> Now, a stale view of the endpoints for this token will be persisted in
> AbstractReplicationStrategy until the next ring change (which may feasibly
> never occur)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14744) TR transient -> full and full -> transient enters pending state reducing availability
[
https://issues.apache.org/jira/browse/CASSANDRA-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613843#comment-16613843
]
Benedict edited comment on CASSANDRA-14744 at 9/13/18 5:54 PM:
---
Have a look at
[ReplicaLayout.haveWriteConflicts|https://github.com/belliottsmith/cassandra/blob/7b7d0025c2470e69fa89053f8c9c9e25f5039941/src/java/org/apache/cassandra/locator/ReplicaLayout.java#L260]
from CASSANDRA-14705, which explains this.
Essentially, a transition is only used to consciously make the right choice at
write time. They never actually end up in the final {{pending}} collection; we
basically just upgrade the {{natural}} replica to full, if it is currently
transient (for writes only). This does mean we could avoid populating entirely
for full->transient transitions, since that's the net effect anyway (though I
prefer having the logic applied in one place, with a dense comment). I hope in
CASSANDRA-14666, we will begin caching the actual layouts (or possibly even the
plans themselves), at which point they'll not even be seen.
TokenMetadata needs to be revisited as a whole, also, as brought up in
CASSANDRA-14660. At which point maybe we can also revisit how this
intermediate state tracks pending/natural.
was (Author: benedict):
Have a look at
[ReplicaLayout.haveWriteConflicts|https://github.com/belliottsmith/cassandra/blob/7b7d0025c2470e69fa89053f8c9c9e25f5039941/src/java/org/apache/cassandra/locator/ReplicaLayout.java#L260]
from CASSANDRA-14705, which explains this.
Essentially, a transition is only used to consciously make the right choice at
write time. I hope in CASSANDRA-14666, we will begin caching the actual
layouts (or possibly even the plans themselves), at which point they'll not
even be seen.
TokenMetadata needs to be revisited as a whole, also, as brought up in
CASSANDRA-14660. At which point maybe we can also revisit how this
intermediate state tracks pending/natural.
> TR transient -> full and full -> transient enters pending state reducing
> availability
> -
>
> Key: CASSANDRA-14744
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14744
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Ariel Weisberg
>Priority: Major
> Fix For: 4.x
>
>
> A transient replica transitioning to full and vice versa doesn't cease to be
> the kind of replica it used to be before. I think this could cause
> availability issues if a node goes down and it could even cause unavailable
> with some ring movements with all nodes up. I haven't convinced myself 100%
> this is true, but I think it should be possible to construct an example.
> We can't read from pending replicas so if a ring movement were to make 2/3
> out of a group of 3 pending we would have this problem.
> The key difference here is that the existing replica should be available
> during this period as the kind of replica it was before. A full replica
> remains full until nodetool cleanup is run. A transient replica remains
> transient the entire time.
> Obviously this doesn't hold for transitions to/from not replicated at all.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14744) TR transient -> full and full -> transient enters pending state reducing availability
[ https://issues.apache.org/jira/browse/CASSANDRA-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613843#comment-16613843 ] Benedict commented on CASSANDRA-14744: -- Have a look at [ReplicaLayout.haveWriteConflicts|https://github.com/belliottsmith/cassandra/blob/7b7d0025c2470e69fa89053f8c9c9e25f5039941/src/java/org/apache/cassandra/locator/ReplicaLayout.java#L260] from CASSANDRA-14705, which explains this. Essentially, a transition is only used to consciously make the right choice at write time. I hope in CASSANDRA-14666, we will begin caching the actual layouts (or possibly even the plans themselves), at which point they'll not even be seen. TokenMetadata needs to be revisited as a whole, also, as brought up in CASSANDRA-14660. At which point maybe we can also revisit how this intermediate state tracks pending/natural. > TR transient -> full and full -> transient enters pending state reducing > availability > - > > Key: CASSANDRA-14744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14744 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Priority: Major > Fix For: 4.x > > > A transient replica transitioning to full and vice versa doesn't cease to be > the kind of replica it used to be before. I think this could cause > availability issues if a node goes down and it could even cause unavailable > with some ring movements with all nodes up. I haven't convinced myself 100% > this is true, but I think it should be possible to construct an example. > We can't read from pending replicas so if a ring movement were to make 2/3 > out of a group of 3 pending we would have this problem. > The key difference here is that the existing replica should be available > during this period as the kind of replica it was before. A full replica > remains full until nodetool cleanup is run. A transient replica remains > transient the entire time. > Obviously this doesn't hold for transitions to/from not replicated at all. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14744) TR transient -> full and full -> transient enters pending state reducing availability
[ https://issues.apache.org/jira/browse/CASSANDRA-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613835#comment-16613835 ] Ariel Weisberg edited comment on CASSANDRA-14744 at 9/13/18 5:41 PM: - I think I get it. Natural replica calculations ignore the presence of pending and since it's there in the old state it will show up in natural with its old transient status and be usable for reads. Writes, full -> transient we need it to be pending for a while because of ring disagreement and readers continuing to use it as full although it's not streaming related that it would cease to be pending. Transient -> full seems like it needs to be pending until it finishes streaming so it doesn't miss writes. There is the issue of how this should impact group size for write quorums? It seems to me like these transitioning replicas maybe shouldn't change the number of responses necessary for a write quorum. Looking at ConsistencyLevel.blockForWrite it seems like they do? was (Author: aweisberg): I think I get it. Natural replica calculations ignore the presence of pending and since it's there in the old state it will show up in natural with its old transient status and be usable for reads. Writes, full -> transient we need it to be pending for a while because of ring disagreement and readers continuing to use it as full although it's not streaming related that it would cease to be pending. Transient -> full doesn't seem like it needs to be pending it just needs to not be natural? There is the issue of how this should impact group size for write quorums? It seems to me like these transitioning replicas maybe shouldn't change the number of responses necessary for a quorum. Looking at ConsistencyLevel.blockForWrite it seems like they do? > TR transient -> full and full -> transient enters pending state reducing > availability > - > > Key: CASSANDRA-14744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14744 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Priority: Major > Fix For: 4.x > > > A transient replica transitioning to full and vice versa doesn't cease to be > the kind of replica it used to be before. I think this could cause > availability issues if a node goes down and it could even cause unavailable > with some ring movements with all nodes up. I haven't convinced myself 100% > this is true, but I think it should be possible to construct an example. > We can't read from pending replicas so if a ring movement were to make 2/3 > out of a group of 3 pending we would have this problem. > The key difference here is that the existing replica should be available > during this period as the kind of replica it was before. A full replica > remains full until nodetool cleanup is run. A transient replica remains > transient the entire time. > Obviously this doesn't hold for transitions to/from not replicated at all. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14744) TR transient -> full and full -> transient enters pending state reducing availability
[ https://issues.apache.org/jira/browse/CASSANDRA-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613835#comment-16613835 ] Ariel Weisberg commented on CASSANDRA-14744: I think I get it. Natural replica calculations ignore the presence of pending and since it's there in the old state it will show up in natural with its old transient status and be usable for reads. Writes, full -> transient we need it to be pending for a while because of ring disagreement and readers continuing to use it as full although it's not streaming related that it would cease to be pending. Transient -> full doesn't seem like it needs to be pending it just needs to not be natural? There is the issue of how this should impact group size for write quorums? It seems to me like these transitioning replicas maybe shouldn't change the number of responses necessary for a quorum. Looking at ConsistencyLevel.blockForWrite it seems like they do? > TR transient -> full and full -> transient enters pending state reducing > availability > - > > Key: CASSANDRA-14744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14744 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Priority: Major > Fix For: 4.x > > > A transient replica transitioning to full and vice versa doesn't cease to be > the kind of replica it used to be before. I think this could cause > availability issues if a node goes down and it could even cause unavailable > with some ring movements with all nodes up. I haven't convinced myself 100% > this is true, but I think it should be possible to construct an example. > We can't read from pending replicas so if a ring movement were to make 2/3 > out of a group of 3 pending we would have this problem. > The key difference here is that the existing replica should be available > during this period as the kind of replica it was before. A full replica > remains full until nodetool cleanup is run. A transient replica remains > transient the entire time. > Obviously this doesn't hold for transitions to/from not replicated at all. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14289) Document sstable tools
[ https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613816#comment-16613816 ] Jeff Jirsa commented on CASSANDRA-14289: [~dataindataout] - if you're able to squash this into a single commit, that sure would be convenient. I'm intending to review it using a pseudo-PR ( like [this|https://github.com/apache/cassandra/compare/trunk...dataindataout:docs_sstable_manually.patch] ), but for some reason it's not merging cleanly. I'm sure it's something minor, but I'd like to keep your authorship AND not have to fight the patch to merge. Again, I can do it, but if it's easy for you, please squash it. > Document sstable tools > -- > > Key: CASSANDRA-14289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14289 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Hannu Kröger >Assignee: Valerie Parham-Thompson >Priority: Major > Attachments: gen-sstable-docs.py, sstabledocs.tar.gz > > > Following tools are missing in the documentation of cassandra tools on the > documentation site (http://cassandra.apache.org/doc/latest/tools/index.html): > * sstabledump > * sstableexpiredblockers > * sstablelevelreset > * sstableloader > * sstablemetadata > * sstableofflinerelevel > * sstablerepairedset > * sstablescrub > * sstablesplit > * sstableupgrade > * sstableutil > * sstableverify -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
[
https://issues.apache.org/jira/browse/CASSANDRA-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613795#comment-16613795
]
Benedict edited comment on CASSANDRA-14749 at 9/13/18 5:08 PM:
---
-Good job I did (split this out) - this time, Aleksey pointed out the
{{getDroppedColumn}} invokes another method {{getDroppedColumn(name,
isStatic)}} with a default parameter of {{false}} for {{isStatic}}- (That's
what I get for not refreshing the page)
I've pushed an update, but I think the defaulting variant of
{{getDroppedColumn}} probably shouldn't exist.
was (Author: benedict):
Good job I did (split this out) - this time, Aleksey pointed out the
{{getDroppedColumn}} invokes another method {{getDroppedColumn(name,
isStatic)}} with a default parameter of {{false}} for {{isStatic}}
I've pushed an update, but I think the defaulting variant of
{{getDroppedColumn}} probably shouldn't exist.
> Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
> --
>
> Key: CASSANDRA-14749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14749
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
> Fix For: 3.0.18
>
>
> Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node
> containing a deletion for a dropped collection column, instead of deleting
> the collection, we will delete the row containing the collection.
>
> This is an admittedly unlikely cluster state but, during such a state, a
> great deal of data loss could happen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
[
https://issues.apache.org/jira/browse/CASSANDRA-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613795#comment-16613795
]
Benedict commented on CASSANDRA-14749:
--
Good job I did (split this out) - this time, Aleksey pointed out the
{{getDroppedColumn}} invokes another method {{getDroppedColumn(name,
isStatic)}} with a default parameter of {{false}} for {{isStatic}}
I've pushed an update, but I think the defaulting variant of
{{getDroppedColumn}} probably shouldn't exist.
> Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
> --
>
> Key: CASSANDRA-14749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14749
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
> Fix For: 3.0.18
>
>
> Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node
> containing a deletion for a dropped collection column, instead of deleting
> the collection, we will delete the row containing the collection.
>
> This is an admittedly unlikely cluster state but, during such a state, a
> great deal of data loss could happen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
[
https://issues.apache.org/jira/browse/CASSANDRA-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613774#comment-16613774
]
Aleksey Yeschenko commented on CASSANDRA-14749:
---
Should probably pass {{isStatic}} to that
{{CFMetaData.getDroppedColumnDefinition()}} call, or else it just defaults to
{{false}}. May or may not have effect here, but since you have that flag in
hand already, might as well use it. +1 otherwise.
On a related note, I'm not 100% certain that other calls of
{{CFMetaData.getDroppedColumnDefinition()}} are oll korrect, it's something to
look into later.
> Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
> --
>
> Key: CASSANDRA-14749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14749
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
> Fix For: 3.0.18
>
>
> Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node
> containing a deletion for a dropped collection column, instead of deleting
> the collection, we will delete the row containing the collection.
>
> This is an admittedly unlikely cluster state but, during such a state, a
> great deal of data loss could happen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14568) Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
[
https://issues.apache.org/jira/browse/CASSANDRA-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613724#comment-16613724
]
Benedict commented on CASSANDRA-14568:
--
[3.0|https://github.com/belliottsmith/cassandra/tree/14568]
[CI|https://circleci.com/workflow-run/70397395-3585-4b4c-904f-a55c070cf359]
Thanks both for your review. I have split out CASSANDRA-14749, and pushed an
updated patch simply missing this part. If either of you could give a quick
cursory +1, I'll commit them both.
> Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
>
>
> Key: CASSANDRA-14568
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14568
> Project: Cassandra
> Issue Type: Bug
>Reporter: Benedict
>Assignee: Benedict
>Priority: Critical
> Fix For: 3.0.17, 3.11.3
>
>
> In 2.1 and 2.2, row and complex deletions were represented as range
> tombstones. LegacyLayout is our compatibility layer, that translates the
> relevant RT patterns in 2.1/2.2 to row/complex deletions in 3.0, and vice
> versa. Unfortunately, it does not handle the special case of static row
> deletions, they are treated as regular row deletions. Since static rows are
> themselves never directly deleted, the only issue is with collection
> deletions.
> Collection deletions in 2.1/2.2 were encoded as a range tombstone, consisting
> of a sequence of the clustering keys’ data for the affected row, followed by
> the bytes representing the name of the collection column. STATIC_CLUSTERING
> contains zero clusterings, so by treating the deletion as for a regular row,
> zero clusterings are written to precede the column name of the erased
> collection, so the column name is written at position zero.
> This can exhibit itself in at least two ways:
> # If the type of your first clustering key is a variable width type, new
> deletes will begin appearing covering the clustering key represented by the
> column name.
> ** If you have multiple clustering keys, you will receive a RT covering all
> those rows with a matching first clustering key.
> ** This RT will be valid as far as the system is concerned, and go
> undetected unless there are outside data quality checks in place.
> # Otherwise, an invalid size of data will be written to the clustering and
> sent over the network to the 2.1 node.
> ** The 2.1/2.2 node will handle this just fine, even though the record is
> junk. Since it is a deletion covering impossible data, there will be no
> user-API visible effect. But if received as a write from a 3.0 node, it will
> dutifully persist the junk record.
> ** The 3.0 node that originally sent this junk, may later coordinate a read
> of the partition, and will notice a digest mismatch, read-repair and
> serialize the junk to disk
> ** The sstable containing this record is now corrupt; the deserialization
> expects fixed-width data, but it encounters too many (or too few) bytes, and
> is now at an incorrect position to read its structural information
> ** (Alternatively when the 2.1 node is upgraded this will occur on eventual
> compaction)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
[ https://issues.apache.org/jira/browse/CASSANDRA-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613722#comment-16613722 ] Benedict commented on CASSANDRA-14749: -- [3.0|https://github.com/belliottsmith/cassandra/tree/14749] [CI|https://circleci.com/workflow-run/c877e18e-ea8e-45ec-b090-4652445f487d] Since this has already been reviewed in CASSANDRA-14568, if either of you could give a quick cursory +1 to the split, that would be great. I felt it warranted its own CHANGES.txt entry and JIRA. > Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows > -- > > Key: CASSANDRA-14749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14749 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Benedict >Priority: Major > Fix For: 3.0.18 > > > Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node > containing a deletion for a dropped collection column, instead of deleting > the collection, we will delete the row containing the collection. > > This is an admittedly unlikely cluster state but, during such a state, a > great deal of data loss could happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
[ https://issues.apache.org/jira/browse/CASSANDRA-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-14749: - Reviewers: Aleksey Yeschenko, Sylvain Lebresne > Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows > -- > > Key: CASSANDRA-14749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14749 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Benedict >Priority: Major > Fix For: 3.0.18 > > > Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node > containing a deletion for a dropped collection column, instead of deleting > the collection, we will delete the row containing the collection. > > This is an admittedly unlikely cluster state but, during such a state, a > great deal of data loss could happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14749) Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows
Benedict created CASSANDRA-14749: Summary: Collection Deletions for Dropped Columns in 2.1/3.0 mixed-mode can delete rows Key: CASSANDRA-14749 URL: https://issues.apache.org/jira/browse/CASSANDRA-14749 Project: Cassandra Issue Type: Bug Components: Core Reporter: Benedict Assignee: Benedict Fix For: 3.0.18 Similar to CASSANDRA-14568, if a 2.1 node sends a response to a 3.0 node containing a deletion for a dropped collection column, instead of deleting the collection, we will delete the row containing the collection. This is an admittedly unlikely cluster state but, during such a state, a great deal of data loss could happen. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14289) Document sstable tools
[ https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa reassigned CASSANDRA-14289: -- Assignee: Valerie Parham-Thompson > Document sstable tools > -- > > Key: CASSANDRA-14289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14289 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Hannu Kröger >Assignee: Valerie Parham-Thompson >Priority: Major > Attachments: gen-sstable-docs.py, sstabledocs.tar.gz > > > Following tools are missing in the documentation of cassandra tools on the > documentation site (http://cassandra.apache.org/doc/latest/tools/index.html): > * sstabledump > * sstableexpiredblockers > * sstablelevelreset > * sstableloader > * sstablemetadata > * sstableofflinerelevel > * sstablerepairedset > * sstablescrub > * sstablesplit > * sstableupgrade > * sstableutil > * sstableverify -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14289) Document sstable tools
[ https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613693#comment-16613693 ] Jeff Jirsa commented on CASSANDRA-14289: Thanks [~dataindataout] - I'll try to review and commit asap. > Document sstable tools > -- > > Key: CASSANDRA-14289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14289 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Hannu Kröger >Assignee: Valerie Parham-Thompson >Priority: Major > Attachments: gen-sstable-docs.py, sstabledocs.tar.gz > > > Following tools are missing in the documentation of cassandra tools on the > documentation site (http://cassandra.apache.org/doc/latest/tools/index.html): > * sstabledump > * sstableexpiredblockers > * sstablelevelreset > * sstableloader > * sstablemetadata > * sstableofflinerelevel > * sstablerepairedset > * sstablescrub > * sstablesplit > * sstableupgrade > * sstableutil > * sstableverify -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14289) Document sstable tools
[ https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-14289: --- Reviewer: Jeff Jirsa Status: Patch Available (was: Open) > Document sstable tools > -- > > Key: CASSANDRA-14289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14289 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Hannu Kröger >Priority: Major > Attachments: gen-sstable-docs.py, sstabledocs.tar.gz > > > Following tools are missing in the documentation of cassandra tools on the > documentation site (http://cassandra.apache.org/doc/latest/tools/index.html): > * sstabledump > * sstableexpiredblockers > * sstablelevelreset > * sstableloader > * sstablemetadata > * sstableofflinerelevel > * sstablerepairedset > * sstablescrub > * sstablesplit > * sstableupgrade > * sstableutil > * sstableverify -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14289) Document sstable tools
[ https://issues.apache.org/jira/browse/CASSANDRA-14289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613663#comment-16613663 ] Valerie Parham-Thompson commented on CASSANDRA-14289: - These are complete and peer-reviewed by (thank you!) [~vidigalp]. Here are the docs: https://github.com/dataindataout/cassandra/tree/docs_sstable_manually/doc/source/tools/sstable Can anyone please advise me on how to mark these as submitted for committer review, and maybe guide me in how to get these in the correct branch? I've created my fork from trunk, but I wonder if they need to be tied to 3.11, for example. [~jjirsa] Could you help with these questions, or direct me to someone else? [~hkroger] Please let me know, if you are able to review, if there's anything you see missing from your original vision. Thank you! > Document sstable tools > -- > > Key: CASSANDRA-14289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14289 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Hannu Kröger >Priority: Major > Attachments: gen-sstable-docs.py, sstabledocs.tar.gz > > > Following tools are missing in the documentation of cassandra tools on the > documentation site (http://cassandra.apache.org/doc/latest/tools/index.html): > * sstabledump > * sstableexpiredblockers > * sstablelevelreset > * sstableloader > * sstablemetadata > * sstableofflinerelevel > * sstablerepairedset > * sstablescrub > * sstablesplit > * sstableupgrade > * sstableutil > * sstableverify -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14703) Document Apache Cassandra Backup Operations
[ https://issues.apache.org/jira/browse/CASSANDRA-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613539#comment-16613539 ] Stefan Podkowinski commented on CASSANDRA-14703: Some feedback that comes to my mind while skimming through your document * Lacks configuration and nodetool examples for implementing any of the described strategies. * Recovery considerations are mentioned, but no steps provided for restoring backups. * I'm having a hard time to convince myself that we should suggest users to think about the described "datacenter backup" strategy, as it fails to adequately address the goals for a backup solution, as outlined in the intro section. > Document Apache Cassandra Backup Operations > --- > > Key: CASSANDRA-14703 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14703 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: pedro vidigal >Priority: Major > Labels: documentation > Fix For: 4.x > > Attachments: backups.rst, backups.rst > > > The documentation for the Backup Operations is missing in on the > documentation site > ([https://github.com/apache/cassandra/blob/trunk/doc/source/operating/backups.rst)] > > Branch with the changes: [https://github.com/vidigalp/cassandra/tree/14703] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14744) TR transient -> full and full -> transient enters pending state reducing availability
[ https://issues.apache.org/jira/browse/CASSANDRA-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613515#comment-16613515 ] Ariel Weisberg commented on CASSANDRA-14744: OK I guess I am unfamiliar with the state transitions that result in it in being in both natural and pending. I need to go do some reading. > TR transient -> full and full -> transient enters pending state reducing > availability > - > > Key: CASSANDRA-14744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14744 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Priority: Major > Fix For: 4.x > > > A transient replica transitioning to full and vice versa doesn't cease to be > the kind of replica it used to be before. I think this could cause > availability issues if a node goes down and it could even cause unavailable > with some ring movements with all nodes up. I haven't convinced myself 100% > this is true, but I think it should be possible to construct an example. > We can't read from pending replicas so if a ring movement were to make 2/3 > out of a group of 3 pending we would have this problem. > The key difference here is that the existing replica should be available > during this period as the kind of replica it was before. A full replica > remains full until nodetool cleanup is run. A transient replica remains > transient the entire time. > Obviously this doesn't hold for transitions to/from not replicated at all. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14705) ReplicaLayout follow-up
[
https://issues.apache.org/jira/browse/CASSANDRA-14705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613475#comment-16613475
]
Ariel Weisberg commented on CASSANDRA-14705:
+1 unless you want to do more changes as part of this.
> ReplicaLayout follow-up
> ---
>
> Key: CASSANDRA-14705
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14705
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
>
> Clarify the new {{ReplicaLayout}} code, separating it into ReplicaPlan (for
> what we want to do) and {{ReplicaLayout}} (for what we know about the
> cluster), with well defined semantics (and comments in the rare cases those
> semantics are weird)
> Found and fixed some bugs:
> * {{commitPaxos}} was using only live nodes, when needed to include down
> * We were not writing to pending transient replicas
> * On write, we were not hinting to full nodes with transient replication
> enabled (since we filtered to {{liveOnly}}, in order to include our transient
> replicas above {{blockFor}})
> * If we speculated, in {{maybeSendAdditionalReads}} (in read repair) we
> would only consult the same node we had speculated too. This also applied to
> {{maybeSendAdditionalWrites}} - and this issue was also true pre-TR.
> * Transient->Full movements mishandled consistency level upgrade
> ** While we need to treat a transitioning node as ‘full’ for writes, so that
> it can safely begin serving full data requests once it has finished, we
> cannot maintain it in the ‘pending’ collection else we will also increase our
> consistency requirements by a node that doesn’t exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14743) Better health based routing for transient reads
[ https://issues.apache.org/jira/browse/CASSANDRA-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613463#comment-16613463 ] Ariel Weisberg commented on CASSANDRA-14743: bq. This is only true while the transient status is healthy. Right so it's still sorted by score it's just that if they are close we prefer transient. If the transient node is unhealthy it's score won't be close right? bq. Ideally, though, we would be tracking separate statistics for each endpoint; Isn't this what the dynamic snitch does? Track read performance by endpoint? > Better health based routing for transient reads > > > Key: CASSANDRA-14743 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14743 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Priority: Major > Fix For: 4.x > > > Right now replica selection for reads with transient replication is purely > health based, but it is health based in the sense that it health information > is shared across both transient and full ranges. This means you can't use the > health information to tell if an endpoint will be faster for a specific query. > As a heuristic a transient replica will be a little faster for a read > compared to a full replica given two endpoints with similar scores so maybe > we don't need additional health information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14705) ReplicaLayout follow-up
[
https://issues.apache.org/jira/browse/CASSANDRA-14705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613459#comment-16613459
]
Benedict commented on CASSANDRA-14705:
--
So, I've pushed a squashed and rebased version
[here|https://github.com/belliottsmith/cassandra/tree/14705-rebase], however
I'm considering on last modification - Alex's most recent patches mean we can
perhaps get rid of the ReplicaPlan.Shared concept. I may try, and see how it
turns out. Previously we needed to construct the readRepair upfront, but now
we could construct it on demand. It might be that it is a useful abstraction
anyway, since we at least make it clear that these are modified. But it would
actually localise more the scope of who can see what modifications to a single
class.
> ReplicaLayout follow-up
> ---
>
> Key: CASSANDRA-14705
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14705
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
>
> Clarify the new {{ReplicaLayout}} code, separating it into ReplicaPlan (for
> what we want to do) and {{ReplicaLayout}} (for what we know about the
> cluster), with well defined semantics (and comments in the rare cases those
> semantics are weird)
> Found and fixed some bugs:
> * {{commitPaxos}} was using only live nodes, when needed to include down
> * We were not writing to pending transient replicas
> * On write, we were not hinting to full nodes with transient replication
> enabled (since we filtered to {{liveOnly}}, in order to include our transient
> replicas above {{blockFor}})
> * If we speculated, in {{maybeSendAdditionalReads}} (in read repair) we
> would only consult the same node we had speculated too. This also applied to
> {{maybeSendAdditionalWrites}} - and this issue was also true pre-TR.
> * Transient->Full movements mishandled consistency level upgrade
> ** While we need to treat a transitioning node as ‘full’ for writes, so that
> it can safely begin serving full data requests once it has finished, we
> cannot maintain it in the ‘pending’ collection else we will also increase our
> consistency requirements by a node that doesn’t exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14657) Handle failures in upgradesstables/cleanup/relocate
[
https://issues.apache.org/jira/browse/CASSANDRA-14657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613413#comment-16613413
]
Benedict commented on CASSANDRA-14657:
--
I guess we could specify per operation if it's valuable, but this all seems out
of scope for this ticket. Ship it, +1.
> Handle failures in upgradesstables/cleanup/relocate
> ---
>
> Key: CASSANDRA-14657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14657
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a compaction in {{parallelAllSSTableOperation}} throws exception, all
> current transactions are closed, this can make us close a transaction that
> has not yet finished (since we can run many of these compactions in
> parallel). This causes this error:
> {code}
> java.lang.IllegalStateException: Cannot prepare to commit unless IN_PROGRESS;
> state is ABORTED
> {code}
> and this can get the leveled manifest (if running LCS) in a bad state causing
> this error message:
> {code}
> Could not acquire references for compacting SSTables ...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14657) Handle failures in upgradesstables/cleanup/relocate
[
https://issues.apache.org/jira/browse/CASSANDRA-14657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613396#comment-16613396
]
Marcus Eriksson commented on CASSANDRA-14657:
-
[~benedict] yeah I considered that but if we run for example `upgradesstables`
it would probably be more wasteful to cancel the potentially successful upgrades
> Handle failures in upgradesstables/cleanup/relocate
> ---
>
> Key: CASSANDRA-14657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14657
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a compaction in {{parallelAllSSTableOperation}} throws exception, all
> current transactions are closed, this can make us close a transaction that
> has not yet finished (since we can run many of these compactions in
> parallel). This causes this error:
> {code}
> java.lang.IllegalStateException: Cannot prepare to commit unless IN_PROGRESS;
> state is ABORTED
> {code}
> and this can get the leveled manifest (if running LCS) in a bad state causing
> this error message:
> {code}
> Could not acquire references for compacting SSTables ...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14657) Handle failures in upgradesstables/cleanup/relocate
[
https://issues.apache.org/jira/browse/CASSANDRA-14657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613392#comment-16613392
]
Benedict commented on CASSANDRA-14657:
--
LGTM. Do we think it's maybe worth cancelling the existing operations? It
looks like there's unfortunately no trivial way of doing it, as we don't
currently check for {{Thread.isInterrupted}} when checking {{isStopRequested}},
so perhaps not worth worrying about. Does seem a shame to leave all that work
on going after an exception, but perhaps we should simply file a follow-up
ticket.
> Handle failures in upgradesstables/cleanup/relocate
> ---
>
> Key: CASSANDRA-14657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14657
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a compaction in {{parallelAllSSTableOperation}} throws exception, all
> current transactions are closed, this can make us close a transaction that
> has not yet finished (since we can run many of these compactions in
> parallel). This causes this error:
> {code}
> java.lang.IllegalStateException: Cannot prepare to commit unless IN_PROGRESS;
> state is ABORTED
> {code}
> and this can get the leveled manifest (if running LCS) in a bad state causing
> this error message:
> {code}
> Could not acquire references for compacting SSTables ...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14657) Handle failures in upgradesstables/cleanup/relocate
[
https://issues.apache.org/jira/browse/CASSANDRA-14657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-14657:
Status: Patch Available (was: Open)
dtest run against the branch above:
https://circleci.com/workflow-run/315d4f3d-18b7-4af2-aa46-ef6a2a57d37e
> Handle failures in upgradesstables/cleanup/relocate
> ---
>
> Key: CASSANDRA-14657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14657
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a compaction in {{parallelAllSSTableOperation}} throws exception, all
> current transactions are closed, this can make us close a transaction that
> has not yet finished (since we can run many of these compactions in
> parallel). This causes this error:
> {code}
> java.lang.IllegalStateException: Cannot prepare to commit unless IN_PROGRESS;
> state is ABORTED
> {code}
> and this can get the leveled manifest (if running LCS) in a bad state causing
> this error message:
> {code}
> Could not acquire references for compacting SSTables ...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14705) ReplicaLayout follow-up
[
https://issues.apache.org/jira/browse/CASSANDRA-14705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613130#comment-16613130
]
Benedict commented on CASSANDRA-14705:
--
Ah, thanks. I must have gone amiss somehow with the summary GitHub UI. It
does look like we can just use the provided replica plan safely, so I've
removed the CL parameters.
> ReplicaLayout follow-up
> ---
>
> Key: CASSANDRA-14705
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14705
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Assignee: Benedict
>Priority: Major
>
> Clarify the new {{ReplicaLayout}} code, separating it into ReplicaPlan (for
> what we want to do) and {{ReplicaLayout}} (for what we know about the
> cluster), with well defined semantics (and comments in the rare cases those
> semantics are weird)
> Found and fixed some bugs:
> * {{commitPaxos}} was using only live nodes, when needed to include down
> * We were not writing to pending transient replicas
> * On write, we were not hinting to full nodes with transient replication
> enabled (since we filtered to {{liveOnly}}, in order to include our transient
> replicas above {{blockFor}})
> * If we speculated, in {{maybeSendAdditionalReads}} (in read repair) we
> would only consult the same node we had speculated too. This also applied to
> {{maybeSendAdditionalWrites}} - and this issue was also true pre-TR.
> * Transient->Full movements mishandled consistency level upgrade
> ** While we need to treat a transitioning node as ‘full’ for writes, so that
> it can safely begin serving full data requests once it has finished, we
> cannot maintain it in the ‘pending’ collection else we will also increase our
> consistency requirements by a node that doesn’t exist.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14743) Better health based routing for transient reads
[
https://issues.apache.org/jira/browse/CASSANDRA-14743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613122#comment-16613122
]
Benedict commented on CASSANDRA-14743:
--
{quote}As a heuristic a transient replica will be a little faster for a read
compared to a full replica given two endpoints with similar scores so maybe we
don't need additional health information.
{quote}
This is only true while the transient status is healthy. When performing a
QUORUM in a multi-DC scenario, for instance, you might prefer a remote digest
over a local transient response if there's a down local node. Or in a scenario
where repair has been unable to run for a period, or inter-DC bandwidth is
saturated, digest queries might entail less time in flight.
If nothing else, separate statistics for transient reads is a useful health
barometer for the system. Ideally, though, we would be tracking separate
statistics for each endpoint; this might be impractical for systems where all
nodes are coordinators (though perhaps we could begin tracking stats for peers
separately, since these are the expected and optimal case.
> Better health based routing for transient reads
>
>
> Key: CASSANDRA-14743
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14743
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Ariel Weisberg
>Priority: Major
> Fix For: 4.x
>
>
> Right now replica selection for reads with transient replication is purely
> health based, but it is health based in the sense that it health information
> is shared across both transient and full ranges. This means you can't use the
> health information to tell if an endpoint will be faster for a specific query.
> As a heuristic a transient replica will be a little faster for a read
> compared to a full replica given two endpoints with similar scores so maybe
> we don't need additional health information.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14744) TR transient -> full and full -> transient enters pending state reducing availability
[ https://issues.apache.org/jira/browse/CASSANDRA-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613117#comment-16613117 ] Benedict commented on CASSANDRA-14744: -- I don't think this is the issue you think it is? We can't read from pending replicas, but any transition from transient->full, or full->transient would see the node exist in *both* the natural *and* pending states (one as transient, the other full). So in either case it will remain available for read, via its 'natural' instance. Ideally we would go a bit further, and on read we might begin treating a full->transient transition as a transient node (to handle races with ring change updates), but this is already an issue for regular ring movements, and would entail a lot more network traffic. So probably the best solution for this is (as with many other things) strongly consistent range movements. > TR transient -> full and full -> transient enters pending state reducing > availability > - > > Key: CASSANDRA-14744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14744 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Priority: Major > Fix For: 4.x > > > A transient replica transitioning to full and vice versa doesn't cease to be > the kind of replica it used to be before. I think this could cause > availability issues if a node goes down and it could even cause unavailable > with some ring movements with all nodes up. I haven't convinced myself 100% > this is true, but I think it should be possible to construct an example. > We can't read from pending replicas so if a ring movement were to make 2/3 > out of a group of 3 pending we would have this problem. > The key difference here is that the existing replica should be available > during this period as the kind of replica it was before. A full replica > remains full until nodetool cleanup is run. A transient replica remains > transient the entire time. > Obviously this doesn't hold for transitions to/from not replicated at all. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14748) Recycler$WeakOrderQueue occupies Heap
[ https://issues.apache.org/jira/browse/CASSANDRA-14748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613112#comment-16613112 ] Benedict commented on CASSANDRA-14748: -- I guess I should go and revisit the WeakOrderQueue and bound its laziness, as this is a recurring them - on the Netty project bug tracker as well. When I wrote it, the spec I was given had that it was intended for all recycling to happen on the same thread, but that we must support occasional recycles from another thread. So it was anticipated that the WeakOrderQueue would be used infrequently. But, it seems to get exercised often in some use cases - ours apparently being one. It's actually probably not too tricky to resolve this, so I'll see about opening a ticket on the netty project. We should anyway try to track down where we are doing this in the project, as it is suboptimal, and not how the recycler is intended to be used. Could you post the full heap dump, or at least the full class histogram? The WeakOrderQueue should be holding on to some child objects. > Recycler$WeakOrderQueue occupies Heap > - > > Key: CASSANDRA-14748 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14748 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: The netty Cassandra using is netty-all-4.0.39.Final.jar >Reporter: HX >Priority: Major > Fix For: 3.10 > > > Heap constantly high on some of the nodes in the cluster, I dump the heap and > open it through Eclipse Memory Analyzer, looks like Recycler$WeakOrderQueue > occupies most of the heap. > > ||Package||Retained Heap||Retained Heap, %||# Top Dominators|| > |!/jira/icons/i5.gif! |7,078,140,136|100.00%|379,627| > |io|5,665,035,800|80.04%|13,306| > |netty|5,665,035,800|80.04%|13,306| > |util|5,568,107,344|78.67%|2,965| > |Recycler$WeakOrderQueue|4,950,021,544|69.93%|2,169| -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
