[jira] [Updated] (CASSANDRA-18513) Limit the read size on the row level
[ https://issues.apache.org/jira/browse/CASSANDRA-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18513: --- Reviewers: maxwellguo > Limit the read size on the row level > > > Key: CASSANDRA-18513 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18513 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > When deserializing a row, we have a limit on amount of data we can read per > cell. If an sstable is corrupted in a certain way, it may read many cells > with extreme length leading to oom errors. When we read a row, it has a row > size in the beginning. The idea is to limit the read size for a cell to the > minimum of the max cell length and the remaining row size. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723920#comment-17723920
]
maxwellguo edited comment on CASSANDRA-18534 at 5/22/23 9:06 AM:
-
Any volunteers ? If not, I think i can try to take this . [~blambov]
was (Author: maxwellguo):
Any volunteers ? If not, I think i can try to take this .
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Branimir Lambov
>Priority: Normal
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723920#comment-17723920
]
maxwellguo commented on CASSANDRA-18534:
Any volunteers ? If not, I think i can try to take this .
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Branimir Lambov
>Priority: Normal
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18500) Add guardrail for partition size
[
https://issues.apache.org/jira/browse/CASSANDRA-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maxwellguo updated CASSANDRA-18500:
---
Reviewers: Berenguer Blasi, maxwellguo (was: Berenguer Blasi)
> Add guardrail for partition size
>
>
> Key: CASSANDRA-18500
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18500
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add a guardrail for max partition size, for example:
> {code:java}
> partition_size_warn_threshold: 50MiB
> partition_size_fail_threshold: 100MiB
> {code}
> Most probably this guardrail would only be checked when writing a new sstable
> to disk (fush/compact). Triggering the guardrail on sstable write would emit
> a log message and a diagnostic event, but it wouldn't reject the write.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18504) Added support for type VECTOR
[ https://issues.apache.org/jira/browse/CASSANDRA-18504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18504: --- Reviewers: maxwellguo, Mike Adamson (was: Mike Adamson) > Added support for type VECTOR > -- > > Key: CASSANDRA-18504 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18504 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema, CQL/Syntax >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 5.x > > Time Spent: 3h > Remaining Estimate: 0h > > Based off several mailing list threads (see "[POLL] Vector type for ML”, > "[DISCUSS] New data type for vector search”, and "Adding vector search to SAI > with heirarchical navigable small world graph index”), its desirable to add a > new type “VECTOR” that has the following properties > 1) fixed length array > 2) elements may not be null > 3) flatten array (aka multi-cell = false) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16555) Add out-of-the-box snitch for Ec2 IMDSv2
[ https://issues.apache.org/jira/browse/CASSANDRA-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720419#comment-17720419 ] maxwellguo commented on CASSANDRA-16555: when doing refactor in trunk should we consider this jira https://issues.apache.org/jira/browse/CASSANDRA-18438 ,haven't gone into some implementation details but it should be valuable.[~brandon.williams][~jlewandowski] > Add out-of-the-box snitch for Ec2 IMDSv2 > > > Key: CASSANDRA-16555 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16555 > Project: Cassandra > Issue Type: New Feature > Components: Consistency/Coordination >Reporter: Paul Rütter (BlueConic) >Assignee: fulco taen >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > Time Spent: 1h 40m > Remaining Estimate: 0h > > In order to patch a vulnerability, Amazon came up with a new version of their > metadata service. > It's no longer unrestricted but now requires a token (in a header), in order > to access the metadata service. > See > [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html] > for more information. > Cassandra currently doesn't offer an out-of-the-box snitch class to support > this. > See > [https://cassandra.apache.org/doc/latest/operating/snitch.html#snitch-classes] > This issue asks to add support for this as a separate snitch class. > We'll probably do a PR for this, as we are in the process of developing one. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18486) LeveledCompactionStrategy does not check its constructor
[
https://issues.apache.org/jira/browse/CASSANDRA-18486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720413#comment-17720413
]
maxwellguo edited comment on CASSANDRA-18486 at 5/8/23 7:06 AM:
my question is , why we need to modify this ? as the way to modify compaction
strategy is through ddl when create or alter table.
but as [~maedhroz] has described , c* will do validate options when doing ddl.
Besides [~manish.c.ghildi...@gmail.com] I think your pr is not very suitable，
at the very least, if we need to modify, we can't just modify LCS, STWCS AND
TWCS may also need.
was (Author: maxwellguo):
my question is , why we need to modify this ? as the way to modify compaction
strategy is through ddl when create or alter table.
but as [~maedhroz]have described , c* will do validate options.
Besides [~manish.c.ghildi...@gmail.com] I think your pr is not very suitable，
at the very least, if we need to modify, we can't just modify LCS, STWCS AND
TWCS may also need.
> LeveledCompactionStrategy does not check its constructor
>
>
> Key: CASSANDRA-18486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18486
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction/LCS
>Reporter: Hao Zhong
>Assignee: Manish Ghildiyal
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> LeveledCompactionStrategy has the following code:
>
> {code:java}
> public static Map validateOptions(Map
> options) throws ConfigurationException
> {
> Map uncheckedOptions =
> AbstractCompactionStrategy.validateOptions(options);
> String size = options.containsKey(SSTABLE_SIZE_OPTION) ?
> options.get(SSTABLE_SIZE_OPTION) : "1";
> try
> {
> int ssSize = Integer.parseInt(size);
> if (ssSize < 1)
> {
> throw new ConfigurationException(String.format("%s must be
> larger than 0, but was %s", SSTABLE_SIZE_OPTION, ssSize));
> }
> }
> catch (NumberFormatException ex)
> {
> throw new ConfigurationException(String.format("%s is not a
> parsable int (base10) for %s", size, SSTABLE_SIZE_OPTION), ex);
> }
> ...
> } {code}
> This method throws ConfigurationException when the configuration file is
> wrong. The thrown exception is easy to understand, and calling code can catch
> this exception to handle configuration files with wrong formats. However, the
> constructor of this class does not check configuration files:
>
>
>
> {code:java}
> public LeveledCompactionStrategy(ColumnFamilyStore cfs, Map
> options)
> {...
>
> configuredMaxSSTableSize = Integer.parseInt(options.get(SSTABLE_SIZE_OPTION));
> ...
> configuredLevelFanoutSize =
> Integer.parseInt(options.get(LEVEL_FANOUT_SIZE_OPTION));
> ...
> }
> {code}
> As a result, this class can throw NumberFormatException when configuration
> files are wrong, and will not throw ConfigurationException.
> It is strange for calling code to call the static validation method before
> calling its constructor. Can this problem be fixed?
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18486) LeveledCompactionStrategy does not check its constructor
[
https://issues.apache.org/jira/browse/CASSANDRA-18486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720413#comment-17720413
]
maxwellguo commented on CASSANDRA-18486:
my question is , why we need to modify this ? as the way to modify compaction
strategy is through ddl when create or alter table.
but as [~maedhroz]have described , c* will do validate options.
Besides [~manish.c.ghildi...@gmail.com] I think your pr is not very suitable，
at the very least, if we need to modify, we can't just modify LCS, STWCS AND
TWCS may also need.
> LeveledCompactionStrategy does not check its constructor
>
>
> Key: CASSANDRA-18486
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18486
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction/LCS
>Reporter: Hao Zhong
>Assignee: Manish Ghildiyal
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> LeveledCompactionStrategy has the following code:
>
> {code:java}
> public static Map validateOptions(Map
> options) throws ConfigurationException
> {
> Map uncheckedOptions =
> AbstractCompactionStrategy.validateOptions(options);
> String size = options.containsKey(SSTABLE_SIZE_OPTION) ?
> options.get(SSTABLE_SIZE_OPTION) : "1";
> try
> {
> int ssSize = Integer.parseInt(size);
> if (ssSize < 1)
> {
> throw new ConfigurationException(String.format("%s must be
> larger than 0, but was %s", SSTABLE_SIZE_OPTION, ssSize));
> }
> }
> catch (NumberFormatException ex)
> {
> throw new ConfigurationException(String.format("%s is not a
> parsable int (base10) for %s", size, SSTABLE_SIZE_OPTION), ex);
> }
> ...
> } {code}
> This method throws ConfigurationException when the configuration file is
> wrong. The thrown exception is easy to understand, and calling code can catch
> this exception to handle configuration files with wrong formats. However, the
> constructor of this class does not check configuration files:
>
>
>
> {code:java}
> public LeveledCompactionStrategy(ColumnFamilyStore cfs, Map
> options)
> {...
>
> configuredMaxSSTableSize = Integer.parseInt(options.get(SSTABLE_SIZE_OPTION));
> ...
> configuredLevelFanoutSize =
> Integer.parseInt(options.get(LEVEL_FANOUT_SIZE_OPTION));
> ...
> }
> {code}
> As a result, this class can throw NumberFormatException when configuration
> files are wrong, and will not throw ConfigurationException.
> It is strange for calling code to call the static validation method before
> calling its constructor. Can this problem be fixed?
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maxwellguo updated CASSANDRA-18260:
---
Status: Needs Committer (was: Review In Progress)
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718389#comment-17718389
]
maxwellguo commented on CASSANDRA-18260:
+1 on this patch
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18397) CEP-26: Unified Compaction Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-18397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18397: --- Reviewers: Jaroslaw Grabowski, Joey Lynch, maxwellguo (was: Jaroslaw Grabowski, Joey Lynch) > CEP-26: Unified Compaction Strategy > --- > > Key: CASSANDRA-18397 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18397 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > Implementation of Unified Compaction Strategy per > [CEP-26|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-26%3A+Unified+Compaction+Strategy]. > Further documentation of the most current state of the solution can be found > in [the included markdown > documentation|https://github.com/blambov/cassandra/blob/CASSANDRA-18397/src/java/org/apache/cassandra/db/compaction/UnifiedCompactionStrategy.md]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18105) TRUNCATED data come back after a restart or upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maxwellguo updated CASSANDRA-18105:
---
Reviewers: maxwellguo
> TRUNCATED data come back after a restart or upgrade
> ---
>
> Key: CASSANDRA-18105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18105
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Ke Han
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When we use the TRUNCATE command to delete all data in the table, the deleted
> data come back after a node restart or upgrade. This problem happens at the
> latest releases (2.2.19, 3.0.28, or 4.0.7)
> h1. Steps to reproduce
> h2. To reproduce it at release (3.0.28 or 4.0.7)
> Start up a single Cassandra node. Using the default configuration and execute
> the following cqlsh commands.
> {code:java}
> CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = { 'class' :
> 'SimpleStrategy', 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c3 TEXT,c4 TEXT,c2 INT,c1 TEXT, PRIMARY KEY (c1, c2, c3
> ));
> INSERT INTO ks.tb (c3, c1, c2) VALUES ('val1','val2',1);
> CREATE INDEX IF NOT EXISTS tb ON ks.tb ( c3);
> TRUNCATE TABLE ks.tb;
> DROP INDEX IF EXISTS ks.tb; {code}
> Execute a read command
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> (0 rows) {code}
> Then, we flush the node and kill the Cassandra daemon by
> {code:java}
> bin/nodetool flush
> pgrep -f cassandra | xargs kill -9 {code}
> We restart the node. When the node has started, perform the same read, and
> the deleted data comes back again.
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> 1
> (1 rows) {code}
> h2. To reproduce it at release (2.2.19)
> We don't need to kill the Cassandra daemon. Use bin/nodetool stopdaemon is
> enough. The other steps are the same as reproducing it at 4.0.7 or 3.0.28.
> {code:java}
> bin/nodetool -h :::127.0.0.1 flush
> bin/nodetool -h :::127.0.0.1 stopdaemon{code}
>
> I have put the full log to reproduce it for release 4.0.7 and 2.2.19 in the
> comments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18105) TRUNCATED data come back after a restart or upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714503#comment-17714503
]
maxwellguo commented on CASSANDRA-18105:
+1 on this path, and left one comment.(y) Thanks [~smiklosovic]
> TRUNCATED data come back after a restart or upgrade
> ---
>
> Key: CASSANDRA-18105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18105
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Ke Han
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When we use the TRUNCATE command to delete all data in the table, the deleted
> data come back after a node restart or upgrade. This problem happens at the
> latest releases (2.2.19, 3.0.28, or 4.0.7)
> h1. Steps to reproduce
> h2. To reproduce it at release (3.0.28 or 4.0.7)
> Start up a single Cassandra node. Using the default configuration and execute
> the following cqlsh commands.
> {code:java}
> CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = { 'class' :
> 'SimpleStrategy', 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c3 TEXT,c4 TEXT,c2 INT,c1 TEXT, PRIMARY KEY (c1, c2, c3
> ));
> INSERT INTO ks.tb (c3, c1, c2) VALUES ('val1','val2',1);
> CREATE INDEX IF NOT EXISTS tb ON ks.tb ( c3);
> TRUNCATE TABLE ks.tb;
> DROP INDEX IF EXISTS ks.tb; {code}
> Execute a read command
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> (0 rows) {code}
> Then, we flush the node and kill the Cassandra daemon by
> {code:java}
> bin/nodetool flush
> pgrep -f cassandra | xargs kill -9 {code}
> We restart the node. When the node has started, perform the same read, and
> the deleted data comes back again.
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> 1
> (1 rows) {code}
> h2. To reproduce it at release (2.2.19)
> We don't need to kill the Cassandra daemon. Use bin/nodetool stopdaemon is
> enough. The other steps are the same as reproducing it at 4.0.7 or 3.0.28.
> {code:java}
> bin/nodetool -h :::127.0.0.1 flush
> bin/nodetool -h :::127.0.0.1 stopdaemon{code}
>
> I have put the full log to reproduce it for release 4.0.7 and 2.2.19 in the
> comments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maxwellguo updated CASSANDRA-18260:
---
Fix Version/s: (was: 5.x)
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714472#comment-17714472
]
maxwellguo edited comment on CASSANDRA-18260 at 4/20/23 8:10 AM:
-
[~henrik.ingo]Hi, I left some comments for your pr. Can you take a look at this
?
And [~mck] can you help to take a look if some comments is inappropriate
Besides,the pr only work for 4.1 and 4.0 , and no pr for trunk .And it seems
that the function do not exist in trunk.
If we will not fix in trunk or 5.0 ,so I can change the fix version to 4.1 and
4.0 only.
was (Author: maxwellguo):
[~henrik.ingo]Hi, I left some comments for your pr. Can you take a look at this
?
And [~mck] can you help to take a look if some comments is inappropriate
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maxwellguo updated CASSANDRA-18260:
---
Reviewers: Brad Schoening, Claude Warren, Kan Maung, maxwellguo, Michael
Semb Wever (was: Brad Schoening, Claude Warren, Kan Maung, Michael Semb Wever)
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714472#comment-17714472
]
maxwellguo commented on CASSANDRA-18260:
[~henrik.ingo]Hi, I left some comments for your pr. Can you take a look at this
?
And [~mck] can you help to take a look if some comments is inappropriate
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Compaction, Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> {noformat}
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18466) Paxos only repair is treated as an incremental repair
[
https://issues.apache.org/jira/browse/CASSANDRA-18466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714388#comment-17714388
]
maxwellguo commented on CASSANDRA-18466:
nodetool repair will doing incremental repair by default if --full is not set.
> Paxos only repair is treated as an incremental repair
> -
>
> Key: CASSANDRA-18466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18466
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Andrew
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> Paxos only repair tries to continue or is treated as an incremental repair.
> This happened on 4.1.0 and 4.1.1 when trying to run repair in preparation for
> enabling paxos_state_purging. The repair was in preparation mode triggered
> multiple anti-compactions on the nodes. Running the command with --full
> behaves in the expected way, ie. only the paxos data is repaired and it's
> finished within a few seconds.
> {code:java}
> nodetool repair --paxos-only // This does not behave as expected, does it
> complete quickly and seems to be waiting on anticompactions
> {code}
> {code:java}
> nodetool repair --full --paxos-only // Completes within a few seconds as
> expected
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18464) Enable Direct I/O For CommitLog Files
[ https://issues.apache.org/jira/browse/CASSANDRA-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17713938#comment-17713938 ] maxwellguo edited comment on CASSANDRA-18464 at 4/19/23 7:55 AM: - I feel that this ci must be able to pass（of course the build /compile stage should pass first) , as the there lack of relevant ut test cases for Direct IO and the use_jna_for_commitlog_io is set to fasle so the related code are not coverd. was (Author: maxwellguo): I feel that this ci must be able to pass（except for build stage）, as the there lack of relevant ut test cases for Direct IO and the use_jna_for_commitlog_io is set to fasle so the related code are not coverd. > Enable Direct I/O For CommitLog Files > - > > Key: CASSANDRA-18464 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18464 > Project: Cassandra > Issue Type: New Feature > Components: Local/Commit Log >Reporter: Josh McKenzie >Assignee: Amit Pawar >Priority: Normal > Fix For: 6.x > > Attachments: UseDirectIOFeatureForCommitLogFiles.patch > > > Relocating from [dev@ email > thread.|https://lists.apache.org/thread/j6ny17q2rhkp7jxvwxm69dd6v1dozjrg] > > I shared my investigation about Commitlog I/O issue on large core count > system in my previous email dated July-22 and link to the thread is given > below. > [https://lists.apache.org/thread/xc5ocog2qz2v2gnj4xlw5hbthfqytx2n] > Basically, two solutions looked possible to improve the CommitLog I/O. > # Multi-threaded syncing > # Using Direct-IO through JNA > I worked on 2nd option considering the following benefit compared to the > first one > # Direct I/O read/write throughput is very high compared to non-Direct I/O. > Learnt through FIO benchmarking. > # Reduces kernel file cache uses which in-turn reduces kernel I/O activity > for Commitlog files only. > # Overall CPU usage reduced for flush activity. JVisualvm shows CPU usage < > 30% for Commitlog syncer thread with Direct I/O feature > # Direct I/O implementation is easier compared to multi-threaded > As per the community suggestion, less in code complex is good to have. Direct > I/O enablement looked promising but there was one issue. > Java version 8 does not have native support to enable Direct I/O. So, JNA > library usage is must. The same implementation should also work across other > versions of Java (like 11 and beyond). > I have completed Direct I/O implementation and summary of the attached patch > changes are given below. > # This implementation is not using Java file channels and file is opened > through JNA to use Direct I/O feature. > # New Segment are defined named “DirectIOSegment” for Direct I/O and > “NonDirectIOSegment” for non-direct I/O (NonDirectIOSegment is test purpose > only). > # JNA write call is used to flush the changes. > # New helper functions are defined in NativeLibrary.java and platform > specific file. Currently tested on Linux only. > # Patch allows user to configure optimum block size and alignment if > default values are not OK for CommitLog disk. > # Following configuration options are provided in Cassandra.yaml file > a. use_jna_for_commitlog_io : to use jna feature > b. use_direct_io_for_commitlog : to use Direct I/O feature. > c. direct_io_minimum_block_alignment: 512 (default) > d. nvme_disk_block_size: 32MiB (default and can be changed as per the > required size) > Test matrix is complex so CommitLog related testcases and TPCx-IOT benchmark > was tested. It works with both Java 8 and 11 versions. Compressed and > Encrypted based segments are not supported yet and it can be enabled later > based on the Community feedback. > Following improvement are seen with Direct I/O enablement. > # 32 cores >= ~15% > # 64 cores >= ~80% > Also, another observation would like to share here. Reading Commitlog files > with Direct I/O might help in reducing node bring-up time after the node > crash. > Tested with commit ID: 91f6a9aca8d3c22a03e68aa901a0b154d960ab07 > The attached patch enables Direct I/O feature for Commitlog files. Please > check and share your feedback. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18464) Enable Direct I/O For CommitLog Files
[ https://issues.apache.org/jira/browse/CASSANDRA-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17713938#comment-17713938 ] maxwellguo commented on CASSANDRA-18464: I feel that this ci must be able to pass（except for build stage）, as the there lack of relevant ut test cases for Direct IO and the use_jna_for_commitlog_io is set to fasle so the related code are not coverd. > Enable Direct I/O For CommitLog Files > - > > Key: CASSANDRA-18464 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18464 > Project: Cassandra > Issue Type: New Feature > Components: Local/Commit Log >Reporter: Josh McKenzie >Assignee: Amit Pawar >Priority: Normal > Fix For: 6.x > > Attachments: UseDirectIOFeatureForCommitLogFiles.patch > > > Relocating from [dev@ email > thread.|https://lists.apache.org/thread/j6ny17q2rhkp7jxvwxm69dd6v1dozjrg] > > I shared my investigation about Commitlog I/O issue on large core count > system in my previous email dated July-22 and link to the thread is given > below. > [https://lists.apache.org/thread/xc5ocog2qz2v2gnj4xlw5hbthfqytx2n] > Basically, two solutions looked possible to improve the CommitLog I/O. > # Multi-threaded syncing > # Using Direct-IO through JNA > I worked on 2nd option considering the following benefit compared to the > first one > # Direct I/O read/write throughput is very high compared to non-Direct I/O. > Learnt through FIO benchmarking. > # Reduces kernel file cache uses which in-turn reduces kernel I/O activity > for Commitlog files only. > # Overall CPU usage reduced for flush activity. JVisualvm shows CPU usage < > 30% for Commitlog syncer thread with Direct I/O feature > # Direct I/O implementation is easier compared to multi-threaded > As per the community suggestion, less in code complex is good to have. Direct > I/O enablement looked promising but there was one issue. > Java version 8 does not have native support to enable Direct I/O. So, JNA > library usage is must. The same implementation should also work across other > versions of Java (like 11 and beyond). > I have completed Direct I/O implementation and summary of the attached patch > changes are given below. > # This implementation is not using Java file channels and file is opened > through JNA to use Direct I/O feature. > # New Segment are defined named “DirectIOSegment” for Direct I/O and > “NonDirectIOSegment” for non-direct I/O (NonDirectIOSegment is test purpose > only). > # JNA write call is used to flush the changes. > # New helper functions are defined in NativeLibrary.java and platform > specific file. Currently tested on Linux only. > # Patch allows user to configure optimum block size and alignment if > default values are not OK for CommitLog disk. > # Following configuration options are provided in Cassandra.yaml file > a. use_jna_for_commitlog_io : to use jna feature > b. use_direct_io_for_commitlog : to use Direct I/O feature. > c. direct_io_minimum_block_alignment: 512 (default) > d. nvme_disk_block_size: 32MiB (default and can be changed as per the > required size) > Test matrix is complex so CommitLog related testcases and TPCx-IOT benchmark > was tested. It works with both Java 8 and 11 versions. Compressed and > Encrypted based segments are not supported yet and it can be enabled later > based on the Community feedback. > Following improvement are seen with Direct I/O enablement. > # 32 cores >= ~15% > # 64 cores >= ~80% > Also, another observation would like to share here. Reading Commitlog files > with Direct I/O might help in reducing node bring-up time after the node > crash. > Tested with commit ID: 91f6a9aca8d3c22a03e68aa901a0b154d960ab07 > The attached patch enables Direct I/O feature for Commitlog files. Please > check and share your feedback. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18336) Sstables were cleared when OOM and best_effort is used
[
https://issues.apache.org/jira/browse/CASSANDRA-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17713839#comment-17713839
]
maxwellguo commented on CASSANDRA-18336:
+1 on this, [~brandon.williams] What's your opinion?
> Sstables were cleared when OOM and best_effort is used
> --
>
> Key: CASSANDRA-18336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18336
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: NAIZHEN QUE
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 4031679897782_.pic.jpg, 4241679905694_.pic.jpg,
> system.log.2023-02-21.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> 1.When this exception occurs in the system
> {code:java}
> //
> ERROR [CompactionExecutor:351627] 2023-02-21 17:59:20,721
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:351627,1,main]
> org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:167)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246)
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61)
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104)
> at
> org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:365)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:172)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:124)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:64)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:137)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:193)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:298)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1016)
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:163)
> ... 23 common frames omitted
> Caused by: java.lang.OutOfMemoryError: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method)
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1013)
> {code}
> 2.Restart the node, Verifying logfile transaction ,All sstables are deleted
> {code:java}
> // code placeholder
> INFO [main] 2023-02-21 18:00:23,350 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Index.db
>
> INFO [main] 2023-02-21 18:00:23,615 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Data.db
>
> INFO [main] 2023-02-21 18:00:46,504 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb_txn_compaction_c923b230-b077-11ed-a081-5d5a5c990823.log
>
> INFO [main] 2023-02-21 18:00:46,510 LogTransaction.java:536 - Verifying
> logfile transaction
> [nb_txn_compaction_461935b0-b1ce-11ed-a081-5d5a5c990823.log in
>
[jira] [Updated] (CASSANDRA-18336) Sstables were cleared when OOM and best_effort is used
[
https://issues.apache.org/jira/browse/CASSANDRA-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maxwellguo updated CASSANDRA-18336:
---
Reviewers: maxwellguo
> Sstables were cleared when OOM and best_effort is used
> --
>
> Key: CASSANDRA-18336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18336
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: NAIZHEN QUE
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 4031679897782_.pic.jpg, 4241679905694_.pic.jpg,
> system.log.2023-02-21.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> 1.When this exception occurs in the system
> {code:java}
> //
> ERROR [CompactionExecutor:351627] 2023-02-21 17:59:20,721
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:351627,1,main]
> org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:167)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246)
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61)
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104)
> at
> org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:365)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:172)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:124)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:64)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:137)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:193)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:298)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1016)
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:163)
> ... 23 common frames omitted
> Caused by: java.lang.OutOfMemoryError: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method)
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1013)
> {code}
> 2.Restart the node, Verifying logfile transaction ,All sstables are deleted
> {code:java}
> // code placeholder
> INFO [main] 2023-02-21 18:00:23,350 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Index.db
>
> INFO [main] 2023-02-21 18:00:23,615 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Data.db
>
> INFO [main] 2023-02-21 18:00:46,504 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb_txn_compaction_c923b230-b077-11ed-a081-5d5a5c990823.log
>
> INFO [main] 2023-02-21 18:00:46,510 LogTransaction.java:536 - Verifying
> logfile transaction
> [nb_txn_compaction_461935b0-b1ce-11ed-a081-5d5a5c990823.log in
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b]
> INFO [main] 2023-02-21 18:00:46,517
[jira] [Commented] (CASSANDRA-18336) Sstables were cleared when OOM and best_effort is used
[
https://issues.apache.org/jira/browse/CASSANDRA-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17713344#comment-17713344
]
maxwellguo commented on CASSANDRA-18336:
I think it's fine, too . and I left some comments on this pr.
> Sstables were cleared when OOM and best_effort is used
> --
>
> Key: CASSANDRA-18336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18336
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: NAIZHEN QUE
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 4031679897782_.pic.jpg, 4241679905694_.pic.jpg,
> system.log.2023-02-21.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> 1.When this exception occurs in the system
> {code:java}
> //
> ERROR [CompactionExecutor:351627] 2023-02-21 17:59:20,721
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:351627,1,main]
> org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:167)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246)
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61)
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104)
> at
> org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:365)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:172)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:124)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:64)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:137)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:193)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:298)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1016)
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:163)
> ... 23 common frames omitted
> Caused by: java.lang.OutOfMemoryError: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method)
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1013)
> {code}
> 2.Restart the node, Verifying logfile transaction ,All sstables are deleted
> {code:java}
> // code placeholder
> INFO [main] 2023-02-21 18:00:23,350 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Index.db
>
> INFO [main] 2023-02-21 18:00:23,615 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Data.db
>
> INFO [main] 2023-02-21 18:00:46,504 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb_txn_compaction_c923b230-b077-11ed-a081-5d5a5c990823.log
>
> INFO [main] 2023-02-21 18:00:46,510 LogTransaction.java:536 - Verifying
> logfile transaction
> [nb_txn_compaction_461935b0-b1ce-11ed-a081-5d5a5c990823.log in
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b]
> INFO
[jira] [Updated] (CASSANDRA-18394) Add nodetool to show top n read/write table
[ https://issues.apache.org/jira/browse/CASSANDRA-18394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18394: --- Resolution: Won't Fix Status: Resolved (was: Triage Needed) > Add nodetool to show top n read/write table > > > Key: CASSANDRA-18394 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18394 > Project: Cassandra > Issue Type: Improvement >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > add nodetool to show the read/write top n table name -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18336) Sstables were cleared when restarting
[
https://issues.apache.org/jira/browse/CASSANDRA-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712193#comment-17712193
]
maxwellguo edited comment on CASSANDRA-18336 at 4/14/23 5:44 AM:
-
It seems the dir data have been remove when using mmap and the memory is
limited and oom occurs
was (Author: maxwellguo):
It seems the dir data have been remove when using mmap and the memmory is
limited
> Sstables were cleared when restarting
> -
>
> Key: CASSANDRA-18336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18336
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: NAIZHEN QUE
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: 4031679897782_.pic.jpg, 4241679905694_.pic.jpg,
> system.log.2023-02-21.0
>
>
> 1.When this exception occurs in the system
> {code:java}
> //
> ERROR [CompactionExecutor:351627] 2023-02-21 17:59:20,721
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:351627,1,main]
> org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:167)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246)
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61)
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104)
> at
> org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:365)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:172)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:124)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:64)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:137)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:193)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:298)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1016)
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:163)
> ... 23 common frames omitted
> Caused by: java.lang.OutOfMemoryError: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method)
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1013)
> {code}
> 2.Restart the node, Verifying logfile transaction ,All sstables are deleted
> {code:java}
> // code placeholder
> INFO [main] 2023-02-21 18:00:23,350 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Index.db
>
> INFO [main] 2023-02-21 18:00:23,615 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Data.db
>
> INFO [main] 2023-02-21 18:00:46,504 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb_txn_compaction_c923b230-b077-11ed-a081-5d5a5c990823.log
>
> INFO [main] 2023-02-21 18:00:46,510 LogTransaction.java:536 - Verifying
> logfile transaction
> [nb_txn_compaction_461935b0-b1ce-11ed-a081-5d5a5c990823.log in
>
[jira] [Commented] (CASSANDRA-18336) Sstables were cleared when restarting
[
https://issues.apache.org/jira/browse/CASSANDRA-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17712193#comment-17712193
]
maxwellguo commented on CASSANDRA-18336:
It seems the dir data have been remove when using mmap and the memmory is
limited
> Sstables were cleared when restarting
> -
>
> Key: CASSANDRA-18336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18336
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: NAIZHEN QUE
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: 4031679897782_.pic.jpg, 4241679905694_.pic.jpg,
> system.log.2023-02-21.0
>
>
> 1.When this exception occurs in the system
> {code:java}
> //
> ERROR [CompactionExecutor:351627] 2023-02-21 17:59:20,721
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:351627,1,main]
> org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:167)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246)
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61)
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104)
> at
> org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:365)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:172)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:124)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:64)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:137)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:193)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:298)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1016)
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:163)
> ... 23 common frames omitted
> Caused by: java.lang.OutOfMemoryError: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method)
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1013)
> {code}
> 2.Restart the node, Verifying logfile transaction ,All sstables are deleted
> {code:java}
> // code placeholder
> INFO [main] 2023-02-21 18:00:23,350 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Index.db
>
> INFO [main] 2023-02-21 18:00:23,615 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Data.db
>
> INFO [main] 2023-02-21 18:00:46,504 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb_txn_compaction_c923b230-b077-11ed-a081-5d5a5c990823.log
>
> INFO [main] 2023-02-21 18:00:46,510 LogTransaction.java:536 - Verifying
> logfile transaction
> [nb_txn_compaction_461935b0-b1ce-11ed-a081-5d5a5c990823.log in
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b]
> INFO [main] 2023-02-21 18:00:46,517 LogTransaction.java:240 - Unfinished
> transaction log, deleting
[jira] [Commented] (CASSANDRA-18336) Sstables were cleared when restarting
[
https://issues.apache.org/jira/browse/CASSANDRA-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711763#comment-17711763
]
maxwellguo commented on CASSANDRA-18336:
of course~~~,and this behavior is also incorrect for databases. and I think
this [~yukim] may do some help too.
> Sstables were cleared when restarting
> -
>
> Key: CASSANDRA-18336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18336
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: NAIZHEN QUE
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: 4031679897782_.pic.jpg, 4241679905694_.pic.jpg,
> system.log.2023-02-21.0
>
>
> 1.When this exception occurs in the system
> {code:java}
> //
> ERROR [CompactionExecutor:351627] 2023-02-21 17:59:20,721
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:351627,1,main]
> org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:167)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246)
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61)
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104)
> at
> org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:365)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:172)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:124)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:64)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:137)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:193)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:298)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1016)
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:163)
> ... 23 common frames omitted
> Caused by: java.lang.OutOfMemoryError: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method)
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1013)
> {code}
> 2.Restart the node, Verifying logfile transaction ,All sstables are deleted
> {code:java}
> // code placeholder
> INFO [main] 2023-02-21 18:00:23,350 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Index.db
>
> INFO [main] 2023-02-21 18:00:23,615 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Data.db
>
> INFO [main] 2023-02-21 18:00:46,504 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb_txn_compaction_c923b230-b077-11ed-a081-5d5a5c990823.log
>
> INFO [main] 2023-02-21 18:00:46,510 LogTransaction.java:536 - Verifying
> logfile transaction
> [nb_txn_compaction_461935b0-b1ce-11ed-a081-5d5a5c990823.log in
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b]
> INFO [main] 2023-02-21 18:00:46,517 LogTransaction.java:240 -
[jira] [Commented] (CASSANDRA-18105) TRUNCATED data come back after a restart or upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711661#comment-17711661
]
maxwellguo commented on CASSANDRA-18105:
[~maedhroz]It seems can also reproduce .
> TRUNCATED data come back after a restart or upgrade
> ---
>
> Key: CASSANDRA-18105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18105
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Ke Han
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x
>
>
> When we use the TRUNCATE command to delete all data in the table, the deleted
> data come back after a node restart or upgrade. This problem happens at the
> latest releases (2.2.19, 3.0.28, or 4.0.7)
> h1. Steps to reproduce
> h2. To reproduce it at release (3.0.28 or 4.0.7)
> Start up a single Cassandra node. Using the default configuration and execute
> the following cqlsh commands.
> {code:java}
> CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = { 'class' :
> 'SimpleStrategy', 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c3 TEXT,c4 TEXT,c2 INT,c1 TEXT, PRIMARY KEY (c1, c2, c3
> ));
> INSERT INTO ks.tb (c3, c1, c2) VALUES ('val1','val2',1);
> CREATE INDEX IF NOT EXISTS tb ON ks.tb ( c3);
> TRUNCATE TABLE ks.tb;
> DROP INDEX IF EXISTS ks.tb; {code}
> Execute a read command
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> (0 rows) {code}
> Then, we flush the node and kill the Cassandra daemon by
> {code:java}
> bin/nodetool flush
> pgrep -f cassandra | xargs kill -9 {code}
> We restart the node. When the node has started, perform the same read, and
> the deleted data comes back again.
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> 1
> (1 rows) {code}
> h2. To reproduce it at release (2.2.19)
> We don't need to kill the Cassandra daemon. Use bin/nodetool stopdaemon is
> enough. The other steps are the same as reproducing it at 4.0.7 or 3.0.28.
> {code:java}
> bin/nodetool -h :::127.0.0.1 flush
> bin/nodetool -h :::127.0.0.1 stopdaemon{code}
>
> I have put the full log to reproduce it for release 4.0.7 and 2.2.19 in the
> comments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18105) TRUNCATED data come back after a restart or upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711468#comment-17711468
]
maxwellguo edited comment on CASSANDRA-18105 at 4/12/23 4:17 PM:
-
I have found out the reason that time as the data still exist in commitlog and
the lowbond of commitlog have changed as index been created（The description may
not be clear because it is too long）.That time I just discard the commitlog ‘s
data at truncate stage，And this problem is solved。At that time, as the patch is
not ready and I was busy with other things temporarily, so this issue did not
progress further. And thank you for your help [~smiklosovic] I think I can
help to do some review latter
was (Author: maxwellguo):
I have found out the reason that time as the data still exist in commitlog and
the lowbond of commitlog have changed as index been created（The description may
not be clear because it is too long）.That time I just discard the commitlog ‘s
data at truncate stage，And this problem is solved。At that time, as the patch is
not ready and I was busy with other things temporarily, so this issue did not
progress further. And thank you for your help [~smiklosovic]
> TRUNCATED data come back after a restart or upgrade
> ---
>
> Key: CASSANDRA-18105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18105
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Ke Han
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x
>
>
> When we use the TRUNCATE command to delete all data in the table, the deleted
> data come back after a node restart or upgrade. This problem happens at the
> latest releases (2.2.19, 3.0.28, or 4.0.7)
> h1. Steps to reproduce
> h2. To reproduce it at release (3.0.28 or 4.0.7)
> Start up a single Cassandra node. Using the default configuration and execute
> the following cqlsh commands.
> {code:java}
> CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = { 'class' :
> 'SimpleStrategy', 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c3 TEXT,c4 TEXT,c2 INT,c1 TEXT, PRIMARY KEY (c1, c2, c3
> ));
> INSERT INTO ks.tb (c3, c1, c2) VALUES ('val1','val2',1);
> CREATE INDEX IF NOT EXISTS tb ON ks.tb ( c3);
> TRUNCATE TABLE ks.tb;
> DROP INDEX IF EXISTS ks.tb; {code}
> Execute a read command
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> (0 rows) {code}
> Then, we flush the node and kill the Cassandra daemon by
> {code:java}
> bin/nodetool flush
> pgrep -f cassandra | xargs kill -9 {code}
> We restart the node. When the node has started, perform the same read, and
> the deleted data comes back again.
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> 1
> (1 rows) {code}
> h2. To reproduce it at release (2.2.19)
> We don't need to kill the Cassandra daemon. Use bin/nodetool stopdaemon is
> enough. The other steps are the same as reproducing it at 4.0.7 or 3.0.28.
> {code:java}
> bin/nodetool -h :::127.0.0.1 flush
> bin/nodetool -h :::127.0.0.1 stopdaemon{code}
>
> I have put the full log to reproduce it for release 4.0.7 and 2.2.19 in the
> comments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18105) TRUNCATED data come back after a restart or upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711468#comment-17711468
]
maxwellguo commented on CASSANDRA-18105:
I have found out the reason that time as the data still exist in commitlog and
the lowbond of commitlog have changed as index been created（The description may
not be clear because it is too long）.That time I just discard the commitlog ‘s
data at truncate stage，And this problem is solved。At that time, as the patch is
not ready and I was busy with other things temporarily, so this issue did not
progress further. And thank you for your help [~smiklosovic]
> TRUNCATED data come back after a restart or upgrade
> ---
>
> Key: CASSANDRA-18105
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18105
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Ke Han
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x
>
>
> When we use the TRUNCATE command to delete all data in the table, the deleted
> data come back after a node restart or upgrade. This problem happens at the
> latest releases (2.2.19, 3.0.28, or 4.0.7)
> h1. Steps to reproduce
> h2. To reproduce it at release (3.0.28 or 4.0.7)
> Start up a single Cassandra node. Using the default configuration and execute
> the following cqlsh commands.
> {code:java}
> CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = { 'class' :
> 'SimpleStrategy', 'replication_factor' : 1 };
> CREATE TABLE ks.tb (c3 TEXT,c4 TEXT,c2 INT,c1 TEXT, PRIMARY KEY (c1, c2, c3
> ));
> INSERT INTO ks.tb (c3, c1, c2) VALUES ('val1','val2',1);
> CREATE INDEX IF NOT EXISTS tb ON ks.tb ( c3);
> TRUNCATE TABLE ks.tb;
> DROP INDEX IF EXISTS ks.tb; {code}
> Execute a read command
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> (0 rows) {code}
> Then, we flush the node and kill the Cassandra daemon by
> {code:java}
> bin/nodetool flush
> pgrep -f cassandra | xargs kill -9 {code}
> We restart the node. When the node has started, perform the same read, and
> the deleted data comes back again.
> {code:java}
> cqlsh> SELECT c2 FROM ks.tb;
> c2
>
> 1
> (1 rows) {code}
> h2. To reproduce it at release (2.2.19)
> We don't need to kill the Cassandra daemon. Use bin/nodetool stopdaemon is
> enough. The other steps are the same as reproducing it at 4.0.7 or 3.0.28.
> {code:java}
> bin/nodetool -h :::127.0.0.1 flush
> bin/nodetool -h :::127.0.0.1 stopdaemon{code}
>
> I have put the full log to reproduce it for release 4.0.7 and 2.2.19 in the
> comments.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18336) Sstables were cleared when restarting
[
https://issues.apache.org/jira/browse/CASSANDRA-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711414#comment-17711414
]
maxwellguo edited comment on CASSANDRA-18336 at 4/12/23 2:14 PM:
-
this is right and will mark the dir unread and unwriteable ，I am trying to find
way to distingush betweent out of memory and real disk failure causes. In this
case all seems to be io exception.
just exploring
was (Author: maxwellguo):
this is right and will mark the dir unread and unwriteable ，I am trying to find
way to distingush betweent out of memory and real disk failure causes. In this
case all seems to be io exception.
> Sstables were cleared when restarting
> -
>
> Key: CASSANDRA-18336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18336
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: NAIZHEN QUE
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: 4031679897782_.pic.jpg, 4241679905694_.pic.jpg,
> system.log.2023-02-21.0
>
>
> 1.When this exception occurs in the system
> {code:java}
> //
> ERROR [CompactionExecutor:351627] 2023-02-21 17:59:20,721
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:351627,1,main]
> org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:167)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246)
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61)
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104)
> at
> org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:365)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:172)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:124)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:64)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:137)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:193)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:298)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1016)
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:163)
> ... 23 common frames omitted
> Caused by: java.lang.OutOfMemoryError: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method)
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1013)
> {code}
> 2.Restart the node, Verifying logfile transaction ,All sstables are deleted
> {code:java}
> // code placeholder
> INFO [main] 2023-02-21 18:00:23,350 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Index.db
>
> INFO [main] 2023-02-21 18:00:23,615 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Data.db
>
> INFO [main] 2023-02-21 18:00:46,504 LogTransaction.java:240 - Unfinished
> transaction log, deleting
>
[jira] [Commented] (CASSANDRA-18336) Sstables were cleared when restarting
[
https://issues.apache.org/jira/browse/CASSANDRA-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711414#comment-17711414
]
maxwellguo commented on CASSANDRA-18336:
this is right and will mark the dir unread and unwriteable ，I am trying to find
way to distingush betweent out of memory and real disk failure causes. In this
case all seems to be io exception.
> Sstables were cleared when restarting
> -
>
> Key: CASSANDRA-18336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18336
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: NAIZHEN QUE
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: 4031679897782_.pic.jpg, 4241679905694_.pic.jpg,
> system.log.2023-02-21.0
>
>
> 1.When this exception occurs in the system
> {code:java}
> //
> ERROR [CompactionExecutor:351627] 2023-02-21 17:59:20,721
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:351627,1,main]
> org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:167)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246)
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61)
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104)
> at
> org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:365)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:172)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:124)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:64)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:137)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:193)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:298)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1016)
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:163)
> ... 23 common frames omitted
> Caused by: java.lang.OutOfMemoryError: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method)
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1013)
> {code}
> 2.Restart the node, Verifying logfile transaction ,All sstables are deleted
> {code:java}
> // code placeholder
> INFO [main] 2023-02-21 18:00:23,350 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Index.db
>
> INFO [main] 2023-02-21 18:00:23,615 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Data.db
>
> INFO [main] 2023-02-21 18:00:46,504 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb_txn_compaction_c923b230-b077-11ed-a081-5d5a5c990823.log
>
> INFO [main] 2023-02-21 18:00:46,510 LogTransaction.java:536 - Verifying
> logfile transaction
> [nb_txn_compaction_461935b0-b1ce-11ed-a081-5d5a5c990823.log in
>
[jira] [Updated] (CASSANDRA-18398) CEP-25: Trie-indexed SSTable format
[ https://issues.apache.org/jira/browse/CASSANDRA-18398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18398: --- Reviewers: Caleb Rackliffe, maxwellguo (was: Caleb Rackliffe) > CEP-25: Trie-indexed SSTable format > --- > > Key: CASSANDRA-18398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18398 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Normal > Fix For: 5.x > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Implementation of Big Trie-Indexed (BTI) SSTable format, per > [CEP-25|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-25%3A+Trie-indexed+SSTable+format]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18438) Refactor the code for public cloud platform snitch to be configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-18438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17710417#comment-17710417 ] maxwellguo commented on CASSANDRA-18438: [~xgerman42] yeah, that is what I want to do ~~~ > Refactor the code for public cloud platform snitch to be configurable > - > > Key: CASSANDRA-18438 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18438 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > Fix For: 5.x > > > Nowadays we have got about four public cloud platform snitchs ： EC2 snitch > for aws, google cloud snitch for google cloud, alibaba cloud snitch for > alibaba cloud and multi region snitch for ec2. And the common place for the > first three is that we just need to query the zone center to get the ec2 / > ecs id , so I think we can refactor the code , and if some new public cloud > platform want to add one more snitch for himself, there is no need to pull a > pr for him and configure some options in yaml is enough . > Besides it would be even better that we may reuse the multic region snitch > for ec2 for other public cloud platform. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18438) Refactor the code for public cloud platform snitch to be configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-18438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18438: --- Change Category: Operability Complexity: Normal Component/s: Local/Other Fix Version/s: 5.x Status: Open (was: Triage Needed) > Refactor the code for public cloud platform snitch to be configurable > - > > Key: CASSANDRA-18438 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18438 > Project: Cassandra > Issue Type: Improvement > Components: Local/Other >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > Fix For: 5.x > > > Nowadays we have got about four public cloud platform snitchs ： EC2 snitch > for aws, google cloud snitch for google cloud, alibaba cloud snitch for > alibaba cloud and multi region snitch for ec2. And the common place for the > first three is that we just need to query the zone center to get the ec2 / > ecs id , so I think we can refactor the code , and if some new public cloud > platform want to add one more snitch for himself, there is no need to pull a > pr for him and configure some options in yaml is enough . > Besides it would be even better that we may reuse the multic region snitch > for ec2 for other public cloud platform. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18438) Refactor the code for public cloud platform snitch to be configurable
maxwellguo created CASSANDRA-18438: -- Summary: Refactor the code for public cloud platform snitch to be configurable Key: CASSANDRA-18438 URL: https://issues.apache.org/jira/browse/CASSANDRA-18438 Project: Cassandra Issue Type: Improvement Reporter: maxwellguo Assignee: maxwellguo Nowadays we have got about four public cloud platform snitchs ： EC2 snitch for aws, google cloud snitch for google cloud, alibaba cloud snitch for alibaba cloud and multi region snitch for ec2. And the common place for the first three is that we just need to query the zone center to get the ec2 / ecs id , so I think we can refactor the code , and if some new public cloud platform want to add one more snitch for himself, there is no need to pull a pr for him and configure some options in yaml is enough . Besides it would be even better that we may reuse the multic region snitch for ec2 for other public cloud platform. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18394) Add nodetool to show top n read/write table
[ https://issues.apache.org/jira/browse/CASSANDRA-18394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707553#comment-17707553 ] maxwellguo commented on CASSANDRA-18394: It seems that there is something differences betweet 3.11.x and trunk(4.x) with toppartitions~~~:( and I think i should backport the feature . 3.11.x needs the keyspace and table informations > Add nodetool to show top n read/write table > > > Key: CASSANDRA-18394 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18394 > Project: Cassandra > Issue Type: Improvement >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > add nodetool to show the read/write top n table name -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18394) Add nodetool to show top n read/write table
[ https://issues.apache.org/jira/browse/CASSANDRA-18394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707119#comment-17707119 ] maxwellguo commented on CASSANDRA-18394: add tool support though we may can get from monitor web site. > Add nodetool to show top n read/write table > > > Key: CASSANDRA-18394 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18394 > Project: Cassandra > Issue Type: Improvement >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > add nodetool to show the read/write top n table name -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18394) Add nodetool to show top n read/write table
maxwellguo created CASSANDRA-18394: -- Summary: Add nodetool to show top n read/write table Key: CASSANDRA-18394 URL: https://issues.apache.org/jira/browse/CASSANDRA-18394 Project: Cassandra Issue Type: Improvement Reporter: maxwellguo Assignee: maxwellguo add nodetool to show the read/write top n table name -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18336) Sstables were cleared when restarting
[
https://issues.apache.org/jira/browse/CASSANDRA-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17705277#comment-17705277
]
maxwellguo commented on CASSANDRA-18336:
Let me check whether it is a bug or not
> Sstables were cleared when restarting
> -
>
> Key: CASSANDRA-18336
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18336
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: NAIZHEN QUE
>Priority: Normal
> Fix For: 4.0.x
>
> Attachments: 4031679897782_.pic.jpg, 4241679905694_.pic.jpg,
> system.log.2023-02-21.0
>
>
> 1.When this exception occurs in the system
> {code:java}
> //
> ERROR [CompactionExecutor:351627] 2023-02-21 17:59:20,721
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:351627,1,main]
> org.apache.cassandra.io.FSReadError: java.io.IOException: Map failed
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:167)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310)
> at
> org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246)
> at
> org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:73)
> at
> org.apache.cassandra.io.util.MmappedRegions.(MmappedRegions.java:61)
> at
> org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104)
> at
> org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:365)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.maybeReopenEarly(SSTableRewriter.java:172)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:124)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:64)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:137)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:193)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:298)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1016)
> at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:163)
> ... 23 common frames omitted
> Caused by: java.lang.OutOfMemoryError: Map failed
> at java.base/sun.nio.ch.FileChannelImpl.map0(Native Method)
> at java.base/sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:1013)
> {code}
> 2.Restart the node, Verifying logfile transaction ,All sstables are deleted
> {code:java}
> // code placeholder
> INFO [main] 2023-02-21 18:00:23,350 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Index.db
>
> INFO [main] 2023-02-21 18:00:23,615 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb-8819408-big-Data.db
>
> INFO [main] 2023-02-21 18:00:46,504 LogTransaction.java:240 - Unfinished
> transaction log, deleting
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b/nb_txn_compaction_c923b230-b077-11ed-a081-5d5a5c990823.log
>
> INFO [main] 2023-02-21 18:00:46,510 LogTransaction.java:536 - Verifying
> logfile transaction
> [nb_txn_compaction_461935b0-b1ce-11ed-a081-5d5a5c990823.log in
> /historyData/cassandra/data/kairosdb/data_points-870fab7087ba11eb8b50d3c6960df21b]
> INFO [main] 2023-02-21 18:00:46,517 LogTransaction.java:240 - Unfinished
> transaction log, deleting
>
[jira] [Comment Edited] (CASSANDRA-18301) Let the user select the sstable version to write
[ https://issues.apache.org/jira/browse/CASSANDRA-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17704259#comment-17704259 ] maxwellguo edited comment on CASSANDRA-18301 at 3/23/23 5:03 PM: - If we support the ability for user to write specified version ，do we need to add a tool to dynamic change the version，or do we permit to change this the version？ If we do ，a new question is ，do we permit more than one version sstable ‘s existence and what is the behavior of compaction ，read ，read repair ，repair and so on ；if we don‘t that means we can only use this every time we restart the cluster？ besides some tools of Cassandra may only support high version，and if let user write lower version ，it seems that some tools may fail for example Cassandra-17698 ，though it is not checkin now .I think we at least shoul support doc for these. was (Author: maxwellguo): If we support the ability for user to write specified version ，do we need to add a tool to dynamic change the version，or do we permit to change this the version？ If we do ，a new question is ，do we permit more than one version sstable ‘s existence and what is the behavior of compaction ，read ，read repair ，repair and so on ；if we don‘t that means we can only use this every time we restart the cluster？ > Let the user select the sstable version to write > > > Key: CASSANDRA-18301 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18301 > Project: Cassandra > Issue Type: New Feature > Components: Local/Config, Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18301) Let the user select the sstable version to write
[ https://issues.apache.org/jira/browse/CASSANDRA-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17704259#comment-17704259 ] maxwellguo commented on CASSANDRA-18301: If we support the ability for user to write specified version ，do we need to add a tool to dynamic change the version，or do we permit to change this the version？ If we do ，a new question is ，do we permit more than one version sstable ‘s existence and what is the behavior of compaction ，read ，read repair ，repair and so on ；if we don‘t that means we can only use this every time we restart the cluster？ > Let the user select the sstable version to write > > > Key: CASSANDRA-18301 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18301 > Project: Cassandra > Issue Type: New Feature > Components: Local/Config, Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18301) Let the user select the sstable version to write
[ https://issues.apache.org/jira/browse/CASSANDRA-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17704246#comment-17704246 ] maxwellguo edited comment on CASSANDRA-18301 at 3/23/23 4:40 PM: - agree on using Cassandra version as using sstable format will be a threshold for users ，because they may only know the Cassandra version number is 4.0 but may hard to know sstable major version is nc for example. But I think the document must be very clear was (Author: maxwellguo): agree on using Cassandra version as using sstable format will be a threshold for users ，because they may only know the Cassandra version number is 4.0 but may hard to know sstable major version is nc for example. > Let the user select the sstable version to write > > > Key: CASSANDRA-18301 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18301 > Project: Cassandra > Issue Type: New Feature > Components: Local/Config, Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18301) Let the user select the sstable version to write
[ https://issues.apache.org/jira/browse/CASSANDRA-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17704246#comment-17704246 ] maxwellguo commented on CASSANDRA-18301: agree on using Cassandra version as using sstable format will be a threshold for users ，because they may only know the Cassandra version number is 4.0 but may hard to know sstable major version is nc for example. > Let the user select the sstable version to write > > > Key: CASSANDRA-18301 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18301 > Project: Cassandra > Issue Type: New Feature > Components: Local/Config, Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703491#comment-17703491
]
maxwellguo commented on CASSANDRA-18102:
Thanks [~paulo] and [~smiklosovic]
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17698) sstabledump errors when dumping data from index
[
https://issues.apache.org/jira/browse/CASSANDRA-17698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17701052#comment-17701052
]
maxwellguo commented on CASSANDRA-17698:
Sorry for the late reply, I didn't notice that the file format version didn't
change yet. I do aggree we should accept the path when the file format come to
the next version, as this patch only made a smll change to file content.
[~adelapena]
As for CASSANDRA-18254 I later think may be CASSANDRA-17698 is enough，So I
closed it.
> sstabledump errors when dumping data from index
> ---
>
> Key: CASSANDRA-17698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17698
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Stefan Miklosovic
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 12h 40m
> Remaining Estimate: 0h
>
> {code:java}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE ks1.tb1 ( id text, name text, primary key (id));
> cqlsh> CREATE INDEX IF NOT EXISTS ON ks1.tb1(name);
> cqlsh> INSERT INTO ks1.tb1 (id, name ) VALUES ( '1', 'Joe');
> cqlsh> exit
> ./bin/nodetool flush
> ./tools/bin/sstabledump
> data/data/ks1/tb1-1c3c5f10ee4711ecab82eda2f44200b3/.tb1_name_idx/nb-1-big-Data.db
>
> [
> {
> "partition" : {
> "key" : [ "Joe" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "clustering" : [ ] } ] } ]Exception in thread "main"
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.db.marshal.PartitionerDefinedOrder.toJSONString(PartitionerDefinedOrder.java:87)
> at
> org.apache.cassandra.db.marshal.AbstractType.toJSONString(AbstractType.java:187)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeClustering(JsonTransformer.java:372)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:269)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:235)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:113)
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:214) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18338) org.apache.cassandra.distributed.test.ByteBuddyExamplesTest.countTest
maxwellguo created CASSANDRA-18338: -- Summary: org.apache.cassandra.distributed.test.ByteBuddyExamplesTest.countTest Key: CASSANDRA-18338 URL: https://issues.apache.org/jira/browse/CASSANDRA-18338 Project: Cassandra Issue Type: Bug Components: CI, Test/unit Reporter: maxwellguo jdk8 and jdk11 all faild at some time, jdk8 failed once, and jdk11 failed twice. for jdk 11: org.apache.cassandra.distributed.test.ByteBuddyExamplesTest.countTest-.jdk11 (from org.apache.cassandra.distributed.test.ByteBuddyExamplesTest-.jdk11) Error Message expected:<1> but was:<2> Stacktrace junit.framework.AssertionFailedError: expected:<1> but was:<2> at org.apache.cassandra.distributed.test.ByteBuddyExamplesTest.lambda$countTest$81c80a4a$1(ByteBuddyExamplesTest.java:93) at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96) at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) for jdk 8: Error Message expected:<1> but was:<4> Stacktrace junit.framework.AssertionFailedError: expected:<1> but was:<4> at org.apache.cassandra.distributed.test.ByteBuddyExamplesTest.lambda$countTest$81c80a4a$1(ByteBuddyExamplesTest.java:93) at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96) at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61) at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18296) Enhance nodetool tablehistograms with ignore options
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700540#comment-17700540 ] maxwellguo commented on CASSANDRA-18296: patch is ready and before that let me ask in ML > Enhance nodetool tablehistograms with ignore options > > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablehistograms to filter the output > of System tables and data tables. > As we know if we nodetool tablehistograms all tables will be output ,but > system table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. > It seems that tablestats got the -i option that can ignore keyspace, just add > this first, and then to see wether this can meet our needs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18296) Enhance nodetool tablehistograms with ignore options
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18296: --- Change Category: Operability Complexity: Normal Status: Open (was: Triage Needed) > Enhance nodetool tablehistograms with ignore options > > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablehistograms to filter the output > of System tables and data tables. > As we know if we nodetool tablehistograms all tables will be output ,but > system table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. > It seems that tablestats got the -i option that can ignore keyspace, just add > this first, and then to see wether this can meet our needs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18296) Enhance nodetool tablehistograms with ignore options
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696101#comment-17696101 ] maxwellguo edited comment on CASSANDRA-18296 at 3/15/23 7:45 AM: - tablehistograms's options are so old ,you can only chose ks table or ks.table nor all table . I want to change the option totally . 1. nodetool tablehistograms ks.tb1 ks.tb2 //list of tables with format keyspace.table 2.nodetool tablehistograms ks1 ks2 ks3 ...//list of keyspaces 3.nodetool tablehistograms -i ks1 ks2 //list of table histograms except for the keyspaces list behind the option -i 4.nodetool tablehistograns -i ks ks.tb // list tables except for table in keyspace ks and ks.tb table. 5.none option specified ,then all tables histograms will be output. this way , we can some what be compatible with the old options for nodetool tablehistograms ks.tb1, but the nodetool tablehistograms ks1 ks2 's result will be different from before, now is two keyspace,ks1 and ks2, but it was a table with keyspace name ks1, and table name ks2 before. [~smiklosovic] [~brandonwilliams] can you help me to take a look at this? was (Author: maxwellguo): tablehistograms's options are so old ,you can only chose ks table or ks.table nor all table . I want to change the option totally . 1. nodetool tablehistograms ks.tb1 ks.tb2 //list of tables with format keyspace.table 2.nodetool tablehistograms ks1 ks2 ks3 ...//list of keyspaces 3.nodetool tablehistograms -i ks1 ks2 //list of table histograms except for the keyspaces list behind the option -i 4.none option specified ,then all tables histograms will be output. this way , we can some what be compatible with the old options for nodetool tablehistograms ks.tb1, but the nodetool tablehistograms ks1 ks2 's result will be different from before, now is two keyspace,ks1 and ks2, but it was a table with keyspace name ks1, and table name ks2 before. [~smiklosovic] [~brandonwilliams] can you help me to take a look at this? > Enhance nodetool tablehistograms with ignore options > > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablehistograms to filter the output > of System tables and data tables. > As we know if we nodetool tablehistograms all tables will be output ,but > system table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. > It seems that tablestats got the -i option that can ignore keyspace, just add > this first, and then to see wether this can meet our needs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18296) Enhance nodetool tablehistograms with ignore options
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18296: --- Summary: Enhance nodetool tablehistograms with ignore options (was: tablehistograms add filter for data table and system table) > Enhance nodetool tablehistograms with ignore options > > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablehistograms to filter the output > of System tables and data tables. > As we know if we nodetool tablehistograms all tables will be output ,but > system table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. > It seems that tablestats got the -i option that can ignore keyspace, just add > this first, and then to see wether this can meet our needs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18325) dtest.bootstrap_test.TestBootstrap.test_cleanup
maxwellguo created CASSANDRA-18325:
--
Summary: dtest.bootstrap_test.TestBootstrap.test_cleanup
Key: CASSANDRA-18325
URL: https://issues.apache.org/jira/browse/CASSANDRA-18325
Project: Cassandra
Issue Type: Bug
Components: Test/dtest/python
Reporter: maxwellguo
assert not True
+ where True = >()
+where > = .is_set
Stacktrace
self =
def test_cleanup(self):
"""
@jira_ticket CASSANDRA-11179
Make sure we remove processed files during cleanup
"""
cluster = self.cluster
cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED',
'True')
cluster.set_configuration_options(values={'concurrent_compactors': 4})
cluster.populate(1)
cluster.start()
node1, = cluster.nodelist()
for x in range(0, 5):
node1.stress(['write', 'n=100k', 'no-warmup', '-schema',
'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)',
'replication(factor=1)', '-rate', 'threads=10'])
node1.flush()
node2 = new_node(cluster)
node2.start(wait_for_binary_proto=True)
event = threading.Event()
failed = threading.Event()
jobs = 1
thread = threading.Thread(target=self._monitor_datadir, args=(node1,
event, len(node1.get_sstables("keyspace1", "standard1")), jobs, failed))
thread.setDaemon(True)
thread.start()
node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs))
event.set()
thread.join()
> assert not failed.is_set()
E assert not True
E+ where True = >()
E+where > = .is_set
bootstrap_test.py:912: AssertionError
failed twice
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698961#comment-17698961
]
maxwellguo commented on CASSANDRA-18102:
I have fixed this problem and it seems to be the the difference for changing
Instant time to date . The fialed ut for java8/11 all passed. I will give this
fixed pr latter .
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698947#comment-17698947
]
maxwellguo commented on CASSANDRA-18102:
ok，let me take over this issue.
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] (CASSANDRA-18102) Add a virtual table to list snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-18102 ]
maxwellguo deleted comment on CASSANDRA-18102:
was (Author: maxwellguo):
yes or add order by in cql
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698930#comment-17698930
]
maxwellguo commented on CASSANDRA-18102:
yes or add order by in cql
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698918#comment-17698918
]
maxwellguo commented on CASSANDRA-18102:
let me take a look
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698291#comment-17698291
]
maxwellguo edited comment on CASSANDRA-18102 at 3/10/23 8:12 AM:
-
I merged your pr and also ran on a instance, thing seems to be all right .
[~smiklosovic]
was (Author: maxwellguo):
I merged your pr and also ran on a instance, thing seems to be all right .
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698291#comment-17698291
]
maxwellguo edited comment on CASSANDRA-18102 at 3/10/23 8:12 AM:
-
I merged your pr and also ran on a instance, thing seems to be all right .
was (Author: maxwellguo):
ok
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18310) Cassandra start up with ERROR of sstable_formats
[ https://issues.apache.org/jira/browse/CASSANDRA-18310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698313#comment-17698313 ] maxwellguo commented on CASSANDRA-18310: I make a new clone from remote trunk branch , And everything is fine ~~~ So It seems it's my own environment's problem. mark this as "not a problem" > Cassandra start up with ERROR of sstable_formats > > > Key: CASSANDRA-18310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18310 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: maxwellguo >Priority: Normal > Fix For: 5.0 > > > Exception (org.apache.cassandra.exceptions.ConfigurationException) > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml > Invalid yaml. Please remove properties [sstable_formats] from your > cassandra.yaml > ERROR [main] 2023-03-09 18:03:34,899 CassandraDaemon.java:911 - Exception > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18310) Cassandra start up with ERROR of sstable_formats
[ https://issues.apache.org/jira/browse/CASSANDRA-18310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698309#comment-17698309 ] maxwellguo commented on CASSANDRA-18310: ok I will try again > Cassandra start up with ERROR of sstable_formats > > > Key: CASSANDRA-18310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18310 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: maxwellguo >Priority: Normal > Fix For: 5.0 > > > Exception (org.apache.cassandra.exceptions.ConfigurationException) > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml > Invalid yaml. Please remove properties [sstable_formats] from your > cassandra.yaml > ERROR [main] 2023-03-09 18:03:34,899 CassandraDaemon.java:911 - Exception > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18310) Cassandra start up with ERROR of sstable_formats
[ https://issues.apache.org/jira/browse/CASSANDRA-18310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698302#comment-17698302 ] maxwellguo edited comment on CASSANDRA-18310 at 3/9/23 10:13 AM: - [~jlewandowski] Can you take a look ? I can't get away for the time being, and it seems that it has something to do with your patch step : 1.git pull the newest version code; 2.ant build && ant clean artifacts -Drelease=true get the bin tar package 3.tar xf the package 4.just ./cassandra -R and met this error. was (Author: maxwellguo): [~jlewandowski] Can you take a look ? I can't get away for the time being, and it seems that it has something to do with your patch > Cassandra start up with ERROR of sstable_formats > > > Key: CASSANDRA-18310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18310 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: maxwellguo >Priority: Normal > Fix For: 5.0 > > > Exception (org.apache.cassandra.exceptions.ConfigurationException) > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml > Invalid yaml. Please remove properties [sstable_formats] from your > cassandra.yaml > ERROR [main] 2023-03-09 18:03:34,899 CassandraDaemon.java:911 - Exception > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18310) Cassandra start up with ERROR of sstable_formats
[ https://issues.apache.org/jira/browse/CASSANDRA-18310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698302#comment-17698302 ] maxwellguo commented on CASSANDRA-18310: [~jlewandowski] Can you take a look ? I can't get away for the time being, and it seems that it has something to do with your patch > Cassandra start up with ERROR of sstable_formats > > > Key: CASSANDRA-18310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18310 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: maxwellguo >Priority: Normal > Fix For: 5.0 > > > Exception (org.apache.cassandra.exceptions.ConfigurationException) > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml > Invalid yaml. Please remove properties [sstable_formats] from your > cassandra.yaml > ERROR [main] 2023-03-09 18:03:34,899 CassandraDaemon.java:911 - Exception > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18310) Cassandra start up with ERROR of sstable_formats
[ https://issues.apache.org/jira/browse/CASSANDRA-18310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18310: --- Bug Category: Parent values: Correctness(12982) Complexity: Normal Component/s: Local/Config Discovered By: User Report Fix Version/s: 5.0 Severity: Critical Status: Open (was: Triage Needed) I just pull the newest version code ,and start the daemon , but failed twice. > Cassandra start up with ERROR of sstable_formats > > > Key: CASSANDRA-18310 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18310 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: maxwellguo >Priority: Normal > Fix For: 5.0 > > > Exception (org.apache.cassandra.exceptions.ConfigurationException) > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml > Invalid yaml. Please remove properties [sstable_formats] from your > cassandra.yaml > ERROR [main] 2023-03-09 18:03:34,899 CassandraDaemon.java:911 - Exception > encountered during startup: Invalid yaml. Please remove properties > [sstable_formats] from your cassandra.yaml -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18310) Cassandra start up with ERROR of sstable_formats
maxwellguo created CASSANDRA-18310: -- Summary: Cassandra start up with ERROR of sstable_formats Key: CASSANDRA-18310 URL: https://issues.apache.org/jira/browse/CASSANDRA-18310 Project: Cassandra Issue Type: Bug Reporter: maxwellguo Exception (org.apache.cassandra.exceptions.ConfigurationException) encountered during startup: Invalid yaml. Please remove properties [sstable_formats] from your cassandra.yaml Invalid yaml. Please remove properties [sstable_formats] from your cassandra.yaml ERROR [main] 2023-03-09 18:03:34,899 CassandraDaemon.java:911 - Exception encountered during startup: Invalid yaml. Please remove properties [sstable_formats] from your cassandra.yaml -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698291#comment-17698291
]
maxwellguo commented on CASSANDRA-18102:
ok
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698190#comment-17698190
]
maxwellguo edited comment on CASSANDRA-18102 at 3/9/23 7:19 AM:
[~paulo][~smiklosovic]I am so sorry that the old pr is discard due to an
operation mistake by me and I feeel very sorry. And I have create a new one .
[old pr link that discard by me, but some conversations is left
|https://github.com/apache/cassandra/pull/2117]
[the new pr that have been modify after resolved the conversations
|https://github.com/apache/cassandra/pull/2205]
[java 8
precommit|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/417/workflows/440693db-00d0-4d86-aaf8-008d94f48ed4]
[java11 precommit
|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/417/workflows/0b7c8599-2146-4f8f-83e7-347b60db5711]
java8 and java11 precommit ut are green
was (Author: maxwellguo):
[~paulo][~smiklosovic]I am so sorry that the old pr is discard due to an
operation mistake by me and I feeel very sorry. And I have create a new one .
[old pr link that discard by me, but some conversations is left
|https://github.com/apache/cassandra/pull/2117]
[the new pr that have been modify after resolved the conversations
|https://github.com/apache/cassandra/pull/2205]
[java 8
precommit|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/417/workflows/440693db-00d0-4d86-aaf8-008d94f48ed4]
[java11 precommit
|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/417/workflows/0b7c8599-2146-4f8f-83e7-347b60db5711]
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698190#comment-17698190
]
maxwellguo commented on CASSANDRA-18102:
[~paulo][~smiklosovic]I am so sorry that the old pr is discard due to an
operation mistake by me and I feeel very sorry. And I have create a new one .
[old pr link that discard by me, but some conversations is left
|https://github.com/apache/cassandra/pull/2117]
[the new pr that have been modify after resolved the conversations
|https://github.com/apache/cassandra/pull/2205]
[java 8
precommit|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/417/workflows/440693db-00d0-4d86-aaf8-008d94f48ed4]
[java11 precommit
|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/417/workflows/0b7c8599-2146-4f8f-83e7-347b60db5711]
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18305) Enhance nodetool compactionstats with existing MBean metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-18305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698140#comment-17698140
]
maxwellguo commented on CASSANDRA-18305:
nobody working on this ? [~smiklosovic] it seems that you have assigned this to
you ?
> Enhance nodetool compactionstats with existing MBean metrics
>
>
> Key: CASSANDRA-18305
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18305
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Brad Schoening
>Priority: Normal
>
> Nodetool compactionstats reports only on active compactions, if nothing is
> active, you see only:
> {quote}$nodetool compactionstats
> pending tasks: 0
> {quote}
> but in the MBean Compaction/TotalCompactionsCompleted there are recent
> statistic in events/second for:
> * Count
> * FifteenMinueRate
> * FiveMinueRate
> * MeanRate
> * OneMinuteRate
> 1) It would be useful to see in addition:
> {quote}pending tasks: 0
> compactions completed: 20
> 1 minute rate: 0/second
> 5 minute rate: 2.3/second
> 15 minute rate: 4.6/second
> {quote}
> 2) Since compaction_throughput_mb_per_sec is a throttling parameter in
> cassandra.yaml (default 64 MBps), it would be nice to show the actual
> compaction throughput and be able to observe if you're close to the limit.
> I.e.,
> {quote}compaction throughput 13.2 MBps / 16 MBps (82.5%)
> {quote}
> 3) for completness, compactionstats should list the number of concurrent
> compactors configured
> {quote}concurrent compactions permitted: 2
> {quote}
> or perhaps simply add to existing 'pending tasks' line:
> {quote}2 concurrent compactors, 0 pending tasks
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18102) Add a virtual table to list snapshots
[
https://issues.apache.org/jira/browse/CASSANDRA-18102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697764#comment-17697764
]
maxwellguo commented on CASSANDRA-18102:
java8 and java11 precommit are all green
||Heading 1||Heading 2||
|pr for trunk|[trunk|https://github.com/apache/cassandra/pull/2117]|
|java8|[java8|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/415/workflows/e86c7bec-5c94-4ea8-91e9-1b2f00e04e49]|
|java11|[java11|https://app.circleci.com/pipelines/github/Maxwell-Guo/cassandra/415/workflows/3a2906c5-d2d0-4a77-8c02-324b04fee26b]|
> Add a virtual table to list snapshots
> -
>
> Key: CASSANDRA-18102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18102
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Virtual Tables, Local/Snapshots
>Reporter: Paulo Motta
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> It should be possible to query a node's snapshots via virtual tables.
> The table should expose the same fields/columns as the
> [TableSnapshot|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/snapshot/TableSnapshot.java]
> class.
> Something along these lines:
> {noformat}
> cqlsh> SELECT * FROM system_views.snapshots;
>
> tag | keyspace_name | table_name | table_id | is_ephemeral | created_at |
> expires_at | directories
> +---++---+--+---++
> 1670460346841 | system | compaction_info |
> 123e4567-e89b-12d3-a456-426614174000 | false | 2022-12-08T00:45:47.108Z |
> null |
> {'/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/snapshots/1670460346841'}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697712#comment-17697712 ] maxwellguo commented on CASSANDRA-18294: [~smiklosovic]can you help to take a look agagin ? > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > Fix For: 3.0.x, 4.0.x, 4.1.x > > Time Spent: 10m > Remaining Estimate: 0h > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17698) sstabledump errors when dumping data from index
[
https://issues.apache.org/jira/browse/CASSANDRA-17698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697706#comment-17697706
]
maxwellguo commented on CASSANDRA-17698:
It seems CASSANDRA-17973 have also been closed, so this patch can be checkin
now ?
[~blambov][~adelapena]:D
> sstabledump errors when dumping data from index
> ---
>
> Key: CASSANDRA-17698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17698
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Stefan Miklosovic
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 12h 40m
> Remaining Estimate: 0h
>
> {code:java}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE ks1.tb1 ( id text, name text, primary key (id));
> cqlsh> CREATE INDEX IF NOT EXISTS ON ks1.tb1(name);
> cqlsh> INSERT INTO ks1.tb1 (id, name ) VALUES ( '1', 'Joe');
> cqlsh> exit
> ./bin/nodetool flush
> ./tools/bin/sstabledump
> data/data/ks1/tb1-1c3c5f10ee4711ecab82eda2f44200b3/.tb1_name_idx/nb-1-big-Data.db
>
> [
> {
> "partition" : {
> "key" : [ "Joe" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "clustering" : [ ] } ] } ]Exception in thread "main"
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.db.marshal.PartitionerDefinedOrder.toJSONString(PartitionerDefinedOrder.java:87)
> at
> org.apache.cassandra.db.marshal.AbstractType.toJSONString(AbstractType.java:187)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeClustering(JsonTransformer.java:372)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:269)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:235)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:113)
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:214) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[
https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697704#comment-17697704
]
maxwellguo edited comment on CASSANDRA-18294 at 3/8/23 3:02 AM:
"private FSError throwFSError(){ " [~curlylrt]
was (Author: maxwellguo):
"private FSError throwFSError(){ "
> die disk failure policy will not kill jvm as documented
> ---
>
> Key: CASSANDRA-18294
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18294
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Runtian Liu
>Assignee: Runtian Liu
>Priority: Normal
> Fix For: 3.0.x, 4.0.x, 4.1.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After Cassandra has successfully starts up with disk_failure_policy die, when
> encounter a file system error, Cassandra server will only throw exception
> instead of shut down gossip and client transport and kill JVM. Document:
> [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy]
>
> The reason for this is the default FS error handler is not handing policy die
> correctly. Instead of shutting down gossip and native transport, it throws an
> error.
>
> The JVMStabilityInspectorTest is passing because the error handler is not set
> so no exception is thrown.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[
https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697704#comment-17697704
]
maxwellguo commented on CASSANDRA-18294:
"private FSError throwFSError(){ "
> die disk failure policy will not kill jvm as documented
> ---
>
> Key: CASSANDRA-18294
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18294
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Runtian Liu
>Assignee: Runtian Liu
>Priority: Normal
> Fix For: 3.0.x, 4.0.x, 4.1.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After Cassandra has successfully starts up with disk_failure_policy die, when
> encounter a file system error, Cassandra server will only throw exception
> instead of shut down gossip and client transport and kill JVM. Document:
> [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy]
>
> The reason for this is the default FS error handler is not handing policy die
> correctly. Instead of shutting down gossip and native transport, it throws an
> error.
>
> The JVMStabilityInspectorTest is passing because the error handler is not set
> so no exception is thrown.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697684#comment-17697684 ] maxwellguo commented on CASSANDRA-18294: there is one left > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > Fix For: 3.0.x, 4.0.x, 4.1.x > > Time Spent: 10m > Remaining Estimate: 0h > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18296) tablehistograms add filter for data table and system table
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697677#comment-17697677 ] maxwellguo commented on CASSANDRA-18296: ok, thanks [~brandon.williams], I think I will ask on ML for consensus. > tablehistograms add filter for data table and system table > --- > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablehistograms to filter the output > of System tables and data tables. > As we know if we nodetool tablehistograms all tables will be output ,but > system table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. > It seems that tablestats got the -i option that can ignore keyspace, just add > this first, and then to see wether this can meet our needs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18294: --- Reviewers: Brandon Williams, maxwellguo, Stefan Miklosovic > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > Fix For: 3.0.x, 4.0.x, 4.1.x > > Time Spent: 10m > Remaining Estimate: 0h > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[
https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697675#comment-17697675
]
maxwellguo commented on CASSANDRA-18294:
I am + 1 on this patch , but there are some very small problems about code
format.
such as : some constructor should start a new line with '{', I have already
pointed out , [~curlylrt]can you help to take a look at them ? thanks.
> die disk failure policy will not kill jvm as documented
> ---
>
> Key: CASSANDRA-18294
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18294
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Runtian Liu
>Assignee: Runtian Liu
>Priority: Normal
> Fix For: 3.0.x, 4.0.x, 4.1.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After Cassandra has successfully starts up with disk_failure_policy die, when
> encounter a file system error, Cassandra server will only throw exception
> instead of shut down gossip and client transport and kill JVM. Document:
> [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy]
>
> The reason for this is the default FS error handler is not handing policy die
> correctly. Instead of shutting down gossip and native transport, it throws an
> error.
>
> The JVMStabilityInspectorTest is passing because the error handler is not set
> so no exception is thrown.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18296) tablehistograms add filter for data table and system table
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17697207#comment-17697207 ] maxwellguo commented on CASSANDRA-18296: If the options are changed, the way that users use it will change. For example : nodetool tablehistograms ks1 ks2 , for the old usage means show the tablehistograms of table : ks1.ks2; for the new usage means show the tablehistograms of all tables beyond to the two keyspace : ks1 and ks2; So I want to know if I can start this work directly now, don't need to pay attention to other things ,right ？ [~brandon.williams] > tablehistograms add filter for data table and system table > --- > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablehistograms to filter the output > of System tables and data tables. > As we know if we nodetool tablehistograms all tables will be output ,but > system table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. > It seems that tablestats got the -i option that can ignore keyspace, just add > this first, and then to see wether this can meet our needs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18301) Let the user select the sstable version to write
[ https://issues.apache.org/jira/browse/CASSANDRA-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696811#comment-17696811 ] maxwellguo commented on CASSANDRA-18301: yes，It will introduce some new params, so we used the extensions map in TableParams to avoid these problems. > Let the user select the sstable version to write > > > Key: CASSANDRA-18301 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18301 > Project: Cassandra > Issue Type: New Feature > Components: Local/Config, Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18301) Let the user select the sstable version to write
[ https://issues.apache.org/jira/browse/CASSANDRA-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696806#comment-17696806 ] maxwellguo commented on CASSANDRA-18301: yeah, in casandra.yaml is ok . Is it necessary to make this feature table level ? > Let the user select the sstable version to write > > > Key: CASSANDRA-18301 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18301 > Project: Cassandra > Issue Type: New Feature > Components: Local/Config, Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18301) Let the user select the sstable version to write
[ https://issues.apache.org/jira/browse/CASSANDRA-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696802#comment-17696802 ] maxwellguo commented on CASSANDRA-18301: Not opensource cassandra ~~~,just explain the similar solutions for common needs. > Let the user select the sstable version to write > > > Key: CASSANDRA-18301 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18301 > Project: Cassandra > Issue Type: New Feature > Components: Local/Config, Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18301) Let the user select the sstable version to write
[ https://issues.apache.org/jira/browse/CASSANDRA-18301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696797#comment-17696797 ] maxwellguo commented on CASSANDRA-18301: So how could user select the sstable version ? When they doing cql write with an attribute or some other cql grammar？ we just have an attribute for the file version both at schema level and cluster level. > Let the user select the sstable version to write > > > Key: CASSANDRA-18301 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18301 > Project: Cassandra > Issue Type: New Feature > Components: Local/Config, Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17698) sstabledump errors when dumping data from index
[
https://issues.apache.org/jira/browse/CASSANDRA-17698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696646#comment-17696646
]
maxwellguo commented on CASSANDRA-17698:
I saw CASSANDRA-17056 change the sstable version to "g", so It means this patch
do not neet to wait for C* 5.0 ? just can be check int after CASSANDRA-17056 ,
right ? [~blambov][~adelapena]
> sstabledump errors when dumping data from index
> ---
>
> Key: CASSANDRA-17698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17698
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Stefan Miklosovic
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 12h 40m
> Remaining Estimate: 0h
>
> {code:java}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE ks1.tb1 ( id text, name text, primary key (id));
> cqlsh> CREATE INDEX IF NOT EXISTS ON ks1.tb1(name);
> cqlsh> INSERT INTO ks1.tb1 (id, name ) VALUES ( '1', 'Joe');
> cqlsh> exit
> ./bin/nodetool flush
> ./tools/bin/sstabledump
> data/data/ks1/tb1-1c3c5f10ee4711ecab82eda2f44200b3/.tb1_name_idx/nb-1-big-Data.db
>
> [
> {
> "partition" : {
> "key" : [ "Joe" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "clustering" : [ ] } ] } ]Exception in thread "main"
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.db.marshal.PartitionerDefinedOrder.toJSONString(PartitionerDefinedOrder.java:87)
> at
> org.apache.cassandra.db.marshal.AbstractType.toJSONString(AbstractType.java:187)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeClustering(JsonTransformer.java:372)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:269)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:235)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:113)
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:214) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18296) tablehistograms add filter for data table and system table
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17696101#comment-17696101 ] maxwellguo commented on CASSANDRA-18296: tablehistograms's options are so old ,you can only chose ks table or ks.table nor all table . I want to change the option totally . 1. nodetool tablehistograms ks.tb1 ks.tb2 //list of tables with format keyspace.table 2.nodetool tablehistograms ks1 ks2 ks3 ...//list of keyspaces 3.nodetool tablehistograms -i ks1 ks2 //list of table histograms except for the keyspaces list behind the option -i 4.none option specified ,then all tables histograms will be output. this way , we can some what be compatible with the old options for nodetool tablehistograms ks.tb1, but the nodetool tablehistograms ks1 ks2 's result will be different from before, now is two keyspace,ks1 and ks2, but it was a table with keyspace name ks1, and table name ks2 before. [~smiklosovic] [~brandonwilliams] can you help me to take a look at this? > tablehistograms add filter for data table and system table > --- > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablehistograms to filter the output > of System tables and data tables. > As we know if we nodetool tablehistograms all tables will be output ,but > system table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. > It seems that tablestats got the -i option that can ignore keyspace, just add > this first, and then to see wether this can meet our needs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18296) tablehistograms add filter for data table and system table
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18296: --- Description: Add some options for nodetool tablehistograms to filter the output of System tables and data tables. As we know if we nodetool tablehistograms all tables will be output ,but system table 's number are too much. Add options to filter only output data tables or system tables, together with the already exist ks.tb lists, the output options are just fine. It seems that tablestats got the -i option that can ignore keyspace, just add this first, and then to see wether this can meet our needs was: Add some options for nodetool tablehistograms to filter the output of System tables and data tables. As we know if we nodetool tablehistograms all tables will be output ,but system table 's number are too much. Add options to filter only output data tables or system tables, together with the already exist ks.tb lists, the output options are just fine. It seems that tablestats got the -i option that can ignore keyspace, just add this > tablehistograms add filter for data table and system table > --- > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablehistograms to filter the output > of System tables and data tables. > As we know if we nodetool tablehistograms all tables will be output ,but > system table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. > It seems that tablestats got the -i option that can ignore keyspace, just add > this first, and then to see wether this can meet our needs -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18296) tablehistograms add filter for data table and system table
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18296: --- Description: Add some options for nodetool tablehistograms to filter the output of System tables and data tables. As we know if we nodetool tablehistograms all tables will be output ,but system table 's number are too much. Add options to filter only output data tables or system tables, together with the already exist ks.tb lists, the output options are just fine. It seems that tablestats got the -i option that can ignore keyspace, just add this was: Add some options for nodetool tablestats and tablehistograms to filter the output of System tables and data tables. As we know if we nodetool tablestats all tables will be output ,but system table 's number are too much. Add options to filter only output data tables or system tables, together with the already exist ks.tb lists, the output options are just fine. > tablehistograms add filter for data table and system table > --- > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablehistograms to filter the output > of System tables and data tables. > As we know if we nodetool tablehistograms all tables will be output ,but > system table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. > It seems that tablestats got the -i option that can ignore keyspace, just add > this -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18296) tablehistograms add filter for data table and system table
[ https://issues.apache.org/jira/browse/CASSANDRA-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18296: --- Summary: tablehistograms add filter for data table and system table (was: tablestats and tablehistograms add filter for data table and system table) > tablehistograms add filter for data table and system table > --- > > Key: CASSANDRA-18296 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: maxwellguo >Assignee: maxwellguo >Priority: Normal > > Add some options for nodetool tablestats and tablehistograms to filter the > output > of System tables and data tables. > As we know if we nodetool tablestats all tables will be output ,but system > table 's number are too much. > Add options to filter only output data tables or system tables, together with > the already exist ks.tb lists, the output options are just fine. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18296) tablestats and tablehistograms add filter for data table and system table
maxwellguo created CASSANDRA-18296: -- Summary: tablestats and tablehistograms add filter for data table and system table Key: CASSANDRA-18296 URL: https://issues.apache.org/jira/browse/CASSANDRA-18296 Project: Cassandra Issue Type: Improvement Components: Tool/nodetool Reporter: maxwellguo Assignee: maxwellguo Add some options for nodetool tablestats and tablehistograms to filter the output of System tables and data tables. As we know if we nodetool tablestats all tables will be output ,but system table 's number are too much. Add options to filter only output data tables or system tables, together with the already exist ks.tb lists, the output options are just fine. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18294: --- Severity: Low Status: Open (was: Triage Needed) > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maxwellguo updated CASSANDRA-18294: --- Fix Version/s: 3.0.x 4.0.x 4.1.x > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > Fix For: 3.0.x, 4.0.x, 4.1.x > > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695456#comment-17695456 ] maxwellguo edited comment on CASSANDRA-18294 at 3/2/23 5:27 AM: I just run your ut, and it seems the bug exist. I think you should add some more jvm-dtest as CASSANDRA-15191 's JVMStabilityInspectorCorruptSSTableExceptionTest . For your ut can just cover gossip is stopped, also I left some comments on this pr. [~smiklosovic] [~brandon.williams] can you take a look at this ? was (Author: maxwellguo): I just run your ut, and it seems the bug exist. I think you should add some more jvm-dtest as CASSANDRA-15191 's JVMStabilityInspectorCorruptSSTableExceptionTest . [~smiklosovic] [~brandon.williams] can you take a look at this ? > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695456#comment-17695456 ] maxwellguo commented on CASSANDRA-18294: I just run your ut, and it seems the bug exist. I think you should add some more jvm-dtest as CASSANDRA-15191 's JVMStabilityInspectorCorruptSSTableExceptionTest . [~smiklosovic] [~brandon.williams] can you take a look at this ? > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695437#comment-17695437 ] maxwellguo edited comment on CASSANDRA-18294 at 3/2/23 3:50 AM: What is the version of your cassandra ? was (Author: maxwellguo): What is the version of your cassandr ? > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695437#comment-17695437 ] maxwellguo commented on CASSANDRA-18294: What is the version of your cassandr ? > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18294) die disk failure policy will not kill jvm as documented
[ https://issues.apache.org/jira/browse/CASSANDRA-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695427#comment-17695427 ] maxwellguo commented on CASSANDRA-18294: can you show more details on the error ? like the stack message or some thing else. > die disk failure policy will not kill jvm as documented > --- > > Key: CASSANDRA-18294 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18294 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Runtian Liu >Assignee: Runtian Liu >Priority: Normal > > After Cassandra has successfully starts up with disk_failure_policy die, when > encounter a file system error, Cassandra server will only throw exception > instead of shut down gossip and client transport and kill JVM. Document: > [https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy] > > The reason for this is the default FS error handler is not handing policy die > correctly. Instead of shutting down gossip and native transport, it throws an > error. > > The JVMStabilityInspectorTest is passing because the error handler is not set > so no exception is thrown. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18290) Test Failure: SecondaryIndexTest.testUpdatesToMemtableData
[
https://issues.apache.org/jira/browse/CASSANDRA-18290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17694201#comment-17694201
]
maxwellguo commented on CASSANDRA-18290:
I think I can take this ticket as I have some work on secondary index recently
> Test Failure: SecondaryIndexTest.testUpdatesToMemtableData
> --
>
> Key: CASSANDRA-18290
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18290
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index, Test/unit
>Reporter: Michael Semb Wever
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 4.1.x
>
>
> from
> -
> https://ci-cassandra.apache.org/job/Cassandra-4.1/279/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testUpdatesToMemtableData_cdc/
> -
> https://ci-cassandra.apache.org/job/Cassandra-4.1-test-cdc/239/jdk=jdk_11_latest,label=cassandra,split=7/testReport/junit/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testUpdatesToMemtableData_cdc/
> Stacktrace
> {noformat}
> junit.framework.AssertionFailedError: expected:<1> but was:<0>
> at
> org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest.testUpdatesToMemtableData(SecondaryIndexTest.java:1010)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
> Standard Output
> {noformat}
> INFO [main] 2023-02-22 19:09:51,801 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml
> DEBUG [main] 2023-02-22 19:09:51,806 YamlConfigurationLoader.java:124 -
> Loading settings from
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml
> INFO [main] 2023-02-22 19:09:52,299 Config.java:1171 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null;
> allow_extra_insecure_udfs=false; allow_filtering_enabled=true;
> allow_insecure_udfs=false;
> audit_logging_options=AuditLogOptions{enabled=false,
> logger='BinAuditLogger{}', included_keyspaces='',
> excluded_keyspaces='system,system_schema,system_virtual_schema',
> included_categories='', excluded_categories='', included_users='',
> excluded_users='', audit_logs_dir='audit', archive_command='',
> roll_cycle='HOURLY', block=true, max_queue_weight=268435456,
> max_log_size=17179869184, max_archive_retries=10};
> auth_cache_warming_enabled=false; auth_read_consistency_level=LOCAL_QUORUM;
> auth_write_consistency_level=EACH_QUORUM; authenticator=null;
> authorizer=null; auto_bootstrap=true; auto_hints_cleanup_enabled=true;
> auto_optimise_full_repair_streams=false;
> auto_optimise_inc_repair_streams=false;
> auto_optimise_preview_repair_streams=false; auto_snapshot=true;
> auto_snapshot_ttl=null; autocompaction_on_startup_enabled=true;
> automatic_sstable_upgrade=false; available_processors=-1;
> back_pressure_enabled=false; back_pressure_strategy=null;
> batch_size_fail_threshold=50KiB; batch_size_warn_threshold=5KiB;
> batchlog_replay_throttle=1024KiB; block_for_peers_in_remote_dcs=false;
> block_for_peers_timeout_in_secs=10; broadcast_address=null;
> broadcast_rpc_address=null; buffer_pool_use_heap_if_exhausted=false;
> cache_load_timeout=30s; cas_contention_timeout=1800ms; cdc_block_writes=true;
> cdc_enabled=true; cdc_free_space_check_interval=250ms;
> cdc_raw_directory=build/test/cassandra/cdc_raw; cdc_total_space=0MiB;
> check_for_duplicate_rows_during_compaction=true;
> check_for_duplicate_rows_during_reads=true;
> client_encryption_options=;
> client_error_reporting_exclusions=SubnetGroups{subnets=[]}; cluster_name=Test
> Cluster; collection_size_fail_threshold=null;
> collection_size_warn_threshold=null; column_index_cache_size=2KiB;
> column_index_size=4KiB; columns_per_table_fail_threshold=-1;
> columns_per_table_warn_threshold=-1; commit_failure_policy=stop;
> commitlog_compression=null;
> commitlog_directory=build/test/cassandra/commitlog;
> commitlog_max_compression_buffers_in_pool=3;
> commitlog_periodic_queue_size=-1; commitlog_segment_size=5MiB;
> commitlog_sync=batch; commitlog_sync_batch_window_in_ms=1.0;
> commitlog_sync_group_window=0ms; commitlog_sync_period=0ms;
> commitlog_total_space=null; compact_tables_enabled=true;
> compaction_large_partition_warning_threshold=100MiB;
> compaction_throughput=0MiB/s; compaction_tombstone_warning_threshold=10;
> concurrent_compactors=4; concurrent_counter_writes=32;
> concurrent_materialized_view_builders=1;
>
[jira] [Assigned] (CASSANDRA-18290) Test Failure: SecondaryIndexTest.testUpdatesToMemtableData
[
https://issues.apache.org/jira/browse/CASSANDRA-18290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maxwellguo reassigned CASSANDRA-18290:
--
Assignee: maxwellguo
> Test Failure: SecondaryIndexTest.testUpdatesToMemtableData
> --
>
> Key: CASSANDRA-18290
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18290
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index, Test/unit
>Reporter: Michael Semb Wever
>Assignee: maxwellguo
>Priority: Normal
> Fix For: 4.1.x
>
>
> from
> -
> https://ci-cassandra.apache.org/job/Cassandra-4.1/279/testReport/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testUpdatesToMemtableData_cdc/
> -
> https://ci-cassandra.apache.org/job/Cassandra-4.1-test-cdc/239/jdk=jdk_11_latest,label=cassandra,split=7/testReport/junit/org.apache.cassandra.cql3.validation.entities/SecondaryIndexTest/testUpdatesToMemtableData_cdc/
> Stacktrace
> {noformat}
> junit.framework.AssertionFailedError: expected:<1> but was:<0>
> at
> org.apache.cassandra.cql3.validation.entities.SecondaryIndexTest.testUpdatesToMemtableData(SecondaryIndexTest.java:1010)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
> Standard Output
> {noformat}
> INFO [main] 2023-02-22 19:09:51,801 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml
> DEBUG [main] 2023-02-22 19:09:51,806 YamlConfigurationLoader.java:124 -
> Loading settings from
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml
> INFO [main] 2023-02-22 19:09:52,299 Config.java:1171 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null;
> allow_extra_insecure_udfs=false; allow_filtering_enabled=true;
> allow_insecure_udfs=false;
> audit_logging_options=AuditLogOptions{enabled=false,
> logger='BinAuditLogger{}', included_keyspaces='',
> excluded_keyspaces='system,system_schema,system_virtual_schema',
> included_categories='', excluded_categories='', included_users='',
> excluded_users='', audit_logs_dir='audit', archive_command='',
> roll_cycle='HOURLY', block=true, max_queue_weight=268435456,
> max_log_size=17179869184, max_archive_retries=10};
> auth_cache_warming_enabled=false; auth_read_consistency_level=LOCAL_QUORUM;
> auth_write_consistency_level=EACH_QUORUM; authenticator=null;
> authorizer=null; auto_bootstrap=true; auto_hints_cleanup_enabled=true;
> auto_optimise_full_repair_streams=false;
> auto_optimise_inc_repair_streams=false;
> auto_optimise_preview_repair_streams=false; auto_snapshot=true;
> auto_snapshot_ttl=null; autocompaction_on_startup_enabled=true;
> automatic_sstable_upgrade=false; available_processors=-1;
> back_pressure_enabled=false; back_pressure_strategy=null;
> batch_size_fail_threshold=50KiB; batch_size_warn_threshold=5KiB;
> batchlog_replay_throttle=1024KiB; block_for_peers_in_remote_dcs=false;
> block_for_peers_timeout_in_secs=10; broadcast_address=null;
> broadcast_rpc_address=null; buffer_pool_use_heap_if_exhausted=false;
> cache_load_timeout=30s; cas_contention_timeout=1800ms; cdc_block_writes=true;
> cdc_enabled=true; cdc_free_space_check_interval=250ms;
> cdc_raw_directory=build/test/cassandra/cdc_raw; cdc_total_space=0MiB;
> check_for_duplicate_rows_during_compaction=true;
> check_for_duplicate_rows_during_reads=true;
> client_encryption_options=;
> client_error_reporting_exclusions=SubnetGroups{subnets=[]}; cluster_name=Test
> Cluster; collection_size_fail_threshold=null;
> collection_size_warn_threshold=null; column_index_cache_size=2KiB;
> column_index_size=4KiB; columns_per_table_fail_threshold=-1;
> columns_per_table_warn_threshold=-1; commit_failure_policy=stop;
> commitlog_compression=null;
> commitlog_directory=build/test/cassandra/commitlog;
> commitlog_max_compression_buffers_in_pool=3;
> commitlog_periodic_queue_size=-1; commitlog_segment_size=5MiB;
> commitlog_sync=batch; commitlog_sync_batch_window_in_ms=1.0;
> commitlog_sync_group_window=0ms; commitlog_sync_period=0ms;
> commitlog_total_space=null; compact_tables_enabled=true;
> compaction_large_partition_warning_threshold=100MiB;
> compaction_throughput=0MiB/s; compaction_tombstone_warning_threshold=10;
> concurrent_compactors=4; concurrent_counter_writes=32;
> concurrent_materialized_view_builders=1;
> concurrent_materialized_view_writes=32; concurrent_reads=32;
> concurrent_replicates=null;
[jira] [Comment Edited] (CASSANDRA-18177) Support AbstractCompositeType with toJSONString method
[ https://issues.apache.org/jira/browse/CASSANDRA-18177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693762#comment-17693762 ] maxwellguo edited comment on CASSANDRA-18177 at 2/27/23 2:45 AM: - hi [~yongjiang], I left some comments . Besides,I think you should start your code based on trunk , for this feature (or CASSANDRA-18177) have little relation with CASSANDRA-17698 ,so I think it may be better to start your code based on trunk. For toJSONString method of AbstractCompositeType does not work only for CASSANDRA-17698, but all cases that need toJSONString method of AbstractCompositeType(or DynamicCompositeType/CompositeType) was (Author: maxwellguo): Some comments was left. Besides,I think you should start your code based on trunk , for this feature (or CASSANDRA-18177) have little relation with CASSANDRA-17698 ,so I think it may be better to start your code based on trunk. For toJSONString method of AbstractCompositeType does not work only for CASSANDRA-17698, but all cases that need toJSONString method of AbstractCompositeType(or DynamicCompositeType/CompositeType) > Support AbstractCompositeType with toJSONString method > -- > > Key: CASSANDRA-18177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18177 > Project: Cassandra > Issue Type: New Feature > Components: Local/SSTable >Reporter: maxwellguo >Assignee: Yong Jiang >Priority: Normal > Fix For: 4.x > > > As we know AbstractCompositeType do not support toJSONString method , but > some times > we do need this. > See the discusstion of > [CASSANDRA-17698|https://issues.apache.org/jira/browse/CASSANDRA-17698]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18177) Support AbstractCompositeType with toJSONString method
[ https://issues.apache.org/jira/browse/CASSANDRA-18177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693762#comment-17693762 ] maxwellguo commented on CASSANDRA-18177: Some comments was left. Besides,I think you should start your code based on trunk , for this feature (or CASSANDRA-18177) have little relation with CASSANDRA-17698 ,so I think it may be better to start your code based on trunk. For toJSONString method of AbstractCompositeType does not work only for CASSANDRA-17698, but all cases that need toJSONString method of AbstractCompositeType(or DynamicCompositeType/CompositeType) > Support AbstractCompositeType with toJSONString method > -- > > Key: CASSANDRA-18177 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18177 > Project: Cassandra > Issue Type: New Feature > Components: Local/SSTable >Reporter: maxwellguo >Assignee: Yong Jiang >Priority: Normal > Fix For: 4.x > > > As we know AbstractCompositeType do not support toJSONString method , but > some times > we do need this. > See the discusstion of > [CASSANDRA-17698|https://issues.apache.org/jira/browse/CASSANDRA-17698]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
