[jira] [Commented] (CASSANDRA-14291) Nodetool command to recreate SSTable components
[
https://issues.apache.org/jira/browse/CASSANDRA-14291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539519#comment-16539519
]
Alexander Ivakov commented on CASSANDRA-14291:
--
Rebuilding some of the components (eg. primary index) requires going through
all the data in the SSTable, so this is basically similar to a compaction. It
is proposed, for the sake of simplicity, to implement this command to call
upgradesstables in the background if recreating any of these components:
primary index, compression info, secondary index, stats.
Note: this will recreate all components and also re-write the data file out to
disk.
Recreating the bloom filter and the index summary (using the saved primary
index) can be done without going through the whole data, so these can be done
separately and without re-writing data and all other components.
> Nodetool command to recreate SSTable components
> ---
>
> Key: CASSANDRA-14291
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14291
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Kurt Greaves
>Assignee: Alexander Ivakov
>Priority: Minor
>
> Need a JMX/Nodetool command to recreate components for SSTables without
> re-writing the data files.
> Possible implementation idea:
> Create a {{nodetool (recreate|regen)component}} command that would enable you
> to recreate specific components of an SSTable, and also allow specifying
> SSTables or columnfamilies.
> I'd say a flag for a list of components and a flag for SSTables with
> keyspace.columnfamilies as positional arguments would work
> Alternatively this could become part of upgradesstables, but would likely
> make that command a bit bloated.
> Background:
> In CASSANDRA-11163 we changed it so summaries and bloomfilters were not
> regenerated or persisted on startup. This means we would rely on
> compactions/upgrades to regenerate the bloomfilter (or other components)
> after a configuration change. While this works, it's pretty inefficient on
> large tables just because you changed the bloomfilter size or summary chunk
> sizes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14432) Docs container leaves build artifacts behind
[ https://issues.apache.org/jira/browse/CASSANDRA-14432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539448#comment-16539448 ] ASF GitHub Bot commented on CASSANDRA-14432: Github user jameslamb commented on the issue: https://github.com/apache/cassandra/pull/222 I do see this commit: https://github.com/apache/cassandra/commit/f2ee3db6537005f9ae8582cecd8e29df29ac30d5 can this PR be closed? > Docs container leaves build artifacts behind > > > Key: CASSANDRA-14432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14432 > Project: Cassandra > Issue Type: Improvement >Reporter: James Lamb >Assignee: James Lamb >Priority: Trivial > Fix For: 4.0 > > > Hello! > I was looking at [the > repo|https://github.com/apache/cassandra/blob/db81f6bffef1a8215fec28bb0522dc9684870627/doc/Dockerfile] > tonight and tried to build the documentation locally. While doing this, I > noticed that the container here does not clean up intermediate build > artifacts. > Will you please consider a PR to address that? When I ran locally, removing > build artifacts reduced the size of the container by 14 MB. I will post a > link to the PR shortly. > > Thank you very much, > > -James > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14432) Docs container leaves build artifacts behind
[ https://issues.apache.org/jira/browse/CASSANDRA-14432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539445#comment-16539445 ] ASF GitHub Bot commented on CASSANDRA-14432: Github user jameslamb commented on the issue: https://github.com/apache/cassandra/pull/222 I'm confused. This PR is still open but it looks like this got closed in: https://issues.apache.org/jira/browse/CASSANDRA-14432 What am I missing? > Docs container leaves build artifacts behind > > > Key: CASSANDRA-14432 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14432 > Project: Cassandra > Issue Type: Improvement >Reporter: James Lamb >Assignee: James Lamb >Priority: Trivial > Fix For: 4.0 > > > Hello! > I was looking at [the > repo|https://github.com/apache/cassandra/blob/db81f6bffef1a8215fec28bb0522dc9684870627/doc/Dockerfile] > tonight and tried to build the documentation locally. While doing this, I > noticed that the container here does not clean up intermediate build > artifacts. > Will you please consider a PR to address that? When I ran locally, removing > build artifacts reduced the size of the container by 14 MB. I will post a > link to the PR shortly. > > Thank you very much, > > -James > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14423) SSTables stop being compacted
[
https://issues.apache.org/jira/browse/CASSANDRA-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539403#comment-16539403
]
mck commented on CASSANDRA-14423:
-
Committed as 3482370df5672c9337a16a8a52baba53b70a4fe8
> SSTables stop being compacted
> -
>
> Key: CASSANDRA-14423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14423
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Blocker
> Fix For: 2.2.13, 3.0.17, 3.11.3
>
>
> So seeing a problem in 3.11.0 where SSTables are being lost from the view and
> not being included in compactions/as candidates for compaction. It seems to
> get progressively worse until there's only 1-2 SSTables in the view which
> happen to be the most recent SSTables and thus compactions completely stop
> for that table.
> The SSTables seem to still be included in reads, just not compactions.
> The issue can be fixed by restarting C*, as it will reload all SSTables into
> the view, but this is only a temporary fix. User defined/major compactions
> still work - not clear if they include the result back in the view but is not
> a good work around.
> This also results in a discrepancy between SSTable count and SSTables in
> levels for any table using LCS.
> {code:java}
> Keyspace : xxx
> Read Count: 57761088
> Read Latency: 0.10527088681224288 ms.
> Write Count: 2513164
> Write Latency: 0.018211106398149903 ms.
> Pending Flushes: 0
> Table: xxx
> SSTable count: 10
> SSTables in each level: [2, 0, 0, 0, 0, 0, 0, 0, 0]
> Space used (live): 894498746
> Space used (total): 894498746
> Space used by snapshots (total): 0
> Off heap memory used (total): 11576197
> SSTable Compression Ratio: 0.6956629530569777
> Number of keys (estimate): 3562207
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 87
> Local read count: 57761088
> Local read latency: 0.108 ms
> Local write count: 2513164
> Local write latency: NaN ms
> Pending flushes: 0
> Percent repaired: 86.33
> Bloom filter false positives: 43
> Bloom filter false ratio: 0.0
> Bloom filter space used: 8046104
> Bloom filter off heap memory used: 8046024
> Index summary off heap memory used: 3449005
> Compression metadata off heap memory used: 81168
> Compacted partition minimum bytes: 104
> Compacted partition maximum bytes: 5722
> Compacted partition mean bytes: 175
> Average live cells per slice (last five minutes): 1.0
> Maximum live cells per slice (last five minutes): 1
> Average tombstones per slice (last five minutes): 1.0
> Maximum tombstones per slice (last five minutes): 1
> Dropped Mutations: 0
> {code}
> Also for STCS we've confirmed that SSTable count will be different to the
> number of SSTables reported in the Compaction Bucket's. In the below example
> there's only 3 SSTables in a single bucket - no more are listed for this
> table. Compaction thresholds haven't been modified for this table and it's a
> very basic KV schema.
> {code:java}
> Keyspace : yyy
> Read Count: 30485
> Read Latency: 0.06708991307200263 ms.
> Write Count: 57044
> Write Latency: 0.02204061776873992 ms.
> Pending Flushes: 0
> Table: yyy
> SSTable count: 19
> Space used (live): 18195482
> Space used (total): 18195482
> Space used by snapshots (total): 0
> Off heap memory used (total): 747376
> SSTable Compression Ratio: 0.7607394576769735
> Number of keys (estimate): 116074
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 39
> Local read count: 30485
> Local read latency: NaN ms
> Local write count: 57044
> Local write latency: NaN ms
> Pending flushes: 0
> Percent repaired: 79.76
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 690912
> Bloom filter off heap memory used: 690760
> Index summary off heap memory used: 54736
> Compression metadata off heap memory used: 1880
> Compacted partition minimum bytes: 73
> Compacted partition maximum bytes: 124
> Compacted partition mean bytes: 96
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> {code:java}
> Apr 27 03:10:39 cassandra[9263]: TRACE o.a.c.d.c.SizeTieredCompactionStrategy
> Compaction buckets are
>
[01/10] cassandra git commit: ninja fix to f8912ce – Stop SSTables being lost from compaction strategy after full repairs
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 9ff78249a -> 3482370df
refs/heads/cassandra-3.0 8ead7af14 -> 6ce887e58
refs/heads/cassandra-3.11 847b84323 -> 6bcc60ae4
refs/heads/trunk 28217f857 -> f081558d6
ninja fix to f8912ce â Stop SSTables being lost from compaction strategy
after full repairs
patch by Mick Semb Wever; reviewed by Michael Shuler for CASSANDRA-14423
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3482370d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3482370d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3482370d
Branch: refs/heads/cassandra-2.2
Commit: 3482370df5672c9337a16a8a52baba53b70a4fe8
Parents: 9ff7824
Author: Mick Semb Wever
Authored: Tue Jul 10 10:25:50 2018 +1000
Committer: Mick Semb Wever
Committed: Wed Jul 11 10:16:06 2018 +1000
--
.../org/apache/cassandra/db/compaction/CompactionManager.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3482370d/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 013fc04..c37049f 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -27,6 +27,7 @@ import javax.management.ObjectName;
import javax.management.openmbean.OpenDataException;
import javax.management.openmbean.TabularData;
+import org.apache.commons.lang3.StringUtils;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Predicate;
import com.google.common.base.Predicates;
@@ -581,7 +582,7 @@ public class CompactionManager implements
CompactionManagerMBean
}
if (!anticompactRanges.isEmpty())
-logger.info("SSTable {} ({}) will be anticompacted on
ranges: {}", sstable, sstableBounds, String.join(", ", anticompactRanges));
+logger.info("SSTable {} ({}) will be anticompacted on
ranges: {}", sstable, sstableBounds, StringUtils.join(anticompactRanges, ", "));
if (!shouldAnticompact)
{
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6ce887e5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6ce887e5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6ce887e5 Branch: refs/heads/cassandra-3.11 Commit: 6ce887e58ac193eb53ecd75d79ead5a93ebda005 Parents: 8ead7af 3482370 Author: Mick Semb Wever Authored: Wed Jul 11 10:16:23 2018 +1000 Committer: Mick Semb Wever Committed: Wed Jul 11 10:16:23 2018 +1000 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[02/10] cassandra git commit: ninja fix to f8912ce – Stop SSTables being lost from compaction strategy after full repairs
ninja fix to f8912ce â Stop SSTables being lost from compaction strategy
after full repairs
patch by Mick Semb Wever; reviewed by Michael Shuler for CASSANDRA-14423
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3482370d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3482370d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3482370d
Branch: refs/heads/cassandra-3.0
Commit: 3482370df5672c9337a16a8a52baba53b70a4fe8
Parents: 9ff7824
Author: Mick Semb Wever
Authored: Tue Jul 10 10:25:50 2018 +1000
Committer: Mick Semb Wever
Committed: Wed Jul 11 10:16:06 2018 +1000
--
.../org/apache/cassandra/db/compaction/CompactionManager.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3482370d/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 013fc04..c37049f 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -27,6 +27,7 @@ import javax.management.ObjectName;
import javax.management.openmbean.OpenDataException;
import javax.management.openmbean.TabularData;
+import org.apache.commons.lang3.StringUtils;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Predicate;
import com.google.common.base.Predicates;
@@ -581,7 +582,7 @@ public class CompactionManager implements
CompactionManagerMBean
}
if (!anticompactRanges.isEmpty())
-logger.info("SSTable {} ({}) will be anticompacted on
ranges: {}", sstable, sstableBounds, String.join(", ", anticompactRanges));
+logger.info("SSTable {} ({}) will be anticompacted on
ranges: {}", sstable, sstableBounds, StringUtils.join(anticompactRanges, ", "));
if (!shouldAnticompact)
{
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6bcc60ae Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6bcc60ae Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6bcc60ae Branch: refs/heads/trunk Commit: 6bcc60ae4c440be1ba9f6c2b83f697ac2f2b3f3e Parents: 847b843 6ce887e Author: Mick Semb Wever Authored: Wed Jul 11 10:16:52 2018 +1000 Committer: Mick Semb Wever Committed: Wed Jul 11 10:16:52 2018 +1000 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[04/10] cassandra git commit: ninja fix to f8912ce – Stop SSTables being lost from compaction strategy after full repairs
ninja fix to f8912ce â Stop SSTables being lost from compaction strategy
after full repairs
patch by Mick Semb Wever; reviewed by Michael Shuler for CASSANDRA-14423
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3482370d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3482370d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3482370d
Branch: refs/heads/trunk
Commit: 3482370df5672c9337a16a8a52baba53b70a4fe8
Parents: 9ff7824
Author: Mick Semb Wever
Authored: Tue Jul 10 10:25:50 2018 +1000
Committer: Mick Semb Wever
Committed: Wed Jul 11 10:16:06 2018 +1000
--
.../org/apache/cassandra/db/compaction/CompactionManager.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3482370d/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 013fc04..c37049f 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -27,6 +27,7 @@ import javax.management.ObjectName;
import javax.management.openmbean.OpenDataException;
import javax.management.openmbean.TabularData;
+import org.apache.commons.lang3.StringUtils;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Predicate;
import com.google.common.base.Predicates;
@@ -581,7 +582,7 @@ public class CompactionManager implements
CompactionManagerMBean
}
if (!anticompactRanges.isEmpty())
-logger.info("SSTable {} ({}) will be anticompacted on
ranges: {}", sstable, sstableBounds, String.join(", ", anticompactRanges));
+logger.info("SSTable {} ({}) will be anticompacted on
ranges: {}", sstable, sstableBounds, StringUtils.join(anticompactRanges, ", "));
if (!shouldAnticompact)
{
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6ce887e5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6ce887e5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6ce887e5 Branch: refs/heads/trunk Commit: 6ce887e58ac193eb53ecd75d79ead5a93ebda005 Parents: 8ead7af 3482370 Author: Mick Semb Wever Authored: Wed Jul 11 10:16:23 2018 +1000 Committer: Mick Semb Wever Committed: Wed Jul 11 10:16:23 2018 +1000 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[03/10] cassandra git commit: ninja fix to f8912ce – Stop SSTables being lost from compaction strategy after full repairs
ninja fix to f8912ce â Stop SSTables being lost from compaction strategy
after full repairs
patch by Mick Semb Wever; reviewed by Michael Shuler for CASSANDRA-14423
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3482370d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3482370d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3482370d
Branch: refs/heads/cassandra-3.11
Commit: 3482370df5672c9337a16a8a52baba53b70a4fe8
Parents: 9ff7824
Author: Mick Semb Wever
Authored: Tue Jul 10 10:25:50 2018 +1000
Committer: Mick Semb Wever
Committed: Wed Jul 11 10:16:06 2018 +1000
--
.../org/apache/cassandra/db/compaction/CompactionManager.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3482370d/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 013fc04..c37049f 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -27,6 +27,7 @@ import javax.management.ObjectName;
import javax.management.openmbean.OpenDataException;
import javax.management.openmbean.TabularData;
+import org.apache.commons.lang3.StringUtils;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Predicate;
import com.google.common.base.Predicates;
@@ -581,7 +582,7 @@ public class CompactionManager implements
CompactionManagerMBean
}
if (!anticompactRanges.isEmpty())
-logger.info("SSTable {} ({}) will be anticompacted on
ranges: {}", sstable, sstableBounds, String.join(", ", anticompactRanges));
+logger.info("SSTable {} ({}) will be anticompacted on
ranges: {}", sstable, sstableBounds, StringUtils.join(anticompactRanges, ", "));
if (!shouldAnticompact)
{
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f081558d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f081558d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f081558d Branch: refs/heads/trunk Commit: f081558d68e0af4968f05687cc9587aa9bdf1bd6 Parents: 28217f8 6bcc60a Author: Mick Semb Wever Authored: Wed Jul 11 10:17:04 2018 +1000 Committer: Mick Semb Wever Committed: Wed Jul 11 10:17:04 2018 +1000 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6bcc60ae Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6bcc60ae Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6bcc60ae Branch: refs/heads/cassandra-3.11 Commit: 6bcc60ae4c440be1ba9f6c2b83f697ac2f2b3f3e Parents: 847b843 6ce887e Author: Mick Semb Wever Authored: Wed Jul 11 10:16:52 2018 +1000 Committer: Mick Semb Wever Committed: Wed Jul 11 10:16:52 2018 +1000 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6ce887e5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6ce887e5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6ce887e5 Branch: refs/heads/cassandra-3.0 Commit: 6ce887e58ac193eb53ecd75d79ead5a93ebda005 Parents: 8ead7af 3482370 Author: Mick Semb Wever Authored: Wed Jul 11 10:16:23 2018 +1000 Committer: Mick Semb Wever Committed: Wed Jul 11 10:16:23 2018 +1000 -- -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14522) sstableloader options assume the rpc/native interface is the same as the internode interface
[
https://issues.apache.org/jira/browse/CASSANDRA-14522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539392#comment-16539392
]
Jeremy commented on CASSANDRA-14522:
[~aweisberg] Thank you very much for the explanation! I will try out the
testing setup and see how it goes.
> sstableloader options assume the rpc/native interface is the same as the
> internode interface
>
>
> Key: CASSANDRA-14522
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14522
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Jeremy Hanna
>Assignee: Jeremy
>Priority: Major
> Labels: lhf
>
> Currently, in the LoaderOptions for the BulkLoader, the user can give a list
> of initial host addresses. That's to do the initial connection to the
> cluster but also to stream the sstables. If you have two physical
> interfaces, one for rpc, the other for internode traffic, then bulk loader
> won't currently work. It will throw an error such as:
> {quote}
> > sstableloader -v -u cassadmin -pw xxx -d
> > 10.133.210.101,10.133.210.102,10.133.210.103,10.133.210.104
> > /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-1-big-Data.db
> /var/lib/cassandra/commitlog/backup_tmp/test_bkup/bkup_tbl/mc-2-big-Data.db
> to [/10.133.210.101, /10.133.210.103, /10.133.210.102, /10.133.210.104]
> progress: total: 100% 0 MB/s(avg: 0 MB/s)ERROR 10:16:05,311 [Stream
> #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:212)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> ~[netty-all-4.0.54.Final.jar:4.0.54.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> ERROR 10:16:05,312 [Stream #9ed00130-6ff6-11e8-965c-93a78bf96e60] Streaming
> error occurred
> java.net.ConnectException: Connection refused
> at sun.nio.ch.Net.connect0(Native Method) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:454) ~[na:1.8.0_101]
> at sun.nio.ch.Net.connect(Net.java:446) ~[na:1.8.0_101]
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:648)
> ~[na:1.8.0_101]
> at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
> ~[na:1.8.0_101]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:60)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:266)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:253)
> ~[cassandra-all-3.0.15.2128.jar:3.0.15.2128]
> at
>
[jira] [Commented] (CASSANDRA-13903) Add JDK9 vectorizedMismatch array compare support
[ https://issues.apache.org/jira/browse/CASSANDRA-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539224#comment-16539224 ] Ariel Weisberg commented on CASSANDRA-13903: I would be interested to see it performs for small and large sizes of arrays. Is there a startup time penalty for using the intrinsic is what I am wondering? > Add JDK9 vectorizedMismatch array compare support > - > > Key: CASSANDRA-13903 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13903 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Yingqi Lu >Priority: Major > Labels: Performance > > Dear All, > JDK9 has recently been released (http://openjdk.java.net/projects/jdk9/). It > introduces a set of new vectorizedMismatch APIs for array comparison. On > supported platforms, the new JDK9 implementation is intrisified to leverage > SIMD instructions. For a byte array comparison, up to 64 bytes (512 bits) can > be compared as a single unit. Feature details please refer to > http://download.java.net/java/jdk9/docs/api/java/util/Arrays.html and > https://bugs.openjdk.java.net/browse/JDK-8033148 > Currently in Cassandra, keys are implemented as ByteBuffers and compared as > byte arrays most of the time. Key comparison, for example decorated key > compare, it is either done with unsafe operation taking 8 bytes at a time, or > pure java operation comparing byte by byte. This can be optimized with new > JDK9 java.util.Arrays.compare APIs on modern CPUs. > Please let us know your feedback. At the meantime, we will submit a patch > with performance studies in couple of weeks. > Thanks, > Yingqi Lu -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539153#comment-16539153
]
Andrés de la Peña commented on CASSANDRA-8272:
--
I have just rebased [the
patch|https://github.com/adelapena/cassandra/commit/07e228824fb0e4ea04bf0b1ed11b347ce654f02a]
with some fixes for indexes on static columns. I have also added a few extra
checks to [the
dtests|https://github.com/apache/cassandra-dtest/compare/master...adelapena:CASSANDRA-8272].
> 2ndary indexes can return stale data
>
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Andrés de la Peña
>Priority: Major
> Fix For: 3.0.x
>
>
> When replica return 2ndary index results, it's possible for a single replica
> to return a stale result and that result will be sent back to the user,
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned (since
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and
> provided we make the index inherit the gcGrace of it's parent CF), instead of
> skipping that tombstone, we'd insert in the result a corresponding range
> tombstone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13957) upgradesstables fails after upgrading from 2.1.x to 3.0.14
[
https://issues.apache.org/jira/browse/CASSANDRA-13957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539100#comment-16539100
]
Joshua McKenzie commented on CASSANDRA-13957:
-
[~dan.priscornic]: did you ever get this resolved?
> upgradesstables fails after upgrading from 2.1.x to 3.0.14
> --
>
> Key: CASSANDRA-13957
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13957
> Project: Cassandra
> Issue Type: Bug
>Reporter: Dan Priscornic
>Priority: Major
>
> After upgrading DSE from 4.8.14 (cassandra 2.1.18.1463) to 5.0.10 (cassandra
> 3.0.14.1862) I ran nodetool upgradesstables and it fails with the following
> stack trace:
> {code:java}
> # nodetool -u cassandra -pwf /etc/dse/cassandra/jmxremote.password
> upgradesstables
> error: null
> -- StackTrace --
> java.lang.AssertionError
> at org.apache.cassandra.db.rows.Rows.collectStats(Rows.java:70)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter$StatsCollector.applyToRow(BigTableWriter.java:197)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:137)
> at
> org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:111)
> at
> org.apache.cassandra.db.ColumnIndex.writeAndBuildIndex(ColumnIndex.java:52)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:149)
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:125)
> at
> org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:57)
> at
> org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:109)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:205)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:99)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> at
> org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:427)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:314)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> at
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$6/61137731.run(Unknown
> Source)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> The bug seems similar to CASSANDRA-13320 which says it should be fixed in
> cassandra 3.0.13 but does not look fixed in 3.0.14
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-13946) Updating row TTL without updating values
[ https://issues.apache.org/jira/browse/CASSANDRA-13946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie resolved CASSANDRA-13946. - Resolution: Incomplete Closing - feel free to add a description and re-open. > Updating row TTL without updating values > > > Key: CASSANDRA-13946 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13946 > Project: Cassandra > Issue Type: New Feature > Components: Core, CQL >Reporter: Tomer >Priority: Trivial > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13932) Stress write order and seed order should be different
[
https://issues.apache.org/jira/browse/CASSANDRA-13932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13932:
Component/s: Stress
> Stress write order and seed order should be different
> -
>
> Key: CASSANDRA-13932
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13932
> Project: Cassandra
> Issue Type: Bug
> Components: Stress, Tools
>Reporter: Daniel Cranford
>Priority: Major
> Labels: stress
> Attachments: 0001-Initial-implementation-cassandra-3.11.patch,
> vmtouch-after.txt, vmtouch-before.txt
>
>
> Read tests get an unrealistic boost in performance because they read data
> from a set of partitions that was written sequentially.
> I ran into this while running a timed read test against a large data set (250
> million partition keys) {noformat}cassandra-stress read
> duration=30m{noformat} While the test was running, I noticed one node was
> performing zero IO after an initial period.
> I discovered each node in the cluster only had blocks from a single SSTable
> loaded in the FS cache. {noformat}vmtouch -v /path/to/sstables{noformat}
> For the node that was performing zero IO, the SSTable in question was small
> enough to fit into the FS cache.
> I realized that when a read test is run for a duration or until rate
> convergenge, the default population for the seeds is a GAUSSIAN distribution
> over the first million seeds. Because of the way compaction works, partitions
> that are written sequentially will (with high probability) always live in the
> same SSTable. That means that while the first million seeds will generate
> partition keys that will be randomly distributed in the token space, they
> will most likely all live in the same SSTable. When this SSTable is small
> enough to fit into the FS cache, you get unbelievably good results for a read
> test. Consider that a dataset 4x the size of the FS cache will have almost
> 1/2 the data in SSTables small enough to fit into the FS cache.
> Adjusting the population of seeds used during the read test to be the entire
> 250 million seeds used to load the cluster does not fix the
> problem.{noformat}cassandra-stress read duration=30m -pop
> dist=gaussian(1..250M){noformat}
> or (same population, larger sample) {noformat}cassandra-stress read
> n=250M{noformat}
> Any distribution other than the uniform distribution has one or more modes,
> and the mode(s) of such a distribution will cluster reads around a certain
> seed range which corresponds to a certain set of sequential writes which
> corresponds to (with high probability) a single SSTable.
> My patch against cassandra-3.11 fixes this by shuffling the sequence of
> generated seeds. Each seed value will still be generated once and only once.
> The old behavior of sequential seed generation (ie seed(n+1) = seed( n) + 1)
> may be selected by using the no-shuffle flag. e.g. {noformat}cassandra-stress
> read duration=30m -pop no-shuffle{noformat}
> Results: In [^vmtouch-before.txt] only pages from a single SSTable are
> present in the FS cache while in [^vmtouch-after.txt] an equal proportion of
> all SSTables are present in the FS cache.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13920) Adding regular column to COMPACT STORAGE with w/o clustering keys table causes an exception
[
https://issues.apache.org/jira/browse/CASSANDRA-13920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13920:
Component/s: CQL
Core
> Adding regular column to COMPACT STORAGE with w/o clustering keys table
> causes an exception
> ---
>
> Key: CASSANDRA-13920
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13920
> Project: Cassandra
> Issue Type: Bug
> Components: Core, CQL
>Reporter: Alex Petrov
>Priority: Minor
>
> When trying to {{ALTER}} a {{COMPACT TABLE}} without clustering keys (i.e.
> non-dense one), you'll get an exception.
> Adding regular columns to non-dense compact tables should be forbidden
> (adding static ones already fails with {{Static columns are not allowed in
> COMPACT STORAGE tables}}), just as adding columns to dense compact tables
> which throws {{Cannot add new column to a COMPACT STORAGE table}} (or the
> error message should be adjusted).
> dtest to reproduce:
> {code}
> from cql_tests import CQLTester
> class StorageProxyCQLTester(CQLTester):
> def test_sparse_compact(self):
> session = self.prepare(nodes=2, rf=2)
> session.execute("CREATE TABLE sparse_compact_table (k int PRIMARY
> KEY, v1 int, v2 int) WITH COMPACT STORAGE;")
> session.execute("ALTER TABLE sparse_compact_table ADD wat int",)
> {code}
> Exception:
> {code}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.CompactTables.getCompactValueColumn(CompactTables.java:67)
> ~[main/:na]
> at
> org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:337)
> ~[main/:na]
> at
> org.apache.cassandra.config.CFMetaData.validate(CFMetaData.java:935)
> ~[main/:na]
> at
> org.apache.cassandra.service.MigrationManager.announceColumnFamilyUpdate(MigrationManager.java:421)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.statements.AlterTableStatement.announceMigration(AlterTableStatement.java:288)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[main/:na]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:222)
> ~[main/:na]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:115)
> ~[main/:na]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513)
> [main/:na]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407)
> [main/:na]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348)
> [netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_121]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> [main/:na]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13917) COMPACT STORAGE inserts on tables without clusterings accept hidden column1 and value columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13917:
Component/s: Core
> COMPACT STORAGE inserts on tables without clusterings accept hidden column1
> and value columns
> -
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Minor
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?,
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1,
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)",
> 1, 1, 1, ByteBufferUtil.bytes('a'));
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?,
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should
> be as easy as just adding validations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13908) Add JDK9 new CRC32C checksum
[ https://issues.apache.org/jira/browse/CASSANDRA-13908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13908: Labels: Performance (was: ) > Add JDK9 new CRC32C checksum > > > Key: CASSANDRA-13908 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13908 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Yingqi Lu >Priority: Major > Labels: Performance > > Dear All, > JDK9 introduces a new API java.util.zip.CRC32C. Our early data show CRC32C > provides additional performance benefits over CRC32 checksum. > We are proposing to add CRC32C as one of the checksum types in addition to > Adler32 and CRC32. Performance data and prototype code changes will be > submitted here soon. At the meantime, please feel free to provide your > feedback and comments. > Thanks, > Yingqi Lu -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13903) Add JDK9 vectorizedMismatch array compare support
[ https://issues.apache.org/jira/browse/CASSANDRA-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-13903: Labels: Performance (was: ) > Add JDK9 vectorizedMismatch array compare support > - > > Key: CASSANDRA-13903 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13903 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Yingqi Lu >Priority: Major > Labels: Performance > > Dear All, > JDK9 has recently been released (http://openjdk.java.net/projects/jdk9/). It > introduces a set of new vectorizedMismatch APIs for array comparison. On > supported platforms, the new JDK9 implementation is intrisified to leverage > SIMD instructions. For a byte array comparison, up to 64 bytes (512 bits) can > be compared as a single unit. Feature details please refer to > http://download.java.net/java/jdk9/docs/api/java/util/Arrays.html and > https://bugs.openjdk.java.net/browse/JDK-8033148 > Currently in Cassandra, keys are implemented as ByteBuffers and compared as > byte arrays most of the time. Key comparison, for example decorated key > compare, it is either done with unsafe operation taking 8 bytes at a time, or > pure java operation comparing byte by byte. This can be optimized with new > JDK9 java.util.Arrays.compare APIs on modern CPUs. > Please let us know your feedback. At the meantime, we will submit a patch > with performance studies in couple of weeks. > Thanks, > Yingqi Lu -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13887) Add SASI metrics to JMX
[
https://issues.apache.org/jira/browse/CASSANDRA-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-13887:
Component/s: Observability
> Add SASI metrics to JMX
> ---
>
> Key: CASSANDRA-13887
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13887
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability, sasi
>Reporter: James Howe
>Priority: Minor
>
> Currently there are MBeans for secondary index metrics
> {{org.apache.cassandra.metrics:type=IndexTable}} but I cannot see SASI
> metrics anywhere.
> The only place they're even mentioned is in the table's {{BuiltIndexes}} list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14423) SSTables stop being compacted
[
https://issues.apache.org/jira/browse/CASSANDRA-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16538761#comment-16538761
]
Michael Shuler commented on CASSANDRA-14423:
Thanks! That commit looks good to me.
- {{ant artifacts}} builds again successfully on JDK7
- {{ant test -Dtest.name=AntiCompactionTest}} passes successfully on JDK7
> SSTables stop being compacted
> -
>
> Key: CASSANDRA-14423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14423
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Blocker
> Fix For: 2.2.13, 3.0.17, 3.11.3
>
>
> So seeing a problem in 3.11.0 where SSTables are being lost from the view and
> not being included in compactions/as candidates for compaction. It seems to
> get progressively worse until there's only 1-2 SSTables in the view which
> happen to be the most recent SSTables and thus compactions completely stop
> for that table.
> The SSTables seem to still be included in reads, just not compactions.
> The issue can be fixed by restarting C*, as it will reload all SSTables into
> the view, but this is only a temporary fix. User defined/major compactions
> still work - not clear if they include the result back in the view but is not
> a good work around.
> This also results in a discrepancy between SSTable count and SSTables in
> levels for any table using LCS.
> {code:java}
> Keyspace : xxx
> Read Count: 57761088
> Read Latency: 0.10527088681224288 ms.
> Write Count: 2513164
> Write Latency: 0.018211106398149903 ms.
> Pending Flushes: 0
> Table: xxx
> SSTable count: 10
> SSTables in each level: [2, 0, 0, 0, 0, 0, 0, 0, 0]
> Space used (live): 894498746
> Space used (total): 894498746
> Space used by snapshots (total): 0
> Off heap memory used (total): 11576197
> SSTable Compression Ratio: 0.6956629530569777
> Number of keys (estimate): 3562207
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 87
> Local read count: 57761088
> Local read latency: 0.108 ms
> Local write count: 2513164
> Local write latency: NaN ms
> Pending flushes: 0
> Percent repaired: 86.33
> Bloom filter false positives: 43
> Bloom filter false ratio: 0.0
> Bloom filter space used: 8046104
> Bloom filter off heap memory used: 8046024
> Index summary off heap memory used: 3449005
> Compression metadata off heap memory used: 81168
> Compacted partition minimum bytes: 104
> Compacted partition maximum bytes: 5722
> Compacted partition mean bytes: 175
> Average live cells per slice (last five minutes): 1.0
> Maximum live cells per slice (last five minutes): 1
> Average tombstones per slice (last five minutes): 1.0
> Maximum tombstones per slice (last five minutes): 1
> Dropped Mutations: 0
> {code}
> Also for STCS we've confirmed that SSTable count will be different to the
> number of SSTables reported in the Compaction Bucket's. In the below example
> there's only 3 SSTables in a single bucket - no more are listed for this
> table. Compaction thresholds haven't been modified for this table and it's a
> very basic KV schema.
> {code:java}
> Keyspace : yyy
> Read Count: 30485
> Read Latency: 0.06708991307200263 ms.
> Write Count: 57044
> Write Latency: 0.02204061776873992 ms.
> Pending Flushes: 0
> Table: yyy
> SSTable count: 19
> Space used (live): 18195482
> Space used (total): 18195482
> Space used by snapshots (total): 0
> Off heap memory used (total): 747376
> SSTable Compression Ratio: 0.7607394576769735
> Number of keys (estimate): 116074
> Memtable cell count: 0
> Memtable data size: 0
> Memtable off heap memory used: 0
> Memtable switch count: 39
> Local read count: 30485
> Local read latency: NaN ms
> Local write count: 57044
> Local write latency: NaN ms
> Pending flushes: 0
> Percent repaired: 79.76
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 690912
> Bloom filter off heap memory used: 690760
> Index summary off heap memory used: 54736
> Compression metadata off heap memory used: 1880
> Compacted partition minimum bytes: 73
> Compacted partition maximum bytes: 124
> Compacted partition mean bytes: 96
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> {code:java}
> Apr 27 03:10:39 cassandra[9263]:
[jira] [Updated] (CASSANDRA-14564) issue while adding a column in a table with compact storage
[
https://issues.apache.org/jira/browse/CASSANDRA-14564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laxmikant Upadhyay updated CASSANDRA-14564:
---
Description:
I have upgraded my system from cassandra 2.1.16 to 3.11.2. We had some tables
with COMPACT STORAGE enabled. We see some weird behaviour of cassandra while
adding a column into it.
Cassandra does not give any error while altering however the added column is
invisible.
Same behaviour when we create a new table with compact storage and try to alter
it. Below is the commands ran in sequence:
{code:java}
x@cqlsh:xuser> CREATE TABLE xuser.employee(emp_id int PRIMARY KEY,emp_name
text, emp_city text, emp_sal varint, emp_phone varint ) WITH COMPACT STORAGE;
x@cqlsh:xuser> desc table xuser.employee ;
CREATE TABLE xuser.employee (
emp_id int PRIMARY KEY,
emp_city text,
emp_name text,
emp_phone varint,
emp_sal varint
) WITH COMPACT STORAGE
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';{code}
Now altering the table by adding a new column:
{code:java}
x@cqlsh:xuser> alter table employee add profile text;
x@cqlsh:xuser> desc table xuser.employee ;
CREATE TABLE xuser.employee (
emp_id int PRIMARY KEY,
emp_city text,
emp_name text,
emp_phone varint,
emp_sal varint
) WITH COMPACT STORAGE
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
{code}
notice that above desc table result does not have newly added column profile.
However when i try to add it again it gives column already exist;
{code:java}
x@cqlsh:xuser> alter table employee add profile text;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid
column name profile because it conflicts with an existing column"
x@cqlsh:xuser> select emp_name,profile from employee;
emp_name | profile
--+-
(0 rows)
x@cqlsh:xuser>
{code}
Inserting also behaves strange:
{code:java}
x@cqlsh:xuser> INSERT INTO employee (emp_id , emp_city , emp_name , emp_phone ,
emp_sal ,profile) VALUES ( 1, 'ggn', 'john', 123456, 5, 'SE');
InvalidRequest: Error from server: code=2200 [Invalid query] message="Some
clustering keys are missing: column1"
x@cqlsh:xuser> INSERT INTO employee (emp_id , emp_city , emp_name , emp_phone ,
emp_sal ,profile,column1) VALUES ( 1, 'ggn', 'john', 123456, 5, 'SE',null);
x@cqlsh:xuser> select * from employee;
emp_id | emp_city | emp_name | emp_phone | emp_sal
+--+--+---+-
(0 rows)
{code}
was:
I have upgraded my system from cassandra 2.1.16 to 3.11.2. We had some tables
with COMPACT STORAGE enabled. We see some weird behaviour of cassandra while
adding a column into it.
The cassandra does not gives any error while altering however the added column
is invisible.
This same behaviour when we create a new table with compact storage and try to
alter it. Below is the commands ran in sequence:
{code:java}
x@cqlsh:xuser> CREATE TABLE xuser.employee(emp_id int PRIMARY KEY,emp_name
text, emp_city text, emp_sal varint, emp_phone varint ) WITH COMPACT STORAGE;
x@cqlsh:xuser> desc table xuser.employee ;
CREATE TABLE xuser.employee (
emp_id int PRIMARY KEY,
emp_city text,
emp_name text,
emp_phone varint,
emp_sal varint
) WITH COMPACT STORAGE
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND
[jira] [Created] (CASSANDRA-14564) issue while adding a column in a table with compact storage
Laxmikant Upadhyay created CASSANDRA-14564:
--
Summary: issue while adding a column in a table with compact
storage
Key: CASSANDRA-14564
URL: https://issues.apache.org/jira/browse/CASSANDRA-14564
Project: Cassandra
Issue Type: Bug
Components: Core, CQL
Reporter: Laxmikant Upadhyay
I have upgraded my system from cassandra 2.1.16 to 3.11.2. We had some tables
with COMPACT STORAGE enabled. We see some weird behaviour of cassandra while
adding a column into it.
The cassandra does not gives any error while altering however the added column
is invisible.
This same behaviour when we create a new table with compact storage and try to
alter it. Below is the commands ran in sequence:
{code:java}
x@cqlsh:xuser> CREATE TABLE xuser.employee(emp_id int PRIMARY KEY,emp_name
text, emp_city text, emp_sal varint, emp_phone varint ) WITH COMPACT STORAGE;
x@cqlsh:xuser> desc table xuser.employee ;
CREATE TABLE xuser.employee (
emp_id int PRIMARY KEY,
emp_city text,
emp_name text,
emp_phone varint,
emp_sal varint
) WITH COMPACT STORAGE
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';{code}
Now altering the table by adding a new column:
{code:java}
x@cqlsh:xuser> alter table employee add profile text;
x@cqlsh:xuser> desc table xuser.employee ;
CREATE TABLE xuser.employee (
emp_id int PRIMARY KEY,
emp_city text,
emp_name text,
emp_phone varint,
emp_sal varint
) WITH COMPACT STORAGE
AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';
{code}
notice that above desc table result does not have newly added column profile.
However when i try to add it again it gives column already exist;
{code:java}
x@cqlsh:xuser> alter table employee add profile text;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid
column name profile because it conflicts with an existing column"
x@cqlsh:xuser> select emp_name,profile from employee;
emp_name | profile
--+-
(0 rows)
x@cqlsh:xuser>
{code}
Inserting also behaves strange:
{code:java}
x@cqlsh:xuser> INSERT INTO employee (emp_id , emp_city , emp_name , emp_phone ,
emp_sal ,profile) VALUES ( 1, 'ggn', 'john', 123456, 5, 'SE');
InvalidRequest: Error from server: code=2200 [Invalid query] message="Some
clustering keys are missing: column1"
x@cqlsh:xuser> INSERT INTO employee (emp_id , emp_city , emp_name , emp_phone ,
emp_sal ,profile,column1) VALUES ( 1, 'ggn', 'john', 123456, 5, 'SE',null);
x@cqlsh:xuser> select * from employee;
emp_id | emp_city | emp_name | emp_phone | emp_sal
+--+--+---+-
(0 rows)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
