[jira] [Updated] (CASSANDRA-9463) ant test-all results incomplete when parsed
[ https://issues.apache.org/jira/browse/CASSANDRA-9463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-9463: -- Fix Version/s: (was: 2.2.x) (was: 3.x) 2.2.0 rc1 3.0 beta 1 ant test-all results incomplete when parsed --- Key: CASSANDRA-9463 URL: https://issues.apache.org/jira/browse/CASSANDRA-9463 Project: Cassandra Issue Type: Test Reporter: Michael Shuler Assignee: Ariel Weisberg Fix For: 3.0 beta 1, 2.2.0 rc1 trunk `ant test` - 1,196 total tests trunk `ant test-all` - 1,353 total tests `ant test-all` runs test,long-test,test-compression,pig-test,test-clientutil-jar, so we should be getting 1196*2 (test, test-compresssion) + N (long-test) + 24 (pig-test) + N (test-clientutil-jar) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9544) Allow specification of TLS protocol to use for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-9544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617012#comment-14617012
]
Tyler Hobbs commented on CASSANDRA-9544:
I like adding the ability to configure the ssl protocol version, but I think
keeping TLSv1 as the default is the best option. It's the Cassandra default,
it has always been the cqlsh default, and it should be the most secure choice.
I've created a [branch with the
changes|https://github.com/thobbs/cassandra/tree/CASSANDRA-9544].
Pending CI test runs:
* [2.1
dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-9544-dtest/]
* [2.2
testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-9544-testall/]
Allow specification of TLS protocol to use for cqlsh
Key: CASSANDRA-9544
URL: https://issues.apache.org/jira/browse/CASSANDRA-9544
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Jesse Szwedko
Labels: cqlsh, tls
Currently when using {{cqlsh}} with {{--ssl}} it tries to use TLS 1.0 to
connect. I have my server only serving TLS 1.2 which means that I cannot
connect.
It would be nice if {{cqlsh}} allowed the TLS protocol it uses to connect to
be configured.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616872#comment-14616872 ] Carl Yeksigian commented on CASSANDRA-6477: --- # MV use the batchlog in order to provide eventual consistency # MV updates follow the same CL as the base update has # No, repairs to the base table only assure that the base table is consistent; same with a repair on a MV In the patch, there is no way to do a repair between the MV and the base table other than dropping and recreating the view. Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9132) resumable_bootstrap_test can hang
[
https://issues.apache.org/jira/browse/CASSANDRA-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616886#comment-14616886
]
Jim Witschey commented on CASSANDRA-9132:
-
Thanks, I'll fix that.
resumable_bootstrap_test can hang
-
Key: CASSANDRA-9132
URL: https://issues.apache.org/jira/browse/CASSANDRA-9132
Project: Cassandra
Issue Type: Bug
Components: Tests
Reporter: Tyler Hobbs
Assignee: Yuki Morishita
Fix For: 2.0.15, 2.1.6, 2.2.0 rc1
Attachments: 9132-2.0.txt, 9132-followup-2.0.txt
The {{bootstrap_test.TestBootstrap.resumable_bootstrap_test}} can hang
sometimes. It looks like the following line never completes:
{noformat}
node3.watch_log_for(Listening for thrift clients...)
{noformat}
I'm not familiar enough with the recent bootstrap changes to know why that's
not happening.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616945#comment-14616945 ] Joshua McKenzie commented on CASSANDRA-6477: bq. In the patch, there is no way to do a repair between the MV and the base table other than dropping and recreating the view. Would this be worth a follow-up ticket, or would the repair overhead / time frame be long enough that it makes sense just to recommend people drop/recreate MV after repair? If so, would it make sense to bolt that functionality as an option onto the repair process? Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[01/12] cassandra git commit: Scrub (recover) sstables even when -Index.db is missing
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 62714a9fe - 452d6a445
refs/heads/cassandra-2.1 4de943fd9 - 4c94ef20d
refs/heads/cassandra-2.2 7d31068da - ebe18bb2f
refs/heads/trunk db68d1cfd - a8bb75a7e
Scrub (recover) sstables even when -Index.db is missing
patch by mck; reviewed by stefania for CASSANDRA-9591
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/452d6a44
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/452d6a44
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/452d6a44
Branch: refs/heads/cassandra-2.0
Commit: 452d6a445a6935d3e7d0e0fdf59e87d2a2ff95e7
Parents: 62714a9
Author: Stefania Alborghetti stefania.alborghe...@datastax.com
Authored: Mon Jun 15 16:49:03 2015 +0800
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 14:27:25 2015 +0100
--
CHANGES.txt | 4 ++
.../cassandra/db/compaction/Scrubber.java | 36 +++
.../cassandra/io/sstable/SSTableReader.java | 47
.../cassandra/tools/StandaloneScrubber.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 25 +++
5 files changed, 96 insertions(+), 18 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/452d6a44/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ca4d4b5..bd1db92 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,7 @@
+2.0.18
+* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
+
+
2.0.17
* Avoid NPE in AuthSuccess#decode (CASSANDRA-9727)
* Add listen_address to system.local (CASSANDRA-9603)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/452d6a44/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
index ea10855..dc60efa 100644
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@ -96,7 +96,16 @@ public class Scrubber implements Closeable
? new ScrubController(cfs)
: new CompactionController(cfs,
Collections.singleton(sstable), CompactionManager.getDefaultGcBefore(cfs));
this.isCommutative =
cfs.metadata.getDefaultValidator().isCommutative();
-this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)));
+
+boolean hasIndexFile = (new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX))).exists();
+if (!hasIndexFile)
+{
+// if there's any corruption in the -Data.db then rows can't be
skipped over. but it's worth a shot.
+outputHandler.warn(Missing component: +
sstable.descriptor.filenameFor(Component.PRIMARY_INDEX));
+}
+
+this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
+hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)) : 0);
// loop through each row, deserializing to check for damage.
// we'll also loop through the index at the same time, using the
position from the index to recover if the
@@ -105,7 +114,11 @@ public class Scrubber implements Closeable
this.dataFile = isOffline
? sstable.openDataReader()
:
sstable.openDataReader(CompactionManager.instance.getRateLimiter());
-this.indexFile = RandomAccessReader.open(new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)));
+
+this.indexFile = hasIndexFile
+? RandomAccessReader.open(new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)))
+: null;
+
this.scrubInfo = new ScrubInfo(dataFile, sstable);
this.currentRowPositionFromIndex = 0;
@@ -117,7 +130,8 @@ public class Scrubber implements Closeable
outputHandler.output(String.format(Scrubbing %s (%s bytes), sstable,
dataFile.length()));
try
{
-nextIndexKey = ByteBufferUtil.readWithShortLength(indexFile);
+nextIndexKey = indexAvailable() ?
ByteBufferUtil.readWithShortLength(indexFile) : null;
+if (indexAvailable())
{
// throw away variable so we don't have a side effect in the
assert
long firstRowPositionFromIndex =
RowIndexEntry.serializer.deserialize(indexFile,
sstable.descriptor.version).position;
@@ -181,7 +195,7 @@ public class Scrubber implements
[02/12] cassandra git commit: Scrub (recover) sstables even when -Index.db is missing
Scrub (recover) sstables even when -Index.db is missing
patch by mck; reviewed by stefania for CASSANDRA-9591
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/452d6a44
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/452d6a44
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/452d6a44
Branch: refs/heads/cassandra-2.1
Commit: 452d6a445a6935d3e7d0e0fdf59e87d2a2ff95e7
Parents: 62714a9
Author: Stefania Alborghetti stefania.alborghe...@datastax.com
Authored: Mon Jun 15 16:49:03 2015 +0800
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 14:27:25 2015 +0100
--
CHANGES.txt | 4 ++
.../cassandra/db/compaction/Scrubber.java | 36 +++
.../cassandra/io/sstable/SSTableReader.java | 47
.../cassandra/tools/StandaloneScrubber.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 25 +++
5 files changed, 96 insertions(+), 18 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/452d6a44/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ca4d4b5..bd1db92 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,7 @@
+2.0.18
+* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
+
+
2.0.17
* Avoid NPE in AuthSuccess#decode (CASSANDRA-9727)
* Add listen_address to system.local (CASSANDRA-9603)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/452d6a44/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
index ea10855..dc60efa 100644
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@ -96,7 +96,16 @@ public class Scrubber implements Closeable
? new ScrubController(cfs)
: new CompactionController(cfs,
Collections.singleton(sstable), CompactionManager.getDefaultGcBefore(cfs));
this.isCommutative =
cfs.metadata.getDefaultValidator().isCommutative();
-this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)));
+
+boolean hasIndexFile = (new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX))).exists();
+if (!hasIndexFile)
+{
+// if there's any corruption in the -Data.db then rows can't be
skipped over. but it's worth a shot.
+outputHandler.warn(Missing component: +
sstable.descriptor.filenameFor(Component.PRIMARY_INDEX));
+}
+
+this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
+hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)) : 0);
// loop through each row, deserializing to check for damage.
// we'll also loop through the index at the same time, using the
position from the index to recover if the
@@ -105,7 +114,11 @@ public class Scrubber implements Closeable
this.dataFile = isOffline
? sstable.openDataReader()
:
sstable.openDataReader(CompactionManager.instance.getRateLimiter());
-this.indexFile = RandomAccessReader.open(new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)));
+
+this.indexFile = hasIndexFile
+? RandomAccessReader.open(new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)))
+: null;
+
this.scrubInfo = new ScrubInfo(dataFile, sstable);
this.currentRowPositionFromIndex = 0;
@@ -117,7 +130,8 @@ public class Scrubber implements Closeable
outputHandler.output(String.format(Scrubbing %s (%s bytes), sstable,
dataFile.length()));
try
{
-nextIndexKey = ByteBufferUtil.readWithShortLength(indexFile);
+nextIndexKey = indexAvailable() ?
ByteBufferUtil.readWithShortLength(indexFile) : null;
+if (indexAvailable())
{
// throw away variable so we don't have a side effect in the
assert
long firstRowPositionFromIndex =
RowIndexEntry.serializer.deserialize(indexFile,
sstable.descriptor.version).position;
@@ -181,7 +195,7 @@ public class Scrubber implements Closeable
outputHandler.debug(String.format(Index doublecheck:
row %s is %s bytes, ByteBufferUtil.bytesToHex(currentIndexKey),
dataSizeFromIndex));
}
-assert
[11/12] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/Scrubber.java
src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ebe18bb2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ebe18bb2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ebe18bb2
Branch: refs/heads/trunk
Commit: ebe18bb2ff62602fd5f55b969ecf665d2d3e5ace
Parents: 7d31068 4c94ef2
Author: Benedict Elliott Smith bened...@apache.org
Authored: Tue Jul 7 16:28:19 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 16:28:19 2015 +0100
--
CHANGES.txt | 5 ++
.../cassandra/db/compaction/Scrubber.java | 38 +++
.../io/sstable/format/SSTableReader.java| 50 ++--
.../cassandra/tools/StandaloneScrubber.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 25 ++
5 files changed, 95 insertions(+), 25 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ebe18bb2/CHANGES.txt
--
diff --cc CHANGES.txt
index 7c6b4ad,2cbc7c4..a863ad8
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,28 -1,12 +1,33 @@@
-2.1.9
++2.2.0-rc3
+ Merged from 2.0:
- * Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
++ * Scrub (recover) sstables even when -Index.db is missing (CASSANDRA-9591)
+
+
-2.1.8
+2.2.0-rc2
+ * Re-enable memory-mapped I/O on Windows (CASSANDRA-9658)
+ * Warn when an extra-large partition is compacted (CASSANDRA-9643)
+ * (cqlsh) Allow setting the initial connection timeout (CASSANDRA-9601)
+ * BulkLoader has --transport-factory option but does not use it
(CASSANDRA-9675)
+ * Allow JMX over SSL directly from nodetool (CASSANDRA-9090)
+ * Update cqlsh for UDFs (CASSANDRA-7556)
+ * Change Windows kernel default timer resolution (CASSANDRA-9634)
+ * Deprected sstable2json and json2sstable (CASSANDRA-9618)
+ * Allow native functions in user-defined aggregates (CASSANDRA-9542)
+ * Don't repair system_distributed by default (CASSANDRA-9621)
+ * Fix mixing min, max, and count aggregates for blob type (CASSANRA-9622)
+ * Rename class for DATE type in Java driver (CASSANDRA-9563)
+ * Duplicate compilation of UDFs on coordinator (CASSANDRA-9475)
+ * Fix connection leak in CqlRecordWriter (CASSANDRA-9576)
+ * Mlockall before opening system sstables remove boot_without_jna option
(CASSANDRA-9573)
+ * Add functions to convert timeuuid to date or time, deprecate dateOf and
unixTimestampOf (CASSANDRA-9229)
+ * Make sure we cancel non-compacting sstables from LifecycleTransaction
(CASSANDRA-9566)
+ * Fix deprecated repair JMX API (CASSANDRA-9570)
+ * Add logback metrics (CASSANDRA-9378)
+ * Update and refactor ant test/test-compression to run the tests in parallel
(CASSANDRA-9583)
+ * Fix upgrading to new directory for secondary index (CASSANDRA-9687)
+Merged from 2.1:
* (cqlsh) Fix bad check for CQL compatibility when DESCRIBE'ing
COMPACT STORAGE tables with no clustering columns
- * Warn when an extra-large partition is compacted (CASSANDRA-9643)
* Eliminate strong self-reference chains in sstable ref tidiers
(CASSANDRA-9656)
* Ensure StreamSession uses canonical sstable reader instances
(CASSANDRA-9700)
* Ensure memtable book keeping is not corrupted in the event we shrink usage
(CASSANDRA-9681)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ebe18bb2/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/Scrubber.java
index 10952e7,b1c12e0..5a0b354
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@@ -109,9 -101,17 +109,18 @@@ public class Scrubber implements Closea
? new ScrubController(cfs)
: new CompactionController(cfs,
Collections.singleton(sstable), CompactionManager.getDefaultGcBefore(cfs));
this.isCommutative = cfs.metadata.isCounter();
+
+ boolean hasIndexFile = (new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX))).exists();
+this.isIndex = cfs.isIndex();
+ if (!hasIndexFile)
+ {
+ // if there's any corruption in the -Data.db then rows can't be
skipped over. but it's worth a shot.
+ outputHandler.warn(Missing component: +
sstable.descriptor.filenameFor(Component.PRIMARY_INDEX));
+ }
-
+this.checkData = checkData !this.isIndex;
[08/12] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ebe18bb2/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
index 0cc425a,000..247d181
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
@@@ -1,2218 -1,0 +1,2238 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.sstable.format;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+import java.util.*;
+import java.util.concurrent.*;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Predicate;
+import com.google.common.collect.Iterables;
+import com.google.common.collect.Iterators;
+import com.google.common.collect.Ordering;
+import com.google.common.primitives.Longs;
+import com.google.common.util.concurrent.RateLimiter;
+
+import com.clearspring.analytics.stream.cardinality.CardinalityMergeException;
+import com.clearspring.analytics.stream.cardinality.HyperLogLogPlus;
+import com.clearspring.analytics.stream.cardinality.ICardinality;
+import com.codahale.metrics.Counter;
+import org.apache.cassandra.cache.CachingOptions;
+import org.apache.cassandra.cache.InstrumentingCache;
+import org.apache.cassandra.cache.KeyCacheKey;
+import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor;
+import org.apache.cassandra.concurrent.ScheduledExecutors;
+import org.apache.cassandra.config.*;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
+import org.apache.cassandra.db.commitlog.ReplayPosition;
+import org.apache.cassandra.db.composites.CellName;
+import org.apache.cassandra.db.filter.ColumnSlice;
+import org.apache.cassandra.db.index.SecondaryIndex;
+import org.apache.cassandra.db.lifecycle.Tracker;
+import org.apache.cassandra.dht.*;
+import org.apache.cassandra.io.compress.CompressionMetadata;
+import org.apache.cassandra.io.sstable.*;
+import org.apache.cassandra.io.sstable.metadata.*;
+import org.apache.cassandra.io.util.*;
+import org.apache.cassandra.metrics.RestorableMeter;
+import org.apache.cassandra.metrics.StorageMetrics;
+import org.apache.cassandra.service.ActiveRepairService;
+import org.apache.cassandra.service.CacheService;
+import org.apache.cassandra.service.StorageService;
+import org.apache.cassandra.utils.*;
+import org.apache.cassandra.utils.concurrent.OpOrder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.cassandra.utils.concurrent.Ref;
+import org.apache.cassandra.utils.concurrent.SelfRefCounted;
+
+import static
org.apache.cassandra.db.Directories.SECONDARY_INDEX_NAME_SEPARATOR;
+
+/**
+ * An SSTableReader can be constructed in a number of places, but typically
is either
+ * read from disk at startup, or constructed from a flushed memtable, or
after compaction
+ * to replace some existing sstables. However once created, an sstablereader
may also be modified.
+ *
+ * A reader's OpenReason describes its current stage in its lifecycle, as
follows:
+ *
+ *
+ * pre {@code
+ * NORMAL
+ * From: None= Reader has been read from disk, either at
startup or from a flushed memtable
+ * EARLY = Reader is the final result of a compaction
+ * MOVED_START = Reader WAS being compacted, but this failed and
it has been restored to NORMAL status
+ *
+ * EARLY
+ * From: None= Reader is a compaction replacement that is
either incomplete and has been opened
+ *to represent its partial result status, or has
been finished but the compaction
+ *it is a part of has not yet completed fully
+ * EARLY = Same as from None, only it is not the first
time it has been
+ *
+ * MOVED_START
+ * From: NORMAL = Reader is being
[05/12] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/Scrubber.java
src/java/org/apache/cassandra/io/sstable/SSTableReader.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4c94ef20
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4c94ef20
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4c94ef20
Branch: refs/heads/cassandra-2.2
Commit: 4c94ef20d3562ab8f0a922945d78464d6c475d98
Parents: 4de943f 452d6a4
Author: Benedict Elliott Smith bened...@apache.org
Authored: Tue Jul 7 16:27:58 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 16:27:58 2015 +0100
--
CHANGES.txt | 5 ++
.../cassandra/db/compaction/Scrubber.java | 37 +++---
.../cassandra/io/sstable/SSTableReader.java | 52 ++--
.../cassandra/tools/StandaloneScrubber.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 25 ++
5 files changed, 97 insertions(+), 24 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c94ef20/CHANGES.txt
--
diff --cc CHANGES.txt
index 95dc8be,bd1db92..2cbc7c4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,8 +1,23 @@@
-2.0.18
-* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
-
-
-2.0.17
++2.1.9
++Merged from 2.0:
++ * Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
++
++
+2.1.8
+ * (cqlsh) Fix bad check for CQL compatibility when DESCRIBE'ing
+ COMPACT STORAGE tables with no clustering columns
+ * Warn when an extra-large partition is compacted (CASSANDRA-9643)
+ * Eliminate strong self-reference chains in sstable ref tidiers
(CASSANDRA-9656)
+ * Ensure StreamSession uses canonical sstable reader instances
(CASSANDRA-9700)
+ * Ensure memtable book keeping is not corrupted in the event we shrink usage
(CASSANDRA-9681)
+ * Update internal python driver for cqlsh (CASSANDRA-9064)
+ * Fix IndexOutOfBoundsException when inserting tuple with too many
+ elements using the string literal notation (CASSANDRA-9559)
+ * Allow JMX over SSL directly from nodetool (CASSANDRA-9090)
+ * Fix incorrect result for IN queries where column not found (CASSANDRA-9540)
+ * Enable describe on indices (CASSANDRA-7814)
+ * ColumnFamilyStore.selectAndReference may block during compaction
(CASSANDRA-9637)
+Merged from 2.0:
* Avoid NPE in AuthSuccess#decode (CASSANDRA-9727)
* Add listen_address to system.local (CASSANDRA-9603)
* Bug fixes to resultset metadata construction (CASSANDRA-9636)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c94ef20/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/Scrubber.java
index ce98a13,dc60efa..b1c12e0
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@@ -100,8 -95,17 +100,18 @@@ public class Scrubber implements Closea
this.controller = isOffline
? new ScrubController(cfs)
: new CompactionController(cfs,
Collections.singleton(sstable), CompactionManager.getDefaultGcBefore(cfs));
-this.isCommutative =
cfs.metadata.getDefaultValidator().isCommutative();
+this.isCommutative = cfs.metadata.isCounter();
- this.expectedBloomFilterSize =
Math.max(cfs.metadata.getMinIndexInterval(),
(int)(SSTableReader.getApproximateKeyCount(toScrub)));
+
+ boolean hasIndexFile = (new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX))).exists();
+ if (!hasIndexFile)
+ {
+ // if there's any corruption in the -Data.db then rows can't be
skipped over. but it's worth a shot.
+ outputHandler.warn(Missing component: +
sstable.descriptor.filenameFor(Component.PRIMARY_INDEX));
+ }
+
-this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
-hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)) : 0);
++this.expectedBloomFilterSize = Math.max(
++cfs.metadata.getMinIndexInterval(),
++hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub)) : 0);
// loop through each row, deserializing to check for damage.
// we'll also loop through the index at the same time, using the
position from the index to recover if the
@@@ -120,14 -128,13 +134,15 @@@
public void scrub()
{
[07/12] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/Scrubber.java
src/java/org/apache/cassandra/io/sstable/SSTableReader.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4c94ef20
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4c94ef20
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4c94ef20
Branch: refs/heads/trunk
Commit: 4c94ef20d3562ab8f0a922945d78464d6c475d98
Parents: 4de943f 452d6a4
Author: Benedict Elliott Smith bened...@apache.org
Authored: Tue Jul 7 16:27:58 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 16:27:58 2015 +0100
--
CHANGES.txt | 5 ++
.../cassandra/db/compaction/Scrubber.java | 37 +++---
.../cassandra/io/sstable/SSTableReader.java | 52 ++--
.../cassandra/tools/StandaloneScrubber.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 25 ++
5 files changed, 97 insertions(+), 24 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c94ef20/CHANGES.txt
--
diff --cc CHANGES.txt
index 95dc8be,bd1db92..2cbc7c4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,8 +1,23 @@@
-2.0.18
-* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
-
-
-2.0.17
++2.1.9
++Merged from 2.0:
++ * Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
++
++
+2.1.8
+ * (cqlsh) Fix bad check for CQL compatibility when DESCRIBE'ing
+ COMPACT STORAGE tables with no clustering columns
+ * Warn when an extra-large partition is compacted (CASSANDRA-9643)
+ * Eliminate strong self-reference chains in sstable ref tidiers
(CASSANDRA-9656)
+ * Ensure StreamSession uses canonical sstable reader instances
(CASSANDRA-9700)
+ * Ensure memtable book keeping is not corrupted in the event we shrink usage
(CASSANDRA-9681)
+ * Update internal python driver for cqlsh (CASSANDRA-9064)
+ * Fix IndexOutOfBoundsException when inserting tuple with too many
+ elements using the string literal notation (CASSANDRA-9559)
+ * Allow JMX over SSL directly from nodetool (CASSANDRA-9090)
+ * Fix incorrect result for IN queries where column not found (CASSANDRA-9540)
+ * Enable describe on indices (CASSANDRA-7814)
+ * ColumnFamilyStore.selectAndReference may block during compaction
(CASSANDRA-9637)
+Merged from 2.0:
* Avoid NPE in AuthSuccess#decode (CASSANDRA-9727)
* Add listen_address to system.local (CASSANDRA-9603)
* Bug fixes to resultset metadata construction (CASSANDRA-9636)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c94ef20/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/Scrubber.java
index ce98a13,dc60efa..b1c12e0
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@@ -100,8 -95,17 +100,18 @@@ public class Scrubber implements Closea
this.controller = isOffline
? new ScrubController(cfs)
: new CompactionController(cfs,
Collections.singleton(sstable), CompactionManager.getDefaultGcBefore(cfs));
-this.isCommutative =
cfs.metadata.getDefaultValidator().isCommutative();
+this.isCommutative = cfs.metadata.isCounter();
- this.expectedBloomFilterSize =
Math.max(cfs.metadata.getMinIndexInterval(),
(int)(SSTableReader.getApproximateKeyCount(toScrub)));
+
+ boolean hasIndexFile = (new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX))).exists();
+ if (!hasIndexFile)
+ {
+ // if there's any corruption in the -Data.db then rows can't be
skipped over. but it's worth a shot.
+ outputHandler.warn(Missing component: +
sstable.descriptor.filenameFor(Component.PRIMARY_INDEX));
+ }
+
-this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
-hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)) : 0);
++this.expectedBloomFilterSize = Math.max(
++cfs.metadata.getMinIndexInterval(),
++hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub)) : 0);
// loop through each row, deserializing to check for damage.
// we'll also loop through the index at the same time, using the
position from the index to recover if the
@@@ -120,14 -128,13 +134,15 @@@
public void scrub()
{
[06/12] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/Scrubber.java
src/java/org/apache/cassandra/io/sstable/SSTableReader.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4c94ef20
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4c94ef20
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4c94ef20
Branch: refs/heads/cassandra-2.1
Commit: 4c94ef20d3562ab8f0a922945d78464d6c475d98
Parents: 4de943f 452d6a4
Author: Benedict Elliott Smith bened...@apache.org
Authored: Tue Jul 7 16:27:58 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 16:27:58 2015 +0100
--
CHANGES.txt | 5 ++
.../cassandra/db/compaction/Scrubber.java | 37 +++---
.../cassandra/io/sstable/SSTableReader.java | 52 ++--
.../cassandra/tools/StandaloneScrubber.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 25 ++
5 files changed, 97 insertions(+), 24 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c94ef20/CHANGES.txt
--
diff --cc CHANGES.txt
index 95dc8be,bd1db92..2cbc7c4
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,18 -1,8 +1,23 @@@
-2.0.18
-* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
-
-
-2.0.17
++2.1.9
++Merged from 2.0:
++ * Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
++
++
+2.1.8
+ * (cqlsh) Fix bad check for CQL compatibility when DESCRIBE'ing
+ COMPACT STORAGE tables with no clustering columns
+ * Warn when an extra-large partition is compacted (CASSANDRA-9643)
+ * Eliminate strong self-reference chains in sstable ref tidiers
(CASSANDRA-9656)
+ * Ensure StreamSession uses canonical sstable reader instances
(CASSANDRA-9700)
+ * Ensure memtable book keeping is not corrupted in the event we shrink usage
(CASSANDRA-9681)
+ * Update internal python driver for cqlsh (CASSANDRA-9064)
+ * Fix IndexOutOfBoundsException when inserting tuple with too many
+ elements using the string literal notation (CASSANDRA-9559)
+ * Allow JMX over SSL directly from nodetool (CASSANDRA-9090)
+ * Fix incorrect result for IN queries where column not found (CASSANDRA-9540)
+ * Enable describe on indices (CASSANDRA-7814)
+ * ColumnFamilyStore.selectAndReference may block during compaction
(CASSANDRA-9637)
+Merged from 2.0:
* Avoid NPE in AuthSuccess#decode (CASSANDRA-9727)
* Add listen_address to system.local (CASSANDRA-9603)
* Bug fixes to resultset metadata construction (CASSANDRA-9636)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c94ef20/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/Scrubber.java
index ce98a13,dc60efa..b1c12e0
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@@ -100,8 -95,17 +100,18 @@@ public class Scrubber implements Closea
this.controller = isOffline
? new ScrubController(cfs)
: new CompactionController(cfs,
Collections.singleton(sstable), CompactionManager.getDefaultGcBefore(cfs));
-this.isCommutative =
cfs.metadata.getDefaultValidator().isCommutative();
+this.isCommutative = cfs.metadata.isCounter();
- this.expectedBloomFilterSize =
Math.max(cfs.metadata.getMinIndexInterval(),
(int)(SSTableReader.getApproximateKeyCount(toScrub)));
+
+ boolean hasIndexFile = (new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX))).exists();
+ if (!hasIndexFile)
+ {
+ // if there's any corruption in the -Data.db then rows can't be
skipped over. but it's worth a shot.
+ outputHandler.warn(Missing component: +
sstable.descriptor.filenameFor(Component.PRIMARY_INDEX));
+ }
+
-this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
-hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)) : 0);
++this.expectedBloomFilterSize = Math.max(
++cfs.metadata.getMinIndexInterval(),
++hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub)) : 0);
// loop through each row, deserializing to check for damage.
// we'll also loop through the index at the same time, using the
position from the index to recover if the
@@@ -120,14 -128,13 +134,15 @@@
public void scrub()
{
[12/12] cassandra git commit: Merge branch 'cassandra-2.2' into trunk
Merge branch 'cassandra-2.2' into trunk
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/Scrubber.java
src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8bb75a7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8bb75a7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8bb75a7
Branch: refs/heads/trunk
Commit: a8bb75a7e1a09ca05ceb8d566f2c9a88d3122c27
Parents: db68d1c ebe18bb
Author: Benedict Elliott Smith bened...@apache.org
Authored: Tue Jul 7 16:45:02 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 16:45:02 2015 +0100
--
CHANGES.txt | 3 ++
.../db/compaction/CompactionManager.java| 15 --
.../cassandra/db/compaction/Scrubber.java | 42 -
.../io/sstable/format/SSTableReader.java| 48 ++--
.../io/sstable/format/big/BigTableReader.java | 3 ++
.../cassandra/tools/StandaloneScrubber.java | 7 +--
.../unit/org/apache/cassandra/db/ScrubTest.java | 22 +
7 files changed, 106 insertions(+), 34 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8bb75a7/CHANGES.txt
--
diff --cc CHANGES.txt
index 53beb26,a863ad8..9dee57d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,19 -1,7 +1,22 @@@
-2.2.0-rc3
+3.0
+ * Storage engine refactor (CASSANDRA-8099, 9743)
+ * Update Guava to 18.0 (CASSANDRA-9653)
+ * Bloom filter false positive ratio is not honoured (CASSANDRA-8413)
+ * New option for cassandra-stress to leave a ratio of columns null
(CASSANDRA-9522)
+ * Change hinted_handoff_enabled yaml setting, JMX (CASSANDRA-9035)
+ * Add algorithmic token allocation (CASSANDRA-7032)
+ * Add nodetool command to replay batchlog (CASSANDRA-9547)
+ * Make file buffer cache independent of paths being read (CASSANDRA-8897)
+ * Remove deprecated legacy Hadoop code (CASSANDRA-9353)
+ * Decommissioned nodes will not rejoin the cluster (CASSANDRA-8801)
+ * Change gossip stabilization to use endpoit size (CASSANDRA-9401)
+ * Change default garbage collector to G1 (CASSANDRA-7486)
+ * Populate TokenMetadata early during startup (CASSANDRA-9317)
+ * undeprecate cache recentHitRate (CASSANDRA-6591)
+ * Add support for selectively varint encoding fields (CASSANDRA-9499)
+ Merged from 2.0:
+ * Scrub (recover) sstables even when -Index.db is missing (CASSANDRA-9591)
+
2.2.0-rc2
* Re-enable memory-mapped I/O on Windows (CASSANDRA-9658)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8bb75a7/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index a6c3d8c,4c94fa0..e3e9b03
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@@ -331,7 -332,7 +331,14 @@@ public class CompactionManager implemen
}
}
--public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final
boolean skipCorrupted, final boolean checkData) throws InterruptedException,
ExecutionException
++public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final
boolean skipCorrupted, final boolean checkData)
++throws InterruptedException, ExecutionException
++{
++return performScrub(cfs, skipCorrupted, checkData, false);
++}
++
++public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final
boolean skipCorrupted, final boolean checkData, final boolean offline)
++throws InterruptedException, ExecutionException
{
return parallelAllSSTableOperation(cfs, new OneSSTableOperation()
{
@@@ -344,7 -345,7 +351,7 @@@
@Override
public void execute(LifecycleTransaction input) throws IOException
{
--scrubOne(cfs, input, skipCorrupted, checkData);
++scrubOne(cfs, input, skipCorrupted, checkData, offline);
}
}, OperationType.SCRUB);
}
@@@ -691,11 -691,11 +698,11 @@@
}
}
--private void scrubOne(ColumnFamilyStore cfs, LifecycleTransaction
modifier, boolean skipCorrupted, boolean checkData) throws IOException
++private void scrubOne(ColumnFamilyStore cfs, LifecycleTransaction
modifier, boolean skipCorrupted, boolean checkData, boolean offline) throws
IOException
{
CompactionInfo.Holder scrubInfo = null;
--try (Scrubber scrubber = new Scrubber(cfs, modifier, skipCorrupted,
[jira] [Resolved] (CASSANDRA-9132) resumable_bootstrap_test can hang
[
https://issues.apache.org/jira/browse/CASSANDRA-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko resolved CASSANDRA-9132.
--
Resolution: Fixed
Fix Version/s: (was: 2.0.x)
2.0.15
resumable_bootstrap_test can hang
-
Key: CASSANDRA-9132
URL: https://issues.apache.org/jira/browse/CASSANDRA-9132
Project: Cassandra
Issue Type: Bug
Components: Tests
Reporter: Tyler Hobbs
Assignee: Yuki Morishita
Fix For: 2.2.0 rc1, 2.1.6, 2.0.15
Attachments: 9132-2.0.txt, 9132-followup-2.0.txt
The {{bootstrap_test.TestBootstrap.resumable_bootstrap_test}} can hang
sometimes. It looks like the following line never completes:
{noformat}
node3.watch_log_for(Listening for thrift clients...)
{noformat}
I'm not familiar enough with the recent bootstrap changes to know why that's
not happening.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9448) Metrics should use up to date nomenclature
[ https://issues.apache.org/jira/browse/CASSANDRA-9448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616885#comment-14616885 ] Nick Bailey commented on CASSANDRA-9448: This will certainly break some monitoring tools, OpsCenter for sure. If the only solution that allows deprecating the old metrics is to duplicate them, then perhaps that is too much overhead and the monitoring tools out there will just have to take that hit, but it would be great to avoid. I also don't know of a way to 'alias' things with the metrics library though. Metrics should use up to date nomenclature -- Key: CASSANDRA-9448 URL: https://issues.apache.org/jira/browse/CASSANDRA-9448 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Sam Tunnicliffe Assignee: Stefania Labels: docs-impacting, jmx Fix For: 3.0 beta 1 There are a number of exposed metrics that currently are named using the old nomenclature of columnfamily and rows (meaning partitions). It would be good to audit all metrics and update any names to match what they actually represent; we should probably do that in a single sweep to avoid a confusing mixture of old and new terminology. As we'd need to do this in a major release, I've initially set the fixver for 3.0 beta1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9444) resumable_bootstrap_test dtest failures
[
https://issues.apache.org/jira/browse/CASSANDRA-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616891#comment-14616891
]
Jim Witschey commented on CASSANDRA-9444:
-
Is this supposed to be closed? I think it was automatically marked as
Resolved since it duplicates CASSANDRA-9132, which has been closed.
resumable_bootstrap_test dtest failures
---
Key: CASSANDRA-9444
URL: https://issues.apache.org/jira/browse/CASSANDRA-9444
Project: Cassandra
Issue Type: Test
Components: Tests
Reporter: Tyler Hobbs
Assignee: Yuki Morishita
The {{bootstrap_test.TestBootstrap.resumable_bootstrap_test}} dtest is
experiencing occasional failures in cassci:
http://cassci.datastax.com/job/cassandra-2.2_dtest/lastCompletedBuild/testReport/bootstrap_test/TestBootstrap/resumable_bootstrap_test/
It looks like the problem is that after one of the streams fail, the
bootstrapping node never gets ready to accept client requests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9750) hashCode in UDFunction broken
Robert Stupp created CASSANDRA-9750:
---
Summary: hashCode in UDFunction broken
Key: CASSANDRA-9750
URL: https://issues.apache.org/jira/browse/CASSANDRA-9750
Project: Cassandra
Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Trivial
Fix For: 2.2.x
{{UDFunction.equals()}} uses the CQL3 type's string representation to compare
argument and return types - but {{UDFunction.hashCode()}} uses the hash of the
{{AbstractType}}s. That's inconsistent and can result in different hash codes
for the (functionally) same function.
Trivial to fix for post-2.2-rc2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9749) CommitLogReplayer continues startup after encountering errors
Blake Eggleston created CASSANDRA-9749: -- Summary: CommitLogReplayer continues startup after encountering errors Key: CASSANDRA-9749 URL: https://issues.apache.org/jira/browse/CASSANDRA-9749 Project: Cassandra Issue Type: Bug Reporter: Blake Eggleston Fix For: 2.2.x There are a few places where the commit log recovery method either skips sections or just returns when it encounters errors. Specifically if it can't read the header here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L298 Or if there are compressor problems here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L314 and here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L366 Whether these are user-fixable or not, I think we should require more direct user intervention (ie: fix what's wrong, or remove the bad file and restart) since we're basically losing data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9132) resumable_bootstrap_test can hang
[
https://issues.apache.org/jira/browse/CASSANDRA-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616880#comment-14616880
]
Aleksey Yeschenko commented on CASSANDRA-9132:
--
bq. I can open a new ticket if you think this is a new bug, but for the moment
I'm reopening.
JIRA-wise, you should not reopen tickets with code that's already in a released
C* version. Always open a new one.
resumable_bootstrap_test can hang
-
Key: CASSANDRA-9132
URL: https://issues.apache.org/jira/browse/CASSANDRA-9132
Project: Cassandra
Issue Type: Bug
Components: Tests
Reporter: Tyler Hobbs
Assignee: Yuki Morishita
Fix For: 2.1.6, 2.0.x, 2.2.0 rc1
Attachments: 9132-2.0.txt, 9132-followup-2.0.txt
The {{bootstrap_test.TestBootstrap.resumable_bootstrap_test}} can hang
sometimes. It looks like the following line never completes:
{noformat}
node3.watch_log_for(Listening for thrift clients...)
{noformat}
I'm not familiar enough with the recent bootstrap changes to know why that's
not happening.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616954#comment-14616954 ] Carl Yeksigian commented on CASSANDRA-6477: --- It should at the very least be a follow-up ticket; it deserves a discussion, including what the overhead will be, and whether it is worth it. Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8974) Need to update to latest dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-8974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616347#comment-14616347 ] Per Malmén commented on CASSANDRA-8974: --- Please consider moving to the new jackson coordinates com.fasterxml.jackson. The 1.x versions at org.codehaus.jackson are no longer actively maintained. https://github.com/FasterXML/jackson Need to update to latest dependencies - Key: CASSANDRA-8974 URL: https://issues.apache.org/jira/browse/CASSANDRA-8974 Project: Cassandra Issue Type: Improvement Components: Packaging Reporter: Joe Fasano Fix For: 3.x Open C* 3.0 to deal with upgrading all the dependencies. This is a general issue to update all dependencies. Specifically for example, I have been told by my team that some of the cassandra dependencies have some security vulnerabilities and should be upgraded. Joda Time 1.6 should be upgraded to 2.7 Jackson 1.9.2 should be upgraded to 1.9.13 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9743) RowUpdateBuilder is broken for maps and lists
[
https://issues.apache.org/jira/browse/CASSANDRA-9743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-9743:
Attachment: 9743.txt
{{PartitionUpdate}} was not properly merging/moving rows when sorting them post
updates. Attaching patch (that include the unit test). Note that I wouldn't
worry on the details of how this all work since CASSANDRA-9705 is going to
greatly simplify all this (it is arguably a bit over-complicated currently).
RowUpdateBuilder is broken for maps and lists
-
Key: CASSANDRA-9743
URL: https://issues.apache.org/jira/browse/CASSANDRA-9743
Project: Cassandra
Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Sylvain Lebresne
Fix For: 3.0 beta 1
Attachments: 9743.txt
Something's not right with {{RowUpdateBuilder}} when adding map or list
entries. It breaks {{SELECT}} queries that follow afterwards, inflating the
resultset, and duplicating the rows multiple times.
It manifests when a table has more than one collection, and we update them
both with more than one item.
Here is a gist with minimal reproducible code:
https://gist.github.com/iamaleksey/b6e9476fece029067729
The bug is currently blocking CASSANDRA-6717 code for migration from old
schema keyspace to a new one.
P.S. I'm assuming that the bug is in {{RowUpdateBuilder}} because I couldn't
reproduce in cqlsh. Could be that something's broken in the iterators, but
trying to debug a cascade of them failed me.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6710) Support union types
[
https://issues.apache.org/jira/browse/CASSANDRA-6710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616381#comment-14616381
]
Sylvain Lebresne commented on CASSANDRA-6710:
-
Frankly, I'd rather shelve that ticket for a while.
If you want to store multiple types in the same value, there is a very simple
workaround with relatively little downsides: use a {{blob}}. Pretty much all
drivers provide a simple way to serialize the existing C* types into blobs
(they kind of must support it internally anyway for bound statements) and CQL
even has a bunch of blob convertion functions (for when you're using cqlsh for
instance), so it's not even that inconvenient to use.
I'm not saying that a union type would be a bad thing per-se (the Approach 1
above since Approach 2 gives basically nothing over blobs outside of some
inefficiencies and I'm -1 on that), it has pros, but they are relatively small:
* some additional server side validation (that you only put something of one of
the type you declared). It's nice, but if it's server validation we're worried
about, then something like CASSANDRA-9745 (which won't require any driver
support) would give us that too (and more).
* small user convenience: you'll be able to do {{UPDATE foo SET v = 2 ...}}
instead of {{UPDATE foo SET v = intAsBlob(2) ...}}. Neat, but not a world of
difference either.
* possibly nice driver support for those languages that have unions natively.
Nice-to-have but far from all languages haves unions.
On the cons side however, on top of the server side work, this will require all
drivers to support this. And while it may not be the hardest thing ever, we're
already adding features that require driver support fairly quickly, which isn't
easy on all drivers maintainer. This is also yet more one feature to learn in
CQL for newcomers, making the language more complex at least in that sense. So,
given that the advantages feels to me rather small compared to the work it'll
requires of everyone to support it, I'd rather leave that on the back burner
for now.
Support union types
---
Key: CASSANDRA-6710
URL: https://issues.apache.org/jira/browse/CASSANDRA-6710
Project: Cassandra
Issue Type: Improvement
Components: API, Core
Reporter: Tupshin Harper
Priority: Minor
Labels: ponies
Fix For: 3.x
I sometimes find myself wanting to abuse Cassandra datatypes when I want to
interleave two different types in the same column.
An example is in CASSANDRA-6167 where an approach is to tag what would
normally be a numeric field with text indicating that it is special in some
ways.
A more elegant approach would be to be able to explicitly define disjoint
unions in the style of Haskell's and Scala's Either types.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9745) Allow validating values through a UDF
Sylvain Lebresne created CASSANDRA-9745:
---
Summary: Allow validating values through a UDF
Key: CASSANDRA-9745
URL: https://issues.apache.org/jira/browse/CASSANDRA-9745
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 3.x
The only server side validation we do for values is currently that they match
their type. Now that we have UDFs, we could allow to optionally specify a
(boolean) value validation function on a per-column basis. This would give us a
pretty generic way of validating value, which sounds useful to me.
Once we have that, we could even add some syntactic sugar for some common type
of validations, like {{v int[0..100]}} for numbers between 0 and 100, or {{v
text[20]}} for a string that is not longer than 20, ... That's more of a follow
up however.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9577) Cassandra not performing GC on stale SStables after compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-9577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616361#comment-14616361 ] Jeff Williams commented on CASSANDRA-9577: -- I am the user mentioned by Rob above. I have run 3 major compactions now and after each compaction it creates a new sstable, but leaves the old. About 5 hours after the compaction, JMX said I had 3 live sstables, though there were 14 in the data directory. After restarting cassandra it reported 8 live sstables, so some came back from the dead. Right now, JMX is reporting 10 sstables and there are 10 sstables in the data directory, but with 6 of them from before the last major compaction. 57372 was the result of the last compaction. $ ls -l /var/lib/cassandra/data/trackcontent/track_content/*Data.db -rw-r--r-- 2 cassandra cassandra 16642164891 Jun 29 06:55 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-30852-Data.db -rw-r--r-- 2 cassandra cassandra 17216513377 Jun 30 08:36 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-30866-Data.db -rw-r--r-- 2 cassandra cassandra 813683923 Jun 30 12:03 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-37136-Data.db -rw-r--r-- 2 cassandra cassandra 855070477 Jun 30 13:15 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-43371-Data.db -rw-r--r-- 2 cassandra cassandra 209921645 Jun 30 13:28 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-44676-Data.db -rw-r--r-- 2 cassandra cassandra 213532360 Jul 2 03:16 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-51845-Data.db -rw-r--r-- 2 cassandra cassandra 16916868412 Jul 6 13:34 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57372-Data.db -rw-r--r-- 2 cassandra cassandra 256127620 Jul 6 20:27 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57377-Data.db -rw-r--r-- 2 cassandra cassandra 8740917 Jul 7 02:10 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57378-Data.db -rw-r--r-- 2 cassandra cassandra 1371689 Jul 7 03:23 /var/lib/cassandra/data/trackcontent/track_content/trackcontent-track_content-jb-57379-Data.db How is this bad for consistency? I have been assuming (perhaps incorrectly) that reads will hit the new sstables first, so they should get the correct data without hitting the old sstables. I now have 3 copies of the dataset stored for this table on this node and am getting close to filling the disk (70%). I am happy to just remove the node, but I am wondering if you want me to try something first? I do not see any errors in the logs, except for a exception due to disk full when I tried a nodetool cleanup after adding a new node to the cluster. I am assuming that this is related. Cassandra not performing GC on stale SStables after compaction -- Key: CASSANDRA-9577 URL: https://issues.apache.org/jira/browse/CASSANDRA-9577 Project: Cassandra Issue Type: Bug Components: Core Environment: 2.0.12.200 / DSE 4.6.1. Reporter: Jeff Ferland Assignee: Marcus Eriksson Space used (live), bytes: 878681716067 Space used (total), bytes: 2227857083852 jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ sudo lsof *-Data.db COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java4473 cassandra 446r REG 0,26 17582559172 39241 trends-trends-jb-144864-Data.db java4473 cassandra 448r REG 0,26 62040962 37431 trends-trends-jb-144731-Data.db java4473 cassandra 449r REG 0,26 829935047545 21150 trends-trends-jb-143581-Data.db java4473 cassandra 452r REG 0,26 8980406 39503 trends-trends-jb-144882-Data.db java4473 cassandra 454r REG 0,26 8980406 39503 trends-trends-jb-144882-Data.db java4473 cassandra 462r REG 0,26 9487703 39542 trends-trends-jb-144883-Data.db java4473 cassandra 463r REG 0,26 36158226 39629 trends-trends-jb-144889-Data.db java4473 cassandra 468r REG 0,26105693505 39447 trends-trends-jb-144881-Data.db java4473 cassandra 530r REG 0,26 17582559172 39241 trends-trends-jb-144864-Data.db java4473 cassandra 535r REG 0,26105693505 39447 trends-trends-jb-144881-Data.db java4473 cassandra 542r REG 0,26 9487703 39542 trends-trends-jb-144883-Data.db java4473 cassandra 553u REG 0,26 6431729821 39556 trends-trends-tmp-jb-144884-Data.db jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ ls *-Data.db trends-trends-jb-142631-Data.db
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616396#comment-14616396
]
Stefania commented on CASSANDRA-9686:
-
Thanks - I'll make sure to apply the correct disk failure policy to this
exception and then submit a complete patch.
FSReadError and LEAK DETECTED after upgrading
-
Key: CASSANDRA-9686
URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
Reporter: Andreas Schnitzerling
Assignee: Stefania
Fix For: 2.2.x
Attachments: cassandra.bat, cassandra.yaml,
compactions_in_progress.zip, sstable_activity.zip, system.log
After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
LEAK DETECTED on start. Deleting the listed files, the failure goes away.
{code:title=system.log}
ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
with 0 chunks encountered: java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at
org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_55]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_55]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_55]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@1c42271
at
org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
... 15 common frames omitted
ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
was not released before the reference was garbage collected
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9745) Allow validating values through a UDF
[
https://issues.apache.org/jira/browse/CASSANDRA-9745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617129#comment-14617129
]
Jonathan Ellis commented on CASSANDRA-9745:
---
I think this should be part of a rewrite of triggers to use UDF. A trigger
already has the concept of vetoing an update; we should formalize this.
Allow validating values through a UDF
-
Key: CASSANDRA-9745
URL: https://issues.apache.org/jira/browse/CASSANDRA-9745
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 3.x
The only server side validation we do for values is currently that they match
their type. Now that we have UDFs, we could allow to optionally specify a
(boolean) value validation function on a per-column basis. This would give us
a pretty generic way of validating value, which sounds useful to me.
Once we have that, we could even add some syntactic sugar for some common
type of validations, like {{v int[0..100]}} for numbers between 0 and 100, or
{{v text[20]}} for a string that is not longer than 20, ... That's more of a
follow up however.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9591) Scrub (recover) sstables even when -Index.db is missing
[ https://issues.apache.org/jira/browse/CASSANDRA-9591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617156#comment-14617156 ] Benedict commented on CASSANDRA-9591: - Thanks. Committed. Could I ask that in future when merging up the branches, you leave the full merge commits in place? i.e. when constructing your branches, start with 2.0 (or whatever the lowest affected branch is), merge that into 2.1 (without --no-commit or --squash), and continue this all the way up to trunk. This is how patches must be committed into mainline, and helps committers with conflict resolutions when applying your patches. AFAICT you have created completely distinct branches all sourced from their mainline branches, and this means it is a lot more labour intensive for me to merge these upwards myself when (inevitably) one of the source branches diverges, as I have to construct my own versions of each, rebase them, perform diffs and patch applications. If the merge commits were in place, I could (in theory) use git rerere to resolve conflicts without any manual intervention, ensuring we do not break upstream due to my misapplication of your changes. Scrub (recover) sstables even when -Index.db is missing --- Key: CASSANDRA-9591 URL: https://issues.apache.org/jira/browse/CASSANDRA-9591 Project: Cassandra Issue Type: Improvement Reporter: mck Assignee: mck Labels: benedict-to-commit, sstablescrub Fix For: 2.0.x Attachments: 9591-2.0.txt, 9591-2.1.txt Today SSTableReader needs at minimum 3 files to load an sstable: - -Data.db - -CompressionInfo.db - -Index.db But during the scrub process the -Index.db file isn't actually necessary, unless there's corruption in the -Data.db and we want to be able to skip over corrupted rows. Given that there is still a fair chance that there's nothing wrong with the -Data.db file and we're just missing the -Index.db file this patch addresses that situation. So the following patch makes it possible for the StandaloneScrubber (sstablescrub) to recover sstables despite missing -Index.db files. This can happen from a catastrophic incident where data directories have been lost and/or corrupted, or wiped and the backup not healthy. I'm aware that normally one depends on replicas or snapshots to avoid such situations, but such catastrophic incidents do occur in the wild. I have not tested this patch against normal c* operations and all the other (more critical) ways SSTableReader is used. i'll happily do that and add the needed units tests if people see merit in accepting the patch. Otherwise the patch can live with the issue, in-case anyone else needs it. There's also a cassandra distribution bundled with the patch [here|https://github.com/michaelsembwever/cassandra/releases/download/2.0.15-recover-sstables-without-indexdb/apache-cassandra-2.0.15-recover-sstables-without-indexdb.tar.gz] to make life a little easier for anyone finding themselves in such a bad situation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6710) Support union types
[ https://issues.apache.org/jira/browse/CASSANDRA-6710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617151#comment-14617151 ] Jonathan Ellis commented on CASSANDRA-6710: --- there is a very simple workaround with relatively little downsides: use a blob Agreed. I'm just not sure this is worth adding extra complexity for little benefit. Support union types --- Key: CASSANDRA-6710 URL: https://issues.apache.org/jira/browse/CASSANDRA-6710 Project: Cassandra Issue Type: Improvement Components: API, Core Reporter: Tupshin Harper Priority: Minor Labels: ponies Fix For: 3.x I sometimes find myself wanting to abuse Cassandra datatypes when I want to interleave two different types in the same column. An example is in CASSANDRA-6167 where an approach is to tag what would normally be a numeric field with text indicating that it is special in some ways. A more elegant approach would be to be able to explicitly define disjoint unions in the style of Haskell's and Scala's Either types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9544) Allow specification of TLS protocol to use for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-9544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617194#comment-14617194
]
Jesse Szwedko commented on CASSANDRA-9544:
--
Makes sense, thanks for reviewing!
Allow specification of TLS protocol to use for cqlsh
Key: CASSANDRA-9544
URL: https://issues.apache.org/jira/browse/CASSANDRA-9544
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Jesse Szwedko
Labels: cqlsh, docs-impacting, tls
Currently when using {{cqlsh}} with {{--ssl}} it tries to use TLS 1.0 to
connect. I have my server only serving TLS 1.2 which means that I cannot
connect.
It would be nice if {{cqlsh}} allowed the TLS protocol it uses to connect to
be configured.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9662) compactionManager reporting wrong pendingtasks
[ https://issues.apache.org/jira/browse/CASSANDRA-9662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617141#comment-14617141 ] Benedict commented on CASSANDRA-9662: - Looking at the linked CASSANDRA-5554, it appears to only partially address this, by ensuring there is always at least one submission for any CF. However it still permits an abitrary number to be submitted, and with many concurrent compactors makes no effort to ensure remotely fair distribution. If we always submit _precisely_ one compaction, when requested, we should be in a much better state AFAICT. compactionManager reporting wrong pendingtasks -- Key: CASSANDRA-9662 URL: https://issues.apache.org/jira/browse/CASSANDRA-9662 Project: Cassandra Issue Type: Bug Components: API Environment: OS: Amazon Linux AMI release 2015.03 Cassandra: 2.0.16 JVM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25, mixed mode) Java: version 1.8.0_40, vendor Oracle Corporation CPU: 8 core Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Memory: 32G Reporter: Tony Xu Assignee: Yuki Morishita Priority: Minor Fix For: 2.0.x Attachments: node1.jpg Original Estimate: 168h Remaining Estimate: 168h Yesterday I upgraded my Cassandra cluster from 2.0.14 to 2.0.16, after upgrade, I am start seeing some strange behaviours of PendingTasks reporting. The Cassandra repository I am using is datastax, steps I performed for upgrade: yum update -y cassandra20 The upgrade went fine, after upgrade cluster is operating okay. nodetool info and nodetool status results looked fine. nodetool version is reporting the correct version. But our monitoring system start reporting some crazy pendingtasks. For example, pending taks for node1 sometimes jump from 0 to 15K for about 1 minute, then drop back to 0. This issue keeps occurring, didn't have this issue with 2.0.14. Our monitoring system is checking the value of MBeans - CompactionManager - PendingTasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9750) hashCode in UDFunction broken
[
https://issues.apache.org/jira/browse/CASSANDRA-9750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-9750:
--
Fix Version/s: (was: 2.2.x)
2.2.0
hashCode in UDFunction broken
-
Key: CASSANDRA-9750
URL: https://issues.apache.org/jira/browse/CASSANDRA-9750
Project: Cassandra
Issue Type: Bug
Reporter: Robert Stupp
Assignee: Robert Stupp
Priority: Trivial
Fix For: 3.0 beta 1, 2.2.0
{{UDFunction.equals()}} uses the CQL3 type's string representation to compare
argument and return types - but {{UDFunction.hashCode()}} uses the hash of
the {{AbstractType}}s. That's inconsistent and can result in different hash
codes for the (functionally) same function.
Trivial to fix for post-2.2-rc2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8894) Our default buffer size for (uncompressed) buffered reads should be smaller, and based on the expected record size
[
https://issues.apache.org/jira/browse/CASSANDRA-8894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617223#comment-14617223
]
Benedict commented on CASSANDRA-8894:
-
bq. the question is should we support more than one type (one per data
directory) or just keep it simple and have a global setting only?
That, and also I was referring to the chance calculations. i.e., the size
percentile on which to calculate the chance of crossing a page boundary, and
the threshold at which we use the resulting chance to add an extra page to the
read. These would be very helpful for some users to have access to, and also
for later performance tuning on our end, however I don't think it warrants
explicit mention in the yaml.
bq. The chance of crossing the page boundary should be calculated as the ratio
between
Well, that depends (if I'm interpreting your use of sigma correctly). Is a
normal distribution the correct assumption? Perhaps we could do better here,
and also reduce the number of knobs, by calculating the chance of every size
percentile (or decile, or some other quanta) crossing a page boundary, so that
we make no assumptions about the distribution. What I was suggesting above was
taking a high size percentile (95%, was my suggestion), and just assuming
everything is that size or smaller. I think we have to assume a uniform
distribution of _start position_ within a page, which for any gives its chance
of crossing a boundary straight-forwardly as {{(size % 4096) / 4096}}.
I don't think it matters _too_ much which strategy we use here, though. So long
as it's something exploiting this basic approach.
Our default buffer size for (uncompressed) buffered reads should be smaller,
and based on the expected record size
--
Key: CASSANDRA-8894
URL: https://issues.apache.org/jira/browse/CASSANDRA-8894
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Benedict
Assignee: Stefania
Labels: benedict-to-commit
Fix For: 3.x
A large contributor to slower buffered reads than mmapped is likely that we
read a full 64Kb at once, when average record sizes may be as low as 140
bytes on our stress tests. The TLB has only 128 entries on a modern core, and
each read will touch 32 of these, meaning we are unlikely to almost ever be
hitting the TLB, and will be incurring at least 30 unnecessary misses each
time (as well as the other costs of larger than necessary accesses). When
working with an SSD there is little to no benefit reading more than 4Kb at
once, and in either case reading more data than we need is wasteful. So, I
propose selecting a buffer size that is the next larger power of 2 than our
average record size (with a minimum of 4Kb), so that we expect to read in one
operation. I also propose that we create a pool of these buffers up-front,
and that we ensure they are all exactly aligned to a virtual page, so that
the source and target operations each touch exactly one virtual page per 4Kb
of expected record size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9749) CommitLogReplayer continues startup after encountering errors
[ https://issues.apache.org/jira/browse/CASSANDRA-9749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-9749: --- Fix Version/s: (was: 2.2.x) 2.2.0 rc2 CommitLogReplayer continues startup after encountering errors - Key: CASSANDRA-9749 URL: https://issues.apache.org/jira/browse/CASSANDRA-9749 Project: Cassandra Issue Type: Bug Reporter: Blake Eggleston Fix For: 2.2.0 rc2 There are a few places where the commit log recovery method either skips sections or just returns when it encounters errors. Specifically if it can't read the header here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L298 Or if there are compressor problems here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L314 and here: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L366 Whether these are user-fixable or not, I think we should require more direct user intervention (ie: fix what's wrong, or remove the bad file and restart) since we're basically losing data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-9751) Ability to change gc_grace period for system tables
Victor Coustenoble created CASSANDRA-9751: - Summary: Ability to change gc_grace period for system tables Key: CASSANDRA-9751 URL: https://issues.apache.org/jira/browse/CASSANDRA-9751 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Victor Coustenoble Priority: Minor For an admin user, or via JMX, give the possibility to change default gc_grace period for system keyspace tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-9744) unrealistic pending compactions displayed
[ https://issues.apache.org/jira/browse/CASSANDRA-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-9744. --- Resolution: Duplicate unrealistic pending compactions displayed - Key: CASSANDRA-9744 URL: https://issues.apache.org/jira/browse/CASSANDRA-9744 Project: Cassandra Issue Type: Bug Reporter: Christophe Attachments: compact.png Hi there, I am on 2.1.7. I am using DateTieredCompactionStrategy with the following parameters: min_threshold 16 max_threshold 16 base_time_seconds 300 max_stable_age_days 0.05 Apart from that, nothing exotic. I dump an sstable of about 18M every 12 sec I would say. Maybe 1 hour after creating my cluster, I can see that: 1. the number of SSTables has constantly been below 60. 2. Most of the time, I have between 1 and 4 pending compaction. 3. I occasionaly have a large number of pending compactions (30-40) for a short period of time (a few min) 4. I witness on 1 node 1610 pending compaction, on another node 367 pending compaction, for a short period of time (a few min) Item 4 makes me thing there is a bug as I don't see how I can have that many pending compaction with less than 50 sstable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9662) compactionManager reporting wrong pendingtasks
[
https://issues.apache.org/jira/browse/CASSANDRA-9662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617137#comment-14617137
]
Benedict commented on CASSANDRA-9662:
-
This patch looks like it would acceptably fix the bouncing report of compaction
tasks outstanding.
However I think we should step back for a moment and consider the context. The
reason this is bouncing around is because we attempt to impose a constraint in
{{submitBackground}} that we enforce very badly. This leads us to accidentally
submit perhaps thousands of no-op compactions. This patch masks that from the
user, but we're still doing it. This constraint being enforced _at all_ is very
questionable, though. It can lead us to starving entire tables from compaction
for extended periods, because they happen to flush at periods during which the
compaction executors are fully occupied. Ironically, CASSANDRA-9592 both
mitigates and worsens this problem, depending on if you're a secondary index or
not (for whom it makes things worse; everyone else it should prevent starvation
for)
So: how about we always submit exactly one background task for any table
whenever one is requested. That way the numbers will never bounce around so
wildly: they'll be capped at a variance of about 1 per table, per flush or per
minute. We can (and perhaps should) also introduce this patch, but it would be
comparatively much less of a problem, and we would have fixed a potential
problematic situation where tables get very far behind on compaction.
compactionManager reporting wrong pendingtasks
--
Key: CASSANDRA-9662
URL: https://issues.apache.org/jira/browse/CASSANDRA-9662
Project: Cassandra
Issue Type: Bug
Components: API
Environment: OS: Amazon Linux AMI release 2015.03
Cassandra: 2.0.16
JVM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25, mixed mode)
Java: version 1.8.0_40, vendor Oracle Corporation
CPU: 8 core Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Memory: 32G
Reporter: Tony Xu
Assignee: Yuki Morishita
Priority: Minor
Fix For: 2.0.x
Attachments: node1.jpg
Original Estimate: 168h
Remaining Estimate: 168h
Yesterday I upgraded my Cassandra cluster from 2.0.14 to 2.0.16, after
upgrade, I am start seeing some strange behaviours of PendingTasks
reporting.
The Cassandra repository I am using is datastax, steps I performed for
upgrade:
yum update -y cassandra20
The upgrade went fine, after upgrade cluster is operating okay. nodetool
info and nodetool status results looked fine. nodetool version is
reporting the correct version.
But our monitoring system start reporting some crazy pendingtasks. For
example, pending taks for node1 sometimes jump from 0 to 15K for about 1
minute, then drop back to 0. This issue keeps occurring, didn't have this
issue with 2.0.14. Our monitoring system is checking the value of MBeans -
CompactionManager - PendingTasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9102) Consistency levels such as non-local quorum need better tests
[ https://issues.apache.org/jira/browse/CASSANDRA-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617109#comment-14617109 ] Ariel Weisberg commented on CASSANDRA-9102: --- That coverage information is very helpful. Do you happen to have a hash for the version you generated the coverage for? I think a side channel like JMX or a property would work. Slightly more flexible might be to be able to annotate the request to indicate that the request wants some specific kind of failure injection. We already have support for tacking stuff onto requests for tracing. Maybe that can be adapted? I think completing the coverage of storage proxy is something should be scoped in a separate ticket. Consistency levels such as non-local quorum need better tests - Key: CASSANDRA-9102 URL: https://issues.apache.org/jira/browse/CASSANDRA-9102 Project: Cassandra Issue Type: Test Reporter: Ariel Weisberg Assignee: Stefania Attachments: jacoco.diff, jacoco.tar.gz We didn't catch unit testing for this functionality. There is dtest consistency_test but it doesn't cover non-local functionality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617191#comment-14617191 ] Carl Yeksigian commented on CASSANDRA-6477: --- That just ensures that the values which are currently in the base table will also be in the MV, but not that extraneous values present in the MV are removed. We can also repair the MV using the current repair process, which should fix the inconsistencies which streaming the updates to the MV would fix. Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617214#comment-14617214 ] Sebastian Estevez edited comment on CASSANDRA-6477 at 7/7/15 7:15 PM: -- i2's are a painpoint during bootstraps and repairs because they are rebuilt after streaming. Will this be the case for MV's as well? Ideally, they could be streamed as well. Much less intensive operation both CPU and IO wise. was (Author: sebastian.este...@datastax.com): i2's are a painpoint during bootstraps and repairs because they are rebuilt after streaming. Will this be the case for MV's as well? Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617214#comment-14617214 ] Sebastian Estevez commented on CASSANDRA-6477: -- i2's are a painpoint during bootstraps and repairs because they are rebuilt after streaming. Will this be the case for MV's as well? Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617221#comment-14617221
]
Joshua McKenzie commented on CASSANDRA-6477:
h6. Overall:
Needs comments. Class level, javadoc where appropriate, etc.
h6. {{CFMetaData}}
* duplicate entry for comparison if triggers added in CFMetadata.triggers
* nit: space after CFMetaData.getMaterializedViews function end
h6. {{MaterializedViewManager}}
* buildIfRequired: scope of whether it's required or skipped isn't in this
method or within view.build(), so it looks like it's just building (or at least
submitting the builder to the CompactionManager)
* You're using SystemKeyspace.setIndexRemoved to mark the MV removed in
removeMaterializedView but not the parallel setIndexBuilt to set them as built.
* allViews has unnecessary genericity
* nits:
** Some unused imports
** Consider renaming reload to indicate what it's reloading (e.g. reloadViews,
reloadChickens, reloadDolphins)
h6. {{MaterializedView}}
* createForDeletionInfo:
** Consider caching/precomputing the results of CFMetaData.hasComplexColumns
rather than iterating through them twice on each call (once on
CFMetaData.hascomplexColumns, again in for loop in createForDeletionInfo)
*** Perhaps hooking into addOrReplaceColumnDefinition, removeColumnDefinition
and updating MV's with the data, keep a set inside MaterializedView and
reference that
* Don't need MaterializedView.reload() as it's just a passthrough to build and
used in 1 place
* Tighten up scoping on member variables - some package private that can be
explicitly private
* ctor: Duplication in the ctor in treatment of MVDefinition partition and
clustering columns - could use a slight refactor to a function.
* createTombstone / createComplexTombstone: refactor out building
viewClusteringValues into method to remove duplication
* targetPartitionKey: collapse spacing on:
{noformat}
return viewCfs
.partitioner
.decorateKey(CFMetaData
.serializePartitionKey(viewCfs
.metadata
.getKeyValidatorAsClusteringComparator()
.make(partitionKey)));
{noformat}
to something like:
{noformat}
return
viewCfs.partitioner.decorateKey(CFMetaData.serializePartitionKey(viewCfs.metadata
.getKeyValidatorAsClusteringComparator()
.make(partitionKey)));
{noformat}
* createPartitionTombstonesForUpdates: Can simply return mutation @ end of
function
* createForDeletionInfo: unused parameter consistency
* Inconsistent naming - using mutationUnits scattered throughout vs.
LiveRowState
* Rename {{query}} to {{queryMaterializedViewData}} or something similar that
denotes what you're querying. Perhaps {{buildMVData}}
* nits:
** Some unused imports
** ctor: Change nonPrimaryKeyCol to allPrimaryKeyCol as true, flip to false
when non found so we're not assigning a double-negative to
targetHasAllPrimaryKeyColumns
** extra space in MaterializedView.createForDeletionInfo
** ctor declaration fits on 1 line 120 char, same for some other lines that
have been wrapped
** hte in javadoc for targetPartitionKey
h6. {{MaterializedViewDefinition}}
* Consider caching sets of columns in the MV rather than pulling from
CFMetaData and iterating. This would allow for faster lookup and simpler code
than having to iterate across all members (MaterializedViewDefinition.selects
for instance).
* Consider using Sets of columns rather than Lists internally, would remove a
lot of O(N) lookups and duplicate code logic scattered throughout the class
(selects, renameColumn, etc)
* selects(): if included.isEmpty() is apparently an indicator that it includes
all. We should 1) comment that and 2) wrap a method around that behavior, e.g.
'boolean isAllInclusive()' that documents that behavior in the code.
* nits:
** I prefer copy constructors to a .copy() method and I think I've seen us err
on the side of the copy constructor. I could be wrong here.
** assert included != null and !isEmpty on ctor
** extra whitespace in renameColumn @ top of method, @ end of method
h6. {{LiveRowState}}
* The name conflates with LivenessInfo (at least for me) which we discussed on
IRC. My biggest hurdle to grokking this class was un-plugging that and
re-plugging the idea that it's really about materializing the data for the
MaterializedView. Perhaps rename to something like {{TemporalRow}}, with
{{LRSCell}} becoming {{TemporalCell}}? While LRSCell doesn't really have any
temporal data above and beyond a general Cell, it does contain its own
{{reconcile}} that takes temporality into account which makes sense.
* Instead of {{addUnit}} / {{getExistingUnit}} in LiveRowState.Set, if going
with
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617178#comment-14617178 ] Jonathan Ellis commented on CASSANDRA-6477: --- Can't we wire up streamed updates to MV replication to make that unnecessary, the way we do with 2i? Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9662) compactionManager reporting wrong pendingtasks
[ https://issues.apache.org/jira/browse/CASSANDRA-9662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-9662: -- Reviewer: Benedict compactionManager reporting wrong pendingtasks -- Key: CASSANDRA-9662 URL: https://issues.apache.org/jira/browse/CASSANDRA-9662 Project: Cassandra Issue Type: Bug Components: API Environment: OS: Amazon Linux AMI release 2015.03 Cassandra: 2.0.16 JVM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25, mixed mode) Java: version 1.8.0_40, vendor Oracle Corporation CPU: 8 core Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz Memory: 32G Reporter: Tony Xu Assignee: Yuki Morishita Priority: Minor Fix For: 2.0.x Attachments: node1.jpg Original Estimate: 168h Remaining Estimate: 168h Yesterday I upgraded my Cassandra cluster from 2.0.14 to 2.0.16, after upgrade, I am start seeing some strange behaviours of PendingTasks reporting. The Cassandra repository I am using is datastax, steps I performed for upgrade: yum update -y cassandra20 The upgrade went fine, after upgrade cluster is operating okay. nodetool info and nodetool status results looked fine. nodetool version is reporting the correct version. But our monitoring system start reporting some crazy pendingtasks. For example, pending taks for node1 sometimes jump from 0 to 15K for about 1 minute, then drop back to 0. This issue keeps occurring, didn't have this issue with 2.0.14. Our monitoring system is checking the value of MBeans - CompactionManager - PendingTasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9544) Allow specification of TLS protocol to use for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-9544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-9544:
--
Labels: cqlsh docs-impacting tls (was: cqlsh doc docs-impacting tls)
Allow specification of TLS protocol to use for cqlsh
Key: CASSANDRA-9544
URL: https://issues.apache.org/jira/browse/CASSANDRA-9544
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Jesse Szwedko
Labels: cqlsh, docs-impacting, tls
Currently when using {{cqlsh}} with {{--ssl}} it tries to use TLS 1.0 to
connect. I have my server only serving TLS 1.2 which means that I cannot
connect.
It would be nice if {{cqlsh}} allowed the TLS protocol it uses to connect to
be configured.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9544) Allow specification of TLS protocol to use for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-9544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-9544:
--
Labels: cqlsh doc docs-impacting tls (was: cqlsh doc-impacting tls)
Allow specification of TLS protocol to use for cqlsh
Key: CASSANDRA-9544
URL: https://issues.apache.org/jira/browse/CASSANDRA-9544
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Jesse Szwedko
Labels: cqlsh, docs-impacting, tls
Currently when using {{cqlsh}} with {{--ssl}} it tries to use TLS 1.0 to
connect. I have my server only serving TLS 1.2 which means that I cannot
connect.
It would be nice if {{cqlsh}} allowed the TLS protocol it uses to connect to
be configured.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617214#comment-14617214 ] Sebastian Estevez edited comment on CASSANDRA-6477 at 7/7/15 7:16 PM: -- 2i's are a painpoint during bootstraps and repairs because they are rebuilt after streaming. Will this be the case for MV's as well? Ideally, they could be streamed as well. Much less intensive operation both CPU and IO wise. was (Author: sebastian.este...@datastax.com): i2's are a painpoint during bootstraps and repairs because they are rebuilt after streaming. Will this be the case for MV's as well? Ideally, they could be streamed as well. Much less intensive operation both CPU and IO wise. Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[03/12] cassandra git commit: Scrub (recover) sstables even when -Index.db is missing
Scrub (recover) sstables even when -Index.db is missing
patch by mck; reviewed by stefania for CASSANDRA-9591
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/452d6a44
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/452d6a44
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/452d6a44
Branch: refs/heads/cassandra-2.2
Commit: 452d6a445a6935d3e7d0e0fdf59e87d2a2ff95e7
Parents: 62714a9
Author: Stefania Alborghetti stefania.alborghe...@datastax.com
Authored: Mon Jun 15 16:49:03 2015 +0800
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 14:27:25 2015 +0100
--
CHANGES.txt | 4 ++
.../cassandra/db/compaction/Scrubber.java | 36 +++
.../cassandra/io/sstable/SSTableReader.java | 47
.../cassandra/tools/StandaloneScrubber.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 25 +++
5 files changed, 96 insertions(+), 18 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/452d6a44/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ca4d4b5..bd1db92 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,7 @@
+2.0.18
+* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
+
+
2.0.17
* Avoid NPE in AuthSuccess#decode (CASSANDRA-9727)
* Add listen_address to system.local (CASSANDRA-9603)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/452d6a44/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
index ea10855..dc60efa 100644
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@ -96,7 +96,16 @@ public class Scrubber implements Closeable
? new ScrubController(cfs)
: new CompactionController(cfs,
Collections.singleton(sstable), CompactionManager.getDefaultGcBefore(cfs));
this.isCommutative =
cfs.metadata.getDefaultValidator().isCommutative();
-this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)));
+
+boolean hasIndexFile = (new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX))).exists();
+if (!hasIndexFile)
+{
+// if there's any corruption in the -Data.db then rows can't be
skipped over. but it's worth a shot.
+outputHandler.warn(Missing component: +
sstable.descriptor.filenameFor(Component.PRIMARY_INDEX));
+}
+
+this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
+hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)) : 0);
// loop through each row, deserializing to check for damage.
// we'll also loop through the index at the same time, using the
position from the index to recover if the
@@ -105,7 +114,11 @@ public class Scrubber implements Closeable
this.dataFile = isOffline
? sstable.openDataReader()
:
sstable.openDataReader(CompactionManager.instance.getRateLimiter());
-this.indexFile = RandomAccessReader.open(new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)));
+
+this.indexFile = hasIndexFile
+? RandomAccessReader.open(new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)))
+: null;
+
this.scrubInfo = new ScrubInfo(dataFile, sstable);
this.currentRowPositionFromIndex = 0;
@@ -117,7 +130,8 @@ public class Scrubber implements Closeable
outputHandler.output(String.format(Scrubbing %s (%s bytes), sstable,
dataFile.length()));
try
{
-nextIndexKey = ByteBufferUtil.readWithShortLength(indexFile);
+nextIndexKey = indexAvailable() ?
ByteBufferUtil.readWithShortLength(indexFile) : null;
+if (indexAvailable())
{
// throw away variable so we don't have a side effect in the
assert
long firstRowPositionFromIndex =
RowIndexEntry.serializer.deserialize(indexFile,
sstable.descriptor.version).position;
@@ -181,7 +195,7 @@ public class Scrubber implements Closeable
outputHandler.debug(String.format(Index doublecheck:
row %s is %s bytes, ByteBufferUtil.bytesToHex(currentIndexKey),
dataSizeFromIndex));
}
-assert
[10/12] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ebe18bb2/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
index 0cc425a,000..247d181
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
@@@ -1,2218 -1,0 +1,2238 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.sstable.format;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+import java.util.*;
+import java.util.concurrent.*;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Predicate;
+import com.google.common.collect.Iterables;
+import com.google.common.collect.Iterators;
+import com.google.common.collect.Ordering;
+import com.google.common.primitives.Longs;
+import com.google.common.util.concurrent.RateLimiter;
+
+import com.clearspring.analytics.stream.cardinality.CardinalityMergeException;
+import com.clearspring.analytics.stream.cardinality.HyperLogLogPlus;
+import com.clearspring.analytics.stream.cardinality.ICardinality;
+import com.codahale.metrics.Counter;
+import org.apache.cassandra.cache.CachingOptions;
+import org.apache.cassandra.cache.InstrumentingCache;
+import org.apache.cassandra.cache.KeyCacheKey;
+import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor;
+import org.apache.cassandra.concurrent.ScheduledExecutors;
+import org.apache.cassandra.config.*;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
+import org.apache.cassandra.db.commitlog.ReplayPosition;
+import org.apache.cassandra.db.composites.CellName;
+import org.apache.cassandra.db.filter.ColumnSlice;
+import org.apache.cassandra.db.index.SecondaryIndex;
+import org.apache.cassandra.db.lifecycle.Tracker;
+import org.apache.cassandra.dht.*;
+import org.apache.cassandra.io.compress.CompressionMetadata;
+import org.apache.cassandra.io.sstable.*;
+import org.apache.cassandra.io.sstable.metadata.*;
+import org.apache.cassandra.io.util.*;
+import org.apache.cassandra.metrics.RestorableMeter;
+import org.apache.cassandra.metrics.StorageMetrics;
+import org.apache.cassandra.service.ActiveRepairService;
+import org.apache.cassandra.service.CacheService;
+import org.apache.cassandra.service.StorageService;
+import org.apache.cassandra.utils.*;
+import org.apache.cassandra.utils.concurrent.OpOrder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.cassandra.utils.concurrent.Ref;
+import org.apache.cassandra.utils.concurrent.SelfRefCounted;
+
+import static
org.apache.cassandra.db.Directories.SECONDARY_INDEX_NAME_SEPARATOR;
+
+/**
+ * An SSTableReader can be constructed in a number of places, but typically
is either
+ * read from disk at startup, or constructed from a flushed memtable, or
after compaction
+ * to replace some existing sstables. However once created, an sstablereader
may also be modified.
+ *
+ * A reader's OpenReason describes its current stage in its lifecycle, as
follows:
+ *
+ *
+ * pre {@code
+ * NORMAL
+ * From: None= Reader has been read from disk, either at
startup or from a flushed memtable
+ * EARLY = Reader is the final result of a compaction
+ * MOVED_START = Reader WAS being compacted, but this failed and
it has been restored to NORMAL status
+ *
+ * EARLY
+ * From: None= Reader is a compaction replacement that is
either incomplete and has been opened
+ *to represent its partial result status, or has
been finished but the compaction
+ *it is a part of has not yet completed fully
+ * EARLY = Same as from None, only it is not the first
time it has been
+ *
+ * MOVED_START
+ * From: NORMAL = Reader is being
[04/12] cassandra git commit: Scrub (recover) sstables even when -Index.db is missing
Scrub (recover) sstables even when -Index.db is missing
patch by mck; reviewed by stefania for CASSANDRA-9591
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/452d6a44
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/452d6a44
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/452d6a44
Branch: refs/heads/trunk
Commit: 452d6a445a6935d3e7d0e0fdf59e87d2a2ff95e7
Parents: 62714a9
Author: Stefania Alborghetti stefania.alborghe...@datastax.com
Authored: Mon Jun 15 16:49:03 2015 +0800
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 14:27:25 2015 +0100
--
CHANGES.txt | 4 ++
.../cassandra/db/compaction/Scrubber.java | 36 +++
.../cassandra/io/sstable/SSTableReader.java | 47
.../cassandra/tools/StandaloneScrubber.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 25 +++
5 files changed, 96 insertions(+), 18 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/452d6a44/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ca4d4b5..bd1db92 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,7 @@
+2.0.18
+* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
+
+
2.0.17
* Avoid NPE in AuthSuccess#decode (CASSANDRA-9727)
* Add listen_address to system.local (CASSANDRA-9603)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/452d6a44/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
index ea10855..dc60efa 100644
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@ -96,7 +96,16 @@ public class Scrubber implements Closeable
? new ScrubController(cfs)
: new CompactionController(cfs,
Collections.singleton(sstable), CompactionManager.getDefaultGcBefore(cfs));
this.isCommutative =
cfs.metadata.getDefaultValidator().isCommutative();
-this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)));
+
+boolean hasIndexFile = (new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX))).exists();
+if (!hasIndexFile)
+{
+// if there's any corruption in the -Data.db then rows can't be
skipped over. but it's worth a shot.
+outputHandler.warn(Missing component: +
sstable.descriptor.filenameFor(Component.PRIMARY_INDEX));
+}
+
+this.expectedBloomFilterSize =
Math.max(cfs.metadata.getIndexInterval(),
+hasIndexFile ?
(int)(SSTableReader.getApproximateKeyCount(toScrub,cfs.metadata)) : 0);
// loop through each row, deserializing to check for damage.
// we'll also loop through the index at the same time, using the
position from the index to recover if the
@@ -105,7 +114,11 @@ public class Scrubber implements Closeable
this.dataFile = isOffline
? sstable.openDataReader()
:
sstable.openDataReader(CompactionManager.instance.getRateLimiter());
-this.indexFile = RandomAccessReader.open(new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)));
+
+this.indexFile = hasIndexFile
+? RandomAccessReader.open(new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX)))
+: null;
+
this.scrubInfo = new ScrubInfo(dataFile, sstable);
this.currentRowPositionFromIndex = 0;
@@ -117,7 +130,8 @@ public class Scrubber implements Closeable
outputHandler.output(String.format(Scrubbing %s (%s bytes), sstable,
dataFile.length()));
try
{
-nextIndexKey = ByteBufferUtil.readWithShortLength(indexFile);
+nextIndexKey = indexAvailable() ?
ByteBufferUtil.readWithShortLength(indexFile) : null;
+if (indexAvailable())
{
// throw away variable so we don't have a side effect in the
assert
long firstRowPositionFromIndex =
RowIndexEntry.serializer.deserialize(indexFile,
sstable.descriptor.version).position;
@@ -181,7 +195,7 @@ public class Scrubber implements Closeable
outputHandler.debug(String.format(Index doublecheck:
row %s is %s bytes, ByteBufferUtil.bytesToHex(currentIndexKey),
dataSizeFromIndex));
}
-assert currentIndexKey !=
[09/12] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/Scrubber.java
src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ebe18bb2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ebe18bb2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ebe18bb2
Branch: refs/heads/cassandra-2.2
Commit: ebe18bb2ff62602fd5f55b969ecf665d2d3e5ace
Parents: 7d31068 4c94ef2
Author: Benedict Elliott Smith bened...@apache.org
Authored: Tue Jul 7 16:28:19 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 16:28:19 2015 +0100
--
CHANGES.txt | 5 ++
.../cassandra/db/compaction/Scrubber.java | 38 +++
.../io/sstable/format/SSTableReader.java| 50 ++--
.../cassandra/tools/StandaloneScrubber.java | 2 +-
.../unit/org/apache/cassandra/db/ScrubTest.java | 25 ++
5 files changed, 95 insertions(+), 25 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ebe18bb2/CHANGES.txt
--
diff --cc CHANGES.txt
index 7c6b4ad,2cbc7c4..a863ad8
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,28 -1,12 +1,33 @@@
-2.1.9
++2.2.0-rc3
+ Merged from 2.0:
- * Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
++ * Scrub (recover) sstables even when -Index.db is missing (CASSANDRA-9591)
+
+
-2.1.8
+2.2.0-rc2
+ * Re-enable memory-mapped I/O on Windows (CASSANDRA-9658)
+ * Warn when an extra-large partition is compacted (CASSANDRA-9643)
+ * (cqlsh) Allow setting the initial connection timeout (CASSANDRA-9601)
+ * BulkLoader has --transport-factory option but does not use it
(CASSANDRA-9675)
+ * Allow JMX over SSL directly from nodetool (CASSANDRA-9090)
+ * Update cqlsh for UDFs (CASSANDRA-7556)
+ * Change Windows kernel default timer resolution (CASSANDRA-9634)
+ * Deprected sstable2json and json2sstable (CASSANDRA-9618)
+ * Allow native functions in user-defined aggregates (CASSANDRA-9542)
+ * Don't repair system_distributed by default (CASSANDRA-9621)
+ * Fix mixing min, max, and count aggregates for blob type (CASSANRA-9622)
+ * Rename class for DATE type in Java driver (CASSANDRA-9563)
+ * Duplicate compilation of UDFs on coordinator (CASSANDRA-9475)
+ * Fix connection leak in CqlRecordWriter (CASSANDRA-9576)
+ * Mlockall before opening system sstables remove boot_without_jna option
(CASSANDRA-9573)
+ * Add functions to convert timeuuid to date or time, deprecate dateOf and
unixTimestampOf (CASSANDRA-9229)
+ * Make sure we cancel non-compacting sstables from LifecycleTransaction
(CASSANDRA-9566)
+ * Fix deprecated repair JMX API (CASSANDRA-9570)
+ * Add logback metrics (CASSANDRA-9378)
+ * Update and refactor ant test/test-compression to run the tests in parallel
(CASSANDRA-9583)
+ * Fix upgrading to new directory for secondary index (CASSANDRA-9687)
+Merged from 2.1:
* (cqlsh) Fix bad check for CQL compatibility when DESCRIBE'ing
COMPACT STORAGE tables with no clustering columns
- * Warn when an extra-large partition is compacted (CASSANDRA-9643)
* Eliminate strong self-reference chains in sstable ref tidiers
(CASSANDRA-9656)
* Ensure StreamSession uses canonical sstable reader instances
(CASSANDRA-9700)
* Ensure memtable book keeping is not corrupted in the event we shrink usage
(CASSANDRA-9681)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ebe18bb2/src/java/org/apache/cassandra/db/compaction/Scrubber.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/Scrubber.java
index 10952e7,b1c12e0..5a0b354
--- a/src/java/org/apache/cassandra/db/compaction/Scrubber.java
+++ b/src/java/org/apache/cassandra/db/compaction/Scrubber.java
@@@ -109,9 -101,17 +109,18 @@@ public class Scrubber implements Closea
? new ScrubController(cfs)
: new CompactionController(cfs,
Collections.singleton(sstable), CompactionManager.getDefaultGcBefore(cfs));
this.isCommutative = cfs.metadata.isCounter();
+
+ boolean hasIndexFile = (new
File(sstable.descriptor.filenameFor(Component.PRIMARY_INDEX))).exists();
+this.isIndex = cfs.isIndex();
+ if (!hasIndexFile)
+ {
+ // if there's any corruption in the -Data.db then rows can't be
skipped over. but it's worth a shot.
+ outputHandler.warn(Missing component: +
sstable.descriptor.filenameFor(Component.PRIMARY_INDEX));
+ }
-
+this.checkData = checkData
[3/3] cassandra git commit: Merge branch 'cassandra-2.2' into trunk
Merge branch 'cassandra-2.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9423109d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9423109d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9423109d Branch: refs/heads/trunk Commit: 9423109de2c11ff88f9ba3b04292428d7c53e7f4 Parents: a8bb75a 0f5dd22 Author: Benedict Elliott Smith bened...@apache.org Authored: Tue Jul 7 17:02:22 2015 +0100 Committer: Benedict Elliott Smith bened...@apache.org Committed: Tue Jul 7 17:02:22 2015 +0100 -- --
[1/3] cassandra git commit: Fix merge
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 ebe18bb2f - 0f5dd225d
refs/heads/trunk a8bb75a7e - 9423109de
Fix merge
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0f5dd225
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0f5dd225
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0f5dd225
Branch: refs/heads/cassandra-2.2
Commit: 0f5dd225de6ae995ddbc6d8d099260ed6eabd501
Parents: ebe18bb
Author: Benedict Elliott Smith bened...@apache.org
Authored: Tue Jul 7 16:48:45 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 17:01:33 2015 +0100
--
.../cassandra/db/compaction/CompactionManager.java | 15 +++
.../io/sstable/format/big/BigTableReader.java| 3 +++
.../apache/cassandra/tools/StandaloneScrubber.java | 5 +
test/unit/org/apache/cassandra/db/ScrubTest.java | 2 +-
4 files changed, 16 insertions(+), 9 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5dd225/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 4c94fa0..150b926 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -332,7 +332,14 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
-public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final
boolean skipCorrupted, final boolean checkData) throws InterruptedException,
ExecutionException
+public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final
boolean skipCorrupted, final boolean checkData)
+throws InterruptedException, ExecutionException
+{
+return performScrub(cfs, skipCorrupted, checkData, false);
+}
+
+public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final
boolean skipCorrupted, final boolean checkData, final boolean offline)
+throws InterruptedException, ExecutionException
{
return parallelAllSSTableOperation(cfs, new OneSSTableOperation()
{
@@ -345,7 +352,7 @@ public class CompactionManager implements
CompactionManagerMBean
@Override
public void execute(LifecycleTransaction input) throws IOException
{
-scrubOne(cfs, input, skipCorrupted, checkData);
+scrubOne(cfs, input, skipCorrupted, checkData, offline);
}
}, OperationType.SCRUB);
}
@@ -691,11 +698,11 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
-private void scrubOne(ColumnFamilyStore cfs, LifecycleTransaction
modifier, boolean skipCorrupted, boolean checkData) throws IOException
+private void scrubOne(ColumnFamilyStore cfs, LifecycleTransaction
modifier, boolean skipCorrupted, boolean checkData, boolean offline) throws
IOException
{
CompactionInfo.Holder scrubInfo = null;
-try (Scrubber scrubber = new Scrubber(cfs, modifier, skipCorrupted,
false, checkData))
+try (Scrubber scrubber = new Scrubber(cfs, modifier, skipCorrupted,
offline, checkData))
{
scrubInfo = scrubber.getScrubInfo();
metrics.beginCompaction(scrubInfo);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5dd225/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
b/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
index 3f375e7..f427389 100644
--- a/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
@@ -167,6 +167,9 @@ public class BigTableReader extends SSTableReader
int effectiveInterval =
indexSummary.getEffectiveIndexIntervalAfterIndex(sampledIndex);
+if (ifile == null)
+return null;
+
// scan the on-disk index, starting at the nearest sampled position.
// The check against IndexInterval is to be exit the loop in the EQ
case when the key looked for is not present
// (bloom filter false positive). But note that for non-EQ cases, we
might need to check the first key of the
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5dd225/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
[2/3] cassandra git commit: Fix merge
Fix merge
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0f5dd225
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0f5dd225
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0f5dd225
Branch: refs/heads/trunk
Commit: 0f5dd225de6ae995ddbc6d8d099260ed6eabd501
Parents: ebe18bb
Author: Benedict Elliott Smith bened...@apache.org
Authored: Tue Jul 7 16:48:45 2015 +0100
Committer: Benedict Elliott Smith bened...@apache.org
Committed: Tue Jul 7 17:01:33 2015 +0100
--
.../cassandra/db/compaction/CompactionManager.java | 15 +++
.../io/sstable/format/big/BigTableReader.java| 3 +++
.../apache/cassandra/tools/StandaloneScrubber.java | 5 +
test/unit/org/apache/cassandra/db/ScrubTest.java | 2 +-
4 files changed, 16 insertions(+), 9 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5dd225/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 4c94fa0..150b926 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -332,7 +332,14 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
-public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final
boolean skipCorrupted, final boolean checkData) throws InterruptedException,
ExecutionException
+public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final
boolean skipCorrupted, final boolean checkData)
+throws InterruptedException, ExecutionException
+{
+return performScrub(cfs, skipCorrupted, checkData, false);
+}
+
+public AllSSTableOpStatus performScrub(final ColumnFamilyStore cfs, final
boolean skipCorrupted, final boolean checkData, final boolean offline)
+throws InterruptedException, ExecutionException
{
return parallelAllSSTableOperation(cfs, new OneSSTableOperation()
{
@@ -345,7 +352,7 @@ public class CompactionManager implements
CompactionManagerMBean
@Override
public void execute(LifecycleTransaction input) throws IOException
{
-scrubOne(cfs, input, skipCorrupted, checkData);
+scrubOne(cfs, input, skipCorrupted, checkData, offline);
}
}, OperationType.SCRUB);
}
@@ -691,11 +698,11 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
-private void scrubOne(ColumnFamilyStore cfs, LifecycleTransaction
modifier, boolean skipCorrupted, boolean checkData) throws IOException
+private void scrubOne(ColumnFamilyStore cfs, LifecycleTransaction
modifier, boolean skipCorrupted, boolean checkData, boolean offline) throws
IOException
{
CompactionInfo.Holder scrubInfo = null;
-try (Scrubber scrubber = new Scrubber(cfs, modifier, skipCorrupted,
false, checkData))
+try (Scrubber scrubber = new Scrubber(cfs, modifier, skipCorrupted,
offline, checkData))
{
scrubInfo = scrubber.getScrubInfo();
metrics.beginCompaction(scrubInfo);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5dd225/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
b/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
index 3f375e7..f427389 100644
--- a/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
@@ -167,6 +167,9 @@ public class BigTableReader extends SSTableReader
int effectiveInterval =
indexSummary.getEffectiveIndexIntervalAfterIndex(sampledIndex);
+if (ifile == null)
+return null;
+
// scan the on-disk index, starting at the nearest sampled position.
// The check against IndexInterval is to be exit the loop in the EQ
case when the key looked for is not present
// (bloom filter false positive). But note that for non-EQ cases, we
might need to check the first key of the
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5dd225/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
--
diff --git a/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
[jira] [Updated] (CASSANDRA-9544) Allow specification of TLS protocol to use for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-9544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-9544:
---
Labels: cqlsh doc-impacting tls (was: cqlsh tls)
Allow specification of TLS protocol to use for cqlsh
Key: CASSANDRA-9544
URL: https://issues.apache.org/jira/browse/CASSANDRA-9544
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Jesse Szwedko
Labels: cqlsh, doc-impacting, tls
Currently when using {{cqlsh}} with {{--ssl}} it tries to use TLS 1.0 to
connect. I have my server only serving TLS 1.2 which means that I cannot
connect.
It would be nice if {{cqlsh}} allowed the TLS protocol it uses to connect to
be configured.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9751) Ability to change gc_grace period for system tables
[ https://issues.apache.org/jira/browse/CASSANDRA-9751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617343#comment-14617343 ] Aleksey Yeschenko commented on CASSANDRA-9751: -- Any tables in particular that you want to be able to alter? Ability to change gc_grace period for system tables --- Key: CASSANDRA-9751 URL: https://issues.apache.org/jira/browse/CASSANDRA-9751 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Victor Coustenoble Priority: Minor For an admin user, or via JMX, give the possibility to change default gc_grace period for system keyspace tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/2] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/12ff1cda Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/12ff1cda Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/12ff1cda Branch: refs/heads/cassandra-2.2 Commit: 12ff1cda7027f76c9b649f674d1e99459164a3fe Parents: 0f5dd22 30df089 Author: Tyler Hobbs tylerlho...@gmail.com Authored: Tue Jul 7 15:49:00 2015 -0500 Committer: Tyler Hobbs tylerlho...@gmail.com Committed: Tue Jul 7 15:49:00 2015 -0500 -- CHANGES.txt | 2 ++ pylib/cqlshlib/sslhandling.py | 15 +-- 2 files changed, 15 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/12ff1cda/CHANGES.txt -- diff --cc CHANGES.txt index a863ad8,0fbadbc..864eed2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,33 -1,14 +1,35 @@@ -2.1.9 +2.2.0-rc3 +Merged from 2.0: + * (cqlsh) Allow the SSL protocol version to be specified through the +config file or environment variables (CASSANDRA-9544) -Merged from 2.0: - * Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591) + * Scrub (recover) sstables even when -Index.db is missing (CASSANDRA-9591) -2.1.8 +2.2.0-rc2 + * Re-enable memory-mapped I/O on Windows (CASSANDRA-9658) + * Warn when an extra-large partition is compacted (CASSANDRA-9643) + * (cqlsh) Allow setting the initial connection timeout (CASSANDRA-9601) + * BulkLoader has --transport-factory option but does not use it (CASSANDRA-9675) + * Allow JMX over SSL directly from nodetool (CASSANDRA-9090) + * Update cqlsh for UDFs (CASSANDRA-7556) + * Change Windows kernel default timer resolution (CASSANDRA-9634) + * Deprected sstable2json and json2sstable (CASSANDRA-9618) + * Allow native functions in user-defined aggregates (CASSANDRA-9542) + * Don't repair system_distributed by default (CASSANDRA-9621) + * Fix mixing min, max, and count aggregates for blob type (CASSANRA-9622) + * Rename class for DATE type in Java driver (CASSANDRA-9563) + * Duplicate compilation of UDFs on coordinator (CASSANDRA-9475) + * Fix connection leak in CqlRecordWriter (CASSANDRA-9576) + * Mlockall before opening system sstables remove boot_without_jna option (CASSANDRA-9573) + * Add functions to convert timeuuid to date or time, deprecate dateOf and unixTimestampOf (CASSANDRA-9229) + * Make sure we cancel non-compacting sstables from LifecycleTransaction (CASSANDRA-9566) + * Fix deprecated repair JMX API (CASSANDRA-9570) + * Add logback metrics (CASSANDRA-9378) + * Update and refactor ant test/test-compression to run the tests in parallel (CASSANDRA-9583) + * Fix upgrading to new directory for secondary index (CASSANDRA-9687) +Merged from 2.1: * (cqlsh) Fix bad check for CQL compatibility when DESCRIBE'ing COMPACT STORAGE tables with no clustering columns - * Warn when an extra-large partition is compacted (CASSANDRA-9643) * Eliminate strong self-reference chains in sstable ref tidiers (CASSANDRA-9656) * Ensure StreamSession uses canonical sstable reader instances (CASSANDRA-9700) * Ensure memtable book keeping is not corrupted in the event we shrink usage (CASSANDRA-9681)
[2/3] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/12ff1cda Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/12ff1cda Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/12ff1cda Branch: refs/heads/trunk Commit: 12ff1cda7027f76c9b649f674d1e99459164a3fe Parents: 0f5dd22 30df089 Author: Tyler Hobbs tylerlho...@gmail.com Authored: Tue Jul 7 15:49:00 2015 -0500 Committer: Tyler Hobbs tylerlho...@gmail.com Committed: Tue Jul 7 15:49:00 2015 -0500 -- CHANGES.txt | 2 ++ pylib/cqlshlib/sslhandling.py | 15 +-- 2 files changed, 15 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/12ff1cda/CHANGES.txt -- diff --cc CHANGES.txt index a863ad8,0fbadbc..864eed2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,33 -1,14 +1,35 @@@ -2.1.9 +2.2.0-rc3 +Merged from 2.0: + * (cqlsh) Allow the SSL protocol version to be specified through the +config file or environment variables (CASSANDRA-9544) -Merged from 2.0: - * Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591) + * Scrub (recover) sstables even when -Index.db is missing (CASSANDRA-9591) -2.1.8 +2.2.0-rc2 + * Re-enable memory-mapped I/O on Windows (CASSANDRA-9658) + * Warn when an extra-large partition is compacted (CASSANDRA-9643) + * (cqlsh) Allow setting the initial connection timeout (CASSANDRA-9601) + * BulkLoader has --transport-factory option but does not use it (CASSANDRA-9675) + * Allow JMX over SSL directly from nodetool (CASSANDRA-9090) + * Update cqlsh for UDFs (CASSANDRA-7556) + * Change Windows kernel default timer resolution (CASSANDRA-9634) + * Deprected sstable2json and json2sstable (CASSANDRA-9618) + * Allow native functions in user-defined aggregates (CASSANDRA-9542) + * Don't repair system_distributed by default (CASSANDRA-9621) + * Fix mixing min, max, and count aggregates for blob type (CASSANRA-9622) + * Rename class for DATE type in Java driver (CASSANDRA-9563) + * Duplicate compilation of UDFs on coordinator (CASSANDRA-9475) + * Fix connection leak in CqlRecordWriter (CASSANDRA-9576) + * Mlockall before opening system sstables remove boot_without_jna option (CASSANDRA-9573) + * Add functions to convert timeuuid to date or time, deprecate dateOf and unixTimestampOf (CASSANDRA-9229) + * Make sure we cancel non-compacting sstables from LifecycleTransaction (CASSANDRA-9566) + * Fix deprecated repair JMX API (CASSANDRA-9570) + * Add logback metrics (CASSANDRA-9378) + * Update and refactor ant test/test-compression to run the tests in parallel (CASSANDRA-9583) + * Fix upgrading to new directory for secondary index (CASSANDRA-9687) +Merged from 2.1: * (cqlsh) Fix bad check for CQL compatibility when DESCRIBE'ing COMPACT STORAGE tables with no clustering columns - * Warn when an extra-large partition is compacted (CASSANDRA-9643) * Eliminate strong self-reference chains in sstable ref tidiers (CASSANDRA-9656) * Ensure StreamSession uses canonical sstable reader instances (CASSANDRA-9700) * Ensure memtable book keeping is not corrupted in the event we shrink usage (CASSANDRA-9681)
[1/2] cassandra git commit: cqlsh: Make SSL protocol version configurable
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 0f5dd225d - 12ff1cda7
cqlsh: Make SSL protocol version configurable
Patch by Jesse Szwedko; reviewed by Tyler Hobbs for CASSANDRA-9544
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30df089d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30df089d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30df089d
Branch: refs/heads/cassandra-2.2
Commit: 30df089d72d7d9889eebacd8c00537e46a2bcaab
Parents: 4c94ef2
Author: Jesse Szwedko jesse.szwe...@gmail.com
Authored: Tue Jul 7 12:12:49 2015 -0500
Committer: Tyler Hobbs tylerlho...@gmail.com
Committed: Tue Jul 7 15:47:57 2015 -0500
--
CHANGES.txt | 2 ++
pylib/cqlshlib/sslhandling.py | 15 +--
2 files changed, 15 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/30df089d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2cbc7c4..0fbadbc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.1.9
+ * (cqlsh) Allow the SSL protocol version to be specified through the
+ config file or environment variables (CASSANDRA-9544)
Merged from 2.0:
* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/30df089d/pylib/cqlshlib/sslhandling.py
--
diff --git a/pylib/cqlshlib/sslhandling.py b/pylib/cqlshlib/sslhandling.py
index 70dd759..2a90e26 100644
--- a/pylib/cqlshlib/sslhandling.py
+++ b/pylib/cqlshlib/sslhandling.py
@@ -19,6 +19,7 @@ import sys
import ConfigParser
import ssl
+
def ssl_settings(host, config_file, env=os.environ):
Function wcich generates SSL setting for cassandra.Cluster
@@ -51,6 +52,17 @@ def ssl_settings(host, config_file, env=os.environ):
ssl_validate = get_option('ssl', 'validate')
ssl_validate = ssl_validate is None or ssl_validate.lower() != 'false'
+ssl_version_str = env.get('SSL_VERSION')
+if ssl_version_str is None:
+ssl_version_str = get_option('ssl', 'version')
+if ssl_version_str is None:
+ssl_version_str = TLSv1
+
+ssl_version = getattr(ssl, PROTOCOL_%s % ssl_version_str, None)
+if ssl_version is None:
+sys.exit(%s is not a valid SSL protocol, please use one of SSLv23,
+ TLSv1, TLSv1.1, or TLSv1.2 % (ssl_version_str,))
+
ssl_certfile = env.get('SSL_CERTFILE')
if ssl_certfile is None:
ssl_certfile = get_option('certfiles', host)
@@ -73,6 +85,5 @@ def ssl_settings(host, config_file, env=os.environ):
return dict(ca_certs=ssl_certfile,
cert_reqs=ssl.CERT_REQUIRED if ssl_validate else ssl.CERT_NONE,
-ssl_version=ssl.PROTOCOL_TLSv1,
+ssl_version=ssl_version,
keyfile=userkey, certfile=usercert)
-
cassandra git commit: cqlsh: Make SSL protocol version configurable
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 4c94ef20d - 30df089d7
cqlsh: Make SSL protocol version configurable
Patch by Jesse Szwedko; reviewed by Tyler Hobbs for CASSANDRA-9544
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30df089d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30df089d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30df089d
Branch: refs/heads/cassandra-2.1
Commit: 30df089d72d7d9889eebacd8c00537e46a2bcaab
Parents: 4c94ef2
Author: Jesse Szwedko jesse.szwe...@gmail.com
Authored: Tue Jul 7 12:12:49 2015 -0500
Committer: Tyler Hobbs tylerlho...@gmail.com
Committed: Tue Jul 7 15:47:57 2015 -0500
--
CHANGES.txt | 2 ++
pylib/cqlshlib/sslhandling.py | 15 +--
2 files changed, 15 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/30df089d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2cbc7c4..0fbadbc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.1.9
+ * (cqlsh) Allow the SSL protocol version to be specified through the
+ config file or environment variables (CASSANDRA-9544)
Merged from 2.0:
* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/30df089d/pylib/cqlshlib/sslhandling.py
--
diff --git a/pylib/cqlshlib/sslhandling.py b/pylib/cqlshlib/sslhandling.py
index 70dd759..2a90e26 100644
--- a/pylib/cqlshlib/sslhandling.py
+++ b/pylib/cqlshlib/sslhandling.py
@@ -19,6 +19,7 @@ import sys
import ConfigParser
import ssl
+
def ssl_settings(host, config_file, env=os.environ):
Function wcich generates SSL setting for cassandra.Cluster
@@ -51,6 +52,17 @@ def ssl_settings(host, config_file, env=os.environ):
ssl_validate = get_option('ssl', 'validate')
ssl_validate = ssl_validate is None or ssl_validate.lower() != 'false'
+ssl_version_str = env.get('SSL_VERSION')
+if ssl_version_str is None:
+ssl_version_str = get_option('ssl', 'version')
+if ssl_version_str is None:
+ssl_version_str = TLSv1
+
+ssl_version = getattr(ssl, PROTOCOL_%s % ssl_version_str, None)
+if ssl_version is None:
+sys.exit(%s is not a valid SSL protocol, please use one of SSLv23,
+ TLSv1, TLSv1.1, or TLSv1.2 % (ssl_version_str,))
+
ssl_certfile = env.get('SSL_CERTFILE')
if ssl_certfile is None:
ssl_certfile = get_option('certfiles', host)
@@ -73,6 +85,5 @@ def ssl_settings(host, config_file, env=os.environ):
return dict(ca_certs=ssl_certfile,
cert_reqs=ssl.CERT_REQUIRED if ssl_validate else ssl.CERT_NONE,
-ssl_version=ssl.PROTOCOL_TLSv1,
+ssl_version=ssl_version,
keyfile=userkey, certfile=usercert)
-
[jira] [Commented] (CASSANDRA-9751) Ability to change gc_grace period for system tables
[ https://issues.apache.org/jira/browse/CASSANDRA-9751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617438#comment-14617438 ] Victor Coustenoble commented on CASSANDRA-9751: --- And you can add all tables that can contain tombstones, we want to avoid all warning messages in log files Ability to change gc_grace period for system tables --- Key: CASSANDRA-9751 URL: https://issues.apache.org/jira/browse/CASSANDRA-9751 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Victor Coustenoble Priority: Minor For an admin user, or via JMX, give the possibility to change default gc_grace period for system keyspace tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9751) Ability to change gc_grace period for system tables
[ https://issues.apache.org/jira/browse/CASSANDRA-9751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617444#comment-14617444 ] Aleksey Yeschenko commented on CASSANDRA-9751: -- The other important one is hints, I guess. But that one will be gone in 3.0 as well (see CASSANDRA-6230). Ability to change gc_grace period for system tables --- Key: CASSANDRA-9751 URL: https://issues.apache.org/jira/browse/CASSANDRA-9751 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Victor Coustenoble Priority: Minor For an admin user, or via JMX, give the possibility to change default gc_grace period for system keyspace tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9751) Ability to change gc_grace period for system tables
[ https://issues.apache.org/jira/browse/CASSANDRA-9751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617441#comment-14617441 ] Aleksey Yeschenko commented on CASSANDRA-9751: -- Right. Guessed so. Schema tables will be much less tombstone heavy after CASSANDRA-6717 (3.0), and there is a reason for gc gs to be set to what it's set. So this will really be a non-issue soon. Closing as Duplicate of that. Ability to change gc_grace period for system tables --- Key: CASSANDRA-9751 URL: https://issues.apache.org/jira/browse/CASSANDRA-9751 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Victor Coustenoble Priority: Minor For an admin user, or via JMX, give the possibility to change default gc_grace period for system keyspace tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9718) Upgrade to OHC 0.4
[ https://issues.apache.org/jira/browse/CASSANDRA-9718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617315#comment-14617315 ] Benedict commented on CASSANDRA-9718: - Shouldn't we now be able to remove those Forwarding input/output implementations? Upgrade to OHC 0.4 -- Key: CASSANDRA-9718 URL: https://issues.apache.org/jira/browse/CASSANDRA-9718 Project: Cassandra Issue Type: Improvement Reporter: Ariel Weisberg Assignee: Ariel Weisberg 0.4 provides a ByteBuffer for serialization/deserialization allowing more efficient encoding/decoding. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9718) Upgrade to OHC 0.4
[
https://issues.apache.org/jira/browse/CASSANDRA-9718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617338#comment-14617338
]
Benedict commented on CASSANDRA-9718:
-
No, I mean we already have a {{DataOutputPlus}}, so surely we don't need to
wrap it in {{new DataOutputPlus.DataOutputPlusAdapter(out)}}? Similarly for
{{DataInputPlus}}
Upgrade to OHC 0.4
--
Key: CASSANDRA-9718
URL: https://issues.apache.org/jira/browse/CASSANDRA-9718
Project: Cassandra
Issue Type: Improvement
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
0.4 provides a ByteBuffer for serialization/deserialization allowing more
efficient encoding/decoding.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9752) incremental repair test flaps on 2.2
Jim Witschey created CASSANDRA-9752:
---
Summary: incremental repair test flaps on 2.2
Key: CASSANDRA-9752
URL: https://issues.apache.org/jira/browse/CASSANDRA-9752
Project: Cassandra
Issue Type: Bug
Reporter: Jim Witschey
Assignee: Yuki Morishita
Fix For: 2.2.x
{{incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test}}
flaps on 2.2. It's hard to tell what failures are repair-specific, but there
are a few distinct failures I've seen recently:
- [an NPE in
StorageService|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/143/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
- [an NPE in
SSTableRewriter|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/135/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/].
I believe this is related to CASSANDRA-9730, but someone should confirm this.
- [an on-disk data size that is too
large|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/133/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
You can find the test itself [here on
GitHub|https://github.com/riptano/cassandra-dtest/blob/master/incremental_repair_test.py#L206]
and run it with the command
{code}
CASSANDRA_VERSION=git:trunk nosetests
incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test
{code}
Assigning [~yukim], since you're the repair person, but feel free to reassign
to whoever's appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9647) Tables created by cassandra-stress are omitted in DESCRIBE KEYSPACE
[
https://issues.apache.org/jira/browse/CASSANDRA-9647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617361#comment-14617361
]
Tyler Hobbs commented on CASSANDRA-9647:
For future reference, this was the version of the driver that was committed:
https://github.com/datastax/python-driver/commit/9f69d53af28cc81ed8511c2d1691eb6cc110126d
Tables created by cassandra-stress are omitted in DESCRIBE KEYSPACE
---
Key: CASSANDRA-9647
URL: https://issues.apache.org/jira/browse/CASSANDRA-9647
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Tyler Hobbs
Priority: Minor
Labels: cqlsh, stress
Fix For: 2.2.0 rc2
CASSANDRA-9374 modified cassandra-stress to only use CQL for creating its
schema. This seems to work, as I'm testing on a cluster with start_rpc:false.
However, when I try to run a DESCRIBE on the schema it omits the tables,
complaining that they were created with a legacy API:
{code}
cqlsh DESCRIBE KEYSPACE keyspace1 ;
CREATE KEYSPACE keyspace1 WITH replication = {'class': 'SimpleStrategy',
'replication_factor': '1'} AND durable_writes = true;
/*
Warning: Table keyspace1.counter1 omitted because it has constructs not
compatible with CQL (was created via legacy API).
Approximate structure, for reference:
(this should not be used to reproduce this schema)
CREATE TABLE keyspace1.counter1 (
key blob PRIMARY KEY,
C0 counter,
C1 counter,
C2 counter,
C3 counter,
C4 counter
) WITH COMPACT STORAGE
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
*/
/*
Warning: Table keyspace1.standard1 omitted because it has constructs not
compatible with CQL (was created via legacy API).
Approximate structure, for reference:
(this should not be used to reproduce this schema)
CREATE TABLE keyspace1.standard1 (
key blob PRIMARY KEY,
C0 blob,
C1 blob,
C2 blob,
C3 blob,
C4 blob
) WITH COMPACT STORAGE
AND bloom_filter_fp_chance = 0.01
AND caching = '{keys:ALL, rows_per_partition:NONE}'
AND comment = ''
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {}
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
*/
cqlsh
{code}
Note that it attempts to describe them anyway, but they are commented out and
shouldn't be used to restore from.
[This is the ccm workflow I used to test
this|https://gist.githubusercontent.com/EnigmaCurry/e779055c8debf6de8ef9/raw/a894e99725b6df599f3ce1db5012dd6d069b1339/gistfile1.txt]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-9751) Ability to change gc_grace period for system tables
[ https://issues.apache.org/jira/browse/CASSANDRA-9751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-9751. -- Resolution: Duplicate Ability to change gc_grace period for system tables --- Key: CASSANDRA-9751 URL: https://issues.apache.org/jira/browse/CASSANDRA-9751 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Victor Coustenoble Priority: Minor For an admin user, or via JMX, give the possibility to change default gc_grace period for system keyspace tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9752) incremental repair dtest flaps on 2.2
[
https://issues.apache.org/jira/browse/CASSANDRA-9752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Witschey updated CASSANDRA-9752:
Summary: incremental repair dtest flaps on 2.2 (was: incremental repair
test flaps on 2.2 )
incremental repair dtest flaps on 2.2
--
Key: CASSANDRA-9752
URL: https://issues.apache.org/jira/browse/CASSANDRA-9752
Project: Cassandra
Issue Type: Bug
Reporter: Jim Witschey
Assignee: Yuki Morishita
Fix For: 2.2.x
{{incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test}}
flaps on 2.2. It's hard to tell what failures are repair-specific, but there
are a few distinct failures I've seen recently:
- [an NPE in
StorageService|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/143/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
- [an NPE in
SSTableRewriter|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/135/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/].
I believe this is related to CASSANDRA-9730, but someone should confirm this.
- [an on-disk data size that is too
large|http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest/133/testReport/junit/incremental_repair_test/TestIncRepair/multiple_subsequent_repair_test/]
You can find the test itself [here on
GitHub|https://github.com/riptano/cassandra-dtest/blob/master/incremental_repair_test.py#L206]
and run it with the command
{code}
CASSANDRA_VERSION=git:trunk nosetests
incremental_repair_test.py:TestIncRepair.multiple_subsequent_repair_test
{code}
Assigning [~yukim], since you're the repair person, but feel free to reassign
to whoever's appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9742) Nodetool verify
[
https://issues.apache.org/jira/browse/CASSANDRA-9742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617318#comment-14617318
]
Aleksey Yeschenko commented on CASSANDRA-9742:
--
I think we should actually have two separate commands. One to check all
checksums, for anything that writes them (including 3.0 hint files), another
one for incremental. Marrying them in one feels unclean.
Nodetool verify
---
Key: CASSANDRA-9742
URL: https://issues.apache.org/jira/browse/CASSANDRA-9742
Project: Cassandra
Issue Type: New Feature
Components: Tools
Reporter: Jonathan Ellis
Fix For: 3.x
We introduced incremental repair in 2.1 but it is difficult to make that the
default without unpleasant surprises for incautious users.
Additionally, while we now store sstable checksums, we leave verification to
the user.
I propose introducing a new command, {{nodetool verify}}, that would address
both of these.
Default operation would be to do an incremental repair, plus validate
checksums on *all* sstables (not just unrepaired ones). We could also have
--local mode (checksums only) and --full (classic repair).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9718) Upgrade to OHC 0.4
[ https://issues.apache.org/jira/browse/CASSANDRA-9718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617335#comment-14617335 ] Ariel Weisberg commented on CASSANDRA-9718: --- Are you suggesting we push the usage of ByteBuffer down into all the dependent data types that have serialization written in terms of streams? I could try it but my guess is that it would reach pretty far. The alternative is creating parallel serialization for the dependent types that take a ByteBuffer. It looks like it eventually hits UnfilteredRowIterator which is shared for things like intra-cluster messaging which support compression. Upgrade to OHC 0.4 -- Key: CASSANDRA-9718 URL: https://issues.apache.org/jira/browse/CASSANDRA-9718 Project: Cassandra Issue Type: Improvement Reporter: Ariel Weisberg Assignee: Ariel Weisberg 0.4 provides a ByteBuffer for serialization/deserialization allowing more efficient encoding/decoding. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/3] cassandra git commit: cqlsh: Make SSL protocol version configurable
Repository: cassandra
Updated Branches:
refs/heads/trunk 9423109de - 6af030a95
cqlsh: Make SSL protocol version configurable
Patch by Jesse Szwedko; reviewed by Tyler Hobbs for CASSANDRA-9544
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/30df089d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/30df089d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/30df089d
Branch: refs/heads/trunk
Commit: 30df089d72d7d9889eebacd8c00537e46a2bcaab
Parents: 4c94ef2
Author: Jesse Szwedko jesse.szwe...@gmail.com
Authored: Tue Jul 7 12:12:49 2015 -0500
Committer: Tyler Hobbs tylerlho...@gmail.com
Committed: Tue Jul 7 15:47:57 2015 -0500
--
CHANGES.txt | 2 ++
pylib/cqlshlib/sslhandling.py | 15 +--
2 files changed, 15 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/30df089d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2cbc7c4..0fbadbc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.1.9
+ * (cqlsh) Allow the SSL protocol version to be specified through the
+ config file or environment variables (CASSANDRA-9544)
Merged from 2.0:
* Scrub (recover) sstables even when -Index.db is missing, (CASSANDRA-9591)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/30df089d/pylib/cqlshlib/sslhandling.py
--
diff --git a/pylib/cqlshlib/sslhandling.py b/pylib/cqlshlib/sslhandling.py
index 70dd759..2a90e26 100644
--- a/pylib/cqlshlib/sslhandling.py
+++ b/pylib/cqlshlib/sslhandling.py
@@ -19,6 +19,7 @@ import sys
import ConfigParser
import ssl
+
def ssl_settings(host, config_file, env=os.environ):
Function wcich generates SSL setting for cassandra.Cluster
@@ -51,6 +52,17 @@ def ssl_settings(host, config_file, env=os.environ):
ssl_validate = get_option('ssl', 'validate')
ssl_validate = ssl_validate is None or ssl_validate.lower() != 'false'
+ssl_version_str = env.get('SSL_VERSION')
+if ssl_version_str is None:
+ssl_version_str = get_option('ssl', 'version')
+if ssl_version_str is None:
+ssl_version_str = TLSv1
+
+ssl_version = getattr(ssl, PROTOCOL_%s % ssl_version_str, None)
+if ssl_version is None:
+sys.exit(%s is not a valid SSL protocol, please use one of SSLv23,
+ TLSv1, TLSv1.1, or TLSv1.2 % (ssl_version_str,))
+
ssl_certfile = env.get('SSL_CERTFILE')
if ssl_certfile is None:
ssl_certfile = get_option('certfiles', host)
@@ -73,6 +85,5 @@ def ssl_settings(host, config_file, env=os.environ):
return dict(ca_certs=ssl_certfile,
cert_reqs=ssl.CERT_REQUIRED if ssl_validate else ssl.CERT_NONE,
-ssl_version=ssl.PROTOCOL_TLSv1,
+ssl_version=ssl_version,
keyfile=userkey, certfile=usercert)
-
[3/3] cassandra git commit: Merge branch 'cassandra-2.2' into trunk
Merge branch 'cassandra-2.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6af030a9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6af030a9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6af030a9 Branch: refs/heads/trunk Commit: 6af030a9569d9cdd9c8f5715c9550b8ea30f1f54 Parents: 9423109 12ff1cd Author: Tyler Hobbs tylerlho...@gmail.com Authored: Tue Jul 7 15:49:29 2015 -0500 Committer: Tyler Hobbs tylerlho...@gmail.com Committed: Tue Jul 7 15:49:29 2015 -0500 -- CHANGES.txt | 2 ++ pylib/cqlshlib/sslhandling.py | 15 +-- 2 files changed, 15 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6af030a9/CHANGES.txt -- diff --cc CHANGES.txt index 9dee57d,864eed2..3b21b52 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,20 -1,7 +1,22 @@@ -2.2.0-rc3 +3.0 + * Storage engine refactor (CASSANDRA-8099, 9743) + * Update Guava to 18.0 (CASSANDRA-9653) + * Bloom filter false positive ratio is not honoured (CASSANDRA-8413) + * New option for cassandra-stress to leave a ratio of columns null (CASSANDRA-9522) + * Change hinted_handoff_enabled yaml setting, JMX (CASSANDRA-9035) + * Add algorithmic token allocation (CASSANDRA-7032) + * Add nodetool command to replay batchlog (CASSANDRA-9547) + * Make file buffer cache independent of paths being read (CASSANDRA-8897) + * Remove deprecated legacy Hadoop code (CASSANDRA-9353) + * Decommissioned nodes will not rejoin the cluster (CASSANDRA-8801) + * Change gossip stabilization to use endpoit size (CASSANDRA-9401) + * Change default garbage collector to G1 (CASSANDRA-7486) + * Populate TokenMetadata early during startup (CASSANDRA-9317) + * undeprecate cache recentHitRate (CASSANDRA-6591) + * Add support for selectively varint encoding fields (CASSANDRA-9499) Merged from 2.0: + * (cqlsh) Allow the SSL protocol version to be specified through the +config file or environment variables (CASSANDRA-9544) * Scrub (recover) sstables even when -Index.db is missing (CASSANDRA-9591)
[jira] [Commented] (CASSANDRA-9683) Get mucher higher load and latencies after upgrading from 2.1.6 to cassandra 2.1.7
[ https://issues.apache.org/jira/browse/CASSANDRA-9683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617426#comment-14617426 ] Ariel Weisberg commented on CASSANDRA-9683: --- I tried to reproduce this on one node and didn't have any luck. I started a node with 2.1.6, loaded 260 gigabytes of 8 megabyte rows along with 7 million small rows in a different column family. I upgraded to 2.1.7, and there was no difference in latency reported by stress. Get mucher higher load and latencies after upgrading from 2.1.6 to cassandra 2.1.7 -- Key: CASSANDRA-9683 URL: https://issues.apache.org/jira/browse/CASSANDRA-9683 Project: Cassandra Issue Type: Bug Environment: Ubuntu 12.04 (3.13 Kernel) * 3 JDK: Oracle JDK 7 RAM: 32GB Cores 4 (+4 HT) Reporter: Loic Lambiel Assignee: Ariel Weisberg Fix For: 2.1.x Attachments: cassandra.yaml, cfstats.txt, os_load.png, pending_compactions.png, read_latency.png, schema.txt, system.log, write_latency.png After upgrading our cassandra staging cluster version from 2.1.6 to 2.1.7, the average load grows from 0.1-0.3 to 1.8. Latencies did increase as well. We see an increase of pending compactions, probably due to CASSANDRA-9592. This cluster has almost no workload (staging environment) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616824#comment-14616824 ] Aleksey Yeschenko commented on CASSANDRA-6477: -- Do we have any performance (cstar) numbers yet? Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9672) Provide a per-table param that would force default ttl on all updates
[
https://issues.apache.org/jira/browse/CASSANDRA-9672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616827#comment-14616827
]
Paulo Motta commented on CASSANDRA-9672:
Would this allow changing the default ttl on a table and enforcing it on
previously inserted data? This would be a desireable feature, since it's quite
common to come up with a initial TTL value, and then decide to change it for
whatever reason. Currently, in order to enforce a new TTL you have to reinsert
all the data, which is a pain for very large datasets, and it's quite a common
use case.
Provide a per-table param that would force default ttl on all updates
-
Key: CASSANDRA-9672
URL: https://issues.apache.org/jira/browse/CASSANDRA-9672
Project: Cassandra
Issue Type: Improvement
Reporter: Aleksey Yeschenko
Priority: Minor
Many users have tables that rely on TTL entirely - no deletes, and only fixed
TTL value.
The way that default ttl works now, we only apply it if none is specified.
We should provide an option that would *enforce* the specified TTL. Not
allowing ttl-less {{INSERT}} or {{UPDATE}}, not allowing ttl that's lower or
higher than the default ttl, and not allowing deletes.
That option when enabled ({{force_default_ttl}}) should allow us to drop more
tables during compaction and do so cheaper. Would also allow the DBAs to
enforce the constraint in a guaranteed manner.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616832#comment-14616832 ] Alan Boudreault commented on CASSANDRA-6477: Is it related to the issue I found when adding a node? Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616843#comment-14616843 ] Alan Boudreault commented on CASSANDRA-6477: I haven't done any performance tests at the moment. [~carlyeks], have you done some? If not, I guess I can do that this week. Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616856#comment-14616856 ] Carl Yeksigian commented on CASSANDRA-6477: --- [~aboudreault]: This was an unrelated issue - it was related to keeping track whether the MV was built or not; I'm still looking at what is causing the inconsistency when a new node has been added. I have only seen it occur once in my testing, but it would be good to catch. [~iamaleksey]: I haven't done any cstar testing yet; have only done local stress. Would be good to run that, though I don't think we can run it through the web interface as we'll need to create the MV. /cc [~enigmacurry] Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616866#comment-14616866 ] Jack Krupansky commented on CASSANDRA-6477: --- 1. Are MV updates still eventually consistent (not guaranteed)? 2. Is there any way for the app to assure that the MV update have been completed to some desired CL? 3. Will a repair to the base table assure that all MV are consistent? 4. Can a single MV be repaired to assure that it is consistent? (Especially since the data for a MV on a node will be derived from data on other nodes due to differences in the partition keys.) Great to see such an exciting new feature take shape! Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9724) Aggregate appears to be causing query to be executed multiple times
[
https://issues.apache.org/jira/browse/CASSANDRA-9724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616869#comment-14616869
]
Tyler Hobbs commented on CASSANDRA-9724:
[~slebresne] it's not trivial, but it is possible. It should take a day or
less of work.
Aggregate appears to be causing query to be executed multiple times
---
Key: CASSANDRA-9724
URL: https://issues.apache.org/jira/browse/CASSANDRA-9724
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Christopher Batey
Assignee: Robert Stupp
Attachments: 9724.txt, data.zip
Not sure if this is intended behaviour.
Example table:
{noformat}
CREATE TABLE raw_weather_data (
wsid text, // Composite of Air Force Datsav3 station number
and NCDC WBAN number
year int,// Year collected
month int, // Month collected
day int, // Day collected
hour int,// Hour collected
temperature double, // Air temperature (degrees Celsius)
dewpoint double, // Dew point temperature (degrees Celsius)
pressure double, // Sea level pressure (hectopascals)
wind_direction int, // Wind direction in degrees. 0-359
wind_speed double,// Wind speed (meters per second)
sky_condition int, // Total cloud cover (coded, see format
documentation)
sky_condition_text text, // Non-coded sky conditions
one_hour_precip double, // One-hour accumulated liquid precipitation
(millimeters)
six_hour_precip double, // Six-hour accumulated liquid precipitation
(millimeters)
PRIMARY KEY ((wsid), year, month, day, hour)
) WITH CLUSTERING ORDER BY (year DESC, month DESC, day DESC, hour DESC);
{noformat}
1 node cluster 2.2rc1. Trace for: select temperature from raw_weather_data
where wsid = '725030:14732' and year = 2008;
{noformat}
activity
| timestamp | source
| source_elapsed
-++---+
Execute CQL3 query | 2015-07-03 09:53:25.002000 |
127.0.0.1 | 0
Parsing select temperature from raw_weather_data where wsid = '725030:14732'
and year = 2008; [SharedPool-Worker-1] | 2015-07-03 09:53:25.002000 |
127.0.0.1 |109
Preparing statement [SharedPool-Worker-1] | 2015-07-03 09:53:25.002000 |
127.0.0.1 |193
Executing single-partition query on
raw_weather_data [SharedPool-Worker-2] | 2015-07-03 09:53:25.002000 |
127.0.0.1 |519
Acquiring
sstable references [SharedPool-Worker-2] | 2015-07-03 09:53:25.002000 |
127.0.0.1 |544
Merging
memtable tombstones [SharedPool-Worker-2] | 2015-07-03 09:53:25.002000 |
127.0.0.1 |558
Skipped 0/0 non-slice-intersecting sstables, included 0
due to tombstones [SharedPool-Worker-2] | 2015-07-03 09:53:25.002001 |
127.0.0.1 |600
Merging data from
memtables and 0 sstables [SharedPool-Worker-2] | 2015-07-03 09:53:25.002001 |
127.0.0.1 |612
Read 92 live and
0 tombstone cells [SharedPool-Worker-2] | 2015-07-03 09:53:25.003000 |
127.0.0.1 |848
Request complete | 2015-07-03 09:53:25.003680 |
127.0.0.1 | 1680
{noformat}
However once i include the min function i get: select min(temperature) from
raw_weather_data where wsid = '725030:14732' and year = 2008;
{noformat}
activity
| timestamp |
source| source_elapsed
--++---+
Execute CQL3 query | 2015-07-03 09:56:15.904000 |
127.0.0.1 |
[jira] [Commented] (CASSANDRA-9742) Nodetool verify
[
https://issues.apache.org/jira/browse/CASSANDRA-9742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617696#comment-14617696
]
Jonathan Ellis commented on CASSANDRA-9742:
---
I thought about that, and I thought about it again after your comment, but I'm
still leaning towards one command.
My reasoning is, at a high level, what the op cares about is make sure this
node has the data on it, that it should have. Validating checksums is part of
that, but comparing with other replicas is too.
Nodetool verify
---
Key: CASSANDRA-9742
URL: https://issues.apache.org/jira/browse/CASSANDRA-9742
Project: Cassandra
Issue Type: New Feature
Components: Tools
Reporter: Jonathan Ellis
Fix For: 3.x
We introduced incremental repair in 2.1 but it is difficult to make that the
default without unpleasant surprises for incautious users.
Additionally, while we now store sstable checksums, we leave verification to
the user.
I propose introducing a new command, {{nodetool verify}}, that would address
both of these.
Default operation would be to do an incremental repair, plus validate
checksums on *all* sstables (not just unrepaired ones). We could also have
--local mode (checksums only) and --full (classic repair).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9742) Nodetool verify
[
https://issues.apache.org/jira/browse/CASSANDRA-9742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617697#comment-14617697
]
Aleksey Yeschenko commented on CASSANDRA-9742:
--
I still disagree, but not strongly enough to argue. Hints checksums weaken the
argument for a single command somewhat, but maybe we just want a separate
command for hints (or just not care about them at all for now).
Nodetool verify
---
Key: CASSANDRA-9742
URL: https://issues.apache.org/jira/browse/CASSANDRA-9742
Project: Cassandra
Issue Type: New Feature
Components: Tools
Reporter: Jonathan Ellis
Fix For: 3.x
We introduced incremental repair in 2.1 but it is difficult to make that the
default without unpleasant surprises for incautious users.
Additionally, while we now store sstable checksums, we leave verification to
the user.
I propose introducing a new command, {{nodetool verify}}, that would address
both of these.
Default operation would be to do an incremental repair, plus validate
checksums on *all* sstables (not just unrepaired ones). We could also have
--local mode (checksums only) and --full (classic repair).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9755) JMX auth driven by C* Auth
sankalp kohli created CASSANDRA-9755: Summary: JMX auth driven by C* Auth Key: CASSANDRA-9755 URL: https://issues.apache.org/jira/browse/CASSANDRA-9755 Project: Cassandra Issue Type: New Feature Reporter: sankalp kohli Priority: Minor Currently we have 2 auth schemes in C*. One is for JMX which could be file based and another is for CQL auth. Nodetool command are driven via JMX auth. It would be great to have both of these auth driven by C* Auth. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617622#comment-14617622 ] Joshua McKenzie commented on CASSANDRA-6477: bq. it's also nice to keep the discussion in Jira so that it's easier to understand past design decisions when you look at tickets I was thinking about that. We absolutely don't want to lose discussion history; I was thinking naming these types of branches ticketnum_review and having a policy of never deleting _review branches so they could be referenced for historical purposes. Not a particularly official / formal approach however with no safeguards other than behavior, and also dependent on github not archiving/removing old branches, space issues on there, etc. The difficulty of transposing review information into a jira comment and convenience is secondary to the fact that this approach takes my comments out of the immediate context of what I'm thinking, requiring a translation from me to here and from here back to whomever; my worry is that there's more chance of something getting lost in translation there. But yeah - dev ML seems the way to go for this topic. Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8224) Checksum Gossip state
[ https://issues.apache.org/jira/browse/CASSANDRA-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli updated CASSANDRA-8224: - Attachment: incomplete_trunk_8224.diff I am attaching this patch to give a sense of what this will do. The patch is not complete as it does not handle that other nodes can alter the state of other nodes when we call remove, etc. Checksum Gossip state - Key: CASSANDRA-8224 URL: https://issues.apache.org/jira/browse/CASSANDRA-8224 Project: Cassandra Issue Type: Improvement Reporter: sankalp kohli Assignee: sankalp kohli Priority: Minor Attachments: incomplete_trunk_8224.diff We have seen that a single machine with bad memory can corrupt the gossip of other nodes and cause entire cluster to be affected. If we store and pass the checksum of the entire state, we can detect corruption. If a bad machine tries to bump the generation number or other things, it will be detected and ignored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9754) Make index info heap friendly for large CQL partitions
[ https://issues.apache.org/jira/browse/CASSANDRA-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617679#comment-14617679 ] Jonathan Ellis commented on CASSANDRA-9754: --- But are most of those in the index summary (should be relatively stable once tenured) or the rowcache (high churn)? Make index info heap friendly for large CQL partitions -- Key: CASSANDRA-9754 URL: https://issues.apache.org/jira/browse/CASSANDRA-9754 Project: Cassandra Issue Type: Improvement Reporter: sankalp kohli Priority: Minor Looking at a heap dump of 2.0 cluster, I found that majority of the objects are IndexInfo and its ByteBuffers. This is specially bad in endpoints with large CQL partitions. If a CQL partition is say 6,4GB, it will have 100K IndexInfo objects and 200K ByteBuffers. This will create a lot of churn for GC. Can this be improved by not creating so many objects? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9591) Scrub (recover) sstables even when -Index.db is missing
[ https://issues.apache.org/jira/browse/CASSANDRA-9591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617691#comment-14617691 ] Stefania commented on CASSANDRA-9591: - Thanks for committing. I actually created the new branches and used rebase --onto. The mistake was also to squash all branches afterwards, thinking it would help to keep things tidier. My bad. I'll use merge in future and keep commits in place. Scrub (recover) sstables even when -Index.db is missing --- Key: CASSANDRA-9591 URL: https://issues.apache.org/jira/browse/CASSANDRA-9591 Project: Cassandra Issue Type: Improvement Reporter: mck Assignee: mck Labels: benedict-to-commit, sstablescrub Fix For: 2.0.x Attachments: 9591-2.0.txt, 9591-2.1.txt Today SSTableReader needs at minimum 3 files to load an sstable: - -Data.db - -CompressionInfo.db - -Index.db But during the scrub process the -Index.db file isn't actually necessary, unless there's corruption in the -Data.db and we want to be able to skip over corrupted rows. Given that there is still a fair chance that there's nothing wrong with the -Data.db file and we're just missing the -Index.db file this patch addresses that situation. So the following patch makes it possible for the StandaloneScrubber (sstablescrub) to recover sstables despite missing -Index.db files. This can happen from a catastrophic incident where data directories have been lost and/or corrupted, or wiped and the backup not healthy. I'm aware that normally one depends on replicas or snapshots to avoid such situations, but such catastrophic incidents do occur in the wild. I have not tested this patch against normal c* operations and all the other (more critical) ways SSTableReader is used. i'll happily do that and add the needed units tests if people see merit in accepting the patch. Otherwise the patch can live with the issue, in-case anyone else needs it. There's also a cassandra distribution bundled with the patch [here|https://github.com/michaelsembwever/cassandra/releases/download/2.0.15-recover-sstables-without-indexdb/apache-cassandra-2.0.15-recover-sstables-without-indexdb.tar.gz] to make life a little easier for anyone finding themselves in such a bad situation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6717) Modernize schema tables
[
https://issues.apache.org/jira/browse/CASSANDRA-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-6717:
-
Priority: Major (was: Minor)
Reviewer: Tyler Hobbs
Modernize schema tables
---
Key: CASSANDRA-6717
URL: https://issues.apache.org/jira/browse/CASSANDRA-6717
Project: Cassandra
Issue Type: Sub-task
Reporter: Sylvain Lebresne
Assignee: Aleksey Yeschenko
Labels: client-impacting
Fix For: 3.0 beta 1
There is a few problems/improvements that can be done with the way we store
schema:
# CASSANDRA-4988: as explained on the ticket, storing the comparator is now
redundant (or almost, we'd need to store whether the table is COMPACT or not
too, which we don't currently is easy and probably a good idea anyway), it
can be entirely reconstructed from the infos in schema_columns (the same is
true of key_validator and subcomparator, and replacing default_validator by a
COMPACT_VALUE column in all case is relatively simple). And storing the
comparator as an opaque string broke concurrent updates of sub-part of said
comparator (concurrent collection addition or altering 2 separate clustering
columns typically) so it's really worth removing it.
# CASSANDRA-4603: it's time to get rid of those ugly json maps. I'll note
that schema_keyspaces is a problem due to its use of COMPACT STORAGE, but I
think we should fix it once and for-all nonetheless (see below).
# For CASSANDRA-6382 and to allow indexing both map keys and values at the
same time, we'd need to be able to have more than one index definition for a
given column.
# There is a few mismatches in table options between the one stored in the
schema and the one used when declaring/altering a table which would be nice
to fix. The compaction, compression and replication maps are one already
mentioned from CASSANDRA-4603, but also for some reason
'dclocal_read_repair_chance' in CQL is called just 'local_read_repair_chance'
in the schema table, and 'min/max_compaction_threshold' are column families
option in the schema but just compaction options for CQL (which makes more
sense).
None of those issues are major, and we could probably deal with them
independently but it might be simpler to just fix them all in one shot so I
wanted to sum them all up here. In particular, the fact that
'schema_keyspaces' uses COMPACT STORAGE is annoying (for the replication map,
but it may limit future stuff too) which suggest we should migrate it to a
new, non COMPACT table. And while that's arguably a detail, it wouldn't hurt
to rename schema_columnfamilies to schema_tables for the years to come since
that's the prefered vernacular for CQL.
Overall, what I would suggest is to move all schema tables to a new keyspace,
named 'schema' for instance (or 'system_schema' but I prefer the shorter
version), and fix all the issues above at once. Since we currently don't
exchange schema between nodes of different versions, all we'd need to do that
is a one shot startup migration, and overall, I think it could be simpler for
clients to deal with one clear migration than to have to handle minor
individual changes all over the place. I also think it's somewhat cleaner
conceptually to have schema tables in their own keyspace since they are
replicated through a different mechanism than other system tables.
If we do that, we could, for instance, migrate to the following schema tables
(details up for discussion of course):
{noformat}
CREATE TYPE user_type (
name text,
column_names listtext,
column_types listtext
)
CREATE TABLE keyspaces (
name text PRIMARY KEY,
durable_writes boolean,
replication mapstring, string,
user_types mapstring, user_type
)
CREATE TYPE trigger_definition (
name text,
options maptex, text
)
CREATE TABLE tables (
keyspace text,
name text,
id uuid,
table_type text, // COMPACT, CQL or SUPER
dropped_columns maptext, bigint,
triggers maptext, trigger_definition,
// options
comment text,
compaction maptext, text,
compression maptext, text,
read_repair_chance double,
dclocal_read_repair_chance double,
gc_grace_seconds int,
caching text,
rows_per_partition_to_cache text,
default_time_to_live int,
min_index_interval int,
max_index_interval int,
speculative_retry text,
populate_io_cache_on_flush boolean,
bloom_filter_fp_chance double
memtable_flush_period_in_ms int,
PRIMARY KEY (keyspace, name)
)
CREATE TYPE index_definition (
name text,
index_type text,
options maptext, text
)
CREATE TABLE columns (
keyspace text,
table text,
name text,
kind text, // PARTITION_KEY, CLUSTERING_COLUMN, REGULAR or COMPACT_VALUE
component_index int;
type
[jira] [Commented] (CASSANDRA-9742) Nodetool verify
[
https://issues.apache.org/jira/browse/CASSANDRA-9742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617728#comment-14617728
]
Jonathan Ellis commented on CASSANDRA-9742:
---
But you'd still want to schedule a repair on hint checksum failure, right?
Just on target node instead of local.
Nodetool verify
---
Key: CASSANDRA-9742
URL: https://issues.apache.org/jira/browse/CASSANDRA-9742
Project: Cassandra
Issue Type: New Feature
Components: Tools
Reporter: Jonathan Ellis
Fix For: 3.x
We introduced incremental repair in 2.1 but it is difficult to make that the
default without unpleasant surprises for incautious users.
Additionally, while we now store sstable checksums, we leave verification to
the user.
I propose introducing a new command, {{nodetool verify}}, that would address
both of these.
Default operation would be to do an incremental repair, plus validate
checksums on *all* sstables (not just unrepaired ones). We could also have
--local mode (checksums only) and --full (classic repair).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9754) Make index info heap friendly for large CQL partitions
sankalp kohli created CASSANDRA-9754: Summary: Make index info heap friendly for large CQL partitions Key: CASSANDRA-9754 URL: https://issues.apache.org/jira/browse/CASSANDRA-9754 Project: Cassandra Issue Type: Improvement Reporter: sankalp kohli Priority: Minor Looking at a heap dump of 2.0 cluster, I found that majority of the objects are IndexInfo and its ByteBuffers. This is specially bad in endpoints with large CQL partitions. If a CQL partition is say 6,4GB, it will have 100K IndexInfo objects and 200K ByteBuffers. This will create a lot of churn for GC. Can this be improved by not creating so many objects? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-9742) Nodetool verify
[
https://issues.apache.org/jira/browse/CASSANDRA-9742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617696#comment-14617696
]
Jonathan Ellis edited comment on CASSANDRA-9742 at 7/8/15 12:07 AM:
My reasoning was, at a high level, that what the operator cares about is make
sure this node has the data on it, that it should have. Validating checksums
is part of that, but comparing with other replicas is too.
On the other hand, sstables corrupted from hardware problems are a lot more
rare than unreplicated data. So maybe insisting that we check for the former
every time we do the latter is not the right call.
was (Author: jbellis):
I thought about that, and I thought about it again after your comment, but I'm
still leaning towards one command.
My reasoning is, at a high level, what the op cares about is make sure this
node has the data on it, that it should have. Validating checksums is part of
that, but comparing with other replicas is too.
Nodetool verify
---
Key: CASSANDRA-9742
URL: https://issues.apache.org/jira/browse/CASSANDRA-9742
Project: Cassandra
Issue Type: New Feature
Components: Tools
Reporter: Jonathan Ellis
Fix For: 3.x
We introduced incremental repair in 2.1 but it is difficult to make that the
default without unpleasant surprises for incautious users.
Additionally, while we now store sstable checksums, we leave verification to
the user.
I propose introducing a new command, {{nodetool verify}}, that would address
both of these.
Default operation would be to do an incremental repair, plus validate
checksums on *all* sstables (not just unrepaired ones). We could also have
--local mode (checksums only) and --full (classic repair).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9448) Metrics should use up to date nomenclature
[ https://issues.apache.org/jira/browse/CASSANDRA-9448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617710#comment-14617710 ] Stefania commented on CASSANDRA-9448: - If it's important and yet no one is aware of an obvious way for duplicating metrics, then it's worth spending a few more hours trying to figure out if it is possible in a non painful way. I need to wrap up a couple of other tickets first, so give me a few more days. I will post another update then. Metrics should use up to date nomenclature -- Key: CASSANDRA-9448 URL: https://issues.apache.org/jira/browse/CASSANDRA-9448 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Sam Tunnicliffe Assignee: Stefania Labels: docs-impacting, jmx Fix For: 3.0 beta 1 There are a number of exposed metrics that currently are named using the old nomenclature of columnfamily and rows (meaning partitions). It would be good to audit all metrics and update any names to match what they actually represent; we should probably do that in a single sweep to avoid a confusing mixture of old and new terminology. As we'd need to do this in a major release, I've initially set the fixver for 3.0 beta1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617545#comment-14617545 ] Tyler Hobbs commented on CASSANDRA-6477: bq. I'm considering doing subsequent reviews as a branch I push to github w/comments inline (similar to what Benedict's been doing lately) with feedback as putting this quantity of feedback in Jira comments is a burden for both you and I - thoughts? We should probably discuss this on the dev ML, but here are my thoughts. It's way more convenient to comment on Github, _but_, it's also nice to keep the discussion in Jira so that it's easier to understand past design decisions when you look at tickets. Maybe there's a nice middle-ground? Materialized Views (was: Global Indexes) Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 beta 1 Attachments: test-view-data.sh Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8224) Checksum Gossip state
[ https://issues.apache.org/jira/browse/CASSANDRA-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617625#comment-14617625 ] sankalp kohli commented on CASSANDRA-8224: -- We can change the implementation to log when checksum does not match instead of ignoring the gossip. This way we can find the bad host. Checksum Gossip state - Key: CASSANDRA-8224 URL: https://issues.apache.org/jira/browse/CASSANDRA-8224 Project: Cassandra Issue Type: Improvement Reporter: sankalp kohli Assignee: sankalp kohli Priority: Minor Attachments: incomplete_trunk_8224.diff We have seen that a single machine with bad memory can corrupt the gossip of other nodes and cause entire cluster to be affected. If we store and pass the checksum of the entire state, we can detect corruption. If a bad machine tries to bump the generation number or other things, it will be detected and ignored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9601) Allow an initial connection timeout to be set in cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617717#comment-14617717 ] Stefania commented on CASSANDRA-9601: - Technically it should be possible. Anyone objects? Allow an initial connection timeout to be set in cqlsh -- Key: CASSANDRA-9601 URL: https://issues.apache.org/jira/browse/CASSANDRA-9601 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Mike Adamson Assignee: Stefania Labels: cqlsh Fix For: 2.2.0 rc2 [PYTHON-206|https://datastax-oss.atlassian.net/browse/PYTHON-206] introduced the ability to change the initial connection timeout on connections from the default of 5s. This change was introduced because some auth providers (kerberos) can take longer than 5s to complete a first time negotiation for a connection. cqlsh should allow this setting to be changed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6717) Modernize schema tables
[
https://issues.apache.org/jira/browse/CASSANDRA-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617726#comment-14617726
]
Aleksey Yeschenko commented on CASSANDRA-6717:
--
Pushed the first commit in the batch to
https://github.com/iamaleksey/cassandra/commits/6717-0
The first commit merely moves the existing tables, with mostly existing
structure, to a new system keyspace ({{system_schema}}). It doesn't contain any
of the deeper improvements whatsoever, and is not supposed to. One exception is
{{system_schema.keyspaces}} table - that one is no longer {{COMPACT STORAGE}}
or using JSON.
What it does contain is migration logic and tests ({{LegacySchemaMigrator}},
{{LegacySchemaMigratorTest}}), the upgraded drivers, and code throughout the
codebase modified to treat {{system_schema}} as a system table.
Things it's missing:
- tests for aggregates migration; this one is because loading aggregates is
broken in general at the moment, in trunk
- upgrade dtests; it includes a reasonable amount of upgrade utests, though,
and I've done some manual testing as well
What it breaks:
- the included python-driver lacks some recent trunk-driver PRs. I will have it
rebased before committing this
I would be grateful if Test Eng could spare some time and help me with writing
some extra upgrade dtests for this. I'm over-capacity at the moment.
No new utests are failing, but dtests needs a new python-driver to properly run.
Once this patch is in, I'll start incrementally adding the remaining patches.
Modernize schema tables
---
Key: CASSANDRA-6717
URL: https://issues.apache.org/jira/browse/CASSANDRA-6717
Project: Cassandra
Issue Type: Sub-task
Reporter: Sylvain Lebresne
Assignee: Aleksey Yeschenko
Priority: Minor
Labels: client-impacting
Fix For: 3.0 beta 1
There is a few problems/improvements that can be done with the way we store
schema:
# CASSANDRA-4988: as explained on the ticket, storing the comparator is now
redundant (or almost, we'd need to store whether the table is COMPACT or not
too, which we don't currently is easy and probably a good idea anyway), it
can be entirely reconstructed from the infos in schema_columns (the same is
true of key_validator and subcomparator, and replacing default_validator by a
COMPACT_VALUE column in all case is relatively simple). And storing the
comparator as an opaque string broke concurrent updates of sub-part of said
comparator (concurrent collection addition or altering 2 separate clustering
columns typically) so it's really worth removing it.
# CASSANDRA-4603: it's time to get rid of those ugly json maps. I'll note
that schema_keyspaces is a problem due to its use of COMPACT STORAGE, but I
think we should fix it once and for-all nonetheless (see below).
# For CASSANDRA-6382 and to allow indexing both map keys and values at the
same time, we'd need to be able to have more than one index definition for a
given column.
# There is a few mismatches in table options between the one stored in the
schema and the one used when declaring/altering a table which would be nice
to fix. The compaction, compression and replication maps are one already
mentioned from CASSANDRA-4603, but also for some reason
'dclocal_read_repair_chance' in CQL is called just 'local_read_repair_chance'
in the schema table, and 'min/max_compaction_threshold' are column families
option in the schema but just compaction options for CQL (which makes more
sense).
None of those issues are major, and we could probably deal with them
independently but it might be simpler to just fix them all in one shot so I
wanted to sum them all up here. In particular, the fact that
'schema_keyspaces' uses COMPACT STORAGE is annoying (for the replication map,
but it may limit future stuff too) which suggest we should migrate it to a
new, non COMPACT table. And while that's arguably a detail, it wouldn't hurt
to rename schema_columnfamilies to schema_tables for the years to come since
that's the prefered vernacular for CQL.
Overall, what I would suggest is to move all schema tables to a new keyspace,
named 'schema' for instance (or 'system_schema' but I prefer the shorter
version), and fix all the issues above at once. Since we currently don't
exchange schema between nodes of different versions, all we'd need to do that
is a one shot startup migration, and overall, I think it could be simpler for
clients to deal with one clear migration than to have to handle minor
individual changes all over the place. I also think it's somewhat cleaner
conceptually to have schema tables in their own keyspace since they are
replicated through a different mechanism than other system tables.
If we do that, we could, for
[jira] [Commented] (CASSANDRA-9577) Cassandra not performing GC on stale SStables after compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-9577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616612#comment-14616612 ] Jeff Williams commented on CASSANDRA-9577: -- We are also running on EC2. The data disk is a raid0 of the instance's ephemeral storage (2 x 40GB SSD) mounted as: /dev/md0 on /var/lib/cassandra type xfs (rw,_netdev). Yes, this is just happening on a single node. There are 8 other nodes in the cluster and only this one is showing this behaviour. Cassandra not performing GC on stale SStables after compaction -- Key: CASSANDRA-9577 URL: https://issues.apache.org/jira/browse/CASSANDRA-9577 Project: Cassandra Issue Type: Bug Components: Core Environment: 2.0.12.200 / DSE 4.6.1. Reporter: Jeff Ferland Assignee: Marcus Eriksson Space used (live), bytes: 878681716067 Space used (total), bytes: 2227857083852 jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ sudo lsof *-Data.db COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java4473 cassandra 446r REG 0,26 17582559172 39241 trends-trends-jb-144864-Data.db java4473 cassandra 448r REG 0,26 62040962 37431 trends-trends-jb-144731-Data.db java4473 cassandra 449r REG 0,26 829935047545 21150 trends-trends-jb-143581-Data.db java4473 cassandra 452r REG 0,26 8980406 39503 trends-trends-jb-144882-Data.db java4473 cassandra 454r REG 0,26 8980406 39503 trends-trends-jb-144882-Data.db java4473 cassandra 462r REG 0,26 9487703 39542 trends-trends-jb-144883-Data.db java4473 cassandra 463r REG 0,26 36158226 39629 trends-trends-jb-144889-Data.db java4473 cassandra 468r REG 0,26105693505 39447 trends-trends-jb-144881-Data.db java4473 cassandra 530r REG 0,26 17582559172 39241 trends-trends-jb-144864-Data.db java4473 cassandra 535r REG 0,26105693505 39447 trends-trends-jb-144881-Data.db java4473 cassandra 542r REG 0,26 9487703 39542 trends-trends-jb-144883-Data.db java4473 cassandra 553u REG 0,26 6431729821 39556 trends-trends-tmp-jb-144884-Data.db jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ ls *-Data.db trends-trends-jb-142631-Data.db trends-trends-jb-143562-Data.db trends-trends-jb-143581-Data.db trends-trends-jb-144731-Data.db trends-trends-jb-144883-Data.db trends-trends-jb-142633-Data.db trends-trends-jb-143563-Data.db trends-trends-jb-144530-Data.db trends-trends-jb-144864-Data.db trends-trends-jb-144889-Data.db trends-trends-jb-143026-Data.db trends-trends-jb-143564-Data.db trends-trends-jb-144551-Data.db trends-trends-jb-144881-Data.db trends-trends-tmp-jb-144884-Data.db trends-trends-jb-143533-Data.db trends-trends-jb-143578-Data.db trends-trends-jb-144552-Data.db trends-trends-jb-144882-Data.db jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ cd - /mnt/cassandra/data/trends/trends jbf@ip-10-0-2-98:/mnt/cassandra/data/trends/trends$ sudo lsof * jbf@ip-10-0-2-98:/mnt/cassandra/data/trends/trends$ ls *-Data.db trends-trends-jb-124502-Data.db trends-trends-jb-141113-Data.db trends-trends-jb-141377-Data.db trends-trends-jb-141846-Data.db trends-trends-jb-144890-Data.db trends-trends-jb-125457-Data.db trends-trends-jb-141123-Data.db trends-trends-jb-141391-Data.db trends-trends-jb-141871-Data.db trends-trends-jb-41121-Data.db trends-trends-jb-130016-Data.db trends-trends-jb-141137-Data.db trends-trends-jb-141538-Data.db trends-trends-jb-141883-Data.db trends-trends.trends_date_idx-jb-2100-Data.db trends-trends-jb-139563-Data.db trends-trends-jb-141358-Data.db trends-trends-jb-141806-Data.db trends-trends-jb-142033-Data.db trends-trends-jb-141102-Data.db trends-trends-jb-141363-Data.db trends-trends-jb-141829-Data.db trends-trends-jb-144553-Data.db Compaction started INFO [CompactionExecutor:6661] 2015-06-05 14:02:36,515 CompactionTask.java (line 120) Compacting [SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-124502-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141358-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141883-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141846-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141871-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141391-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-139563-Data.db'),
[jira] [Commented] (CASSANDRA-9577) Cassandra not performing GC on stale SStables after compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-9577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14616615#comment-14616615 ] Marcus Eriksson commented on CASSANDRA-9577: Extremely strange, nothing different about this node compared to the others? Kernel version, file system, errors in syslog etc? Cassandra not performing GC on stale SStables after compaction -- Key: CASSANDRA-9577 URL: https://issues.apache.org/jira/browse/CASSANDRA-9577 Project: Cassandra Issue Type: Bug Components: Core Environment: 2.0.12.200 / DSE 4.6.1. Reporter: Jeff Ferland Assignee: Marcus Eriksson Space used (live), bytes: 878681716067 Space used (total), bytes: 2227857083852 jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ sudo lsof *-Data.db COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME java4473 cassandra 446r REG 0,26 17582559172 39241 trends-trends-jb-144864-Data.db java4473 cassandra 448r REG 0,26 62040962 37431 trends-trends-jb-144731-Data.db java4473 cassandra 449r REG 0,26 829935047545 21150 trends-trends-jb-143581-Data.db java4473 cassandra 452r REG 0,26 8980406 39503 trends-trends-jb-144882-Data.db java4473 cassandra 454r REG 0,26 8980406 39503 trends-trends-jb-144882-Data.db java4473 cassandra 462r REG 0,26 9487703 39542 trends-trends-jb-144883-Data.db java4473 cassandra 463r REG 0,26 36158226 39629 trends-trends-jb-144889-Data.db java4473 cassandra 468r REG 0,26105693505 39447 trends-trends-jb-144881-Data.db java4473 cassandra 530r REG 0,26 17582559172 39241 trends-trends-jb-144864-Data.db java4473 cassandra 535r REG 0,26105693505 39447 trends-trends-jb-144881-Data.db java4473 cassandra 542r REG 0,26 9487703 39542 trends-trends-jb-144883-Data.db java4473 cassandra 553u REG 0,26 6431729821 39556 trends-trends-tmp-jb-144884-Data.db jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ ls *-Data.db trends-trends-jb-142631-Data.db trends-trends-jb-143562-Data.db trends-trends-jb-143581-Data.db trends-trends-jb-144731-Data.db trends-trends-jb-144883-Data.db trends-trends-jb-142633-Data.db trends-trends-jb-143563-Data.db trends-trends-jb-144530-Data.db trends-trends-jb-144864-Data.db trends-trends-jb-144889-Data.db trends-trends-jb-143026-Data.db trends-trends-jb-143564-Data.db trends-trends-jb-144551-Data.db trends-trends-jb-144881-Data.db trends-trends-tmp-jb-144884-Data.db trends-trends-jb-143533-Data.db trends-trends-jb-143578-Data.db trends-trends-jb-144552-Data.db trends-trends-jb-144882-Data.db jbf@ip-10-0-2-98:/ebs/cassandra/data/trends/trends$ cd - /mnt/cassandra/data/trends/trends jbf@ip-10-0-2-98:/mnt/cassandra/data/trends/trends$ sudo lsof * jbf@ip-10-0-2-98:/mnt/cassandra/data/trends/trends$ ls *-Data.db trends-trends-jb-124502-Data.db trends-trends-jb-141113-Data.db trends-trends-jb-141377-Data.db trends-trends-jb-141846-Data.db trends-trends-jb-144890-Data.db trends-trends-jb-125457-Data.db trends-trends-jb-141123-Data.db trends-trends-jb-141391-Data.db trends-trends-jb-141871-Data.db trends-trends-jb-41121-Data.db trends-trends-jb-130016-Data.db trends-trends-jb-141137-Data.db trends-trends-jb-141538-Data.db trends-trends-jb-141883-Data.db trends-trends.trends_date_idx-jb-2100-Data.db trends-trends-jb-139563-Data.db trends-trends-jb-141358-Data.db trends-trends-jb-141806-Data.db trends-trends-jb-142033-Data.db trends-trends-jb-141102-Data.db trends-trends-jb-141363-Data.db trends-trends-jb-141829-Data.db trends-trends-jb-144553-Data.db Compaction started INFO [CompactionExecutor:6661] 2015-06-05 14:02:36,515 CompactionTask.java (line 120) Compacting [SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-124502-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141358-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141883-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141846-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141871-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141391-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-139563-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-125457-Data.db'), SSTableReader(path='/mnt/cassandra/data/trends/trends/trends-trends-jb-141806-Data.db'),
[jira] [Updated] (CASSANDRA-9746) Querying sometimes broken after non-durable writes
[
https://issues.apache.org/jira/browse/CASSANDRA-9746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-9746:
Attachment: 9746.txt
Simple fix attached (the {{PartitionUpdate.rowCount()}} method was improperly
protected).
Querying sometimes broken after non-durable writes
--
Key: CASSANDRA-9746
URL: https://issues.apache.org/jira/browse/CASSANDRA-9746
Project: Cassandra
Issue Type: Bug
Reporter: Aleksey Yeschenko
Assignee: Sylvain Lebresne
Fix For: 3.0 beta 1
Attachments: 9746.txt
Depending on whether or not {{Mutation}}'s {{PartitionUpdate}}s get iterated
over before applying (iterated manually, or by having the commit log iterate
over them for serialization), the subsequent reads may fail.
A follow up to CASSANDRA-9743, reusing the same (but properly named) test
now: https://gist.github.com/iamaleksey/ed55a1f38a8869c2e229
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
