[jira] [Comment Edited] (CASSANDRA-11053) COPY FROM on large datasets: fix progress report and debug performance

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173201#comment-15173201
 ] 

Stefania edited comment on CASSANDRA-11053 at 3/3/16 6:03 AM:
--

Thank you for your latest comments and for introducing 
{{deserializers.DesBytesTypeByteArray}}.

 I've fixed the typo, removed {{LibevConnection}} (as it was already the 
default as you pointed out) and added {{DesBytesTypeByteArray}} if available, 
in [this 
commit|https://github.com/stef1927/cassandra/commit/2d10a8dc7d369324ecfd5d2457c15cf716243d98].
 I've saved the commit history on this [historical 
branch|https://github.com/stef1927/cassandra/commits/11053-2.1-historical].

I've squashed and up-merged:

||2.1||2.2||2.2 win||3.0||3.5||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/11053-2.1]|[patch|https://github.com/stef1927/cassandra/commits/11053-2.2]|
 
|[patch|https://github.com/stef1927/cassandra/commits/11053-3.0]|[patch|https://github.com/stef1927/cassandra/commits/11053-3.5]|[patch|https://github.com/stef1927/cassandra/commits/11053]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-2.2-dtest/]|[win
 
dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-2.2-windows-dtest_win32/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-3.5-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-dtest/]|

-No patch merges cleanly, all up-merges have conflicts.-
There are conflicts all the way up to 3.5, only patch to merge cleanly is 3.5 
into trunk.

CI is still pending.


was (Author: stefania):
Thank you for your latest comments and for introducing 
{{deserializers.DesBytesTypeByteArray}}.

 I've fixed the typo, removed {{LibevConnection}} (as it was already the 
default as you pointed out) and added {{DesBytesTypeByteArray}} if available, 
in [this 
commit|https://github.com/stef1927/cassandra/commit/2d10a8dc7d369324ecfd5d2457c15cf716243d98].
 I've saved the commit history on this [historical 
branch|https://github.com/stef1927/cassandra/commits/11053-2.1-historical].

I've squashed and up-merged:

||2.1||2.2||2.2 win||3.0||trunk||
|[patch|https://github.com/stef1927/cassandra/commits/11053-2.1]|[patch|https://github.com/stef1927/cassandra/commits/11053-2.2]|
 
|[patch|https://github.com/stef1927/cassandra/commits/11053-3.0]|[patch|https://github.com/stef1927/cassandra/commits/11053]|
|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-2.2-dtest/]|[win
 
dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-2.2-windows-dtest_win32/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-3.0-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-11053-dtest/]|

No patch merges cleanly, all up-merges have conflicts.

CI is still pending.

> COPY FROM on large datasets: fix progress report and debug performance
> --
>
> Key: CASSANDRA-11053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: copy_from_large_benchmark.txt, 
> copy_from_large_benchmark_2.txt, parent_profile.txt, parent_profile_2.txt, 
> worker_profiles.txt, worker_profiles_2.txt
>
>
> Running COPY from on a large dataset (20G divided in 20M records) revealed 
> two issues:
> * The progress report is incorrect, it is very slow until almost the end of 
> the test at which point it catches up extremely quickly.
> * The performance in rows per second is similar to running smaller tests with 
> a smaller cluster locally (approx 35,000 rows per second). As a comparison, 
> cassandra-stress manages 50,000 rows per second under the same set-up, 
> therefore resulting 1.5 times faster. 
> See attached file _copy_from_large_benchmark.txt_ for the benchmark details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11053) COPY FROM on large datasets: fix progress report and debug performance

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177280#comment-15177280
 ] 

Stefania commented on CASSANDRA-11053:
--

One more test timing out, fix is 
[here|https://github.com/stef1927/cassandra/commit/28f77134bd1d96bc6a4235ba3232082db5f92043].

> COPY FROM on large datasets: fix progress report and debug performance
> --
>
> Key: CASSANDRA-11053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: copy_from_large_benchmark.txt, 
> copy_from_large_benchmark_2.txt, parent_profile.txt, parent_profile_2.txt, 
> worker_profiles.txt, worker_profiles_2.txt
>
>
> Running COPY from on a large dataset (20G divided in 20M records) revealed 
> two issues:
> * The progress report is incorrect, it is very slow until almost the end of 
> the test at which point it catches up extremely quickly.
> * The performance in rows per second is similar to running smaller tests with 
> a smaller cluster locally (approx 35,000 rows per second). As a comparison, 
> cassandra-stress manages 50,000 rows per second under the same set-up, 
> therefore resulting 1.5 times faster. 
> See attached file _copy_from_large_benchmark.txt_ for the benchmark details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177248#comment-15177248
 ] 

Marcus Eriksson commented on CASSANDRA-11176:
-

[~aweisberg] what do you mean? I posted the test result with that leak in the 
first comment?

> SSTableRewriter.InvalidateKeys should have a weak reference to cache
> 
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.14, 2.2.6, 3.5, 3.0.5
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references 
> SSTableRewriter.InvalidateKeys which references the cache. The cache 
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111  
> NoSpamLogger.java:97 - Strong self-ref loop detected 
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache 
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture 
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue 
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue 
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable 
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable 
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore 
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace 
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap 
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap 
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker 
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference 
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object 
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List 
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object 
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator 
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final 
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator 
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool 
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool 
> 

[jira] [Commented] (CASSANDRA-11274) cqlsh: interpret CQL type for formatting blob types

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177077#comment-15177077
 ] 

Stefania commented on CASSANDRA-11274:
--

Thank you.

I will keep this ticket in patch available until CASSANDRA-11053 is committed 
to make like easier for the committer of that ticket (the only clean merge is 
from 3.5 -> trunk at the moment and if this ticket is committed this is no 
longer true).

Once CASSANDRA-11053 is committed, I will rebase, repeat the tests and if 
everything is fine with the tests then this patch can be committed.

> cqlsh: interpret CQL type for formatting blob types
> ---
>
> Key: CASSANDRA-11274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
>
> During the development of CASSANDRA-11053 we have added changes to the cqlsh 
> formatting code so that we can format {{blob}} types correctly even if they 
> are represented as {{str}} rather than {{bytearray}}.
> At the moment we ensure {{blob}} are of type {{bytearray}} via the following 
> shortcut:
> {code}
> cassandra.cqltypes.BytesType.deserialize = staticmethod(lambda byts, 
> protocol_version: bytearray(byts))
> {code}
> After CASSANDRA-11053 is committed there will be a similar shortcut to 
> override the fast serializers implemented in cython. 
> Decoding the CQL type is safer in that it decouples cqlsh formatting from the 
> types returned by the driver deserializers but it is also unnecessary for 
> CASSANDRA-11053 performance goals and risky for older releases. 
> Therefore this ticket delivers this functionality but only on trunk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11053) COPY FROM on large datasets: fix progress report and debug performance

[Stefania (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177068#comment-15177068
 ] 

Stefania commented on CASSANDRA-11053:
--

Thank you for the latest review. 

Unfortunately there was one more small problem; I noticed it on Windows but it 
is actually happening on Linux too. If a child process crashes, 
{{import_records}} will not terminate because the parent process is unable to 
get the lock required to write termination messages to the pipes. The reason is 
that the feeder process is hanging on a send and not releasing the lock. To fix 
this properly, we would have to introduce a bounded semaphore to keep track of 
how many messages are in transit on a pipe. However, since the problem only 
occurs when a child process crashes, and in this case we just want to 
terminate, I simply added a workaround to avoid sending termination messages to 
processes if at least one has crashed. In this case the processes will simply 
terminate. The only consequence should be that any profiling results won't be 
available. 

Please check [this 
commit|https://github.com/stef1927/cassandra/commit/7186cf803fe6cff126b310d7b7785623688b9aa4].

I've restarted CI on all branches.

> COPY FROM on large datasets: fix progress report and debug performance
> --
>
> Key: CASSANDRA-11053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: copy_from_large_benchmark.txt, 
> copy_from_large_benchmark_2.txt, parent_profile.txt, parent_profile_2.txt, 
> worker_profiles.txt, worker_profiles_2.txt
>
>
> Running COPY from on a large dataset (20G divided in 20M records) revealed 
> two issues:
> * The progress report is incorrect, it is very slow until almost the end of 
> the test at which point it catches up extremely quickly.
> * The performance in rows per second is similar to running smaller tests with 
> a smaller cluster locally (approx 35,000 rows per second). As a comparison, 
> cassandra-stress manages 50,000 rows per second under the same set-up, 
> therefore resulting 1.5 times faster. 
> See attached file _copy_from_large_benchmark.txt_ for the benchmark details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-11275) sstableloader fails with java.lang.IllegalArgumentException: flags is not a column defined in this metadata

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams reassigned CASSANDRA-11275:


Assignee: Sergey Kirillov

> sstableloader fails with java.lang.IllegalArgumentException: flags is not a 
> column defined in this metadata
> ---
>
> Key: CASSANDRA-11275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11275
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: ubuntu, cassandra 3.0.3
>Reporter: Sergey Kirillov
>Assignee: Sergey Kirillov
> Fix For: 3.5, 3.0.5
>
>
> When used on a cluster with materialized view sstableloader fails with:
> {noformat}
> flags is not a column defined in this metadata
> java.lang.IllegalArgumentException: flags is not a column defined in this 
> metadata
> at 
> com.datastax.driver.core.ColumnDefinitions.getAllIdx(ColumnDefinitions.java:272)
> at 
> com.datastax.driver.core.ColumnDefinitions.getFirstIdx(ColumnDefinitions.java:278)
> at 
> com.datastax.driver.core.ArrayBackedRow.getIndexOf(ArrayBackedRow.java:83)
> at 
> com.datastax.driver.core.AbstractGettableData.getSet(AbstractGettableData.java:217)
> at 
> org.apache.cassandra.utils.NativeSSTableLoaderClient.createTableMetadata(NativeSSTableLoaderClient.java:172)
> at 
> org.apache.cassandra.utils.NativeSSTableLoaderClient.fetchViews(NativeSSTableLoaderClient.java:157)
> at 
> org.apache.cassandra.utils.NativeSSTableLoaderClient.init(NativeSSTableLoaderClient.java:93)
> at 
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:159)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:103)
> {noformat}
> This happens because there is no column `flags` in `system_schema.views`, but 
> NativeSSTableLoaderClient want's to read it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11275) sstableloader fails with java.lang.IllegalArgumentException: flags is not a column defined in this metadata

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11275:
---
   Resolution: Fixed
Fix Version/s: (was: 3.0.x)
   (was: 3.x)
   3.0.5
   3.5
   Status: Resolved  (was: Ready to Commit)

Committed, thanks!

> sstableloader fails with java.lang.IllegalArgumentException: flags is not a 
> column defined in this metadata
> ---
>
> Key: CASSANDRA-11275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11275
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
> Environment: ubuntu, cassandra 3.0.3
>Reporter: Sergey Kirillov
> Fix For: 3.5, 3.0.5
>
>
> When used on a cluster with materialized view sstableloader fails with:
> {noformat}
> flags is not a column defined in this metadata
> java.lang.IllegalArgumentException: flags is not a column defined in this 
> metadata
> at 
> com.datastax.driver.core.ColumnDefinitions.getAllIdx(ColumnDefinitions.java:272)
> at 
> com.datastax.driver.core.ColumnDefinitions.getFirstIdx(ColumnDefinitions.java:278)
> at 
> com.datastax.driver.core.ArrayBackedRow.getIndexOf(ArrayBackedRow.java:83)
> at 
> com.datastax.driver.core.AbstractGettableData.getSet(AbstractGettableData.java:217)
> at 
> org.apache.cassandra.utils.NativeSSTableLoaderClient.createTableMetadata(NativeSSTableLoaderClient.java:172)
> at 
> org.apache.cassandra.utils.NativeSSTableLoaderClient.fetchViews(NativeSSTableLoaderClient.java:157)
> at 
> org.apache.cassandra.utils.NativeSSTableLoaderClient.init(NativeSSTableLoaderClient.java:93)
> at 
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:159)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:103)
> {noformat}
> This happens because there is no column `flags` in `system_schema.views`, but 
> NativeSSTableLoaderClient want's to read it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/6] cassandra git commit: Fix sstableloader fail when using materialized view

[yukim]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 66d1b8f22 -> 19bccb4ea
  refs/heads/cassandra-3.5 bec103826 -> a8dc14d74
  refs/heads/trunk c19910419 -> 980e476bd


Fix sstableloader fail when using materialized view

patch by Sergey Kirillov; reviewed by yukim for CASSANDRA-11275


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19bccb4e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19bccb4e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19bccb4e

Branch: refs/heads/cassandra-3.0
Commit: 19bccb4eaea7a6b33b72bb55e6923206526009e8
Parents: 66d1b8f
Author: Sergey Kirillov 
Authored: Mon Feb 29 20:42:25 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 18:55:03 2016 -0600

--
 CHANGES.txt  | 1 +
 .../org/apache/cassandra/utils/NativeSSTableLoaderClient.java| 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19bccb4e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0c721e8..520e66e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.5
+ * Fix sstableloader fail when using materialized view (CASSANDRA-11275)
 Merged from 2.2:
  * Only log yaml config once, at startup (CASSANDRA-11217)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/19bccb4e/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
--
diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java 
b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
index 225e453..5bcbcf7 100644
--- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
+++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
@@ -169,12 +169,12 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
   Types types)
 {
 UUID id = row.getUUID("id");
-Set flags = 
CFMetaData.flagsFromStrings(row.getSet("flags", String.class));
+Set flags = isView ? Collections.emptySet() : 
CFMetaData.flagsFromStrings(row.getSet("flags", String.class));
 
 boolean isSuper = flags.contains(CFMetaData.Flag.SUPER);
 boolean isCounter = flags.contains(CFMetaData.Flag.COUNTER);
 boolean isDense = flags.contains(CFMetaData.Flag.DENSE);
-boolean isCompound = flags.contains(CFMetaData.Flag.COMPOUND);
+boolean isCompound = isView || 
flags.contains(CFMetaData.Flag.COMPOUND);
 
 String columnsQuery = String.format("SELECT * FROM %s.%s WHERE 
keyspace_name = ? AND table_name = ?",
 SchemaKeyspace.NAME,

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.5


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8dc14d7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8dc14d7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8dc14d7

Branch: refs/heads/trunk
Commit: a8dc14d74902f5b04fb0a06b935be146c719ae61
Parents: bec1038 19bccb4
Author: Yuki Morishita 
Authored: Wed Mar 2 18:56:15 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 18:56:15 2016 -0600

--
 CHANGES.txt  | 2 ++
 .../org/apache/cassandra/utils/NativeSSTableLoaderClient.java| 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8dc14d7/CHANGES.txt
--
diff --cc CHANGES.txt
index 2073ab0,520e66e..a6cae21
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,4 -1,5 +1,6 @@@
 -3.0.5
 +3.5
++Merged from 3.0:
+  * Fix sstableloader fail when using materialized view (CASSANDRA-11275)
  Merged from 2.2:
   * Only log yaml config once, at startup (CASSANDRA-11217)
   * Reference leak with parallel repairs on the same table (CASSANDRA-11215)

[3/6] cassandra git commit: Fix sstableloader fail when using materialized view

[yukim]

Fix sstableloader fail when using materialized view

patch by Sergey Kirillov; reviewed by yukim for CASSANDRA-11275


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19bccb4e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19bccb4e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19bccb4e

Branch: refs/heads/trunk
Commit: 19bccb4eaea7a6b33b72bb55e6923206526009e8
Parents: 66d1b8f
Author: Sergey Kirillov 
Authored: Mon Feb 29 20:42:25 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 18:55:03 2016 -0600

--
 CHANGES.txt  | 1 +
 .../org/apache/cassandra/utils/NativeSSTableLoaderClient.java| 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19bccb4e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0c721e8..520e66e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.5
+ * Fix sstableloader fail when using materialized view (CASSANDRA-11275)
 Merged from 2.2:
  * Only log yaml config once, at startup (CASSANDRA-11217)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/19bccb4e/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
--
diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java 
b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
index 225e453..5bcbcf7 100644
--- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
+++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
@@ -169,12 +169,12 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
   Types types)
 {
 UUID id = row.getUUID("id");
-Set flags = 
CFMetaData.flagsFromStrings(row.getSet("flags", String.class));
+Set flags = isView ? Collections.emptySet() : 
CFMetaData.flagsFromStrings(row.getSet("flags", String.class));
 
 boolean isSuper = flags.contains(CFMetaData.Flag.SUPER);
 boolean isCounter = flags.contains(CFMetaData.Flag.COUNTER);
 boolean isDense = flags.contains(CFMetaData.Flag.DENSE);
-boolean isCompound = flags.contains(CFMetaData.Flag.COMPOUND);
+boolean isCompound = isView || 
flags.contains(CFMetaData.Flag.COMPOUND);
 
 String columnsQuery = String.format("SELECT * FROM %s.%s WHERE 
keyspace_name = ? AND table_name = ?",
 SchemaKeyspace.NAME,

[2/6] cassandra git commit: Fix sstableloader fail when using materialized view

[yukim]

Fix sstableloader fail when using materialized view

patch by Sergey Kirillov; reviewed by yukim for CASSANDRA-11275


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19bccb4e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19bccb4e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19bccb4e

Branch: refs/heads/cassandra-3.5
Commit: 19bccb4eaea7a6b33b72bb55e6923206526009e8
Parents: 66d1b8f
Author: Sergey Kirillov 
Authored: Mon Feb 29 20:42:25 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 18:55:03 2016 -0600

--
 CHANGES.txt  | 1 +
 .../org/apache/cassandra/utils/NativeSSTableLoaderClient.java| 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19bccb4e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0c721e8..520e66e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0.5
+ * Fix sstableloader fail when using materialized view (CASSANDRA-11275)
 Merged from 2.2:
  * Only log yaml config once, at startup (CASSANDRA-11217)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/19bccb4e/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
--
diff --git a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java 
b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
index 225e453..5bcbcf7 100644
--- a/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
+++ b/src/java/org/apache/cassandra/utils/NativeSSTableLoaderClient.java
@@ -169,12 +169,12 @@ public class NativeSSTableLoaderClient extends 
SSTableLoader.Client
   Types types)
 {
 UUID id = row.getUUID("id");
-Set flags = 
CFMetaData.flagsFromStrings(row.getSet("flags", String.class));
+Set flags = isView ? Collections.emptySet() : 
CFMetaData.flagsFromStrings(row.getSet("flags", String.class));
 
 boolean isSuper = flags.contains(CFMetaData.Flag.SUPER);
 boolean isCounter = flags.contains(CFMetaData.Flag.COUNTER);
 boolean isDense = flags.contains(CFMetaData.Flag.DENSE);
-boolean isCompound = flags.contains(CFMetaData.Flag.COMPOUND);
+boolean isCompound = isView || 
flags.contains(CFMetaData.Flag.COMPOUND);
 
 String columnsQuery = String.format("SELECT * FROM %s.%s WHERE 
keyspace_name = ? AND table_name = ?",
 SchemaKeyspace.NAME,

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.5


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a8dc14d7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a8dc14d7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a8dc14d7

Branch: refs/heads/cassandra-3.5
Commit: a8dc14d74902f5b04fb0a06b935be146c719ae61
Parents: bec1038 19bccb4
Author: Yuki Morishita 
Authored: Wed Mar 2 18:56:15 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 18:56:15 2016 -0600

--
 CHANGES.txt  | 2 ++
 .../org/apache/cassandra/utils/NativeSSTableLoaderClient.java| 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a8dc14d7/CHANGES.txt
--
diff --cc CHANGES.txt
index 2073ab0,520e66e..a6cae21
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,4 -1,5 +1,6 @@@
 -3.0.5
 +3.5
++Merged from 3.0:
+  * Fix sstableloader fail when using materialized view (CASSANDRA-11275)
  Merged from 2.2:
   * Only log yaml config once, at startup (CASSANDRA-11217)
   * Reference leak with parallel repairs on the same table (CASSANDRA-11215)

[6/6] cassandra git commit: Merge branch 'cassandra-3.5' into trunk

[yukim]

Merge branch 'cassandra-3.5' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/980e476b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/980e476b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/980e476b

Branch: refs/heads/trunk
Commit: 980e476bdefd3642f95e374685de7fb524c544c6
Parents: c199104 a8dc14d
Author: Yuki Morishita 
Authored: Wed Mar 2 18:58:16 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 18:58:16 2016 -0600

--
 CHANGES.txt  | 2 ++
 .../org/apache/cassandra/utils/NativeSSTableLoaderClient.java| 4 ++--
 2 files changed, 4 insertions(+), 2 deletions(-)
--

[jira] [Updated] (CASSANDRA-11299) AssertionError when quering by secondary index

[JIRA]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michał Matłoka updated CASSANDRA-11299:
---
Description: 
Hi,
Recently we have upgraded from Cassandra 2.2.4 to 3.3. I have issues with one 
table. When I try to query using any secondary index I get e.g. in cqlsh
{code}
Traceback (most recent call last):
  File "/usr/bin/cqlsh.py", line 1249, in perform_simple_statement
result = future.result()
  File 
"/usr/share/cassandra/lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/cluster.py",
 line 3122, in result
raise self._final_exception
ReadFailure: code=1300 [Replica(s) failed to execute read] message="Operation 
failed - received 0 responses and 1 failures" info={'failures': 1, 
'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
{code}

Node logs shows then:
{code}
[[AWARN  [SharedPool-Worker-2] 2016-03-03 00:47:01,679 
AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
Thread[SharedPool-Worker-2,5,main]: {}
java.lang.AssertionError: null
at 
org.apache.cassandra.index.internal.composites.CompositesSearcher$1Transform.findEntry(CompositesSearcher.java:225)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.index.internal.composites.CompositesSearcher$1Transform.applyToRow(CompositesSearcher.java:215)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116) 
~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:133)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:89)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:79)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:294)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65) 
~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:292) 
~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1789)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2457)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
~[na:1.8.0_66]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
 ~[apache-cassandra-3.3.0.jar:3.3.0]
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
 [apache-cassandra-3.3.0.jar:3.3.0]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
[apache-cassandra-3.3.0.jar:3.3.0]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
{code}


SStables are upgraded, I have tried repair and scrub. I have tried to rebuild 
indexes, and even remove them and re-add them.It occurs on every cluster node.

Additionally I had seen in this table case where PRIMARY KEY was duplicated!!! 
(there were two rows with same primary key, by seeing what columns were set I 
can say one was older, and second was from newer query which sets only a subset 
of columns)

  was:
Hi,
Recently we have upgraded from Cassandra 2.2.4 to 3.3. I have issues with one 
table. When I try to query using any secondary index I get e.g. in cqlsh
{code}
Traceback (most recent call last):
  File "/usr/bin/cqlsh.py", line 1249, in perform_simple_statement
result = future.result()
  File 
"/usr/share/cassandra/lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/cluster.py",
 line 3122, in result
raise self._final_exception
ReadFailure: code=1300 [Replica(s) failed to execute read] message="Operation 
failed - received 0 responses and 1 failures" info={'failures': 1, 
'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
{code}

Node logs shows then:

[jira] [Commented] (CASSANDRA-11279) dtest failure in disk_balance_test.TestDiskBalance.disk_balance_bootstrap_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176716#comment-15176716
 ] 

Russ Hatch commented on CASSANDRA-11279:


This assert is possibly just a bit too stringent, values above would be about 
26% below max and the assertion error wants 20% or less.

> dtest failure in disk_balance_test.TestDiskBalance.disk_balance_bootstrap_test
> --
>
> Key: CASSANDRA-11279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11279
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: DS Test Eng
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1011/testReport/disk_balance_test/TestDiskBalance/disk_balance_bootstrap_test
> Failed on CassCI build trunk_dtest #1011
> This looks likely to be a test issue, perhaps we need to relax the assertion 
> here a bit:
> {noformat}
> values not within 20.00% of the max: (474650, 382235, 513385) (node1)
> {noformat}
> This is flaky with several flaps in the last few weeks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11168) Hint Metrics are updated even if hinted_hand-offs=false

[Anubhav Kale (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176707#comment-15176707
 ] 

Anubhav Kale commented on CASSANDRA-11168:
--

So, making sure I understand Aleksey's thought this correctly, what we want is 
below. Can you confirm (only increment if hint window expired) ?

if (DatabaseDescriptor.hintedHandoffEnabled())
{
Set disabledDCs = 
DatabaseDescriptor.hintedHandoffDisabledDCs();
if (!disabledDCs.isEmpty())
{
final String dc = 
DatabaseDescriptor.getEndpointSnitch().getDatacenter(ep);
if (disabledDCs.contains(dc))
{
Tracing.trace("Not hinting {} since its 
data center {} has been disabled {}", ep, dc, disabledDCs);

return false;
}
}

boolean hintWindowExpired = 
Gossiper.instance.getEndpointDowntime(ep) > 
DatabaseDescriptor.getMaxHintWindow();
if (hintWindowExpired)
{

HintsService.instance.metrics.incrPastWindow(ep);
Tracing.trace("Not hinting {} which has been 
down {} ms", ep, Gossiper.instance.getEndpointDowntime(ep));
}
return !hintWindowExpired;
}
else
{
return false;
}

> Hint Metrics are updated even if hinted_hand-offs=false
> ---
>
> Key: CASSANDRA-11168
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11168
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Anubhav Kale
>Assignee: Anubhav Kale
>Priority: Minor
> Attachments: 0001-Hinted-Handoff-Fix.patch
>
>
> In our PROD logs, we noticed a lot of hint metrics even though we have 
> disabled hinted handoffs.
> The reason is StorageProxy.ShouldHint has an inverted if condition. 
> We should also wrap the if (hintWindowExpired) block in if 
> (DatabaseDescriptor.hintedHandoffEnabled()).
> The fix is easy, and I can provide a patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11297) AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11297:
---
Reviewer: Jeremiah Jordan
  Status: Patch Available  (was: Open)

> AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)
> -
>
> Key: CASSANDRA-11297
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11297
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Jeremiah Jordan
>Assignee: Yuki Morishita
> Fix For: 2.2.x, 3.0.x
>
>
> Looks like CASSANDRA-10859 is an issue in cassandra-2.2 and 3.0 as well, if 
> you add the -ea to the nodetool script in those you get the problem in those 
> as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11297) AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176691#comment-15176691
 ] 

Yuki Morishita commented on CASSANDRA-11297:


Backported to 2.2 and 3.0, and tests look good.

||branch||testall||dtest||
|[11297-2.2|https://github.com/yukim/cassandra/tree/11297-2.2]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11297-2.2-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11297-2.2-dtest/lastCompletedBuild/testReport/]|
|[11297-3.0|https://github.com/yukim/cassandra/tree/11297-3.0]|[testall|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11297-3.0-testall/lastCompletedBuild/testReport/]|[dtest|http://cassci.datastax.com/view/Dev/view/yukim/job/yukim-11297-3.0-dtest/lastCompletedBuild/testReport/]|


> AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)
> -
>
> Key: CASSANDRA-11297
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11297
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Jeremiah Jordan
>Assignee: Yuki Morishita
> Fix For: 2.2.x, 3.0.x
>
>
> Looks like CASSANDRA-10859 is an issue in cassandra-2.2 and 3.0 as well, if 
> you add the -ea to the nodetool script in those you get the problem in those 
> as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache

[Ariel Weisberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176638#comment-15176638
 ] 

Ariel Weisberg commented on CASSANDRA-11176:


What about the leak that's already been detected? It's not in the new ticket 
and it's kind of orthogonal.

> SSTableRewriter.InvalidateKeys should have a weak reference to cache
> 
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.14, 2.2.6, 3.5, 3.0.5
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references 
> SSTableRewriter.InvalidateKeys which references the cache. The cache 
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111  
> NoSpamLogger.java:97 - Strong self-ref loop detected 
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache 
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture 
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue 
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue 
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable 
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable 
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore 
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace 
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap 
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap 
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker 
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference 
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object 
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List 
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object 
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator 
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final 
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator 
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool 
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool 
> 

[jira] [Updated] (CASSANDRA-11176) SSTableRewriter.InvalidateKeys should have a weak reference to cache

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11176:
---
Fix Version/s: (was: 3.0.4)
   3.0.5

> SSTableRewriter.InvalidateKeys should have a weak reference to cache
> 
>
> Key: CASSANDRA-11176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11176
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Jeremiah Jordan
>Assignee: Marcus Eriksson
> Fix For: 2.1.14, 2.2.6, 3.5, 3.0.5
>
>
> From [~aweisberg]
> bq. The SSTableReader.DropPageCache runnable references 
> SSTableRewriter.InvalidateKeys which references the cache. The cache 
> reference should be a WeakReference.
> {noformat}
> ERROR [Strong-Reference-Leak-Detector:1] 2016-02-17 14:51:52,111  
> NoSpamLogger.java:97 - Strong self-ref loop detected 
> [/var/lib/cassandra/data/keyspace1/standard1-990bc741d56411e591d5590d7a7ad312/ma-20-big,
> private java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.runOnClose-org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache,
> final java.lang.Runnable 
> org.apache.cassandra.io.sstable.format.SSTableReader$DropPageCache.andThen-org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys,
> final org.apache.cassandra.cache.InstrumentingCache 
> org.apache.cassandra.io.sstable.SSTableRewriter$InvalidateKeys.cache-org.apache.cassandra.cache.AutoSavingCache,
> protected volatile java.util.concurrent.ScheduledFuture 
> org.apache.cassandra.cache.AutoSavingCache.saveTask-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> final java.util.concurrent.ScheduledThreadPoolExecutor 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.this$0-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor,
> private final java.util.concurrent.BlockingQueue 
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue,
> private final java.util.concurrent.BlockingQueue 
> java.util.concurrent.ThreadPoolExecutor.workQueue-java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask,
> private java.util.concurrent.Callable 
> java.util.concurrent.FutureTask.callable-java.util.concurrent.Executors$RunnableAdapter,
> final java.lang.Runnable 
> java.util.concurrent.Executors$RunnableAdapter.task-org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable,
> private final java.lang.Runnable 
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.runnable-org.apache.cassandra.db.ColumnFamilyStore$3,
> final org.apache.cassandra.db.ColumnFamilyStore 
> org.apache.cassandra.db.ColumnFamilyStore$3.this$0-org.apache.cassandra.db.ColumnFamilyStore,
> public final org.apache.cassandra.db.Keyspace 
> org.apache.cassandra.db.ColumnFamilyStore.keyspace-org.apache.cassandra.db.Keyspace,
> private final java.util.concurrent.ConcurrentMap 
> org.apache.cassandra.db.Keyspace.columnFamilyStores-java.util.concurrent.ConcurrentHashMap,
> private final java.util.concurrent.ConcurrentMap 
> org.apache.cassandra.db.Keyspace.columnFamilyStores-org.apache.cassandra.db.ColumnFamilyStore,
> private final org.apache.cassandra.db.lifecycle.Tracker 
> org.apache.cassandra.db.ColumnFamilyStore.data-org.apache.cassandra.db.lifecycle.Tracker,
> final java.util.concurrent.atomic.AtomicReference 
> org.apache.cassandra.db.lifecycle.Tracker.view-java.util.concurrent.atomic.AtomicReference,
> private volatile java.lang.Object 
> java.util.concurrent.atomic.AtomicReference.value-org.apache.cassandra.db.lifecycle.View,
> public final java.util.List 
> org.apache.cassandra.db.lifecycle.View.liveMemtables-com.google.common.collect.SingletonImmutableList,
> final transient java.lang.Object 
> com.google.common.collect.SingletonImmutableList.element-org.apache.cassandra.db.Memtable,
> private final org.apache.cassandra.utils.memory.MemtableAllocator 
> org.apache.cassandra.db.Memtable.allocator-org.apache.cassandra.utils.memory.SlabAllocator,
> private final 
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator 
> org.apache.cassandra.utils.memory.MemtableAllocator.onHeap-org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator,
> private final org.apache.cassandra.utils.memory.MemtablePool$SubPool 
> org.apache.cassandra.utils.memory.MemtableAllocator$SubAllocator.parent-org.apache.cassandra.utils.memory.MemtablePool$SubPool,
> final org.apache.cassandra.utils.memory.MemtablePool 
> org.apache.cassandra.utils.memory.MemtablePool$SubPool.this$0-org.apache.cassandra.utils.memory.SlabPool,
> final org.apache.cassandra.utils.memory.MemtableCleanerThread 
> 

[jira] [Updated] (CASSANDRA-11215) Reference leak with parallel repairs on the same table

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11215:
---
Fix Version/s: (was: 3.0.4)
   3.0.5

> Reference leak with parallel repairs on the same table
> --
>
> Key: CASSANDRA-11215
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11215
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
> Fix For: 2.2.6, 3.5, 3.0.5
>
>
> When starting multiple repairs on the same table Cassandra starts to log 
> about reference leak as:
> {noformat}
> ERROR [Reference-Reaper:1] 2016-02-23 15:02:05,516 Ref.java:187 - LEAK 
> DETECTED: a reference 
> (org.apache.cassandra.utils.concurrent.Ref$State@5213f926) to class 
> org.apache.cassandra.io.sstable.format.SSTableReader
> $InstanceTidier@605893242:.../testrepair/standard1-dcf311a0da3411e5a5c0c1a39c091431/la-30-big
>  was not released before the reference was garbage collected
> {noformat}
> Reproducible with:
> {noformat}
> ccm create repairtest -v 2.2.5 -n 3
> ccm start
> ccm stress write n=100 -schema 
> replication(strategy=SimpleStrategy,factor=3) keyspace=testrepair
> # And then perform two repairs concurrently with:
> ccm node1 nodetool repair testrepair
> {noformat}
> I know that starting multiple repairs in parallel on the same table isn't 
> very wise, but this shouldn't result in reference leaks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/6] cassandra git commit: Move fixes for 3.0.5 under the correct version

[yukim]

Move fixes for 3.0.5 under the correct version


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66d1b8f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66d1b8f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66d1b8f2

Branch: refs/heads/cassandra-3.5
Commit: 66d1b8f22735a7b22e4a209afcefcc5743873f4a
Parents: 621f446
Author: Yuki Morishita 
Authored: Wed Mar 2 16:32:25 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 16:32:25 2016 -0600

--
 CHANGES.txt | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66d1b8f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 819fa22..0c721e8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,11 @@
+3.0.5
+Merged from 2.2:
+ * Only log yaml config once, at startup (CASSANDRA-11217)
+ * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
+Merged from 2.1:
+ * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
+
+
 3.0.4
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * MV should only query complex columns included in the view (CASSANDRA-11069)
@@ -21,8 +29,6 @@
properly (CASSANDRA-11050)
  * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
 Merged from 2.2:
- * Only log yaml config once, at startup (CASSANDRA-11217)
- * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)
  * Avoid NPE when serializing ErrorMessage with null message (CASSANDRA-11167)
  * Replacing an aggregate with a new version doesn't reset INITCOND 
(CASSANDRA-10840)
@@ -43,7 +49,6 @@ Merged from 2.2:
(CASSANDRA-10010)
  * cqlsh: change default encoding to UTF-8 (CASSANDRA-11124)
 Merged from 2.1:
- * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
  * Don't remove FailureDetector history on removeEndpoint (CASSANDRA-10371)
  * Only notify if repair status changed (CASSANDRA-11172)
  * Use logback setting for 'cassandra -v' command (CASSANDRA-10767)

[6/6] cassandra git commit: Merge branch 'cassandra-3.5' into trunk

[yukim]

Merge branch 'cassandra-3.5' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c1991041
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c1991041
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c1991041

Branch: refs/heads/trunk
Commit: c199104195bd58c13afcc354769ff560e5f290a6
Parents: 2d420e6 bec1038
Author: Yuki Morishita 
Authored: Wed Mar 2 16:40:17 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 16:40:17 2016 -0600

--
 CHANGES.txt | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)
--

[3/6] cassandra git commit: Move fixes for 3.0.5 under the correct version

[yukim]

Move fixes for 3.0.5 under the correct version


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66d1b8f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66d1b8f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66d1b8f2

Branch: refs/heads/trunk
Commit: 66d1b8f22735a7b22e4a209afcefcc5743873f4a
Parents: 621f446
Author: Yuki Morishita 
Authored: Wed Mar 2 16:32:25 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 16:32:25 2016 -0600

--
 CHANGES.txt | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66d1b8f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 819fa22..0c721e8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,11 @@
+3.0.5
+Merged from 2.2:
+ * Only log yaml config once, at startup (CASSANDRA-11217)
+ * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
+Merged from 2.1:
+ * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
+
+
 3.0.4
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * MV should only query complex columns included in the view (CASSANDRA-11069)
@@ -21,8 +29,6 @@
properly (CASSANDRA-11050)
  * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
 Merged from 2.2:
- * Only log yaml config once, at startup (CASSANDRA-11217)
- * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)
  * Avoid NPE when serializing ErrorMessage with null message (CASSANDRA-11167)
  * Replacing an aggregate with a new version doesn't reset INITCOND 
(CASSANDRA-10840)
@@ -43,7 +49,6 @@ Merged from 2.2:
(CASSANDRA-10010)
  * cqlsh: change default encoding to UTF-8 (CASSANDRA-11124)
 Merged from 2.1:
- * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
  * Don't remove FailureDetector history on removeEndpoint (CASSANDRA-10371)
  * Only notify if repair status changed (CASSANDRA-11172)
  * Use logback setting for 'cassandra -v' command (CASSANDRA-10767)

[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.5


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bec10382
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bec10382
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bec10382

Branch: refs/heads/cassandra-3.5
Commit: bec103826609add72068f3f9a4801ab2015d3db0
Parents: 965ee08 66d1b8f
Author: Yuki Morishita 
Authored: Wed Mar 2 16:39:42 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 16:39:42 2016 -0600

--
 CHANGES.txt | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bec10382/CHANGES.txt
--
diff --cc CHANGES.txt
index 729d491,0c721e8..2073ab0
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,36 -1,13 +1,44 @@@
 -3.0.5
++3.5
+ Merged from 2.2:
+  * Only log yaml config once, at startup (CASSANDRA-11217)
+  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
+ Merged from 2.1:
+  * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
+ 
+ 
 -3.0.4
 - * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
 +3.4
 + * (cqlsh) add cqlshrc option to always connect using ssl (CASSANDRA-10458)
 + * Cleanup a few resource warnings (CASSANDRA-11085)
 + * Allow custom tracing implementations (CASSANDRA-10392)
 + * Extract LoaderOptions to be able to be used from outside (CASSANDRA-10637)
 + * fix OnDiskIndexTest to properly treat empty ranges (CASSANDRA-11205)
 + * fix TrackerTest to handle new notifications (CASSANDRA-11178)
 + * add SASI validation for partitioner and complex columns (CASSANDRA-11169)
 + * Add caching of encrypted credentials in PasswordAuthenticator 
(CASSANDRA-7715)
 + * fix SASI memtable switching on flush (CASSANDRA-11159)
 + * Remove duplicate offline compaction tracking (CASSANDRA-11148)
 + * fix EQ semantics of analyzed SASI indexes (CASSANDRA-11130)
 + * Support long name output for nodetool commands (CASSANDRA-7950)
 + * Encrypted hints (CASSANDRA-11040)
 + * SASI index options validation (CASSANDRA-11136)
 + * Optimize disk seek using min/max column name meta data when the LIMIT 
clause is used
 +   (CASSANDRA-8180)
 + * Add LIKE support to CQL3 (CASSANDRA-11067)
 + * Generic Java UDF types (CASSANDRA-10819)
 + * cqlsh: Include sub-second precision in timestamps by default 
(CASSANDRA-10428)
 + * Set javac encoding to utf-8 (CASSANDRA-11077)
 + * Integrate SASI index into Cassandra (CASSANDRA-10661)
 + * Add --skip-flush option to nodetool snapshot
 + * Skip values for non-queried columns (CASSANDRA-10657)
 + * Add support for secondary indexes on static columns (CASSANDRA-8103)
 + * CommitLogUpgradeTestMaker creates broken commit logs (CASSANDRA-11051)
 + * Add metric for number of dropped mutations (CASSANDRA-10866)
 + * Simplify row cache invalidation code (CASSANDRA-10396)
 + * Support user-defined compaction through nodetool (CASSANDRA-10660)
 + * Stripe view locks by key and table ID to reduce contention 
(CASSANDRA-10981)
 + * Add nodetool gettimeout and settimeout commands (CASSANDRA-10953)
 + * Add 3.0 metadata to sstablemetadata output (CASSANDRA-10838)
 +Merged from 3.0:
   * MV should only query complex columns included in the view (CASSANDRA-11069)
   * Failed aggregate creation breaks server permanently (CASSANDRA-11064)
   * Add sstabledump tool (CASSANDRA-7464)
@@@ -52,9 -29,6 +60,7 @@@
 properly (CASSANDRA-11050)
   * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
  Merged from 2.2:
-  * Only log yaml config once, at startup (CASSANDRA-11217)
 + * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
-  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
   * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)
   * Avoid NPE when serializing ErrorMessage with null message (CASSANDRA-11167)
   * Replacing an aggregate with a new version doesn't reset INITCOND 
(CASSANDRA-10840)
@@@ -73,10 -47,8 +79,9 @@@
   * (cqlsh) Support utf-8/cp65001 encoding on Windows (CASSANDRA-11030)
   * Fix paging on DISTINCT queries repeats result when first row in partition 
changes
 (CASSANDRA-10010)
 + * (cqlsh) Support timezone conversion using pytz (CASSANDRA-10397)
   * cqlsh: change default encoding to UTF-8 (CASSANDRA-11124)
  Merged from 2.1:
-  * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
   * Don't remove FailureDetector history on removeEndpoint (CASSANDRA-10371)
   * Only notify if repair status changed (CASSANDRA-11172)
   * Use logback setting for 'cassandra -v' command (CASSANDRA-10767)

[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5

[yukim]

Merge branch 'cassandra-3.0' into cassandra-3.5


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bec10382
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bec10382
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bec10382

Branch: refs/heads/trunk
Commit: bec103826609add72068f3f9a4801ab2015d3db0
Parents: 965ee08 66d1b8f
Author: Yuki Morishita 
Authored: Wed Mar 2 16:39:42 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 16:39:42 2016 -0600

--
 CHANGES.txt | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/bec10382/CHANGES.txt
--
diff --cc CHANGES.txt
index 729d491,0c721e8..2073ab0
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,36 -1,13 +1,44 @@@
 -3.0.5
++3.5
+ Merged from 2.2:
+  * Only log yaml config once, at startup (CASSANDRA-11217)
+  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
+ Merged from 2.1:
+  * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
+ 
+ 
 -3.0.4
 - * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
 +3.4
 + * (cqlsh) add cqlshrc option to always connect using ssl (CASSANDRA-10458)
 + * Cleanup a few resource warnings (CASSANDRA-11085)
 + * Allow custom tracing implementations (CASSANDRA-10392)
 + * Extract LoaderOptions to be able to be used from outside (CASSANDRA-10637)
 + * fix OnDiskIndexTest to properly treat empty ranges (CASSANDRA-11205)
 + * fix TrackerTest to handle new notifications (CASSANDRA-11178)
 + * add SASI validation for partitioner and complex columns (CASSANDRA-11169)
 + * Add caching of encrypted credentials in PasswordAuthenticator 
(CASSANDRA-7715)
 + * fix SASI memtable switching on flush (CASSANDRA-11159)
 + * Remove duplicate offline compaction tracking (CASSANDRA-11148)
 + * fix EQ semantics of analyzed SASI indexes (CASSANDRA-11130)
 + * Support long name output for nodetool commands (CASSANDRA-7950)
 + * Encrypted hints (CASSANDRA-11040)
 + * SASI index options validation (CASSANDRA-11136)
 + * Optimize disk seek using min/max column name meta data when the LIMIT 
clause is used
 +   (CASSANDRA-8180)
 + * Add LIKE support to CQL3 (CASSANDRA-11067)
 + * Generic Java UDF types (CASSANDRA-10819)
 + * cqlsh: Include sub-second precision in timestamps by default 
(CASSANDRA-10428)
 + * Set javac encoding to utf-8 (CASSANDRA-11077)
 + * Integrate SASI index into Cassandra (CASSANDRA-10661)
 + * Add --skip-flush option to nodetool snapshot
 + * Skip values for non-queried columns (CASSANDRA-10657)
 + * Add support for secondary indexes on static columns (CASSANDRA-8103)
 + * CommitLogUpgradeTestMaker creates broken commit logs (CASSANDRA-11051)
 + * Add metric for number of dropped mutations (CASSANDRA-10866)
 + * Simplify row cache invalidation code (CASSANDRA-10396)
 + * Support user-defined compaction through nodetool (CASSANDRA-10660)
 + * Stripe view locks by key and table ID to reduce contention 
(CASSANDRA-10981)
 + * Add nodetool gettimeout and settimeout commands (CASSANDRA-10953)
 + * Add 3.0 metadata to sstablemetadata output (CASSANDRA-10838)
 +Merged from 3.0:
   * MV should only query complex columns included in the view (CASSANDRA-11069)
   * Failed aggregate creation breaks server permanently (CASSANDRA-11064)
   * Add sstabledump tool (CASSANDRA-7464)
@@@ -52,9 -29,6 +60,7 @@@
 properly (CASSANDRA-11050)
   * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
  Merged from 2.2:
-  * Only log yaml config once, at startup (CASSANDRA-11217)
 + * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
-  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
   * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)
   * Avoid NPE when serializing ErrorMessage with null message (CASSANDRA-11167)
   * Replacing an aggregate with a new version doesn't reset INITCOND 
(CASSANDRA-10840)
@@@ -73,10 -47,8 +79,9 @@@
   * (cqlsh) Support utf-8/cp65001 encoding on Windows (CASSANDRA-11030)
   * Fix paging on DISTINCT queries repeats result when first row in partition 
changes
 (CASSANDRA-10010)
 + * (cqlsh) Support timezone conversion using pytz (CASSANDRA-10397)
   * cqlsh: change default encoding to UTF-8 (CASSANDRA-11124)
  Merged from 2.1:
-  * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
   * Don't remove FailureDetector history on removeEndpoint (CASSANDRA-10371)
   * Only notify if repair status changed (CASSANDRA-11172)
   * Use logback setting for 'cassandra -v' command (CASSANDRA-10767)

[1/6] cassandra git commit: Move fixes for 3.0.5 under the correct version

[yukim]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.0 621f446ca -> 66d1b8f22
  refs/heads/cassandra-3.5 965ee082d -> bec103826
  refs/heads/trunk 2d420e6c2 -> c19910419


Move fixes for 3.0.5 under the correct version


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66d1b8f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66d1b8f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66d1b8f2

Branch: refs/heads/cassandra-3.0
Commit: 66d1b8f22735a7b22e4a209afcefcc5743873f4a
Parents: 621f446
Author: Yuki Morishita 
Authored: Wed Mar 2 16:32:25 2016 -0600
Committer: Yuki Morishita 
Committed: Wed Mar 2 16:32:25 2016 -0600

--
 CHANGES.txt | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/66d1b8f2/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 819fa22..0c721e8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,11 @@
+3.0.5
+Merged from 2.2:
+ * Only log yaml config once, at startup (CASSANDRA-11217)
+ * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
+Merged from 2.1:
+ * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
+
+
 3.0.4
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * MV should only query complex columns included in the view (CASSANDRA-11069)
@@ -21,8 +29,6 @@
properly (CASSANDRA-11050)
  * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
 Merged from 2.2:
- * Only log yaml config once, at startup (CASSANDRA-11217)
- * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)
  * Avoid NPE when serializing ErrorMessage with null message (CASSANDRA-11167)
  * Replacing an aggregate with a new version doesn't reset INITCOND 
(CASSANDRA-10840)
@@ -43,7 +49,6 @@ Merged from 2.2:
(CASSANDRA-10010)
  * cqlsh: change default encoding to UTF-8 (CASSANDRA-11124)
 Merged from 2.1:
- * InvalidateKeys should have a weak ref to key cache (CASSANDRA-11176)
  * Don't remove FailureDetector history on removeEndpoint (CASSANDRA-10371)
  * Only notify if repair status changed (CASSANDRA-11172)
  * Use logback setting for 'cassandra -v' command (CASSANDRA-10767)

[jira] [Updated] (CASSANDRA-11217) Only log yaml config once, at startup

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-11217:
--
Fix Version/s: (was: 3.0.4)
   (was: 3.x)
   3.0.5

> Only log yaml config once, at startup
> -
>
> Key: CASSANDRA-11217
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11217
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration, Core
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Fix For: 2.2.6, 3.5, 3.0.5
>
>
> CASSANDRA-6456 introduced a feature where the yaml is dumped in the log. At 
> startup this is a nice feature, but I see that it’s actually triggered every 
> time it handshakes with a node and fails to connect and the node happens to 
> be a seed ([see 
> here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L435]).
>  Calling {{DD.getseeds()}} calls the {{SeedProvider}}, and if you happen to 
> use {{SimpleSeedProvider}} it will reload the yaml config, and once again 
> dump it out to the log.
> It's debatable if {{DD.getseeds()}} should trigger a reload (which I added in 
> CASSANDRA-5459) or whether reloading the seeds should be a different method 
> (it probably should), but we shouldn't keep logging the yaml config on every 
> connection failure to a seed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10070) Automatic repair scheduling

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176519#comment-15176519
 ] 

Paulo Motta commented on CASSANDRA-10070:
-

bq. We start with the resource locking and then move on to the maintenance 
scheduling API. And after that I think most tasks could be discussed in 
parallel. Also I removed the task for management commands since I think it 
would be easier to add them while implementing the features. 

+1

bq. I've now created the sub-tasks and linked them to this issue. I didn't 
include the node configuration since it might be redundant but we might add it 
later on if we feel the need to.

Awesome!

> Automatic repair scheduling
> ---
>
> Key: CASSANDRA-10070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10070
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Minor
> Fix For: 3.x
>
> Attachments: Distributed Repair Scheduling.doc
>
>
> Scheduling and running repairs in a Cassandra cluster is most often a 
> required task, but this can both be hard for new users and it also requires a 
> bit of manual configuration. There are good tools out there that can be used 
> to simplify things, but wouldn't this be a good feature to have inside of 
> Cassandra? To automatically schedule and run repairs, so that when you start 
> up your cluster it basically maintains itself in terms of normal 
> anti-entropy, with the possibility for manual configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11258) Repair scheduling - Resource locking API

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176516#comment-15176516
 ] 

Paulo Motta commented on CASSANDRA-11258:
-

While 4 feels like a workaround, and 3 would probably make LWT more complex, I 
think 2 is probably the best option here, and this wouldn't prevent 1 from 
being pursued in the future.

While I think we could add a new VERB ({{REMOTE_CAS}}) to the messaging service 
without a protocol bump (by reusing the {{UNUSED_X}} verbs), I think we could 
this in a separate ticket to avoid losing focus here.

So I propose we use a global CAS ({{SERIAL}} consistency) for each DC lock for 
the first version, which should make multi-dc schedule repairs work when there 
is no network partition, and improve later when the {{REMOTE_CAS}} verb is in 
place. WDYT?

> Repair scheduling - Resource locking API
> 
>
> Key: CASSANDRA-11258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11258
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Minor
>
> Create a resource locking API & implementation that is able to lock a 
> resource in a specified data center. It should handle priorities to avoid 
> node starvation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11254) (windows) dtest failure in auth_test.TestAuth.restart_node_doesnt_lose_auth_data_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176514#comment-15176514
 ] 

Russ Hatch commented on CASSANDRA-11254:


known_failure decorator removed

> (windows) dtest failure in 
> auth_test.TestAuth.restart_node_doesnt_lose_auth_data_test
> -
>
> Key: CASSANDRA-11254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11254
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Philip Thompson
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/194/testReport/auth_test/TestAuth/restart_node_doesnt_lose_auth_data_test
> Failed on CassCI build cassandra-2.2_dtest_win32 #194
> looks like a problem with dtest code:
> {noformat}
> code=2200 [Invalid query] message="Keyspace ks does not exist"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11221) replication_test.ReplicationTest.network_topology_test flaps

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176502#comment-15176502
 ] 

Russ Hatch commented on CASSANDRA-11221:


known_failure decorator removed from dtest code

> replication_test.ReplicationTest.network_topology_test flaps
> 
>
> Key: CASSANDRA-11221
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11221
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Philip Thompson
>  Labels: dtest
>
> Test intermittently failing with set comparison errors that differ from one 
> failure to the next. Looks a bit more stable recently since #203 failed, but 
> probably worth keeping an eye on, and check if there's a problem with the 
> test code.
> most recent failure:
> http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/203/testReport/replication_test/ReplicationTest/network_topology_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11217) Only log yaml config once, at startup

[Jason Brown (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Brown updated CASSANDRA-11217:

   Resolution: Fixed
Fix Version/s: 3.5
   3.x
   3.0.4
   2.2.6
   Status: Resolved  (was: Ready to Commit)

Committed to 2.2, 3.0, 3.5, and trunk

Commit sha is 093454205498bad7094cb8fb362d60e95296b092

Thanks [~pauloricardomg]

> Only log yaml config once, at startup
> -
>
> Key: CASSANDRA-11217
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11217
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration, Core
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
> Fix For: 2.2.6, 3.0.4, 3.x, 3.5
>
>
> CASSANDRA-6456 introduced a feature where the yaml is dumped in the log. At 
> startup this is a nice feature, but I see that it’s actually triggered every 
> time it handshakes with a node and fails to connect and the node happens to 
> be a seed ([see 
> here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L435]).
>  Calling {{DD.getseeds()}} calls the {{SeedProvider}}, and if you happen to 
> use {{SimpleSeedProvider}} it will reload the yaml config, and once again 
> dump it out to the log.
> It's debatable if {{DD.getseeds()}} should trigger a reload (which I added in 
> CASSANDRA-5459) or whether reloading the seeds should be a different method 
> (it probably should), but we shouldn't keep logging the yaml config on every 
> connection failure to a seed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11221) replication_test.ReplicationTest.network_topology_test flaps

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson resolved CASSANDRA-11221.
-
   Resolution: Fixed
Reproduced In:   (was: 2.1.x)

This was fixed the day after it failed, in dtest commit cd4f8fd19f39a254

> replication_test.ReplicationTest.network_topology_test flaps
> 
>
> Key: CASSANDRA-11221
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11221
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Philip Thompson
>  Labels: dtest
>
> Test intermittently failing with set comparison errors that differ from one 
> failure to the next. Looks a bit more stable recently since #203 failed, but 
> probably worth keeping an eye on, and check if there's a problem with the 
> test code.
> most recent failure:
> http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/203/testReport/replication_test/ReplicationTest/network_topology_test/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[09/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/621f446c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/621f446c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/621f446c

Branch: refs/heads/trunk
Commit: 621f446ca4a319adcb50d2479e3fa7121eac6ebd
Parents: 49a33e4
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:59:37 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 48 
 .../cassandra/config/DatabaseDescriptor.java| 12 -
 .../config/YamlConfigurationLoader.java | 18 
 4 files changed, 59 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/621f446c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b77d811..819fa22 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -21,6 +21,7 @@
properly (CASSANDRA-11050)
  * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
 Merged from 2.2:
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)
  * Avoid NPE when serializing ErrorMessage with null message (CASSANDRA-11167)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/621f446c/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index ca5bcea..32c0bfa 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,11 +17,21 @@
  */
 package org.apache.cassandra.config;
 
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 
@@ -32,6 +42,8 @@ import 
org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -383,4 +395,40 @@ public class Config
 ssd,
 spinning
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/621f446c/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index c903775..7bcfb97 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -53,7 +53,6 @@ import org.apache.cassandra.scheduler.IRequestScheduler;
 import org.apache.cassandra.scheduler.NoScheduler;
 import 

[12/16] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.5


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cd6f57ec
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cd6f57ec
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cd6f57ec

Branch: refs/heads/cassandra-3.5
Commit: cd6f57ec07ff046ab599df1707eb7b952c139c23
Parents: 87992c3 621f446
Author: Jason Brown 
Authored: Wed Mar 2 13:00:41 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 13:00:41 2016 -0800

--

--

[15/16] cassandra git commit: Merge branch 'cassandra-3.5' into trunk

[jasobrown]

Merge branch 'cassandra-3.5' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9991db24
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9991db24
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9991db24

Branch: refs/heads/trunk
Commit: 9991db24bc0f9c0a3160687da635119a89e601a3
Parents: 87992c3 965ee08
Author: Jason Brown 
Authored: Wed Mar 2 13:02:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 13:02:27 2016 -0800

--

--

[06/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jasobrown]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49a33e49
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49a33e49
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49a33e49

Branch: refs/heads/trunk
Commit: 49a33e49c362f1ebdbf214320842a6f3a5ac16b0
Parents: 3c8a097 0934542
Author: Jason Brown 
Authored: Wed Mar 2 12:58:16 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:58:16 2016 -0800

--

--

[11/16] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5

[jasobrown]

Merge branch 'cassandra-3.0' into cassandra-3.5


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cd6f57ec
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cd6f57ec
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cd6f57ec

Branch: refs/heads/trunk
Commit: cd6f57ec07ff046ab599df1707eb7b952c139c23
Parents: 87992c3 621f446
Author: Jason Brown 
Authored: Wed Mar 2 13:00:41 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 13:00:41 2016 -0800

--

--

[07/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jasobrown]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49a33e49
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49a33e49
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49a33e49

Branch: refs/heads/cassandra-3.5
Commit: 49a33e49c362f1ebdbf214320842a6f3a5ac16b0
Parents: 3c8a097 0934542
Author: Jason Brown 
Authored: Wed Mar 2 12:58:16 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:58:16 2016 -0800

--

--

[13/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/965ee082
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/965ee082
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/965ee082

Branch: refs/heads/trunk
Commit: 965ee082da2b8a3ff5e932e91d261018d9232c75
Parents: cd6f57e
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 13:01:47 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 48 
 .../cassandra/config/DatabaseDescriptor.java| 11 -
 .../config/YamlConfigurationLoader.java | 18 
 4 files changed, 59 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/965ee082/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a4b1aab..729d491 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -52,6 +52,7 @@ Merged from 3.0:
properly (CASSANDRA-11050)
  * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
 Merged from 2.2:
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/965ee082/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index c783960..fea1b04 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,11 +17,21 @@
  */
 package org.apache.cassandra.config;
 
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 
@@ -32,6 +42,8 @@ import 
org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -387,4 +399,40 @@ public class Config
 ssd,
 spinning
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/965ee082/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index a3a9b4a..7057c37 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -103,6 +103,7 @@ public class DatabaseDescriptor
 private static String localDC;
 private static Comparator localComparator;
 private 

[16/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d420e6c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d420e6c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d420e6c

Branch: refs/heads/trunk
Commit: 2d420e6c2a362c69d1a66076ac7dd2db37ee9306
Parents: 9991db2
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 13:03:09 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 48 
 .../cassandra/config/DatabaseDescriptor.java| 11 -
 .../config/YamlConfigurationLoader.java | 18 
 4 files changed, 59 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d420e6c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a4b1aab..729d491 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -52,6 +52,7 @@ Merged from 3.0:
properly (CASSANDRA-11050)
  * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
 Merged from 2.2:
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d420e6c/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index c783960..fea1b04 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,11 +17,21 @@
  */
 package org.apache.cassandra.config;
 
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 
@@ -32,6 +42,8 @@ import 
org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -387,4 +399,40 @@ public class Config
 ssd,
 spinning
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d420e6c/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index a3a9b4a..7057c37 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -103,6 +103,7 @@ public class DatabaseDescriptor
 private static String localDC;
 private static Comparator localComparator;
 private 

[14/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/965ee082
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/965ee082
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/965ee082

Branch: refs/heads/cassandra-3.5
Commit: 965ee082da2b8a3ff5e932e91d261018d9232c75
Parents: cd6f57e
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 13:01:47 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 48 
 .../cassandra/config/DatabaseDescriptor.java| 11 -
 .../config/YamlConfigurationLoader.java | 18 
 4 files changed, 59 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/965ee082/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a4b1aab..729d491 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -52,6 +52,7 @@ Merged from 3.0:
properly (CASSANDRA-11050)
  * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
 Merged from 2.2:
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/965ee082/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index c783960..fea1b04 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,11 +17,21 @@
  */
 package org.apache.cassandra.config;
 
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 
@@ -32,6 +42,8 @@ import 
org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -387,4 +399,40 @@ public class Config
 ssd,
 spinning
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/965ee082/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index a3a9b4a..7057c37 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -103,6 +103,7 @@ public class DatabaseDescriptor
 private static String localDC;
 private static Comparator localComparator;
 

[04/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/09345420
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/09345420
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/09345420

Branch: refs/heads/trunk
Commit: 093454205498bad7094cb8fb362d60e95296b092
Parents: d1f6a1d
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:57:06 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 51 
 .../cassandra/config/DatabaseDescriptor.java| 12 -
 .../config/YamlConfigurationLoader.java | 18 +--
 4 files changed, 63 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 481916b..db6d79d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.6
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index e7ac0c0..c7ad837 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,14 +17,27 @@
  */
 package org.apache.cassandra.config;
 
+import java.io.ByteArrayInputStream;
 import java.io.IOException;
 import java.io.StringReader;
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.HashMap;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.apache.commons.lang3.builder.ToStringBuilder;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 import org.apache.cassandra.exceptions.ConfigurationException;
@@ -38,6 +51,8 @@ import org.supercsv.prefs.CsvPreference;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -368,4 +383,40 @@ public class Config
 {
 keyspace
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index af0595d..cbea2ff 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -53,7 +53,6 @@ import org.apache.cassandra.scheduler.IRequestScheduler;
 import 

[08/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/621f446c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/621f446c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/621f446c

Branch: refs/heads/cassandra-3.5
Commit: 621f446ca4a319adcb50d2479e3fa7121eac6ebd
Parents: 49a33e4
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:59:37 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 48 
 .../cassandra/config/DatabaseDescriptor.java| 12 -
 .../config/YamlConfigurationLoader.java | 18 
 4 files changed, 59 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/621f446c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b77d811..819fa22 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -21,6 +21,7 @@
properly (CASSANDRA-11050)
  * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
 Merged from 2.2:
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)
  * Avoid NPE when serializing ErrorMessage with null message (CASSANDRA-11167)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/621f446c/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index ca5bcea..32c0bfa 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,11 +17,21 @@
  */
 package org.apache.cassandra.config;
 
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 
@@ -32,6 +42,8 @@ import 
org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -383,4 +395,40 @@ public class Config
 ssd,
 spinning
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/621f446c/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index c903775..7bcfb97 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -53,7 +53,6 @@ import org.apache.cassandra.scheduler.IRequestScheduler;
 import org.apache.cassandra.scheduler.NoScheduler;
 import 

[01/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.2 d1f6a1d95 -> 093454205
  refs/heads/cassandra-3.0 3c8a09767 -> 621f446ca
  refs/heads/cassandra-3.5 87992c3a4 -> 965ee082d
  refs/heads/trunk 87992c3a4 -> 2d420e6c2


Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/09345420
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/09345420
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/09345420

Branch: refs/heads/cassandra-2.2
Commit: 093454205498bad7094cb8fb362d60e95296b092
Parents: d1f6a1d
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:57:06 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 51 
 .../cassandra/config/DatabaseDescriptor.java| 12 -
 .../config/YamlConfigurationLoader.java | 18 +--
 4 files changed, 63 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 481916b..db6d79d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.6
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index e7ac0c0..c7ad837 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,14 +17,27 @@
  */
 package org.apache.cassandra.config;
 
+import java.io.ByteArrayInputStream;
 import java.io.IOException;
 import java.io.StringReader;
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.HashMap;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.apache.commons.lang3.builder.ToStringBuilder;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 import org.apache.cassandra.exceptions.ConfigurationException;
@@ -38,6 +51,8 @@ import org.supercsv.prefs.CsvPreference;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -368,4 +383,40 @@ public class Config
 {
 keyspace
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 

[03/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/09345420
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/09345420
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/09345420

Branch: refs/heads/cassandra-3.5
Commit: 093454205498bad7094cb8fb362d60e95296b092
Parents: d1f6a1d
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:57:06 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 51 
 .../cassandra/config/DatabaseDescriptor.java| 12 -
 .../config/YamlConfigurationLoader.java | 18 +--
 4 files changed, 63 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 481916b..db6d79d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.6
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index e7ac0c0..c7ad837 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,14 +17,27 @@
  */
 package org.apache.cassandra.config;
 
+import java.io.ByteArrayInputStream;
 import java.io.IOException;
 import java.io.StringReader;
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.HashMap;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.apache.commons.lang3.builder.ToStringBuilder;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 import org.apache.cassandra.exceptions.ConfigurationException;
@@ -38,6 +51,8 @@ import org.supercsv.prefs.CsvPreference;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -368,4 +383,40 @@ public class Config
 {
 keyspace
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index af0595d..cbea2ff 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -53,7 +53,6 @@ import org.apache.cassandra.scheduler.IRequestScheduler;
 

[05/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0

[jasobrown]

Merge branch 'cassandra-2.2' into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49a33e49
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49a33e49
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49a33e49

Branch: refs/heads/cassandra-3.0
Commit: 49a33e49c362f1ebdbf214320842a6f3a5ac16b0
Parents: 3c8a097 0934542
Author: Jason Brown 
Authored: Wed Mar 2 12:58:16 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:58:16 2016 -0800

--

--

[02/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/09345420
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/09345420
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/09345420

Branch: refs/heads/cassandra-3.0
Commit: 093454205498bad7094cb8fb362d60e95296b092
Parents: d1f6a1d
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:57:06 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 51 
 .../cassandra/config/DatabaseDescriptor.java| 12 -
 .../config/YamlConfigurationLoader.java | 18 +--
 4 files changed, 63 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 481916b..db6d79d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.2.6
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index e7ac0c0..c7ad837 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,14 +17,27 @@
  */
 package org.apache.cassandra.config;
 
+import java.io.ByteArrayInputStream;
 import java.io.IOException;
 import java.io.StringReader;
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.HashMap;
 import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.apache.commons.lang3.builder.ToStringBuilder;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 import org.apache.cassandra.exceptions.ConfigurationException;
@@ -38,6 +51,8 @@ import org.supercsv.prefs.CsvPreference;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -368,4 +383,40 @@ public class Config
 {
 keyspace
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/09345420/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index af0595d..cbea2ff 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -53,7 +53,6 @@ import org.apache.cassandra.scheduler.IRequestScheduler;
 

[10/16] cassandra git commit: Only log yaml config once, at startup

[jasobrown]

Only log yaml config once, at startup

patch by jasobrown, reviewed by Paulo Motta for CASSANDRA-11217


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/621f446c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/621f446c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/621f446c

Branch: refs/heads/cassandra-3.0
Commit: 621f446ca4a319adcb50d2479e3fa7121eac6ebd
Parents: 49a33e4
Author: Jason Brown 
Authored: Fri Feb 12 14:55:27 2016 -0800
Committer: Jason Brown 
Committed: Wed Mar 2 12:59:37 2016 -0800

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/config/Config.java | 48 
 .../cassandra/config/DatabaseDescriptor.java| 12 -
 .../config/YamlConfigurationLoader.java | 18 
 4 files changed, 59 insertions(+), 20 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/621f446c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b77d811..819fa22 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -21,6 +21,7 @@
properly (CASSANDRA-11050)
  * Fix NPE when using forceRepairRangeAsync without DC (CASSANDRA-11239)
 Merged from 2.2:
+ * Only log yaml config once, at startup (CASSANDRA-11217)
  * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
  * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)
  * Avoid NPE when serializing ErrorMessage with null message (CASSANDRA-11167)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/621f446c/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index ca5bcea..32c0bfa 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -17,11 +17,21 @@
  */
 package org.apache.cassandra.config;
 
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
 import java.util.Set;
+import java.util.TreeMap;
 import java.util.concurrent.TimeUnit;
 
+import com.google.common.base.Joiner;
 import com.google.common.collect.Sets;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.config.EncryptionOptions.ClientEncryptionOptions;
 import org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
 
@@ -32,6 +42,8 @@ import 
org.apache.cassandra.config.EncryptionOptions.ServerEncryptionOptions;
  */
 public class Config
 {
+private static final Logger logger = LoggerFactory.getLogger(Config.class);
+
 /*
  * Prefix for Java properties for internal Cassandra configuration options
  */
@@ -383,4 +395,40 @@ public class Config
 ssd,
 spinning
 }
+
+private static final List SENSITIVE_KEYS = new ArrayList() 
{{
+add("client_encryption_options");
+add("server_encryption_options");
+}};
+public void log()
+{
+Map configMap = new TreeMap<>();
+for (Field field : getClass().getFields())
+{
+// ignore the constants
+if (Modifier.isFinal(field.getModifiers()))
+continue;
+
+String name = field.getName();
+if (SENSITIVE_KEYS.contains(name))
+{
+configMap.put(name, "");
+continue;
+}
+
+String value;
+try
+{
+// Field.get() can throw NPE if the value of the field is null
+value = field.get(this).toString();
+}
+catch (NullPointerException | IllegalAccessException npe)
+{
+value = "null";
+}
+configMap.put(name, value);
+}
+
+logger.info("Node configuration:[" + Joiner.on("; 
").join(configMap.entrySet()) + "]");
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/621f446c/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index c903775..7bcfb97 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -53,7 +53,6 @@ import org.apache.cassandra.scheduler.IRequestScheduler;
 import org.apache.cassandra.scheduler.NoScheduler;
 import 

[jira] [Commented] (CASSANDRA-11298) (windows) dtest failure in repair_tests.repair_test.TestRepairDataSystemTable.repair_table_test

[Russ Hatch (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176361#comment-15176361
 ] 

Russ Hatch commented on CASSANDRA-11298:


message:
{noformat}
ERROR [SharedPool-Worker-1] 2016-03-01 03:12:11,513 Keyspace.java:477 - 
Attempting to mutate non-existant table 5bc52802-de25-35ed-aeab-188eecebb090 
(system_auth.roles)
{noformat}

> (windows) dtest failure in 
> repair_tests.repair_test.TestRepairDataSystemTable.repair_table_test
> ---
>
> Key: CASSANDRA-11298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11298
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: DS Test Eng
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/191/testReport/repair_tests.repair_test/TestRepairDataSystemTable/repair_table_test
> Failed on CassCI build cassandra-3.0_dtest_win32 #191
> This is a singular new failure, but the error message looks suspicious and 
> worth digging into.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-11254) (windows) dtest failure in auth_test.TestAuth.restart_node_doesnt_lose_auth_data_test

[Philip Thompson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson resolved CASSANDRA-11254.
-
   Resolution: Fixed
Fix Version/s: (was: 2.2.x)

Test code problem. Fixed in https://github.com/pcmanus/ccm/pull/457

> (windows) dtest failure in 
> auth_test.TestAuth.restart_node_doesnt_lose_auth_data_test
> -
>
> Key: CASSANDRA-11254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11254
> Project: Cassandra
>  Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Philip Thompson
>  Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/194/testReport/auth_test/TestAuth/restart_node_doesnt_lose_auth_data_test
> Failed on CassCI build cassandra-2.2_dtest_win32 #194
> looks like a problem with dtest code:
> {noformat}
> code=2200 [Invalid query] message="Keyspace ks does not exist"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11298) (windows) dtest failure in repair_tests.repair_test.TestRepairDataSystemTable.repair_table_test

[Russ Hatch (JIRA)]

Russ Hatch created CASSANDRA-11298:
--

 Summary: (windows) dtest failure in 
repair_tests.repair_test.TestRepairDataSystemTable.repair_table_test
 Key: CASSANDRA-11298
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11298
 Project: Cassandra
  Issue Type: Test
Reporter: Russ Hatch
Assignee: DS Test Eng


example failure:

http://cassci.datastax.com/job/cassandra-3.0_dtest_win32/191/testReport/repair_tests.repair_test/TestRepairDataSystemTable/repair_table_test

Failed on CassCI build cassandra-3.0_dtest_win32 #191

This is a singular new failure, but the error message looks suspicious and 
worth digging into.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11217) Only log yaml config once, at startup

[Paulo Motta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paulo Motta updated CASSANDRA-11217:

Status: Ready to Commit  (was: Patch Available)

> Only log yaml config once, at startup
> -
>
> Key: CASSANDRA-11217
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11217
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration, Core
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
>
> CASSANDRA-6456 introduced a feature where the yaml is dumped in the log. At 
> startup this is a nice feature, but I see that it’s actually triggered every 
> time it handshakes with a node and fails to connect and the node happens to 
> be a seed ([see 
> here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L435]).
>  Calling {{DD.getseeds()}} calls the {{SeedProvider}}, and if you happen to 
> use {{SimpleSeedProvider}} it will reload the yaml config, and once again 
> dump it out to the log.
> It's debatable if {{DD.getseeds()}} should trigger a reload (which I added in 
> CASSANDRA-5459) or whether reloading the seeds should be a different method 
> (it probably should), but we shouldn't keep logging the yaml config on every 
> connection failure to a seed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11217) Only log yaml config once, at startup

[Paulo Motta (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176339#comment-15176339
 ] 

Paulo Motta commented on CASSANDRA-11217:
-

LGTM, marking as ready to commit. Thanks!

> Only log yaml config once, at startup
> -
>
> Key: CASSANDRA-11217
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11217
> Project: Cassandra
>  Issue Type: Bug
>  Components: Configuration, Core
>Reporter: Jason Brown
>Assignee: Jason Brown
>Priority: Minor
>
> CASSANDRA-6456 introduced a feature where the yaml is dumped in the log. At 
> startup this is a nice feature, but I see that it’s actually triggered every 
> time it handshakes with a node and fails to connect and the node happens to 
> be a seed ([see 
> here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L435]).
>  Calling {{DD.getseeds()}} calls the {{SeedProvider}}, and if you happen to 
> use {{SimpleSeedProvider}} it will reload the yaml config, and once again 
> dump it out to the log.
> It's debatable if {{DD.getseeds()}} should trigger a reload (which I added in 
> CASSANDRA-5459) or whether reloading the seeds should be a different method 
> (it probably should), but we shouldn't keep logging the yaml config on every 
> connection failure to a seed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11232) CentOS 7.1 cassandra startup - Exception in thread

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176301#comment-15176301
 ] 

Michael Shuler commented on CASSANDRA-11232:


I opened an internal ticket to update the RPM build's cassandra-env.sh that 
includes the Java version check - 
https://datastax.jira.com/browse/COMMUNITY-325 (not public).

Without the version check with the nice clear error, as long as the Java 
version is 8u40 or later, this should probably work fine, but it would be nice 
to know from the user.

> CentOS 7.1 cassandra startup - Exception in thread
> --
>
> Key: CASSANDRA-11232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11232
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ionut Chisanovici
>
> Hi,
> I'm having issues in starting cassandra30 on a CentOS 7.1 box.
> Below is the error:
> {noformat}
> ERROR 10:54:28 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
>   at 
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:95) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at org.apache.cassandra.io.util.Memory.(Memory.java:74) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:514)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:506)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:472)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:441)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:375)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:361) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_31]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_31]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_31]
> {noformat}
> OS informations:
> # cat /etc/redhat-release
> CentOS Linux release 7.1.1503 (Core)
> # java -version
> openjdk version "1.8.0_31"
> OpenJDK Runtime Environment (build 1.8.0_31-b13)
> OpenJDK 64-Bit Server VM (build 25.31-b07, mixed mode)
> Cassandra installation was done from the official datastax.repo:
> yum install dsc30 cassandra30-tools
> Thank you,
> Ionut



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11232) CentOS 7.1 cassandra startup - Exception in thread

[Michael Shuler (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176287#comment-15176287
 ] 

Michael Shuler commented on CASSANDRA-11232:


Cassandra 3.0 requires at least Java 1.8.0_40 - {{cassandra-ev.sh}} should 
throw an error that states this. Does your install work with Java >= 1.8.0_40?

{noformat}
if [ "$JVM_VERSION" \< "1.8" ] && [ "$JVM_PATCH_VERSION" \< "40" ] ; then
echo "Cassandra 3.0 and later require Java 8u40 or later."
exit 1;
fi
{noformat}

> CentOS 7.1 cassandra startup - Exception in thread
> --
>
> Key: CASSANDRA-11232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11232
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Ionut Chisanovici
>
> Hi,
> I'm having issues in starting cassandra30 on a CentOS 7.1 box.
> Below is the error:
> {noformat}
> ERROR 10:54:28 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
>   at 
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:95) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at org.apache.cassandra.io.util.Memory.(Memory.java:74) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:514)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:506)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:472)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:441)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:375)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:361) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
> ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
>  ~[guava-18.0.jar:na]
>   at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
>  ~[apache-cassandra-3.0.3.jar:3.0.3]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_31]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_31]
>   at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_31]
> {noformat}
> OS informations:
> # cat /etc/redhat-release
> CentOS Linux release 7.1.1503 (Core)
> # java -version
> openjdk version "1.8.0_31"
> OpenJDK Runtime Environment (build 1.8.0_31-b13)
> OpenJDK 64-Bit Server VM (build 25.31-b07, mixed mode)
> Cassandra installation was done from the official datastax.repo:
> yum install dsc30 cassandra30-tools
> Thank you,
> Ionut



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11232) CentOS 7.1 cassandra startup - Exception in thread

[Michael Shuler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-11232:
---
Description: 
Hi,

I'm having issues in starting cassandra30 on a CentOS 7.1 box.
Below is the error:
{noformat}
ERROR 10:54:28 Exception in thread Thread[MemtableFlushWriter:1,5,main]
java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
at 
org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:95) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at org.apache.cassandra.io.util.Memory.(Memory.java:74) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:514)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:506)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:472)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:441)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:375)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.db.Memtable$FlushRunnable.runMayThrow(Memtable.java:361) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at 
com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
 ~[guava-18.0.jar:na]
at 
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1037)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
~[na:1.8.0_31]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
~[na:1.8.0_31]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_31]
{noformat}

OS informations:
# cat /etc/redhat-release
CentOS Linux release 7.1.1503 (Core)
# java -version
openjdk version "1.8.0_31"
OpenJDK Runtime Environment (build 1.8.0_31-b13)
OpenJDK 64-Bit Server VM (build 25.31-b07, mixed mode)

Cassandra installation was done from the official datastax.repo:
yum install dsc30 cassandra30-tools

Thank you,
Ionut

  was:
Hi,

I'm having issues in starting cassandra30 on a CentOS 7.1 box.
Below is the error:
ERROR 10:54:28 Exception in thread Thread[MemtableFlushWriter:1,5,main]
java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
at 
org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:95) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at org.apache.cassandra.io.util.Memory.(Memory.java:74) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
 ~[apache-cassandra-3.0.3.jar:3.0.3]
at 
org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163) 
~[apache-cassandra-3.0.3.jar:3.0.3]
at 

[jira] [Resolved] (CASSANDRA-7842) Work out and implement an initialization order for singletons

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston resolved CASSANDRA-7842.

Resolution: Later

> Work out and implement an initialization order for singletons
> -
>
> Key: CASSANDRA-7842
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7842
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>
> Prefer final members passed into their constructor, using volatile members 
> assigned during startup where there are circular dependencies.
> We should consider splitting some of the singletons into more component 
> specific classes before this step. There is definitely a case for making that 
> another ticket, but with the non-singleton dependencies clearly defined at 
> this point, we will have a clear idea of what can be split up. Doing that 
> would reduce the number of circular dependencies between singletons, and make 
> this stage a lot easier, with the additional benefit of further modularizing 
> the code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7841) Pass static singleton instances into constructors of dependent classes

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston resolved CASSANDRA-7841.

Resolution: Later

> Pass static singleton instances into constructors of dependent classes
> --
>
> Key: CASSANDRA-7841
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7841
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>
> Identify all non-singleton usages of static state (grep for '.instance.'), 
> and refactor to pass dependencies into their constructors from their 
> instantiating services.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7840) Refactor static state & functions into static singletons

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston resolved CASSANDRA-7840.

Resolution: Later

> Refactor static state & functions into static singletons
> 
>
> Key: CASSANDRA-7840
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7840
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
> Fix For: 3.x
>
> Attachments: 
> 0001-splitting-StorageService-executors-into-a-separate-c.patch, 
> 0002-making-DatabaseDescriptor-a-singleton.patch, 
> 0003-refactoring-StorageService-static-methods.patch, 
> 0004-making-StorageProxy-a-singleton.patch, 
> 0005-making-MigrationManager-a-singleton.patch, 
> 0006-making-SystemKeyspace-a-singleton.patch, 
> 0007-making-Auth-a-singleton.patch, 
> 0008-removing-static-methods-and-initialization-from-Comp.patch, 
> 0009-making-SinkManager-a-singleton.patch, 
> 0010-making-DefsTables-a-singleton.patch, 
> 0011-making-StageManager-a-singleton.patch, 
> 0012-making-MessagingService-a-singleton.patch, 
> 0013-making-QueryProcessor-a-singleton.patch, 
> 0014-refactoring-static-methods-on-Tracing.patch, 
> 0015-removing-static-state-from-BatchlogManager.patch, 
> 0016-removing-static-method-from-CommitLog.patch, 
> 0017-OutboundTcpConnection-removing-singleton-access-from.patch, 
> 0018-FBUtilities-removing-getLocalAddress-getBroadcastAdd.patch, 
> 0019-PendingRangeCalculatorService-removing-singleton-acc.patch, 
> 0020-ActiveRepairService-removing-static-members-and-meth.patch, 
> 0021-RowDataResolver-removing-static-singleton-access-fro.patch, 
> 0022-AbstractReadExecutor-removing-static-method.patch, 
> 0023-StorageServiceAccessor-removing-static-singleton-acc.patch, 
> 0024-FileUtils-removing-static-singleton-accesses-from-st.patch, 
> 0025-ResourceWatcher-removing-static-singleton-access-fro.patch, 
> 0026-TokenMetadata-removing-static-singleton-access-from-.patch, 
> 0027-OutboundTcpConnectionPool-removing-static-singleton-.patch, 
> 0028-Cassandra-PasswordAuthenticator-making-static-method.patch, 
> 0029-CompactionMetrics-making-static-method-instance-meth.patch, 
> 0030-ClientState-splitting-configured-QueryHandler-instan.patch, 
> 0031-SSTableReader-splitting-static-factory-methods-into-.patch, 
> 0032-Keyspace-splitting-static-factory-methods-and-state-.patch, 
> 0033-ColumnFamilyStore-splitting-static-factory-methods-a.patch, 
> 0034-TriggerDefinition-removing-static-singleton-access-f.patch, 
> 0035-CFMetaData-splitting-off-static-factory-methods-onto.patch, 
> 0036-KSMetaData-splitting-off-static-factory-methods-onto.patch, 
> 0037-SystemKeyspace-moving-system-keyspace-definitions-on.patch, 
> 0038-UTMetaData-refactoring-static-singleton-accesses-for.patch, 
> 0039-CounterId-removing-static-singleton-accesses-from-st.patch, 
> 0040-AtomicBtreeColumns-replacing-SystemKeyspace-CFMetaDa.patch
>
>
> 1st step of CASSANDRA-7837.
> Things like DatabaseDescriptor.getPartitioner() should become 
> DatabaseDescriptor.instance.getPartitioner(). In cases where there is a mix 
> of instance state and static functionality (Keyspace & ColumnFamilyStore 
> classes), the static portion should be split off into singleton factory 
> classes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7843) Remove static singleton instances

[Blake Eggleston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston resolved CASSANDRA-7843.

Resolution: Later

> Remove static singleton instances
> -
>
> Key: CASSANDRA-7843
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7843
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>
> Initialize and start C* from a static method that takes a configuration as an 
> argument.
> CassandraDaemon and utilities will need to be updated, and we'll need a 
> helper method so tests can instantiate the database with a one or two liner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-11297) AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita reassigned CASSANDRA-11297:
--

Assignee: Yuki Morishita

> AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)
> -
>
> Key: CASSANDRA-11297
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11297
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Jeremiah Jordan
>Assignee: Yuki Morishita
> Fix For: 2.2.x, 3.0.x
>
>
> Looks like CASSANDRA-10859 is an issue in cassandra-2.2 and 3.0 as well, if 
> you add the -ea to the nodetool script in those you get the problem in those 
> as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11297) AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11297:
---
Component/s: Tools

> AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)
> -
>
> Key: CASSANDRA-11297
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11297
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Jeremiah Jordan
> Fix For: 2.2.x, 3.0.x
>
>
> Looks like CASSANDRA-10859 is an issue in cassandra-2.2 and 3.0 as well, if 
> you add the -ea to the nodetool script in those you get the problem in those 
> as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11297) AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)

[Yuki Morishita (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-11297:
---
Fix Version/s: 3.0.x
   2.2.x

> AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)
> -
>
> Key: CASSANDRA-11297
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11297
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Jeremiah Jordan
> Fix For: 2.2.x, 3.0.x
>
>
> Looks like CASSANDRA-10859 is an issue in cassandra-2.2 and 3.0 as well, if 
> you add the -ea to the nodetool script in those you get the problem in those 
> as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7837) Factor out static initialization / singletons

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-7837.
--
Resolution: Later

Removing all singletons should not be a standalone goal, but rather a means to 
some end. Instead of boiling the ocean - or instead letting the ticket stay 
undead - we are going to address concrete issues caused by some of the 
singletons one by one. Probably starting with CASSANDRA-10283 - eventually.

> Factor out static initialization / singletons
> -
>
> Key: CASSANDRA-7837
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7837
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>
> I've been spending some time experimenting with ways to factor out static 
> initialization and static singletons in Cassandra.
> This is a known issue, with obvious implications on testability and 
> extensibility. It also leads to subtle changes in behavior of the database 
> and tests, as access patterns to static methods reorder the initialization 
> order of the static state. For instance, the test 
> org.apache.cassandra.config.DatabaseDescriptorTest.testKSMetaDataSerialization
>  is no longer testing anything, because Schema.getKeyspaceDefinitions() is 
> now returning an empty array. Once the initialization order is changed, the 
> test fails because system keyspaces aren't being filtered out, and fromThrift 
> throws an exceptions when it sees LocalStrategy. Removing static state and 
> singletons will make testing each component in isolation easier, leading to 
> fewer regressions.
> With a large refactor like this, we'd want to avoid having the refactor 
> branch, and the trunk branch diverging too far, or for too long. After a few 
> failed attempts, I've worked out a way to perform the refactor in a way that 
> can be performed with a series of (relatively) small patches, and avoid a 
> big, scary merge. Each patch will build, the tests will pass, and can be 
> merged into trunk daily/regularly as they are completed.
> The process is as follows:
> # Condense all static state into static singletons. 
> (DatabaseDescriptor.getPartitioner() -> 
> DatabaseDescriptor.instance.getPartitioner())
> ** In some cases, classes will need to be split into pairs of instance and 
> factory classes, like in the case of Keyspace and ColumnFamilyStore
> # Identify all non-singleton usages of static state (grep for '.instance.'), 
> and refactor to pass dependencies into their constructors from their 
> instantiating services.
> # Work out and implement an initialization order for the singletons, 
> preferring final members passed into their constructor, using volatile 
> members assigned during startup where there are circular dependencies.
> ** We should consider splitting some of the singletons into more component 
> specific classes before this step. There is definitely a case for making that 
> another ticket, but with the non-singleton dependencies clearly defined at 
> this point, we will have a clear idea of what can be split up. Doing that 
> would reduce the number of circular dependencies between singletons, and make 
> this stage a lot easier, with the additional benefit of further modularizing 
> the code.
> # Remove static singleton instances. Initialize and start C* from a static 
> method that takes a configuration as an argument.
> ** CassandraDaemon and utilities will need to be updated, and we'll need a 
> helper method so tests can instantiate the database with a one or two liner.
> Obviously, this is a lot of work, but I think it's worth the effort. I've 
> already gone through a few practice runs, and it's doable. I'm happy to do 
> the work, with guidance and input from the C* team.
> Thoughts?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7850) Composite Aware Partitioner

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-7850.
--
Resolution: Duplicate

The ticket duplicates CASSANDRA-9231, which we decided to not implement for 
reasons listed in the comments section.

> Composite Aware Partitioner
> ---
>
> Key: CASSANDRA-7850
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7850
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Drew Kutcharian
>
> Since C* supports composites for partition keys, I think it'd be useful to 
> have the ability to only use first (or first few) components of the key to 
> calculate the token hash.
> A naive use case would be multi-tenancy:
> Say we have accounts and accounts have users. So we would have the following 
> tables:
> {code}
> CREATE TABLE account (
>   id timeuuid PRIMARY KEY,
>   company text
> );
> {code}
> {code}
> CREATE TABLE user (
>   id  timeuuid PRIMARY KEY, 
>   accountId timeuuid,
>   emailtext,
>   password text
> );
> {code}
> {code}
> // Get users by account
> CREATE TABLE user_account_index (
>   accountId  timeuuid,
>   userIdtimeuuid,
>   PRIMARY KEY(acid, id)
> );
> {code}
> Say we want to get all the users that belong to an account. We would first 
> have to get the results from user_account_index and then use a multi-get 
> (WHERE IN) to get the records from user table. Now this multi-get part could 
> potentially query a lot of different nodes in the cluster. It’d be great if 
> there was a way to limit storage of users of an account to a single node so 
> that way multi-get would only need to query a single node.
> With this improvement we would be able to define the user table like so:
> {code}
> CREATE TABLE user (
>   id  timeuuid, 
>   accountId timeuuid,
>   emailtext,
>   password text,
>   PRIMARY KEY(((accountId),id))  //extra parentheses
> );
> {code}
> I'm not too sure about the notation, it could be something like PRIMARY 
> KEY(((accountId),id)) where the "(accountId)" means use this part to 
> calculate the hash and ((accountId),id) is the actual partition key.
> The main complication I see with this is that we would have to use the table 
> definition when calculating hashes so we know what components of the 
> partition keys need to be used for hash calculation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7848) Additional keystore configurations for SSL with HSMs

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-7848.
--
Resolution: Not A Problem

bq. Patch to be submitted soon. Comments are welcome in the meantime.

It's been a long time now without a follow up. Are you still interested in 
providing a patch? If so and when so, feel free to reopen. Thanks!

> Additional keystore configurations for SSL with HSMs
> 
>
> Key: CASSANDRA-7848
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7848
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Hendrik van Huyssteen
>Priority: Minor
>
> In order to use Cassandra with a Hardware Security Module (HSM) for encrypted 
> communications, additional configuration options are required in terms of 
> keystore configurations. 
> A user configuring Cassandra must be able to:
> # Specify the truststore and keystore type independently (eg. keystore would 
> be in hardware and truststore in software)
> # Specify the desired certificate and private key entry that should be used, 
> by setting an alias
> # Specify the keystore and keypair passwords independently
>  
> At the moment Cassandra only allows:
> # A global keystore type
> # Expects one keypair per keystore and
> # Uses the same password for the keystore and keypair
>  
> The appropriate changes have been made to Cassandra 1.2 to support the above 
> mentioned configuration.
> The proposed cassandra.yaml would then look as follows, with the new changes 
> marked with *:
> {noformat}
> server_encryption_options:
> internode_encryption: all
> keystore: 
> keystore_password: 
> store_type: 
> *keystore_entry_alias: *
> *keystore_entry_password: *
>  
> truststore: 
> truststore_password: 
> # More advanced defaults below:
> # protocol: TLS
> *truststore_type: JKS*
> # cipher_suites: 
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA]
> {noformat}
>  
> In terms of backwards compatibility, the following defaults should be used 
> for the newly proposed settings:
> * truststore_type = store_type;
> * keystore_entry_password = keystore_password;
> * keystore_entry_alias = autoselect
> Example use case with HSM:
> * Keystore is stored in HSM.
> * store_type is set to the HSM store type.
> * keystore_password is set to the slot password of the HSM.
> * keystore_entry_password set to the keypair password.
> * Truststore is stored on disk, with type set to JKS. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7877) Tunable consistency across multiple datacenters - LOCAL_QUORUM + quorum at 1 out of N other DCs

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-7877.
--
Resolution: Duplicate

> Tunable consistency across multiple datacenters - LOCAL_QUORUM + quorum at 1 
> out of N other DCs
> ---
>
> Key: CASSANDRA-7877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7877
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Hari Sekhon
>Priority: Minor
>
> Right now tunable consistency allows for things like local quorum and each 
> quorum across multiple datacenters.
> If you have 3 or more datacenters then perhaps there should be an option to 
> do local_quorum + quorum in one other DC but not all the multiple DCs before 
> confirming write to prevent data loss on site failure?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7928) Digest queries do not require alder32 checks

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176152#comment-15176152
 ] 

Aleksey Yeschenko commented on CASSANDRA-7928:
--

[~kohlisankalp] I'm inclined to close the ticket now. What's your current take 
on it?

> Digest queries do not require alder32 checks
> 
>
> Key: CASSANDRA-7928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7928
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: sankalp kohli
>Priority: Minor
>  Labels: performance
> Fix For: 3.x
>
>
>  While reading data from sstables, C* does Alder32 checks for any data being 
> read. We have seen that this causes higher CPU usage while doing kernel 
> profiling. These checks might not be useful for digest queries as they will 
> have a different digest in case of corruption. 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7931) Single Host CPU profiling data dump

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-7931.
--
Resolution: Not A Problem

Closing the issue now as it's not exactly suggesting any actionable 
improvements. Thanks for doing the profiling though.

> Single Host CPU profiling data dump
> ---
>
> Key: CASSANDRA-7931
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7931
> Project: Cassandra
>  Issue Type: Improvement
> Environment: 2.1.0 on my local machine
>Reporter: Michael Nelson
>Priority: Minor
> Attachments: traces.txt
>
>
> At Boot Camp today I did some CPU profiling and wanted to turn in my findings 
> somehow. So here they are in a JIRA.
> I ran cassandra-stress read test against my local machine (single node: SSD, 
> 8 proc, plenty of RAM). I'm using straight 2.1.0 with the "Lightweight Java 
> Profiler" by Jeremy Manson 
> (https://code.google.com/p/lightweight-java-profiler/). This is a lower-level 
> profiler that more accurately reports what the CPU is actually doing rather 
> than where time is going.
> Attached is the report. Here are a few high points with some commentary from 
> some investigation I did today:
> 1. The number one _consumer_of_CPU_ is this stack trace:
> sun.misc.Unsafe.park(Unsafe.java:-1)
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
> org.apache.cassandra.concurrent.SEPWorker.doWaitSpin(SEPWorker.java:236)
> org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:74)
> java.lang.Thread.run(Thread.java:745)
> There is a CPU cost to going into and out of Unsafe.park() and the SEPWorkers 
> are doing it so much when they are spinning that they were the largest single 
> consumer of CPU. Note that this profiler is sampling the CPU so it isn't the 
> _blocking_ that this call does that is the issue. It is the CPU actually 
> being consumed as it prepares to block and then later comes out. This kind of 
> lower level fine grained control of threads is tricky and there appears to be 
> room for improvement here. I tried a few approaches that didn't pan out. 
> There really needs to be a way for threads to block waiting for the executor 
> queues. That way they will be immediately available (rather than waiting for 
> them to poll the executors) and will not consume CPU when they aren't needed. 
> Maybe block for some short period of time and then become available for other 
> queues after that?
> 2. Second is Netty writing to sockets. I didn't investigate this. Netty is 
> pretty optimized. Someone mentioned there are native options for Netty. 
> Probably worth trying.
> 3. Third is similar to #1:
> sun.misc.Unsafe.park(Unsafe.java:-1)
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
> org.apache.cassandra.utils.concurrent.WaitQueue$AbstractSignal.awaitUntil(WaitQueue.java:303)
>  
> org.apache.cassandra.utils.concurrent.SimpleCondition.await(SimpleCondition.java:63)
> org.apache.cassandra.service.ReadCallback.await(ReadCallback.java:90)
> org.apache.cassandra.service.ReadCallback.get(ReadCallback.java:100)
> org.apache.cassandra.service.AbstractReadExecutor.get(AbstractReadExecutor.java:144)
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1226)
> I looked at this a little. The asynchronous callback using Unsafe.park() is 
> being invoked even for local reads (like my local laptop). It only rarely 
> blocked. It just paid the CPU cost of going in and immediately coming out 
> because the read was already satisfied from the buffer cache. :-(
> 4. Fourth is Netty doing epoll. Again, worth considering Netty's native 
> optimizations to help here.
> The others are interesting but, for this little benchmark, not as big a deal. 
> For example, ArrayList.grow() shows up more than I would have expected.
> The Boot Camp was great. Feel free to contact me if I can help somehow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-11297) AssertionError in nodetool cfstats in 3.0 (Back port CASSANDRA-10859)

[Jeremiah Jordan (JIRA)]

Jeremiah Jordan created CASSANDRA-11297:
---

 Summary: AssertionError in nodetool cfstats in 3.0 (Back port 
CASSANDRA-10859)
 Key: CASSANDRA-11297
 URL: https://issues.apache.org/jira/browse/CASSANDRA-11297
 Project: Cassandra
  Issue Type: Bug
Reporter: Jeremiah Jordan


Looks like CASSANDRA-10859 is an issue in cassandra-2.2 and 3.0 as well, if you 
add the -ea to the nodetool script in those you get the problem in those as 
well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-10859) AssertionError in nodetool cfhistograms

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176115#comment-15176115
 ] 

Jeremiah Jordan edited comment on CASSANDRA-10859 at 3/2/16 6:18 PM:
-

I think cassandra-3.0 (and maybe 2.2) has this problem as well.  We should 
apply this there.

Opened CASSANDRA-11297


was (Author: jjordan):
I think cassandra-3.0 (and maybe 2.2) has this problem as well.  We should 
apply this there.

> AssertionError in nodetool cfhistograms
> ---
>
> Key: CASSANDRA-10859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10859
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.2
>
>
> {{nodetool cfhistograms}} raises an {{AssertionError}} on the CassCI 
> {{trunk}} jobs:
> http://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/jmx_test/TestJMX/cfhistograms_test/history/
> This regression started in this build on CassCI and has failed consistently 
> since then:
> http://cassci.datastax.com/job/trunk_dtest/806/testReport/junit/jmx_test/TestJMX/cfhistograms_test/
> I can't find any recent dtest changes to this method, so I believe it's a 
> Cassandra bug. Here's the changeset for the first failing build:
> http://cassci.datastax.com/job/trunk_dtest/806/changes
> Maybe something in the changes to the scripts here introduced the bug:
> https://github.com/apache/cassandra/commit/b5240204d7aa2a32c6649d19da2b961c856cde28
> [~jjordan] could you have a look at this please?
> This appears to only affect {{trunk}} at present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8098) Allow CqlInputFormat to be restricted to more than one data-center

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-8098.
--
Resolution: Won't Fix

Hadoop input/output formats are likely to move off-tree soon, and as such we 
aren't going to allocate any resources to new Hadoop-related functionality.

If you come up with a 3.x patch, however, feel free to reopen the ticket.

> Allow CqlInputFormat to be restricted to more than one data-center
> --
>
> Key: CASSANDRA-8098
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8098
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: mck
>Assignee: mck
>
> Today, using CqlInputFormat, it's only possible to 
>  - enforce data-locality to one specific data-center, or
>  - disable it by changing CL from LOCAL_ONE to ONE.
> We need a way to enforce data-locality to specific *data-centers*, and would 
> like to contribute a solution.
> Suggested ideas
>  - CqlInputFormat (gently) calls describeLocalRing against all the listed 
> connection addresses and merge the results into one masterRangeNodes list, or 
>  - changing the signature of describeLocalRing(..) to describeRings(String 
> keyspace, String[] dc) and having the job specify which DCs it will be 
> running within.
> *Long description*
> A lot has changed since CASSANDRA-2388 that has made life a lot easier with 
> integrating c* and hadoop, for example: CqlInputFormat, CL.LOCAL_ONE, 
> LimitedLocalNodeFirstLocalBalancingPolicy, vnodes, and describe_local_ring.
> When using CqlInputFormat, if you don't want to be stuck within 
> datacenter-locality you can for example change the consistency level from 
> LOCAL_ONE to ONE. That's great, but describe_local_ring + CL.LOCAL_ONE in its 
> current implementation isn't enough for us. We have multiple datacenters for 
> offline, multiple for online, because we still want the availability 
> advantages that come from aligning virtual datacenters to physical 
> datacenters for the offline stuff too. That is using hadoop for aggregation 
> purposes on top of c* doesn't always imply one can settle with an CP solution.
> Some of our jobs have their own InputFormat implementation that uses 
> describe_ring, LOCAL_ONE, and data with only replica in the offline 
> datacenters. Works very well, except the last point kinda sucks because we 
> have online clients that want to read this data and have to then do so 
> through nodes in the offline datacenters. Underlying performance 
> improvements: eg cross_node_timeout and speculative requests; have helped but 
> there's still the need to separate online and offline. If we wanted to push 
> replica out on to the online nodes, i think the best approach is for us is to 
> have to filter out those splits/locations in getRangeMap(..)
> Back to this issue we also have jobs using CqlInputFormat. Specifying 
> multiple client input addresses doesn't help take advantage of the multiple 
> offline datacenters because the Cassandra.Client only makes one call to 
> describe_local_ring, and StorageService.describeLocalRing(..) only checks 
> against its own address. It would work to have either a) CqlInputFormat call 
> describeLocalRing against all the listed connection addresses and merge the 
> results into one masterRangeNodes list, or b) something along the lines of 
> changing the signature of describeLocalRing(..) to describeRings(String 
> keyspace, String[] dc) and having the job specify which DCs it will be 
> running within.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-10859) AssertionError in nodetool cfhistograms

[Jeremiah Jordan (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-10859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176115#comment-15176115
 ] 

Jeremiah Jordan commented on CASSANDRA-10859:
-

I think cassandra-3.0 (and maybe 2.2) has this problem as well.  We should 
apply this there.

> AssertionError in nodetool cfhistograms
> ---
>
> Key: CASSANDRA-10859
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10859
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Tools
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.2
>
>
> {{nodetool cfhistograms}} raises an {{AssertionError}} on the CassCI 
> {{trunk}} jobs:
> http://cassci.datastax.com/job/trunk_dtest/lastCompletedBuild/testReport/jmx_test/TestJMX/cfhistograms_test/history/
> This regression started in this build on CassCI and has failed consistently 
> since then:
> http://cassci.datastax.com/job/trunk_dtest/806/testReport/junit/jmx_test/TestJMX/cfhistograms_test/
> I can't find any recent dtest changes to this method, so I believe it's a 
> Cassandra bug. Here's the changeset for the first failing build:
> http://cassci.datastax.com/job/trunk_dtest/806/changes
> Maybe something in the changes to the scripts here introduced the bug:
> https://github.com/apache/cassandra/commit/b5240204d7aa2a32c6649d19da2b961c856cde28
> [~jjordan] could you have a look at this please?
> This appears to only affect {{trunk}} at present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11053) COPY FROM on large datasets: fix progress report and debug performance

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176106#comment-15176106
 ] 

Adam Holmberg commented on CASSANDRA-11053:
---

+1 @2aff646, 8c798c2
Sorry I missed the date overflow on the first pass here.

> COPY FROM on large datasets: fix progress report and debug performance
> --
>
> Key: CASSANDRA-11053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: copy_from_large_benchmark.txt, 
> copy_from_large_benchmark_2.txt, parent_profile.txt, parent_profile_2.txt, 
> worker_profiles.txt, worker_profiles_2.txt
>
>
> Running COPY from on a large dataset (20G divided in 20M records) revealed 
> two issues:
> * The progress report is incorrect, it is very slow until almost the end of 
> the test at which point it catches up extremely quickly.
> * The performance in rows per second is similar to running smaller tests with 
> a smaller cluster locally (approx 35,000 rows per second). As a comparison, 
> cassandra-stress manages 50,000 rows per second under the same set-up, 
> therefore resulting 1.5 times faster. 
> See attached file _copy_from_large_benchmark.txt_ for the benchmark details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11274) cqlsh: interpret CQL type for formatting blob types

[Adam Holmberg (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176102#comment-15176102
 ] 

Adam Holmberg commented on CASSANDRA-11274:
---

+1 @20d8e58

> cqlsh: interpret CQL type for formatting blob types
> ---
>
> Key: CASSANDRA-11274
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11274
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.x
>
>
> During the development of CASSANDRA-11053 we have added changes to the cqlsh 
> formatting code so that we can format {{blob}} types correctly even if they 
> are represented as {{str}} rather than {{bytearray}}.
> At the moment we ensure {{blob}} are of type {{bytearray}} via the following 
> shortcut:
> {code}
> cassandra.cqltypes.BytesType.deserialize = staticmethod(lambda byts, 
> protocol_version: bytearray(byts))
> {code}
> After CASSANDRA-11053 is committed there will be a similar shortcut to 
> override the fast serializers implemented in cython. 
> Decoding the CQL type is safer in that it decouples cqlsh formatting from the 
> types returned by the driver deserializers but it is also unnecessary for 
> CASSANDRA-11053 performance goals and risky for older releases. 
> Therefore this ticket delivers this functionality but only on trunk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7423) Allow updating individual subfields of UDT

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176045#comment-15176045
 ] 

Tyler Hobbs commented on CASSANDRA-7423:


That makes sense -- I'm not sure why I didn't think of that yesterday.  I've 
force-pushed an update to the branch that checks for non-frozen collections 
nested within non-frozen UDTs when creating a table.

> Allow updating individual subfields of UDT
> --
>
> Key: CASSANDRA-7423
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7423
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Tupshin Harper
>Assignee: Tyler Hobbs
>  Labels: client-impacting, cql, docs-impacting
> Fix For: 3.x
>
>
> Since user defined types were implemented in CASSANDRA-5590 as blobs (you 
> have to rewrite the entire type in order to make any modifications), they 
> can't be safely used without LWT for any operation that wants to modify a 
> subset of the UDT's fields by any client process that is not authoritative 
> for the entire blob. 
> When trying to use UDTs to model complex records (particularly with nesting), 
> this is not an exceptional circumstance, this is the totally expected normal 
> situation. 
> The use of UDTs for anything non-trivial is harmful to either performance or 
> consistency or both.
> edit: to clarify, i believe that most potential uses of UDTs should be 
> considered anti-patterns until/unless we have field-level r/w access to 
> individual elements of the UDT, with individual timestamps and standard LWW 
> semantics



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9161) Add random interleaving for flush/compaction when running CQL unit tests

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175991#comment-15175991
 ] 

Benjamin Lerer commented on CASSANDRA-9161:
---

{quote}Just to clarify, we're talking about unit tests here.{quote}
I know. I was just mentioning them because of the problems we faced with 
upgrade tests due to the randomness of how the data is distributed.

{quote}To be clear, that ticket wasn't trying to suggest we randomize 
absolutely everything.{quote}

I know that a lot of thing cannot be randomize. I am absolutly not in favor of 
random testing. For flushing, the main problem that I have seen is that only 
the read or write path for memtables was tested not the one for SSTables. This 
problem is for me a problem of test coverage (we are not testing properly some 
part of the code). Personnaly, I would prefer to have a tool able to detect 
that type of issues than having some random flushes. Now, it is possible that 
you have other problems in mind which my be more difficult to discover. 



> Add random interleaving for flush/compaction when running CQL unit tests
> 
>
> Key: CASSANDRA-9161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9161
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sylvain Lebresne
>  Labels: retrospective_generated
>
> Most CQL tests don't bother flushing, which means that they overwhelmingly 
> test the memtable path and not the sstables one. A simple way to improve on 
> that would be to make {{CQLTester}} issue flushes and compactions randomly 
> between statements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7957) improve active/pending compaction monitoring

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175908#comment-15175908
 ] 

Sylvain Lebresne commented on CASSANDRA-7957:
-

bq. I see hundreds of pending compactions. What I was asking for is the 
additional information that shows what is in that list of pending compactions

"pending compactions" is probably one of our most confusing metric: a "pending 
tasks" is just a task to look for whether there is more things to do, they have 
not yet chosen what to compact. So there is just no additional information for 
those tasks to expose I'm afraid. In particular, pending tasks are only 
"blocked" by the sheer fact that we only do one compaction at a time.
To put it another way, the only time where the code decide which sstable it 
compacts together is at the very beginning of a concrete compaction, and so the 
only information we could return in your case is the sstables being compacted 
by the one compaction that is running, which as I said, is already logged at 
the beginning of said compaction.

Please note that I'm not pretending that compaction performance problems are 
easy to understand/debug, just that there isn't anything regarding the "pending 
tasks" that we don't currently expose, and exposing just the sstables involves 
in the current compaction through JMX doesn't seem all that crucial to me since 
it's already logged.

> improve active/pending compaction monitoring
> 
>
> Key: CASSANDRA-7957
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7957
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Nikolai Grigoriev
>Priority: Minor
>
> I think it might be useful to create a way to see what sstables are being 
> compacted into what new sstable. Something like an extension of "nodetool 
> compactionstats". I think it would be easier with this feature to 
> troubleshoot and understand how compactions are happening on your data. Not 
> sure how it is useful in everyday life but I could use such a feature when 
> dealing with CASSANDRA-7949.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-11207) Can not remove TTL on table with default_time_to_live

[Tyler Hobbs (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-11207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-11207:

Reviewer: Tyler Hobbs

Sure, I'll review.

> Can not remove TTL on table with default_time_to_live
> -
>
> Key: CASSANDRA-11207
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11207
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Matthieu Nantern
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> I've created a table with a default TTL:
> {code:sql}
> CREATE TABLE testmna.ndr (
> device_id text,
> event_year text,
> event_time timestamp,
> active boolean,
> PRIMARY KEY ((device_id, event_year), event_time)
> ) WITH CLUSTERING ORDER BY (event_time DESC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> AND comment = ''
> AND compaction = {'class': 
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'}
> AND compression = {'sstable_compression': 
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 600
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99.0PERCENTILE';
> {code}
> When I insert data with a "runtime TTL" (INSERT ... USING TTL 86400) 
> everything works as expected (ttl is set to 86400).
> But I can't insert data without TTL at runtime: INSERT ... USING TTL 0; does 
> not work.
> Tested on C* 2.2.4, CentOS 7



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6377) ALLOW FILTERING should allow seq scan filtering

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175880#comment-15175880
 ] 

Benjamin Lerer commented on CASSANDRA-6377:
---

I think we should propably change the type of this ticket to bug. Filtering has 
been mostly implemented in CASSANDRA-8099. The patches are just adding tests 
for it and fixing some issues with validation, compact static tables and static 
columns.  

> ALLOW FILTERING should allow seq scan filtering
> ---
>
> Key: CASSANDRA-6377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6377
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Benjamin Lerer
>  Labels: cql
> Fix For: 3.0.x, 3.x
>
>
> CREATE TABLE emp_table2 (
> empID int PRIMARY KEY,
> firstname text,
> lastname text,
> b_mon text,
> b_day text,
> b_yr text,
> );
> INSERT INTO emp_table2 (empID,firstname,lastname,b_mon,b_day,b_yr) 
>VALUES (100,'jane','doe','oct','31','1980');
> INSERT INTO emp_table2 (empID,firstname,lastname,b_mon,b_day,b_yr) 
>VALUES (101,'john','smith','jan','01','1981');
> INSERT INTO emp_table2 (empID,firstname,lastname,b_mon,b_day,b_yr) 
>VALUES (102,'mary','jones','apr','15','1982');
> INSERT INTO emp_table2 (empID,firstname,lastname,b_mon,b_day,b_yr) 
>VALUES (103,'tim','best','oct','25','1982');
>
> SELECT b_mon,b_day,b_yr,firstname,lastname FROM emp_table2 
> WHERE b_mon='oct' ALLOW FILTERING;
> Bad Request: No indexed columns present in by-columns clause with Equal 
> operator



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-9161) Add random interleaving for flush/compaction when running CQL unit tests

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175869#comment-15175869
 ] 

Sylvain Lebresne commented on CASSANDRA-9161:
-

bq. DTests are more random in nature

Just to clarify, we're talking about unit tests here.

bq. finding the problem behind a flapping test requires a lot more time that 
for a normal test failure

It doesn't have to be. It's very clear to me that if we do this we'll use a 
specific seed to initialize our random generator and that we'll log that seed 
on any error. That and some added convenience so we can pass said seed when 
running a particular test and debugging a problem won't be harder than for any 
failure. Further, CQLTester logs every statements it runs I believe, so if we 
also log the randomly added flushes/compactions, a simple inspection of the log 
would be enough to tell you how to modify the test to reproduce the issue.

bq. The most commons type of error in writting tests are

To be clear, that ticket wasn't trying to suggest we randomize absolutely 
everything. Most of what you suggest can't be easily randomized and is worth 
having specific tests for, I agree. But I've seen way more than one case where 
a problem hadn't been found because a test that could have found it wasn't 
flushing or compacting at the right time, and for a single test there can be 
many places where you could flush and compact: having a test for every 
permutation is just not doable. Adding some randomization is just a practical 
way to get testing of more interleaving over time and maybe find a few bugs 
along the way.

> Add random interleaving for flush/compaction when running CQL unit tests
> 
>
> Key: CASSANDRA-9161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9161
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sylvain Lebresne
>  Labels: retrospective_generated
>
> Most CQL tests don't bother flushing, which means that they overwhelmingly 
> test the memtable path and not the sstables one. A simple way to improve on 
> that would be to make {{CQLTester}} issue flushes and compactions randomly 
> between statements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6377) ALLOW FILTERING should allow seq scan filtering

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175867#comment-15175867
 ] 

Benjamin Lerer commented on CASSANDRA-6377:
---

I have pushed the fix for the review comments.

The tombstones were added to make sure that the filtering was working properly 
with them. I added a comment to clarify the code.

||Branch||Unit tests||DTests||
|[2.2|https://github.com/blerer/cassandra/tree/6377-2.2]|[2.2|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-6377-2.2-testall/4/]|[2.2|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-6377-2.2-dtest/4/]|
|[3.0|https://github.com/blerer/cassandra/tree/6377-3.0]|[3.0|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-6377-3.0-testall/4/]|[3.0|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-6377-3.0-dtest/4/]|
|[trunk|https://github.com/blerer/cassandra/tree/6377-trunk]|[trunk|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-6377-trunk-testall/5/]|[trunk|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-6377-trunk-dtest/6/]|

> ALLOW FILTERING should allow seq scan filtering
> ---
>
> Key: CASSANDRA-6377
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6377
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Benjamin Lerer
>  Labels: cql
> Fix For: 3.0.x, 3.x
>
>
> CREATE TABLE emp_table2 (
> empID int PRIMARY KEY,
> firstname text,
> lastname text,
> b_mon text,
> b_day text,
> b_yr text,
> );
> INSERT INTO emp_table2 (empID,firstname,lastname,b_mon,b_day,b_yr) 
>VALUES (100,'jane','doe','oct','31','1980');
> INSERT INTO emp_table2 (empID,firstname,lastname,b_mon,b_day,b_yr) 
>VALUES (101,'john','smith','jan','01','1981');
> INSERT INTO emp_table2 (empID,firstname,lastname,b_mon,b_day,b_yr) 
>VALUES (102,'mary','jones','apr','15','1982');
> INSERT INTO emp_table2 (empID,firstname,lastname,b_mon,b_day,b_yr) 
>VALUES (103,'tim','best','oct','25','1982');
>
> SELECT b_mon,b_day,b_yr,firstname,lastname FROM emp_table2 
> WHERE b_mon='oct' ALLOW FILTERING;
> Bad Request: No indexed columns present in by-columns clause with Equal 
> operator



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11258) Repair scheduling - Resource locking API

[Marcus Olsson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175849#comment-15175849
 ] 

Marcus Olsson commented on CASSANDRA-11258:
---

cc [~pauloricardomg]

bq. All data centers involved in a repair must be available for a repair to 
start/succeed, so if we make the lock resource dc-aware and try to create the 
lock by contacting a node in each involved data center with LOCAL_SERIAL 
consistency that should be sufficient to ensure correctness without the need 
for a global lock. This will also play along well with both dc_parallelism 
global option and with the --local or --dcs table repair options.

As it is right now I don't think it's possible to forward a LWT query to 
another node (at least not through the MessagingService). I guess we wouldn't 
want to do the access through the native layer since that would require that we 
have auth and encryption configured. So I think there are a couple of options 
here:
# More expressive consistency levels, perhaps CASSANDRA-8119? (e.g. "QUORUM in 
DC1")
# Make it possible to forward a LWT requests and have a node in the other dc 
act as a coordinator for the request.
# Extend the current LWT implementation to be able to coordinate "LOCAL_SERIAL" 
in a remote dc.
# Create separate lock keyspaces for each dc.

I think that 3 & 4 should be possible to implement directly but 1 & 2 would 
probably require a protocol change. It should be possible to start by extending 
the current LWT implementation and then switch to either 1 or 2 later on if 
needed.

WDYT?

> Repair scheduling - Resource locking API
> 
>
> Key: CASSANDRA-11258
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11258
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Minor
>
> Create a resource locking API & implementation that is able to lock a 
> resource in a specified data center. It should handle priorities to avoid 
> node starvation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7957) improve active/pending compaction monitoring

[Nikolai Grigoriev (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175833#comment-15175833
 ] 

Nikolai Grigoriev commented on CASSANDRA-7957:
--

I was talking specifically about the *current* status. That's why I mentioned 
"nodetool compactionstats". I think that in case of Leveled Compaction we could 
expose a bit more data to make nodetool more informative.

Here is the use case. I look at "nodetool compactionstats" and I see that my 
compaction process is doing almost nothing, like doing one compaction. At the 
same time, I see hundreds of pending compactions. What I was asking for is the 
additional information that shows what is in that list of pending compactions 
and what is blocked by what. I do not think that with the default log settings 
you can get this info today easily.

If you look at CASSANDRA-7949 you will probably better understand why I was 
looking for this information.

> improve active/pending compaction monitoring
> 
>
> Key: CASSANDRA-7957
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7957
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Nikolai Grigoriev
>Priority: Minor
>
> I think it might be useful to create a way to see what sstables are being 
> compacted into what new sstable. Something like an extension of "nodetool 
> compactionstats". I think it would be easier with this feature to 
> troubleshoot and understand how compactions are happening on your data. Not 
> sure how it is useful in everyday life but I could use such a feature when 
> dealing with CASSANDRA-7949.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7839) Support standard EC2 naming conventions in Ec2Snitch

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-7839:
---
Reviewer: Paulo Motta  (was: Brandon Williams)

> Support standard EC2 naming conventions in Ec2Snitch
> 
>
> Key: CASSANDRA-7839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7839
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Gregory Ramsperger
>Assignee: Gregory Ramsperger
> Attachments: conditionally-use-full-Amazon-style-naming-for-dc.patch
>
>
> The EC2 snitches use datacenter and rack naming conventions inconsistent with 
> those presented in Amazon EC2 APIs as region and availability zone. A 
> discussion of this is found in CASSANDRA-4026. This has not been changed for 
> valid backwards compatibility reasons. Using SnitchProperties, it is possible 
> to switch between the legacy naming and the full, AWS-style naming. 
> Proposal:
> * introduce a property (ec2_naming_scheme) to switch naming schemes.
> * default to current/legacy naming scheme
> * add support for a new scheme ("full") which is consistent AWS conventions
> ** data centers will be the region name, including the number
> ** racks will be the availability zone name, including the region name
> Examples:
> * * legacy* : datacenter is the part of the availability zone name preceding 
> the last "\-" when the zone ends in \-1 and includes the number if not \-1. 
> Rack is the portion of the availability zone name following  the last "\-".
> ** us-west-1a => dc: us-west, rack: 1a
> ** us-west-2b => dc: us-west-2, rack: 2b; 
> * *full* : datacenter is the part of the availability zone name preceding 
> zone letter. rack is the entire availability zone name.
> ** us-west-1a => dc: us-west-1, rack: us-west-1a
> ** us-west-2b => dc: us-west-2, rack: us-west-2b; 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-9161) Add random interleaving for flush/compaction when running CQL unit tests

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175823#comment-15175823
 ] 

Benjamin Lerer edited comment on CASSANDRA-9161 at 3/2/16 3:51 PM:
---

I am not really in favor of random operations. DTests are more random in nature 
and finding the problem behind a flapping test requires a lot more time that 
for a normal test failure. I honestly prefer writing more tests but having them 
deterministic.

My experience with CQL is that we usually forgot to test a certain amount of 
use cases. 

The most commons type of error in writting tests are:
# forgetting to test the different types of tables:
#* CQL table without clustering column
#* CQL table with clustering columns
#* Compact table without clustering columns 
#* Compact table with clustering columns
# forgetting to test with static columns. A use case which is often forgotten 
is the case of partitions containing static data but no rows. For such a case 
it is important to make sure that the partition with only static data is in the 
set being queried.
# if collection are involved, forgetting to test all the possible types (List, 
Set, Maps, Tuples and UDFs) when they are frozen and non-frozen
# if the processing of data can be different when the data is read from 
Memtables or SSTables forgetting to test with and without flush
# for paging not testing all the possible use cases:
#* Range queries (e.g. {{SELECT * FROM myTable}})
#* Range query with LIMIT  (e.g. {{SELECT * FROM myTable LIMIT 3}})
#* Range query with DISTINCT (e.g. {{SELECT DISTINCT pk, s  FROM myTable}})
#* Range query with DISTINCT and LIMIT  (e.g. {{SELECT DISTINCT pk, s  FROM 
myTable LIMIT 3}})
#* Range query with ORDER BY (should always be invalid)
#* Single partition queries (e.g. {{SELECT * FROM myTable WHERE pk = 1}})
#* Single partition queries with LIMIT (e.g. {{SELECT * FROM myTable WHERE pk = 
1 LIMIT 3}})
#* Single partition queries  with DISTINCT (e.g. {{SELECT DISTINCT pk, s FROM 
myTable WHERE pk = 1}})
#* Single partition queries  with DISTINCT and LIMIT (e.g. {{SELECT DISTINCT 
pk, s FROM myTable WHERE pk = 1 LIMIT 3}})
#* Single partition queries with ORDER BY (e.g. {{SELECT * FROM myTable WHERE 
pk = 1 ORDER BY clustering1 DESC}})
#* Single partition queries with ORDER BY and LIMIT (e.g. {{SELECT * FROM 
myTable WHERE pk = 1 ORDER BY clustering1 DESC LIMIT 3}})
#* Multi-partitions queries (e.g. {{SELECT * FROM myTable WHERE pk IN (1, 2)}})
#* Multi-partitions queries with LIMIT (e.g. {{SELECT * FROM myTable WHERE pk 
IN (1, 2) LIMIT 3}})
#* Multi-partitions queries  with DISTINCT (e.g. {{SELECT DISTINCT pk, s FROM 
myTable WHERE  pk IN (1, 2)}})
#* Multi-partitions queries  with DISTINCT and LIMIT (e.g. {{SELECT DISTINCT 
pk, s FROM myTable WHERE pk IN (1, 2) LIMIT 3}})
For paging the tests should be written as DTests with multiple nodes in order 
to also check the serialization protocol used between the nodes and the non 
local path. The tests should use page sizes smaller and greater than the 
dataset. Testing with a paging of 1 is highly recommanded as it makes sure that 
the last page return a full page.
# forgetting to test some invalid conditions. I found that this one is the 
harder to test properly. There are so many wrong type of inputs that it is 
difficult to not miss some.





was (Author: blerer):
I am not really in favor of random operations. DTests are more random in nature 
and finding the problem behind a flapping test require a lot more time that for 
a normal test failure. I honnestly prefer writing more tests but having them 
deterministic.

My experience with CQL is that we usually forgot to test a certain amount of 
use cases. 

The most commons type of error in writting tests are:
# forgetting to test the different types of tables:
#* CQL table without clustering column
#* CQL table with clustering columns
#* Compact table without clustering columns 
#* Compact table with clustering columns
# forgetting to test with static columns. A use case which is often forgotten 
is the case of partitions containing static data but no rows. For such a case 
it is important to make sure that the partition with only static data is in the 
set being queried.
# if collection are involved, forgetting to test all the possible types (List, 
Set, Maps, Tuples and UDFs) when they are frozen and non-frozen
# if the processing of data can be different when the data is read from 
Memtables or SSTables forgetting to test with and without flush
# for paging not testing all the possible use cases:
#* Range queries (e.g. {{SELECT * FROM myTable}})
#* Range query with LIMIT  (e.g. {{SELECT * FROM myTable LIMIT 3}})
#* Range query with DISTINCT (e.g. {{SELECT DISTINCT pk, s  FROM myTable}})
#* Range query with DISTINCT and LIMIT  (e.g. {{SELECT DISTINCT pk, s  FROM 
myTable LIMIT 3}})
#* Range query with ORDER BY (should 

[jira] [Commented] (CASSANDRA-9161) Add random interleaving for flush/compaction when running CQL unit tests

[Benjamin Lerer (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175823#comment-15175823
 ] 

Benjamin Lerer commented on CASSANDRA-9161:
---

I am not really in favor of random operations. DTests are more random in nature 
and finding the problem behind a flapping test require a lot more time that for 
a normal test failure. I honnestly prefer writing more tests but having them 
deterministic.

My experience with CQL is that we usually forgot to test a certain amount of 
use cases. 

The most commons type of error in writting tests are:
# forgetting to test the different types of tables:
#* CQL table without clustering column
#* CQL table with clustering columns
#* Compact table without clustering columns 
#* Compact table with clustering columns
# forgetting to test with static columns. A use case which is often forgotten 
is the case of partitions containing static data but no rows. For such a case 
it is important to make sure that the partition with only static data is in the 
set being queried.
# if collection are involved, forgetting to test all the possible types (List, 
Set, Maps, Tuples and UDFs) when they are frozen and non-frozen
# if the processing of data can be different when the data is read from 
Memtables or SSTables forgetting to test with and without flush
# for paging not testing all the possible use cases:
#* Range queries (e.g. {{SELECT * FROM myTable}})
#* Range query with LIMIT  (e.g. {{SELECT * FROM myTable LIMIT 3}})
#* Range query with DISTINCT (e.g. {{SELECT DISTINCT pk, s  FROM myTable}})
#* Range query with DISTINCT and LIMIT  (e.g. {{SELECT DISTINCT pk, s  FROM 
myTable LIMIT 3}})
#* Range query with ORDER BY (should always be invalid)
#* Single partition queries (e.g. {{SELECT * FROM myTable WHERE pk = 1}})
#* Single partition queries with LIMIT (e.g. {{SELECT * FROM myTable WHERE pk = 
1 LIMIT 3}})
#* Single partition queries  with DISTINCT (e.g. {{SELECT DISTINCT pk, s FROM 
myTable WHERE pk = 1}})
#* Single partition queries  with DISTINCT and LIMIT (e.g. {{SELECT DISTINCT 
pk, s FROM myTable WHERE pk = 1 LIMIT 3}})
#* Single partition queries with ORDER BY (e.g. {{SELECT * FROM myTable WHERE 
pk = 1 ORDER BY clustering1 DESC}})
#* Single partition queries with ORDER BY and LIMIT (e.g. {{SELECT * FROM 
myTable WHERE pk = 1 ORDER BY clustering1 DESC LIMIT 3}})
#* Multi-partitions queries (e.g. {{SELECT * FROM myTable WHERE pk IN (1, 2)}})
#* Multi-partitions queries with LIMIT (e.g. {{SELECT * FROM myTable WHERE pk 
IN (1, 2) LIMIT 3}})
#* Multi-partitions queries  with DISTINCT (e.g. {{SELECT DISTINCT pk, s FROM 
myTable WHERE  pk IN (1, 2)}})
#* Multi-partitions queries  with DISTINCT and LIMIT (e.g. {{SELECT DISTINCT 
pk, s FROM myTable WHERE pk IN (1, 2) LIMIT 3}})
For paging the tests should be written as DTests with multiple nodes in order 
to also check the serialization protocol used between the nodes and the non 
local path. The tests should use page sizes smaller and greater than the 
dataset. Testing with a paging of 1 is highly recommanded as it makes sure that 
the last page return a full page.
# forgetting to test some invalid conditions. I found that this one is the 
harder to test properly. There are so many wrong type of inputs that it is 
difficult to not miss some.




> Add random interleaving for flush/compaction when running CQL unit tests
> 
>
> Key: CASSANDRA-9161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9161
> Project: Cassandra
>  Issue Type: Test
>Reporter: Sylvain Lebresne
>  Labels: retrospective_generated
>
> Most CQL tests don't bother flushing, which means that they overwhelmingly 
> test the memtable path and not the sstables one. A simple way to improve on 
> that would be to make {{CQLTester}} issue flushes and compactions randomly 
> between statements.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7902) Introduce the ability to ignore RR based on consistency

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-7902:
---
Summary: Introduce the ability to ignore RR based on consistency  (was: 
Introduce the ability to ignore RR based on consistencfy)

> Introduce the ability to ignore RR based on consistency
> ---
>
> Key: CASSANDRA-7902
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7902
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Brandon Williams
>  Labels: docs-impacting
> Fix For: 3.x
>
>
> There exists a case for LOCAL_* consistency levels where you really want them 
> *local only*.  This implies that you don't ever want to do cross-dc RR, but 
> do for other levels.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7957) improve active/pending compaction monitoring

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175801#comment-15175801
 ] 

Sylvain Lebresne commented on CASSANDRA-7957:
-

I believe we log this information, isn't it enough?

> improve active/pending compaction monitoring
> 
>
> Key: CASSANDRA-7957
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7957
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Nikolai Grigoriev
>Priority: Minor
>
> I think it might be useful to create a way to see what sstables are being 
> compacted into what new sstable. Something like an extension of "nodetool 
> compactionstats". I think it would be easier with this feature to 
> troubleshoot and understand how compactions are happening on your data. Not 
> sure how it is useful in everyday life but I could use such a feature when 
> dealing with CASSANDRA-7949.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8094) Heavy writes in RangeSlice read requests

[Sylvain Lebresne (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-8094:

Labels: lhf  (was: )

> Heavy writes in RangeSlice read  requests 
> --
>
> Key: CASSANDRA-8094
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8094
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Minh Do
>Assignee: Minh Do
>  Labels: lhf
> Fix For: 2.1.x
>
>
> RangeSlice requests always do a scheduled read repair when coordinators try 
> to resolve replicas' responses no matter read_repair_chance is set or not.
> Because of this, in low writes and high reads clusters, there are very high 
> write requests going on between nodes.  
> We should have an option to turn this off and this can be different than the 
> read_repair_chance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8148) sstableloader needs ability to stream to private IP

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-8148:
-
Priority: Minor  (was: Major)

> sstableloader needs ability to stream to private IP
> ---
>
> Key: CASSANDRA-8148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8148
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Yuki Morishita
>Priority: Minor
>
> sstableloader gets where to stream from the contacting node, but destinations 
> returned are all broadcast address.
> It is nice if we can somehow tell sstableloader to stream to private IP 
> instead.
> To do this, we have to find the way to load cassandra topology to 
> sstableloader.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8148) sstableloader needs ability to stream to private IP

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-8148:
-
Fix Version/s: (was: 2.1.x)

> sstableloader needs ability to stream to private IP
> ---
>
> Key: CASSANDRA-8148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8148
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Yuki Morishita
>
> sstableloader gets where to stream from the contacting node, but destinations 
> returned are all broadcast address.
> It is nice if we can somehow tell sstableloader to stream to private IP 
> instead.
> To do this, we have to find the way to load cassandra topology to 
> sstableloader.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8148) sstableloader needs ability to stream to private IP

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-8148:
-
Component/s: Tools

> sstableloader needs ability to stream to private IP
> ---
>
> Key: CASSANDRA-8148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8148
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Yuki Morishita
>Priority: Minor
>
> sstableloader gets where to stream from the contacting node, but destinations 
> returned are all broadcast address.
> It is nice if we can somehow tell sstableloader to stream to private IP 
> instead.
> To do this, we have to find the way to load cassandra topology to 
> sstableloader.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8217) Fail snapshot based repair if Merkle tree does not use the same snapshot

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-8217:
-
Component/s: Streaming and Messaging

> Fail snapshot based repair if Merkle tree does not use the same snapshot
> 
>
> Key: CASSANDRA-8217
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8217
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Streaming and Messaging
>Reporter: sankalp kohli
>Assignee: Paulo Motta
>Priority: Minor
>
> Snapshot based repair rely on having a snapshot on each replica. If for some 
> reason, snapshot is not found or is different, we should fail the repair. 
> This will avoid streaming lot of data around.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8150) Revaluate Default JVM tuning parameters

[Joshua McKenzie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-8150:
---
Assignee: (was: Ryan McGuire)

> Revaluate Default JVM tuning parameters
> ---
>
> Key: CASSANDRA-8150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8150
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Configuration
>Reporter: Matt Stump
> Attachments: upload.png
>
>
> It's been found that the old twitter recommendations of 100m per core up to 
> 800m is harmful and should no longer be used.
> Instead the formula used should be 1/3 or 1/4 max heap with a max of 2G. 1/3 
> or 1/4 is debatable and I'm open to suggestions. If I were to hazard a guess 
> 1/3 is probably better for releases greater than 2.1.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8186) Revisit defaults for max rpc threads

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-8186.
--
Resolution: Won't Fix

Closing as {{Won't Fix}} as it's easily workaroundable, and Thrift is on its 
way out.

> Revisit defaults for max rpc threads
> 
>
> Key: CASSANDRA-8186
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8186
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jeremy Hanna
>
> Currently the rpc max threads defaults to unlimited.  The dangers of doing 
> that in certain circumstances were discussed on CASSANDRA-8116 and a warning 
> was added to the yaml.
> I've worked with two clusters in the last couple of weeks that has had the 
> heap full of rpc threads for this reason.  I'm wondering if we should set a 
> more reasonable default so that people don't hurt themselves unnecessarily.
> Is there a reason to not default this to perhaps 2048, what's commented out 
> in the yaml?  We could have a yaml comment about it as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-11086) trunk eclipse-warnings

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-11086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175751#comment-15175751
 ] 

Jason Brown commented on CASSANDRA-11086:
-

OK, added annotations for all of SASI.

|| trunk ||
| [branch|https://github.com/jasobrown/cassandra/tree/11086_trunk] |
| 
[testall|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-11086_trunk-testall/]
 |
| 
[dtest|http://cassci.datastax.com/view/Dev/view/jasobrown/job/jasobrown-11086_trunk-dtest/]
 |


> trunk eclipse-warnings
> --
>
> Key: CASSANDRA-11086
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11086
> Project: Cassandra
>  Issue Type: Bug
>  Components: Testing
>Reporter: Michael Shuler
>Assignee: Jason Brown
>Priority: Minor
> Fix For: 3.x
>
>
> REF = origin/trunk 
> COMMIT = 7230a66318ce8add742959d095900d5870689f0c
> {noformat}
> # 1/27/16 8:50:25 PM UTC
> # Eclipse Compiler for Java(TM) v20150120-1634, 3.10.2, Copyright IBM Corp 
> 2000, 2013. All rights reserved.
> --
> 1. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/metrics/TableMetrics.java
>  (at line 338)
>   return 
> computeCompressionRatio(Iterables.concat(Iterables.transform(Keyspace.all(),
>  ^^^
> The method computeCompressionRatio(Iterable) in the type 
> TableMetrics is not applicable for the arguments (Iterable)
> --
> --
> 2. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/net/OutboundTcpConnectionPool.java
>  (at line 141)
>   return channel.socket();
>   
> Potential resource leak: 'channel' may not be closed at this location
> --
> --
> 3. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/index/sasi/disk/OnDiskBlock.java
>  (at line 93)
>   return cast(dup);
>   ^
> Potential resource leak: 'dup' may not be closed at this location
> --
> --
> 4. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
>  (at line 142)
>   indexFile = new MappedBuffer(new ChannelProxy(indexPath, 
> backingFile.getChannel()));
>
> ^
> Potential resource leak: '' may not be closed
> --
> 5. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
>  (at line 161)
>   throw new FSReadError(e, index);
>   
> Potential resource leak: 'backingFile' may not be closed at this location
> --
> 6. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndex.java
>  (at line 249)
>   RangeIterator range = searchRange(e);
>  ^
> Potential resource leak: 'range' may not be closed
> --
> --
> 7. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/index/sasi/disk/OnDiskIndexBuilder.java
>  (at line 286)
>   throw new FSWriteError(e, file);
>   
> Potential resource leak: 'out' may not be closed at this location
> --
> --
> 8. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/index/sasi/disk/PerSSTableIndexWriter.java
>  (at line 284)
>   OnDiskIndex last = scheduleSegmentFlush(false).call();
>   
> Potential resource leak: 'last' may not be closed
> --
> 9. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/index/sasi/disk/PerSSTableIndexWriter.java
>  (at line 293)
>   OnDiskIndex part = Futures.getUnchecked(f);
>   
> Potential resource leak: 'part' may not be closed
> --
> --
> 10. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/index/sasi/TermIterator.java
>  (at line 118)
>   RangeIterator keyIterator = index.search(e);
>  ^^^
> Potential resource leak: 'keyIterator' may not be closed
> --
> 11. ERROR in 
> /var/lib/jenkins/jobs/trunk_eclipse-warnings/workspace/src/java/org/apache/cassandra/index/sasi/TermIterator.java
>  (at line 148)
>   return ranges == null ? null : new TermIterator(e, ranges, 
> referencedIndexes);
>   
> ^^
> Potential resource leak: 'ranges' may not be closed at this 

[jira] [Resolved] (CASSANDRA-8195) Tools for discovering problem/largest partitions

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-8195.
--
Resolution: Duplicate

> Tools for discovering problem/largest partitions
> 
>
> Key: CASSANDRA-8195
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8195
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Matt Stump
>
> A very common problem is that a customer will have one or two very hot 
> partitions. This could be a result of an unforeseen usage pattern or a 
> problem user. Tracking down the offending parition/row currently involves a 
> linear scan of the data. 
> We should keep stats on the largest N partitions encountered and make those 
> visible via nodetool as we do with cfhistograms. A reasonable suggestion for 
> N is 10, meaning the top 10 partition keys and their size would be visible. 
> We should also probably keep this info as part of the sstable metadata.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)