[jira] [Comment Edited] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh

[ZhaoYang (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081143#comment-17081143
 ] 

ZhaoYang edited comment on CASSANDRA-15660 at 4/11/20, 5:37 AM:


[CI|https://circleci.com/workflow-run/8e4b4739-cb7c-40ad-a24d-8d7c0027eb4b] 
looks good


was (Author: jasonstack):
[CI](https://circleci.com/workflow-run/8e4b4739-cb7c-40ad-a24d-8d7c0027eb4b) 
looks good

> Unable to specify -e/--execute flag in cqlsh
> 
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a 
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file 
> and use "-f" flag.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15657) Improve zero-copy-streaming containment check by using file sections

[ZhaoYang (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081144#comment-17081144
 ] 

ZhaoYang edited comment on CASSANDRA-15657 at 4/11/20, 5:37 AM:


[CI|https://circleci.com/workflow-run/a36ff018-cbb5-455f-88bb-40bfd1ca4cea] 
looks good...


was (Author: jasonstack):
[CI](https://circleci.com/workflow-run/a36ff018-cbb5-455f-88bb-40bfd1ca4cea) 
looks good...

> Improve zero-copy-streaming containment check by using file sections
> 
>
> Key: CASSANDRA-15657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15657
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Streaming and Messaging
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0
>
>
> Currently zero copy streaming is only enabled for leveled-compaction strategy 
> and it checks if all keys in the sstables are included in the transferred 
> ranges.
> This is very inefficient. The containment check can be improved by checking 
> if transferred sections (the transferred file positions) cover entire sstable.
> I also enabled ZCS for all compaction strategies since the new containment 
> check is very fast..



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh

[ZhaoYang (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081143#comment-17081143
 ] 

ZhaoYang edited comment on CASSANDRA-15660 at 4/11/20, 5:21 AM:


[CI](https://circleci.com/workflow-run/8e4b4739-cb7c-40ad-a24d-8d7c0027eb4b) 
looks good


was (Author: jasonstack):
[CI]([https://circleci.com/workflow-run/8e4b4739-cb7c-40ad-a24d-8d7c0027eb4b]) 
looks good

> Unable to specify -e/--execute flag in cqlsh
> 
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a 
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file 
> and use "-f" flag.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15657) Improve zero-copy-streaming containment check by using file sections

[ZhaoYang (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081144#comment-17081144
 ] 

ZhaoYang commented on CASSANDRA-15657:
--

[CI](https://circleci.com/workflow-run/a36ff018-cbb5-455f-88bb-40bfd1ca4cea) 
looks good...

> Improve zero-copy-streaming containment check by using file sections
> 
>
> Key: CASSANDRA-15657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15657
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Streaming and Messaging
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0
>
>
> Currently zero copy streaming is only enabled for leveled-compaction strategy 
> and it checks if all keys in the sstables are included in the transferred 
> ranges.
> This is very inefficient. The containment check can be improved by checking 
> if transferred sections (the transferred file positions) cover entire sstable.
> I also enabled ZCS for all compaction strategies since the new containment 
> check is very fast..



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh

[ZhaoYang (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081143#comment-17081143
 ] 

ZhaoYang commented on CASSANDRA-15660:
--

[CI]([https://circleci.com/workflow-run/8e4b4739-cb7c-40ad-a24d-8d7c0027eb4b]) 
looks good

> Unable to specify -e/--execute flag in cqlsh
> 
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a 
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file 
> and use "-f" flag.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15394) Remove list iterators

[Blake Eggleston (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-15394:

Source Control Link: 
https://github.com/apache/cassandra/commit/543bbdd9b630a77cff40cc266b427946688805ed
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

committed to trunk as 
[543bbdd9b630a77cff40cc266b427946688805ed|https://github.com/apache/cassandra/commit/543bbdd9b630a77cff40cc266b427946688805ed]
 

> Remove list iterators
> -
>
> Key: CASSANDRA-15394
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15394
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0
>
>
> We allocate list iterators in several places in hot paths. This converts them 
> to get by index. This provides a ~4% improvement in relvant workloads.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Replace array iterators with get by index

[bdeggleston]

This is an automated email from the ASF dual-hosted git repository.

bdeggleston pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 543bbdd  Replace array iterators with get by index
543bbdd is described below

commit 543bbdd9b630a77cff40cc266b427946688805ed
Author: Blake Eggleston 
AuthorDate: Thu Oct 17 15:37:20 2019 -0700

Replace array iterators with get by index

Patch by Blake Eggleston; Reviewed by Benedict Elliott Smith for 
CASSANDRA-15394
---
 CHANGES.txt  |  1 +
 .../cassandra/db/compaction/AbstractCompactionStrategy.java  | 12 +++-
 .../apache/cassandra/db/compaction/CompactionIterator.java   |  8 ++--
 src/java/org/apache/cassandra/db/rows/EncodingStats.java |  3 ++-
 src/java/org/apache/cassandra/db/rows/Row.java   |  7 ---
 src/java/org/apache/cassandra/utils/MergeIterator.java   |  3 ++-
 6 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/CHANGES.txt b/CHANGES.txt
index c1e5aeb..dc04d30 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0-alpha4
+ * Replace array iterators with get by index (CASSANDRA-15394)
  * Minimize BTree iterator allocations (CASSANDRA-15389)
  * Add client request size server metrics (CASSANDRA-15704)
  * Add additional logging around FileUtils and compaction leftover cleanup 
(CASSANDRA-15705)
diff --git 
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
index ad494b1..30b4cb8 100644
--- 
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java
@@ -316,8 +316,8 @@ public abstract class AbstractCompactionStrategy
 public long getTotalBytesScanned()
 {
 long bytesScanned = 0L;
-for (ISSTableScanner scanner : scanners)
-bytesScanned += scanner.getBytesScanned();
+for (int i=0, isize=scanners.size(); i 
versions)
 {
 int merged = 0;
-for (UnfilteredRowIterator iter : versions)
+for (int i=0, isize=versions.size(); i
 if (column.isSimple())
 {
 Cell merged = null;
-for (ColumnData data : versions)
+for (int i=0, isize=versions.size(); i
 complexBuilder.newColumn(column);
 complexCells.clear();
 DeletionTime complexDeletion = DeletionTime.LIVE;
-for (ColumnData data : versions)
+for (int i=0, isize=versions.size(); i extends 
AbstractIterator implem
 
 public void close()
 {
-for (Iterator iterator : this.iterators)
+for (int i=0, isize=iterators.size(); i iterator = iterators.get(i);
 try
 {
 if (iterator instanceof AutoCloseable)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15389) Minimize BTree iterator allocations

[Blake Eggleston (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-15389:

Source Control Link: 
https://github.com/apache/cassandra/commit/8576e769d13b3e887ea604074641fd4c42af5e8a
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

thanks, committed as 
[8576e769d13b3e887ea604074641fd4c42af5e8a|https://github.com/apache/cassandra/commit/8576e769d13b3e887ea604074641fd4c42af5e8a]

> Minimize BTree iterator allocations
> ---
>
> Key: CASSANDRA-15389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15389
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0
>
>
> Allocations of BTree iterators contribute a lot amount of garbage to the 
> compaction and read paths.
> This patch removes most btree iterator allocations on hot paths by:
>  • using Row#apply where appropriate on frequently called methods 
> (Row#digest, Row#validateData
>  • adding BTree accumulate method. Like the apply method, this method walks 
> the btree with a function that takes and returns a long argument, this 
> eliminates iterator allocations without adding helper object allocations 
> (BTreeRow#hasComplex, BTreeRow#hasInvalidDeletions, BTreeRow#dataSize, 
> BTreeRow#unsharedHeapSizeExcludingData, Rows#collectStats, 
> UnfilteredSerializer#serializedRowBodySize) as well as eliminating the 
> allocation of helper objects in places where apply was used previously^[1]^.
>  • Create map of columns in SerializationHeader, this lets us avoid 
> allocating a btree search iterator for each row we serialize.
> These optimizations reduce garbage created during compaction by up to 13.5%
>  
> [1] the memory test does measure memory allocated by lambdas capturing objects



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15389) Minimize BTree iterator allocations

[Blake Eggleston (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-15389:

Status: Ready to Commit  (was: Changes Suggested)

> Minimize BTree iterator allocations
> ---
>
> Key: CASSANDRA-15389
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15389
> Project: Cassandra
>  Issue Type: Sub-task
>  Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0
>
>
> Allocations of BTree iterators contribute a lot amount of garbage to the 
> compaction and read paths.
> This patch removes most btree iterator allocations on hot paths by:
>  • using Row#apply where appropriate on frequently called methods 
> (Row#digest, Row#validateData
>  • adding BTree accumulate method. Like the apply method, this method walks 
> the btree with a function that takes and returns a long argument, this 
> eliminates iterator allocations without adding helper object allocations 
> (BTreeRow#hasComplex, BTreeRow#hasInvalidDeletions, BTreeRow#dataSize, 
> BTreeRow#unsharedHeapSizeExcludingData, Rows#collectStats, 
> UnfilteredSerializer#serializedRowBodySize) as well as eliminating the 
> allocation of helper objects in places where apply was used previously^[1]^.
>  • Create map of columns in SerializationHeader, this lets us avoid 
> allocating a btree search iterator for each row we serialize.
> These optimizations reduce garbage created during compaction by up to 13.5%
>  
> [1] the memory test does measure memory allocated by lambdas capturing objects



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Minimize BTree iterator allocations

[bdeggleston]

This is an automated email from the ASF dual-hosted git repository.

bdeggleston pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 8576e76  Minimize BTree iterator allocations
8576e76 is described below

commit 8576e769d13b3e887ea604074641fd4c42af5e8a
Author: Blake Eggleston 
AuthorDate: Tue Oct 15 20:10:42 2019 -0700

Minimize BTree iterator allocations

Patch by Blake Eggleston; Reviewed by Benedict Elliott Smith for 
CASSANDRA-15389
---
 CHANGES.txt|   1 +
 src/java/org/apache/cassandra/db/ColumnIndex.java  |   6 +-
 src/java/org/apache/cassandra/db/Columns.java  |   9 +-
 src/java/org/apache/cassandra/db/Mutation.java |   6 +-
 src/java/org/apache/cassandra/db/ReadResponse.java |   8 +-
 .../apache/cassandra/db/SerializationHeader.java   |   1 +
 .../cassandra/db/UnfilteredDeserializer.java   |   6 +-
 .../db/columniterator/AbstractSSTableIterator.java |   6 +-
 .../cassandra/db/commitlog/CommitLogReader.java|   4 +-
 .../db/partitions/CachedBTreePartition.java|   4 +-
 .../cassandra/db/partitions/PartitionUpdate.java   |   7 +-
 .../partitions/UnfilteredPartitionIterators.java   |   2 +-
 .../org/apache/cassandra/db/rows/AbstractRow.java  |  18 +-
 .../org/apache/cassandra/db/rows/BTreeRow.java | 103 +++
 src/java/org/apache/cassandra/db/rows/Cell.java|   2 +-
 .../org/apache/cassandra/db/rows/ColumnData.java   |   5 +
 .../cassandra/db/rows/ComplexColumnData.java   |  18 +-
 ...ationHelper.java => DeserializationHelper.java} |   6 +-
 src/java/org/apache/cassandra/db/rows/Row.java |  23 ++-
 src/java/org/apache/cassandra/db/rows/Rows.java|  72 ---
 .../cassandra/db/rows/SerializationHelper.java | 134 +++---
 .../db/rows/UnfilteredRowIteratorSerializer.java   |  21 ++-
 .../cassandra/db/rows/UnfilteredSerializer.java|  95 +-
 .../db/streaming/CassandraStreamReader.java|   4 +-
 .../io/sstable/SSTableIdentityIterator.java|   4 +-
 .../io/sstable/SSTableSimpleIterator.java  |  12 +-
 .../io/sstable/SSTableSimpleUnsortedWriter.java|   5 +-
 .../org/apache/cassandra/service/paxos/Commit.java |   2 +-
 .../{WrappedInt.java => BiLongAccumulator.java}|  32 +---
 .../{WrappedInt.java => LongAccumulator.java}  |  32 +---
 .../org/apache/cassandra/utils/btree/BTree.java| 206 -
 .../cassandra/utils/btree/BTreeSearchIterator.java |   4 +
 .../utils/btree/LeafBTreeSearchIterator.java   |  29 ++-
 .../db/commitlog/CommitLogStressTest.java  |   5 +-
 .../org/apache/cassandra/db/ReadCommandTest.java   |   3 +-
 .../org/apache/cassandra/db/RowIndexEntryTest.java |   7 +-
 .../cassandra/db/commitlog/CDCTestReplayer.java|   4 +-
 .../db/commitlog/CommitLogTestReplayer.java|   4 +-
 .../apache/cassandra/utils/btree/BTreeTest.java|  55 +-
 39 files changed, 492 insertions(+), 473 deletions(-)

diff --git a/CHANGES.txt b/CHANGES.txt
index 96ce286..c1e5aeb 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0-alpha4
+ * Minimize BTree iterator allocations (CASSANDRA-15389)
  * Add client request size server metrics (CASSANDRA-15704)
  * Add additional logging around FileUtils and compaction leftover cleanup 
(CASSANDRA-15705)
  * Mark system_views/system_virtual_schema as non-alterable keyspaces in cqlsh 
(CASSANDRA-15711)
diff --git a/src/java/org/apache/cassandra/db/ColumnIndex.java 
b/src/java/org/apache/cassandra/db/ColumnIndex.java
index 74ad264..e11f784 100644
--- a/src/java/org/apache/cassandra/db/ColumnIndex.java
+++ b/src/java/org/apache/cassandra/db/ColumnIndex.java
@@ -53,6 +53,7 @@ public class ColumnIndex
 public int columnIndexCount;
 private int[] indexOffsets;
 
+private final SerializationHelper helper;
 private final SerializationHeader header;
 private final int version;
 private final SequentialWriter writer;
@@ -79,6 +80,7 @@ public class ColumnIndex
 Collection observers,
 ISerializer indexInfoSerializer)
 {
+this.helper = new SerializationHelper(header);
 this.header = header;
 this.writer = writer;
 this.version = version.correspondingMessagingVersion();
@@ -126,7 +128,7 @@ public class ColumnIndex
 {
 Row staticRow = iterator.staticRow();
 
-UnfilteredSerializer.serializer.serializeStaticRow(staticRow, 
header, writer, version);
+UnfilteredSerializer.serializer.serializeStaticRow(staticRow, 
helper, writer, version);
 if (!observers.isEmpty())
 observers.forEach((o) -> o.nextUnfilteredCluster(staticRow));
 }
@@ -248,7 +250,7 @@ public class ColumnIndex
 startPosition = pos;
 }
 
-UnfilteredSerializer.serializer.serialize(unfiltered, 

[jira] [Updated] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15667:

Attachment: log_bootstrap_resumable

> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
> Attachments: log_bootstrap_resumable
>
>
> {{StreamResultFuture#maybeComplete()}} uses 
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are 
> completed, but then accesses each session state via 
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the 
> former relies on the actual {{StreamSession}} state, while the latter on the 
> {{SessionInfo}} state, and the two are concurrently updated with no 
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as 
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc 
> [~e.dimitrova].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17081026#comment-17081026
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15667:
-

Attached a log of 100 successful runs.
(Turned out the issue was false alarm, my local config got messed up, nothing 
related to the test or the patch)

> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamResultFuture#maybeComplete()}} uses 
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are 
> completed, but then accesses each session state via 
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the 
> former relies on the actual {{StreamSession}} state, while the latter on the 
> {{SessionInfo}} state, and the two are concurrently updated with no 
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as 
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc 
> [~e.dimitrova].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated (ca2dc1b -> d00c004)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from ca2dc1b  Updated the introductory section of the Installation page
 add d00c004  Prepare debian changelog for 4.0-alpha4

No new revisions were added by this update.

Summary of changes:
 debian/changelog | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

svn commit: r38896 - in /dev/cassandra/4.0-alpha4/redhat: ./ repodata/

[mck]

Author: mck
Date: Fri Apr 10 22:50:43 2020
New Revision: 38896

Log:
staging cassandra rpm packages for 4.0-alpha4

Added:
dev/cassandra/4.0-alpha4/redhat/
dev/cassandra/4.0-alpha4/redhat/cassandra-4.0~alpha4-1.noarch.rpm   (with 
props)
dev/cassandra/4.0-alpha4/redhat/cassandra-4.0~alpha4-1.src.rpm   (with 
props)
dev/cassandra/4.0-alpha4/redhat/cassandra-tools-4.0~alpha4-1.noarch.rpm   
(with props)
dev/cassandra/4.0-alpha4/redhat/repodata/

dev/cassandra/4.0-alpha4/redhat/repodata/4314a61d3f5e23815d797e71eba74702348a1960704ad6cc43e93273f4866672-filelists.xml.gz
   (with props)

dev/cassandra/4.0-alpha4/redhat/repodata/4314a61d3f5e23815d797e71eba74702348a1960704ad6cc43e93273f4866672-filelists.xml.gz.asc

dev/cassandra/4.0-alpha4/redhat/repodata/4da88c95f55278442539f0f685aa13d0c73bb7fc2edbd515babfce2fe84dbdb6-other.xml.gz
   (with props)

dev/cassandra/4.0-alpha4/redhat/repodata/4da88c95f55278442539f0f685aa13d0c73bb7fc2edbd515babfce2fe84dbdb6-other.xml.gz.asc

dev/cassandra/4.0-alpha4/redhat/repodata/6e357c6e3835956e46b492d64bd752ed944702a0ddda1e26fa84367e5f252e35-primary.sqlite.bz2
   (with props)

dev/cassandra/4.0-alpha4/redhat/repodata/6e357c6e3835956e46b492d64bd752ed944702a0ddda1e26fa84367e5f252e35-primary.sqlite.bz2.asc

dev/cassandra/4.0-alpha4/redhat/repodata/88e434b20334db0385b8e9ed22b17923d37b5f8c193f78e302c01ca7f6941b77-filelists.sqlite.bz2
   (with props)

dev/cassandra/4.0-alpha4/redhat/repodata/88e434b20334db0385b8e9ed22b17923d37b5f8c193f78e302c01ca7f6941b77-filelists.sqlite.bz2.asc

dev/cassandra/4.0-alpha4/redhat/repodata/aadc8038e1f86f8a7fd129568b32a2cc77f9f9b2d8da42286881e04c5fcb83fd-primary.xml.gz
   (with props)

dev/cassandra/4.0-alpha4/redhat/repodata/aadc8038e1f86f8a7fd129568b32a2cc77f9f9b2d8da42286881e04c5fcb83fd-primary.xml.gz.asc

dev/cassandra/4.0-alpha4/redhat/repodata/fc2ea9cd4e27637049c3454d0dcc76898f67d029eb9443df3c9ba04aaf0fc30e-other.sqlite.bz2
   (with props)

dev/cassandra/4.0-alpha4/redhat/repodata/fc2ea9cd4e27637049c3454d0dcc76898f67d029eb9443df3c9ba04aaf0fc30e-other.sqlite.bz2.asc
dev/cassandra/4.0-alpha4/redhat/repodata/repomd.xml
dev/cassandra/4.0-alpha4/redhat/repodata/repomd.xml.asc

Added: dev/cassandra/4.0-alpha4/redhat/cassandra-4.0~alpha4-1.noarch.rpm
==
Binary file - no diff available.

Propchange: dev/cassandra/4.0-alpha4/redhat/cassandra-4.0~alpha4-1.noarch.rpm
--
svn:mime-type = application/octet-stream

Added: dev/cassandra/4.0-alpha4/redhat/cassandra-4.0~alpha4-1.src.rpm
==
Binary file - no diff available.

Propchange: dev/cassandra/4.0-alpha4/redhat/cassandra-4.0~alpha4-1.src.rpm
--
svn:mime-type = application/octet-stream

Added: dev/cassandra/4.0-alpha4/redhat/cassandra-tools-4.0~alpha4-1.noarch.rpm
==
Binary file - no diff available.

Propchange: 
dev/cassandra/4.0-alpha4/redhat/cassandra-tools-4.0~alpha4-1.noarch.rpm
--
svn:mime-type = application/octet-stream

Added: 
dev/cassandra/4.0-alpha4/redhat/repodata/4314a61d3f5e23815d797e71eba74702348a1960704ad6cc43e93273f4866672-filelists.xml.gz
==
Binary file - no diff available.

Propchange: 
dev/cassandra/4.0-alpha4/redhat/repodata/4314a61d3f5e23815d797e71eba74702348a1960704ad6cc43e93273f4866672-filelists.xml.gz
--
svn:mime-type = application/octet-stream

Added: 
dev/cassandra/4.0-alpha4/redhat/repodata/4314a61d3f5e23815d797e71eba74702348a1960704ad6cc43e93273f4866672-filelists.xml.gz.asc
==
--- 
dev/cassandra/4.0-alpha4/redhat/repodata/4314a61d3f5e23815d797e71eba74702348a1960704ad6cc43e93273f4866672-filelists.xml.gz.asc
 (added)
+++ 
dev/cassandra/4.0-alpha4/redhat/repodata/4314a61d3f5e23815d797e71eba74702348a1960704ad6cc43e93273f4866672-filelists.xml.gz.asc
 Fri Apr 10 22:50:43 2020
@@ -0,0 +1,16 @@
+-BEGIN PGP SIGNATURE-
+
+iQIzBAABCgAdFiEEpMRl/qDFUlYaOSph6RM1134+h8sFAl6Q+CwACgkQ6RM1134+
+h8unrBAAsYlBQ6Z8clm8oQ9fztxHZ4WlHwF3XcLRWYuYQR8kc8OIDSIrR/Z2MdMJ
+y8WjqN1xUgaf5b/DtiROWON8mSz8eeZ5YHDjV/sQKVzL58kjlw+RHVWrOhRXAuwS
+mBpATHPqWxMSeAKXCbbmxjZ8q1Ir8zBiZZCslyHPkBS00ho7/Vbhku4FIfIR8r45
+VeVWE8eLeiH2dgrv9Dij6Qi0B/p+gBOFAus9U0OQOIJVble26mJ0qAfs8+bNrK0B
+Rd+wzLDTfMklvDFO/oCNYTndni3EdWs99+EgcLVkgDe+uCln60zjExm1j59Qxaw0
+wPRYxT7wTV1VEImn8oL7/ekW6bN/cmMsIAwJWMX1cLZ/1W2VcL2k8bvGRHGAYDFs

[jira] [Updated] (CASSANDRA-15714) Support in cassandra-in-jvm-dtest-api for replacing logback with alternate logger

[Jon Meredith (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Meredith updated CASSANDRA-15714:
-
Test and Documentation Plan: Manual check it worked in upstream build.
 Status: Patch Available  (was: In Progress)

PR on github

>  Support in cassandra-in-jvm-dtest-api for replacing logback with alternate 
> logger
> --
>
> Key: CASSANDRA-15714
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15714
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Not all forks use logback, and there is an (prematurely) closed ticket 
> indicating that it would be valuable CASSANDRA-13212.
>  
> Add support for making the log file configuration property and log file 
> pathname configurable rather than hard-coding to logback.
>  
> Also had to add 'org.w3c.dom' to the InstanceClassLoader so that log4j2 could 
> load its configuration, but looks like that can be handled with the changes 
> in CASSANDRA-15713



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15714) Support in cassandra-in-jvm-dtest-api for replacing logback with alternate logger

[Jon Meredith (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Meredith updated CASSANDRA-15714:
-
 Bug Category: Parent values: Code(13163)
   Complexity: Low Hanging Fruit
Discovered By: Unit Test
 Severity: Low
 Assignee: Jon Meredith
   Status: Open  (was: Triage Needed)

>  Support in cassandra-in-jvm-dtest-api for replacing logback with alternate 
> logger
> --
>
> Key: CASSANDRA-15714
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15714
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Not all forks use logback, and there is an (prematurely) closed ticket 
> indicating that it would be valuable CASSANDRA-13212.
>  
> Add support for making the log file configuration property and log file 
> pathname configurable rather than hard-coding to logback.
>  
> Also had to add 'org.w3c.dom' to the InstanceClassLoader so that log4j2 could 
> load its configuration, but looks like that can be handled with the changes 
> in CASSANDRA-15713



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15714) Support in cassandra-in-jvm-dtest-api for replacing logback with alternate logger

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CASSANDRA-15714:
---
Labels: pull-request-available  (was: )

>  Support in cassandra-in-jvm-dtest-api for replacing logback with alternate 
> logger
> --
>
> Key: CASSANDRA-15714
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15714
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
>
> Not all forks use logback, and there is an (prematurely) closed ticket 
> indicating that it would be valuable CASSANDRA-13212.
>  
> Add support for making the log file configuration property and log file 
> pathname configurable rather than hard-coding to logback.
>  
> Also had to add 'org.w3c.dom' to the InstanceClassLoader so that log4j2 could 
> load its configuration, but looks like that can be handled with the changes 
> in CASSANDRA-15713



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

svn commit: r38895 - in /dev/cassandra/4.0-alpha4/debian: ./ dists/ dists/40x/ dists/40x/main/ dists/40x/main/binary-amd64/ dists/40x/main/binary-i386/ dists/40x/main/source/ pool/ pool/main/ pool/mai

[mck]

Author: mck
Date: Fri Apr 10 22:39:22 2020
New Revision: 38895

Log:
staging cassandra debian packages for 4.0-alpha4

Added:
dev/cassandra/4.0-alpha4/debian/
dev/cassandra/4.0-alpha4/debian/cassandra-tools_4.0~alpha4_all.deb   (with 
props)
dev/cassandra/4.0-alpha4/debian/cassandra_4.0~alpha4.dsc
dev/cassandra/4.0-alpha4/debian/cassandra_4.0~alpha4.tar.gz   (with props)
dev/cassandra/4.0-alpha4/debian/cassandra_4.0~alpha4_all.deb   (with props)
dev/cassandra/4.0-alpha4/debian/cassandra_4.0~alpha4_amd64.buildinfo
dev/cassandra/4.0-alpha4/debian/cassandra_4.0~alpha4_amd64.changes
dev/cassandra/4.0-alpha4/debian/dists/
dev/cassandra/4.0-alpha4/debian/dists/40x/
dev/cassandra/4.0-alpha4/debian/dists/40x/InRelease
dev/cassandra/4.0-alpha4/debian/dists/40x/Release
dev/cassandra/4.0-alpha4/debian/dists/40x/Release.gpg
dev/cassandra/4.0-alpha4/debian/dists/40x/main/
dev/cassandra/4.0-alpha4/debian/dists/40x/main/binary-amd64/
dev/cassandra/4.0-alpha4/debian/dists/40x/main/binary-amd64/Packages
dev/cassandra/4.0-alpha4/debian/dists/40x/main/binary-amd64/Packages.gz   
(with props)
dev/cassandra/4.0-alpha4/debian/dists/40x/main/binary-amd64/Release
dev/cassandra/4.0-alpha4/debian/dists/40x/main/binary-i386/
dev/cassandra/4.0-alpha4/debian/dists/40x/main/binary-i386/Packages
dev/cassandra/4.0-alpha4/debian/dists/40x/main/binary-i386/Packages.gz   
(with props)
dev/cassandra/4.0-alpha4/debian/dists/40x/main/binary-i386/Release
dev/cassandra/4.0-alpha4/debian/dists/40x/main/source/
dev/cassandra/4.0-alpha4/debian/dists/40x/main/source/Release
dev/cassandra/4.0-alpha4/debian/dists/40x/main/source/Sources.gz   (with 
props)
dev/cassandra/4.0-alpha4/debian/pool/
dev/cassandra/4.0-alpha4/debian/pool/main/
dev/cassandra/4.0-alpha4/debian/pool/main/c/
dev/cassandra/4.0-alpha4/debian/pool/main/c/cassandra/

dev/cassandra/4.0-alpha4/debian/pool/main/c/cassandra/cassandra-tools_4.0~alpha4_all.deb
   (with props)

dev/cassandra/4.0-alpha4/debian/pool/main/c/cassandra/cassandra_4.0~alpha4.dsc

dev/cassandra/4.0-alpha4/debian/pool/main/c/cassandra/cassandra_4.0~alpha4.tar.gz
   (with props)

dev/cassandra/4.0-alpha4/debian/pool/main/c/cassandra/cassandra_4.0~alpha4_all.deb
   (with props)

Added: dev/cassandra/4.0-alpha4/debian/cassandra-tools_4.0~alpha4_all.deb
==
Binary file - no diff available.

Propchange: dev/cassandra/4.0-alpha4/debian/cassandra-tools_4.0~alpha4_all.deb
--
svn:mime-type = application/octet-stream

Added: dev/cassandra/4.0-alpha4/debian/cassandra_4.0~alpha4.dsc
==
--- dev/cassandra/4.0-alpha4/debian/cassandra_4.0~alpha4.dsc (added)
+++ dev/cassandra/4.0-alpha4/debian/cassandra_4.0~alpha4.dsc Fri Apr 10 
22:39:22 2020
@@ -0,0 +1,41 @@
+-BEGIN PGP SIGNED MESSAGE-
+Hash: SHA512
+
+Format: 1.0
+Source: cassandra
+Binary: cassandra, cassandra-tools
+Architecture: all
+Version: 4.0~alpha4
+Maintainer: Eric Evans 
+Uploaders: Sylvain Lebresne 
+Homepage: http://cassandra.apache.org
+Standards-Version: 3.8.3
+Vcs-Browser: https://gitbox.apache.org/repos/asf?p=cassandra.git
+Vcs-Git: https://gitbox.apache.org/repos/asf/cassandra.git
+Build-Depends: debhelper (>= 5), openjdk-8-jdk | java8-jdk, ant (>= 1.9), 
ant-optional (>= 1.9), dh-python, python-dev (>= 2.7), quilt, bash-completion
+Package-List:
+ cassandra deb misc extra arch=all
+ cassandra-tools deb misc extra arch=all
+Checksums-Sha1:
+ 8e14760ece060ff6143587b8354812e7dce90492 45126226 cassandra_4.0~alpha4.tar.gz
+Checksums-Sha256:
+ 7634e61bd1e0ddb2052dca1f33bafa95241dba341fdbfa77e37cf4a0a5836c23 45126226 
cassandra_4.0~alpha4.tar.gz
+Files:
+ 87cbef5a449c8283d1c7d64289f9a43c 45126226 cassandra_4.0~alpha4.tar.gz
+
+-BEGIN PGP SIGNATURE-
+
+iQIzBAEBCgAdFiEEpMRl/qDFUlYaOSph6RM1134+h8sFAl6Q9XUACgkQ6RM1134+
+h8vu4w/+Izz/rPLQ2O0llZqaNFWpz/I81FwPT8csb/Wp/3fXs/ip+GhHBv8M7MJB
+nOd+wZhq2FruJ8mZMf6PV9c1/e9iixHyy4tcfXMwpAnUFriy1FnAeP02KYsEc98y
+0zh48rwLzLW85SYZVFVvIy/QVML4f0CQia+zOdGb4LCKdTTCmLEWl83OEH8STr/n
+NjiyVV9/FEzabg5cLTaFynQAdlSwnV9PrgMWAbFSbzhykDFurPULjL/bcWoQ9iVa
+yBNkx/ymCBsdfyZlOhI7T/bYqRIbRK1IbzxlKZXluVuzu4FcW1Omghf+VIN+VzM0
+dqgyfkmT+xgMyVzDyDoFykoM/hT/sbT73dH0G20utnm+2wJCQ/6cG28FG3tPqoep
+rkt8U6RyYUbtEkULhGjXOAmQiV/JGk51XCZ3KBuFN0faXVwqW15ggcv7jSxy86Nd
+8XfouDSAdYZyE3eNgf+OJMZ7vSy5JHUcmqpbimtvhrrZyL9m1Ee6Gsl5wqdfam4Y
+KBhuxxKV5SfNIifDmWQc/vIb4OxGcRWyC4SEHhazAuHtgE6RUYDhO57fvvQ97w9P
+0LFBvO5iBEjzrOGYdNYnUYKT4af/FxbS5TeqGWdCp1ETaso9vUZTlfLFfhQM9sWq
+4lHvFUXP+c8/6Z8SelHZ33btnKJs0ciURFbK57lHKUR1EB6i7AQ=
+=/rq2
+-END PGP SIGNATURE-

Added: dev/cassandra/4.0-alpha4/debian/cassandra_4.0~alpha4.tar.gz
==
Binary 

svn commit: r38894 - /dev/cassandra/4.0-alpha4/

[mck]

Author: mck
Date: Fri Apr 10 22:37:07 2020
New Revision: 38894

Log:
staging cassandra 4.0-alpha4

Added:
dev/cassandra/4.0-alpha4/
dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz   (with 
props)
dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.asc
dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.sha256
dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.sha512
dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz   (with 
props)
dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz.asc
dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz.sha256
dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz.sha512

Added: dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz
==
Binary file - no diff available.

Propchange: dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz
--
svn:mime-type = application/octet-stream

Added: dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.asc
==
--- dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.asc (added)
+++ dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.asc Fri Apr 
10 22:37:07 2020
@@ -0,0 +1,16 @@
+-BEGIN PGP SIGNATURE-
+
+iQIzBAABCgAdFiEEpMRl/qDFUlYaOSph6RM1134+h8sFAl6Q9CkACgkQ6RM1134+
+h8uG7BAAvW5pApyqFmudosvSVe5vE6bwoagdzVlSnS8Ws3Hd0BfCN8K1A8QuVw+Y
+CxAZhVTcbqcwmMwCk69F6r1iYmISzxF479gl9aLnHRTSlErFiqRKFvfqpvko0fz8
+Q3o1lVufVK5IfMUKwPHM/ypEBdZyw1sfoxNPE4jpHtSGYQfhh5z2mXgEqpbNwHYz
+hP+xPiK8KsFu7BIunlWxcmFBklc/B53/dIOCpoVtksB6TdVCmUEBGDaSYWCK9NI8
+qpxlvUY6LZWj4VVaVABhVD2RH0o0bo9DgAwsRg1Q/D2Rq2m7d4Xl7kUbeLba8Bcu
+InordCc6gpBFfZCCYV+CK/p2P8+Y7Qa/P1YeB/jgBj7O0clt9cKZIUmDj5gR6gdA
+7YK5tlu3HAny4+Tzox8S4kmZ4t7mu04nY5R2P4WaeXWUvQ+hXvzDdghKwOEfX8I2
+OpeIurfQ7MIY+mzHC8f/OmV1t2u3x66ABcI+nmx+3NFe8pJxrmR49jHcG0QoOnYT
+7G1IPdambaj/nezXkhGXF+ZhKoZf9hfEqO9x8vJEVaKYktz9el+cmR2haZDe3l/k
+Owvzek94wT0Vy8rROIKTNTXnWQxbmb8o473SAnNa6Y9E4KB/uVTvepVoLZSDfkEr
+MzqtEZTP8p6DfOs60eoduEUuZwuyvOWLkQRjhqqPQEcyu92mvcg=
+=rWfm
+-END PGP SIGNATURE-

Added: dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.sha256
==
--- dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.sha256 
(added)
+++ dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.sha256 Fri 
Apr 10 22:37:07 2020
@@ -0,0 +1 @@
+2a7547f533e1840f8b8ebc74ef12d1ad11b53a70fbb6479e6140cc947fceefbd

Added: dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.sha512
==
--- dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.sha512 
(added)
+++ dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-bin.tar.gz.sha512 Fri 
Apr 10 22:37:07 2020
@@ -0,0 +1 @@
+58171a002dd9d5a8f990bc485fa38c513c5a2a7dbb5adad5c19f9f09e9c7759888eef7b2790c73c7583e509b00519b20058687a65b20c0a1b9f3b8bb1d4e9d99

Added: dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz
==
Binary file - no diff available.

Propchange: dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz
--
svn:mime-type = application/octet-stream

Added: dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz.asc
==
--- dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz.asc (added)
+++ dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz.asc Fri Apr 
10 22:37:07 2020
@@ -0,0 +1,16 @@
+-BEGIN PGP SIGNATURE-
+
+iQIzBAABCgAdFiEEpMRl/qDFUlYaOSph6RM1134+h8sFAl6Q9CsACgkQ6RM1134+
+h8v1vg/+MWEJ6ZAaOtSPGyhvS76ecptda7VhewFQ5/1xZ9N3SHGhoXbCmH9sqGI8
+kig0Cpy4xrcbOx+BEiVwrCnJpAKsDZzWnLpwdYp/3iRnXlYqvvjvieuQb7tiUapN
+Ry1cN7wEMxvO4i6pT8+HdYfJuV3VWG8gFNTMsE1iis/5DHMGrTO+YJTOTYS6lNU6
+OTXT4+jbENafo/VzfMWSBu36i0TVDMEnk8lP1mEGD+UNYC4Jw+Ec8I2b2VwvVU0a
+LWEm0j0LQw2t6s/qSrAZxJJj5+1no74A+oTJ9uzGPz7EhMVQN4cZrTQPsX9PIW0U
+pGI1OLO5bOdi5awvM/PFeFOAzLFruzrL04R5i548pk0NZqelxjT4RI+udmlufsjX
+Ep1mLewJPY4L8NumlcNlzUNp/4u4kMhAlb4VG1BDVRh0YLpJtlbp/2YraqbPlMly
+uJdXz7VWfVd3qq5QCINPjiuDFmDOBr5rqTF8mAA4rj/uWPs9yvbTJ+550p/Jz266
+v8hDILLzU52TvxBerBh+tzFKD/SmCe8vr5CzLpkxLYQjPY15kvVL8IBtLERGT5Xy
+Hw6c7yww5ibxVoDvbhY/l0F9PgoNKZNihKGVWs8CnY9J4QETyhB/29xTAHVcOCYI
+EHCmsA2hdD/DqiLUOZ0715mMAPA7wxuXQnWxYjq/nezGvGfUZKk=
+=rn5Q
+-END PGP SIGNATURE-

Added: dev/cassandra/4.0-alpha4/apache-cassandra-4.0-alpha4-src.tar.gz.sha256
==
--- 

[jira] [Created] (CASSANDRA-15714) Support in cassandra-in-jvm-dtest-api for replacing logback with alternate logger

[Jon Meredith (Jira)]

Jon Meredith created CASSANDRA-15714:


 Summary:  Support in cassandra-in-jvm-dtest-api for replacing 
logback with alternate logger
 Key: CASSANDRA-15714
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15714
 Project: Cassandra
  Issue Type: Bug
  Components: Test/dtest
Reporter: Jon Meredith


Not all forks use logback, and there is an (prematurely) closed ticket 
indicating that it would be valuable CASSANDRA-13212.

 
Add support for making the log file configuration property and log file 
pathname configurable rather than hard-coding to logback.
 
Also had to add 'org.w3c.dom' to the InstanceClassLoader so that log4j2 could 
load its configuration, but looks like that can be handled with the changes in 
CASSANDRA-15713



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15713) InstanceClassLoader fails to load with the following previously initiated loading for a different type with name "org/w3c/dom/Document"

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CASSANDRA-15713:
---
Labels: pull-request-available  (was: )

> InstanceClassLoader fails to load with the following previously initiated 
> loading for a different type with name "org/w3c/dom/Document"
> ---
>
> Key: CASSANDRA-15713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15713
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
>
> java.lang.LinkageError: loader constraint violation: loader (instance of 
> org/apache/cassandra/distributed/shared/InstanceClassLoader) previously 
> initiated loading for a different type with name "org/w3c/dom/Document”
> This is caused when using dtest outside of the normal Cassandra context.  
> There is no API to add more exclusions so unable to work around this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] tag 4.0-alpha4-tentative created (now d00c004)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to tag 4.0-alpha4-tentative
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


  at d00c004  (commit)
This tag includes the following new commits:

 new d00c004  Prepare debian changelog for 4.0-alpha4

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Prepare debian changelog for 4.0-alpha4

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to tag 4.0-alpha4-tentative
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit d00c004cc10986fc41c2070f9c5d0007e03a45c3
Author: Mick Semb Wever 
AuthorDate: Sat Apr 11 00:28:22 2020 +0200

Prepare debian changelog for 4.0-alpha4
---
 debian/changelog | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 6e44a80..fea0317 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,8 +1,8 @@
-cassandra (4.0~alpha4) UNRELEASED; urgency=medium
+cassandra (4.0~alpha4) unstable; urgency=medium
 
   * New release
 
- --
+ -- Mick Semb Wever   Sat, 11 Apr 2020 00:27:13 +0200
 
 cassandra (4.0~alpha3) unstable; urgency=medium
 


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15713) InstanceClassLoader fails to load with the following previously initiated loading for a different type with name "org/w3c/dom/Document"

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15713:
--
Test and Documentation Plan: manual verification
 Status: Patch Available  (was: Open)

> InstanceClassLoader fails to load with the following previously initiated 
> loading for a different type with name "org/w3c/dom/Document"
> ---
>
> Key: CASSANDRA-15713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15713
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> java.lang.LinkageError: loader constraint violation: loader (instance of 
> org/apache/cassandra/distributed/shared/InstanceClassLoader) previously 
> initiated loading for a different type with name "org/w3c/dom/Document”
> This is caused when using dtest outside of the normal Cassandra context.  
> There is no API to add more exclusions so unable to work around this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15713) InstanceClassLoader fails to load with the following previously initiated loading for a different type with name "org/w3c/dom/Document"

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15713:
--
Change Category: Quality Assurance
 Complexity: Low Hanging Fruit
 Status: Open  (was: Triage Needed)

> InstanceClassLoader fails to load with the following previously initiated 
> loading for a different type with name "org/w3c/dom/Document"
> ---
>
> Key: CASSANDRA-15713
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15713
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>
> java.lang.LinkageError: loader constraint violation: loader (instance of 
> org/apache/cassandra/distributed/shared/InstanceClassLoader) previously 
> initiated loading for a different type with name "org/w3c/dom/Document”
> This is caused when using dtest outside of the normal Cassandra context.  
> There is no API to add more exclusions so unable to work around this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-15713) InstanceClassLoader fails to load with the following previously initiated loading for a different type with name "org/w3c/dom/Document"

[David Capwell (Jira)]

David Capwell created CASSANDRA-15713:
-

 Summary: InstanceClassLoader fails to load with the following 
previously initiated loading for a different type with name 
"org/w3c/dom/Document"
 Key: CASSANDRA-15713
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15713
 Project: Cassandra
  Issue Type: Improvement
  Components: Test/dtest
Reporter: David Capwell
Assignee: David Capwell


java.lang.LinkageError: loader constraint violation: loader (instance of 
org/apache/cassandra/distributed/shared/InstanceClassLoader) previously 
initiated loading for a different type with name "org/w3c/dom/Document”

This is caused when using dtest outside of the normal Cassandra context.  There 
is no API to add more exclusions so unable to work around this.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15466) DOC - Improve installation pages, include YUM procedure

[Jon Haddad (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRA-15466:
---
Status: Patch Available  (was: In Progress)

> DOC - Improve installation pages, include YUM procedure
> ---
>
> Key: CASSANDRA-15466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15466
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15466-trunk-2.txt, 15466-trunk.txt
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> h2. Background
> The installation instructions are a little lacking. I'm proposing we add more 
> explicit steps which are targeted towards users who are very new to Cassandra.
> h2. Proposal
> We can improve the installation pages with explicit instructions for the 
> following:
> * steps for installing older versions to match nodes in an existing cluster
> * steps for installing RHEL/CentOS package with YUM
> I am preparing a draft with details to follow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15466) DOC - Improve installation pages, include YUM procedure

[Jon Haddad (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRA-15466:
---
Source Control Link: 
https://github.com/apache/cassandra/commit/ca2dc1bfa9a7555d565c6b8a4fb957e35a5c0f13
 Status: Resolved  (was: Ready to Commit)

Committed, thanks!

> DOC - Improve installation pages, include YUM procedure
> ---
>
> Key: CASSANDRA-15466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15466
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15466-trunk-2.txt, 15466-trunk.txt
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> h2. Background
> The installation instructions are a little lacking. I'm proposing we add more 
> explicit steps which are targeted towards users who are very new to Cassandra.
> h2. Proposal
> We can improve the installation pages with explicit instructions for the 
> following:
> * steps for installing older versions to match nodes in an existing cluster
> * steps for installing RHEL/CentOS package with YUM
> I am preparing a draft with details to follow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15466) DOC - Improve installation pages, include YUM procedure

[Jon Haddad (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRA-15466:
---
Reviewers: Jon Haddad, Jon Haddad  (was: Jon Haddad)
   Jon Haddad, Jon Haddad  (was: Jon Haddad)
   Status: Review In Progress  (was: Patch Available)

> DOC - Improve installation pages, include YUM procedure
> ---
>
> Key: CASSANDRA-15466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15466
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15466-trunk-2.txt, 15466-trunk.txt
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> h2. Background
> The installation instructions are a little lacking. I'm proposing we add more 
> explicit steps which are targeted towards users who are very new to Cassandra.
> h2. Proposal
> We can improve the installation pages with explicit instructions for the 
> following:
> * steps for installing older versions to match nodes in an existing cluster
> * steps for installing RHEL/CentOS package with YUM
> I am preparing a draft with details to follow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15466) DOC - Improve installation pages, include YUM procedure

[Jon Haddad (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRA-15466:
---
Status: In Progress  (was: Patch Available)

> DOC - Improve installation pages, include YUM procedure
> ---
>
> Key: CASSANDRA-15466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15466
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15466-trunk-2.txt, 15466-trunk.txt
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> h2. Background
> The installation instructions are a little lacking. I'm proposing we add more 
> explicit steps which are targeted towards users who are very new to Cassandra.
> h2. Proposal
> We can improve the installation pages with explicit instructions for the 
> following:
> * steps for installing older versions to match nodes in an existing cluster
> * steps for installing RHEL/CentOS package with YUM
> I am preparing a draft with details to follow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15466) DOC - Improve installation pages, include YUM procedure

[Jon Haddad (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRA-15466:
---
Status: Patch Available  (was: Open)

> DOC - Improve installation pages, include YUM procedure
> ---
>
> Key: CASSANDRA-15466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15466
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15466-trunk-2.txt, 15466-trunk.txt
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> h2. Background
> The installation instructions are a little lacking. I'm proposing we add more 
> explicit steps which are targeted towards users who are very new to Cassandra.
> h2. Proposal
> We can improve the installation pages with explicit instructions for the 
> following:
> * steps for installing older versions to match nodes in an existing cluster
> * steps for installing RHEL/CentOS package with YUM
> I am preparing a draft with details to follow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15466) DOC - Improve installation pages, include YUM procedure

[Jon Haddad (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jon Haddad updated CASSANDRA-15466:
---
Status: Ready to Commit  (was: Review In Progress)

> DOC - Improve installation pages, include YUM procedure
> ---
>
> Key: CASSANDRA-15466
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15466
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15466-trunk-2.txt, 15466-trunk.txt
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> h2. Background
> The installation instructions are a little lacking. I'm proposing we add more 
> explicit steps which are targeted towards users who are very new to Cassandra.
> h2. Proposal
> We can improve the installation pages with explicit instructions for the 
> following:
> * steps for installing older versions to match nodes in an existing cluster
> * steps for installing RHEL/CentOS package with YUM
> I am preparing a draft with details to follow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Updated the introductory section of the Installation page

[rustyrazorblade]

This is an automated email from the ASF dual-hosted git repository.

rustyrazorblade pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new ca2dc1b  Updated the introductory section of the Installation page
ca2dc1b is described below

commit ca2dc1bfa9a7555d565c6b8a4fb957e35a5c0f13
Author: Erick Ramirez 
AuthorDate: Sun Apr 5 06:52:40 2020 +

Updated the introductory section of the Installation page

Patch by Erick Ramirez; Reviewed by Jon Haddad for CASSANDRA-15466
---
 doc/source/getting_started/installing.rst | 306 +-
 1 file changed, 262 insertions(+), 44 deletions(-)

diff --git a/doc/source/getting_started/installing.rst 
b/doc/source/getting_started/installing.rst
index a87589d..f3a22f2 100644
--- a/doc/source/getting_started/installing.rst
+++ b/doc/source/getting_started/installing.rst
@@ -19,88 +19,306 @@
 Installing Cassandra
 
 
+These are the instructions for deploying the supported releases of Apache 
Cassandra on Linux servers.
+
+Cassandra runs on a wide array of Linux distributions including (but not 
limited to):
+
+- Ubuntu, most notably LTS releases 16.04 to 18.04
+- CentOS & RedHat Enterprise Linux (RHEL) including 6.6 to 7.7
+- Amazon Linux AMIs including 2016.09 through to Linux 2
+- Debian versions 8 & 9
+- SUSE Enterprise Linux 12
+
+This is not an exhaustive list of operating system platforms, nor is it 
prescriptive. However users will be
+well-advised to conduct exhaustive tests of their own particularly for 
less-popular distributions of Linux.
+Deploying on older versions is not recommended unless you have previous 
experience with the older distribution
+in a production environment.
+
 Prerequisites
 ^
 
-- The latest version of Java 8, either the `Oracle Java Standard Edition 8
+- Install the latest version of Java 8, either the `Oracle Java Standard 
Edition 8
   `__ or 
`OpenJDK 8 `__. To
   verify that you have the correct version of java installed, type ``java 
-version``.
-
-- For using cqlsh, the latest version of `Python 2.7 
`__. To verify that you have
+- **NOTE**: *Experimental* support for Java 11 was added in Cassandra 4.0 
(`CASSANDRA-9608 `__).
+  Running Cassandra on Java 11 is *experimental*. Do so at your own risk. For 
more information, see
+  `NEWS.txt `__.
+- For using cqlsh, the latest version of `Python 2.7 
`__ or Python 3.6+. To verify that you have
   the correct version of Python installed, type ``python --version``.
 
-Installation from binary tarball files
-^^
+Choosing an installation method
+^^^
+
+For most users, installing the binary tarball is the simplest choice. The 
tarball unpacks all its contents
+into a single location with binaries and configuration files located in their 
own subdirectories. The most
+obvious attribute of the tarball installation is it does not require ``root`` 
permissions and can be
+installed on any Linux distribution.
+
+Packaged installations require ``root`` permissions. Install the RPM build on 
CentOS and RHEL-based
+distributions if you want to install Cassandra using YUM. Install the Debian 
build on Ubuntu and other
+Debian-based distributions if you want to install Cassandra using APT. Note 
that both the YUM and APT
+methods required ``root`` permissions and will install the binaries and 
configuration files as the
+``cassandra`` OS user.
+
+Installing the binary tarball
+^
+
+1. Verify the version of Java installed. For example:
+
+::
+
+   $ java -version
+   openjdk version "1.8.0_222"
+   OpenJDK Runtime Environment (build 1.8.0_222-8u222-b10-1ubuntu1~16.04.1-b10)
+   OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode)
+
+2. Download the binary tarball from one of the mirrors on the `Apache 
Cassandra Download `__
+   site. For example, to download 4.0:
+
+::
+
+   $ curl -OL 
http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz
+
+NOTE: The mirrors only host the latest versions of each major supported 
release. To download an earlier
+version of Cassandra, visit the `Apache Archives 
`__.
+
+3. OPTIONAL: Verify the integrity of the downloaded tarball using one of the 
methods `here `__.
+   For example, to verify the hash of the downloaded file using GPG:
+
+::
+
+   $ gpg --print-md SHA256 apache-cassandra-4.0.0-bin.tar.gz 
+   apache-cassandra-4.0.0-bin.tar.gz: 28757DDE 

[cassandra] branch trunk updated: Update docs describing when a CHANGES.txt entry is necessary and remove 4.0-alpha* entries in CHANGES.txt that don't touch runtime code ref: https://lists.apache.org/

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 141ea26  Update docs describing when a CHANGES.txt entry is necessary  
and remove 4.0-alpha*  entries in CHANGES.txt that don't touch runtime code  
ref: 
https://lists.apache.org/thread.html/rde1128131a621e43b0a9c88778398c053a234da0f4c654b82dcbbe0e%40%3Cdev.cassandra.apache.org%3E
141ea26 is described below

commit 141ea26733e7e8fe022bc78f7fd68225013e8d14
Author: Eduard Tudenhöfner 
AuthorDate: Thu Apr 9 09:26:40 2020 +0200

Update docs describing when a CHANGES.txt entry is necessary
 and remove 4.0-alpha*  entries in CHANGES.txt that don't touch runtime code
 ref: 
https://lists.apache.org/thread.html/rde1128131a621e43b0a9c88778398c053a234da0f4c654b82dcbbe0e%40%3Cdev.cassandra.apache.org%3E

 patch by Eduard Tudenhöfner, Mick Semb Wever; reviewed by Mick Semb Wever, 
Jon Haddad
---
 CHANGES.txt| 23 ---
 doc/source/development/patches.rst |  2 +-
 2 files changed, 1 insertion(+), 24 deletions(-)

diff --git a/CHANGES.txt b/CHANGES.txt
index 878770b..96ce286 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -17,38 +17,20 @@
  * Set now in seconds in the future for validation repairs (CASSANDRA-15655)
  * Emit metric on preview repair failure (CASSANDRA-15654)
  * Use more appropriate logging levels (CASSANDRA-15661)
- * Added production recommendations and improved compaction doc organization
- * Document usage of EC2Snitch with intra-region VPC peering (CASSANDRA-15337)
- * Fixed flakey test in SASIIndexTest by shutting down its ExecutorService 
(CASSANDRA-15528)
  * Fixed empty check in TrieMemIndex due to potential state inconsistency in 
ConcurrentSkipListMap (CASSANDRA-15526)
- * Add compaction allocation measurement test (CASSANDRA-15388)
  * Added UnleveledSSTables global and table level metric (CASSANDRA-15620)
  * Added Virtual Table exposing Cassandra relevant system properties 
(CASSANDRA-15616, CASSANDRA-15643)
- * Add data modeling introduction (CASSANDRA-15481)
  * Improve the algorithmic token allocation in case racks = RF 
(CASSANDRA-15600)
  * Fix ConnectionTest.testAcquireReleaseOutbound (CASSANDRA-15308)
  * Include finalized pending sstables in preview repair (CASSANDRA-15553)
  * Reverted to the original behavior of CLUSTERING ORDER on CREATE TABLE 
(CASSANDRA-15271)
  * Correct inaccurate logging message (CASSANDRA-15549)
- * Add documentation of dynamo (CASSANDRA-15486)
- * Added documentation for Guarantees (CASSANDRA-15482)
- * Added documentation for audit logging (CASSANDRA-15474)
  * Unset GREP_OPTIONS (CASSANDRA-14487)
- * Added streaming documentation (CASSANDRA-15477)
  * Update to Python driver 3.21 for cqlsh (CASSANDRA-14872)
- * Added bulk loading documentation (CASSANDRA-15480)
- * Updated overview documentation (CASSANDRA-15483)
- * Added CDC and speculative retry documentation to DDL section 
(CASSANDRA-15492)
  * Fix missing Keyspaces in cqlsh describe output (CASSANDRA-15576)
  * Fix multi DC nodetool status output (CASSANDRA-15305)
- * Added documentation covering new Netty based internode messaging 
(CASSANDRA-15478)
- * Add documentation of hints (CASSANDRA-15491)
  * updateCoordinatorWriteLatencyTableMetric can produce misleading metrics 
(CASSANDRA-15569)
- * Added documentation for read repair and an example of full repair 
(CASSANDRA-15485)
  * Make cqlsh and cqlshlib Python 2 & 3 compatible (CASSANDRA-10190)
- * Added documentation for Full Query Logging (CASSANDRA-15475)
- * Added documentation for backups (CASSANDRA-15479)
- * Documentation gives the wrong instruction to activate remote jmx 
(CASSANDRA-15535)
  * Improve the description of nodetool listsnapshots command (CASSANDRA-14587)
  * allow embedded cassandra launched from a one-jar or uno-jar 
(CASSANDRA-15494)
  * Update hppc library to version 0.8.1 (CASSANDRA-12995)
@@ -90,7 +72,6 @@ Merged from 2.1:
  * Make sure all exceptions are propagated in DebuggableThreadPoolExecutor 
(CASSANDRA-15332)
  * Make it possible to resize concurrent read / write thread pools at runtime 
(CASSANDRA-15277)
  * Close channels on error (CASSANDRA-15407)
- * Add documentation for Java 11 support in Cassandra (CASSANDRA-15428)
  * Integrate SJK into nodetool (CASSANDRA-12197)
  * Ensure that empty clusterings with kind==CLUSTERING are Clustering.EMPTY 
(CASSANDRA-15498)
  * The flag 'cross_node_timeout' has been set as true by default. This change
@@ -246,7 +227,6 @@ Merged from 3.0:
  * Add a virtual table to expose thread pools (CASSANDRA-14523)
  * Add a virtual table to expose caches (CASSANDRA-14538, CASSANDRA-14626)
  * Fix toDate function for timestamp arguments (CASSANDRA-14502)
- * Revert running dtests by default in circleci (CASSANDRA-14614)
  * Stream entire SSTables when possible (CASSANDRA-14556)
  

[jira] [Commented] (CASSANDRA-15709) SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080855#comment-17080855
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15709:
-

[~djoshi], [~dcapwell], is there anything else we are looking for in this one?
Or only a committer? (I know David is not a committer, same as me :D ) 

> SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure
> ---
>
> Key: CASSANDRA-15709
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15709
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> {code}
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest 
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.656 sec
> [junit-timeout] 
> [junit-timeout] Testcase: 
> changingMaxWorkersMeetsConcurrencyGoalsTest(org.apache.cassandra.concurrent.SEPExecutorTest):
>FAILED
> [junit-timeout] expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but 
> was:
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.assertMaxTaskConcurrency(SEPExecutorTest.java:180)
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.changingMaxWorkersMeetsConcurrencyGoalsTest(SEPExecutorTest.java:110)
> [junit-timeout] 
> [junit-timeout] 
> [junit-timeout] Test org.apache.cassandra.concurrent.SEPExecutorTest FAILED
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15674) liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if IndexSummaryRedistribution gets interrupted

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080854#comment-17080854
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15674:
-

Hi [~jrwest],
I see you are assigned as a reviewer, are you looking into this?
Do you need help?

> liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if 
> IndexSummaryRedistribution gets interrupted
> -
>
> Key: CASSANDRA-15674
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15674
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction, Observability/Metrics
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> IndexSummaryRedistribution is a compaction task and as such extends Holder 
> and supports cancelation by throwing a CompactionInterruptedException.  The 
> issue is that IndexSummaryRedistribution tries to use transactions, but 
> mutates the sstable in-place; transaction is unable to roll back.
> This would be fine (only updates summary) if it wasn’t for the fact the task 
> attempts to also mutate the two metrics liveDiskSpaceUsed and 
> totalDiskSpaceUsed, since these can’t be rolled back any cancelation could 
> corrupt these metrics.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15690) Single partition queries can mistakenly omit partition deletions and resurrect data

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080853#comment-17080853
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15690 at 4/10/20, 6:47 PM:
---

Hi [~aleksey],
I see you are assigned as a reviewer. Are you looking into this one? Need help 
if you are busy?


was (Author: e.dimitrova):
Hi [~aleksey],
I see you are assigned as a reviewer. Are you looking into this one?

> Single partition queries can mistakenly omit partition deletions and 
> resurrect data
> ---
>
> Key: CASSANDRA-15690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-alpha
>
>
> We have logic that allows us to exclude sstables with partition deletions 
> that are older than the minimum collected timestamp in a local request. 
> However, it’s possible that another node could have rows that aren’t known to 
> the local node that are in turn older than the excluded partition deletion. 
> In such a scenario, those will be mistakenly resurrected, which is a 
> correctness issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15601) Ensure repaired data tracking reads a consistent amount of data across replicas

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080852#comment-17080852
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15601 at 4/10/20, 6:47 PM:
---

Hi [~aleksey],
I see you are assigned as a reviewer, are you looking into this one? Need help 
if you are busy?



was (Author: e.dimitrova):
Hi [~aleksey],
I see you are assigned as a reviewer, are you looking into this one?


> Ensure repaired data tracking reads a consistent amount of data across 
> replicas
> ---
>
> Key: CASSANDRA-15601
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15601
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> When generating a digest for repaired data tracking, the amount of repaired 
> data that needs to be read may depend on the unrepaired data on the replica. 
> As this may vary between replicas, digest mismatches can be reported even 
> though the repaired data may actually be in sync.
> For example, two replicas, A & B and a table like
> {code}
> CREATE TABLE t  (pk int, ck int, PRIMARY KEY (pk, ck)) WITH CLUSTERING ORDER 
> BY ck DESC; 
> Unrepaired
> ===
> Instance A
> (0, 5)
> Instance B
> (0, 6)
> (0, 5)
> Repaired (Both A & B)
> =
> (0, 4)
> (0, 3)
> (0, 2)
> (0, 1)
> (0, 0)
> SELECT * FROM tbl WHERE pk = 0 LIMIT 3;
> {code}
> Instance A would read (0, 5) from the unrepaired set and (0, 4) (0, 3) from 
> the repaired set. 
>  Instance B would read (0, 6) (0, 5) from its unrepaired set and just (0, 4) 
> from repaired data.
> Unrepaired row/range/partition tombstones shadowing repaired data and present 
> on some replicas but not others will have the opposite effect, with more 
> repaired data being read in comparison.
>  To fix this, when repaired data tracking is in effect each replica needs to 
> overread during a full data read. Replicas should read up to {{LIMIT}} (i.e. 
> the {{DataLimit}} of the {{ReadCommand}}) from the repaired set, regardless 
> of how much is read from the unrepaired data. At the point where that amount 
> of repaired data has been read, replica should stop updating the digest. So 
> if unrepaired tombstones cause more than {{LIMIT}} repaired data to be read, 
> the digest is only calculated over the first {{LIMIT}}-worth of repaired data.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15690) Single partition queries can mistakenly omit partition deletions and resurrect data

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080853#comment-17080853
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15690:
-

Hi [~aleksey],
I see you are assigned as a reviewer. Are you looking into this one?

> Single partition queries can mistakenly omit partition deletions and 
> resurrect data
> ---
>
> Key: CASSANDRA-15690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-alpha
>
>
> We have logic that allows us to exclude sstables with partition deletions 
> that are older than the minimum collected timestamp in a local request. 
> However, it’s possible that another node could have rows that aren’t known to 
> the local node that are in turn older than the excluded partition deletion. 
> In such a scenario, those will be mistakenly resurrected, which is a 
> correctness issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15601) Ensure repaired data tracking reads a consistent amount of data across replicas

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080852#comment-17080852
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15601:
-

Hi [~aleksey],
I see you are assigned as a reviewer, are you looking into this one?


> Ensure repaired data tracking reads a consistent amount of data across 
> replicas
> ---
>
> Key: CASSANDRA-15601
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15601
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> When generating a digest for repaired data tracking, the amount of repaired 
> data that needs to be read may depend on the unrepaired data on the replica. 
> As this may vary between replicas, digest mismatches can be reported even 
> though the repaired data may actually be in sync.
> For example, two replicas, A & B and a table like
> {code}
> CREATE TABLE t  (pk int, ck int, PRIMARY KEY (pk, ck)) WITH CLUSTERING ORDER 
> BY ck DESC; 
> Unrepaired
> ===
> Instance A
> (0, 5)
> Instance B
> (0, 6)
> (0, 5)
> Repaired (Both A & B)
> =
> (0, 4)
> (0, 3)
> (0, 2)
> (0, 1)
> (0, 0)
> SELECT * FROM tbl WHERE pk = 0 LIMIT 3;
> {code}
> Instance A would read (0, 5) from the unrepaired set and (0, 4) (0, 3) from 
> the repaired set. 
>  Instance B would read (0, 6) (0, 5) from its unrepaired set and just (0, 4) 
> from repaired data.
> Unrepaired row/range/partition tombstones shadowing repaired data and present 
> on some replicas but not others will have the opposite effect, with more 
> repaired data being read in comparison.
>  To fix this, when repaired data tracking is in effect each replica needs to 
> overread during a full data read. Replicas should read up to {{LIMIT}} (i.e. 
> the {{DataLimit}} of the {{ReadCommand}}) from the repaired set, regardless 
> of how much is read from the unrepaired data. At the point where that amount 
> of repaired data has been read, replica should stop updating the digest. So 
> if unrepaired tombstones cause more than {{LIMIT}} repaired data to be read, 
> the digest is only calculated over the first {{LIMIT}}-worth of repaired data.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15262) server_encryption_options is not backwards compatible with 3.11

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080596#comment-17080596
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15262 at 4/10/20, 6:39 PM:
---

Hey [~jolynch],
Thanks for the update. Are you still working on this? Should we find anyone 
else to complete the rest of it if you are overloaded?


was (Author: e.dimitrova):
Hey [~jolynch],
Thanks for the update. Are you still working on this? Should we find anyone 
else?

> server_encryption_options is not backwards compatible with 3.11
> ---
>
> Key: CASSANDRA-15262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15262
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Config
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> The current `server_encryption_options` configuration options are as follows:
> {noformat}
> server_encryption_options:
> # set to true for allowing secure incoming connections
> enabled: false
> # If enabled and optional are both set to true, encrypted and unencrypted 
> connections are handled on the storage_port
> optional: false
> # if enabled, will open up an encrypted listening socket on 
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
> enable_legacy_ssl_storage_port: false
> # on outbound connections, determine which type of peers to securely 
> connect to. 'enabled' must be set to true.
> internode_encryption: none
> keystore: conf/.keystore
> keystore_password: cassandra
> truststore: conf/.truststore
> truststore_password: cassandra
> # More advanced defaults below:
> # protocol: TLS
> # store_type: JKS
> # cipher_suites: 
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> # require_client_auth: false
> # require_endpoint_verification: false
> {noformat}
> A couple of issues here:
> 1. optional defaults to false, which will break existing TLS configurations 
> for (from what I can tell) no particularly good reason
> 2. The provided protocol and cipher suites are not good ideas (in particular 
> encouraging anyone to use CBC ciphers is a bad plan
> I propose that before the 4.0 cut we fixup server_encryption_options and even 
> client_encryption_options :
> # Change the default {{optional}} setting to true. As the new Netty code 
> intelligently decides to open a TLS connection or not this is the more 
> sensible default (saves operators a step while transitioning to TLS as well)
> # Update the defaults to what netty actually defaults to



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-15624) Avoid lazy initializing shut down instances when trying to send them messages

[Gianluca Righetto (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gianluca Righetto reassigned CASSANDRA-15624:
-

Assignee: Gianluca Righetto  (was: Sujay Hegde)

> Avoid lazy initializing shut down instances when trying to send them messages
> -
>
> Key: CASSANDRA-15624
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15624
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest
>Reporter: Marcus Eriksson
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> We currently use {{to.broadcastAddressAndPort()}} when figuring out if we 
> should send a message to an instance, if that instance has been shut down it 
> will get re-initialized but not startup:ed which makes the tests fail.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15709) SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080643#comment-17080643
 ] 

David Capwell commented on CASSANDRA-15709:
---

update looks good, +1

> SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure
> ---
>
> Key: CASSANDRA-15709
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15709
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> {code}
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest 
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.656 sec
> [junit-timeout] 
> [junit-timeout] Testcase: 
> changingMaxWorkersMeetsConcurrencyGoalsTest(org.apache.cassandra.concurrent.SEPExecutorTest):
>FAILED
> [junit-timeout] expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but 
> was:
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.assertMaxTaskConcurrency(SEPExecutorTest.java:180)
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.changingMaxWorkersMeetsConcurrencyGoalsTest(SEPExecutorTest.java:110)
> [junit-timeout] 
> [junit-timeout] 
> [junit-timeout] Test org.apache.cassandra.concurrent.SEPExecutorTest FAILED
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15686) Improvements in circle CI default config

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080641#comment-17080641
 ] 

David Capwell commented on CASSANDRA-15686:
---

bq. Do you think these could be fixed, or put in another bucket to run in 
another non-parallel suite?

They could be fixed, but we would need to find all cases first.  I don't assume 
its a small number but a lot of subtle cases (actual unit tests are safe, but 
there are a lot which touch disk/network and those are the main ones worry me).

bq. You can see the runtime goes from 19min to 12.

I ran with 8 cores with 8 runners.  I ran twice and in both cases saw a latency 
regression for the builds, but also saw some cases where the JVM hangs because 
of memory issues.  its possible that a lower value would be better, but didn't 
bother testing further.  I "feel" that it really depends on what tests are in 
the same container, but this is just a guess.

> Improvements in circle CI default config
> 
>
> Key: CASSANDRA-15686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15686
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: Kevin Gallardo
>Priority: Normal
>
> I have been looking at and played around with the [default CircleCI 
> config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml], 
> a few comments/questions regarding the following topics:
>  * Python dtests do not run successfully (200-300 failures) on {{medium}} 
> instances, they seem to only run with small flaky failures on {{large}} 
> instances or higher
>  * Python Upgrade tests:
>  ** Do not seem to run without many failures on any instance types / any 
> parallelism setting
>  ** Do not seem to parallelize well, it seems each container is going to 
> download multiple C* versions
>  ** Additionally it seems the configuration is not up to date, as currently 
> we get errors because {{JAVA8_HOME}} is not set
>  * Unit tests do not seem to parallelize optimally, number of test runners do 
> not reflect the available CPUs on the container. Ideally if # of runners == # 
> of CPUs, build time is improved, on any type of instances.
>  ** For instance when using the current configuration, running on medium 
> instances, build will use 1 junit test runner, but 2 CPUs are available. If 
> using 2 runners, the build time is reduced from 19min (at the current main 
> config of parallelism=4) to 12min.
>  * There are some typos in the file, some dtests say "Run Unit Tests" but 
> they are JVM dtests (see 
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077],
>  
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])
> So some ways to process these would be:
>  * Do the Python dtests run successfully for anyone on {{medium}} instances? 
> If not, would it make sense to bump them to {{large}} so that they can be run 
> successfully?
>  * Does anybody ever run the python upgrade tests on CircleCI and what is the 
> configuration that makes it work?
>  * Would it make sense to either hardcode the number of test runners in the 
> unit tests with `-Dtest.runners` in the config file to reflect the number of 
> CPUs on the instances, or change the build so that it is able to detect the 
> appropriate number of core available automatically?
> Additionally, it seems this default config file (config.yml) is not as well 
> maintained as the 
> [{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml]
>  (+its lowres/highres) version in the same folder (from CASSANDRA-14806). 
> What is the reasoning for maintaining these 2 versions of the build? Could 
> the better maintained version be used as the default? We could generate a 
> lowres version of the new config-2_1.yml, and rename it {{config.yml}} so 
> that it gets picked up by CircleCI automatically instead of the current 
> default.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15338) Fix flakey testMessagePurging - org.apache.cassandra.net.ConnectionTest

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080639#comment-17080639
 ] 

David Capwell commented on CASSANDRA-15338:
---

nope, still too much on my plate.  This patch requires me to review closer 
which I am struggling with the time on atm =(

> Fix flakey testMessagePurging - org.apache.cassandra.net.ConnectionTest
> ---
>
> Key: CASSANDRA-15338
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15338
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Yifan Cai
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: CASS-15338-Docker.zip
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Example failure: 
> [https://circleci.com/gh/dcapwell/cassandra/11#artifacts/containers/1]
>   
> {code:java}
> Testcase: testMessagePurging(org.apache.cassandra.net.ConnectionTest):  FAILED
>  expected:<0> but was:<1>
>  junit.framework.AssertionFailedError: expected:<0> but was:<1>
>    at 
> org.apache.cassandra.net.ConnectionTest.lambda$testMessagePurging$38(ConnectionTest.java:625)
>    at 
> org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:258)
>    at 
> org.apache.cassandra.net.ConnectionTest.testManual(ConnectionTest.java:231)
>    at 
> org.apache.cassandra.net.ConnectionTest.testMessagePurging(ConnectionTest.java:584){code}
>   
>  Looking closer at 
> org.apache.cassandra.net.OutboundConnection.Delivery#stopAndRun it seems that 
> the run method is called before 
> org.apache.cassandra.net.OutboundConnection.Delivery#doRun which may lead to 
> a test race condition where the CountDownLatch completes before executing



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15709) SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080632#comment-17080632
 ] 

David Capwell commented on CASSANDRA-15709:
---

Been running in a loop for awhile without issue.  Most of my comments are small 
or nits, so assuming you resolve them I am +1

> SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure
> ---
>
> Key: CASSANDRA-15709
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15709
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> {code}
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest 
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.656 sec
> [junit-timeout] 
> [junit-timeout] Testcase: 
> changingMaxWorkersMeetsConcurrencyGoalsTest(org.apache.cassandra.concurrent.SEPExecutorTest):
>FAILED
> [junit-timeout] expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but 
> was:
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.assertMaxTaskConcurrency(SEPExecutorTest.java:180)
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.changingMaxWorkersMeetsConcurrencyGoalsTest(SEPExecutorTest.java:110)
> [junit-timeout] 
> [junit-timeout] 
> [junit-timeout] Test org.apache.cassandra.concurrent.SEPExecutorTest FAILED
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15709) SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080625#comment-17080625
 ] 

David Capwell commented on CASSANDRA-15709:
---

patch mostly LGTM.  I am currently testing to see how flaky it is before/after. 
 

Without limiting resources I was able to cause it to fail quickly, running with 
your patch now.

> SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure
> ---
>
> Key: CASSANDRA-15709
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15709
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> {code}
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest 
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.656 sec
> [junit-timeout] 
> [junit-timeout] Testcase: 
> changingMaxWorkersMeetsConcurrencyGoalsTest(org.apache.cassandra.concurrent.SEPExecutorTest):
>FAILED
> [junit-timeout] expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but 
> was:
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.assertMaxTaskConcurrency(SEPExecutorTest.java:180)
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.changingMaxWorkersMeetsConcurrencyGoalsTest(SEPExecutorTest.java:110)
> [junit-timeout] 
> [junit-timeout] 
> [junit-timeout] Test org.apache.cassandra.concurrent.SEPExecutorTest FAILED
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15709) SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CASSANDRA-15709:
---
Labels: pull-request-available  (was: )

> SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure
> ---
>
> Key: CASSANDRA-15709
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15709
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>
> {code}
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest 
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.656 sec
> [junit-timeout] 
> [junit-timeout] Testcase: 
> changingMaxWorkersMeetsConcurrencyGoalsTest(org.apache.cassandra.concurrent.SEPExecutorTest):
>FAILED
> [junit-timeout] expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but 
> was:
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.assertMaxTaskConcurrency(SEPExecutorTest.java:180)
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.changingMaxWorkersMeetsConcurrencyGoalsTest(SEPExecutorTest.java:110)
> [junit-timeout] 
> [junit-timeout] 
> [junit-timeout] Test org.apache.cassandra.concurrent.SEPExecutorTest FAILED
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15709) SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CASSANDRA-15709:
---
Labels: pull-request-available  (was: )

> SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure
> ---
>
> Key: CASSANDRA-15709
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15709
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Jon Meredith
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>
> {code}
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest 
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.656 sec
> [junit-timeout] 
> [junit-timeout] Testcase: 
> changingMaxWorkersMeetsConcurrencyGoalsTest(org.apache.cassandra.concurrent.SEPExecutorTest):
>FAILED
> [junit-timeout] expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but 
> was:
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.assertMaxTaskConcurrency(SEPExecutorTest.java:180)
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.changingMaxWorkersMeetsConcurrencyGoalsTest(SEPExecutorTest.java:110)
> [junit-timeout] 
> [junit-timeout] 
> [junit-timeout] Test org.apache.cassandra.concurrent.SEPExecutorTest FAILED
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15551) Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-15551:
-
  Since Version: 4.0-alpha
Source Control Link: 
https://github.com/apache/cassandra/commit/2344e77c026f1ecfed4f9ba84dab21ede2cf5b26
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Nice work, great analysis.  Committed, thanks!

> Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal 
> and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes
> 
>
> Key: CASSANDRA-15551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15551
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> testStateJumpToNormal failure was on java 11
> {code}
> java.lang.NullPointerException
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1028)
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1023)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2513)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testStateJumpToNormal(MoveTest.java:935)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes failure was on 
> java 8
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:2174)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2511)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes(MoveTest.java:199)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15551) Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-15551:
-
Reviewers: Brandon Williams, Brandon Williams  (was: Brandon Williams)
   Brandon Williams, Brandon Williams
   Status: Review In Progress  (was: Patch Available)

> Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal 
> and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes
> 
>
> Key: CASSANDRA-15551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15551
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> testStateJumpToNormal failure was on java 11
> {code}
> java.lang.NullPointerException
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1028)
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1023)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2513)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testStateJumpToNormal(MoveTest.java:935)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes failure was on 
> java 8
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:2174)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2511)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes(MoveTest.java:199)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15551) Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-15551:
-
Status: Ready to Commit  (was: Review In Progress)

> Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal 
> and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes
> 
>
> Key: CASSANDRA-15551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15551
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> testStateJumpToNormal failure was on java 11
> {code}
> java.lang.NullPointerException
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1028)
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1023)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2513)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testStateJumpToNormal(MoveTest.java:935)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes failure was on 
> java 8
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:2174)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2511)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes(MoveTest.java:199)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Fixed race condition in MoveTest by waiting for stale endpoints to be evicted from membership before starting another test.

[brandonwilliams]

This is an automated email from the ASF dual-hosted git repository.

brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 2344e77  Fixed race condition in MoveTest by waiting for stale 
endpoints to be evicted from membership before starting another test.
2344e77 is described below

commit 2344e77c026f1ecfed4f9ba84dab21ede2cf5b26
Author: Gianluca Righetto 
AuthorDate: Thu Apr 9 18:55:10 2020 -0300

Fixed race condition in MoveTest by waiting for stale endpoints to be 
evicted from membership before starting another test.

Patch by Gianluca Righetto, reviewed by brandonwilliams for CASSANDRA-15551.
---
 .../org/apache/cassandra/service/MoveTest.java | 36 +-
 1 file changed, 35 insertions(+), 1 deletion(-)

diff --git a/test/unit/org/apache/cassandra/service/MoveTest.java 
b/test/unit/org/apache/cassandra/service/MoveTest.java
index a4da7b8..9777602 100644
--- a/test/unit/org/apache/cassandra/service/MoveTest.java
+++ b/test/unit/org/apache/cassandra/service/MoveTest.java
@@ -21,10 +21,18 @@ package org.apache.cassandra.service;
 
 import java.net.UnknownHostException;
 import java.util.*;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.locks.Condition;
+import java.util.function.Consumer;
 
 import com.google.common.collect.HashMultimap;
 import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.Multimap;
+
+import org.apache.cassandra.config.OverrideConfigurationLoader;
+import org.apache.cassandra.diag.DiagnosticEventService;
+import org.apache.cassandra.gms.GossiperEvent;
 import org.apache.cassandra.locator.EndpointsForRange;
 import org.apache.cassandra.locator.EndpointsForToken;
 import org.apache.cassandra.locator.RangesAtEndpoint;
@@ -32,7 +40,11 @@ import org.apache.cassandra.locator.RangesByEndpoint;
 import org.junit.AfterClass;
 import org.junit.Before;
 import org.junit.BeforeClass;
+import org.junit.Rule;
 import org.junit.Test;
+import org.junit.rules.TestRule;
+import org.junit.rules.TestWatcher;
+import org.junit.runner.Description;
 
 import org.apache.cassandra.locator.InetAddressAndPort;
 import org.apache.cassandra.locator.Replica;
@@ -99,6 +111,7 @@ public class MoveTest
 addNetworkTopologyKeyspace(Network_11_KeyspaceName, 1, 1);
 addNetworkTopologyKeyspace(Network_22_KeyspaceName, 2, 2);
 addNetworkTopologyKeyspace(Network_33_KeyspaceName, 3, 3);
+DatabaseDescriptor.setDiagnosticEventsEnabled(true);
 }
 
 @AfterClass
@@ -108,10 +121,31 @@ public class MoveTest
 }
 
 @Before
-public void clearTokenMetadata()
+public void clearTokenMetadata() throws InterruptedException
 {
+// we expect to have a single endpoint before running each test method,
+// so we have to wait for the GossipStage thread to evict stale 
endpoints
+// from membership before moving on, otherwise it may break other 
tests as
+// things change in the background
+final int endpointCount = Gossiper.instance.getEndpointCount() - 1;
+final CountDownLatch latch = new CountDownLatch(endpointCount);
+Consumer onEndpointEvicted = event -> latch.countDown();
+DiagnosticEventService.instance().subscribe(GossiperEvent.class,
+
GossiperEvent.GossiperEventType.EVICTED_FROM_MEMBERSHIP,
+onEndpointEvicted);
+
 PendingRangeCalculatorService.instance.blockUntilFinished();
 StorageService.instance.getTokenMetadata().clearUnsafe();
+
+try
+{
+if (!latch.await(1, TimeUnit.MINUTES))
+throw new RuntimeException("Took too long to evict stale 
endpoints.");
+}
+finally
+{
+DiagnosticEventService.instance().unsubscribe(onEndpointEvicted);
+}
 }
 
 private static void addNetworkTopologyKeyspace(String keyspaceName, 
Integer... replicas) throws Exception


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15709) SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15709:
--
Reviewers: David Capwell, Dinesh Joshi, David Capwell  (was: David Capwell, 
Dinesh Joshi)
   David Capwell, Dinesh Joshi, David Capwell  (was: Dinesh Joshi)
   Status: Review In Progress  (was: Patch Available)

> SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure
> ---
>
> Key: CASSANDRA-15709
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15709
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> {code}
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest 
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.656 sec
> [junit-timeout] 
> [junit-timeout] Testcase: 
> changingMaxWorkersMeetsConcurrencyGoalsTest(org.apache.cassandra.concurrent.SEPExecutorTest):
>FAILED
> [junit-timeout] expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but 
> was:
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.assertMaxTaskConcurrency(SEPExecutorTest.java:180)
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.changingMaxWorkersMeetsConcurrencyGoalsTest(SEPExecutorTest.java:110)
> [junit-timeout] 
> [junit-timeout] 
> [junit-timeout] Test org.apache.cassandra.concurrent.SEPExecutorTest FAILED
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15551) Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes

[Gianluca Righetto (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080606#comment-17080606
 ] 

Gianluca Righetto edited comment on CASSANDRA-15551 at 4/10/20, 4:11 PM:
-

The issue here is that once the this line is executed in MoveTest's @Before 
method, {{StorageService.instance.getTokenMetadata().clearUnsafe()}}, the 
{{GossipStage}} thread kicks in and starts evicting the stale endpoints from 
membership, which may happen in parallel while another test method is already 
running.

To reproduce this in an IDE, you can set breakpoints at:

[https://github.com/apache/cassandra/blob/1ce3c1c039561c15892115af37e0c7abf260bc6b/test/unit/org/apache/cassandra/Util.java#L222]

and

[https://github.com/apache/cassandra/blob/1ce3c1c039561c15892115af37e0c7abf260bc6b/src/java/org/apache/cassandra/gms/Gossiper.java#L524]

If the main thread starts executing the second iteration of the loop in 
{{createInitialRing}} while the GossipStage thread is removing the endpoints in 
{{evictFromMembership}}, it will throw a NPE down the road.

The fix I submitted basically makes the main thread wait for all endpoints to 
be evicted in between tests, such that the next test starts in a clean state.

Pull request: [https://github.com/apache/cassandra/pull/533]
 Java 11 Unit Tests results: [https://circleci.com/gh/grighetto/cassandra/68]
 Java 8 Unit Tests results: [https://circleci.com/gh/grighetto/cassandra/65]


was (Author: gianluca):
The issue here is that once the this line is executed in the @Before setup 
method, {{StorageService.instance.getTokenMetadata().clearUnsafe()}}, the 
{{GossipStage}} thread kicks in and starts evicting the stale endpoints from 
membership, which may happen in parallel while another test method is already 
running.

To reproduce this in an IDE, you can set breakpoints at:

https://github.com/apache/cassandra/blob/1ce3c1c039561c15892115af37e0c7abf260bc6b/test/unit/org/apache/cassandra/Util.java#L222

and

https://github.com/apache/cassandra/blob/1ce3c1c039561c15892115af37e0c7abf260bc6b/src/java/org/apache/cassandra/gms/Gossiper.java#L524

If the main thread starts executing the second iteration of the loop in 
{{createInitialRing}} while the GossipStage thread is removing the endpoints in 
{{evictFromMembership}}, it will throw a NPE down the road.

The fix I submitted basically makes the main thread wait for all endpoints to 
be evicted in between tests, such that the next test starts in a clean state.

Pull request: https://github.com/apache/cassandra/pull/533
Java 11 Unit Tests results: https://circleci.com/gh/grighetto/cassandra/68
Java 8 Unit Tests results: https://circleci.com/gh/grighetto/cassandra/65

> Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal 
> and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes
> 
>
> Key: CASSANDRA-15551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15551
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> testStateJumpToNormal failure was on java 11
> {code}
> java.lang.NullPointerException
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1028)
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1023)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2513)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testStateJumpToNormal(MoveTest.java:935)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes failure was on 
> java 8
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:2174)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2511)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> 

[jira] [Updated] (CASSANDRA-15551) Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes

[Gianluca Righetto (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gianluca Righetto updated CASSANDRA-15551:
--
Test and Documentation Plan: 
For testing, check the reproduction steps in my comment below and verify that 
with the patch, the main thread waits while the endpoints are being evicted, 
before running another test method.
Unit tests results attached for both Java 11 and 8.
Since this is a change only in the test realm, no further documentation is 
required.
 Status: Patch Available  (was: In Progress)

> Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal 
> and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes
> 
>
> Key: CASSANDRA-15551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15551
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> testStateJumpToNormal failure was on java 11
> {code}
> java.lang.NullPointerException
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1028)
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1023)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2513)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testStateJumpToNormal(MoveTest.java:935)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes failure was on 
> java 8
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:2174)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2511)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes(MoveTest.java:199)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-15709) SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure

[Dinesh Joshi (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi reassigned CASSANDRA-15709:


Reviewers: Dinesh Joshi
 Assignee: Jon Meredith

> SEPExecutorTest changingMaxWorkersMeetsConcurrencyGoalsTest failure
> ---
>
> Key: CASSANDRA-15709
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15709
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> {code}
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest
> [junit-timeout] Testsuite: org.apache.cassandra.concurrent.SEPExecutorTest 
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.656 sec
> [junit-timeout] 
> [junit-timeout] Testcase: 
> changingMaxWorkersMeetsConcurrencyGoalsTest(org.apache.cassandra.concurrent.SEPExecutorTest):
>FAILED
> [junit-timeout] expected: but was:
> [junit-timeout] junit.framework.AssertionFailedError: expected: but 
> was:
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.assertMaxTaskConcurrency(SEPExecutorTest.java:180)
> [junit-timeout]   at 
> org.apache.cassandra.concurrent.SEPExecutorTest.changingMaxWorkersMeetsConcurrencyGoalsTest(SEPExecutorTest.java:110)
> [junit-timeout] 
> [junit-timeout] 
> [junit-timeout] Test org.apache.cassandra.concurrent.SEPExecutorTest FAILED
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15551) Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes

[Gianluca Righetto (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080606#comment-17080606
 ] 

Gianluca Righetto commented on CASSANDRA-15551:
---

The issue here is that once the this line is executed in the @Before setup 
method, {{StorageService.instance.getTokenMetadata().clearUnsafe()}}, the 
{{GossipStage}} thread kicks in and starts evicting the stale endpoints from 
membership, which may happen in parallel while another test method is already 
running.

To reproduce this in an IDE, you can set breakpoints at:

https://github.com/apache/cassandra/blob/1ce3c1c039561c15892115af37e0c7abf260bc6b/test/unit/org/apache/cassandra/Util.java#L222

and

https://github.com/apache/cassandra/blob/1ce3c1c039561c15892115af37e0c7abf260bc6b/src/java/org/apache/cassandra/gms/Gossiper.java#L524

If the main thread starts executing the second iteration of the loop in 
{{createInitialRing}} while the GossipStage thread is removing the endpoints in 
{{evictFromMembership}}, it will throw a NPE down the road.

The fix I submitted basically makes the main thread wait for all endpoints to 
be evicted in between tests, such that the next test starts in a clean state.

Pull request: https://github.com/apache/cassandra/pull/533
Java 11 Unit Tests results: https://circleci.com/gh/grighetto/cassandra/68
Java 8 Unit Tests results: https://circleci.com/gh/grighetto/cassandra/65

> Fix flaky tests org.apache.cassandra.service.MoveTest testStateJumpToNormal 
> and testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes
> 
>
> Key: CASSANDRA-15551
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15551
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> testStateJumpToNormal failure was on java 11
> {code}
> java.lang.NullPointerException
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1028)
>   at org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1023)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2513)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testStateJumpToNormal(MoveTest.java:935)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes failure was on 
> java 8
> {code}
> java.lang.NullPointerException
>   at 
> org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:2174)
>   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2511)
>   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2055)
>   at org.apache.cassandra.Util.createInitialRing(Util.java:225)
>   at 
> org.apache.cassandra.service.MoveTest.testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes(MoveTest.java:199)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15338) Fix flakey testMessagePurging - org.apache.cassandra.net.ConnectionTest

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080602#comment-17080602
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15338:
-

I [~dcapwell] are you still on that?

> Fix flakey testMessagePurging - org.apache.cassandra.net.ConnectionTest
> ---
>
> Key: CASSANDRA-15338
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15338
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Yifan Cai
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: CASS-15338-Docker.zip
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Example failure: 
> [https://circleci.com/gh/dcapwell/cassandra/11#artifacts/containers/1]
>   
> {code:java}
> Testcase: testMessagePurging(org.apache.cassandra.net.ConnectionTest):  FAILED
>  expected:<0> but was:<1>
>  junit.framework.AssertionFailedError: expected:<0> but was:<1>
>    at 
> org.apache.cassandra.net.ConnectionTest.lambda$testMessagePurging$38(ConnectionTest.java:625)
>    at 
> org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:258)
>    at 
> org.apache.cassandra.net.ConnectionTest.testManual(ConnectionTest.java:231)
>    at 
> org.apache.cassandra.net.ConnectionTest.testMessagePurging(ConnectionTest.java:584){code}
>   
>  Looking closer at 
> org.apache.cassandra.net.OutboundConnection.Delivery#stopAndRun it seems that 
> the run method is called before 
> org.apache.cassandra.net.OutboundConnection.Delivery#doRun which may lead to 
> a test race condition where the CountDownLatch completes before executing



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15262) server_encryption_options is not backwards compatible with 3.11

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080596#comment-17080596
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15262:
-

Hey [~jolynch],
Thanks for the update. Are you still working on this? Should we find anyone 
else?

> server_encryption_options is not backwards compatible with 3.11
> ---
>
> Key: CASSANDRA-15262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15262
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Config
>Reporter: Joey Lynch
>Assignee: Joey Lynch
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> The current `server_encryption_options` configuration options are as follows:
> {noformat}
> server_encryption_options:
> # set to true for allowing secure incoming connections
> enabled: false
> # If enabled and optional are both set to true, encrypted and unencrypted 
> connections are handled on the storage_port
> optional: false
> # if enabled, will open up an encrypted listening socket on 
> ssl_storage_port. Should be used
> # during upgrade to 4.0; otherwise, set to false.
> enable_legacy_ssl_storage_port: false
> # on outbound connections, determine which type of peers to securely 
> connect to. 'enabled' must be set to true.
> internode_encryption: none
> keystore: conf/.keystore
> keystore_password: cassandra
> truststore: conf/.truststore
> truststore_password: cassandra
> # More advanced defaults below:
> # protocol: TLS
> # store_type: JKS
> # cipher_suites: 
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> # require_client_auth: false
> # require_endpoint_verification: false
> {noformat}
> A couple of issues here:
> 1. optional defaults to false, which will break existing TLS configurations 
> for (from what I can tell) no particularly good reason
> 2. The provided protocol and cipher suites are not good ideas (in particular 
> encouraging anyone to use CBC ciphers is a bad plan
> I propose that before the 4.0 cut we fixup server_encryption_options and even 
> client_encryption_options :
> # Change the default {{optional}} setting to true. As the new Netty code 
> intelligently decides to open a TLS connection or not this is the more 
> sensible default (saves operators a step while transitioning to TLS as well)
> # Update the defaults to what netty actually defaults to



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15712) Introduce MIDRES config in CircleCI

[Kevin Gallardo (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080555#comment-17080555
 ] 

Kevin Gallardo commented on CASSANDRA-15712:


Note that it would be better to solve 
https://issues.apache.org/jira/browse/CASSANDRA-15686 before this issue

> Introduce MIDRES config in CircleCI
> ---
>
> Key: CASSANDRA-15712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15712
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest, Test/unit
>Reporter: Kevin Gallardo
>Priority: Normal
>
> From document: 
> [https://gist.github.com/newkek/bb79dccbe7d2f5e41b2a3daac3858fde]
> We have identified that the current HIGHRES configuration seems to require 
> resources that might not bring the best cost efficiency to the build.
> We have also identified several "good compromise" configurations that allow 
> to get decent performance out of the test suites, without going all out on 
> the big config.
> It seems it would be useful for a lot of people to have this available as a 
> {{config.yml.MIDRES}} configuration in the {{.circleci}} folder. This way we 
> do not need to argue on modifying the {{HIGHRES}} configuration so as to not 
> impact the people already using it , but can still have easy access the 
> "compromise" config.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15712) Introduce MIDRES config in CircleCI

[Kevin Gallardo (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080556#comment-17080556
 ] 

Kevin Gallardo commented on CASSANDRA-15712:


[~dcapwell] FYI, if you have any feedback

> Introduce MIDRES config in CircleCI
> ---
>
> Key: CASSANDRA-15712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15712
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest, Test/unit
>Reporter: Kevin Gallardo
>Priority: Normal
>
> From document: 
> [https://gist.github.com/newkek/bb79dccbe7d2f5e41b2a3daac3858fde]
> We have identified that the current HIGHRES configuration seems to require 
> resources that might not bring the best cost efficiency to the build.
> We have also identified several "good compromise" configurations that allow 
> to get decent performance out of the test suites, without going all out on 
> the big config.
> It seems it would be useful for a lot of people to have this available as a 
> {{config.yml.MIDRES}} configuration in the {{.circleci}} folder. This way we 
> do not need to argue on modifying the {{HIGHRES}} configuration so as to not 
> impact the people already using it, but can still have easy access the 
> "compromise" config.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15712) Introduce MIDRES config in CircleCI

[Kevin Gallardo (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Gallardo updated CASSANDRA-15712:
---
Description: 
>From document: 
>[https://gist.github.com/newkek/bb79dccbe7d2f5e41b2a3daac3858fde]

We have identified that the current HIGHRES configuration seems to require 
resources that might not bring the best cost efficiency to the build.

We have also identified several "good compromise" configurations that allow to 
get decent performance out of the test suites, without going all out on the big 
config.

It seems it would be useful for a lot of people to have this available as a 
{{config.yml.MIDRES}} configuration in the {{.circleci}} folder. This way we do 
not need to argue on modifying the {{HIGHRES}} configuration so as to not 
impact the people already using it, but can still have easy access the 
"compromise" config.

  was:
>From document: 
>[https://gist.github.com/newkek/bb79dccbe7d2f5e41b2a3daac3858fde]

We have identified that the current HIGHRES configuration seems to require 
resources that might not bring the best cost efficiency to the build.

We have also identified several "good compromise" configurations that allow to 
get decent performance out of the test suites, without going all out on the big 
config.

It seems it would be useful for a lot of people to have this available as a 
{{config.yml.MIDRES}} configuration in the {{.circleci}} folder. This way we do 
not need to argue on modifying the {{HIGHRES}} configuration so as to not 
impact the people already using it , but can still have easy access the 
"compromise" config.


> Introduce MIDRES config in CircleCI
> ---
>
> Key: CASSANDRA-15712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15712
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest, Test/unit
>Reporter: Kevin Gallardo
>Priority: Normal
>
> From document: 
> [https://gist.github.com/newkek/bb79dccbe7d2f5e41b2a3daac3858fde]
> We have identified that the current HIGHRES configuration seems to require 
> resources that might not bring the best cost efficiency to the build.
> We have also identified several "good compromise" configurations that allow 
> to get decent performance out of the test suites, without going all out on 
> the big config.
> It seems it would be useful for a lot of people to have this available as a 
> {{config.yml.MIDRES}} configuration in the {{.circleci}} folder. This way we 
> do not need to argue on modifying the {{HIGHRES}} configuration so as to not 
> impact the people already using it, but can still have easy access the 
> "compromise" config.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15712) Introduce MIDRES config in CircleCI

[Kevin Gallardo (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Gallardo updated CASSANDRA-15712:
---
  Workflow: Cassandra Default Workflow  (was: Cassandra Bug Workflow)
Issue Type: Improvement  (was: Bug)

> Introduce MIDRES config in CircleCI
> ---
>
> Key: CASSANDRA-15712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15712
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest, Test/unit
>Reporter: Kevin Gallardo
>Priority: Normal
>
> From document: 
> [https://gist.github.com/newkek/bb79dccbe7d2f5e41b2a3daac3858fde]
> We have identified that the current HIGHRES configuration seems to require 
> resources that might not bring the best cost efficiency to the build.
> We have also identified several "good compromise" configurations that allow 
> to get decent performance out of the test suites, without going all out on 
> the big config.
> It seems it would be useful for a lot of people to have this available as a 
> {{config.yml.MIDRES}} configuration in the {{.circleci}} folder. This way we 
> do not need to argue on modifying the {{HIGHRES}} configuration so as to not 
> impact the people already using it , but can still have easy access the 
> "compromise" config.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-15712) Introduce MIDRES config in CircleCI

[Kevin Gallardo (Jira)]

Kevin Gallardo created CASSANDRA-15712:
--

 Summary: Introduce MIDRES config in CircleCI
 Key: CASSANDRA-15712
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15712
 Project: Cassandra
  Issue Type: Bug
  Components: Test/dtest, Test/unit
Reporter: Kevin Gallardo


>From document: 
>[https://gist.github.com/newkek/bb79dccbe7d2f5e41b2a3daac3858fde]

We have identified that the current HIGHRES configuration seems to require 
resources that might not bring the best cost efficiency to the build.

We have also identified several "good compromise" configurations that allow to 
get decent performance out of the test suites, without going all out on the big 
config.

It seems it would be useful for a lot of people to have this available as a 
{{config.yml.MIDRES}} configuration in the {{.circleci}} folder. This way we do 
not need to argue on modifying the {{HIGHRES}} configuration so as to not 
impact the people already using it , but can still have easy access the 
"compromise" config.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15686) Improvements in circle CI default config

[Kevin Gallardo (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080546#comment-17080546
 ] 

Kevin Gallardo commented on CASSANDRA-15686:


{quote}and saw more failures than normal (normally ~1-3 failures, had 9; most 
known flaky tests)
{quote}
Hm that's a bummer, I think I saw similar failures when testing out too, but 
did not investigate more and thought they were just flaky. Do you think these 
could be fixed, or put in another bucket to run in another non-parallel suite?

For reference, this is the tests I had run a little while ago (about 2 weeks) 
for the *unit* tests only, on parallelizing the tests on current LOWRES config:
 - [Run of p=4 and i=medium with no change to the current build (which means it 
uses 1 runner by 
default)|https://app.circleci.com/pipelines/github/newkek/cassandra/4/workflows/16897387-cbf0-49e8-82bc-327d75885256]
 - [Same hardware, but hardcoding 
{{-Dtest.runners=2}}|https://app.circleci.com/pipelines/github/newkek/cassandra/5/workflows/56b341b5-8f74-414e-852e-52cba8acd28a]

You can see the runtime goes from 19min to 12.

I am looking at old experiments I ran, and it seems I was able to run with 8 
runners on unit tests, and all that was reported seem like failures from flaky 
tests: 
[run|https://app.circleci.com/pipelines/github/newkek/cassandra/22/workflows/68bf0582-2b43-4b71-a610-e1e76c6ff418/jobs/90]
 and [corresponding 
config|https://github.com/newkek/cassandra/blob/00103c51d3c6b1fd0c534b649e146bbe45bcdf11/.circleci/config.yml#L975]

> Improvements in circle CI default config
> 
>
> Key: CASSANDRA-15686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15686
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: Kevin Gallardo
>Priority: Normal
>
> I have been looking at and played around with the [default CircleCI 
> config|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml], 
> a few comments/questions regarding the following topics:
>  * Python dtests do not run successfully (200-300 failures) on {{medium}} 
> instances, they seem to only run with small flaky failures on {{large}} 
> instances or higher
>  * Python Upgrade tests:
>  ** Do not seem to run without many failures on any instance types / any 
> parallelism setting
>  ** Do not seem to parallelize well, it seems each container is going to 
> download multiple C* versions
>  ** Additionally it seems the configuration is not up to date, as currently 
> we get errors because {{JAVA8_HOME}} is not set
>  * Unit tests do not seem to parallelize optimally, number of test runners do 
> not reflect the available CPUs on the container. Ideally if # of runners == # 
> of CPUs, build time is improved, on any type of instances.
>  ** For instance when using the current configuration, running on medium 
> instances, build will use 1 junit test runner, but 2 CPUs are available. If 
> using 2 runners, the build time is reduced from 19min (at the current main 
> config of parallelism=4) to 12min.
>  * There are some typos in the file, some dtests say "Run Unit Tests" but 
> they are JVM dtests (see 
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1077],
>  
> [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config.yml#L1386])
> So some ways to process these would be:
>  * Do the Python dtests run successfully for anyone on {{medium}} instances? 
> If not, would it make sense to bump them to {{large}} so that they can be run 
> successfully?
>  * Does anybody ever run the python upgrade tests on CircleCI and what is the 
> configuration that makes it work?
>  * Would it make sense to either hardcode the number of test runners in the 
> unit tests with `-Dtest.runners` in the config file to reflect the number of 
> CPUs on the instances, or change the build so that it is able to detect the 
> appropriate number of core available automatically?
> Additionally, it seems this default config file (config.yml) is not as well 
> maintained as the 
> [{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml]
>  (+its lowres/highres) version in the same folder (from CASSANDRA-14806). 
> What is the reasoning for maintaining these 2 versions of the build? Could 
> the better maintained version be used as the default? We could generate a 
> lowres version of the new config-2_1.yml, and rename it {{config.yml}} so 
> that it gets picked up by CircleCI automatically instead of the current 
> default.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15229) BufferPool Regression

[Benedict Elliott Smith (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080534#comment-17080534
 ] 

Benedict Elliott Smith edited comment on CASSANDRA-15229 at 4/10/20, 2:39 PM:
--

Hi [~jasonstack], I'm on leave at the moment, so I cannot properly review your 
patch.  This was in the vicinity of what I had originally been considering, 
however I see one potential problem with this specific instantiation of the 
approach, and wonder anyway if we shouldn't take a different tack:

Recirculating immediately will lead to greater inefficiency in allocation, as 
we will attempt to reuse partially freed chunks in preference to entirely freed 
chunks, leading to a great deal more churn in the active blocks. This will 
affect the networking pooling as much as the chunk cache.  At the very least 
this behaviour should be enabled only for the {{ChunkCache}}, but ideally might 
have e.g. two queues, one with guaranteed-free chunks, another (perhaps for 
ease a superset) containing those chunks that might or mightn't be free.

This also isn't a _trivial_ behavioural change, and I continue to wonder if 
using {{Unsafe.allocateMemory}} wouldn't be simpler, more efficient, less risky 
and produce less fragmentation.


was (Author: benedict):
Hi [~jasonstack], I'm on leave at the moment, so I cannot properly review your 
patch.  However I see some potential problems:

Recirculating immediately will lead to greater inefficiency in allocation, as 
we will attempt to reuse partially freed chunks in preference to entirely freed 
chunks, leading to a great deal more churn in the active blocks. This will 
affect the networking pooling as much as the chunk cache.  At the very least 
this behaviour should be enabled only for the {{ChunkCache}}, but ideally might 
have e.g. two queues, one with guaranteed-free chunks, another (perhaps for 
ease a superset) containing those chunks that might or mightn't be free.

This isn't a _trivial_ behavioural change, and I continue to wonder if using 
{{Unsafe.allocateMemory}} wouldn't be simpler, more efficient, less risky and 
produce less fragmentation.

> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-hit-rate.png, 
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png, 
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we 
> need to either change our behaviour to handle uncorrelated lifetimes or use 
> something else.  This is particularly important with the default chunk size 
> for compressed sstables being reduced.  If we address the problem, we should 
> also utilise the BufferPool for native transport connections like we do for 
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used 
> for things with uncorrelated lifetimes, which essentially boils down to 
> tracking those chunks that have not been freed and re-circulating them when 
> we run out of completely free blocks.  We should probably also permit 
> instantiating separate {{BufferPool}}, so that we can insulate internode 
> messaging from the {{ChunkCache}}, or at least have separate memory bounds 
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk 
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce 
> the amount of global coordination and per-allocation overhead.  We don’t need 
> 1KiB granularity for allocations, nor 16 byte granularity for tiny 
> allocations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15229) BufferPool Regression

[Benedict Elliott Smith (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080534#comment-17080534
 ] 

Benedict Elliott Smith commented on CASSANDRA-15229:


Hi [~jasonstack], I'm on leave at the moment, so I cannot properly review your 
patch.  However I see some potential problems:

Recirculating immediately will lead to greater inefficiency in allocation, as 
we will attempt to reuse partially freed chunks in preference to entirely freed 
chunks, leading to a great deal more churn in the active blocks. This will 
affect the networking pooling as much as the chunk cache.  At the very least 
this behaviour should be enabled only for the {{ChunkCache}}, but ideally might 
have e.g. two queues, one with guaranteed-free chunks, another (perhaps for 
ease a superset) containing those chunks that might or mightn't be free.

This isn't a _trivial_ behavioural change, and I continue to wonder if using 
{{Unsafe.allocateMemory}} wouldn't be simpler, more efficient, less risky and 
produce less fragmentation.

> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-hit-rate.png, 
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png, 
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we 
> need to either change our behaviour to handle uncorrelated lifetimes or use 
> something else.  This is particularly important with the default chunk size 
> for compressed sstables being reduced.  If we address the problem, we should 
> also utilise the BufferPool for native transport connections like we do for 
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used 
> for things with uncorrelated lifetimes, which essentially boils down to 
> tracking those chunks that have not been freed and re-circulating them when 
> we run out of completely free blocks.  We should probably also permit 
> instantiating separate {{BufferPool}}, so that we can insulate internode 
> messaging from the {{ChunkCache}}, or at least have separate memory bounds 
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk 
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce 
> the amount of global coordination and per-allocation overhead.  We don’t need 
> 1KiB granularity for allocations, nor 16 byte granularity for tiny 
> allocations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15671) Testcase: testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): FAILED

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-15671:
-
  Since Version: 4.0
Source Control Link: 
https://github.com/apache/cassandra/commit/1ce3c1c039561c15892115af37e0c7abf260bc6b
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Committed, thanks!

> Testcase: 
> testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest):
>FAILED
> --
>
> Key: CASSANDRA-15671
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15671
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Francisco Fernandez
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The following test failure was observed:
> [junit-timeout] Testcase: 
> testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest):
>FAILED
> [junit-timeout] expected:<4> but was:<5>
> [junit-timeout] junit.framework.AssertionFailedError: expected:<4> but was:<5>
> [junit-timeout]   at 
> org.apache.cassandra.db.compaction.CancelCompactionsTest.testSubrangeCompaction(CancelCompactionsTest.java:190)
> Java 8



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15671) Testcase: testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest): FAILED

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-15671:
-
Reviewers: Brandon Williams, Ekaterina Dimitrova  (was: Ekaterina Dimitrova)

> Testcase: 
> testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest):
>FAILED
> --
>
> Key: CASSANDRA-15671
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15671
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Francisco Fernandez
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0, 4.0-beta
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The following test failure was observed:
> [junit-timeout] Testcase: 
> testSubrangeCompaction(org.apache.cassandra.db.compaction.CancelCompactionsTest):
>FAILED
> [junit-timeout] expected:<4> but was:<5>
> [junit-timeout] junit.framework.AssertionFailedError: expected:<4> but was:<5>
> [junit-timeout]   at 
> org.apache.cassandra.db.compaction.CancelCompactionsTest.testSubrangeCompaction(CancelCompactionsTest.java:190)
> Java 8



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Use only test triggered compactions for test assertions on CancelCompactionsTest.

[brandonwilliams]

This is an automated email from the ASF dual-hosted git repository.

brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 1ce3c1c  Use only test triggered compactions for test assertions on 
CancelCompactionsTest.
1ce3c1c is described below

commit 1ce3c1c039561c15892115af37e0c7abf260bc6b
Author: Francisco Fernandez Castano 
AuthorDate: Tue Mar 31 16:21:28 2020 +0200

Use only test triggered compactions for test assertions on 
CancelCompactionsTest.

Patch by Francisco Fernandez Castano, reviewed by Ekaterina Dimitrova
and brandonwilliams for CASSANDRA-15671
---
 .../db/compaction/CancelCompactionsTest.java   | 34 +-
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git 
a/test/unit/org/apache/cassandra/db/compaction/CancelCompactionsTest.java 
b/test/unit/org/apache/cassandra/db/compaction/CancelCompactionsTest.java
index 60433bb..beed019 100644
--- a/test/unit/org/apache/cassandra/db/compaction/CancelCompactionsTest.java
+++ b/test/unit/org/apache/cassandra/db/compaction/CancelCompactionsTest.java
@@ -82,7 +82,7 @@ public class CancelCompactionsTest extends CQLTester
 {
 tct.start();
 
-List activeCompactions = 
CompactionManager.instance.active.getCompactions();
+List activeCompactions = 
getActiveCompactionsForTable(cfs);
 assertEquals(1, activeCompactions.size());
 
assertEquals(activeCompactions.get(0).getCompactionInfo().getSSTables(), 
toMarkCompacting);
 // predicate requires the non-compacting sstables, should not 
cancel the one currently compacting:
@@ -128,7 +128,7 @@ public class CancelCompactionsTest extends CQLTester
 {
 tcts.forEach(TestCompactionTask::start);
 
-List activeCompactions = 
CompactionManager.instance.active.getCompactions();
+List activeCompactions = 
getActiveCompactionsForTable(cfs);
 assertEquals(2, activeCompactions.size());
 
 Set> compactingSSTables = new HashSet<>();
@@ -147,7 +147,7 @@ public class CancelCompactionsTest extends CQLTester
 // start a compaction which only needs the sstables where first 
token is > 50 - these are the sstables compacted by tcts.get(1)
 Thread t = new Thread(() -> cfs.runWithCompactionsDisabled(() -> { 
cdl.countDown(); return null; }, (sstable) -> first(sstable) > 50, false, 
false, true));
 t.start();
-activeCompactions = 
CompactionManager.instance.active.getCompactions();
+activeCompactions = getActiveCompactionsForTable(cfs);
 assertEquals(2, activeCompactions.size());
 Thread.sleep(500);
 for (CompactionInfo.Holder holder : activeCompactions)
@@ -186,7 +186,7 @@ public class CancelCompactionsTest extends CQLTester
 {
 tcts.forEach(TestCompactionTask::start);
 
-List activeCompactions = 
CompactionManager.instance.active.getCompactions();
+List activeCompactions = 
getActiveCompactionsForTable(cfs);
 assertEquals(4, activeCompactions.size());
 Range range = new Range<>(token(0), token(49));
 Thread t = new Thread(() -> {
@@ -203,9 +203,9 @@ public class CancelCompactionsTest extends CQLTester
 t.start();
 
 Thread.sleep(500);
-assertEquals(4, 
CompactionManager.instance.active.getCompactions().size());
+assertEquals(4, getActiveCompactionsForTable(cfs).size());
 List toAbort = new ArrayList<>();
-for (CompactionInfo.Holder holder : 
CompactionManager.instance.active.getCompactions())
+for (CompactionInfo.Holder holder : 
getActiveCompactionsForTable(cfs))
 {
 if 
(holder.getCompactionInfo().getSSTables().stream().anyMatch(sstable -> 
sstable.intersects(Collections.singleton(range
 {
@@ -250,7 +250,7 @@ public class CancelCompactionsTest extends CQLTester
 {
 tcts.forEach(TestCompactionTask::start);
 nonAffectedTcts.forEach(TestCompactionTask::start);
-List activeCompactions = 
CompactionManager.instance.active.getCompactions();
+List activeCompactions = 
getActiveCompactionsForTable(cfs);
 assertEquals(5, activeCompactions.size());
 // make sure that sstables are fully contained so that the 
metadata gets mutated
 Range range = new Range<>(token(-1), token(49));
@@ -265,7 +265,7 @@ public class CancelCompactionsTest extends CQLTester
 Future fut = pac.run();
 Thread.sleep(600);
 List toAbort = new ArrayList<>();
-for (CompactionInfo.Holder holder : 
CompactionManager.instance.active.getCompactions())
+for (CompactionInfo.Holder holder : 

[jira] [Assigned] (CASSANDRA-1291) nodetool ring prints incorrect IP address

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams reassigned CASSANDRA-1291:
---

Assignee: Brandon Williams  (was: Amikam Kaminsky)

> nodetool ring prints incorrect IP address
> -
>
> Key: CASSANDRA-1291
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1291
> Project: Cassandra
>  Issue Type: Bug
>Affects Versions: 0.7 beta 1
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 0.7 beta 2
>
> Attachments: 1291.txt
>
>
> Nodetool's ring output on trunk seems to duplicate an ipaddress instead of 
> printing them all.
> To reproduce: spin up a 3 node cluster and create a keyspace, examine 
> nodetool ring.  One ip address will show up twice, though the tokens are 
> correctly displayed for the three machines.
> This is a cosmetic error, if you call getLiveNodes via JMX you can see all 
> three IPs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080472#comment-17080472
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15667 at 4/10/20, 1:11 PM:
---

I am looking now at the test as I triggered it to run again 100 times yesterday 
and it seems there is some error. I am gonna check it and update the ticket 
later



was (Author: e.dimitrova):
I am looking now at the test as I triggered it to run again 100 times yesterday 
and it seems there is some error. I am gonna figure it out and update the 
ticket later


> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamResultFuture#maybeComplete()}} uses 
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are 
> completed, but then accesses each session state via 
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the 
> former relies on the actual {{StreamSession}} state, while the latter on the 
> {{SessionInfo}} state, and the two are concurrently updated with no 
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as 
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc 
> [~e.dimitrova].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080472#comment-17080472
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15667:
-

I am looking now at the test as I triggered it to run again 100 times yesterday 
and it seems there is some error. I am gonna figure it out and update the 
ticket later


> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamResultFuture#maybeComplete()}} uses 
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are 
> completed, but then accesses each session state via 
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the 
> former relies on the actual {{StreamSession}} state, while the latter on the 
> {{SessionInfo}} state, and the two are concurrently updated with no 
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as 
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc 
> [~e.dimitrova].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-1291) nodetool ring prints incorrect IP address

[Amikam Kaminsky (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amikam Kaminsky reassigned CASSANDRA-1291:
--

Assignee: Amikam Kaminsky  (was: Brandon Williams)

> nodetool ring prints incorrect IP address
> -
>
> Key: CASSANDRA-1291
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1291
> Project: Cassandra
>  Issue Type: Bug
>Affects Versions: 0.7 beta 1
>Reporter: Brandon Williams
>Assignee: Amikam Kaminsky
>Priority: Normal
> Fix For: 0.7 beta 2
>
> Attachments: 1291.txt
>
>
> Nodetool's ring output on trunk seems to duplicate an ipaddress instead of 
> printing them all.
> To reproduce: spin up a 3 node cluster and create a keyspace, examine 
> nodetool ring.  One ip address will show up twice, though the tokens are 
> correctly displayed for the three machines.
> This is a cosmetic error, if you call getLiveNodes via JMX you can see all 
> three IPs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15666) Race condition when completing stream sessions

[ZhaoYang (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-15666:
-
Status: Patch Available  (was: Review In Progress)

Found some bootstrap dtest failures.. looking into them

> Race condition when completing stream sessions
> --
>
> Key: CASSANDRA-15666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15666
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamSession#prepareAsync()}} executes, as the name implies, 
> asynchronously from the IO thread: this opens up for race conditions between 
> the sending of the {{PrepareSynAckMessage}} and the call to 
> {{StreamSession#maybeCompleted()}}. I.e., the following could happen:
> 1) Node A sends {{PrepareSynAckMessage}} from the {{prepareAsync()}} thread.
> 2) Node B receives it and starts streaming.
> 3) Node A receives the streamed file and sends {{ReceivedMessage}}.
> 4) At this point, if this was the only file to stream, both nodes are ready 
> to close the session via {{maybeCompleted()}}, but:
> a) Node A will call it twice from both the IO thread and the thread at #1, 
> closing the session and its channels.
> b) Node B will attempt to send a {{CompleteMessage}}, but will fail because 
> the session has been closed in the meantime.
> There are other subtle variations of the pattern above, depending on the 
> order of concurrently sent/received messages.
> I believe the best fix would be to modify the message exchange so that:
> 1) Only the "follower" is allowed to send the {{CompleteMessage}}.
> 2) Only the "initiator" is allowed to close the session and its channels 
> after receiving the {{CompleteMessage}}.
> By doing so, the message exchange logic would be easier to reason about, 
> which is overall a win anyway.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15666) Race condition when completing stream sessions

[ZhaoYang (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-15666:
-
Status: In Progress  (was: Patch Available)

> Race condition when completing stream sessions
> --
>
> Key: CASSANDRA-15666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15666
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamSession#prepareAsync()}} executes, as the name implies, 
> asynchronously from the IO thread: this opens up for race conditions between 
> the sending of the {{PrepareSynAckMessage}} and the call to 
> {{StreamSession#maybeCompleted()}}. I.e., the following could happen:
> 1) Node A sends {{PrepareSynAckMessage}} from the {{prepareAsync()}} thread.
> 2) Node B receives it and starts streaming.
> 3) Node A receives the streamed file and sends {{ReceivedMessage}}.
> 4) At this point, if this was the only file to stream, both nodes are ready 
> to close the session via {{maybeCompleted()}}, but:
> a) Node A will call it twice from both the IO thread and the thread at #1, 
> closing the session and its channels.
> b) Node B will attempt to send a {{CompleteMessage}}, but will fail because 
> the session has been closed in the meantime.
> There are other subtle variations of the pattern above, depending on the 
> order of concurrently sent/received messages.
> I believe the best fix would be to modify the message exchange so that:
> 1) Only the "follower" is allowed to send the {{CompleteMessage}}.
> 2) Only the "initiator" is allowed to close the session and its channels 
> after receiving the {{CompleteMessage}}.
> By doing so, the message exchange logic would be easier to reason about, 
> which is overall a win anyway.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15657) Improve zero-copy-streaming containment check by using file sections

[ZhaoYang (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-15657:
-
Test and Documentation Plan: 
CI: [https://circleci.com/workflow-run/a36ff018-cbb5-455f-88bb-40bfd1ca4cea]
 

  was:CI: https://circleci.com/workflow-run/f74e2716-554d-489a-abc8-7893610bd93d


> Improve zero-copy-streaming containment check by using file sections
> 
>
> Key: CASSANDRA-15657
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15657
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/Streaming and Messaging
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0
>
>
> Currently zero copy streaming is only enabled for leveled-compaction strategy 
> and it checks if all keys in the sstables are included in the transferred 
> ranges.
> This is very inefficient. The containment check can be improved by checking 
> if transferred sections (the transferred file positions) cover entire sstable.
> I also enabled ZCS for all compaction strategies since the new containment 
> check is very fast..



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15660) Unable to specify -e/--execute flag in cqlsh

[ZhaoYang (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-15660:
-
Test and Documentation Plan: 
CI pending: 
[https://circleci.com/workflow-run/8e4b4739-cb7c-40ad-a24d-8d7c0027eb4b]

Added regression cqlsh dtest.
 

  was:
CI pending: 
https://circleci.com/workflow-run/aa05f724-4ade-4ed1-9c54-884c77c2bd16

Added regression cqlsh dtest.


> Unable to specify -e/--execute flag in cqlsh
> 
>
> Key: CASSANDRA-15660
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15660
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Stefan Miklosovic
>Assignee: ZhaoYang
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-alpha
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> From mailing list:
> [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E]
> The bug looks like this:
> {code:java}
> $ /usr/bin/cqlsh -e 'describe keyspaces' -u cassandra -p cassandra 127.0.0.1
> Usage: cqlsh.py [options] [host [port]]cqlsh.py: error: '127.0.0.1' is not a 
> valid port number.
> {code}
> This is working in 3.x releases just fine but fails on 4.
> The workaround for 4.x code as of today is to put these statements into file 
> and use "-f" flag.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15666) Race condition when completing stream sessions

[ZhaoYang (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-15666:
-
Test and Documentation Plan: 
Added interceptor to verify stream messages and state transition.

 CI pending: 
[https://circleci.com/workflow-run/54fa1412-65ec-4580-9f62-a81eaa0e6ec2]
 

  was:
Added interceptor to verify stream messages and state transition.

 CI pending: 
https://circleci.com/workflow-run/adc6807e-eed5-4ecd-b5c0-537437633e72


> Race condition when completing stream sessions
> --
>
> Key: CASSANDRA-15666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15666
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamSession#prepareAsync()}} executes, as the name implies, 
> asynchronously from the IO thread: this opens up for race conditions between 
> the sending of the {{PrepareSynAckMessage}} and the call to 
> {{StreamSession#maybeCompleted()}}. I.e., the following could happen:
> 1) Node A sends {{PrepareSynAckMessage}} from the {{prepareAsync()}} thread.
> 2) Node B receives it and starts streaming.
> 3) Node A receives the streamed file and sends {{ReceivedMessage}}.
> 4) At this point, if this was the only file to stream, both nodes are ready 
> to close the session via {{maybeCompleted()}}, but:
> a) Node A will call it twice from both the IO thread and the thread at #1, 
> closing the session and its channels.
> b) Node B will attempt to send a {{CompleteMessage}}, but will fail because 
> the session has been closed in the meantime.
> There are other subtle variations of the pattern above, depending on the 
> order of concurrently sent/received messages.
> I believe the best fix would be to modify the message exchange so that:
> 1) Only the "follower" is allowed to send the {{CompleteMessage}}.
> 2) Only the "initiator" is allowed to close the session and its channels 
> after receiving the {{CompleteMessage}}.
> By doing so, the message exchange logic would be easier to reason about, 
> which is overall a win anyway.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15665) StreamManager should clearly differentiate between "initiator" and "receiver" sessions

[ZhaoYang (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-15665:
-
Test and Documentation Plan: CI pending: 
[https://circleci.com/workflow-run/b8aed041-cb39-4e86-9a8f-189c02a08857]  (was: 
CI pending: 
[https://circleci.com/workflow-run/896f8753-3b8a-47ed-959e-f2806ae3d052])

> StreamManager should clearly differentiate between "initiator" and "receiver" 
> sessions
> --
>
> Key: CASSANDRA-15665
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15665
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamManager}} does currently a suboptimal job in differentiating between 
> stream sessions (in form of {{StreamResultFuture}}) which have been either 
> initiated or "received", for the following reasons:
> 1) Naming is IMO confusing: a "receiver" session could actually both send and 
> receive files, so technically an initiator is also a receiver.
> 2) {{StreamManager#findSession()}}  assumes we should first looking into 
> "initiator" sessions, then into "receiver" ones: this is a dangerous 
> assumptions, in particular for test environments where the same process could 
> work as both an initiator and a receiver.
> I would recommend the following changes:
> 1) Rename "receiver" with "follower" everywhere the former is used.
> 2) Introduce a new flag into {{StreamMessageHeader}} to signal if the message 
> comes from an initiator or follower session, in order to correctly 
> differentiate and look for sessions in {{StreamManager}}.
> While my arguments above might seem trivial, I believe they will improve 
> clarity and save from potential bugs/headaches at testing time, and doing 
> such changes now that we're revamping streaming for 4.0 seems the right time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15229) BufferPool Regression

[ZhaoYang (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080314#comment-17080314
 ] 

ZhaoYang edited comment on CASSANDRA-15229 at 4/10/20, 7:51 AM:


Discussed with [~stefania] offline, there are two issues with buffer pool:
 * Chunk cache holds a piece of buffer preventing entire chunk from recycling 
for arbitrary period.
 * Even if we recirculate the partially freed chunk, due to different 
allocation sizes, fragmentation will reduce utilization. That's why forked 
version uses uniform allocation size.

The first issue should be solvable and less risky for 4.0.. Here is the 
performance comparison against recirculating partially freed chunk.

Setup: single node 16T - 8GB heap - 250m rows - mixed read 40k qps - write 10k 
qps - with 128 file cache
 [baseline|https://github.com/jasonstack/cassandra/pull/8]: initiate 2 buffer 
pools, one for chunk cache, one for network.
 
[recirculate-partially-freed-chunk|https://github.com/jasonstack/cassandra/pull/11/files]:
 baseline + partially freed chunk recirculation.

baseline:
| !15229-hit-rate.png|height=400,width=400! | 
!15229-count.png|height=400,width=400! |

recirculation: 
| !15229-recirculate-hit-rate.png|height=400,width=400! | 
!15229-recirculate-count.png|height=400,width=400! |

QPS:  
!15229-recirculate.png|height=600,width=600! 

With partially freed chunk recirculation, latency is improved and buffer pool 
misses are reduced..

Should we proceed with recirculating partially freed chunk + a separate pool 
for network cache in 4.0 and then port forked buffer pool with uniform 
allocation size in 4.x?


was (Author: jasonstack):
Discussed with [~stefania] offline, there are two issues with buffer pool:
 * Chunk cache holds a piece of buffer preventing entire chunk from recycling 
for arbitrary period.
 * Even if we recirculate the partially freed chunk, due to different 
allocation sizes, fragmentation will reduce utilization. That's why forked 
version uses uniform allocation size.

The first issue should be solvable and less risky for 4.0.. Here is the 
performance comparison against recirculating partially freed chunk.

Setup: single node 16T - 8GB heap - 250m rows - mixed read 40k qps - write 10k 
qps - with 128 file cache
 [baseline|https://github.com/jasonstack/cassandra/pull/8]: initiate 2 buffer 
pools, one for chunk cache, one for network.
 
[recirculate-partially-freed-chunk|https://github.com/jasonstack/cassandra/pull/11/files]:
 baseline + partially freed chunk recirculation.

baseline:
| !15229-hit-rate.png|height=250,width=250! | 
!15229-count.png|height=250,width=250! |

recirculation: 
| !15229-recirculate-hit-rate.png|height=250,width=250! | 
!15229-recirculate-count.png|height=250,width=250! |



QPS:  
!15229-recirculate.png|height=400,width=400! 

With partially freed chunk recirculation, latency is improved and buffer pool 
misses are reduced..

Should we proceed with recirculating partially freed chunk + a separate pool 
for network cache in 4.0 and then port forked buffer pool with uniform 
allocation size in 4.x?

> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-hit-rate.png, 
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png, 
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we 
> need to either change our behaviour to handle uncorrelated lifetimes or use 
> something else.  This is particularly important with the default chunk size 
> for compressed sstables being reduced.  If we address the problem, we should 
> also utilise the BufferPool for native transport connections like we do for 
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used 
> for things with uncorrelated lifetimes, which essentially boils down to 
> tracking those chunks that have not been freed and re-circulating them when 
> we run out of completely free blocks.  We should probably also permit 
> instantiating separate {{BufferPool}}, so that we can insulate internode 
> messaging from the {{ChunkCache}}, or at least have separate memory bounds 
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk 
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce 
> the amount of global coordination and per-allocation 

[jira] [Comment Edited] (CASSANDRA-15229) BufferPool Regression

[ZhaoYang (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080314#comment-17080314
 ] 

ZhaoYang edited comment on CASSANDRA-15229 at 4/10/20, 7:50 AM:


Discussed with [~stefania] offline, there are two issues with buffer pool:
 * Chunk cache holds a piece of buffer preventing entire chunk from recycling 
for arbitrary period.
 * Even if we recirculate the partially freed chunk, due to different 
allocation sizes, fragmentation will reduce utilization. That's why forked 
version uses uniform allocation size.

The first issue should be solvable and less risky for 4.0.. Here is the 
performance comparison against recirculating partially freed chunk.

Setup: single node 16T - 8GB heap - 250m rows - mixed read 40k qps - write 10k 
qps - with 128 file cache
 [baseline|https://github.com/jasonstack/cassandra/pull/8]: initiate 2 buffer 
pools, one for chunk cache, one for network.
 
[recirculate-partially-freed-chunk|https://github.com/jasonstack/cassandra/pull/11/files]:
 baseline + partially freed chunk recirculation.

baseline:
| !15229-hit-rate.png|height=250,width=250! | 
!15229-count.png|height=250,width=250! |

recirculation: 
| !15229-recirculate-hit-rate.png|height=250,width=250! | 
!15229-recirculate-count.png|height=250,width=250! |



QPS:  
!15229-recirculate.png|height=400,width=400! 

With partially freed chunk recirculation, latency is improved and buffer pool 
misses are reduced..

Should we proceed with recirculating partially freed chunk + a separate pool 
for network cache in 4.0 and then port forked buffer pool with uniform 
allocation size in 4.x?


was (Author: jasonstack):
Discussed with [~stefania] offline, there are two issues with buffer pool:
 * Chunk cache holds a piece of buffer preventing entire chunk from recycling 
for arbitrary period.
 * Even if we recirculate the partially freed chunk, due to different 
allocation sizes, fragmentation will reduce utilization. That's why forked 
version uses uniform allocation size.

The first issue should be solvable and less risky for 4.0.. Here is the 
performance comparison against recirculating partially freed chunk.

Setup: single node 16T - 8GB heap - 250m rows - mixed read 40k qps - write 10k 
qps - with 128 file cache
 [baseline|https://github.com/jasonstack/cassandra/pull/8]: initiate 2 buffer 
pools, one for chunk cache, one for network.
 
[recirculate-partially-freed-chunk|https://github.com/jasonstack/cassandra/pull/11/files]:
 baseline + partially freed chunk recirculation.

baseline: | !15229-hit-rate.png|thumbnail! | !15229-count.png|thumbnail! | 
!15229-size.png|thumbnail! |

recirculation: | !15229-recirculate-hit-rate.png|thumbnail! | 
!15229-recirculate-count.png|thumbnail! | 
!15229-recirculate-size.png|thumbnail! |

QPS:  !15229-recirculate.png|thumbnail! 

With partially freed chunk recirculation, latency is improved and buffer pool 
misses are reduced..

Should we proceed with recirculating partially freed chunk + a separate pool 
for network cache in 4.0 and then port forked buffer pool with uniform 
allocation size in 4.x?

> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-hit-rate.png, 
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png, 
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we 
> need to either change our behaviour to handle uncorrelated lifetimes or use 
> something else.  This is particularly important with the default chunk size 
> for compressed sstables being reduced.  If we address the problem, we should 
> also utilise the BufferPool for native transport connections like we do for 
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used 
> for things with uncorrelated lifetimes, which essentially boils down to 
> tracking those chunks that have not been freed and re-circulating them when 
> we run out of completely free blocks.  We should probably also permit 
> instantiating separate {{BufferPool}}, so that we can insulate internode 
> messaging from the {{ChunkCache}}, or at least have separate memory bounds 
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk 
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce 
> the amount of global coordination and 

[jira] [Commented] (CASSANDRA-15229) BufferPool Regression

[ZhaoYang (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080314#comment-17080314
 ] 

ZhaoYang commented on CASSANDRA-15229:
--

Discussed with [~stefania] offline, there are two issues with buffer pool:
 * Chunk cache holds a piece of buffer preventing entire chunk from recycling 
for arbitrary period.
 * Even if we recirculate the partially freed chunk, due to different 
allocation sizes, fragmentation will reduce utilization. That's why forked 
version uses uniform allocation size.

The first issue should be solvable and less risky for 4.0.. Here is the 
performance comparison against recirculating partially freed chunk.

Setup: single node 16T - 8GB heap - 250m rows - mixed read 40k qps - write 10k 
qps - with 128 file cache
 [baseline|https://github.com/jasonstack/cassandra/pull/8]: initiate 2 buffer 
pools, one for chunk cache, one for network.
 
[recirculate-partially-freed-chunk|https://github.com/jasonstack/cassandra/pull/11/files]:
 baseline + partially freed chunk recirculation.

baseline: | !15229-hit-rate.png|thumbnail! | !15229-count.png|thumbnail! | 
!15229-size.png|thumbnail! |

recirculation: | !15229-recirculate-hit-rate.png|thumbnail! | 
!15229-recirculate-count.png|thumbnail! | 
!15229-recirculate-size.png|thumbnail! |

QPS:  !15229-recirculate.png|thumbnail! 

With partially freed chunk recirculation, latency is improved and buffer pool 
misses are reduced..

Should we proceed with recirculating partially freed chunk + a separate pool 
for network cache in 4.0 and then port forked buffer pool with uniform 
allocation size in 4.x?

> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-hit-rate.png, 
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png, 
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we 
> need to either change our behaviour to handle uncorrelated lifetimes or use 
> something else.  This is particularly important with the default chunk size 
> for compressed sstables being reduced.  If we address the problem, we should 
> also utilise the BufferPool for native transport connections like we do for 
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used 
> for things with uncorrelated lifetimes, which essentially boils down to 
> tracking those chunks that have not been freed and re-circulating them when 
> we run out of completely free blocks.  We should probably also permit 
> instantiating separate {{BufferPool}}, so that we can insulate internode 
> messaging from the {{ChunkCache}}, or at least have separate memory bounds 
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk 
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce 
> the amount of global coordination and per-allocation overhead.  We don’t need 
> 1KiB granularity for allocations, nor 16 byte granularity for tiny 
> allocations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15229) BufferPool Regression

[ZhaoYang (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-15229:
-
Attachment: 15229-recirculate.png

> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-hit-rate.png, 
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png, 
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we 
> need to either change our behaviour to handle uncorrelated lifetimes or use 
> something else.  This is particularly important with the default chunk size 
> for compressed sstables being reduced.  If we address the problem, we should 
> also utilise the BufferPool for native transport connections like we do for 
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used 
> for things with uncorrelated lifetimes, which essentially boils down to 
> tracking those chunks that have not been freed and re-circulating them when 
> we run out of completely free blocks.  We should probably also permit 
> instantiating separate {{BufferPool}}, so that we can insulate internode 
> messaging from the {{ChunkCache}}, or at least have separate memory bounds 
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk 
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce 
> the amount of global coordination and per-allocation overhead.  We don’t need 
> 1KiB granularity for allocations, nor 16 byte granularity for tiny 
> allocations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15229) BufferPool Regression

[ZhaoYang (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-15229:
-
Attachment: 15229-size.png
15229-recirculate-size.png
15229-recirculate-hit-rate.png
15229-recirculate-count.png
15229-hit-rate.png
15229-count.png

> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-hit-rate.png, 
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png, 
> 15229-recirculate-size.png, 15229-size.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we 
> need to either change our behaviour to handle uncorrelated lifetimes or use 
> something else.  This is particularly important with the default chunk size 
> for compressed sstables being reduced.  If we address the problem, we should 
> also utilise the BufferPool for native transport connections like we do for 
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used 
> for things with uncorrelated lifetimes, which essentially boils down to 
> tracking those chunks that have not been freed and re-circulating them when 
> we run out of completely free blocks.  We should probably also permit 
> instantiating separate {{BufferPool}}, so that we can insulate internode 
> messaging from the {{ChunkCache}}, or at least have separate memory bounds 
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk 
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce 
> the amount of global coordination and per-allocation overhead.  We don’t need 
> 1KiB granularity for allocations, nor 16 byte granularity for tiny 
> allocations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races

[ZhaoYang (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080281#comment-17080281
 ] 

ZhaoYang commented on CASSANDRA-15667:
--

[~maxtomassi] thanks for the patch.. should we also fix 3.0 and 3.11 ?

> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamResultFuture#maybeComplete()}} uses 
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are 
> completed, but then accesses each session state via 
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the 
> former relies on the actual {{StreamSession}} state, while the latter on the 
> {{SessionInfo}} state, and the two are concurrently updated with no 
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as 
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc 
> [~e.dimitrova].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org