[jira] [Commented] (CASSANDRA-15369) Fake row deletions and range tombstones, causing digest mismatch and sstable growth

2020-10-02 Thread Zhao Yang (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206597#comment-17206597
 ] 

Zhao Yang commented on CASSANDRA-15369:
---

[~marcuse] do you still plan to review this ticket? should I find another 
reviewer? thanks..

> Fake row deletions and range tombstones, causing digest mismatch and sstable 
> growth
> ---
>
> Key: CASSANDRA-15369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15369
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination, Local/Memtable, Local/SSTable
>Reporter: Benedict Elliott Smith
>Assignee: Zhao Yang
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta, 4.0-triage
>
>
> As assessed in CASSANDRA-15363, we generate fake row deletions and fake 
> tombstone markers under various circumstances:
>  * If we perform a clustering key query (or select a compact column):
>  * Serving from a {{Memtable}}, we will generate fake row deletions
>  * Serving from an sstable, we will generate fake row tombstone markers
>  * If we perform a slice query, we will generate only fake row tombstone 
> markers for any range tombstone that begins or ends outside of the limit of 
> the requested slice
>  * If we perform a multi-slice or IN query, this will occur for each 
> slice/clustering
> Unfortunately, these different behaviours can lead to very different data 
> stored in sstables until a full repair is run.  When we read-repair, we only 
> send these fake deletions or range tombstones.  A fake row deletion, 
> clustering RT and slice RT, each produces a different digest.  So for each 
> single point lookup we can produce a digest mismatch twice, and until a full 
> repair is run we can encounter an unlimited number of digest mismatches 
> across different overlapping queries.
> Relatedly, this seems a more problematic variant of our atomicity failures 
> caused by our monotonic reads, since RTs can have an atomic effect across (up 
> to) the entire partition, whereas the propagation may happen on an 
> arbitrarily small portion.  If the RT exists on only one node, this could 
> plausibly lead to fairly problematic scenario if that node fails before the 
> range can be repaired. 
> At the very least, this behaviour can lead to an almost unlimited amount of 
> extraneous data being stored until the range is repaired and compaction 
> happens to overwrite the sub-range RTs and row deletions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15921) 4.0 quality testing: Materialized View

2020-10-02 Thread Zhao Yang (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206596#comment-17206596
 ] 

Zhao Yang commented on CASSANDRA-15921:
---

[~jmckenzie] I am now compiling a list of tickets as areas to improve, they are 
all marked as 4.x ..

> 4.0 quality testing: Materialized View
> --
>
> Key: CASSANDRA-15921
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15921
> Project: Cassandra
>  Issue Type: Task
>  Components: Feature/Materialized Views
>Reporter: Zhao Yang
>Assignee: Zhao Yang
>Priority: Normal
> Fix For: 4.x
>
> Attachments: C40_MV.png
>
>
> The main purpose of this ticket to get a better understanding about 4.0 MV 
> status as a guideline for future improvements. I don't think it should block 
> 4.0 release since it's already marked as experimental.
> Main areas to test:
>  * Write perf: We expect to see [10% write throughput drop per MV 
> added|https://www.datastax.com/blog/2016/05/materialized-view-performance-cassandra-3x].
> ** Attached C40_MV.png  is alpha-4, 5-node, rf3 MV write tests: with 1 mv, 
> throughput dropped 50%
>  * Read perf: identical to normal table
>  * Bootstrap/Decommision: no write-path required since CASSANDRA-13065
>  * Repair: write path required
>  * Chaos monkey: take down coordinator/base-replica/view-replica during 
> read/write/token-movement and verify data consistency (may need a tool)
>  * Hint Replay: able to throttle if table has MV - CASSANDRA-13810
>  * Schema race: create/drop - CASSANDRA-15845/CASSANDRA-15918



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16175) Avoid removing batch when it's not created during view replication

2020-10-02 Thread Zhao Yang (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhao Yang updated CASSANDRA-16175:
--
Fix Version/s: 4.x

> Avoid removing batch when it's not created during view replication
> --
>
> Key: CASSANDRA-16175
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16175
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Materialized Views
>Reporter: Zhao Yang
>Priority: Normal
> Fix For: 4.x
>
>
> When the base replica is also a view replica we don't write a local batchlog, 
> but they are unnecessarily removed when the view write is successful, what 
> creates (and persists) a tombstone into the system.batches table.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16174) During bootstrap streaming, skipping write path may create orphan MV rows

2020-10-02 Thread Zhao Yang (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhao Yang updated CASSANDRA-16174:
--
Fix Version/s: 4.x

> During bootstrap streaming, skipping write path may create orphan MV rows 
> --
>
> Key: CASSANDRA-16174
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16174
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Materialized Views
>Reporter: Zhao Yang
>Priority: Normal
> Fix For: 4.x
>
>
> CASSANDRA-13065 improves the speed of bootstrap streaming by skipping write 
> path for base table with MV.
> Unfortunately during bootstrapping, the bootstrapping node may receive a base 
> write that is already deleted or shadowed on other replicas, but the 
> bootstrapping node didn't receive all sstables yet thus bootstapping node may 
> create an orphan view row based on received sstables. Without write path, the 
> newer version data will not create tombstone to shadow orphan view row.
> For example, node-A has a base sstable containing: "k=1, v=1@2019", but 
> bootstrapping node didn't receive it yet. A write "k=1, v=0@2018" hitting on 
> bootstrapping node will create an orphan view row: "v=0@2018, k=1". Applying 
> streaming sstables from bootstrap or rebuild through write path can help, but 
> that's something we want to get rid of.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16172) materialized view can return stale data even with R+W>N

2020-10-02 Thread Zhao Yang (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhao Yang updated CASSANDRA-16172:
--
Fix Version/s: 4.x

> materialized view can return stale data even with R+W>N
> ---
>
> Key: CASSANDRA-16172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16172
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Materialized Views
>Reporter: Zhao Yang
>Priority: Normal
> Fix For: 4.x
>
>
> This is a similar issue as CASSANDRA-8272.
> With a key-value table on a 2-node cluster with RF2 and there is a MV that 
> put `value` as partition key.
> Insert with ConsistencyLevel.ONE twice with different data, assuming there is 
> a network partition between 2 nodes:
> - Node 1 received: pk="a" -> value=1 @ts1
> - Node 2 received: pk="a" -> value=2 @ts2 where ts2 > ts1
> When querying `value`=1 on MV with ConsistencyLevel.ALL, we got stale row: 
> "a" -> 1.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables

2020-10-02 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16063:

Fix Version/s: (was: 4.0-triage)

> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Compact_storage_upgrade_tests.txt
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The code to handle compact tables has been removed from 4.0, and the intended 
> upgrade path to 4.0 for users having compact tables on 3.x is that they must 
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables 
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table) 
> and may try upgrading despite still having compact tables. If they do so, the 
> intent is that the node will _not_ start, with a message clearly indicating 
> the pre-upgrade step the user has missed. The user will then downgrade back 
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and 
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables 
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within 
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
>   But by then, we've _at least_ called 
> {{SystemKeyspace.persistLocalMetadata()}}} and 
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log, 
> and even possibly flush new {{na}} format sstables. As a results, a user 
> might not be able to seemlessly restart the node on 3.x (to drop compact 
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0 
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0 
> with compact storage, you can downgrade back with no intervention whatsoever).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16109) Don't adjust nodeCount when setting node id topology in in-jvm dtests

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206566#comment-17206566
 ] 

David Capwell commented on CASSANDRA-16109:
---

CI has been unstable caused by the new feature to fail if unexpected errors 
happened... been running tests and trying to patch...

> Don't adjust nodeCount when setting node id topology in in-jvm dtests
> -
>
> Key: CASSANDRA-16109
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16109
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Low
>  Labels: pull-request-available
>
> We update the node count when setting the node id topology in in-jvm dtests, 
> this should only happen if node count is smaller than the node id topology, 
> otherwise bootstrap tests error out.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15958:
--
Fix Version/s: (was: 4.0-triage)

> org.apache.cassandra.net.ConnectionTest testMessagePurging
> --
>
> Key: CASSANDRA-15958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15958
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Build: 
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> Build: 
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> java.util.concurrent.TimeoutException
>   at org.apache.cassandra.net.AsyncPromise.get(AsyncPromise.java:258)
>   at org.apache.cassandra.net.FutureDelegate.get(FutureDelegate.java:143)
>   at 
> org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:268)
>   at 
> org.apache.cassandra.net.ConnectionTest.testManual(ConnectionTest.java:236)
>   at 
> org.apache.cassandra.net.ConnectionTest.testMessagePurging(ConnectionTest.java:679)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16078) Performance regression for queries accessing multiple rows

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16078:
--
Fix Version/s: (was: 4.0-triage)

> Performance regression for queries accessing multiple rows
> --
>
> Key: CASSANDRA-16078
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16078
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: image.png
>
>
> This is spin off from CASSANDRA-16036.
> In testing 4.0 relative to 3.0* I found that queries which accessed multiple 
> rows to have a noticeable performance decrease; two queries were used in the 
> test (more may be impacted, others might not): query partition (table has 
> clustering keys) with LIMIT, and query clustering keys using IN clause.
> In the below graphs the green line is 3.0 and the other lines are 4.0 (with 
> and without chunk cache)
> Partition with LIMIT
> !https://issues.apache.org/jira/secure/attachment/13009751/clustering-slice_latency_selects_baseline.png!
> !https://issues.apache.org/jira/secure/attachment/13009750/clustering-slice_latency_under90_selects_baseline.png!
> Cluster with IN clause
> !https://issues.apache.org/jira/secure/attachment/13009749/clustering-in-clause_latency_selects_baseline.png!
> !https://issues.apache.org/jira/secure/attachment/13009748/clustering-in-clause_latency_under90_selects_baseline.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15992) Fix flaky python dtest test_13595 - consistency_test.TestConsistency

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15992:
--
Fix Version/s: (was: 4.0-triage)

> Fix flaky python dtest test_13595 - consistency_test.TestConsistency
> 
>
> Key: CASSANDRA-15992
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15992
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/355/workflows/7b8df61d-706f-4094-a206-7cdc6b4e0451/jobs/1818
> {code}
> >   assert 9 == jmx.read_attribute(srp, 'Count')
> E   AssertionError: assert 9 == 5
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16097) DigestResolver.getData throws AssertionError since dataResponse is null

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16097:
--
Fix Version/s: (was: 4.0-triage)

> DigestResolver.getData throws AssertionError since dataResponse is null
> ---
>
> Key: CASSANDRA-16097
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16097
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception
> {code}
> 2020-09-02 21:08:59,872 ERROR [Native-Transport-Requests-35] 
> org.apache.cassandra.transport.Message - Unexpected exception during request; 
> channel = [id: 0x13bb89d4, L:/10.14.92.74:9042 - R:/10.14.89.248:47112]
> java.lang.AssertionError
>at 
> org.apache.cassandra.service.reads.DigestResolver.getData(DigestResolver.java:77)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.service.reads.AbstractReadExecutor.awaitResponses(AbstractReadExecutor.java:390)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1821) 
> ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1711) 
> ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1628) 
> ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:1097)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:294)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:246)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:498)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:476)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:138)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253) 
> ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
>  ~[apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>  [?:?]
>at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
>  [apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119) 
> [apache-cassandra-4.0.0-beta3.jar:4.0.0-beta3]
>at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>  [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.base/java.lang.Thread.run(Thread.java:834) [?:?]
> {code}
> This exception was not frequent, out of the whole run (3h) only saw this 
> twice.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16083) Missing JMX objects and attributes upgrading from 3.0 to 4.0

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16083:
--
Fix Version/s: (was: 4.0-triage)

> Missing JMX objects and attributes upgrading from 3.0 to 4.0
> 
>
> Key: CASSANDRA-16083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16083
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: David Capwell
>Assignee: Uchenna
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Using the tools added in CASSANDRA-16082, below are the list of metrics 
> missing in 4.0 but present in 3.0.  The work here is to make sure we had 
> proper deprecation for each metric, and if not to add it back.
> {code}
> $ tools/bin/jmxtool diff -f yaml cassandra-3.0-jmx.yaml trunk-jmx.yaml 
> --ignore-missing-on-left
> Objects not in right:
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=EstimatedPartitionSizeHistogram
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=BloomFilterFalseRatio
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ReplicaFilteringProtectionRequests
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=RowCacheHitOutOfRange
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=CounterMutationStage,name=MaxPoolSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=TombstoneScannedHistogram
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=WaitingOnFreeMemtableSpace
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasCommitTotalLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_aggregates,name=CasProposeLatency
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=ViewReadTime
> org.apache.cassandra.db:type=HintedHandoffManager
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=BloomFilterDiskSpaceUsed
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=RequestResponseStage,name=PendingTasks
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=MemtableSwitchCount
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=range_xfers,name=ReplicaFilteringProtectionRowsCachedPerQuery
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=SnapshotsSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=RecentBloomFilterFalsePositives
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=SpeculativeRetries
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=LiveDiskSpaceUsed
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ViewReadTime
> org.apache.cassandra.metrics:type=ThreadPools,path=internal,scope=MigrationStage,name=CompletedTasks
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=ViewReadTime
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=BloomFilterFalsePositives
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=CompressionMetadataOffHeapMemoryUsed
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=TotalBlockedTasks
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=LiveScannedHistogram
> org.apache.cassandra.db:type=Tables,keyspace=system,table=views_builds_in_progress
> 

[jira] [Updated] (CASSANDRA-16103) Invalid serialized size for responses

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16103:
--
Fix Version/s: (was: 4.0-triage)

> Invalid serialized size for responses
> -
>
> Key: CASSANDRA-16103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16103
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception; 
> this is related to CASSANDRA-16097 as it was found during the same test.
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized 
> size; expected 14, actual size at least 13, for verb HINT_RSP
>   at 
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
>   at 
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
>   at 
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
>   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>   at 
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
> {code}
> org.apache.cassandra.net.InvalidSerializedSizeException: Invalid serialized 
> size; expected 14, actual size at least 13, for verb MUTATION_RSP
>   at 
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:816)
>   at 
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
>   at 
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
>   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>   at 
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16106) BufferOverflow exception while writing response to buffer

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16106:
--
Fix Version/s: (was: 4.0-triage)

> BufferOverflow exception while writing response to buffer
> -
>
> Key: CASSANDRA-16106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16106
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Was running a benchmark at LOCAL_ONE and eventually saw the below exception; 
> this is related to CASSANDRA-16097 as it was found during the same test.
> {code}
> message="...SMALL_MESSAGES-1bb47c27 dropping message of type HINT_RSP due to 
> error"
> exception="java.nio.BufferOverflowException
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:153)
>   at 
> org.apache.cassandra.utils.vint.VIntCoding.writeUnsignedVInt(VIntCoding.java:191)
>   at 
> org.apache.cassandra.io.util.DataOutputPlus.writeUnsignedVInt(DataOutputPlus.java:55)
>   at 
> org.apache.cassandra.net.Message$Serializer.serializeHeaderPost40(Message.java:688)
>   at 
> org.apache.cassandra.net.Message$Serializer.serializePost40(Message.java:758)
>   at 
> org.apache.cassandra.net.Message$Serializer.serialize(Message.java:618)
>   at 
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:813)
>   at 
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
>   at 
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
>   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>   at 
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.base/java.lang.Thread.run(Thread.java:834)"
> {code}
> {code}
> message="...-SMALL_MESSAGES-e72423f4 dropping message of type MUTATION_RSP 
> due to error"
> exception="java.nio.BufferOverflowException
>   at 
> org.apache.cassandra.io.util.DataOutputBufferFixed.doFlush(DataOutputBufferFixed.java:52)
>   at 
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:153)
>   at 
> org.apache.cassandra.utils.vint.VIntCoding.writeUnsignedVInt(VIntCoding.java:191)
>   at 
> org.apache.cassandra.io.util.DataOutputPlus.writeUnsignedVInt(DataOutputPlus.java:55)
>   at 
> org.apache.cassandra.net.Message$Serializer.serializeParams(Message.java:1112)
>   at 
> org.apache.cassandra.net.Message$Serializer.serializeHeaderPost40(Message.java:689)
>   at 
> org.apache.cassandra.net.Message$Serializer.serializePost40(Message.java:758)
>   at 
> org.apache.cassandra.net.Message$Serializer.serialize(Message.java:618)
>   at 
> org.apache.cassandra.net.OutboundConnection$EventLoopDelivery.doRun(OutboundConnection.java:813)
>   at 
> org.apache.cassandra.net.OutboundConnection$Delivery.run(OutboundConnection.java:687)
>   at 
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
>   at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
>   at 
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>   at 
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.base/java.lang.Thread.run(Thread.java:834)"
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16114) Fix tests CQLTester.assertLastSchemaChange causes ClassCastException

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16114:
--
Fix Version/s: (was: 4.0-triage)

> Fix tests CQLTester.assertLastSchemaChange causes ClassCastException
> 
>
> Key: CASSANDRA-16114
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16114
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Cedric Nabaa
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Build: 
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/494/workflows/b3765545-7b09-48dd-85ff-830c4f348329/jobs/2681
> {code}
> java.lang.ClassCastException: 
> org.apache.cassandra.transport.messages.ResultMessage$Void cannot be cast to 
> org.apache.cassandra.transport.messages.ResultMessage$SchemaChange
>   at 
> org.apache.cassandra.cql3.CQLTester.assertLastSchemaChange(CQLTester.java:916)
>   at 
> org.apache.cassandra.cql3.validation.entities.UFTest.testSchemaChange(UFTest.java:94)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16120:
--
Fix Version/s: (was: 4.0-triage)

> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest 
> supports the ability to grep the logs of an instance; we need this capability 
> as some tests require validating logs were triggered.
> Pydocs for common log methods 
> {code}
> |  grep_log(self, expr, filename='system.log', from_mark=None)
> |  Returns a list of lines matching the regular expression in parameter
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors(self, filename='system.log')
> |  Returns a list of errors with stack traces
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> |  watch_log_for(self, exprs, from_mark=None, timeout=600, process=None, 
> verbose=False, filename='system.log')
> |  Watch the log until one or more (regular) expression are found.
> |  This methods when all the expressions have been found or the method
> |  timeouts (a TimeoutError is then raised). On successful completion,
> |  a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag", 
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename", 
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag, 
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l -> 
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws 
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws 
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new 
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it, 
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag", 
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename", 
> "suitename_IS_UNDEFINED");
>//TODO missing way to get node id
> //cluster.get(1);
>String log = 
> 

[jira] [Updated] (CASSANDRA-16176) Python dtests should run at debug

2020-10-02 Thread Brandon Williams (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16176:
-
Change Category: Quality Assurance
 Complexity: Normal
Component/s: CI
  Fix Version/s: 4.0-beta
   Assignee: Brandon Williams
 Status: Open  (was: Triage Needed)

> Python dtests should run at debug
> -
>
> Key: CASSANDRA-16176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16176
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta
>
>
> At least in our circle configs, we are specifying --log-level="INFO" which 
> often leaves us with nothing to go on when we have a flaky test that only 
> fails in a certain environment.  In general it would be more useful to always 
> run at DEBUG. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16176) Python dtests should run at debug

2020-10-02 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206542#comment-17206542
 ] 

Brandon Williams commented on CASSANDRA-16176:
--

I wholesale replaced 'INFO' with 'DEBUG': 
https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-16176

Nothing broke, but it also didn't produce debug output (outside of CCM, which 
is odd.)

> Python dtests should run at debug
> -
>
> Key: CASSANDRA-16176
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16176
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Brandon Williams
>Priority: Normal
>
> At least in our circle configs, we are specifying --log-level="INFO" which 
> often leaves us with nothing to go on when we have a flaky test that only 
> fails in a certain environment.  In general it would be more useful to always 
> run at DEBUG. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16169) Rename master branch to trunk in cassandra-in-jvm-dtest-api

2020-10-02 Thread Michael Semb Wever (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206540#comment-17206540
 ] 

Michael Semb Wever commented on CASSANDRA-16169:


https://lists.apache.org/thread.html/r6ba3e73a9d347f91d18167e5194f0179cabd59940c3a902e2acbf924%40%3Cdev.cassandra.apache.org%3E

> Rename master branch to trunk in cassandra-in-jvm-dtest-api
> ---
>
> Key: CASSANDRA-16169
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16169
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Michael Semb Wever
>Priority: Normal
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15902) OOM because repair session thread not closed when terminating repair

2020-10-02 Thread Swen Fuhrmann (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206541#comment-17206541
 ] 

Swen Fuhrmann commented on CASSANDRA-15902:
---

Thanks a lot [~adejanovski] for starting the review and the testing! 

> OOM because repair session thread not closed when terminating repair
> 
>
> Key: CASSANDRA-15902
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15902
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair
>Reporter: Swen Fuhrmann
>Assignee: Swen Fuhrmann
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
> Attachments: heap-mem-histo.txt, repair-terminated.txt
>
>
> In our cluster, after a while some nodes running slowly out of memory. On 
> that nodes we observed that Cassandra Reaper terminate repairs with a JMX 
> call to {{StorageServiceMBean.forceTerminateAllRepairSessions()}} because 
> reaching timeout of 30 min.
> In the memory heap dump we see lot of instances of 
> {{io.netty.util.concurrent.FastThreadLocalThread}} occupy most of the memory:
> {noformat}
> 119 instances of "io.netty.util.concurrent.FastThreadLocalThread", loaded by 
> "sun.misc.Launcher$AppClassLoader @ 0x51a80" occupy 8.445.684.480 (93,96 
> %) bytes. {noformat}
> In the thread dump we see lot of repair threads:
> {noformat}
> grep "Repair#" threaddump.txt | wc -l
>   50 {noformat}
>  
> The repair jobs are waiting for the validation to finish:
> {noformat}
> "Repair#152:1" #96170 daemon prio=5 os_prio=0 tid=0x12fc5000 
> nid=0x542a waiting on condition [0x7f81ee414000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x0007939bcfc8> (a 
> com.google.common.util.concurrent.AbstractFuture$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at 
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:285)
> at 
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at 
> com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137)
> at 
> com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1509)
> at org.apache.cassandra.repair.RepairJob.run(RepairJob.java:160)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at 
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> at 
> org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$13/480490520.run(Unknown
>  Source)
> at java.lang.Thread.run(Thread.java:748) {noformat}
>  
> Thats the line where the threads stuck:
> {noformat}
> // Wait for validation to complete
> Futures.getUnchecked(validations); {noformat}
>  
> The call to {{StorageServiceMBean.forceTerminateAllRepairSessions()}} stops 
> the thread pool executor. It looks like that futures which are in progress 
> will therefor never be completed and the repair thread waits forever and 
> won't be finished.
>  
> Environment:
> Cassandra version: 3.11.4 and 3.11.6
> Cassandra Reaper: 1.4.0
> JVM memory settings:
> {noformat}
> -Xms11771M -Xmx11771M -XX:+UseG1GC -XX:MaxGCPauseMillis=100 
> -XX:+ParallelRefProcEnabled -XX:MaxMetaspaceSize=100M {noformat}
> on another cluster with same issue:
> {noformat}
> -Xms31744M -Xmx31744M -XX:+UseG1GC -XX:MaxGCPauseMillis=100 
> -XX:+ParallelRefProcEnabled -XX:MaxMetaspaceSize=100M {noformat}
> Java Runtime:
> {noformat}
> openjdk version "1.8.0_212"
> OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_212-b03)
> OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.212-b03, mixed mode) 
> {noformat}
>  
> The same issue described in this comment: 
> https://issues.apache.org/jira/browse/CASSANDRA-14355?focusedCommentId=16992973=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16992973
> As suggested in the comments I created this new specific ticket.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For 

[jira] [Commented] (CASSANDRA-16167) Rename master branch to trunk in cassandra-sidecar

2020-10-02 Thread Michael Semb Wever (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206539#comment-17206539
 ] 

Michael Semb Wever commented on CASSANDRA-16167:


https://lists.apache.org/thread.html/r2215c44605a1de601b1958f93df84473e7b591df594e86f2e8d795e4%40%3Cdev.cassandra.apache.org%3E

> Rename master branch to trunk in cassandra-sidecar
> --
>
> Key: CASSANDRA-16167
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16167
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Michael Semb Wever
>Priority: Normal
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15944) ASF CI unit tests on JDK11

2020-10-02 Thread Michael Semb Wever (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15944:
---
Fix Version/s: (was: 4.0-triage)

> ASF CI unit tests on JDK11
> --
>
> Key: CASSANDRA-15944
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15944
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, CI
>Reporter: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0-beta
>
>
> ASF CI tests today only run on JDK1.8
> On the Jenkins cluster JDKs from 1.4 through to 15 are available. See 
> screenshot for naming specifics, attached in  CASSANDRA-15809
>  
> This ticket is to add JDK11 test targets on Cassandra-trunk and 
> Cassandra-devbranch, for parity to CircleCI's workflows.
>  
> The JDK is specified in the groovy DSL:
>  
> [https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L11]
>  
>  This is a continuation from CASSANDRA-15809 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16146) Node state incorrectly set to NORMAL after nodetool disablegossip and enablegossip during bootstrap

2020-10-02 Thread Yifan Cai (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206535#comment-17206535
 ] 

Yifan Cai commented on CASSANDRA-16146:
---

yep. 3.11 and trunk.

> Node state incorrectly set to NORMAL after nodetool disablegossip and 
> enablegossip during bootstrap
> ---
>
> Key: CASSANDRA-16146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16146
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta3
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> At high level, {{StorageService#setGossipTokens}} set the gossip state to 
> {{NORMAL}} blindly. Therefore, re-enabling gossip (stop and start gossip) 
> overrides the actual gossip state.
>   
> It could happen in the below scenario.
> # Bootstrap failed. The gossip state remains in {{BOOT}} / {{JOINING}} and 
> code execution exits StorageService#initServer.
> # Operator runs nodetool to stop and re-start gossip. The gossip state gets 
> flipped to {{NORMAL}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Assigned] (CASSANDRA-16166) Rename master branch to trunk in cassandra-dtest

2020-10-02 Thread Michael Semb Wever (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever reassigned CASSANDRA-16166:
--

Assignee: Michael Semb Wever

> Rename master branch to trunk in cassandra-dtest
> 
>
> Key: CASSANDRA-16166
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16166
> Project: Cassandra
>  Issue Type: Sub-task
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16146) Node state incorrectly set to NORMAL after nodetool disablegossip and enablegossip during bootstrap

2020-10-02 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206534#comment-17206534
 ] 

Brandon Williams commented on CASSANDRA-16146:
--

I am +1 on 3.0, are you still planning to do patches for other versions?

> Node state incorrectly set to NORMAL after nodetool disablegossip and 
> enablegossip during bootstrap
> ---
>
> Key: CASSANDRA-16146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16146
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta3
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> At high level, {{StorageService#setGossipTokens}} set the gossip state to 
> {{NORMAL}} blindly. Therefore, re-enabling gossip (stop and start gossip) 
> overrides the actual gossip state.
>   
> It could happen in the below scenario.
> # Bootstrap failed. The gossip state remains in {{BOOT}} / {{JOINING}} and 
> code execution exits StorageService#initServer.
> # Operator runs nodetool to stop and re-start gossip. The gossip state gets 
> flipped to {{NORMAL}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16146) Node state incorrectly set to NORMAL after nodetool disablegossip and enablegossip during bootstrap

2020-10-02 Thread Brandon Williams (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16146:
-
Fix Version/s: (was: 4.0-triage)

> Node state incorrectly set to NORMAL after nodetool disablegossip and 
> enablegossip during bootstrap
> ---
>
> Key: CASSANDRA-16146
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16146
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta3
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> At high level, {{StorageService#setGossipTokens}} set the gossip state to 
> {{NORMAL}} blindly. Therefore, re-enabling gossip (stop and start gossip) 
> overrides the actual gossip state.
>   
> It could happen in the below scenario.
> # Bootstrap failed. The gossip state remains in {{BOOT}} / {{JOINING}} and 
> code execution exits StorageService#initServer.
> # Operator runs nodetool to stop and re-start gossip. The gossip state gets 
> flipped to {{NORMAL}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16178) ByteBufferAccessor throws ClassCastException when trying to query system_views.local_read_latency

2020-10-02 Thread David Capwell (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16178:
--
Reviewers: Blake Eggleston, David Capwell, David Capwell  (was: Blake 
Eggleston, David Capwell)
   Blake Eggleston, David Capwell, David Capwell  (was: Blake 
Eggleston)
   Status: Review In Progress  (was: Patch Available)

> ByteBufferAccessor throws ClassCastException when trying to query 
> system_views.local_read_latency
> -
>
> Key: CASSANDRA-16178
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16178
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Tables
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>
> If you start up a fresh trunk/4.0 node and execute the query “SELECT * FROM 
> system_views.local_read_latency”, you’ll get the following error:
> ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099 
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.ClassCastException: 
> org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to 
> java.lang.String
> at 
> org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
> at 
> org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
> at 
> org.apache.cassandra.db.marshal.CompositeType.decompose(CompositeType.java:192)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.makeDecoratedKey(SimpleDataSet.java:87)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.row(SimpleDataSet.java:63)
> at 
> org.apache.cassandra.db.virtual.TableMetricTables$TableMetricTable.data(TableMetricTables.java:196)
> at 
> org.apache.cassandra.db.virtual.AbstractVirtualTable.select(AbstractVirtualTable.java:91)
> at 
> org.apache.cassandra.db.VirtualTablePartitionRangeReadQuery.queryVirtualTable(VirtualTablePartitionRangeReadQuery.java:93)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeLocally(VirtualTableReadQuery.java:61)
> at 
> org.apache.cassandra.db.AbstractReadQuery.executeInternal(AbstractReadQuery.java:64)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeInternal(VirtualTableReadQuery.java:32)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.execute(VirtualTableReadQuery.java:54)
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:75)
> at 
> org.apache.cassandra.service.pager.PartitionRangeQueryPager.fetchPage(PartitionRangeQueryPager.java:29)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:253)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
> at 
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16178) ByteBufferAccessor throws ClassCastException when trying to query system_views.local_read_latency

2020-10-02 Thread Blake Eggleston (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Blake Eggleston updated CASSANDRA-16178:

Reviewers: Blake Eggleston

> ByteBufferAccessor throws ClassCastException when trying to query 
> system_views.local_read_latency
> -
>
> Key: CASSANDRA-16178
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16178
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Tables
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>
> If you start up a fresh trunk/4.0 node and execute the query “SELECT * FROM 
> system_views.local_read_latency”, you’ll get the following error:
> ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099 
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.ClassCastException: 
> org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to 
> java.lang.String
> at 
> org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
> at 
> org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
> at 
> org.apache.cassandra.db.marshal.CompositeType.decompose(CompositeType.java:192)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.makeDecoratedKey(SimpleDataSet.java:87)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.row(SimpleDataSet.java:63)
> at 
> org.apache.cassandra.db.virtual.TableMetricTables$TableMetricTable.data(TableMetricTables.java:196)
> at 
> org.apache.cassandra.db.virtual.AbstractVirtualTable.select(AbstractVirtualTable.java:91)
> at 
> org.apache.cassandra.db.VirtualTablePartitionRangeReadQuery.queryVirtualTable(VirtualTablePartitionRangeReadQuery.java:93)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeLocally(VirtualTableReadQuery.java:61)
> at 
> org.apache.cassandra.db.AbstractReadQuery.executeInternal(AbstractReadQuery.java:64)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeInternal(VirtualTableReadQuery.java:32)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.execute(VirtualTableReadQuery.java:54)
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:75)
> at 
> org.apache.cassandra.service.pager.PartitionRangeQueryPager.fetchPage(PartitionRangeQueryPager.java:29)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:253)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
> at 
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16178) ByteBufferAccessor throws ClassCastException when trying to query system_views.local_read_latency

2020-10-02 Thread Caleb Rackliffe (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206519#comment-17206519
 ] 

Caleb Rackliffe commented on CASSANDRA-16178:
-

Possible reviewers look like [~bdeggleston], [~ifesdjeen], or [~dcapwell]...

> ByteBufferAccessor throws ClassCastException when trying to query 
> system_views.local_read_latency
> -
>
> Key: CASSANDRA-16178
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16178
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Tables
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>
> If you start up a fresh trunk/4.0 node and execute the query “SELECT * FROM 
> system_views.local_read_latency”, you’ll get the following error:
> ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099 
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.ClassCastException: 
> org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to 
> java.lang.String
> at 
> org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
> at 
> org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
> at 
> org.apache.cassandra.db.marshal.CompositeType.decompose(CompositeType.java:192)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.makeDecoratedKey(SimpleDataSet.java:87)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.row(SimpleDataSet.java:63)
> at 
> org.apache.cassandra.db.virtual.TableMetricTables$TableMetricTable.data(TableMetricTables.java:196)
> at 
> org.apache.cassandra.db.virtual.AbstractVirtualTable.select(AbstractVirtualTable.java:91)
> at 
> org.apache.cassandra.db.VirtualTablePartitionRangeReadQuery.queryVirtualTable(VirtualTablePartitionRangeReadQuery.java:93)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeLocally(VirtualTableReadQuery.java:61)
> at 
> org.apache.cassandra.db.AbstractReadQuery.executeInternal(AbstractReadQuery.java:64)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeInternal(VirtualTableReadQuery.java:32)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.execute(VirtualTableReadQuery.java:54)
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:75)
> at 
> org.apache.cassandra.service.pager.PartitionRangeQueryPager.fetchPage(PartitionRangeQueryPager.java:29)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:253)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
> at 
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16109) Don't adjust nodeCount when setting node id topology in in-jvm dtests

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206511#comment-17206511
 ] 

David Capwell edited comment on CASSANDRA-16109 at 10/2/20, 9:03 PM:
-

starting commit (upgrade tests use these branches since mixed version was 
failing); includes CASSANDRA-16109 & CASSANDRA-16101

CI results (pending)
2.2
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-cassandra-2.2-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/58/

3.0
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-cassandra-3.0-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/59/

3.11
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-cassandra-3.11-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/60/

trunk
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-trunk-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/61/


was (Author: dcapwell):
starting commit (upgrade tests use these branches since mixed version was 
failing).

CI results (pending)
2.2
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-cassandra-2.2-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/58/

3.0
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-cassandra-3.0-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/59/

3.11
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-cassandra-3.11-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/60/

trunk
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-trunk-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/61/

> Don't adjust nodeCount when setting node id topology in in-jvm dtests
> -
>
> Key: CASSANDRA-16109
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16109
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Low
>  Labels: pull-request-available
>
> We update the node count when setting the node id topology in in-jvm dtests, 
> this should only happen if node count is smaller than the node id topology, 
> otherwise bootstrap tests error out.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16178) ByteBufferAccessor throws ClassCastException when trying to query system_views.local_read_latency

2020-10-02 Thread Caleb Rackliffe (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-16178:

Test and Documentation Plan: a new unit test that isolates and verifies the 
fix
 Status: Patch Available  (was: In Progress)

Looks like this was just a minor mistake w/ passing an accessor to a method 
that only needed the raw values. It obviously only affects trunk.

[branch|https://github.com/apache/cassandra/pull/766]
[j8 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/122/workflows/1cd714e6-859d-46a0-95bd-68930fc7f78f]
[j11 
tests|https://app.circleci.com/pipelines/github/maedhroz/cassandra/122/workflows/8664a5a1-f0f4-46d0-9713-b0188220ddcf]

> ByteBufferAccessor throws ClassCastException when trying to query 
> system_views.local_read_latency
> -
>
> Key: CASSANDRA-16178
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16178
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Tables
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>
> If you start up a fresh trunk/4.0 node and execute the query “SELECT * FROM 
> system_views.local_read_latency”, you’ll get the following error:
> ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099 
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.ClassCastException: 
> org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to 
> java.lang.String
> at 
> org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
> at 
> org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
> at 
> org.apache.cassandra.db.marshal.CompositeType.decompose(CompositeType.java:192)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.makeDecoratedKey(SimpleDataSet.java:87)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.row(SimpleDataSet.java:63)
> at 
> org.apache.cassandra.db.virtual.TableMetricTables$TableMetricTable.data(TableMetricTables.java:196)
> at 
> org.apache.cassandra.db.virtual.AbstractVirtualTable.select(AbstractVirtualTable.java:91)
> at 
> org.apache.cassandra.db.VirtualTablePartitionRangeReadQuery.queryVirtualTable(VirtualTablePartitionRangeReadQuery.java:93)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeLocally(VirtualTableReadQuery.java:61)
> at 
> org.apache.cassandra.db.AbstractReadQuery.executeInternal(AbstractReadQuery.java:64)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeInternal(VirtualTableReadQuery.java:32)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.execute(VirtualTableReadQuery.java:54)
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:75)
> at 
> org.apache.cassandra.service.pager.PartitionRangeQueryPager.fetchPage(PartitionRangeQueryPager.java:29)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:253)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
> at 
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To 

[jira] [Updated] (CASSANDRA-16178) ByteBufferAccessor throws ClassCastException when trying to query system_views.local_read_latency

2020-10-02 Thread Caleb Rackliffe (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-16178:

Fix Version/s: 4.0-beta

> ByteBufferAccessor throws ClassCastException when trying to query 
> system_views.local_read_latency
> -
>
> Key: CASSANDRA-16178
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16178
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Tables
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0-beta
>
>
> If you start up a fresh trunk/4.0 node and execute the query “SELECT * FROM 
> system_views.local_read_latency”, you’ll get the following error:
> ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099 
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.ClassCastException: 
> org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to 
> java.lang.String
> at 
> org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
> at 
> org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
> at 
> org.apache.cassandra.db.marshal.CompositeType.decompose(CompositeType.java:192)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.makeDecoratedKey(SimpleDataSet.java:87)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.row(SimpleDataSet.java:63)
> at 
> org.apache.cassandra.db.virtual.TableMetricTables$TableMetricTable.data(TableMetricTables.java:196)
> at 
> org.apache.cassandra.db.virtual.AbstractVirtualTable.select(AbstractVirtualTable.java:91)
> at 
> org.apache.cassandra.db.VirtualTablePartitionRangeReadQuery.queryVirtualTable(VirtualTablePartitionRangeReadQuery.java:93)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeLocally(VirtualTableReadQuery.java:61)
> at 
> org.apache.cassandra.db.AbstractReadQuery.executeInternal(AbstractReadQuery.java:64)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeInternal(VirtualTableReadQuery.java:32)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.execute(VirtualTableReadQuery.java:54)
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:75)
> at 
> org.apache.cassandra.service.pager.PartitionRangeQueryPager.fetchPage(PartitionRangeQueryPager.java:29)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:253)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
> at 
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16178) ByteBufferAccessor throws ClassCastException when trying to query system_views.local_read_latency

2020-10-02 Thread Caleb Rackliffe (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-16178:

 Bug Category: Parent values: Correctness(12982)Level 1 values: API / 
Semantic Implementation(12988)
   Complexity: Normal
Discovered By: User Report
 Severity: Normal
   Status: Open  (was: Triage Needed)

> ByteBufferAccessor throws ClassCastException when trying to query 
> system_views.local_read_latency
> -
>
> Key: CASSANDRA-16178
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16178
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Tables
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
>
> If you start up a fresh trunk/4.0 node and execute the query “SELECT * FROM 
> system_views.local_read_latency”, you’ll get the following error:
> ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099 
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.ClassCastException: 
> org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to 
> java.lang.String
> at 
> org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
> at 
> org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
> at 
> org.apache.cassandra.db.marshal.CompositeType.decompose(CompositeType.java:192)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.makeDecoratedKey(SimpleDataSet.java:87)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.row(SimpleDataSet.java:63)
> at 
> org.apache.cassandra.db.virtual.TableMetricTables$TableMetricTable.data(TableMetricTables.java:196)
> at 
> org.apache.cassandra.db.virtual.AbstractVirtualTable.select(AbstractVirtualTable.java:91)
> at 
> org.apache.cassandra.db.VirtualTablePartitionRangeReadQuery.queryVirtualTable(VirtualTablePartitionRangeReadQuery.java:93)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeLocally(VirtualTableReadQuery.java:61)
> at 
> org.apache.cassandra.db.AbstractReadQuery.executeInternal(AbstractReadQuery.java:64)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeInternal(VirtualTableReadQuery.java:32)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.execute(VirtualTableReadQuery.java:54)
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:75)
> at 
> org.apache.cassandra.service.pager.PartitionRangeQueryPager.fetchPage(PartitionRangeQueryPager.java:29)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:253)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
> at 
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-16178) ByteBufferAccessor throws ClassCastException when trying to query system_views.local_read_latency

2020-10-02 Thread Caleb Rackliffe (Jira)
Caleb Rackliffe created CASSANDRA-16178:
---

 Summary: ByteBufferAccessor throws ClassCastException when trying 
to query system_views.local_read_latency
 Key: CASSANDRA-16178
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16178
 Project: Cassandra
  Issue Type: Bug
  Components: Feature/Virtual Tables
Reporter: Caleb Rackliffe


If you start up a fresh trunk/4.0 node and execute the query “SELECT * FROM 
system_views.local_read_latency”, you’ll get the following error:

ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099 
ErrorMessage.java:457 - Unexpected exception during request
java.lang.ClassCastException: 
org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to 
java.lang.String
at 
org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
at 
org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
at 
org.apache.cassandra.db.marshal.CompositeType.decompose(CompositeType.java:192)
at 
org.apache.cassandra.db.virtual.SimpleDataSet.makeDecoratedKey(SimpleDataSet.java:87)
at 
org.apache.cassandra.db.virtual.SimpleDataSet.row(SimpleDataSet.java:63)
at 
org.apache.cassandra.db.virtual.TableMetricTables$TableMetricTable.data(TableMetricTables.java:196)
at 
org.apache.cassandra.db.virtual.AbstractVirtualTable.select(AbstractVirtualTable.java:91)
at 
org.apache.cassandra.db.VirtualTablePartitionRangeReadQuery.queryVirtualTable(VirtualTablePartitionRangeReadQuery.java:93)
at 
org.apache.cassandra.db.VirtualTableReadQuery.executeLocally(VirtualTableReadQuery.java:61)
at 
org.apache.cassandra.db.AbstractReadQuery.executeInternal(AbstractReadQuery.java:64)
at 
org.apache.cassandra.db.VirtualTableReadQuery.executeInternal(VirtualTableReadQuery.java:32)
at 
org.apache.cassandra.db.VirtualTableReadQuery.execute(VirtualTableReadQuery.java:54)
at 
org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:75)
at 
org.apache.cassandra.service.pager.PartitionRangeQueryPager.fetchPage(PartitionRangeQueryPager.java:29)
at 
org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
at 
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:253)
at 
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
at 
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
at 
org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
at 
org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
at 
org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at 
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
at 
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Assigned] (CASSANDRA-16178) ByteBufferAccessor throws ClassCastException when trying to query system_views.local_read_latency

2020-10-02 Thread Caleb Rackliffe (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe reassigned CASSANDRA-16178:
---

Assignee: Caleb Rackliffe

> ByteBufferAccessor throws ClassCastException when trying to query 
> system_views.local_read_latency
> -
>
> Key: CASSANDRA-16178
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16178
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Tables
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
>
> If you start up a fresh trunk/4.0 node and execute the query “SELECT * FROM 
> system_views.local_read_latency”, you’ll get the following error:
> ERROR [Native-Transport-Requests-1] 2020-09-30 09:44:45,099 
> ErrorMessage.java:457 - Unexpected exception during request
> java.lang.ClassCastException: 
> org.apache.cassandra.db.marshal.ByteBufferAccessor cannot be cast to 
> java.lang.String
> at 
> org.apache.cassandra.serializers.AbstractTextSerializer.serialize(AbstractTextSerializer.java:29)
> at 
> org.apache.cassandra.db.marshal.AbstractType.decompose(AbstractType.java:131)
> at 
> org.apache.cassandra.db.marshal.CompositeType.decompose(CompositeType.java:192)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.makeDecoratedKey(SimpleDataSet.java:87)
> at 
> org.apache.cassandra.db.virtual.SimpleDataSet.row(SimpleDataSet.java:63)
> at 
> org.apache.cassandra.db.virtual.TableMetricTables$TableMetricTable.data(TableMetricTables.java:196)
> at 
> org.apache.cassandra.db.virtual.AbstractVirtualTable.select(AbstractVirtualTable.java:91)
> at 
> org.apache.cassandra.db.VirtualTablePartitionRangeReadQuery.queryVirtualTable(VirtualTablePartitionRangeReadQuery.java:93)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeLocally(VirtualTableReadQuery.java:61)
> at 
> org.apache.cassandra.db.AbstractReadQuery.executeInternal(AbstractReadQuery.java:64)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.executeInternal(VirtualTableReadQuery.java:32)
> at 
> org.apache.cassandra.db.VirtualTableReadQuery.execute(VirtualTableReadQuery.java:54)
> at 
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(AbstractQueryPager.java:75)
> at 
> org.apache.cassandra.service.pager.PartitionRangeQueryPager.fetchPage(PartitionRangeQueryPager.java:29)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement$Pager$NormalPager.fetchPage(SelectStatement.java:352)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:400)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:250)
> at 
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:88)
> at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:216)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:253)
> at 
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:240)
> at 
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:108)
> at 
> org.apache.cassandra.transport.Message$Request.execute(Message.java:253)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.processRequest(Message.java:725)
> at 
> org.apache.cassandra.transport.Message$Dispatcher.lambda$channelRead0$0(Message.java:630)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16109) Don't adjust nodeCount when setting node id topology in in-jvm dtests

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206511#comment-17206511
 ] 

David Capwell commented on CASSANDRA-16109:
---

starting commit (upgrade tests use these branches since mixed version was 
failing).

CI results (pending)
2.2
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-cassandra-2.2-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/58/

3.0
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-cassandra-3.0-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/59/

3.11
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-cassandra-3.11-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/60/

trunk
Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16109-trunk-6EE6E6D9-1C91-4F6B-BE9A-745867F247DD
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/61/

> Don't adjust nodeCount when setting node id topology in in-jvm dtests
> -
>
> Key: CASSANDRA-16109
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16109
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Low
>  Labels: pull-request-available
>
> We update the node count when setting the node id topology in in-jvm dtests, 
> this should only happen if node count is smaller than the node id topology, 
> otherwise bootstrap tests error out.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15703) When CDC is disabled bootstrapping breaks

2020-10-02 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15703:

 Bug Category: Parent values: Correctness(12982)
   Complexity: Normal
Discovered By: User Report
 Severity: Normal
   Status: Open  (was: Triage Needed)

> When CDC is disabled bootstrapping breaks
> -
>
> Key: CASSANDRA-15703
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15703
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Bootstrap and Decommission
>Reporter: T Jake Luciani
>Priority: Normal
> Fix For: 3.11.x
>
>
> Related to CASSANDRA-12697
> There is an edge case left over.  If a cluster had enabled CDC on a table 
> then subsequently set cdc=false, subsequent bootstraps break. 
>  
> This is because the cdc column is false on the existing nodes but null on the 
> bootstrapping node, causing the schema sha to never match.
>  
> There are a couple possible fixes:
>   1.  Since 12697 was only about upgrades we can serialize the cdc column IFF 
> the cluster nodes are all on the same version.
>   2.  We can force cdc=false on all tables where it's null.
>  
> I think #1 is probably simpler. #2 would probably cause more of the same 
> problem if nodes are not all updated with the fix.
>  
>   



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16177) jvm_upgrade_dtests job issue in CircleCI MIDRES

2020-10-02 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16177:

Fix Version/s: 4.0-beta

> jvm_upgrade_dtests job issue in CircleCI MIDRES
> ---
>
> Key: CASSANDRA-16177
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16177
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
>
> jvm_upgrade_dtests work well in HIGHRES, but we see the following issue with 
> MIDRES:
> https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/349/workflows/04bccc52-4e3e-41e2-9c04-93501ea4ce77/jobs/2167/steps



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-16177) jvm_upgrade_dtests job issue in CircleCI MIDRES

2020-10-02 Thread Ekaterina Dimitrova (Jira)
Ekaterina Dimitrova created CASSANDRA-16177:
---

 Summary: jvm_upgrade_dtests job issue in CircleCI MIDRES
 Key: CASSANDRA-16177
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16177
 Project: Cassandra
  Issue Type: Bug
Reporter: Ekaterina Dimitrova


jvm_upgrade_dtests work well in HIGHRES, but we see the following issue with 
MIDRES:

https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/349/workflows/04bccc52-4e3e-41e2-9c04-93501ea4ce77/jobs/2167/steps



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16177) jvm_upgrade_dtests job issue in CircleCI MIDRES

2020-10-02 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16177:

 Bug Category: Parent values: Correctness(12982)
   Complexity: Normal
  Component/s: CI
Discovered By: User Report
 Severity: Low
   Status: Open  (was: Triage Needed)

> jvm_upgrade_dtests job issue in CircleCI MIDRES
> ---
>
> Key: CASSANDRA-16177
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16177
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Ekaterina Dimitrova
>Priority: Normal
>
> jvm_upgrade_dtests work well in HIGHRES, but we see the following issue with 
> MIDRES:
> https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/349/workflows/04bccc52-4e3e-41e2-9c04-93501ea4ce77/jobs/2167/steps



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-15997) TestBootstrap::test_cleanup failing on unexpected number of SSTables

2020-10-02 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206498#comment-17206498
 ] 

Brandon Williams edited comment on CASSANDRA-15997 at 10/2/20, 8:33 PM:


I'm unable to repro this after thousands of runs, and jenkins has never seen it 
fail, so there's something special about circle here.  Unfortunately, for 
whatever reason our circle configs force all the logging to INFO, so we never 
see _why_ this failed.  Since I can't repro, I propose pushing the debug 
logging to error when the error occurs so regardless of the log level we can 
see what happened: 
https://github.com/apache/cassandra-dtest/compare/master...driftx:CASSANDRA-15997
 and the next time it fails we'll have something to work with.


was (Author: brandon.williams):
I'm unable to repro this after thousands of runs, and jenkins has never seen it 
fail, so there's something special about circle here.  Unfortunately, for 
whatever reason our circle configs force all the logging to INFO, so we never 
see _why_ this failed.  Since I can repro, I propose pushing the debug logging 
to error when the error occurs so regardless of the log level we can see what 
happened: 
https://github.com/apache/cassandra-dtest/compare/master...driftx:CASSANDRA-15997
 and the next time it fails we'll have something to work with.

> TestBootstrap::test_cleanup failing on unexpected number of SSTables
> 
>
> Key: CASSANDRA-15997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15997
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Caleb Rackliffe
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> This failure has now occurred in a number of places on trunk (4.0), including 
> both Java 8 and 11 dtest runs. Nominally, there appear to be more SSTables 
> after cleanup than the test is expecting.
> {noformat}
> if len(sstables) > basecount + jobs:
> logger.debug("Current count is {}, basecount was 
> {}".format(len(sstables), basecount))
> failed.set()
> {noformat}
> Examples:
> https://app.circleci.com/pipelines/github/maedhroz/cassandra/92/workflows/c59be4f8-329e-4d76-9c59-d49c38e58dd2/jobs/448
> https://app.circleci.com/pipelines/github/jolynch/cassandra/20/workflows/9d6c3b86-6207-4ead-aa4b-79022fc84182/jobs/893



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206502#comment-17206502
 ] 

David Capwell commented on CASSANDRA-16120:
---

stopped commit as upgrade is broken; switching to 
https://issues.apache.org/jira/browse/CASSANDRA-16109 which tries to update all 
4 branches.

> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta, 4.0-triage
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest 
> supports the ability to grep the logs of an instance; we need this capability 
> as some tests require validating logs were triggered.
> Pydocs for common log methods 
> {code}
> |  grep_log(self, expr, filename='system.log', from_mark=None)
> |  Returns a list of lines matching the regular expression in parameter
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors(self, filename='system.log')
> |  Returns a list of errors with stack traces
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> |  watch_log_for(self, exprs, from_mark=None, timeout=600, process=None, 
> verbose=False, filename='system.log')
> |  Watch the log until one or more (regular) expression are found.
> |  This methods when all the expressions have been found or the method
> |  timeouts (a TimeoutError is then raised). On successful completion,
> |  a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag", 
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename", 
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag, 
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l -> 
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws 
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws 
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new 
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it, 
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag", 
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename", 
> 

[jira] [Created] (CASSANDRA-16176) Python dtests should run at debug

2020-10-02 Thread Brandon Williams (Jira)
Brandon Williams created CASSANDRA-16176:


 Summary: Python dtests should run at debug
 Key: CASSANDRA-16176
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16176
 Project: Cassandra
  Issue Type: Improvement
Reporter: Brandon Williams


At least in our circle configs, we are specifying --log-level="INFO" which 
often leaves us with nothing to go on when we have a flaky test that only fails 
in a certain environment.  In general it would be more useful to always run at 
DEBUG. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15997) TestBootstrap::test_cleanup failing on unexpected number of SSTables

2020-10-02 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206501#comment-17206501
 ] 

Brandon Williams commented on CASSANDRA-15997:
--

CASSANDRA-16176 to follow up on the log level more generally.

> TestBootstrap::test_cleanup failing on unexpected number of SSTables
> 
>
> Key: CASSANDRA-15997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15997
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Caleb Rackliffe
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> This failure has now occurred in a number of places on trunk (4.0), including 
> both Java 8 and 11 dtest runs. Nominally, there appear to be more SSTables 
> after cleanup than the test is expecting.
> {noformat}
> if len(sstables) > basecount + jobs:
> logger.debug("Current count is {}, basecount was 
> {}".format(len(sstables), basecount))
> failed.set()
> {noformat}
> Examples:
> https://app.circleci.com/pipelines/github/maedhroz/cassandra/92/workflows/c59be4f8-329e-4d76-9c59-d49c38e58dd2/jobs/448
> https://app.circleci.com/pipelines/github/jolynch/cassandra/20/workflows/9d6c3b86-6207-4ead-aa4b-79022fc84182/jobs/893



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15997) TestBootstrap::test_cleanup failing on unexpected number of SSTables

2020-10-02 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206498#comment-17206498
 ] 

Brandon Williams commented on CASSANDRA-15997:
--

I'm unable to repro this after thousands of runs, and jenkins has never seen it 
fail, so there's something special about circle here.  Unfortunately, for 
whatever reason our circle configs force all the logging to INFO, so we never 
see _why_ this failed.  Since I can repro, I propose pushing the debug logging 
to error when the error occurs so regardless of the log level we can see what 
happened: 
https://github.com/apache/cassandra-dtest/compare/master...driftx:CASSANDRA-15997
 and the next time it fails we'll have something to work with.

> TestBootstrap::test_cleanup failing on unexpected number of SSTables
> 
>
> Key: CASSANDRA-15997
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15997
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Caleb Rackliffe
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> This failure has now occurred in a number of places on trunk (4.0), including 
> both Java 8 and 11 dtest runs. Nominally, there appear to be more SSTables 
> after cleanup than the test is expecting.
> {noformat}
> if len(sstables) > basecount + jobs:
> logger.debug("Current count is {}, basecount was 
> {}".format(len(sstables), basecount))
> failed.set()
> {noformat}
> Examples:
> https://app.circleci.com/pipelines/github/maedhroz/cassandra/92/workflows/c59be4f8-329e-4d76-9c59-d49c38e58dd2/jobs/448
> https://app.circleci.com/pipelines/github/jolynch/cassandra/20/workflows/9d6c3b86-6207-4ead-aa4b-79022fc84182/jobs/893



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15586) 4.0 quality testing: Cluster Setup and Maintenance

2020-10-02 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206489#comment-17206489
 ] 

Josh McKenzie commented on CASSANDRA-15586:
---

+1 here.

Sad my amazing house of cards in stop-server.bat and stop-server.ps1 is going 
away, but it's time.

> 4.0 quality testing: Cluster Setup and Maintenance
> --
>
> Key: CASSANDRA-15586
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15586
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>  Labels: 4.0-QA
> Fix For: 4.0-beta, 4.0-triage
>
>
> We want 4.0 to be easy for users to setup out of the box and just work. This 
> means having low friction when users download the Cassandra package and start 
> running it. For example, users should be able to easily configure and start 
> new 4.0 clusters and have tokens distributed evenly. Another example is 
> packaging, it should be easy to install Cassandra on all supported platforms 
> (e.g. packaging) and have Cassandra use standard platform integrations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15582) 4.0 quality testing: metrics

2020-10-02 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206488#comment-17206488
 ] 

Josh McKenzie commented on CASSANDRA-15582:
---

Sounds good. Also sounds like it's be pretty solid LHF for new contributors to 
get involved in if we enumerate it clearly.

> 4.0 quality testing: metrics
> 
>
> Key: CASSANDRA-15582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15582
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
> Attachments: Screen Shot 2020-04-07 at 5.47.17 PM.png
>
>
> In past releases we've unknowingly broken metrics integrations and introduced 
> performance regressions in metrics collection and reporting. We strive in 4.0 
> to not do that. Metrics should work well!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15538) 4.0 quality testing: Local Read/Write Path: Other Areas

2020-10-02 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206487#comment-17206487
 ] 

Josh McKenzie commented on CASSANDRA-15538:
---

What's our appetite for taking Harry from "it is possible to do this with 
Harry" to "we are in fact doing this with Harry" in the 4.0 time frame? Any 
points of view on blocking the release based on that work being done?

Are there areas of the codebase we feel still have significant risk that we 
should further exercise with things like Harry and Fallout before release (i.e. 
local read/write paths, etc), or are we confident that the combination of no 
LegacyLayout for 4.0 + diff testing will give users a solid 4.0 experience and 
we can wire up the other stuff after?

> 4.0 quality testing: Local Read/Write Path: Other Areas
> ---
>
> Key: CASSANDRA-15538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15538
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> *Shepherd: Aleksey Yeschenko*
> Testing in this area refers to the local read/write path (StorageProxy, 
> ColumnFamilyStore, Memtable, SSTable reading/writing, etc). We are still 
> finding numerous bugs and issues with the 3.0 storage engine rewrite 
> (CASSANDRA-8099). For 4.0 we want to ensure that we thoroughly cover the 
> local read/write path with techniques such as property-based testing, fuzzing 
> ([example|http://cassandra.apache.org/blog/2018/10/17/finding_bugs_with_property_based_testing.html]),
>  and a source audit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15537) 4.0 quality testing: Local Read/Write Path: Upgrade and Diff Test

2020-10-02 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206486#comment-17206486
 ] 

Josh McKenzie commented on CASSANDRA-15537:
---

{quote}The diff tool is open sourced. It would be great if others can run diff 
on their clusters. People should have restore functionality in their automation
{quote}
Yep! Big help; should have something for the community soon there.

As for the "keep this open until GA", makes complete sense. Also true for 
another ticket so we should codify that somehow and exclude from our collective 
kanban / view of scope. Thanks for the clarification.

> 4.0 quality testing: Local Read/Write Path: Upgrade and Diff Test
> -
>
> Key: CASSANDRA-15537
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15537
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> Execution of upgrade and diff tests via cassandra-diff have proven to be one 
> of the most effective approaches toward identifying issues with the local 
> read/write path. These include instances of data loss, data corruption, data 
> resurrection, incorrect responses to queries, incomplete responses, and 
> others. Upgrade and diff tests can be executed concurrent with fault 
> injection (such as host or network failure); as well as during mixed-version 
> scenarios (such as upgrading half of the instances in a cluster, and running 
> upgradesstables on only half of the upgraded instances).
> Upgrade and diff tests are expected to continue through the release cycle, 
> and are a great way for contributors to gain confidence in the correctness of 
> the database under their own workloads.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206411#comment-17206411
 ] 

David Capwell edited comment on CASSANDRA-16120 at 10/2/20, 7:46 PM:
-

starting commit

CI results (pending):

Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-trunk-F6C5BD3F-51DF-4CCE-85A5-2A5EFA7F311E
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/57/


was (Author: dcapwell):
starting commit

CI results (pending):

Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-trunk-5D6B2C72-CBAF-45B9-8DBE-B27730C3668B
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/55/

> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta, 4.0-triage
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest 
> supports the ability to grep the logs of an instance; we need this capability 
> as some tests require validating logs were triggered.
> Pydocs for common log methods 
> {code}
> |  grep_log(self, expr, filename='system.log', from_mark=None)
> |  Returns a list of lines matching the regular expression in parameter
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors(self, filename='system.log')
> |  Returns a list of errors with stack traces
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> |  watch_log_for(self, exprs, from_mark=None, timeout=600, process=None, 
> verbose=False, filename='system.log')
> |  Watch the log until one or more (regular) expression are found.
> |  This methods when all the expressions have been found or the method
> |  timeouts (a TimeoutError is then raised). On successful completion,
> |  a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag", 
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename", 
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag, 
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l -> 
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws 
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws 
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new 
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};

[jira] [Commented] (CASSANDRA-16153) Cassandra 4b2 - JVM options from *.options not read/set

2020-10-02 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206456#comment-17206456
 ] 

Brandon Williams commented on CASSANDRA-16153:
--

do you have the debug log as well?



> Cassandra 4b2 - JVM options from *.options not read/set
> ---
>
> Key: CASSANDRA-16153
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16153
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Scripts
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: system.log.2020-10-01.0.zip
>
>
> Trying out Cassandra 4 beta 2 with Java 8 (AdoptOpenJDK) in AWS.
> {noformat}
> NAME="Amazon Linux AMI"
> VERSION="2018.03"
> ID="amzn"
> ID_LIKE="rhel fedora"
> VERSION_ID="2018.03"
> PRETTY_NAME="Amazon Linux AMI 2018.03"
> ANSI_COLOR="0;33"
> CPE_NAME="cpe:/o:amazon:linux:2018.03:ga"
> HOME_URL="http://aws.amazon.com/amazon-linux-ami/;
> {noformat}
> It seems the Cassandra JVM results in using Parallel GC.
> {noformat}
> INFO  [Service Thread] 2020-10-01 00:00:56,233 GCInspector.java:299 - PS 
> Scavenge GC in 541ms.  PS Old Gen: 5152844776 -> 5726724752;
> WARN  [Service Thread] 2020-10-01 00:00:56,234 GCInspector.java:297 - PS 
> MarkSweep GC in 1969ms.  PS Eden Space: 2111307776 -> 0; PS Old Gen: 
> 5726724752 -> 2581334376; PS Survivor Space: 363850224 -> 0
> {noformat}
> Although {{jvm8-server.options}} is using CMS.
> {noformat}
> #
> #  GC SETTINGS  #
> #
> ### CMS Settings
> -XX:+UseParNewGC
> -XX:+UseConcMarkSweepGC
> -XX:+CMSParallelRemarkEnabled
> -XX:SurvivorRatio=8
> -XX:MaxTenuringThreshold=1
> -XX:CMSInitiatingOccupancyFraction=75
> -XX:+UseCMSInitiatingOccupancyOnly
> -XX:CMSWaitDuration=1
> -XX:+CMSParallelInitialMarkEnabled
> -XX:+CMSEdenChunksRecordAlways
> ## some JVMs will fill up their heap when accessed via JMX, see CASSANDRA-6541
> -XX:+CMSClassUnloadingEnabled
> ...
> {noformat}
> In Cassandra 3, default has been CMS.
> So, possibly there is something wrong in reading/processing 
> {{jvm8-server.options}}?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206382#comment-17206382
 ] 

David Capwell edited comment on CASSANDRA-16150 at 10/2/20, 7:09 PM:
-

Trigged build: https://ci-cassandra.apache.org/job/Cassandra-devbranch/56/


was (Author: dcapwell):
Trigged build: https://ci-cassandra.apache.org/job/Cassandra-devbranch/54/

> Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
> ---
>
> Key: CASSANDRA-16150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16150
> Project: Cassandra
>  Issue Type: Bug
>  Components: Dependencies
>Reporter: Rahul Nandi
>Assignee: Rahul Nandi
>Priority: Normal
> Fix For: 4.x
>
>
> There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml 
> version earlier to 1.26. This has been patched into snakeyaml version 1.26.
> Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640]
> This card is expected to upgrade the snakeyaml version to 1.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206411#comment-17206411
 ] 

David Capwell commented on CASSANDRA-16120:
---

starting commit

CI results (pending):

Circle: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16120-trunk-5D6B2C72-CBAF-45B9-8DBE-B27730C3668B
Jenkins: https://ci-cassandra.apache.org/job/Cassandra-devbranch/55/

> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta, 4.0-triage
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest 
> supports the ability to grep the logs of an instance; we need this capability 
> as some tests require validating logs were triggered.
> Pydocs for common log methods 
> {code}
> |  grep_log(self, expr, filename='system.log', from_mark=None)
> |  Returns a list of lines matching the regular expression in parameter
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors(self, filename='system.log')
> |  Returns a list of errors with stack traces
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> |  watch_log_for(self, exprs, from_mark=None, timeout=600, process=None, 
> verbose=False, filename='system.log')
> |  Watch the log until one or more (regular) expression are found.
> |  This methods when all the expressions have been found or the method
> |  timeouts (a TimeoutError is then raised). On successful completion,
> |  a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag", 
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename", 
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag, 
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l -> 
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws 
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws 
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new 
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it, 
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = 

[jira] [Commented] (CASSANDRA-16120) Add ability for jvm-dtest to grep instance logs

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206405#comment-17206405
 ] 

David Capwell commented on CASSANDRA-16120:
---

rebasing and will merge once CI is clean.

> Add ability for jvm-dtest to grep instance logs
> ---
>
> Key: CASSANDRA-16120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16120
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>  Labels: pull-request-available
> Fix For: 4.0-beta, 4.0-triage
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> One of the main gaps between python dtest and jvm dtest is python dtest 
> supports the ability to grep the logs of an instance; we need this capability 
> as some tests require validating logs were triggered.
> Pydocs for common log methods 
> {code}
> |  grep_log(self, expr, filename='system.log', from_mark=None)
> |  Returns a list of lines matching the regular expression in parameter
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors(self, filename='system.log')
> |  Returns a list of errors with stack traces
> |  in the Cassandra log of this node
> |
> |  grep_log_for_errors_from(self, filename='system.log', seek_start=0)
> {code}
> {code}
> |  watch_log_for(self, exprs, from_mark=None, timeout=600, process=None, 
> verbose=False, filename='system.log')
> |  Watch the log until one or more (regular) expression are found.
> |  This methods when all the expressions have been found or the method
> |  timeouts (a TimeoutError is then raised). On successful completion,
> |  a list of pair (line matched, match object) is returned.
> {code}
> Below is a POC showing a way to do such logic
> {code}
> package org.apache.cassandra.distributed.test;
> import java.io.BufferedReader;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.InputStreamReader;
> import java.io.UncheckedIOException;
> import java.nio.charset.StandardCharsets;
> import java.util.Iterator;
> import java.util.Spliterator;
> import java.util.Spliterators;
> import java.util.regex.Matcher;
> import java.util.regex.Pattern;
> import java.util.stream.Stream;
> import java.util.stream.StreamSupport;
> import com.google.common.io.Closeables;
> import org.junit.Test;
> import org.apache.cassandra.distributed.Cluster;
> import org.apache.cassandra.utils.AbstractIterator;
> public class AllTheLogs extends TestBaseImpl
> {
>@Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag", 
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename", 
> "suitename_IS_UNDEFINED");
>String log = String.format("build/test/logs/%s/TEST-%s.log", tag, 
> suite);
>grep(log, "Enqueuing flush of tables").forEach(l -> 
> System.out.println("I found the thing: " + l));
>}
>}
>private static Stream grep(String file, String regex) throws 
> IOException
>{
>return grep(file, Pattern.compile(regex));
>}
>private static Stream grep(String file, Pattern regex) throws 
> IOException
>{
>BufferedReader reader = new BufferedReader(new InputStreamReader(new 
> FileInputStream(file), StandardCharsets.UTF_8));
>Iterator it = new AbstractIterator()
>{
>protected String computeNext()
>{
>try
>{
>String s;
>while ((s = reader.readLine()) != null)
>{
>Matcher m = regex.matcher(s);
>if (m.find())
>return s;
>}
>reader.close();
>return endOfData();
>}
>catch (IOException e)
>{
>Closeables.closeQuietly(reader);
>throw new UncheckedIOException(e);
>}
>}
>};
>return StreamSupport.stream(Spliterators.spliteratorUnknownSize(it, 
> Spliterator.ORDERED), false);
>}
> }
> {code}
> And
> {code}
> @Test
>public void test() throws IOException
>{
>try (final Cluster cluster = init(Cluster.build(1).start()))
>{
>String tag = System.getProperty("cassandra.testtag", 
> "cassandra.testtag_IS_UNDEFINED");
>String suite = System.getProperty("suitename", 
> "suitename_IS_UNDEFINED");
>//TODO missing way to get node id
> //cluster.get(1);
>

[jira] [Assigned] (CASSANDRA-16083) Missing JMX objects and attributes upgrading from 3.0 to 4.0

2020-10-02 Thread Uchenna (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uchenna reassigned CASSANDRA-16083:
---

Assignee: Uchenna

> Missing JMX objects and attributes upgrading from 3.0 to 4.0
> 
>
> Key: CASSANDRA-16083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16083
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: David Capwell
>Assignee: Uchenna
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Using the tools added in CASSANDRA-16082, below are the list of metrics 
> missing in 4.0 but present in 3.0.  The work here is to make sure we had 
> proper deprecation for each metric, and if not to add it back.
> {code}
> $ tools/bin/jmxtool diff -f yaml cassandra-3.0-jmx.yaml trunk-jmx.yaml 
> --ignore-missing-on-left
> Objects not in right:
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=EstimatedPartitionSizeHistogram
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=BloomFilterFalseRatio
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ReplicaFilteringProtectionRequests
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=RowCacheHitOutOfRange
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=CounterMutationStage,name=MaxPoolSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=TombstoneScannedHistogram
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=WaitingOnFreeMemtableSpace
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasCommitTotalLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_aggregates,name=CasProposeLatency
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=ViewReadTime
> org.apache.cassandra.db:type=HintedHandoffManager
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=BloomFilterDiskSpaceUsed
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=RequestResponseStage,name=PendingTasks
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=MemtableSwitchCount
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=range_xfers,name=ReplicaFilteringProtectionRowsCachedPerQuery
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=SnapshotsSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=RecentBloomFilterFalsePositives
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=SpeculativeRetries
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=LiveDiskSpaceUsed
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ViewReadTime
> org.apache.cassandra.metrics:type=ThreadPools,path=internal,scope=MigrationStage,name=CompletedTasks
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=ViewReadTime
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=BloomFilterFalsePositives
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=CompressionMetadataOffHeapMemoryUsed
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=TotalBlockedTasks
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=LiveScannedHistogram
> org.apache.cassandra.db:type=Tables,keyspace=system,table=views_builds_in_progress
> org.apache.cassandra.metrics:type=ThreadPools,path=internal,scope=MiscStage,name=ActiveTasks
> 

[jira] [Updated] (CASSANDRA-16144) TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore

2020-10-02 Thread Dinesh Joshi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dinesh Joshi updated CASSANDRA-16144:
-
Reviewers: Dinesh Joshi, Dinesh Joshi  (was: Dinesh Joshi)
   Dinesh Joshi, Dinesh Joshi  (was: Dinesh Joshi)
   Status: Review In Progress  (was: Patch Available)

> TLS connections to the storage port on a node without server encryption 
> configured causes java.io.IOException accessing missing keystore
> 
>
> Key: CASSANDRA-16144
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16144
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> If a TLS connection is requested against a node with all encryption disabled 
> by configuration,
> configured with
> {code}
> server_encryption_options: {optional:false, internode_encryption: none}
> {code}
> it logs the following error if no keystore exists for the node.
> {code}
> INFO  [Messaging-EventLoop-3-3] 2020-09-15T14:30:02,952 : - 
> 127.0.0.1:7000->127.0.1.1:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection 
> refused: local1-i1/127.0.1.1:7000
> Caused by: java.net.ConnectException: Connection refused
>at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
>at 
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779) ~[?:?]
>at 
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
>  ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
>  ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702) 
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
>  ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) 
> ~[netty-all-4.1.50.Final.jar:4.1.50.Final]
>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) 
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
>  [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>  [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at java.lang.Thread.run(Thread.java:834) [?:?]
> WARN  [Messaging-EventLoop-3-9] 2020-09-15T14:30:06,375 : - Failed to 
> initialize a channel. Closing: [id: 0x0746c157, L:/127.0.0.1:7000 - 
> R:/127.0.0.1:59623]
> java.io.IOException: failed to build trust manager store for secure 
> connections
>at 
> org.apache.cassandra.security.SSLFactory.buildKeyManagerFactory(SSLFactory.java:232)
>  ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at 
> org.apache.cassandra.security.SSLFactory.createNettySslContext(SSLFactory.java:300)
>  ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at 
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:276)
>  ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at 
> org.apache.cassandra.security.SSLFactory.getOrCreateSslContext(SSLFactory.java:257)
>  ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at 
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:107)
>  ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at 
> org.apache.cassandra.net.InboundConnectionInitiator$Initializer.initChannel(InboundConnectionInitiator.java:71)
>  ~[apache-cassandra-4.0-beta1-SNAPSHOT.jar:4.0-beta1-SNAPSHOT]
>at 
> io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129) 
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112) 
> [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938)
>  [netty-all-4.1.50.Final.jar:4.1.50.Final]
>at 
> 

[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206382#comment-17206382
 ] 

David Capwell commented on CASSANDRA-16150:
---

Trigged build: https://ci-cassandra.apache.org/job/Cassandra-devbranch/54/

> Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
> ---
>
> Key: CASSANDRA-16150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16150
> Project: Cassandra
>  Issue Type: Bug
>  Components: Dependencies
>Reporter: Rahul Nandi
>Assignee: Rahul Nandi
>Priority: Normal
> Fix For: 4.x
>
>
> There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml 
> version earlier to 1.26. This has been patched into snakeyaml version 1.26.
> Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640]
> This card is expected to upgrade the snakeyaml version to 1.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16175) Avoid removing batch when it's not created during view replication

2020-10-02 Thread Zhao Yang (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhao Yang updated CASSANDRA-16175:
--
  Workflow: Cassandra Bug Workflow  (was: Cassandra Default Workflow)
Issue Type: Bug  (was: Improvement)

> Avoid removing batch when it's not created during view replication
> --
>
> Key: CASSANDRA-16175
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16175
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Materialized Views
>Reporter: Zhao Yang
>Priority: Normal
>
> When the base replica is also a view replica we don't write a local batchlog, 
> but they are unnecessarily removed when the view write is successful, what 
> creates (and persists) a tombstone into the system.batches table.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-16175) Avoid removing batch when it's not created during view replication

2020-10-02 Thread Zhao Yang (Jira)
Zhao Yang created CASSANDRA-16175:
-

 Summary: Avoid removing batch when it's not created during view 
replication
 Key: CASSANDRA-16175
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16175
 Project: Cassandra
  Issue Type: Improvement
  Components: Feature/Materialized Views
Reporter: Zhao Yang


When the base replica is also a view replica we don't write a local batchlog, 
but they are unnecessarily removed when the view write is successful, what 
creates (and persists) a tombstone into the system.batches table.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16150) Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206376#comment-17206376
 ] 

David Capwell commented on CASSANDRA-16150:
---

I am going to run the branch through CI just to make sure nothing breaks like 
the upgrade to 1.23 did.

[~ifesdjeen] you switched the branch to 1.23 for harry reasons, can you also 
take a look at this to make sure this works well with harry?

> Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix
> ---
>
> Key: CASSANDRA-16150
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16150
> Project: Cassandra
>  Issue Type: Bug
>  Components: Dependencies
>Reporter: Rahul Nandi
>Assignee: Rahul Nandi
>Priority: Normal
> Fix For: 4.x
>
>
> There have been critical level CVE (CVE-2017-18640) discovered in snakeyaml 
> version earlier to 1.26. This has been patched into snakeyaml version 1.26.
> Reference: [https://nvd.nist.gov/vuln/detail/CVE-2017-18640]
> This card is expected to upgrade the snakeyaml version to 1.26.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833

2020-10-02 Thread Jordan West (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206372#comment-17206372
 ] 

Jordan West commented on CASSANDRA-16148:
-

Hehe this is why I wanted to split them up initially but was encouraged to keep 
them together. Do we then just have two source tracking links, etc? Can we get 
the process for in-jvm dtest started ASAP (let me know what I have to do)?

 

> Test failures caused by merging CASSANDRA-15833
> ---
>
> Key: CASSANDRA-16148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16148
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> Three issues were caused by merging CASSANDRA-15833:
> 1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk: 
> https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> 2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
> 3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an 
> issue with how CASSANDRA-15833 changes interacted with in-jvm dtests running 
> without {{Feature.GOSSIP}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16148) Test failures caused by merging CASSANDRA-15833

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206371#comment-17206371
 ] 

David Capwell commented on CASSANDRA-16148:
---

Overall the patch LGTM, one thing I would love is to split it in two (can keep 
the same JIRA): Gossiper change (fix unit and python dtest), jvm dtest when not 
gossiping.  Main reason is updating jvm-dtest is rather slow at the moment so 
would be good to get the build green.

> Test failures caused by merging CASSANDRA-15833
> ---
>
> Key: CASSANDRA-16148
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16148
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> Three issues were caused by merging CASSANDRA-15833:
> 1. `GossiperTest#testHaveAnyVersion3Nodes` was failing on trunk: 
> https://app.circleci.com/pipelines/github/jrwest/cassandra/53/workflows/95f9f401-1ef8-4b8d-9c64-3703d9669d95/jobs/771
> 2. python dtest ReadRepairTest#test_atomic_writes[blocking] was failing
> 3. In-jvm dtests being worked on as part of CASSANDRA-15977 uncovered an 
> issue with how CASSANDRA-15833 changes interacted with in-jvm dtests running 
> without {{Feature.GOSSIP}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16083) Missing JMX objects and attributes upgrading from 3.0 to 4.0

2020-10-02 Thread David Capwell (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206359#comment-17206359
 ] 

David Capwell commented on CASSANDRA-16083:
---

Updated the test 
https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/tools/JMXCompatabilityTest.java#L73
 to flag some of the JMX objects/attributes as expected.

{code}
List excludeObjects = 
Arrays.asList("org.apache.cassandra.metrics:type=ThreadPools.*",

"org.apache.cassandra.internal:.*",

"org.apache.cassandra.metrics:type=DroppedMessage.*",

"org.apache.cassandra.metrics:type=ClientRequest,scope=CASRead,name=ConditionNotMet",

"org.apache.cassandra.metrics:type=Client,name=connectedThriftClients", // 
removed in CASSANDRA-5

"org.apache.cassandra.request:type=ReadRepairStage", // removed in 
CASSANDRA-13910

"org.apache.cassandra.db:type=HintedHandoffManager", // removed in 
CASSANDRA-15939

// dropped tables

"org.apache.cassandra.metrics:type=Table,keyspace=system,scope=(schema_aggregates|schema_columnfamilies|schema_columns|schema_functions|schema_keyspaces|schema_triggers|schema_usertypes),name=.*",

".*keyspace=system,(scope|table|columnfamily)=views_builds_in_progress.*",

".*keyspace=system,(scope|table|columnfamily)=range_xfers.*",

".*keyspace=system,(scope|table|columnfamily)=hints.*",

".*keyspace=system,(scope|table|columnfamily)=batchlog.*");
List excludeAttributes = Arrays.asList("RPCServerRunning", // 
removed in CASSANDRA-5
   
"MaxNativeProtocolVersion");
List excludeOperations = Arrays.asList("startRPCServer", 
"stopRPCServer", // removed in CASSANDRA-5
   // nodetool apis that 
were changed,
   "decommission", // -> 
decommission(boolean)
   "forceRepairAsync", // 
-> repairAsync
   "forceRepairRangeAsync", 
// -> repairAsync
   "beginLocalSampling", // 
-> beginLocalSampling(p1: java.lang.String, p2: int, p3: int): void
   "finishLocalSampling" // 
-> finishLocalSampling(p1: java.lang.String, p2: int): java.util.List
);
{code}

given this list, the main ones remaining (outside of operations, would be good 
to be backwards compatible but I see them as sidecar/tools related where as 
object/attributes are metrics, so I am more focused on metrics side) are:

Objects:
"org.apache.cassandra.metrics:type=ThreadPools.*",
"org.apache.cassandra.internal:.*",
"org.apache.cassandra.metrics:type=DroppedMessage.*",
"org.apache.cassandra.metrics:type=ClientRequest,scope=CASRead,name=ConditionNotMet",

Attributes:
MaxNativeProtocolVersion

> Missing JMX objects and attributes upgrading from 3.0 to 4.0
> 
>
> Key: CASSANDRA-16083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16083
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Using the tools added in CASSANDRA-16082, below are the list of metrics 
> missing in 4.0 but present in 3.0.  The work here is to make sure we had 
> proper deprecation for each metric, and if not to add it back.
> {code}
> $ tools/bin/jmxtool diff -f yaml cassandra-3.0-jmx.yaml trunk-jmx.yaml 
> --ignore-missing-on-left
> Objects not in right:
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=EstimatedPartitionSizeHistogram
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=BloomFilterFalseRatio
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ReplicaFilteringProtectionRequests
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=RowCacheHitOutOfRange
> 

[jira] [Commented] (CASSANDRA-15537) 4.0 quality testing: Local Read/Write Path: Upgrade and Diff Test

2020-10-02 Thread Yifan Cai (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206325#comment-17206325
 ] 

Yifan Cai commented on CASSANDRA-15537:
---

Hi Josh. The diff test is the type of test that discovers new things with 
different workloads / configurations. We gain confidence in 4.0 with more and 
more tests passed. But there is no fine line to say the test is done. 
Therefore, I'd prefer to have this ticket "open until others are closed" and 
tested with hundreds of clusters. 
The diff tool is open sourced. It would be great if others can run diff on 
their clusters. People should have restore functionality in their automation.

> 4.0 quality testing: Local Read/Write Path: Upgrade and Diff Test
> -
>
> Key: CASSANDRA-15537
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15537
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> Execution of upgrade and diff tests via cassandra-diff have proven to be one 
> of the most effective approaches toward identifying issues with the local 
> read/write path. These include instances of data loss, data corruption, data 
> resurrection, incorrect responses to queries, incomplete responses, and 
> others. Upgrade and diff tests can be executed concurrent with fault 
> injection (such as host or network failure); as well as during mixed-version 
> scenarios (such as upgrading half of the instances in a cluster, and running 
> upgradesstables on only half of the upgraded instances).
> Upgrade and diff tests are expected to continue through the release cycle, 
> and are a great way for contributors to gain confidence in the correctness of 
> the database under their own workloads.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch trunk updated (f866753 -> 08315ea)

2020-10-02 Thread blerer
This is an automated email from the ASF dual-hosted git repository.

blerer pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from f866753  Read repair test with moving token should read at QUORUM
 add 3f73c16  Fix memory leak in CompressedChunkReader
 add 08315ea  Merge branch cassandra-3.11 into trunk

No new revisions were added by this update.

Summary of changes:
 CHANGES.txt|  1 +
 .../commitlog/AbstractCommitLogSegmentManager.java | 18 -
 .../cassandra/db/commitlog/CompressedSegment.java  |  3 +-
 .../cassandra/db/commitlog/EncryptedSegment.java   |  4 +-
 .../cassandra/io/util/CompressedChunkReader.java   | 26 ++-
 .../util}/SimpleCachedBufferPool.java  | 79 ++--
 .../io/util/ThreadLocalByteBufferHolder.java   | 83 ++
 7 files changed, 147 insertions(+), 67 deletions(-)
 rename src/java/org/apache/cassandra/{db/commitlog => 
io/util}/SimpleCachedBufferPool.java (57%)
 create mode 100644 
src/java/org/apache/cassandra/io/util/ThreadLocalByteBufferHolder.java


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch cassandra-3.11 updated (f5d5e72 -> 3f73c16)

2020-10-02 Thread blerer
This is an automated email from the ASF dual-hosted git repository.

blerer pushed a change to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from f5d5e72  Merge branch 'cassandra-3.0' into cassandra-3.11
 add 3f73c16  Fix memory leak in CompressedChunkReader

No new revisions were added by this update.

Summary of changes:
 CHANGES.txt|  1 +
 .../commitlog/AbstractCommitLogSegmentManager.java | 17 -
 .../cassandra/db/commitlog/CompressedSegment.java  |  3 +-
 .../cassandra/db/commitlog/EncryptedSegment.java   |  4 +-
 .../cassandra/io/util/CompressedChunkReader.java   | 27 +--
 .../util}/SimpleCachedBufferPool.java  | 79 ++--
 .../io/util/ThreadLocalByteBufferHolder.java   | 83 ++
 7 files changed, 145 insertions(+), 69 deletions(-)
 rename src/java/org/apache/cassandra/{db/commitlog => 
io/util}/SimpleCachedBufferPool.java (57%)
 create mode 100644 
src/java/org/apache/cassandra/io/util/ThreadLocalByteBufferHolder.java


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15880) Memory leak in CompressedChunkReader

2020-10-02 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15880:
---
  Fix Version/s: (was: 4.0-triage)
 (was: 3.11.x)
 (was: 4.0)
 4.0-beta3
 3.11.9
Source Control Link: 
https://github.com/apache/cassandra/commit/3f73c16d50a80c574dd004c59bfaa6042dcd5781
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

The patch was committed into cassandra-3.11 at 
3f73c16d50a80c574dd004c59bfaa6042dcd5781  and merged into trunk

> Memory leak in CompressedChunkReader
> 
>
> Key: CASSANDRA-15880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15880
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Compression
>Reporter: Jaroslaw Grabowski
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 3.11.9, 4.0-beta3
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> CompressedChunkReader uses java.lang.ThreadLocal to reuse ByteBuffer for 
> compressed data. ByteBuffers leak due to peculiar ThreadLocal quality.
> ThreadLocals are stored in a map, where the key is a weak reference to a 
> ThreadLocal and the value is the user's object (ByteBuffer in this case). 
> When a last strong reference to a ThreadLocal is lost, weak reference to 
> ThreadLocal (key) is removed but the value (ByteBuffer) is kept until cleaned 
> by ThreadLocal heuristic expunge mechanism. See ThreadLocal's "stale entries" 
> for details.
> When a number of long-living threads is high enough this results in thousands 
> of ByteBuffers stored as stale entries in ThreadLocals. In a not-so-lucky 
> scenario we get OutOfMemoryException.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16171) Remove Windows scripts

2020-10-02 Thread Yuki Morishita (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-16171:
---
Test and Documentation Plan: Alternative setup for Windows users should be 
documented. See CASSANDRA-16173.
 Status: Patch Available  (was: Open)

[https://github.com/apache/cassandra/pull/765]

 

PR removes all the *.bat and *.ps1 scripts, as well as reference to the files 
from cqlshlib test and the doc.

> Remove Windows scripts
> --
>
> Key: CASSANDRA-16171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16171
> Project: Cassandra
>  Issue Type: Task
>  Components: Packaging
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Normal
> Fix For: 4.0-rc
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As per the email thread in cassandra-dev mailing list[1], remove windows 
> scripts from Cassandra 4.0 onwards, due to the lack of maintenance and tests.
> 1: https://www.mail-archive.com/dev@cassandra.apache.org/msg15583.html 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16171) Remove Windows scripts

2020-10-02 Thread Yuki Morishita (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-16171:
---
Change Category: Quality Assurance
 Complexity: Normal
Component/s: Packaging
 Status: Open  (was: Triage Needed)

> Remove Windows scripts
> --
>
> Key: CASSANDRA-16171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16171
> Project: Cassandra
>  Issue Type: Task
>  Components: Packaging
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Normal
> Fix For: 4.0-rc
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As per the email thread in cassandra-dev mailing list[1], remove windows 
> scripts from Cassandra 4.0 onwards, due to the lack of maintenance and tests.
> 1: https://www.mail-archive.com/dev@cassandra.apache.org/msg15583.html 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-16174) During bootstrap streaming, skipping write path may create orphan MV rows

2020-10-02 Thread Zhao Yang (Jira)
Zhao Yang created CASSANDRA-16174:
-

 Summary: During bootstrap streaming, skipping write path may 
create orphan MV rows 
 Key: CASSANDRA-16174
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16174
 Project: Cassandra
  Issue Type: Bug
  Components: Feature/Materialized Views
Reporter: Zhao Yang


CASSANDRA-13065 improves the speed of bootstrap streaming by skipping write 
path for base table with MV.

Unfortunately during bootstrapping, the bootstrapping node may receive a base 
write that is already deleted or shadowed on other replicas, but the 
bootstrapping node didn't receive all sstables yet thus bootstapping node may 
create an orphan view row based on received sstables. Without write path, the 
newer version data will not create tombstone to shadow orphan view row.

For example, node-A has a base sstable containing: "k=1, v=1@2019", but 
bootstrapping node didn't receive it yet. A write "k=1, v=0@2018" hitting on 
bootstrapping node will create an orphan view row: "v=0@2018, k=1". Applying 
streaming sstables from bootstrap or rebuild through write path can help, but 
that's something we want to get rid of.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16171) Remove Windows scripts

2020-10-02 Thread Yuki Morishita (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-16171:
---
Platform: Windows  (was: All)

> Remove Windows scripts
> --
>
> Key: CASSANDRA-16171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16171
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Normal
> Fix For: 4.0-rc
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As per the email thread in cassandra-dev mailing list[1], remove windows 
> scripts from Cassandra 4.0 onwards, due to the lack of maintenance and tests.
> 1: https://www.mail-archive.com/dev@cassandra.apache.org/msg15583.html 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Assigned] (CASSANDRA-16171) Remove Windows scripts

2020-10-02 Thread Yuki Morishita (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita reassigned CASSANDRA-16171:
--

Assignee: Yuki Morishita

> Remove Windows scripts
> --
>
> Key: CASSANDRA-16171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16171
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Normal
> Fix For: 4.0-rc
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> As per the email thread in cassandra-dev mailing list[1], remove windows 
> scripts from Cassandra 4.0 onwards, due to the lack of maintenance and tests.
> 1: https://www.mail-archive.com/dev@cassandra.apache.org/msg15583.html 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15902) OOM because repair session thread not closed when terminating repair

2020-10-02 Thread Alexander Dejanovski (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206250#comment-17206250
 ] 

Alexander Dejanovski commented on CASSANDRA-15902:
--

So far, so good.
I've reproduced the issue in 3.11 using a low timeout in Reaper and repair 
sessions started to pile up indefinitely:

{code:java}
% x_all "sudo su -s /bin/bash -c \"jstack \$(ps -ef |grep CassandraDaemon |grep 
-v grep| cut -d' ' -f3) |grep 'Repair#'\" cassandra"
"Repair#11:1" #2193 daemon prio=5 os_prio=0 tid=0x7fe15b19f530 nid=0x74d8 
waiting on condition [0x7fe145968000]
"Repair#10:1" #2154 daemon prio=5 os_prio=0 tid=0x7fe16d7eceb0 nid=0x7471 
waiting on condition [0x7fe12bf12000]
"Repair#8:1" #2116 daemon prio=5 os_prio=0 tid=0x7fe150316b40 nid=0x73f1 
waiting on condition [0x7fe12ce09000]
"Repair#7:1" #2084 daemon prio=5 os_prio=0 tid=0x7fe150162f80 nid=0x73a9 
waiting on condition [0x7fe137894000]
"Repair#3:1" #1704 daemon prio=5 os_prio=0 tid=0x7fe10f1b98d0 nid=0x6b9a 
waiting on condition [0x7fe1428fc000]
"Repair#14:1" #1778 daemon prio=5 os_prio=0 tid=0x565030775bb0 nid=0x6d58 
waiting on condition [0x7f8d08659000]
"Repair#9:1" #1573 daemon prio=5 os_prio=0 tid=0x7f8d28770af0 nid=0x6b88 
waiting on condition [0x7f8d1ff39000]
"Repair#2:1" #1397 daemon prio=5 os_prio=0 tid=0x7f8d2815eb70 nid=0x6851 
waiting on condition [0x7f8d1f9a]
"Repair#1:1" #1375 daemon prio=5 os_prio=0 tid=0x7f8c67dcee40 nid=0x66a8 
waiting on condition [0x7f8d1cc6f000]
"Repair#1:1" #2412 daemon prio=5 os_prio=0 tid=0x7fc61d2a38f0 nid=0x6ed9 
waiting on condition [0x7fc60736d000]
{code}

Then I built the patched version and waited again for repairs to time out for a 
little while.
I never got more than one repair thread:

{code:java}
% x_all "sudo su -s /bin/bash -c \"jstack \$(ps -ef |grep CassandraDaemon |grep 
-v grep| cut -d' ' -f2) |grep 'Repair#'\" cassandra"
"Repair#21:1" #682 daemon prio=5 os_prio=0 tid=0x7f249854cc10 nid=0x7ced 
waiting on condition [0x7f246f779000]
{code}
 
I'm currently checking that repair still go through as expected with a regular 
timeout and that is still running. 
Once that's done, I'll check again against 3.0 and then perform a code review.

> OOM because repair session thread not closed when terminating repair
> 
>
> Key: CASSANDRA-15902
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15902
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair
>Reporter: Swen Fuhrmann
>Assignee: Swen Fuhrmann
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
> Attachments: heap-mem-histo.txt, repair-terminated.txt
>
>
> In our cluster, after a while some nodes running slowly out of memory. On 
> that nodes we observed that Cassandra Reaper terminate repairs with a JMX 
> call to {{StorageServiceMBean.forceTerminateAllRepairSessions()}} because 
> reaching timeout of 30 min.
> In the memory heap dump we see lot of instances of 
> {{io.netty.util.concurrent.FastThreadLocalThread}} occupy most of the memory:
> {noformat}
> 119 instances of "io.netty.util.concurrent.FastThreadLocalThread", loaded by 
> "sun.misc.Launcher$AppClassLoader @ 0x51a80" occupy 8.445.684.480 (93,96 
> %) bytes. {noformat}
> In the thread dump we see lot of repair threads:
> {noformat}
> grep "Repair#" threaddump.txt | wc -l
>   50 {noformat}
>  
> The repair jobs are waiting for the validation to finish:
> {noformat}
> "Repair#152:1" #96170 daemon prio=5 os_prio=0 tid=0x12fc5000 
> nid=0x542a waiting on condition [0x7f81ee414000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x0007939bcfc8> (a 
> com.google.common.util.concurrent.AbstractFuture$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at 
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:285)
> at 
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
> at 
> com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137)
> at 
> com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1509)
> at 

[jira] [Created] (CASSANDRA-16173) Update "Getting Started" document for Windows users

2020-10-02 Thread Yuki Morishita (Jira)
Yuki Morishita created CASSANDRA-16173:
--

 Summary: Update "Getting Started" document for Windows users
 Key: CASSANDRA-16173
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16173
 Project: Cassandra
  Issue Type: Task
Reporter: Yuki Morishita


This is a documentation follow up to CASSANDRA-16171.

Since we are removing support for Windows, we should update ["Getting Started" 
guide|https://cassandra.apache.org/doc/latest/getting_started/index.html] to 
include how-to's for Windows users for setting up Cassandra for dev evnironment.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-16172) materialized view can return stale data even with R+W>N

2020-10-02 Thread Zhao Yang (Jira)
Zhao Yang created CASSANDRA-16172:
-

 Summary: materialized view can return stale data even with R+W>N
 Key: CASSANDRA-16172
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16172
 Project: Cassandra
  Issue Type: Bug
  Components: Feature/Materialized Views
Reporter: Zhao Yang


This is a similar issue as CASSANDRA-8272.

With a key-value table on a 2-node cluster with RF2 and there is a MV that put 
`value` as partition key.

Insert with ConsistencyLevel.ONE twice with different data, assuming there is a 
network partition between 2 nodes:
- Node 1 received: pk="a" -> value=1 @ts1
- Node 2 received: pk="a" -> value=2 @ts2 where ts2 > ts1

When querying `value`=1 on MV with ConsistencyLevel.ALL, we got stale row: "a" 
-> 1.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16171) Remove Windows scripts

2020-10-02 Thread Yuki Morishita (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-16171:
---
Fix Version/s: 4.0-rc

> Remove Windows scripts
> --
>
> Key: CASSANDRA-16171
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16171
> Project: Cassandra
>  Issue Type: Task
>Reporter: Yuki Morishita
>Priority: Normal
> Fix For: 4.0-rc
>
>
> As per the email thread in cassandra-dev mailing list[1], remove windows 
> scripts from Cassandra 4.0 onwards, due to the lack of maintenance and tests.
> 1: https://www.mail-archive.com/dev@cassandra.apache.org/msg15583.html 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-16171) Remove Windows scripts

2020-10-02 Thread Yuki Morishita (Jira)
Yuki Morishita created CASSANDRA-16171:
--

 Summary: Remove Windows scripts
 Key: CASSANDRA-16171
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16171
 Project: Cassandra
  Issue Type: Task
Reporter: Yuki Morishita


As per the email thread in cassandra-dev mailing list[1], remove windows 
scripts from Cassandra 4.0 onwards, due to the lack of maintenance and tests.

1: https://www.mail-archive.com/dev@cassandra.apache.org/msg15583.html 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15586) 4.0 quality testing: Cluster Setup and Maintenance

2020-10-02 Thread Yuki Morishita (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206241#comment-17206241
 ] 

Yuki Morishita commented on CASSANDRA-15586:


Hi Josh and Ekaterina,

Yes, the plan is to open two tickets, one for removing Windows scripts and one 
for updating docs.

I have patch for the former, so I will work on that right away.

> 4.0 quality testing: Cluster Setup and Maintenance
> --
>
> Key: CASSANDRA-15586
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15586
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>  Labels: 4.0-QA
> Fix For: 4.0-beta, 4.0-triage
>
>
> We want 4.0 to be easy for users to setup out of the box and just work. This 
> means having low friction when users download the Cassandra package and start 
> running it. For example, users should be able to easily configure and start 
> new 4.0 clusters and have tokens distributed evenly. Another example is 
> packaging, it should be easy to install Cassandra on all supported platforms 
> (e.g. packaging) and have Cassandra use standard platform integrations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16161) Validation Compactions causing Java GC pressure

2020-10-02 Thread Chris Lohfink (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lohfink updated CASSANDRA-16161:
--
Component/s: Tool/nodetool
 Local/Config
 Local/Compaction
   Assignee: Cameron Zemek
 Status: Open  (was: Triage Needed)

> Validation Compactions causing Java GC pressure
> ---
>
> Key: CASSANDRA-16161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16161
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Compaction, Local/Config, Tool/nodetool
>Reporter: Cameron Zemek
>Assignee: Cameron Zemek
>Priority: Normal
> Fix For: 3.11.x, 3.11.8
>
> Attachments: 16161.patch
>
>
> Validation Compactions are not rate limited which can cause Java GC pressure 
> and result in spikes in latency.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16161) Validation Compactions causing Java GC pressure

2020-10-02 Thread Chris Lohfink (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206225#comment-17206225
 ] 

Chris Lohfink edited comment on CASSANDRA-16161 at 10/2/20, 2:44 PM:
-

While tangental to adding a throttle, offheap merkle trees and some of the perf 
improvements (ie xor impl and more aggressive dereferencing) has helped a lot 
as well (CASSANDRA-15202). Its a lot better in 4.0 from my experiences.

For 3.x though I agree that validation compaction is brutal and this could 
provide a workaround to keep functioning.  Only question I think is if the 
compaction throughput throttle should be shared vs creating a new one but I am 
happy with this approach.

The code looks straight forward and am +1 to it. not sure about versions though


was (Author: clohfink):
While tangental to adding a throttle, offheap merkle trees and some of the perf 
improvements (ie xor impl and more aggressive dereferencing) has helped a lot 
as well (CASSANDRA-15202). Its a lot better in 4.0 from my experiences.

For 3.x though I agree that validation compaction is brutal and this could 
provide a workaround to keep functioning.  Only question I think is if the 
compaction throughput throttle should be shared vs creating a new one but I am 
happy with this approach.

The code looks straight forward, +1

> Validation Compactions causing Java GC pressure
> ---
>
> Key: CASSANDRA-16161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16161
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Cameron Zemek
>Priority: Normal
> Fix For: 3.11.x, 3.11.8
>
> Attachments: 16161.patch
>
>
> Validation Compactions are not rate limited which can cause Java GC pressure 
> and result in spikes in latency.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16161) Validation Compactions causing Java GC pressure

2020-10-02 Thread Chris Lohfink (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206225#comment-17206225
 ] 

Chris Lohfink commented on CASSANDRA-16161:
---

While tangental to adding a throttle, offheap merkle trees and some of the perf 
improvements (ie xor impl and more aggressive dereferencing) has helped a lot 
as well (CASSANDRA-15202). Its a lot better in 4.0 from my experiences.

For 3.x though I agree that validation compaction is brutal and this could 
provide a workaround to keep functioning.  Only question I think is if the 
compaction throughput throttle should be shared vs creating a new one but I am 
happy with this approach.

The code looks straight forward, +1

> Validation Compactions causing Java GC pressure
> ---
>
> Key: CASSANDRA-16161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16161
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Cameron Zemek
>Priority: Normal
> Fix For: 3.11.x, 3.11.8
>
> Attachments: 16161.patch
>
>
> Validation Compactions are not rate limited which can cause Java GC pressure 
> and result in spikes in latency.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Assigned] (CASSANDRA-16083) Missing JMX objects and attributes upgrading from 3.0 to 4.0

2020-10-02 Thread Brandon Williams (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams reassigned CASSANDRA-16083:


Assignee: (was: Uchenna)

> Missing JMX objects and attributes upgrading from 3.0 to 4.0
> 
>
> Key: CASSANDRA-16083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16083
> Project: Cassandra
>  Issue Type: Task
>  Components: Observability/Metrics
>Reporter: David Capwell
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Using the tools added in CASSANDRA-16082, below are the list of metrics 
> missing in 4.0 but present in 3.0.  The work here is to make sure we had 
> proper deprecation for each metric, and if not to add it back.
> {code}
> $ tools/bin/jmxtool diff -f yaml cassandra-3.0-jmx.yaml trunk-jmx.yaml 
> --ignore-missing-on-left
> Objects not in right:
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=EstimatedPartitionSizeHistogram
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=BloomFilterFalseRatio
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ReplicaFilteringProtectionRequests
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=RowCacheHitOutOfRange
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=CasPrepareLatency
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=CounterMutationStage,name=MaxPoolSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=TombstoneScannedHistogram
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=ActiveTasks
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=WaitingOnFreeMemtableSpace
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_columnfamilies,name=CasCommitTotalLatency
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=schema_aggregates,name=CasProposeLatency
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=ViewReadTime
> org.apache.cassandra.db:type=HintedHandoffManager
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=BloomFilterDiskSpaceUsed
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=RequestResponseStage,name=PendingTasks
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=MemtableSwitchCount
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=MemtableOnHeapSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=range_xfers,name=ReplicaFilteringProtectionRowsCachedPerQuery
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=SnapshotsSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=RecentBloomFilterFalsePositives
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ColUpdateTimeDeltaHistogram
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=SpeculativeRetries
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=LiveDiskSpaceUsed
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=views_builds_in_progress,name=ViewReadTime
> org.apache.cassandra.metrics:type=ThreadPools,path=internal,scope=MigrationStage,name=CompletedTasks
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=AllMemtablesLiveDataSize
> org.apache.cassandra.metrics:type=Table,keyspace=system,scope=batchlog,name=ViewReadTime
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=hints,name=BloomFilterFalsePositives
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=range_xfers,name=CompressionMetadataOffHeapMemoryUsed
> org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=TotalBlockedTasks
> org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system,scope=views_builds_in_progress,name=LiveScannedHistogram
> org.apache.cassandra.db:type=Tables,keyspace=system,table=views_builds_in_progress
> org.apache.cassandra.metrics:type=ThreadPools,path=internal,scope=MiscStage,name=ActiveTasks
> 

[jira] [Commented] (CASSANDRA-15582) 4.0 quality testing: metrics

2020-10-02 Thread Benjamin Lerer (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206223#comment-17206223
 ] 

Benjamin Lerer commented on CASSANDRA-15582:


I want to go throught the list of metrics that we have and which tests/coverage 
we have for each of them next week. Once we know what is not cover we should 
have a clear idea of the amount of work to be done.

> 4.0 quality testing: metrics
> 
>
> Key: CASSANDRA-15582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15582
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
> Attachments: Screen Shot 2020-04-07 at 5.47.17 PM.png
>
>
> In past releases we've unknowingly broken metrics integrations and introduced 
> performance regressions in metrics collection and reporting. We strive in 4.0 
> to not do that. Metrics should work well!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15586) 4.0 quality testing: Cluster Setup and Maintenance

2020-10-02 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206197#comment-17206197
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15586:
-

This was the thread:
[https://www.mail-archive.com/dev@cassandra.apache.org/msg15583.html]

I see most of the people expressed a support to [~yukim]'s idea to remove the 
support with the new release with:
 * Note in CHANGES.txt
 * Update "Getting Started" documents for Windows users (to direct them
to use docker or cloud)

So I believe I should open a ticket for point 2? [~yukim], [~jmckenzie] any 
different opinion?

> 4.0 quality testing: Cluster Setup and Maintenance
> --
>
> Key: CASSANDRA-15586
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15586
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Ekaterina Dimitrova
>Priority: Normal
>  Labels: 4.0-QA
> Fix For: 4.0-beta, 4.0-triage
>
>
> We want 4.0 to be easy for users to setup out of the box and just work. This 
> means having low friction when users download the Cassandra package and start 
> running it. For example, users should be able to easily configure and start 
> new 4.0 clusters and have tokens distributed evenly. Another example is 
> packaging, it should be easy to install Cassandra on all supported platforms 
> (e.g. packaging) and have Cassandra use standard platform integrations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15585) 4.0 quality testing: Test Frameworks, Tooling, Infra / Automation

2020-10-02 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206190#comment-17206190
 ] 

Josh McKenzie commented on CASSANDRA-15585:
---

{quote}It would be nice to see some Harry tests running regularly on say ASF 
hardware
{quote}
Strongly agree. I _wouldn't_, however, advocate for using 4.0 as a forcing 
function to motivate us to wire up these tests (and fallout tests for 
instance). If we believe that configuring recurring runs of these longer 
integration tests will either surface new defects or provide protection against 
regression in the run up to rc, 100% agree we should do it. If it's about 
blocking the release of 4.0 because we don't trust ourselves as a project to 
have the discipline to wire these things up...

Past is prologue so that resonates. That said, I'm talking to users multiple 
times a week who are asking about 4.0's availability, so I'd advocate we try 
and balance those needs.

We as a project could come together and define 4.0.1 as "8 weeks after 4.0 
GA's, the goal is to 1) fix anything operators surface in the release, and 2) 
have a suite of Harry tests running 24/7 on ASF hardware, and 3) have a fallout 
server for ASF C* up and running with adverse scenario tests running".

A little hand-wavy since we don't have a precedent for scoping work to include 
in releases up front but maybe moving the "we're going to block" force to the 
subsequent patch release could be a reasonable compromise for both the project 
community and user community?

> 4.0 quality testing: Test Frameworks, Tooling, Infra / Automation
> -
>
> Key: CASSANDRA-15585
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15585
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/python
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> *Shepherd: Jordan West*
> This area refers to contributions to test frameworks/tooling (e.g., dtests, 
> QuickTheories, CASSANDRA-14821), and automation enabling those tools to be 
> applied at scale (e.g., replay testing via Spark-based replay of captured FQL 
> logs).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15584) 4.0 quality testing: Tooling - External Ecosystem

2020-10-02 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206172#comment-17206172
 ] 

Josh McKenzie commented on CASSANDRA-15584:
---

Seems like we need some kind of metadata indicator for "working on during the 
run up to 4.0 but doesn't block the release" re: optionality of things like 
this.

4.0.x doesn't cover that because then it'd evict it from the kanban / query.

Maybe a label of some sort for it? I'll chew on this and try and come up with 
something sane, probably bring it up in slack. Thanks for the insight both of 
you.

> 4.0 quality testing: Tooling - External Ecosystem
> -
>
> Key: CASSANDRA-15584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15584
> Project: Cassandra
>  Issue Type: Task
>  Components: Tool/external
>Reporter: Josh McKenzie
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-rc, 4.0-triage
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> *Shepherd: Benjamin Lerer*
> Many users of Apache Cassandra employ open source tooling to automate 
> Cassandra configuration, runtime management, and repair scheduling. Prior to 
> release, we need to confirm that popular third-party tools function properly. 
> Current list of tools:
> || Name || Status || Contact ||
> | [Priam|http://netflix.github.io/Priam/] |{color:#00875A} *DONE WITH 
> ALPHA*{color} (need to be tested with beta) | [~sumanth.pasupuleti]| 
> | [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] | 
> *NOT STARTED* | [~stefan.miklosovic]| 
> | [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]| 
> *NOT STARTED* | [~stefan.miklosovic]|
> | [Instaclustr Cassandra 
> operator|https://github.com/instaclustr/cassandra-operator]|  
> {color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
> | [Instaclustr Cassandra Backup Restore | 
> https://github.com/instaclustr/cassandra-backup]|{color:#00875A}*DONE*{color} 
> | [~stefan.miklosovic]|
> | [Instaclustr Cassandra Sidecar | 
> https://github.com/instaclustr/cassandra-sidecar]|{color:#00875A}*DONE*{color}
>  | [~stefan.miklosovic]|
> | [Cassandra SSTable generator | 
> https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
>  [~stefan.miklosovic]|
> | [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} | 
> [~adejanovski]|
> | [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *NOT STARTED*| 
> [~adejanovski]|
> | [Casskop|https://orange-opensource.github.io/casskop/]| *NOT STARTED*| 
> Franck Dehay|
> | 
> [spark-cassandra-connector|https://github.com/datastax/spark-cassandra-connector]|
>  {color:#00875A}*DONE*{color}| [~jtgrabowski]|
> | [cass operator|https://github.com/datastax/cass-operator]| 
> {color:#00875A}*DONE*{color}| [~jimdickinson]|
> | [metric 
> collector|https://github.com/datastax/metric-collector-for-apache-cassandra]| 
> {color:#00875A}*DONE*{color}| [~tjake]|
> | [managment 
> API|https://github.com/datastax/management-api-for-apache-cassandra]| 
> {color:#00875A}*DONE*{color}| [~tjake]|  
> Columns descriptions:
> * *Name*: Name and link to the tool official page
> * *Status*: {{NOT STARTED}}, {{IN PROGRESS}}, {{BLOCKED}} if you hit any 
> issue and have to wait for it to be solved, {{DONE}}, {{AUTOMATIC}} if 
> testing 4.0 is part of your CI process.
> * *Contact*: The person acting as the contact point for that tool. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15582) 4.0 quality testing: metrics

2020-10-02 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206171#comment-17206171
 ] 

Josh McKenzie commented on CASSANDRA-15582:
---

[~blerer] - do we have a point of view on what work is remaining before we have 
confidence in our metrics?

> 4.0 quality testing: metrics
> 
>
> Key: CASSANDRA-15582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15582
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
> Attachments: Screen Shot 2020-04-07 at 5.47.17 PM.png
>
>
> In past releases we've unknowingly broken metrics integrations and introduced 
> performance regressions in metrics collection and reporting. We strive in 4.0 
> to not do that. Metrics should work well!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15538) 4.0 quality testing: Local Read/Write Path: Other Areas

2020-10-02 Thread Alex Petrov (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206170#comment-17206170
 ] 

Alex Petrov commented on CASSANDRA-15538:
-

bq. it's incredibly helpful reproducing things discovered in diff testing.

Diff testing is a great tool, and I think it's super useful, but Harry 
hopefully goes much further than it does. First, we exercise all kinds of 
queries in harry: we insert, update, and delete data, so it exercises code 
paths that are not touched by diff. 

> Is there also a "generative fuzz without specific workload context" role it 
> serves? If so, is that something we can enumerate here? Thinking something 
> like a gdoc to brain dump "here's the scenarios we can and should test with 
> tool X to confirm subsystem Y" or something like that.

We've tried our best to make Harry functional enough to test pretty much 
anything, but we don't have a single-button solution for anything just yet. In 
other words, if you want to exerice "SRP after RR", you can still use Harry 
machinery to create and validate data, but you'll have to create specific 
message filters that create a situation yourself. In future, we'll create 
failure scenarios and improving the way we simulate stuff, too. I'm always 
happy to answer any questions about how to implement certain things using Harry.

> 4.0 quality testing: Local Read/Write Path: Other Areas
> ---
>
> Key: CASSANDRA-15538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15538
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> *Shepherd: Aleksey Yeschenko*
> Testing in this area refers to the local read/write path (StorageProxy, 
> ColumnFamilyStore, Memtable, SSTable reading/writing, etc). We are still 
> finding numerous bugs and issues with the 3.0 storage engine rewrite 
> (CASSANDRA-8099). For 4.0 we want to ensure that we thoroughly cover the 
> local read/write path with techniques such as property-based testing, fuzzing 
> ([example|http://cassandra.apache.org/blog/2018/10/17/finding_bugs_with_property_based_testing.html]),
>  and a source audit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15538) 4.0 quality testing: Local Read/Write Path: Other Areas

2020-10-02 Thread Josh McKenzie (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206169#comment-17206169
 ] 

Josh McKenzie commented on CASSANDRA-15538:
---

{quote}bq. Having it run 24/7 on as large a cluster of boxes as possible, 
reporting and collecting invariant failures
{quote}
Indeed. The example yaml in the repo is a 2 hour test with a random schema; is 
there an example or could the codebase be augmented with a more robust schema 
intended to be run for a longer period of time for people to base subsequent 
work off?

Given how nascent the Harry project is + the lack of examples like that (no 
judgement; new code is new code), I'm wary of us establishing a release 
dependency between Harry being wired into our "official" CI/CD checklist 
pipeline vs. having built our confidence in this release with the dirtier, more 
disappointing "tested X schemas with cassandra-diff and FQLTool and fixed 
defects we reproduced with Harry" and having an orthogonal task of wiring up 
24/7 long-running soaks using Harry and Fallout as we move forward during the 
4.0.1 etc time frame.

You certainly have more understanding re: the proximity of Harry's current 
state to "is ready to be wired into our CI/CD and appropriate to block the 
release on" [~aleksey]  and/or [~ifesdjeen] - what do you guys think?

> 4.0 quality testing: Local Read/Write Path: Other Areas
> ---
>
> Key: CASSANDRA-15538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15538
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Reference [doc from 
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
>  for context.
> *Shepherd: Aleksey Yeschenko*
> Testing in this area refers to the local read/write path (StorageProxy, 
> ColumnFamilyStore, Memtable, SSTable reading/writing, etc). We are still 
> finding numerous bugs and issues with the 3.0 storage engine rewrite 
> (CASSANDRA-8099). For 4.0 we want to ensure that we thoroughly cover the 
> local read/write path with techniques such as property-based testing, fuzzing 
> ([example|http://cassandra.apache.org/blog/2018/10/17/finding_bugs_with_property_based_testing.html]),
>  and a source audit.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16121) Circleci should run cqlshlib tests as well

2020-10-02 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206158#comment-17206158
 ] 

Berenguer Blasi commented on CASSANDRA-16121:
-

[~e.dimitrova] iirc the only thing that survives the execution of the sh script 
is the xml junit results file. So there are no other artifacts to store 
available. You get the logs in stdout and the junit results but nothing else.

Also yes the patch applies cleanly for me.

> Circleci should run cqlshlib tests as well
> --
>
> Key: CASSANDRA-16121
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16121
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI, Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Currently circleci is not running cqlshlib tests. This resulted in some bugs 
> not being caught before committing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16121) Circleci should run cqlshlib tests as well

2020-10-02 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206155#comment-17206155
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-16121 at 10/2/20, 1:12 PM:
---

Thank you for your work.

One last thing to add and I will +1. Artifacts are not stored.

Otherwise I apply the patch and the only thing I see is this line but the patch 
applies properly so it doesn't look as an issue?
{code:java}
missing header for unified diff at line 1 of patch{code}
Also, [~Bereng], you mentioned this line doesn't appear on your machine...

[~dcapwell], can you also check it, please?


was (Author: e.dimitrova):
Thank you for your work.

One last thing to add and I will +1. Artifacts are not stored.

Otherwise I apply the patch and the only thing I see is this line but the patch 
applies properly so it doesn't look as an issue?
{code:java}
missing header for unified diff at line 1 of patch{code}
Also you mentioned this line doesn't appear on your machine...

[~dcapwell], can you also check it, please?

> Circleci should run cqlshlib tests as well
> --
>
> Key: CASSANDRA-16121
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16121
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI, Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Currently circleci is not running cqlshlib tests. This resulted in some bugs 
> not being caught before committing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16121) Circleci should run cqlshlib tests as well

2020-10-02 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206155#comment-17206155
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16121:
-

Thank you for your work.

One last thing to add and I will +1. Artifacts are not stored.

Otherwise I apply the patch and the only thing I see is this line but the patch 
applies properly so it doesn't look as an issue?
{code:java}
missing header for unified diff at line 1 of patch{code}
Also you mentioned this line doesn't appear on your machine...

[~dcapwell], can you also check it, please?

> Circleci should run cqlshlib tests as well
> --
>
> Key: CASSANDRA-16121
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16121
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI, Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Currently circleci is not running cqlshlib tests. This resulted in some bugs 
> not being caught before committing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16117) Improve docs about frozen types and invert UDT/Tuple order

2020-10-02 Thread Jira


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17206144#comment-17206144
 ] 

Fábio Takeo Ueno commented on CASSANDRA-16117:
--

[~benedict] and [~blerer], I addressed the comments you made, but there's a 
point in which I replied and I'm not quite sure how to proceed...

When you have time, could you guys do a re-review?

Hope I'm not being inconvenient!

> Improve docs about frozen types and invert UDT/Tuple order
> --
>
> Key: CASSANDRA-16117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16117
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website
>Reporter: Fábio Takeo Ueno
>Assignee: Fábio Takeo Ueno
>Priority: Low
> Fix For: 4.0, 4.0-beta, 4.0-triage
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Currently, there's no documentation regarding frozen/non-frozen types.
> Also, a tuple is mentioned after the definition of UDT: "tuples can be though 
> as anonymous UDT with anonymous fields". Since in the code base a UDT is an 
> extension of a tuple, it would be nice to invert these in the docs.
> This issue addresses both topics.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-13917) COMPACT STORAGE queries on dense static tables accept hidden column1 and value columns

2020-10-02 Thread Marcus Eriksson (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-13917:

Fix Version/s: (was: 3.11.x)
   (was: 3.0.x)
   3.0.21
   3.11.7

> COMPACT STORAGE queries on dense static tables accept hidden column1 and 
> value columns
> --
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Low
>  Labels: lhf
> Fix For: 3.0.21, 3.11.7
>
> Attachments: 13917-3.0-testall-13.12.2019, 
> 13917-3.0-testall-16.01.2020, 13917-3.0-testall-2.png, 
> 13917-3.0-testall-20.11.2019.png, 13917-3.0-upgrade-16.01.2020, 
> 13917-3.0.png, 13917-3.11-testall-13.12.2019, 
> 13917-3.11-testall-16.01.2020.png, 13917-3.11-testall-2.png, 
> 13917-3.11-testall-20.11.2019.png, 13917-3.11-upgrade-16.01.2020.png, 
> 13917-3.11.png
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH 
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?, 
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1, 
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)", 
> 1, 1, 1, ByteBufferUtil.bytes('a'));   
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?, 
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should 
> be as easy as just adding validations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15880) Memory leak in CompressedChunkReader

2020-10-02 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15880:
---
Status: Ready to Commit  (was: Review In Progress)

The PRs have been approved and the CI runs look good.  

> Memory leak in CompressedChunkReader
> 
>
> Key: CASSANDRA-15880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15880
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Compression
>Reporter: Jaroslaw Grabowski
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0, 3.11.x, 4.0-triage
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> CompressedChunkReader uses java.lang.ThreadLocal to reuse ByteBuffer for 
> compressed data. ByteBuffers leak due to peculiar ThreadLocal quality.
> ThreadLocals are stored in a map, where the key is a weak reference to a 
> ThreadLocal and the value is the user's object (ByteBuffer in this case). 
> When a last strong reference to a ThreadLocal is lost, weak reference to 
> ThreadLocal (key) is removed but the value (ByteBuffer) is kept until cleaned 
> by ThreadLocal heuristic expunge mechanism. See ThreadLocal's "stale entries" 
> for details.
> When a number of long-living threads is high enough this results in thousands 
> of ByteBuffers stored as stale entries in ThreadLocals. In a not-so-lucky 
> scenario we get OutOfMemoryException.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15880) Memory leak in CompressedChunkReader

2020-10-02 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15880:
---
Reviewers: Benjamin Lerer, Branimir Lambov  (was: Benjamin Lerer)

> Memory leak in CompressedChunkReader
> 
>
> Key: CASSANDRA-15880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15880
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Compression
>Reporter: Jaroslaw Grabowski
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0, 3.11.x, 4.0-triage
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> CompressedChunkReader uses java.lang.ThreadLocal to reuse ByteBuffer for 
> compressed data. ByteBuffers leak due to peculiar ThreadLocal quality.
> ThreadLocals are stored in a map, where the key is a weak reference to a 
> ThreadLocal and the value is the user's object (ByteBuffer in this case). 
> When a last strong reference to a ThreadLocal is lost, weak reference to 
> ThreadLocal (key) is removed but the value (ByteBuffer) is kept until cleaned 
> by ThreadLocal heuristic expunge mechanism. See ThreadLocal's "stale entries" 
> for details.
> When a number of long-living threads is high enough this results in thousands 
> of ByteBuffers stored as stale entries in ThreadLocals. In a not-so-lucky 
> scenario we get OutOfMemoryException.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15880) Memory leak in CompressedChunkReader

2020-10-02 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15880:
---
Reviewers: Benjamin Lerer, Branimir Lambov, Benjamin Lerer  (was: Benjamin 
Lerer, Branimir Lambov)
   Benjamin Lerer, Branimir Lambov, Benjamin Lerer  (was: Benjamin 
Lerer, Branimir Lambov)
   Status: Review In Progress  (was: Patch Available)

> Memory leak in CompressedChunkReader
> 
>
> Key: CASSANDRA-15880
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15880
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Compression
>Reporter: Jaroslaw Grabowski
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0, 3.11.x, 4.0-triage
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> CompressedChunkReader uses java.lang.ThreadLocal to reuse ByteBuffer for 
> compressed data. ByteBuffers leak due to peculiar ThreadLocal quality.
> ThreadLocals are stored in a map, where the key is a weak reference to a 
> ThreadLocal and the value is the user's object (ByteBuffer in this case). 
> When a last strong reference to a ThreadLocal is lost, weak reference to 
> ThreadLocal (key) is removed but the value (ByteBuffer) is kept until cleaned 
> by ThreadLocal heuristic expunge mechanism. See ThreadLocal's "stale entries" 
> for details.
> When a number of long-living threads is high enough this results in thousands 
> of ByteBuffers stored as stale entries in ThreadLocals. In a not-so-lucky 
> scenario we get OutOfMemoryException.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-15584) 4.0 quality testing: Tooling - External Ecosystem

2020-10-02 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15584:
---
Description: 
Reference [doc from 
NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
 for context.

*Shepherd: Benjamin Lerer*

Many users of Apache Cassandra employ open source tooling to automate Cassandra 
configuration, runtime management, and repair scheduling. Prior to release, we 
need to confirm that popular third-party tools function properly. 

Current list of tools:
|| Name || Status || Contact ||
| [Priam|http://netflix.github.io/Priam/] |{color:#00875A} *DONE WITH 
ALPHA*{color} (need to be tested with beta) | [~sumanth.pasupuleti]| 
| [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] | *NOT 
STARTED* | [~stefan.miklosovic]| 
| [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]| *NOT 
STARTED* | [~stefan.miklosovic]|
| [Instaclustr Cassandra 
operator|https://github.com/instaclustr/cassandra-operator]|  
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Instaclustr Cassandra Backup Restore | 
https://github.com/instaclustr/cassandra-backup]|{color:#00875A}*DONE*{color} | 
[~stefan.miklosovic]|
| [Instaclustr Cassandra Sidecar | 
https://github.com/instaclustr/cassandra-sidecar]|{color:#00875A}*DONE*{color} 
| [~stefan.miklosovic]|
| [Cassandra SSTable generator | 
https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
 [~stefan.miklosovic]|
| [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} | 
[~adejanovski]|
| [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *NOT STARTED*| 
[~adejanovski]|
| [Casskop|https://orange-opensource.github.io/casskop/]| *NOT STARTED*| Franck 
Dehay|
| 
[spark-cassandra-connector|https://github.com/datastax/spark-cassandra-connector]|
 {color:#00875A}*DONE*{color}| [~jtgrabowski]|
| [cass operator|https://github.com/datastax/cass-operator]| 
{color:#00875A}*DONE*{color}| [~jimdickinson]|
| [metric 
collector|https://github.com/datastax/metric-collector-for-apache-cassandra]| 
{color:#00875A}*DONE*{color}| [~tjake]|
| [managment 
API|https://github.com/datastax/management-api-for-apache-cassandra]| 
{color:#00875A}*DONE*{color}| [~tjake]|  

Columns descriptions:
* *Name*: Name and link to the tool official page
* *Status*: {{NOT STARTED}}, {{IN PROGRESS}}, {{BLOCKED}} if you hit any issue 
and have to wait for it to be solved, {{DONE}}, {{AUTOMATIC}} if testing 4.0 is 
part of your CI process.
* *Contact*: The person acting as the contact point for that tool. 

  was:
Reference [doc from 
NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
 for context.

*Shepherd: Benjamin Lerer*

Many users of Apache Cassandra employ open source tooling to automate Cassandra 
configuration, runtime management, and repair scheduling. Prior to release, we 
need to confirm that popular third-party tools function properly. 

Current list of tools:
|| Name || Status || Contact ||
| [Priam|http://netflix.github.io/Priam/] | *DONE WITH ALPHA* (need to be 
tested with beta) | [~sumanth.pasupuleti]| 
| [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] | *NOT 
STARTED* | [~stefan.miklosovic]| 
| [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]| *NOT 
STARTED* | [~stefan.miklosovic]|
| [Instaclustr Cassandra 
operator|https://github.com/instaclustr/cassandra-operator]|  
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Instaclustr Cassandra Backup Restore | 
https://github.com/instaclustr/cassandra-backup]|{color:#00875A}*DONE*{color} | 
[~stefan.miklosovic]|
| [Instaclustr Cassandra Sidecar | 
https://github.com/instaclustr/cassandra-sidecar]|{color:#00875A}*DONE*{color} 
| [~stefan.miklosovic]|
| [Cassandra SSTable generator | 
https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
 [~stefan.miklosovic]|
| [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} | 
[~adejanovski]|
| [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *NOT STARTED*| 
[~adejanovski]|
| [Casskop|https://orange-opensource.github.io/casskop/]| *NOT STARTED*| Franck 
Dehay|
| 
[spark-cassandra-connector|https://github.com/datastax/spark-cassandra-connector]|
 {color:#00875A}*DONE*{color}| [~jtgrabowski]|
| [cass operator|https://github.com/datastax/cass-operator]| 
{color:#00875A}*DONE*{color}| [~jimdickinson]|
| [metric 
collector|https://github.com/datastax/metric-collector-for-apache-cassandra]| 
{color:#00875A}*DONE*{color}| [~tjake]|
| [managment 
API|https://github.com/datastax/management-api-for-apache-cassandra]| 
{color:#00875A}*DONE*{color}| [~tjake]|  

Columns descriptions:
* *Name*: Name and link to the tool official page
* *Status*: {{NOT STARTED}}, {{IN PROGRESS}}, {{BLOCKED}} if you 

[jira] [Updated] (CASSANDRA-15584) 4.0 quality testing: Tooling - External Ecosystem

2020-10-02 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15584:
---
Description: 
Reference [doc from 
NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
 for context.

*Shepherd: Benjamin Lerer*

Many users of Apache Cassandra employ open source tooling to automate Cassandra 
configuration, runtime management, and repair scheduling. Prior to release, we 
need to confirm that popular third-party tools function properly. 

Current list of tools:
|| Name || Status || Contact ||
| [Priam|http://netflix.github.io/Priam/] | *DONE WITH ALPHA* (need to be 
tested with beta) | [~sumanth.pasupuleti]| 
| [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] | *NOT 
STARTED* | [~stefan.miklosovic]| 
| [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]| *NOT 
STARTED* | [~stefan.miklosovic]|
| [Instaclustr Cassandra 
operator|https://github.com/instaclustr/cassandra-operator]|  
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Instaclustr Cassandra Backup Restore | 
https://github.com/instaclustr/cassandra-backup]|{color:#00875A}*DONE*{color} | 
[~stefan.miklosovic]|
| [Instaclustr Cassandra Sidecar | 
https://github.com/instaclustr/cassandra-sidecar]|{color:#00875A}*DONE*{color} 
| [~stefan.miklosovic]|
| [Cassandra SSTable generator | 
https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
 [~stefan.miklosovic]|
| [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} | 
[~adejanovski]|
| [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *NOT STARTED*| 
[~adejanovski]|
| [Casskop|https://orange-opensource.github.io/casskop/]| *NOT STARTED*| Franck 
Dehay|
| 
[spark-cassandra-connector|https://github.com/datastax/spark-cassandra-connector]|
 {color:#00875A}*DONE*{color}| [~jtgrabowski]|
| [cass operator|https://github.com/datastax/cass-operator]| 
{color:#00875A}*DONE*{color}| [~jimdickinson]|
| [metric 
collector|https://github.com/datastax/metric-collector-for-apache-cassandra]| 
{color:#00875A}*DONE*{color}| [~tjake]|
| [managment 
API|https://github.com/datastax/management-api-for-apache-cassandra]| 
{color:#00875A}*DONE*{color}| [~tjake]|  

Columns descriptions:
* *Name*: Name and link to the tool official page
* *Status*: {{NOT STARTED}}, {{IN PROGRESS}}, {{BLOCKED}} if you hit any issue 
and have to wait for it to be solved, {{DONE}}, {{AUTOMATIC}} if testing 4.0 is 
part of your CI process.
* *Contact*: The person acting as the contact point for that tool. 

  was:
Reference [doc from 
NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
 for context.

*Shepherd: Benjamin Lerer*

Many users of Apache Cassandra employ open source tooling to automate Cassandra 
configuration, runtime management, and repair scheduling. Prior to release, we 
need to confirm that popular third-party tools function properly. 

Current list of tools:
|| Name || Status || Contact ||
| [Priam|http://netflix.github.io/Priam/] | *NOT STARTED* | 
[~sumanth.pasupuleti]| 
| [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] | *NOT 
STARTED* | [~stefan.miklosovic]| 
| [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]| *NOT 
STARTED* | [~stefan.miklosovic]|
| [Instaclustr Cassandra 
operator|https://github.com/instaclustr/cassandra-operator]|  
{color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
| [Instaclustr Cassandra Backup Restore | 
https://github.com/instaclustr/cassandra-backup]|{color:#00875A}*DONE*{color} | 
[~stefan.miklosovic]|
| [Instaclustr Cassandra Sidecar | 
https://github.com/instaclustr/cassandra-sidecar]|{color:#00875A}*DONE*{color} 
| [~stefan.miklosovic]|
| [Cassandra SSTable generator | 
https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
 [~stefan.miklosovic]|
| [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} | 
[~adejanovski]|
| [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *NOT STARTED*| 
[~adejanovski]|
| [Casskop|https://orange-opensource.github.io/casskop/]| *NOT STARTED*| Franck 
Dehay|
| 
[spark-cassandra-connector|https://github.com/datastax/spark-cassandra-connector]|
 {color:#00875A}*DONE*{color}| [~jtgrabowski]|
| [cass operator|https://github.com/datastax/cass-operator]| 
{color:#00875A}*DONE*{color}| [~jimdickinson]|
| [metric 
collector|https://github.com/datastax/metric-collector-for-apache-cassandra]| 
{color:#00875A}*DONE*{color}| [~tjake]|
| [managment 
API|https://github.com/datastax/management-api-for-apache-cassandra]| 
{color:#00875A}*DONE*{color}| [~tjake]|  

Columns descriptions:
* *Name*: Name and link to the tool official page
* *Status*: {{NOT STARTED}}, {{IN PROGRESS}}, {{BLOCKED}} if you hit any issue 
and have to wait for it to be solved, 

[jira] [Updated] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging

2020-10-02 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-15958:
---
Reviewers: Benjamin Lerer, Yifan Cai, Benjamin Lerer  (was: Benjamin Lerer, 
Yifan Cai)
   Benjamin Lerer, Yifan Cai, Benjamin Lerer  (was: Benjamin Lerer, 
Yifan Cai)
   Status: Review In Progress  (was: Patch Available)

> org.apache.cassandra.net.ConnectionTest testMessagePurging
> --
>
> Key: CASSANDRA-15958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15958
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta, 4.0-triage
>
>
> Build: 
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> Build: 
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> java.util.concurrent.TimeoutException
>   at org.apache.cassandra.net.AsyncPromise.get(AsyncPromise.java:258)
>   at org.apache.cassandra.net.FutureDelegate.get(FutureDelegate.java:143)
>   at 
> org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:268)
>   at 
> org.apache.cassandra.net.ConnectionTest.testManual(ConnectionTest.java:236)
>   at 
> org.apache.cassandra.net.ConnectionTest.testMessagePurging(ConnectionTest.java:679)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



  1   2   >