[jira] [Updated] (CASSANDRA-15459) Short read protection doesn't work on group-by queries

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-15459:

Reviewers: Benjamin Lerer, Caleb Rackliffe, Caleb Rackliffe  (was: Benjamin 
Lerer, Caleb Rackliffe)
   Benjamin Lerer, Caleb Rackliffe, Caleb Rackliffe  (was: Benjamin 
Lerer, Caleb Rackliffe)
   Status: Review In Progress  (was: Patch Available)

> Short read protection doesn't work on group-by queries
> --
>
> Key: CASSANDRA-15459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15459
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Coordination
>Reporter: ZhaoYang
>Assignee: Andres de la Peña
>Priority: Normal
>  Labels: correctness
> Fix For: 3.11.x, 4.0-beta
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> [DTest to 
> reproduce|https://github.com/apache/cassandra-dtest/compare/master...jasonstack:srp_group_by_trunk?expand=1]:
>  it affects all versions..
> {code}
> In a two-node cluster with RF = 2
> Execute only on Node1:
> * Insert pk=1 and ck=1 with timestamp 9
> * Delete pk=0 and ck=0 with timestamp 10
> * Insert pk=2 and ck=2 with timestamp 9
> Execute only on Node2:
> * Delete pk=1 and ck=1 with timestamp 10
> * Insert pk=0 and ck=0 with timestamp 9
> * Delete pk=2 and ck=2 with timestamp 10
> Query: "SELECT pk, c FROM %s GROUP BY pk LIMIT 1"
> * Expect no live data, but got [0, 0]
> {code}
> Note: for group-by queries, SRP should use "group counted" to calculate 
> limits used for SRP query, rather than "row counted".



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15938) Fix support for adding UDT fields to clustering keys

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170486#comment-17170486
 ] 

Caleb Rackliffe commented on CASSANDRA-15938:
-

Taking a quick look at the tests, it seems like the 3.11 run [failed to 
build|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra/36/workflows/5d23d9e7-1bf9-4538-a69d-b355adf5650f/jobs/2300].
 The unit tests also weren't run w/ J11 on trunk. Otherwise, things look about 
as expected.

> Fix support for adding UDT fields to clustering keys
> 
>
> Key: CASSANDRA-15938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15938
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/UDT
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Adding UDT fields to clustering keys is broken in all versions, however 
> slightly differently.
> In 4.0, there will be a brief moment while schema changes are propagated 
> during which we won’t be able to decode and compare byte sequences. 
> Unfortunately, it is unclear what we should do in such cases, since we can’t 
> just come up with a comparator, and we can’t ignore non-null trailing values, 
> since this will lead to cases where compare for tuples `a;1` and `a;2` would 
> return 0, effectively making them equal, and we don’t know how to compare 
> unknown trailing values. Probably we should reject such query since we can’t 
> sort correctly, but we should make the error message more descriptive than 
> just "Index 1 out of bounds for length 1”. The only problem is that we get 
> this exception only on flush right now, so data already propagates to the 
> node by that time.
> In 3.0, the problem is a bit worse than that, since in 3.0 we do not ignore 
> trailing nulls, so some of the values, written before `ALTER TYPE .. ADD` 
> become inaccessible. Both old values, and the new ones should always be 
> accessible.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15861) Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170484#comment-17170484
 ] 

Caleb Rackliffe commented on CASSANDRA-15861:
-

bq. if you could guarantee that no more than 1 index resample can happen at 
once for a given sstable, the only thing you'd need to synchronize in 
`cloneWithNewSummarySamplingLevel` is `saveSummary`. If you did that, you could 
just synchronize hard link creation on `tidy.global`, instead of introducing a 
new lock.

I think we can guarantee that only one index summary resampling is going on at 
a time for an SSTable, but not necessarily on the same thread from run to run. 
What non-final state does the {{synchronized (tidy.global)}} block in 
{{recloneWithNewSummarySamplingLevel()}} protect? It's probably not the 
metadata, since that's already a {{TableMetadataRef}} (and effectively 
volatile). That leaves {{indexSummary}}, which perhaps we cold make 
{{volatile}}, and all the state used in {{cloneAndReplace()}}...but we could 
just extend the {{synchronized (tidy.global)}} block to include the latter. 
Nothing expensive happens inside {{cloneAndReplace()}}, AFAICT.

I tried this locally, just with a quick and dirty substitution in the current 
wrapper methods for the read and write lock, and it does seem to pass the newly 
added tests (and fail with locking removed completely).

CC [~jasonstack]

> Mutating sstable component may race with entire-sstable-streaming(ZCS) 
> causing checksum validation failure
> --
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair, Consistency/Streaming, 
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator - 
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors: 
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081 
> CassandraEntireSSTableStreamReader.java:145 - [Stream 
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream 
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
>   at 
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
>   at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
>   at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>   at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>   at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
>   at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for 
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>  
> In the above test, it executes "nodetool repair" on node1 and kills node2 
> during repair. At the end, node3 reports checksum validation failure on 
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies 
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be 
> transferred to node3.
> 3. Before node1 actually sends the files 

[jira] [Commented] (CASSANDRA-15861) Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170479#comment-17170479
 ] 

Caleb Rackliffe commented on CASSANDRA-15861:
-

bq. 1) Orphaned hard links need to be cleaned up on startup.
bq. 2) Using the streaming session id for the hard link name, instead of a time 
uuid, would make debugging some issues easier.

+1

bq. 3) ComponentManifest: the changes to this class feel a bit awkward to me. I 
think it would be cleaner if hard link creation and management was handled by a 
separate class. It should also be autocloseable so we're not deleting hard 
links in a finally block.

Agree (and commented to this effect) that we should make whatever that class is 
auto-closable. I'm just not sure what level we want to do this at. The creation 
of what we're now calling {{ComponentManifest}} might not be simple to break 
up. We could leave {{ComponentManifest}} the way it was before this patch and 
have a separate class, let's call it {{ComponentContext}}, that embeds it. The 
{{ComponentContext}} would only be used on the write side, where it would 
mostly be responsible for what the manifest doesn't do: provide a channel and 
file size. There could even be an interface, let's call it 
{{StreamingComponent}}, which just provides a size and channel, and the 
{{ComponentContext}} could be an {{Iterable}}. (It seems 
like this would fit the usage in {{CassandraEntireSSTableStreamWriter}}.) 
Either way, the receiving side of the streaming logic could continue to deal 
with just a clean {{ComponentManifest}}, free of things like the unused 
{{hardLinks}} (which doesn't need to be transient?).

> Mutating sstable component may race with entire-sstable-streaming(ZCS) 
> causing checksum validation failure
> --
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair, Consistency/Streaming, 
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator - 
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors: 
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081 
> CassandraEntireSSTableStreamReader.java:145 - [Stream 
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream 
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
>   at 
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
>   at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
>   at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>   at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>   at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
>   at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for 
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>  
> In the above test, it executes "nodetool repair" on node1 and kills node2 
> during repair. At the end, node3 reports checksum validation failure on 
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started 

[jira] [Commented] (CASSANDRA-15981) jvm-dtests crash on java 11

[Jon Meredith (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170478#comment-17170478
 ] 

Jon Meredith commented on CASSANDRA-15981:
--

LGTM

I reproduced the SEGV on my laptop against a commit I know fails (

43c19878e38fbe260f9e6143aa43836e85cf2f44) then cherry-picked the patch on top 
and ran 140 iterations without failure.
 
Checked the process arguments

Under J11 - you can see {{-CMSClassUnloadingEnabled}}

{code}
-Dstorage-config=/test/conf -Djava.awt.headless=true 
-javaagent:/lib/jamm-0.3.2.jar -ea -Djava.io.tmpdir= 
-Dcassandra.debugrefcount=true -Xss256k -XX:SoftRefLRUPolicyMSPerMB=0 
-Dcassandra.memtable_row_overhead_computation_step=100 
-Dcassandra.test.use_prepared=true -Dcassandra.test.offsetseed=0 
-Dcassandra.test.sstableformatdevelopment=true 
-Djava.security.egd=file:/dev/urandom -Dcassandra.testtag= 
-Dcassandra.keepBriefBrief=true -Dcassandra.strict.runtime.checks=true 
-Djdk.attach.allowAttachSelf=true -XX:+UseConcMarkSweepGC 
-XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly 
-XX:CMSWaitDuration=1 -XX:+CMSParallelInitialMarkEnabled 
-XX:+CMSEdenChunksRecordAlways --add-exports 
java.base/jdk.internal.misc=ALL-UNNAMED --add-exports 
java.base/jdk.internal.ref=ALL-UNNAMED --add-exports 
java.base/sun.nio.ch=ALL-UNNAMED --add-exports 
java.management.rmi/com.sun.jmx.remote.internal.rmi=ALL-UNNAMED --add-exports 
java.rmi/sun.rmi.registry=ALL-UNNAMED --add-exports 
java.rmi/sun.rmi.server=ALL-UNNAMED --add-exports java.sql/java.sql=ALL-UNNAMED 
--add-opens java.base/java.lang.module=ALL-UNNAMED --add-opens 
java.base/java.net=ALL-UNNAMED --add-opens 
java.base/jdk.internal.loader=ALL-UNNAMED --add-opens 
java.base/jdk.internal.ref=ALL-UNNAMED --add-opens 
java.base/jdk.internal.reflect=ALL-UNNAMED --add-opens 
java.base/jdk.internal.math=ALL-UNNAMED --add-opens 
java.base/jdk.internal.module=ALL-UNNAMED --add-opens 
java.base/jdk.internal.util.jar=ALL-UNNAMED --add-opens 
jdk.management/com.sun.management.internal=ALL-UNNAMED -DQT_SHRINKS=0 
-XX:-CMSClassUnloadingEnabled 
-Dlegacy-sstable-root=/test/data/legacy-sstables 
-Dinvalid-legacy-sstable-root=/test/data/invalid-legacy-sstables 
-Dcassandra.ring_delay_ms=1000 -Dcassandra.tolerate_sstable_size=true 
-Dcassandra.config.loader=org.apache.cassandra.OffsetAwareConfigurationLoader 
-Dcassandra.skip_sync=true -Xmx1024m
{code}

Under J8 - and has {{-XX:MaxMetaspaceExpansion=64M -XX:MaxMetaspaceSize=512M 
-XX:MetaspaceSize=128M}}

{code}
-Dstorage-config=/test/conf -Djava.awt.headless=true 
-javaagent:/lib/jamm-0.3.2.jar -ea -Djava.io.tmpdir= 
-Dcassandra.debugrefcount=true -Xss256k -XX:SoftRefLRUPolicyMSPerMB=0 
-Dcassandra.memtable_row_overhead_computation_step=100 
-Dcassandra.test.use_prepared=true -Dcassandra.test.offsetseed=0 
-Dcassandra.test.sstableformatdevelopment=true 
-Djava.security.egd=file:/dev/urandom -Dcassandra.testtag= 
-Dcassandra.keepBriefBrief=true -Dcassandra.strict.runtime.checks=true 
-DQT_SHRINKS=0 -XX:MaxMetaspaceExpansion=64M -XX:MaxMetaspaceSize=512M 
-XX:MetaspaceSize=128M -Dlegacy-sstable-root=/test/data/legacy-sstables 
-Dinvalid-legacy-sstable-root=/test/data/invalid-legacy-sstables 
-Dcassandra.ring_delay_ms=1000 -Dcassandra.tolerate_sstable_size=true 
-Dcassandra.config.loader=org.apache.cassandra.OffsetAwareConfigurationLoader 
-Dcassandra.skip_sync=true -Xmx1024m
{code}

> jvm-dtests crash on java 11
> ---
>
> Key: CASSANDRA-15981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15981
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> There is a race condition bug with CMS and class unloading which cause the 
> JVM to crash.  Since jvm-dtests rely on class loaders and unloading, this 
> causes sporadic JVM crashes that look like the following in CI logs
> {code}
> junit.framework.AssertionFailedError: Forked Java VM exited abnormally. 
> Please note the time in the report does not reflect the time until the VM 
> exit.
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.util.Vector.forEach(Vector.java:1387)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> 

[jira] [Commented] (CASSANDRA-15937) JMX output inconsistencies from CASSANDRA-7544 storage-port-configurable-per-node

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170459#comment-17170459
 ] 

David Capwell commented on CASSANDRA-15937:
---

This patch looks to be a few things
* refactor to centralize logic (toString(true), hostAndPortForJmx, etc.)
* bug fixes for JMX (InetAddressAndPort, Guava collections, etc.)
* check output to match 3.x output

Overall I am good with this patch, but would like to see tests enforcing the 
behavioral changes; mostly looks like updating existing tests impacted (like 
repair tests).

I will need to do another pass again as this is large, but looks good so far.

> JMX output inconsistencies from CASSANDRA-7544 
> storage-port-configurable-per-node
> -
>
> Key: CASSANDRA-15937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15937
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability/JMX
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> CASSANDRA-7544 introduced changes to allow the storage port number to be 
> configured per-node. As part of that work it introduces new MBeans for 
> MessagingService, FailureDetector providing new 'WithPort' versions that 
> include the new port information, however there are some mistakes and 
> inconsistencies.
> {code:java}
>                            3.11.6                trunk                  trunk 
> w/Port          Notes
>   
>  AllEndpointStates        /127.0.0.1\n...       /127.0.0.3\n...        
> 127.0.0.3:7000\n        (trunk /w port different)
>  SimpleStates             /127.0.0.2=UP         /127.0.0.2=UP          
> 127.0.0.3:7000=UP       (trunk /w port different)
>  LargeMessagePendingTasks /127.0.0.1=0          /127.0.0.1=0           
> 127.0.0.3:7000=0        (trunk /w port different)
>  TimeoutsPerHost          127.0.0.1=0           /127.0.0.1=0           
> 127.0.0.3:7000=0        3.0/3.11.6 & trunk differ.
>  BackPressurePerHost      127.0.0.1=Infinity    /127.0.0.2=Infinity    
> /127.0.0.2=Infinity     3.11 & trunk differ, missing port number for 
> BackPressurePerHostWithPort
>  SchemaVersions          {...=[127.0.0.1,...]} {...=[127.0.0.1,...]}  
> {...=[127.0.0.1:7000,...]
>   
>  TokenToEndpointMap      {-92...8=127.0.0.1,   -92...8=127.0.0.1      
> -92..8=127.0.0.1:7000
>  HostIdMap               127.0.0.1=1ee..6f0af  127.0.0.1=e06...7e     MISSING 
>                  Deprecated for EndpointToHostId
>  EndpointToHostId        127.0.0.1=1ee..6f0a   127.0.0.1=e06...7e     
> 127.0.0.1:7000=e0..7e
>  HostIdToEndpoint        1ee..6f0a=127.0.0.1   e06..7e=127.0.0.1      
> e06..7e=127.0.0.1:7000
>  LoadMap                 127.0.0.1=185.01 KiB  127.0.0.1:7000=106.08 KiB  
> 127.0.0.1=106.08 Ki  LoadMap and LoadMapWithPort are flipped.
>  LiveNodes               127.0.0.1             127.0.0.1              
> 127.0.0.1:7000
>  Ownership               /127.0.0.1=0.33   /127.0.0.1=0.33    
> 127.0.0.1:7000=0.33
>  Scores                  /127.0.0.1=0.0        /127.0.0.1=0.0         
> 127.0.0.1:7000=0.0
>   {code}
>  
>  Proposed changes
>   
>  1) AllEndpointStats, SimpleStates, Connection message tracking, 
> TimeoutsPerHost - include the host/ip:port in the WithPort version
> 2) Add port number to BackPressurePerHostWithPort
> 3) Correct LoadMap to omit port / LoadMapWithPort to include port
> 4) Ownership - update with port to host/ip:port version
> 5) Scores - update with port to host/ip:port version
>   
>   
>  Additionally while dumping out all of the JMX info with `sjk mxdump`
>   
> 6) DynamicEndpointSnitch.getScoresWithPort now returns an InetAddressAndPort 
> which should just be a String
> 7) ClientMetrics.clientsByProtocolVersion returns a Guava Immutable map
> 8) StorageService.getIdealConsistencyLevel fails if none set (as we try and 
> call ConsistencyLevel.toString on a null pointer).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15981) jvm-dtests crash on java 11

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-15981:
--
Test and Documentation Plan: multiple attempts with java 11 without issue 
(hit repeatedly without changes)
 Status: Patch Available  (was: In Progress)

> jvm-dtests crash on java 11
> ---
>
> Key: CASSANDRA-15981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15981
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> There is a race condition bug with CMS and class unloading which cause the 
> JVM to crash.  Since jvm-dtests rely on class loaders and unloading, this 
> causes sporadic JVM crashes that look like the following in CI logs
> {code}
> junit.framework.AssertionFailedError: Forked Java VM exited abnormally. 
> Please note the time in the report does not reflect the time until the VM 
> exit.
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.util.Vector.forEach(Vector.java:1387)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.util.Vector.forEach(Vector.java:1387)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16004) When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect this and will use the jars from the older version

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16004:
--
  Fix Version/s: (was: 4.0-beta)
 (was: 3.11.x)
 (was: 3.0.x)
 4.0-beta2
 3.11.8
 3.0.22
  Since Version: 3.0.22
Source Control Link: 
https://github.com/apache/cassandra/commit/8b8ec94744c70b5770c98b22f641ca7f492e0534
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

CI results:

3.0: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra/401/workflows/b5254a01-a0ec-46b6-8f4b-e164bd4145d4
3.11: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra/402/workflows/1e4ecc06-9ee1-42d9-a12f-0572aee136c3
trunk: 
https://app.circleci.com/pipelines/github/dcapwell/cassandra/403/workflows/eba134d3-0725-4590-aa46-4076b2725422

there were failed tests, but not in the upgrade path, so ignored.

> When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect 
> this and will use the jars from the older version
> 
>
> Key: CASSANDRA-16004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16004
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.22, 3.11.8, 4.0-beta2
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/389/workflows/ea7776ac-5be0-4cb4-ab4c-61f524397c07/jobs/2019
> {code}
> build-test:
> [javac] Compiling 510 source files to 
> /home/cassandra/cassandra/build/test/classes
> [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.BenchmarkProcessor' less than -source 
> '1.8'
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/impl/RowUtil.java:49:
>  error: no suitable constructor found for 
> SimpleQueryResult(String[],Object[][],warnings =[...]nings)
> [javac] return new SimpleQueryResult(names, results, warnings 
> == null ? Collections.emptyList() : warnings);
> [javac]^
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][]) is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][],Predicate,int) 
> is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/test/ReplicaFilteringProtectionTest.java:212:
>  error: cannot find symbol
> [javac] List futureWarnings = futureResult.warnings();
> [javac]   ^
> [javac]   symbol:   method warnings()
> [javac]   location: variable futureResult of type SimpleQueryResult
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> [javac] 1 warning
> BUILD FAILED
> /home/cassandra/cassandra/build.xml:1176: Compile failed; see the compiler 
> error output for details.
> {code}
> cassandra-3.0 compiled against the dtest jar provided by cassandra-2.2, so 
> failed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk

[dcapwell]

This is an automated email from the ASF dual-hosted git repository.

dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit e163855c4540d1d4a6d2173bf29fcd0d6bec573f
Merge: 0e65e85 45ac5c5
Author: David Capwell 
AuthorDate: Mon Aug 3 15:16:18 2020 -0700

Merge branch 'cassandra-3.11' into trunk

 .circleci/config-2_1.yml | 14 ++
 .circleci/config.yml | 14 ++
 .circleci/config.yml.HIGHRES | 14 ++
 .circleci/config.yml.LOWRES  | 14 ++
 .circleci/config.yml.MIDRES  | 14 ++
 5 files changed, 50 insertions(+), 20 deletions(-)

diff --cc .circleci/config.yml.MIDRES
index f1e3c8c,000..979ee15
mode 100644,00..100644
--- a/.circleci/config.yml.MIDRES
+++ b/.circleci/config.yml.MIDRES
@@@ -1,2403 -1,0 +1,2409 @@@
 +version: 2
 +jobs:
 +  j8_jvm_upgrade_dtests:
 +docker:
 +- image: 
nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603
 +resource_class: large
 +working_directory: ~/
 +shell: /bin/bash -eo pipefail -l
 +parallelism: 10
 +steps:
 +- attach_workspace:
 +at: /home/cassandra
 +- run:
 +name: Determine distributed Tests to Run
 +command: |
 +  # reminder: this code (along with all the steps) is independently 
executed on every circle container
 +  # so the goal here is to get the circleci script to return the 
tests *this* container will run
 +  # which we do via the `circleci` cli tool.
 +
 +  rm -fr ~/cassandra-dtest/upgrade_tests
 +  echo "***java tests***"
 +
 +  # get all of our unit test filenames
 +  set -eo pipefail && circleci tests glob 
"$HOME/cassandra/test/distributed/**/*.java" > /tmp/all_java_unit_tests.txt
 +
 +  # split up the unit tests into groups based on the number of 
containers we have
 +  set -eo pipefail && circleci tests split --split-by=timings 
--timings-type=filename --index=${CIRCLE_NODE_INDEX} 
--total=${CIRCLE_NODE_TOTAL} /tmp/all_java_unit_tests.txt > 
/tmp/java_tests_${CIRCLE_NODE_INDEX}.txt
 +  set -eo pipefail && cat /tmp/java_tests_${CIRCLE_NODE_INDEX}.txt | 
sed "s;^/home/cassandra/cassandra/test/distributed/;;g" | grep "Test\.java$" | 
grep upgrade > /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
 +  echo "** /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt"
 +  cat /tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt
 +no_output_timeout: 15m
 +- run:
 +name: Log Environment Information
 +command: |
 +  echo '*** id ***'
 +  id
 +  echo '*** cat /proc/cpuinfo ***'
 +  cat /proc/cpuinfo
 +  echo '*** free -m ***'
 +  free -m
 +  echo '*** df -m ***'
 +  df -m
 +  echo '*** ifconfig -a ***'
 +  ifconfig -a
 +  echo '*** uname -a ***'
 +  uname -a
 +  echo '*** mount ***'
 +  mount
 +  echo '*** env ***'
 +  env
 +  echo '*** java ***'
 +  which java
 +  java -version
 +- run:
 +name: Run Unit Tests (testclasslist)
 +command: |
 +  set -x
 +  export PATH=$JAVA_HOME/bin:$PATH
 +  time mv ~/cassandra /tmp
 +  cd /tmp/cassandra
 +  if [ -d ~/dtest_jars ]; then
 +cp ~/dtest_jars/dtest* /tmp/cassandra/build/
 +  fi
 +  test_timeout=$(grep 'name="test.distributed.timeout"' build.xml | 
awk -F'"' '{print $4}' || true)
 +  if [ -z "$test_timeout" ]; then
 +test_timeout=$(grep 'name="test.timeout"' build.xml | awk -F'"' 
'{print $4}')
 +  fi
 +  ant testclasslist -Dtest.timeout="$test_timeout" 
-Dtest.classlistfile=/tmp/java_tests_${CIRCLE_NODE_INDEX}_final.txt  
-Dtest.classlistprefix=distributed
 +no_output_timeout: 15m
 +- store_test_results:
 +path: /tmp/cassandra/build/test/output/
 +- store_artifacts:
 +path: /tmp/cassandra/build/test/output
 +destination: junitxml
 +- store_artifacts:
 +path: /tmp/cassandra/build/test/logs
 +destination: logs
 +environment:
 +- ANT_HOME: /usr/share/ant
 +- LANG: en_US.UTF-8
 +- KEEP_TEST_DIR: true
 +- DEFAULT_DIR: /home/cassandra/cassandra-dtest
 +- PYTHONIOENCODING: utf-8
 +- PYTHONUNBUFFERED: true
 +- CASS_DRIVER_NO_EXTENSIONS: true
 +- CASS_DRIVER_NO_CYTHON: true
 +- CASSANDRA_SKIP_SYNC: true
 +- DTEST_REPO: git://github.com/apache/cassandra-dtest.git
 +- DTEST_BRANCH: master
 +- CCM_MAX_HEAP_SIZE: 1024M
 +- CCM_HEAP_NEWSIZE: 256M
 +- JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
 +- JDK_HOME: /usr/lib/jvm/java-8-openjdk-amd64
 +  j8_cqlsh-dtests-py2-with-vnodes:
 +docker:
 +- image: 
nastra/cassandra-testing-ubuntu1910-java11-w-dependencies:20200603
 +resource_class: large
 +working_directory: ~/
 +

[cassandra] branch cassandra-3.11 updated (8957c08 -> 45ac5c5)

[dcapwell]

This is an automated email from the ASF dual-hosted git repository.

dcapwell pushed a change to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 8957c08  Merge branch 'cassandra-3.0' into cassandra-3.11
 new 8b8ec94  When jvm dtest apis differ Circle CI's dtest_jars_build can 
fail to detect this and will use the jars from the older version
 new 45ac5c5  Merge branch 'cassandra-3.0' into cassandra-3.11

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .circleci/config-2_1.yml | 14 ++
 .circleci/config.yml | 14 ++
 .circleci/config.yml.HIGHRES | 14 ++
 .circleci/config.yml.LOWRES  | 14 ++
 4 files changed, 40 insertions(+), 16 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-3.0 updated: When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect this and will use the jars from the older version

[dcapwell]

This is an automated email from the ASF dual-hosted git repository.

dcapwell pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
 new 8b8ec94  When jvm dtest apis differ Circle CI's dtest_jars_build can 
fail to detect this and will use the jars from the older version
8b8ec94 is described below

commit 8b8ec94744c70b5770c98b22f641ca7f492e0534
Author: David Capwell 
AuthorDate: Mon Aug 3 13:58:42 2020 -0700

When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect 
this and will use the jars from the older version

patch by David Capwell; reviewed by Caleb Rackliffe for CASSANDRA-16004
---
 .circleci/config-2_1.yml | 14 ++
 .circleci/config.yml | 14 ++
 .circleci/config.yml.HIGHRES | 14 ++
 .circleci/config.yml.LOWRES  | 14 ++
 4 files changed, 40 insertions(+), 16 deletions(-)

diff --git a/.circleci/config-2_1.yml b/.circleci/config-2_1.yml
index f2c4f50..d7d48c0 100644
--- a/.circleci/config-2_1.yml
+++ b/.circleci/config-2_1.yml
@@ -318,6 +318,7 @@ commands:
 command: |
   export PATH=$JAVA_HOME/bin:$PATH
   cd ~/cassandra
+  mkdir ~/dtest_jars
   git remote add apache git://github.com/apache/cassandra.git
   for branch in cassandra-2.2 cassandra-3.0 cassandra-3.11 trunk; do
 # check out the correct cassandra version:
@@ -326,9 +327,10 @@ commands:
 git checkout $branch
 # Loop to prevent failure due to maven-ant-tasks not downloading a 
jar..
 for x in $(seq 1 3); do
-${ANT_HOME}/bin/ant clean jar dtest-jar
+${ANT_HOME}/bin/ant realclean; ${ANT_HOME}/bin/ant jar 
dtest-jar
 RETURN="$?"
 if [ "${RETURN}" -eq "0" ]; then
+cp build/dtest*.jar ~/dtest_jars
 break
 fi
 done
@@ -341,14 +343,18 @@ commands:
   # and build the dtest-jar for the branch under test
   git checkout origin/$CIRCLE_BRANCH
   for x in $(seq 1 3); do
-  ${ANT_HOME}/bin/ant clean jar dtest-jar
+  ${ANT_HOME}/bin/ant realclean; ${ANT_HOME}/bin/ant jar dtest-jar
   RETURN="$?"
   if [ "${RETURN}" -eq "0" ]; then
+  cp build/dtest*.jar ~/dtest_jars
   break
   fi
   done
-  mkdir ~/dtest_jars
-  cp build/dtest*.jar ~/dtest_jars
+  # Exit, if we didn't build successfully
+  if [ "${RETURN}" -ne "0" ]; then
+  echo "Build failed with exit code: ${RETURN}"
+  exit ${RETURN}
+  fi
   ls -l ~/dtest_jars
 no_output_timeout: 15m
 
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 3c62b4a..f8526be 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -725,6 +725,7 @@ jobs:
 command: |
   export PATH=$JAVA_HOME/bin:$PATH
   cd ~/cassandra
+  mkdir ~/dtest_jars
   git remote add apache git://github.com/apache/cassandra.git
   for branch in cassandra-2.2 cassandra-3.0 cassandra-3.11 trunk; do
 # check out the correct cassandra version:
@@ -733,9 +734,10 @@ jobs:
 git checkout $branch
 # Loop to prevent failure due to maven-ant-tasks not downloading a 
jar..
 for x in $(seq 1 3); do
-${ANT_HOME}/bin/ant clean jar dtest-jar
+${ANT_HOME}/bin/ant realclean; ${ANT_HOME}/bin/ant jar 
dtest-jar
 RETURN="$?"
 if [ "${RETURN}" -eq "0" ]; then
+cp build/dtest*.jar ~/dtest_jars
 break
 fi
 done
@@ -748,14 +750,18 @@ jobs:
   # and build the dtest-jar for the branch under test
   git checkout origin/$CIRCLE_BRANCH
   for x in $(seq 1 3); do
-  ${ANT_HOME}/bin/ant clean jar dtest-jar
+  ${ANT_HOME}/bin/ant realclean; ${ANT_HOME}/bin/ant jar dtest-jar
   RETURN="$?"
   if [ "${RETURN}" -eq "0" ]; then
+  cp build/dtest*.jar ~/dtest_jars
   break
   fi
   done
-  mkdir ~/dtest_jars
-  cp build/dtest*.jar ~/dtest_jars
+  # Exit, if we didn't build successfully
+  if [ "${RETURN}" -ne "0" ]; then
+  echo "Build failed with exit code: ${RETURN}"
+  exit ${RETURN}
+  fi
   ls -l ~/dtest_jars
 no_output_timeout: 15m
 - persist_to_workspace:
diff --git a/.circleci/config.yml.HIGHRES b/.circleci/config.yml.HIGHRES
index 81802ad..2e40513 100644
--- a/.circleci/config.yml.HIGHRES
+++ b/.circleci/config.yml.HIGHRES
@@ -725,6 +725,7 @@ jobs:
 command: |
   export 

[cassandra] branch trunk updated (0e65e85 -> e163855)

[dcapwell]

This is an automated email from the ASF dual-hosted git repository.

dcapwell pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 0e65e85  Throw FSWriteError upon write failures in order to apply 
DiskFailurePolicy
 new 8b8ec94  When jvm dtest apis differ Circle CI's dtest_jars_build can 
fail to detect this and will use the jars from the older version
 new 45ac5c5  Merge branch 'cassandra-3.0' into cassandra-3.11
 new e163855  Merge branch 'cassandra-3.11' into trunk

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .circleci/config-2_1.yml | 14 ++
 .circleci/config.yml | 14 ++
 .circleci/config.yml.HIGHRES | 14 ++
 .circleci/config.yml.LOWRES  | 14 ++
 .circleci/config.yml.MIDRES  | 14 ++
 5 files changed, 50 insertions(+), 20 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11

[dcapwell]

This is an automated email from the ASF dual-hosted git repository.

dcapwell pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 45ac5c52753b133f208dadb259451c12c67258ea
Merge: 8957c08 8b8ec94
Author: David Capwell 
AuthorDate: Mon Aug 3 15:15:36 2020 -0700

Merge branch 'cassandra-3.0' into cassandra-3.11

 .circleci/config-2_1.yml | 14 ++
 .circleci/config.yml | 14 ++
 .circleci/config.yml.HIGHRES | 14 ++
 .circleci/config.yml.LOWRES  | 14 ++
 4 files changed, 40 insertions(+), 16 deletions(-)



-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16008) 2.2.17 fails to start up with ExceptionInInitializerError

[Tianon Gravi (Jira)]

Tianon Gravi created CASSANDRA-16008:


 Summary: 2.2.17 fails to start up with ExceptionInInitializerError
 Key: CASSANDRA-16008
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16008
 Project: Cassandra
  Issue Type: Bug
Reporter: Tianon Gravi


After the upgrade to 2.2.17, Cassandra fails to start with the following error:
{noformat}
INFO  20:28:57 JVM Arguments: [-Dcom.sun.management.jmxremote.port=7199, 
-Dcom.sun.management.jmxremote.ssl=false, 
-Dcom.sun.management.jmxremote.authenticate=false, -ea, 
-javaagent:/opt/cassandra/lib/jamm-0.3.0.jar, -XX:+CMSClassUnloadingEnabled, 
-XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -Xms128m, -Xmx128m, 
-Xmn32m, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, 
-XX:StringTableSize=103, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, 
-XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, 
-XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, 
-XX:+UseTLAB, -XX:+PerfDisableSharedMem, 
-XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, 
-XX:CMSWaitDuration=1, -XX:+CMSParallelInitialMarkEnabled, 
-XX:+CMSEdenChunksRecordAlways, -XX:CMSWaitDuration=1, -XX:+PrintGCDetails, 
-XX:+PrintGCDateStamps, -XX:+PrintHeapAtGC, -XX:+PrintTenuringDistribution, 
-XX:+PrintGCApplicationStoppedTime, -XX:+PrintPromotionFailure, 
-Xloggc:/opt/cassandra/logs/gc.log, -XX:+UseGCLogFileRotation, 
-XX:NumberOfGCLogFiles=10, -XX:GCLogFileSize=10M, 
-Djava.net.preferIPv4Stack=true, -Dcassandra.jmx.local.port=7199, 
-XX:+DisableExplicitGC, -Djava.library.path=/opt/cassandra/lib/sigar-bin, 
-Dcassandra.libjemalloc=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1, 
-XX:OnOutOfMemoryError=kill -9 %p, -Dlogback.configurationFile=logback.xml, 
-Dcassandra.logdir=/opt/cassandra/logs, 
-Dcassandra.storagedir=/opt/cassandra/data, -Dcassandra-foreground=yes]
WARN  20:28:57 Unable to lock JVM memory (ENOMEM). This can result in part of 
the JVM being swapped out, especially with mmapped I/O enabled. Increase 
RLIMIT_MEMLOCK or run Cassandra as root.
INFO  20:28:57 jemalloc seems to be preloaded from 
/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
INFO  20:28:57 JMX is enabled to receive remote connections on port: 7199
WARN  20:28:57 OpenJDK is not recommended. Please upgrade to the newest Oracle 
Java release
INFO  20:28:57 Initializing SIGAR library
INFO  20:28:57 Checked OS settings and found them configured for optimal 
performance.
WARN  20:28:57 Directory /opt/cassandra/data/commitlog doesn't exist
WARN  20:28:57 Directory /opt/cassandra/data/saved_caches doesn't exist
Exception (java.lang.ExceptionInInitializerError) encountered during startup: 
null
java.lang.ExceptionInInitializerError
at 
org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:709)
at 
org.apache.cassandra.service.StartupChecks$9.execute(StartupChecks.java:351)
at 
org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:109)
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:188)
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:607)
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:717)
Caused by: java.lang.IllegalArgumentException: Bad configuration; unable to 
start server: At least one DataFileDirectory must be specified
at 
org.apache.cassandra.config.DatabaseDescriptor.createAllDirectories(DatabaseDescriptor.java:846)
at org.apache.cassandra.db.Keyspace.(Keyspace.java:66)
... 6 more
ERROR 20:28:58 Exception encountered during startup
java.lang.ExceptionInInitializerError: null
at 
org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:709) 
~[apache-cassandra-2.2.17.jar:2.2.17]
at 
org.apache.cassandra.service.StartupChecks$9.execute(StartupChecks.java:351) 
~[apache-cassandra-2.2.17.jar:2.2.17]
at 
org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:109) 
~[apache-cassandra-2.2.17.jar:2.2.17]
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:188) 
[apache-cassandra-2.2.17.jar:2.2.17]
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:607) 
[apache-cassandra-2.2.17.jar:2.2.17]
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:717) 
[apache-cassandra-2.2.17.jar:2.2.17]
Caused by: java.lang.IllegalArgumentException: Bad configuration; unable to 
start server: At least one DataFileDirectory must be specified
at 
org.apache.cassandra.config.DatabaseDescriptor.createAllDirectories(DatabaseDescriptor.java:846)
 ~[apache-cassandra-2.2.17.jar:2.2.17]
at org.apache.cassandra.db.Keyspace.(Keyspace.java:66) 
~[apache-cassandra-2.2.17.jar:2.2.17]
... 6 common frames omitted
at 

[jira] [Commented] (CASSANDRA-15938) Fix support for adding UDT fields to clustering keys

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170438#comment-17170438
 ] 

Caleb Rackliffe commented on CASSANDRA-15938:
-

[~ifesdjeen] After looking at the 3.0+ patches, the basic fix looks reasonable. 
There are only two things that seem worth mentioning:

- It seems like {{allRemainingComponentsAreNull()}} corrected the previous 
mistake of potentially treating a size of zero as null, but it doesn't look 
like there was a test that covered this case. Not sure how important it is for 
us to add that, but it's worth considering.
- The extra bit of diagnostic information in {{DuplicateRowChecker}} might be 
useful, but I'm not sure exactly how much overhead the extra {{equals()}} check 
will introduce on compaction and read.

+1 otherwise

> Fix support for adding UDT fields to clustering keys
> 
>
> Key: CASSANDRA-15938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15938
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/UDT
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Adding UDT fields to clustering keys is broken in all versions, however 
> slightly differently.
> In 4.0, there will be a brief moment while schema changes are propagated 
> during which we won’t be able to decode and compare byte sequences. 
> Unfortunately, it is unclear what we should do in such cases, since we can’t 
> just come up with a comparator, and we can’t ignore non-null trailing values, 
> since this will lead to cases where compare for tuples `a;1` and `a;2` would 
> return 0, effectively making them equal, and we don’t know how to compare 
> unknown trailing values. Probably we should reject such query since we can’t 
> sort correctly, but we should make the error message more descriptive than 
> just "Index 1 out of bounds for length 1”. The only problem is that we get 
> this exception only on flush right now, so data already propagates to the 
> node by that time.
> In 3.0, the problem is a bit worse than that, since in 3.0 we do not ignore 
> trailing nulls, so some of the values, written before `ALTER TYPE .. ADD` 
> become inaccessible. Both old values, and the new ones should always be 
> accessible.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16004) When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect this and will use the jars from the older version

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170424#comment-17170424
 ] 

David Capwell commented on CASSANDRA-16004:
---

upgrade tests passed, builds only failed because of ant OOM issue (tracked by 
different JIRA).

> When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect 
> this and will use the jars from the older version
> 
>
> Key: CASSANDRA-16004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16004
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/389/workflows/ea7776ac-5be0-4cb4-ab4c-61f524397c07/jobs/2019
> {code}
> build-test:
> [javac] Compiling 510 source files to 
> /home/cassandra/cassandra/build/test/classes
> [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.BenchmarkProcessor' less than -source 
> '1.8'
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/impl/RowUtil.java:49:
>  error: no suitable constructor found for 
> SimpleQueryResult(String[],Object[][],warnings =[...]nings)
> [javac] return new SimpleQueryResult(names, results, warnings 
> == null ? Collections.emptyList() : warnings);
> [javac]^
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][]) is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][],Predicate,int) 
> is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/test/ReplicaFilteringProtectionTest.java:212:
>  error: cannot find symbol
> [javac] List futureWarnings = futureResult.warnings();
> [javac]   ^
> [javac]   symbol:   method warnings()
> [javac]   location: variable futureResult of type SimpleQueryResult
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> [javac] 1 warning
> BUILD FAILED
> /home/cassandra/cassandra/build.xml:1176: Compile failed; see the compiler 
> error output for details.
> {code}
> cassandra-3.0 compiled against the dtest jar provided by cassandra-2.2, so 
> failed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16004) When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect this and will use the jars from the older version

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16004:
--
Status: Ready to Commit  (was: Review In Progress)

> When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect 
> this and will use the jars from the older version
> 
>
> Key: CASSANDRA-16004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16004
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/389/workflows/ea7776ac-5be0-4cb4-ab4c-61f524397c07/jobs/2019
> {code}
> build-test:
> [javac] Compiling 510 source files to 
> /home/cassandra/cassandra/build/test/classes
> [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.BenchmarkProcessor' less than -source 
> '1.8'
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/impl/RowUtil.java:49:
>  error: no suitable constructor found for 
> SimpleQueryResult(String[],Object[][],warnings =[...]nings)
> [javac] return new SimpleQueryResult(names, results, warnings 
> == null ? Collections.emptyList() : warnings);
> [javac]^
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][]) is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][],Predicate,int) 
> is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/test/ReplicaFilteringProtectionTest.java:212:
>  error: cannot find symbol
> [javac] List futureWarnings = futureResult.warnings();
> [javac]   ^
> [javac]   symbol:   method warnings()
> [javac]   location: variable futureResult of type SimpleQueryResult
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> [javac] 1 warning
> BUILD FAILED
> /home/cassandra/cassandra/build.xml:1176: Compile failed; see the compiler 
> error output for details.
> {code}
> cassandra-3.0 compiled against the dtest jar provided by cassandra-2.2, so 
> failed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15981) jvm-dtests crash on java 11

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170423#comment-17170423
 ] 

David Capwell commented on CASSANDRA-15981:
---

so far in testing I see that normal and upgrade tests are stable with the 
change, and I only run with a 4gb container (matches LOWER, see 
https://circleci.com/docs/2.0/configuration-reference/#docker-executor)

> jvm-dtests crash on java 11
> ---
>
> Key: CASSANDRA-15981
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15981
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> There is a race condition bug with CMS and class unloading which cause the 
> JVM to crash.  Since jvm-dtests rely on class loaders and unloading, this 
> causes sporadic JVM crashes that look like the following in CI logs
> {code}
> junit.framework.AssertionFailedError: Forked Java VM exited abnormally. 
> Please note the time in the report does not reflect the time until the VM 
> exit.
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.util.Vector.forEach(Vector.java:1387)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.util.Vector.forEach(Vector.java:1387)
>   at jdk.internal.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15861) Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-15861:

Status: Changes Suggested  (was: Review In Progress)

> Mutating sstable component may race with entire-sstable-streaming(ZCS) 
> causing checksum validation failure
> --
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair, Consistency/Streaming, 
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator - 
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors: 
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081 
> CassandraEntireSSTableStreamReader.java:145 - [Stream 
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream 
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
>   at 
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
>   at 
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
>   at 
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
>   at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
>   at 
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
>   at 
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
>   at 
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
>   at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for 
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>  
> In the above test, it executes "nodetool repair" on node1 and kills node2 
> during repair. At the end, node3 reports checksum validation failure on 
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies 
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be 
> transferred to node3.
> 3. Before node1 actually sends the files to node3, node2 is killed and node1 
> starts to broadcast repair-failure-message to all participants in 
> {{CoordinatorSession#fail}}
> 4. Node1 receives its own repair-failure-message and fails its local repair 
> sessions at {{LocalSessions#failSession}} which triggers async background 
> compaction.
> 5. Node1's background compaction will mutate sstable's repairAt to 0 and 
> pending repair id to null via  
> {{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more 
> in-progress repair.
> 6. Node1 actually sends the sstable to node3 where the sstable's STATS 
> component size is different from the original size recorded in the manifest.
> 7. At the end, node3 reports checksum validation failure when it tries to 
> mutate sstable level and "isTransient" attribute in 
> {{CassandraEntireSSTableStreamReader#read}}.
> {code}
> Currently, entire-sstable-streaming requires sstable components to be 
> immutable, because \{{ComponentManifest}}
> with component sizes are sent before sending actual files. This isn't a 
> problem in legacy streaming as STATS file length didn't matter.
>  
> Ideally it will be great to make sstable STATS metadata immutable, just like 
> 

[jira] [Updated] (CASSANDRA-16004) When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect this and will use the jars from the older version

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-16004:

Reviewers: Caleb Rackliffe, Caleb Rackliffe  (was: Caleb Rackliffe)
   Caleb Rackliffe, Caleb Rackliffe  (was: Caleb Rackliffe)
   Status: Review In Progress  (was: Patch Available)

> When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect 
> this and will use the jars from the older version
> 
>
> Key: CASSANDRA-16004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16004
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/389/workflows/ea7776ac-5be0-4cb4-ab4c-61f524397c07/jobs/2019
> {code}
> build-test:
> [javac] Compiling 510 source files to 
> /home/cassandra/cassandra/build/test/classes
> [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.BenchmarkProcessor' less than -source 
> '1.8'
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/impl/RowUtil.java:49:
>  error: no suitable constructor found for 
> SimpleQueryResult(String[],Object[][],warnings =[...]nings)
> [javac] return new SimpleQueryResult(names, results, warnings 
> == null ? Collections.emptyList() : warnings);
> [javac]^
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][]) is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][],Predicate,int) 
> is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/test/ReplicaFilteringProtectionTest.java:212:
>  error: cannot find symbol
> [javac] List futureWarnings = futureResult.warnings();
> [javac]   ^
> [javac]   symbol:   method warnings()
> [javac]   location: variable futureResult of type SimpleQueryResult
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> [javac] 1 warning
> BUILD FAILED
> /home/cassandra/cassandra/build.xml:1176: Compile failed; see the compiler 
> error output for details.
> {code}
> cassandra-3.0 compiled against the dtest jar provided by cassandra-2.2, so 
> failed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16004) When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect this and will use the jars from the older version

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170395#comment-17170395
 ] 

Caleb Rackliffe commented on CASSANDRA-16004:
-

LGTM

> When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect 
> this and will use the jars from the older version
> 
>
> Key: CASSANDRA-16004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16004
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/389/workflows/ea7776ac-5be0-4cb4-ab4c-61f524397c07/jobs/2019
> {code}
> build-test:
> [javac] Compiling 510 source files to 
> /home/cassandra/cassandra/build/test/classes
> [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.BenchmarkProcessor' less than -source 
> '1.8'
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/impl/RowUtil.java:49:
>  error: no suitable constructor found for 
> SimpleQueryResult(String[],Object[][],warnings =[...]nings)
> [javac] return new SimpleQueryResult(names, results, warnings 
> == null ? Collections.emptyList() : warnings);
> [javac]^
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][]) is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][],Predicate,int) 
> is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/test/ReplicaFilteringProtectionTest.java:212:
>  error: cannot find symbol
> [javac] List futureWarnings = futureResult.warnings();
> [javac]   ^
> [javac]   symbol:   method warnings()
> [javac]   location: variable futureResult of type SimpleQueryResult
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> [javac] 1 warning
> BUILD FAILED
> /home/cassandra/cassandra/build.xml:1176: Compile failed; see the compiler 
> error output for details.
> {code}
> cassandra-3.0 compiled against the dtest jar provided by cassandra-2.2, so 
> failed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16004) When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect this and will use the jars from the older version

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-16004:

Reviewers: Caleb Rackliffe

> When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect 
> this and will use the jars from the older version
> 
>
> Key: CASSANDRA-16004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16004
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/389/workflows/ea7776ac-5be0-4cb4-ab4c-61f524397c07/jobs/2019
> {code}
> build-test:
> [javac] Compiling 510 source files to 
> /home/cassandra/cassandra/build/test/classes
> [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.BenchmarkProcessor' less than -source 
> '1.8'
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/impl/RowUtil.java:49:
>  error: no suitable constructor found for 
> SimpleQueryResult(String[],Object[][],warnings =[...]nings)
> [javac] return new SimpleQueryResult(names, results, warnings 
> == null ? Collections.emptyList() : warnings);
> [javac]^
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][]) is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][],Predicate,int) 
> is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/test/ReplicaFilteringProtectionTest.java:212:
>  error: cannot find symbol
> [javac] List futureWarnings = futureResult.warnings();
> [javac]   ^
> [javac]   symbol:   method warnings()
> [javac]   location: variable futureResult of type SimpleQueryResult
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> [javac] 1 warning
> BUILD FAILED
> /home/cassandra/cassandra/build.xml:1176: Compile failed; see the compiler 
> error output for details.
> {code}
> cassandra-3.0 compiled against the dtest jar provided by cassandra-2.2, so 
> failed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16004) When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect this and will use the jars from the older version

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16004:
--
Test and Documentation Plan: ran upgrade tests in circle ci
 Status: Patch Available  (was: Open)

> When jvm dtest apis differ Circle CI's dtest_jars_build can fail to detect 
> this and will use the jars from the older version
> 
>
> Key: CASSANDRA-16004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16004
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/389/workflows/ea7776ac-5be0-4cb4-ab4c-61f524397c07/jobs/2019
> {code}
> build-test:
> [javac] Compiling 510 source files to 
> /home/cassandra/cassandra/build/test/classes
> [javac] warning: Supported source version 'RELEASE_6' from annotation 
> processor 'org.openjdk.jmh.generators.BenchmarkProcessor' less than -source 
> '1.8'
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/impl/RowUtil.java:49:
>  error: no suitable constructor found for 
> SimpleQueryResult(String[],Object[][],warnings =[...]nings)
> [javac] return new SimpleQueryResult(names, results, warnings 
> == null ? Collections.emptyList() : warnings);
> [javac]^
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][]) is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] constructor 
> SimpleQueryResult.SimpleQueryResult(String[],Object[][],Predicate,int) 
> is not applicable
> [javac]   (actual and formal argument lists differ in length)
> [javac] 
> /home/cassandra/cassandra/test/distributed/org/apache/cassandra/distributed/test/ReplicaFilteringProtectionTest.java:212:
>  error: cannot find symbol
> [javac] List futureWarnings = futureResult.warnings();
> [javac]   ^
> [javac]   symbol:   method warnings()
> [javac]   location: variable futureResult of type SimpleQueryResult
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] Note: Some input files use unchecked or unsafe operations.
> [javac] Note: Recompile with -Xlint:unchecked for details.
> [javac] 2 errors
> [javac] 1 warning
> BUILD FAILED
> /home/cassandra/cassandra/build.xml:1176: Compile failed; see the compiler 
> error output for details.
> {code}
> cassandra-3.0 compiled against the dtest jar provided by cassandra-2.2, so 
> failed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15980) Improve log messages for socket connection/disconnection

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170371#comment-17170371
 ] 

David Capwell commented on CASSANDRA-15980:
---

LGTM +1

> Improve log messages for socket connection/disconnection
> 
>
> Key: CASSANDRA-15980
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15980
> Project: Cassandra
>  Issue Type: Bug
>  Components: Observability/Logging
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Logging for inbound SSL connections can take place before protocol 
> negotiation has taken place and logs a misleading cipher that could cause 
> problems for security auditing.
>   
>   
> {code:java}
> INFO  2020-07-03T13:57:58,380 [Messaging-EventLoop-3-1] 
> org.apache.cassandra.net.InboundConnectionInitiator:242 - connection from 
> peer /1.1.1.1:57899 to /2.2.2.2:7000, protocol = TLSv1.2, cipher suite = 
> SSL_NULL_WITH_NULL_NULL
> {code}
>  
>  Instead Cassandra should log the connection & protocol, then once the cipher 
> has been negotiated log the agreed upon cipher.
>   
>   
>  If the inbound SSL connection does not present a client certificate, 
> Cassandra logs this error, even if the client wasn't required to.
> {code:java}
> ERROR 2020-07-14T11:58:45,925 [Native-Transport-Requests-1] 
> org.apache.cassandra.transport.ServerConnection:140 - Failed to get peer 
> certificates for peer /4.3.2.1:59263
> {code}
>  
>  Logging the absense of verified certificates should be a concern of the 
> SaslNegotiator if it requires it, and not something worth alerting the 
> operator for generally. Downgrade to debug message to make investigation 
> possible if needed.
>   
>   
>  Finally, to help with logging issues related to disconnection, add a log 
> statement when an instance decides it no longer needs to keep a gossip 
> connection open when cleaning up connections in 
> org.apache.cassandra.net.OutboundConnections.UnusedConnectionMonitor#closeUnusedSinceLastRun



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta

[Alan Boudreault (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170358#comment-17170358
 ] 

Alan Boudreault commented on CASSANDRA-15299:
-

FYI, we plan to start working on the python driver implementation soon. Since 
this driver does not have a write coalescer, we want to provide an 
_experimental_ checksumming support that simply wrap every message. We will 
evaluate the performance and the addition of a write coalescer in a second 
iteration.

> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol 
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
>  Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it 
> introduced checksumming/CRC32 to request and response bodies. It’s an 
> important step forward, but it doesn’t cover the entire stream. In 
> particular, the message header is not covered by a checksum or a crc, which 
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of 
> computing the CRC32 on the bytes written on the wire - losing the properties 
> of the CRC32. In some cases, due to this sequencing, attempting to decompress 
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order, 
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for 
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since 
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression 
> operate on individual message bodies rather than frames of multiple complete 
> messages. In reality, this has several important additional downsides. To 
> name a couple:
> # For compression, we are getting poor compression ratios for smaller 
> messages - when operating on tiny sequences of bytes. In reality, for most 
> small requests and responses we are discarding the compressed value as it’d 
> be smaller than the uncompressed one - incurring both redundant allocations 
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages. 
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the 
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we 
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode 
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed, 
> and that we do it before native protocol v5 graduates from beta. See 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
>  and 
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16007) Standardize audit logging and FQL with same options

[Lorina Poland (Jira)]

Lorina Poland created CASSANDRA-16007:
-

 Summary: Standardize audit logging and FQL with same options
 Key: CASSANDRA-16007
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16007
 Project: Cassandra
  Issue Type: Improvement
  Components: Tool/fql, Tool/nodetool
Reporter: Lorina Poland


In Cassandra 4.0:

 

FQL and audit logging use similar `nodetool` enable commands. But FQL uses:

`nodetool enablefullquerylog `

and audit logging uses:

`nodetool enableauditlog` with no `audit_log_dir` as an option.

 

Additionally, FQL uses `log_dir` in the `cassandra.yaml` file and audit logging 
uses `audit_logs_dir` (note the plural `logs`). 

Both of these differences will likely lead to user confusion, no matter how 
well explained in the docs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15977) 4.0 quality testing: Read Repair

[Jira]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres de la Peña updated CASSANDRA-15977:
--
Change Category: Quality Assurance
 Complexity: Normal
 Status: Open  (was: Triage Needed)

> 4.0 quality testing: Read Repair
> 
>
> Key: CASSANDRA-15977
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15977
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest, Test/unit
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> This is a subtask of CASSANDRA-15579 focusing on read repair.
> [This 
> document|https://docs.google.com/document/d/1-gldHcdLSMRbDhhI8ahs_tPeAZsjurjXr38xABVjWHE/edit?usp=sharing]
>  lists and describes the existing functional tests for read repair, so we can 
> have a broad view of what is currently covered. We can comment on this 
> document and add ideas for new cases/tests, so it can gradually evolve to a 
> more or less detailed test plan.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16003) ToolRunner added in CASSANDRA-15942 stdErr checks improvements

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170171#comment-17170171
 ] 

David Capwell commented on CASSANDRA-16003:
---

Overall LGTM +1 (with small comment).

> ToolRunner added in CASSANDRA-15942 stdErr checks improvements
> --
>
> Key: CASSANDRA-16003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16003
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: David Capwell
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> The JVM will output to stderr on some environments to show what flags that 
> were picked up, when this happens all tests which validate stderr start to 
> fail.  This was found in the org.apache.cassandra.tools.ClearSnapshotTest as 
> it switched to use the ToolRunner; below is a sample failure on my laptop (I 
> had to modify the asserts since they don’t include the input)
> {code}
> java.lang.AssertionError: 
> Expecting empty but was:<"Picked up _JAVA_OPTIONS: 
> -Djava.net.preferIPv4Stack=true
> ">
>   at 
> org.apache.cassandra.tools.ToolRunner.assertEmptyStdErr(ToolRunner.java:339)
>   at 
> org.apache.cassandra.tools.ToolRunner.waitAndAssertOnCleanExit(ToolRunner.java:334)
>   at 
> org.apache.cassandra.tools.ClearSnapshotTest.testClearSnapshot_RemoveMultiple(ClearSnapshotTest.java:91)
> {code}
> Here _JAVA_OPTIONS is used globally on my system so fails the test; there is 
> also JAVA_TOOL_RUNNER which is used the same way.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15928) Throw FSWriteError upon write failures in order to apply DiskFailurePolicy

[Aleksey Yeschenko (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15928:
--
  Fix Version/s: 4.0-beta2
  Since Version: 3.0.0
Source Control Link: 
[0e65e85861b84e87691d576ba80c5c3b3b7020d5|https://github.com/apache/cassandra/commit/0e65e85861b84e87691d576ba80c5c3b3b7020d5
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Committed to trunk as 
[0e65e85861b84e87691d576ba80c5c3b3b7020d5|https://github.com/apache/cassandra/commit/0e65e85861b84e87691d576ba80c5c3b3b7020d5],
 thanks.

> Throw FSWriteError upon write failures in order to apply DiskFailurePolicy
> --
>
> Key: CASSANDRA-15928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15928
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Other
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta2
>
> Attachments: intellij_custom_insepction.xml
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Active handling of FSError in AbstractLocalAwareExecutorService was 
> introduced in CASSANDRA-14993. 
>  
> However, in the current code base, there are places that catch the 
> IOException but not wrap it with FSError, hence not triggering the 
> DiskFailurePolicy. Especially when the bad disk no longer permits write 
> operations, it could leads to the mutation stage backup. Therefore I propose 
> to fix the IOException handling in those cases. 
>  
> From the code inspection, those 6 places are current not re-throwing an 
> IOException with FSWriteError.
>  # org.apache.cassandra.triggers.CustomClassLoader#addClassPath throws 
> IOError. Invoked in TriggerExecutor's constructor
>  # org.apache.cassandra.io.util.FileUtils#renameWithConfirm throws 
> RuntimeException
>  # org.apache.cassandra.io.util.FileUtils#truncate throws RuntimeException
>  # org.apache.cassandra.io.util.FileUtils#write throws RuntimeException
>  # org.apache.cassandra.db.compaction.LeveledManifest#sendBackToL0 throws 
> RuntimeException. Invokes rewriteSSTableMetadata
>  # org.apache.cassandra.io.sstable.SSTableHeaderFix#writeNewMetadata throws 
> RuntimeException. Invokes rewriteSSTableMetadata



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15928) Throw FSWriteError upon write failures in order to apply DiskFailurePolicy

[Aleksey Yeschenko (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15928:
--
Status: Ready to Commit  (was: Review In Progress)

> Throw FSWriteError upon write failures in order to apply DiskFailurePolicy
> --
>
> Key: CASSANDRA-15928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15928
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Other
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Attachments: intellij_custom_insepction.xml
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Active handling of FSError in AbstractLocalAwareExecutorService was 
> introduced in CASSANDRA-14993. 
>  
> However, in the current code base, there are places that catch the 
> IOException but not wrap it with FSError, hence not triggering the 
> DiskFailurePolicy. Especially when the bad disk no longer permits write 
> operations, it could leads to the mutation stage backup. Therefore I propose 
> to fix the IOException handling in those cases. 
>  
> From the code inspection, those 6 places are current not re-throwing an 
> IOException with FSWriteError.
>  # org.apache.cassandra.triggers.CustomClassLoader#addClassPath throws 
> IOError. Invoked in TriggerExecutor's constructor
>  # org.apache.cassandra.io.util.FileUtils#renameWithConfirm throws 
> RuntimeException
>  # org.apache.cassandra.io.util.FileUtils#truncate throws RuntimeException
>  # org.apache.cassandra.io.util.FileUtils#write throws RuntimeException
>  # org.apache.cassandra.db.compaction.LeveledManifest#sendBackToL0 throws 
> RuntimeException. Invokes rewriteSSTableMetadata
>  # org.apache.cassandra.io.sstable.SSTableHeaderFix#writeNewMetadata throws 
> RuntimeException. Invokes rewriteSSTableMetadata



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Throw FSWriteError upon write failures in order to apply DiskFailurePolicy

[aleksey]

This is an automated email from the ASF dual-hosted git repository.

aleksey pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 0e65e85  Throw FSWriteError upon write failures in order to apply 
DiskFailurePolicy
0e65e85 is described below

commit 0e65e85861b84e87691d576ba80c5c3b3b7020d5
Author: yifan-c 
AuthorDate: Mon Jul 13 10:16:30 2020 -0700

Throw FSWriteError upon write failures in order to apply DiskFailurePolicy

patch by Yifan Cai; reviewed by Aleksey Yeschenko for CASSANDRA-15928
---
 CHANGES.txt |  1 +
 .../apache/cassandra/io/compress/CompressionMetadata.java   |  6 +-
 .../cassandra/io/sstable/metadata/MetadataSerializer.java   |  8 
 src/java/org/apache/cassandra/io/util/FileUtils.java| 13 +++--
 .../org/apache/cassandra/triggers/CustomClassLoader.java| 10 +-
 5 files changed, 30 insertions(+), 8 deletions(-)

diff --git a/CHANGES.txt b/CHANGES.txt
index e74f330..05ec961 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0-beta2
+ * Throw FSWriteError upon write failures in order to apply DiskFailurePolicy 
(CASSANDRA-15928)
  * Forbid altering UDTs used in partition keys (CASSANDRA-15933)
  * Fix version parsing logic when upgrading from 3.0 (CASSANDRA-15973)
  * Optimize NoSpamLogger use in hot paths (CASSANDRA-15766)
diff --git a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java 
b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
index 3f08fe2..45cf7af 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
@@ -422,9 +422,13 @@ public class CompressionMetadata
 out.flush();
 SyncUtil.sync(fos);
 }
+catch (FileNotFoundException fnfe)
+{
+throw Throwables.propagate(fnfe);
+}
 catch (IOException e)
 {
-throw Throwables.propagate(e);
+throw new FSWriteError(e, filePath);
 }
 }
 
diff --git 
a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java 
b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java
index 0ae7c32..d886338 100644
--- a/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java
+++ b/src/java/org/apache/cassandra/io/sstable/metadata/MetadataSerializer.java
@@ -22,10 +22,13 @@ import java.util.*;
 import java.util.function.Function;
 import java.util.zip.CRC32;
 
+import com.google.common.base.Throwables;
 import com.google.common.collect.Lists;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.io.FSWriteError;
 import org.apache.cassandra.io.sstable.Component;
 import org.apache.cassandra.io.sstable.CorruptSSTableException;
 import org.apache.cassandra.io.sstable.Descriptor;
@@ -255,6 +258,11 @@ public class MetadataSerializer implements 
IMetadataSerializer
 serialize(currentComponents, out, descriptor.version);
 out.flush();
 }
+catch (IOException e)
+{
+Throwables.throwIfInstanceOf(e, FileNotFoundException.class);
+throw new FSWriteError(e, filePath);
+}
 // we cant move a file on top of another file in windows:
 if (FBUtilities.isWindows)
 FileUtils.delete(descriptor.filenameFor(Component.STATS));
diff --git a/src/java/org/apache/cassandra/io/util/FileUtils.java 
b/src/java/org/apache/cassandra/io/util/FileUtils.java
index 7c940c5..c24ce81 100644
--- a/src/java/org/apache/cassandra/io/util/FileUtils.java
+++ b/src/java/org/apache/cassandra/io/util/FileUtils.java
@@ -23,6 +23,7 @@ import java.lang.invoke.MethodHandles;
 import java.lang.reflect.Method;
 import java.nio.ByteBuffer;
 import java.nio.channels.Channels;
+import java.nio.channels.ClosedChannelException;
 import java.nio.channels.FileChannel;
 import java.nio.charset.Charset;
 import java.nio.charset.StandardCharsets;
@@ -280,9 +281,13 @@ public final class FileUtils
 {
 channel.truncate(size);
 }
+catch (NoSuchFileException | FileNotFoundException nfe)
+{
+throw new RuntimeException(nfe);
+}
 catch (IOException e)
 {
-throw new RuntimeException(e);
+throw new FSWriteError(e, path);
 }
 }
 
@@ -730,9 +735,13 @@ public final class FileUtils
 SyncUtil.force(fc, false);
 }
 }
+catch (ClosedChannelException cce)
+{
+throw new RuntimeException(cce);
+}
 catch (IOException ex)
 {
-throw new RuntimeException(ex);
+throw new FSWriteError(ex, file);
 }
 

[jira] [Comment Edited] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169968#comment-17169968
 ] 

Michael Semb Wever edited comment on CASSANDRA-15925 at 8/3/20, 12:09 PM:
--

bq. Even if you switch back to the 'classic' view the stages that fail are not 
being reported. Which is a pity as that's partially how we came to detect this 
ticket in the first place...

Yes, and why I had initially not taken the approach. Unfortunately it looked 
like taking the long-way-around to the same end place :(

I'd suggest we revisit it later on. As stages become reliably SUCCESS (rather 
than UNSTABLE) we'll have more options open to us. The incompatibilities and 
support between Jenkins APIs may improve as well. Otherwise I am in favour of 
keeping things as simple as they can be (i imagine very little joy on an 
elaborate system on top of jenkins).


was (Author: michaelsembwever):
bq. Even if you switch back to the 'classic' view the stages that fail are not 
being reported. Which is a pity as that's partially how we came to detect this 
ticket in the first place...

Yes, and why I had initially not taken the approach. Unfortunately it looked 
like taking the long-way-around to the same end place :(

I'd suggest we revisit it later on. As stages become reliable SUCCESS (rather 
than UNSTABLE) we'll have more options open to us. The incompatibilities and 
support between Jenkins APIs may improve as well. Otherwise I am in favour of 
keeping things as simple as they can be (i imagine very little joy on an 
elaborate system on top of jenkins).

> Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>
> Spotted in 
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169968#comment-17169968
 ] 

Michael Semb Wever commented on CASSANDRA-15925:


bq. Even if you switch back to the 'classic' view the stages that fail are not 
being reported. Which is a pity as that's partially how we came to detect this 
ticket in the first place...

Yes, and why I had initially not taken the approach. Unfortunately it looked 
like taking the long-way-around to the same end place :(

I'd suggest we revisit it later on. As stages become reliable SUCCESS (rather 
than UNSTABLE) we'll have more options open to us. The incompatibilities and 
support between Jenkins APIs may improve as well. Otherwise I am in favour of 
keeping things as simple as they can be (i imagine very little joy on an 
elaborate system on top of jenkins).

> Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>
> Spotted in 
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169913#comment-17169913
 ] 

Berenguer Blasi commented on CASSANDRA-15925:
-

Even if you switch back to the 'classic' view the stages that fail are not 
being reported. Which is a pity as that's partially how we came to detect this 
ticket in the first place...

> Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>
> Spotted in 
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15925:
---
Reviewers: Berenguer Blasi  (was: Berenguer Blasi, Michael Semb Wever)

> Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>
> Spotted in 
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15925:
---
Reviewers: Berenguer Blasi, Michael Semb Wever  (was: Berenguer Blasi)
   Status: Review In Progress  (was: Patch Available)

> Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>
> Spotted in 
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15925:
---
Status: Ready to Commit  (was: Review In Progress)

> Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>
> Spotted in 
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169897#comment-17169897
 ] 

Michael Semb Wever commented on CASSANDRA-15925:


bq. Why are stage failures not being reported now? They used to appear before 
the final total report.

Nice spot! Yes, there's some muffins there they way pipelines, blue ocean, and 
unstable builds are visualised. 
A little bit of background to this is in blog post: 
https://www.jenkins.io/blog/2019/07/05/jenkins-pipeline-stage-result-visualization-improvements/

I might be wrong, but I believe with the patch we are at "as good as it gets" 
with the current Jenkins API and plugins.

> Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>
> Spotted in 
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169881#comment-17169881
 ] 

Berenguer Blasi edited comment on CASSANDRA-15925 at 8/3/20, 10:22 AM:
---

[~mck] LGTM +1. Note: i have focused on trunk, I trust you for the  Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>
> Spotted in 
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15925) Jenkins pipeline can copy wrong test report artefacts from stage builds

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169881#comment-17169881
 ] 

Berenguer Blasi commented on CASSANDRA-15925:
-

[~mck] LGTM +1. Note: i have focused on trunk, I trust you for the  Jenkins pipeline can copy wrong test report artefacts from stage builds
> ---
>
> Key: CASSANDRA-15925
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15925
> Project: Cassandra
>  Issue Type: Bug
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>
> Spotted in 
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/196/console
> Looks like copyArtifact will need to be specific to a build.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16006) Parallelise Jenkins dtests

[Eduard Tudenhoefner (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eduard Tudenhoefner updated CASSANDRA-16006:

Status: Ready to Commit  (was: Review In Progress)

> Parallelise Jenkins dtests
> --
>
> Key: CASSANDRA-16006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16006
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Currently dtests in Jenkins take ~10 hours.
> Using the Jenkins Matrix plugin these jobs can be split into smaller lists of 
> dtests and run in parallel. This is the approach CircleCI takes.
> This approach was 
> [trialed|https://github.com/apache/cassandra-builds/pull/29] with the 
> dtest-upgrade jobs (which are not yet part of the branch pipelines, and 
> haven't previously worked at all due to their duration).
> In addition to the Matrix plugin, the Priority-Sorter and Matrix Reloaded 
> plugins also needed to be added.
> The splits will occupy all executors, and multiple builds will lead to a long 
> build queue. More important builds (artifacts and unit tests) need a way to 
> be prioritised in such saturated situations.
> Splits can fail for silly reasons (false-positive), like 
> [full|https://issues.apache.org/jira/browse/INFRA-19793] /tmp disks, or 
> [connectivity issues|https://issues.apache.org/jira/browse/INFRA-20107] 
> between the donated agent servers. The Matrix Reloaded plugin makes it easy 
> to rebuilt just those failed splits.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16006) Parallelise Jenkins dtests

[Eduard Tudenhoefner (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eduard Tudenhoefner updated CASSANDRA-16006:

Reviewers: Eduard Tudenhoefner, Eduard Tudenhoefner  (was: Eduard 
Tudenhoefner)
   Eduard Tudenhoefner, Eduard Tudenhoefner
   Status: Review In Progress  (was: Patch Available)

changes LGTM

> Parallelise Jenkins dtests
> --
>
> Key: CASSANDRA-16006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16006
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Currently dtests in Jenkins take ~10 hours.
> Using the Jenkins Matrix plugin these jobs can be split into smaller lists of 
> dtests and run in parallel. This is the approach CircleCI takes.
> This approach was 
> [trialed|https://github.com/apache/cassandra-builds/pull/29] with the 
> dtest-upgrade jobs (which are not yet part of the branch pipelines, and 
> haven't previously worked at all due to their duration).
> In addition to the Matrix plugin, the Priority-Sorter and Matrix Reloaded 
> plugins also needed to be added.
> The splits will occupy all executors, and multiple builds will lead to a long 
> build queue. More important builds (artifacts and unit tests) need a way to 
> be prioritised in such saturated situations.
> Splits can fail for silly reasons (false-positive), like 
> [full|https://issues.apache.org/jira/browse/INFRA-19793] /tmp disks, or 
> [connectivity issues|https://issues.apache.org/jira/browse/INFRA-20107] 
> between the donated agent servers. The Matrix Reloaded plugin makes it easy 
> to rebuilt just those failed splits.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16006) Parallelise Jenkins dtests

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169608#comment-17169608
 ] 

Michael Semb Wever edited comment on CASSANDRA-16006 at 8/3/20, 7:55 AM:
-

[Patch|https://github.com/apache/cassandra-builds/compare/master...thelastpickle:mck/jenkins-dtest-parallel]
 ready. Tested locally.

When committing this, all existing dtest (mainline branch dtest-upgrade 
excluded) jobs will first need to be renamed, as the job type is changing.


was (Author: michaelsembwever):
Patch ready. Tested locally.

When committing this, all existing dtest (mainline branch dtest-upgrade 
excluded) jobs will first need to be renamed, as the job type is changing.

> Parallelise Jenkins dtests
> --
>
> Key: CASSANDRA-16006
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16006
> Project: Cassandra
>  Issue Type: Improvement
>  Components: CI
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Currently dtests in Jenkins take ~10 hours.
> Using the Jenkins Matrix plugin these jobs can be split into smaller lists of 
> dtests and run in parallel. This is the approach CircleCI takes.
> This approach was 
> [trialed|https://github.com/apache/cassandra-builds/pull/29] with the 
> dtest-upgrade jobs (which are not yet part of the branch pipelines, and 
> haven't previously worked at all due to their duration).
> In addition to the Matrix plugin, the Priority-Sorter and Matrix Reloaded 
> plugins also needed to be added.
> The splits will occupy all executors, and multiple builds will lead to a long 
> build queue. More important builds (artifacts and unit tests) need a way to 
> be prioritised in such saturated situations.
> Splits can fail for silly reasons (false-positive), like 
> [full|https://issues.apache.org/jira/browse/INFRA-19793] /tmp disks, or 
> [connectivity issues|https://issues.apache.org/jira/browse/INFRA-20107] 
> between the donated agent servers. The Matrix Reloaded plugin makes it easy 
> to rebuilt just those failed splits.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15961) Reference CASSANDRA-12607

[Jeremy Hanna (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremy Hanna updated CASSANDRA-15961:
-
Resolution: Duplicate
Status: Resolved  (was: Triage Needed)

> Reference CASSANDRA-12607
> -
>
> Key: CASSANDRA-15961
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15961
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Kapil Shewate
>Assignee: Mattias W
>Priority: Normal
>
> In cassandra 3.11.0 , the issue of commit logs being corrupted is still 
> observed. Will this be fixed in higher versions of Cassandra?
>  
> 02 19:58:33,677 JVMStabilityInspector.java:82 - Exiting due to error while 
> processing commit log during 
> initialization.org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException:
>  Mutation checksum failure at 191598541 in Next section at 191590263 in 
> CommitLog-6-1592895482005.log at 
> org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:344)
>  [apache-cassandra-3.11.0.jar:3.11.0] at 
> org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:201)
>  [apache-cassandra-3.11.0.jar:3.11.0] at 
> org.apache.cassandra.db.commitlog.CommitLogReader.readAllFiles(CommitLogReader.java:84)
>  [apache-cassandra-3.11.0.jar:3.11.0] at 
> org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:140)
>  [apache-cassandra-3.11.0.jar:3.11.0] at 
> org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:177) 
> [apache-cassandra-3.11.0.jar:3.11.0] at 
> org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:158)
>  [apache-cassandra-3.11.0.jar:3.11.0] at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:325) 
> [apache-cassandra-3.11.0.jar:3.11.0] at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:600)
>  [apache-cassandra-3.11.0.jar:3.11.0] at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:689) 
> [apache-cassandra-3.11.0.jar:3.11.0][WARN] [main] 2020-07-02 20:31:30,334 
> DatabaseDescriptor.java:540 - Only 2.339GiB free across all data volumes. 
> Consider adding more capacity to your cluster or removing obsolete 
> snapshots[WARN] [main] 2020-07-02 20:31:30,763 NativeLibrary.java:187 - 
> Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being 
> swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or 
> run Cassandra as root.[WARN] [main] 2020-07-02 20:31:30,764 
> StartupChecks.java:127 - jemalloc shared library could not be preloaded to 
> speed up memory allocations[WARN] [main] 2020-07-02 20:31:30,764 
> StartupChecks.java:201 - Non-Oracle JVM detected.  Some features, such as 
> immediate unmap of compacted SSTables, may not work as intended[WARN] [main] 
> 2020-07-02 20:31:30,786 SigarLibrary.java:174 - Cassandra server running in 
> degraded mode. Is swap disabled? : false,  Address space adequate? : true,  
> nofile limit adequate? : false, nproc limit adequate? : true [WARN] [main] 
> 2020-07-02 20:31:30,789 StartupChecks.java:265 - Maximum number of memory map 
> areas per process (vm.max_map_count) 65530 is too low, recommended value: 
> 1048575, you can change it with sysctl.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15947) nodetool gossipinfo doc does not document the output

[Jeremy Hanna (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremy Hanna updated CASSANDRA-15947:
-
Complexity: Low Hanging Fruit

> nodetool gossipinfo doc does not document the output
> 
>
> Key: CASSANDRA-15947
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15947
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jens Rantil
>Priority: Low
>
> [https://cassandra.apache.org/doc/latest/tools/nodetool/gossipinfo.html] does 
> not contain any sample output, nor does does it explain what the fields mean.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15947) nodetool gossipinfo doc does not document the output

[Jeremy Hanna (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremy Hanna updated CASSANDRA-15947:
-
Priority: Normal  (was: Low)

> nodetool gossipinfo doc does not document the output
> 
>
> Key: CASSANDRA-15947
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15947
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jens Rantil
>Priority: Normal
>
> [https://cassandra.apache.org/doc/latest/tools/nodetool/gossipinfo.html] does 
> not contain any sample output, nor does does it explain what the fields mean.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15947) nodetool gossipinfo doc does not document the output

[Jeremy Hanna (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeremy Hanna updated CASSANDRA-15947:
-
Priority: Low  (was: Normal)

> nodetool gossipinfo doc does not document the output
> 
>
> Key: CASSANDRA-15947
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15947
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Jens Rantil
>Priority: Low
>
> [https://cassandra.apache.org/doc/latest/tools/nodetool/gossipinfo.html] does 
> not contain any sample output, nor does does it explain what the fields mean.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-14559) Check for endpoint collision with hibernating nodes

[Stefan Miklosovic (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-14559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169758#comment-17169758
 ] 

Stefan Miklosovic commented on CASSANDRA-14559:
---

PR for branch 3.0

[https://github.com/apache/cassandra/pull/707]

tests for branch 3.0

[https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/227/pipeline]

> Check for endpoint collision with hibernating nodes 
> 
>
> Key: CASSANDRA-14559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14559
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Bootstrap and Decommission
>Reporter: Vincent White
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> I ran across an edge case when replacing a node with the same address. This 
> issue results in the node(and its tokens) being unsafely removed from gossip.
> Steps to replicate:
> 1. Create 3 node cluster.
> 2. Stop a node
> 3. Replace the stopped node with a node using the same address using the 
> replace_address flag
> 4. Stop the node before it finishes bootstrapping
> 5. Remove the replace_address flag and restart the node to resume 
> bootstrapping (if the data dir is also cleared at this point the node will 
> also generate new tokens when it starts)
> 6. Stop the node before it finishes bootstrapping again
> 7. 30 Seconds later the node will be removed from gossip because it now 
> matches the check for a FatClient
> I think this is only an issue when replacing a node with the same address 
> because other replacements now use STATUS_BOOTSTRAPPING_REPLACE and leave the 
> dead node unchanged.
> I believe the simplest fix for this is to add a check that prevents a 
> non-bootstrapped node (without the replaces_address flag) starting if there 
> is a gossip entry for the same address in the hibernate state. 
> [3.11 PoC 
> |https://github.com/apache/cassandra/compare/trunk...vincewhite:check_for_hibernate_on_start]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org