[jira] [Commented] (CASSANDRA-15969) jvm dtest execute APIs do not support collections
[ https://issues.apache.org/jira/browse/CASSANDRA-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164153#comment-17164153 ] David Capwell commented on CASSANDRA-15969: --- this was fixed in trunk (via CASSANDRA-15542) so objectToBytes now allows ByteBuffer, but its annoying from the API as you must build the collection type, convert to byte buffer, then send that. Don't have a lot of time and found out I could replicate CASSANDRA-15970 without a collection insert (only need range tombstone), so going to remove myself as the assigned. > jvm dtest execute APIs do not support collections > - > > Key: CASSANDRA-15969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15969 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta > > > If you use a collection type they will be transferred to the instance and we > call org.apache.cassandra.utils.ByteBufferUtil#objectToBytes to convert to > ByteBuffers; this doesn’t support collections. If you try to work around > this by converting before sending, it will fail since that method doesn’t > support ByteBuffer as input -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15969) jvm dtest execute APIs do not support collections
[ https://issues.apache.org/jira/browse/CASSANDRA-15969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell reassigned CASSANDRA-15969: - Assignee: (was: David Capwell) > jvm dtest execute APIs do not support collections > - > > Key: CASSANDRA-15969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15969 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: David Capwell >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta > > > If you use a collection type they will be transferred to the instance and we > call org.apache.cassandra.utils.ByteBufferUtil#objectToBytes to convert to > ByteBuffers; this doesn’t support collections. If you try to work around > this by converting before sending, it will fail since that method doesn’t > support ByteBuffer as input -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15968) nodetool drain fails in jvm dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164092#comment-17164092
]
David Capwell edited comment on CASSANDRA-15968 at 7/24/20, 5:26 AM:
-
CI results
||branch||status||link||
|trunk |(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968]|
|3.11|(!)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11]|
|3.0|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0]|
|2.2|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2]|
3.11 failed with the following
testCapWarnExpirationOverflowPolicy -
org.apache.cassandra.cql3.validation.operations.TTLTest - see CASSANDRA-15959
testIndexMemtableSwitching - org.apache.cassandra.index.sasi.SASIIndexTest -
see CASSANDRA-15881
Actions (Type . to access issue actions)
was (Author: dcapwell):
CI results
||branch||status||link||
|trunk |(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968]|
|3.11|(!)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11]|
|3.0|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0]|
|2.2|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2]|
> nodetool drain fails in jvm dtest
> -
>
> Key: CASSANDRA-15968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15968
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Drain fails since MessagingService doesn’t expect to be shutdown twice.
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:766)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$parallelRun$28(Instance.java:805)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15968) nodetool drain fails in jvm dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164092#comment-17164092
]
David Capwell edited comment on CASSANDRA-15968 at 7/24/20, 5:26 AM:
-
CI results
||branch||status||link||
|trunk |(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968]|
|3.11|(!)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11]|
|3.0|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0]|
|2.2|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2]|
3.11 failed with the following
testCapWarnExpirationOverflowPolicy -
org.apache.cassandra.cql3.validation.operations.TTLTest - see CASSANDRA-15959
testIndexMemtableSwitching - org.apache.cassandra.index.sasi.SASIIndexTest -
see CASSANDRA-15881
was (Author: dcapwell):
CI results
||branch||status||link||
|trunk |(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968]|
|3.11|(!)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11]|
|3.0|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0]|
|2.2|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2]|
3.11 failed with the following
testCapWarnExpirationOverflowPolicy -
org.apache.cassandra.cql3.validation.operations.TTLTest - see CASSANDRA-15959
testIndexMemtableSwitching - org.apache.cassandra.index.sasi.SASIIndexTest -
see CASSANDRA-15881
Actions (Type . to access issue actions)
> nodetool drain fails in jvm dtest
> -
>
> Key: CASSANDRA-15968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15968
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Drain fails since MessagingService doesn’t expect to be shutdown twice.
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:766)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$parallelRun$28(Instance.java:805)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15968) nodetool drain fails in jvm dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15968:
--
Test and Documentation Plan: circle ci + jvm test
Status: Patch Available (was: In Progress)
> nodetool drain fails in jvm dtest
> -
>
> Key: CASSANDRA-15968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15968
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Drain fails since MessagingService doesn’t expect to be shutdown twice.
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:766)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$parallelRun$28(Instance.java:805)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15968) nodetool drain fails in jvm dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164092#comment-17164092
]
David Capwell edited comment on CASSANDRA-15968 at 7/24/20, 5:24 AM:
-
CI results
||branch||status||link||
|trunk |(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968]|
|3.11|(!)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11]|
|3.0|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0]|
|2.2|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2]|
was (Author: dcapwell):
CI results
||branch||status||link||
|trunk |(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968]|
|3.11|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11]|
|3.0|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0]|
|2.2|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2]|
> nodetool drain fails in jvm dtest
> -
>
> Key: CASSANDRA-15968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15968
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Drain fails since MessagingService doesn’t expect to be shutdown twice.
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:766)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$parallelRun$28(Instance.java:805)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15968) nodetool drain fails in jvm dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164092#comment-17164092
]
David Capwell edited comment on CASSANDRA-15968 at 7/24/20, 5:23 AM:
-
CI results
||branch||status||link||
|trunk |(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968]|
|3.11|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11]|
|3.0|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0]|
|2.2|(/)|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2]|
was (Author: dcapwell):
CI results
||branch||status||link||
|trunk |?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968]|
|3.11|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11]|
|3.0|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0]|
|2.2|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2]|
> nodetool drain fails in jvm dtest
> -
>
> Key: CASSANDRA-15968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15968
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Drain fails since MessagingService doesn’t expect to be shutdown twice.
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:766)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$parallelRun$28(Instance.java:805)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15968) nodetool drain fails in jvm dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164092#comment-17164092
]
David Capwell edited comment on CASSANDRA-15968 at 7/24/20, 2:42 AM:
-
CI results
||branch||status||link||
|trunk |?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968]|
|3.11|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11]|
|3.0|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0]|
|2.2|?|[Circle
CI|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2]|
was (Author: dcapwell):
CI results
||branch||status||link||
|trunk
|?|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968|
|3.11|?|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11|
|3.0|?|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0|
|2.2|?|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2|
> nodetool drain fails in jvm dtest
> -
>
> Key: CASSANDRA-15968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15968
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Drain fails since MessagingService doesn’t expect to be shutdown twice.
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:766)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$parallelRun$28(Instance.java:805)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15968) nodetool drain fails in jvm dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164092#comment-17164092
]
David Capwell commented on CASSANDRA-15968:
---
CI results
||branch||status||link||
|trunk
|?|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968|
|3.11|?|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.11|
|3.0|?|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_3.0|
|2.2|?|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-15968_2.2|
> nodetool drain fails in jvm dtest
> -
>
> Key: CASSANDRA-15968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15968
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Drain fails since MessagingService doesn’t expect to be shutdown twice.
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:766)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$parallelRun$28(Instance.java:805)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15665) StreamManager should clearly differentiate between "initiator" and "receiver" sessions
[
https://issues.apache.org/jira/browse/CASSANDRA-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164075#comment-17164075
]
ZhaoYang edited comment on CASSANDRA-15665 at 7/24/20, 1:57 AM:
[~maedhroz] does it fail anything? I think we don't allow cross-version
streaming between 3.x and 4.0..It's guarded by version when establishing
connections.
was (Author: jasonstack):
[~maedhroz] does it fail anything? I think we don't allow cross-version
streaming between 3.x and 4.0..
> StreamManager should clearly differentiate between "initiator" and "receiver"
> sessions
> --
>
> Key: CASSANDRA-15665
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15665
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta1
>
>
> {{StreamManager}} does currently a suboptimal job in differentiating between
> stream sessions (in form of {{StreamResultFuture}}) which have been either
> initiated or "received", for the following reasons:
> 1) Naming is IMO confusing: a "receiver" session could actually both send and
> receive files, so technically an initiator is also a receiver.
> 2) {{StreamManager#findSession()}} assumes we should first looking into
> "initiator" sessions, then into "receiver" ones: this is a dangerous
> assumptions, in particular for test environments where the same process could
> work as both an initiator and a receiver.
> I would recommend the following changes:
> 1) Rename "receiver" with "follower" everywhere the former is used.
> 2) Introduce a new flag into {{StreamMessageHeader}} to signal if the message
> comes from an initiator or follower session, in order to correctly
> differentiate and look for sessions in {{StreamManager}}.
> While my arguments above might seem trivial, I believe they will improve
> clarity and save from potential bugs/headaches at testing time, and doing
> such changes now that we're revamping streaming for 4.0 seems the right time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15665) StreamManager should clearly differentiate between "initiator" and "receiver" sessions
[
https://issues.apache.org/jira/browse/CASSANDRA-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164075#comment-17164075
]
ZhaoYang commented on CASSANDRA-15665:
--
[~maedhroz] does it fail anything? I think we don't allow cross-version
streaming between 3.x and 4.0..
> StreamManager should clearly differentiate between "initiator" and "receiver"
> sessions
> --
>
> Key: CASSANDRA-15665
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15665
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta1
>
>
> {{StreamManager}} does currently a suboptimal job in differentiating between
> stream sessions (in form of {{StreamResultFuture}}) which have been either
> initiated or "received", for the following reasons:
> 1) Naming is IMO confusing: a "receiver" session could actually both send and
> receive files, so technically an initiator is also a receiver.
> 2) {{StreamManager#findSession()}} assumes we should first looking into
> "initiator" sessions, then into "receiver" ones: this is a dangerous
> assumptions, in particular for test environments where the same process could
> work as both an initiator and a receiver.
> I would recommend the following changes:
> 1) Rename "receiver" with "follower" everywhere the former is used.
> 2) Introduce a new flag into {{StreamMessageHeader}} to signal if the message
> comes from an initiator or follower session, in order to correctly
> differentiate and look for sessions in {{StreamManager}}.
> While my arguments above might seem trivial, I believe they will improve
> clarity and save from potential bugs/headaches at testing time, and doing
> such changes now that we're revamping streaming for 4.0 seems the right time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15966) Circle CI start_ jobs should not require build
[ https://issues.apache.org/jira/browse/CASSANDRA-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15966: -- Fix Version/s: (was: 4.0-rc) 4.0-beta2 Source Control Link: https://github.com/apache/cassandra/commit/b619d21ed2b0de72da11daad7ce3938a6b348dd4 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Circle CI start_ jobs should not require build > -- > > Key: CASSANDRA-15966 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15966 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: 4.0, 4.0-beta2 > > > The Circle CI start_ jobs all depend on build, so can’t approve them until > the build is done, but their downstream also depends on build. Given this, > we can remove this dependency which will allow someone to approve the > downstream pipelines even if the build isn’t complete yet. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Circle CI approval jobs should not require build to run first
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new b619d21 Circle CI approval jobs should not require build to run first b619d21 is described below commit b619d21ed2b0de72da11daad7ce3938a6b348dd4 Author: David Capwell AuthorDate: Thu Jul 23 17:41:20 2020 -0700 Circle CI approval jobs should not require build to run first patch by David Capwell; reviewed by Ekaterina Dimitrova for CASSANDRA-15966 --- .circleci/config-2_1.yml | 63 ++-- .circleci/config.yml | 63 ++-- .circleci/config.yml.HIGHRES | 63 ++-- .circleci/config.yml.LOWRES | 63 ++-- .circleci/config.yml.MIDRES | 63 ++-- .circleci/generate.sh| 7 + .circleci/generate_midres.sh | 8 -- .circleci/readme.md | 4 +-- 8 files changed, 164 insertions(+), 170 deletions(-) diff --git a/.circleci/config-2_1.yml b/.circleci/config-2_1.yml index ab62124..68b83b7 100644 --- a/.circleci/config-2_1.yml +++ b/.circleci/config-2_1.yml @@ -65,40 +65,35 @@ j8_with_dtests_jobs: _with_dtests_jobs # Java 11 unit tests (on request, currently not working) - start_j11_unit_tests: type: approval -requires: - - j8_build - j11_unit_tests: requires: - start_j11_unit_tests + - j8_build # specialized unit tests (all run on request using Java 8) - start_utests_long: type: approval -requires: - - j8_build - utests_long: requires: - start_utests_long + - j8_build - start_utests_compression: type: approval -requires: - - j8_build - utests_compression: requires: - start_utests_compression + - j8_build - start_utests_stress: type: approval -requires: - - j8_build - utests_stress: requires: - start_utests_stress + - j8_build - start_utests_fqltool: type: approval -requires: - - j8_build - utests_fqltool: requires: - start_utests_fqltool + - j8_build - start_jvm_upgrade_dtest: type: approval - j8_dtest_jars_build: @@ -111,85 +106,88 @@ j8_with_dtests_jobs: _with_dtests_jobs # Java 8 dtests (on request) - start_j8_dtests: type: approval -requires: - - j8_build - j8_dtests-with-vnodes: requires: - start_j8_dtests + - j8_build - j8_dtests-no-vnodes: requires: - start_j8_dtests + - j8_build # Java 11 dtests (on request) - start_j11_dtests: type: approval -requires: - - j8_build - j11_dtests-with-vnodes: requires: - start_j11_dtests +- j8_build - j11_dtests-no-vnodes: requires: - start_j11_dtests +- j8_build # Java 8 upgrade tests - start_upgrade_tests: type: approval -requires: - - j8_build - j8_upgradetests-no-vnodes: requires: - start_upgrade_tests + - j8_build - start_j8_cqlsh_tests-with-vnodes: type: approval -requires: -- j8_build - j8_cqlsh-dtests-py2-with-vnodes: requires: - start_j8_cqlsh_tests-with-vnodes +- j8_build - j8_cqlsh-dtests-py3-with-vnodes: requires: - start_j8_cqlsh_tests-with-vnodes +- j8_build - j8_cqlsh-dtests-py38-with-vnodes: requires: - start_j8_cqlsh_tests-with-vnodes +- j8_build - start_j8_cqlsh_tests-no-vnodes: type: approval -requires: -- j8_build - j8_cqlsh-dtests-py2-no-vnodes: requires: - start_j8_cqlsh_tests-no-vnodes +- j8_build - j8_cqlsh-dtests-py3-no-vnodes: requires: - start_j8_cqlsh_tests-no-vnodes +- j8_build - j8_cqlsh-dtests-py38-no-vnodes: requires: - start_j8_cqlsh_tests-no-vnodes + - j8_build - start_j11_cqlsh_tests-with-vnodes: type: approval -requires: -- j8_build - j11_cqlsh-dtests-py2-with-vnodes: requires: - start_j11_cqlsh_tests-with-vnodes +- j8_build - j11_cqlsh-dtests-py3-with-vnodes: requires: - start_j11_cqlsh_tests-with-vnodes +- j8_build - j11_cqlsh-dtests-py38-with-vnodes: requires: - start_j11_cqlsh_tests-with-vnodes + - j8_build - start_j11_cqlsh_tests-no-vnodes: type:
[jira] [Updated] (CASSANDRA-15975) SEP.shutdownAndWait does not wait for termination and SEPExecutor.awaitTermination may hang
[ https://issues.apache.org/jira/browse/CASSANDRA-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Meredith updated CASSANDRA-15975: - Bug Category: Parent values: Code(13163) Complexity: Normal Discovered By: Unit Test Severity: Low Status: Open (was: Triage Needed) > SEP.shutdownAndWait does not wait for termination and > SEPExecutor.awaitTermination may hang > --- > > Key: CASSANDRA-15975 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15975 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > SharedExecutorPool.shutdownAndWait calls shutdownNow on the executors it > owns, which has the side effect of removing the executor from the list, then > uses the same emptied list to try and wait for them to complete shutting > down, which completes immediately. > > Once fixed by copying the list in SEP.shutdownAndWait, the SEPExecutorTest > suite fails with a timeout due to a startup race in SEPWorker. > > If a SEPExecutor is shutdown immediately after maybeSchedule creates a new > SEPWorker, > but before it is able to execute the first task then it exits immediately on > entering the work loop with a work and task permit reserved for it, which > invalidates the work/task permit accounting and prevents calling the shutdown > SimpleCondition.signalAll when the last SEPWorker exits. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15975) SEP.shutdownAndWait does not wait for termination and SEPExecutor.awaitTermination may hang
[ https://issues.apache.org/jira/browse/CASSANDRA-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Meredith reassigned CASSANDRA-15975: Assignee: Jon Meredith > SEP.shutdownAndWait does not wait for termination and > SEPExecutor.awaitTermination may hang > --- > > Key: CASSANDRA-15975 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15975 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > > SharedExecutorPool.shutdownAndWait calls shutdownNow on the executors it > owns, which has the side effect of removing the executor from the list, then > uses the same emptied list to try and wait for them to complete shutting > down, which completes immediately. > > Once fixed by copying the list in SEP.shutdownAndWait, the SEPExecutorTest > suite fails with a timeout due to a startup race in SEPWorker. > > If a SEPExecutor is shutdown immediately after maybeSchedule creates a new > SEPWorker, > but before it is able to execute the first task then it exits immediately on > entering the work loop with a work and task permit reserved for it, which > invalidates the work/task permit accounting and prevents calling the shutdown > SimpleCondition.signalAll when the last SEPWorker exits. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15966) Circle CI start_ jobs should not require build
[ https://issues.apache.org/jira/browse/CASSANDRA-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164058#comment-17164058 ] David Capwell commented on CASSANDRA-15966: --- updated CI link to show that it works with latest trunk; enabled the dtest builds as well. > Circle CI start_ jobs should not require build > -- > > Key: CASSANDRA-15966 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15966 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: 4.0, 4.0-rc > > > The Circle CI start_ jobs all depend on build, so can’t approve them until > the build is done, but their downstream also depends on build. Given this, > we can remove this dependency which will allow someone to approve the > downstream pipelines even if the build isn’t complete yet. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15975) SEP.shutdownAndWait does not wait for termination and SEPExecutor.awaitTermination may hang
Jon Meredith created CASSANDRA-15975: Summary: SEP.shutdownAndWait does not wait for termination and SEPExecutor.awaitTermination may hang Key: CASSANDRA-15975 URL: https://issues.apache.org/jira/browse/CASSANDRA-15975 Project: Cassandra Issue Type: Bug Components: Local/Other Reporter: Jon Meredith SharedExecutorPool.shutdownAndWait calls shutdownNow on the executors it owns, which has the side effect of removing the executor from the list, then uses the same emptied list to try and wait for them to complete shutting down, which completes immediately. Once fixed by copying the list in SEP.shutdownAndWait, the SEPExecutorTest suite fails with a timeout due to a startup race in SEPWorker. If a SEPExecutor is shutdown immediately after maybeSchedule creates a new SEPWorker, but before it is able to execute the first task then it exits immediately on entering the work loop with a work and task permit reserved for it, which invalidates the work/task permit accounting and prevents calling the shutdown SimpleCondition.signalAll when the last SEPWorker exits. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15191) stop_paranoid disk failure policy is ignored on CorruptSSTableException after node is up
[
https://issues.apache.org/jira/browse/CASSANDRA-15191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164051#comment-17164051
]
David Capwell commented on CASSANDRA-15191:
---
CI 3.11 -
https://app.circleci.com/pipelines/github/dcapwell/cassandra/309/workflows/b4cbed8d-868f-4640-a697-471fa03fd4bf
CI trunk -
https://app.circleci.com/pipelines/github/dcapwell/cassandra/310/workflows/62969c9b-9c65-4558-9ec0-3fcc3f17d79e
Looks like this patch doesn't play nicely with commit log, this breaks the
following tests
commitlog_test.py
- test_ignore_failure_policy
- test_stop_commit_failure_policy
Here is the log from the ignore policy test
https://1573-209217594-gh.circle-artifacts.com/62/dtest_j8_without_vnodes_logs/1595547611103_test_ignore_failure_policy/node1.log
sample that stands out
{code}
ERROR [COMMIT-LOG-ALLOCATOR] 2020-07-23 23:40:08,735 CommitLog.java:499 -
Failed managing commit log segments
org.apache.cassandra.io.FSWriteError: java.nio.file.AccessDeniedException:
/tmp/dtest-zt17lw0m/test/node1/commitlogs/CommitLog-7-1595547598804.log
at
org.apache.cassandra.db.commitlog.CommitLogSegment.(CommitLogSegment.java:180)
at
org.apache.cassandra.db.commitlog.MemoryMappedSegment.(MemoryMappedSegment.java:45)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.createSegment(CommitLogSegment.java:137)
at
org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.createSegment(CommitLogSegmentManagerStandard.java:66)
at
org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager$1.runMayThrow(AbstractCommitLogSegmentManager.java:114)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.AccessDeniedException:
/tmp/dtest-zt17lw0m/test/node1/commitlogs/CommitLog-7-1595547598804.log
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at
sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
at java.nio.channels.FileChannel.open(FileChannel.java:287)
at java.nio.channels.FileChannel.open(FileChannel.java:335)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.(CommitLogSegment.java:175)
... 7 common frames omitted
ERROR [COMMIT-LOG-ALLOCATOR] 2020-07-23 23:40:09,736
DefaultFSErrorHandler.java:66 - Stopping transports as disk_failure_policy is
stop
{code}
Looks like the commit policy isn't respected and instead we fall back to the
normal disk policy.
[~stefan.miklosovic] can you look into this?
> stop_paranoid disk failure policy is ignored on CorruptSSTableException after
> node is up
>
>
> Key: CASSANDRA-15191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15191
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Vincent White
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
> Attachments: log.txt
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> There is a bug when disk_failure_policy is set to stop_paranoid and
> CorruptSSTableException is thrown after server is up. The problem is that
> this setting is ignored. Normally, it should stop gossip and transport but it
> just continues to serve requests and an exception is just logged.
>
> This patch unifies the exception handling in JVMStabilityInspector and code
> is reworked in such way that this inspector acts as a central place where
> such exceptions are inspected.
>
> The core reason for ignoring that exception is that thrown exception in
> AbstractLocalAwareExecturorService is not CorruptSSTableException but it is
> RuntimeException and that exception is as its cause. Hence it is better if we
> handle this in JVMStabilityInspector which can recursively examine it, hence
> act accordingly.
> Behaviour before:
> stop_paranoid of disk_failure_policy is ignored when CorruptSSTableException
> is thrown, e.g. on a regular select statement
> Behaviour after:
> Gossip and transport (cql) is turned off, JVM is still up for further
> investigation e.g. by jmx.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15906) Queries on KEYS 2i are broken by DROP COMPACT STORAGE on 3.0
[
https://issues.apache.org/jira/browse/CASSANDRA-15906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15906:
-
Fix Version/s: (was: 3.0.x)
4.0-beta1
3.11.7
3.0.21
Since Version: 3.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/64c80f4ef89f0cc88c15febed8c01eb07ae0a84e
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Yes, though not by me. I've updated the ticket to reflect the correct versions.
> Queries on KEYS 2i are broken by DROP COMPACT STORAGE on 3.0
>
>
> Key: CASSANDRA-15906
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15906
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 3.0.21, 3.11.7, 4.0-beta1
>
>
> From 3.0 onwards, the declared columns of a thrift table are internally
> static columns. While the table is compact, this
> After DROP COMPACT STORAGE is used on a table that has a KEYS 2i, queries
> that uses that index will start failing with:
> {noformat}
> Queries using 2ndary indexes don't support selecting only static columns
> {noformat}
> In 3.0, we don't support index on static columns and have that validation
> that rejects 2i queries on static columns. But the declared columns of
> compact table are static under the hood, and while this specific validation
> is skipped while the table is compact, it isn't anymore after the DROP
> COMPACT STORAGE.
> Note that internally, nothing changes with the DROP COMPACT STORAGE, and the
> 2i queries would still work as well as before, it is just that they are
> rejected.
> Also not that this is only a problem in 3.0. In 3.11, static column indexes
> were added (CASSANDRA-8103) and thus this validation has been removed, and
> everything works as it should.
> However, since DROP COMPACT STORAGE is a mandatory step for compact tables
> before upgrading to 4.0, fixing this annoying in 3.0 would avoid forcing
> users with KEYS 2i on 3.0 to upgrade to 3.11 before going to 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15906) Queries on KEYS 2i are broken by DROP COMPACT STORAGE on 3.0
[
https://issues.apache.org/jira/browse/CASSANDRA-15906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164020#comment-17164020
]
Caleb Rackliffe commented on CASSANDRA-15906:
-
[~brandon.williams] This was committed and merged up through trunk, right?
> Queries on KEYS 2i are broken by DROP COMPACT STORAGE on 3.0
>
>
> Key: CASSANDRA-15906
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15906
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Normal
> Fix For: 3.0.x
>
>
> From 3.0 onwards, the declared columns of a thrift table are internally
> static columns. While the table is compact, this
> After DROP COMPACT STORAGE is used on a table that has a KEYS 2i, queries
> that uses that index will start failing with:
> {noformat}
> Queries using 2ndary indexes don't support selecting only static columns
> {noformat}
> In 3.0, we don't support index on static columns and have that validation
> that rejects 2i queries on static columns. But the declared columns of
> compact table are static under the hood, and while this specific validation
> is skipped while the table is compact, it isn't anymore after the DROP
> COMPACT STORAGE.
> Note that internally, nothing changes with the DROP COMPACT STORAGE, and the
> 2i queries would still work as well as before, it is just that they are
> rejected.
> Also not that this is only a problem in 3.0. In 3.11, static column indexes
> were added (CASSANDRA-8103) and thus this validation has been removed, and
> everything works as it should.
> However, since DROP COMPACT STORAGE is a mandatory step for compact tables
> before upgrading to 4.0, fixing this annoying in 3.0 would avoid forcing
> users with KEYS 2i on 3.0 to upgrade to 3.11 before going to 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164015#comment-17164015
]
Ekaterina Dimitrova commented on CASSANDRA-15693:
-
Thanks [~brandon.williams] :)
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta2
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: In Jenkins use `propagate: false` to ensure copyArtifact always works with a valid build number
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new 0f1e7c2 In Jenkins use `propagate: false` to ensure copyArtifact
always works with a valid build number
0f1e7c2 is described below
commit 0f1e7c27b950cdba7692f9313917c621324c355d
Author: mck
AuthorDate: Mon Jul 20 22:00:12 2020 +0200
In Jenkins use `propagate: false` to ensure copyArtifact always works with
a valid build number
---
jenkins-dsl/cassandra_pipeline.groovy | 123 +-
1 file changed, 33 insertions(+), 90 deletions(-)
diff --git a/jenkins-dsl/cassandra_pipeline.groovy
b/jenkins-dsl/cassandra_pipeline.groovy
index b13b818..348c77e 100644
--- a/jenkins-dsl/cassandra_pipeline.groovy
+++ b/jenkins-dsl/cassandra_pipeline.groovy
@@ -1,5 +1,8 @@
// Cassandra-devbranch needs custom Jenkinsfile because of the parameters
passed into the build jobs.
//
+// When updating this file you will need to go to
https://ci-cassandra.apache.org/scriptApproval/
+// and approve the change.
+//
// Validate/lint this file using the following command
// `curl -X POST -F "jenkinsfile=https://ci-cassandra.apache.org/pipeline-model-converter/validate`
@@ -27,19 +30,13 @@ pipeline {
steps {
warnError('Tests unstable') {
script {
-stress = build job: "${env.JOB_NAME}-stress-test",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)]
+stress = build job: "${env.JOB_NAME}-stress-test",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)], propagate: false
+if (stress.result != 'SUCCESS') unstable('stress test
failures')
}
}
}
post {
-success {
-warnError('missing test xml files') {
-script {
-copyTestResults('stress-test', stress.getNumber())
-}
-}
-}
-unstable {
+always {
warnError('missing test xml files') {
script {
copyTestResults('stress-test', stress.getNumber())
@@ -52,19 +49,13 @@ pipeline {
steps {
warnError('Tests unstable') {
script {
-fqltool = build job: "${env.JOB_NAME}-fqltool-test",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)]
+fqltool = build job: "${env.JOB_NAME}-fqltool-test",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)], propagate: false
+if (fqltool.result != 'SUCCESS') unstable('fqltool
test failures')
}
}
}
post {
-success {
-warnError('missing test xml files') {
-script {
-copyTestResults('fqltool-test',
fqltool.getNumber())
-}
-}
-}
-unstable {
+always {
warnError('missing test xml files') {
script {
copyTestResults('fqltool-test',
fqltool.getNumber())
@@ -77,19 +68,13 @@ pipeline {
steps {
warnError('Tests unstable') {
script {
-jvm_dtest = build job: "${env.JOB_NAME}-jvm-dtest",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)]
+jvm_dtest = build job: "${env.JOB_NAME}-jvm-dtest",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)], propagate: false
+if (jvm_dtest.result != 'SUCCESS') unstable('jvm-dtest
failures')
}
}
}
post {
-success {
-warnError('missing test xml files') {
-script {
-copyTestResults('jvm-dtest', jvm_dtest.getNumber())
-}
-}
-}
-unstable {
+always {
warnError('missing test xml files') {
script {
copyTestResults('jvm-dtest', jvm_dtest.getNumber())
@@ -102,19 +87,13 @@ pipeline {
steps {
[jira] [Commented] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164002#comment-17164002
]
Brandon Williams commented on CASSANDRA-15693:
--
Committed before I saw your branch, but I elected not to add this to
CHANGES.txt anyway.
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta2
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15693:
-
Fix Version/s: (was: 4.0-beta)
4.0-beta2
Since Version: 4.0-alpha1
Source Control Link:
https://github.com/apache/cassandra/commit/a47be7eddd5855fc7723d4080ca1a63c611efdab
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta2
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15693:
-
Reviewers: Brandon Williams, Ekaterina Dimitrova (was: Ekaterina Dimitrova)
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Fix gen-doc for python 3
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new a47be7e Fix gen-doc for python 3 a47be7e is described below commit a47be7eddd5855fc7723d4080ca1a63c611efdab Author: Angelo Polo AuthorDate: Sat Apr 4 11:51:44 2020 +0200 Fix gen-doc for python 3 Patch by Angelo Polo, reviewed by Ekaterina Dimitrova and brandonwilliams for CASSANDRA-15693 --- doc/gen-nodetool-docs.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/gen-nodetool-docs.py b/doc/gen-nodetool-docs.py index cc784c2..67e56c1 100644 --- a/doc/gen-nodetool-docs.py +++ b/doc/gen-nodetool-docs.py @@ -57,7 +57,7 @@ def create_rst(command): cmdName = command.group(0).strip() cmdFilename = outdir + "/" + cmdName + ".txt" rstFilename = outdir + "/" + cmdName + ".rst" -with open(cmdFilename, "w+") as cmdFile: +with open(cmdFilename, "w+b") as cmdFile: proc = Popen([nodetool, "help", cmdName], stdin=PIPE, stdout=PIPE) (out, err) = proc.communicate() cmdFile.write(out) @@ -76,7 +76,7 @@ with open(outdir + "/nodetool.rst", "w+") as output: output.write(command) # create the command usage pages -with open(helpfilename, "rw+") as helpfile: +with open(helpfilename, "r+") as helpfile: for commandLine in helpfile: command = command_re.match(commandLine) create_rst(command) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164001#comment-17164001
]
Ekaterina Dimitrova edited comment on CASSANDRA-15693 at 7/23/20, 10:18 PM:
Pushed a
[branch|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15693]
with updated CHANGES.txt
was (Author: e.dimitrova):
Pushed a
[branch|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15693]
with CHANGES.txt
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164001#comment-17164001
]
Ekaterina Dimitrova commented on CASSANDRA-15693:
-
Pushed a
[branch|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15693]
with CHANGES.txt
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Increment version to 4.0-beta2
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 39c8ec2 Increment version to 4.0-beta2 39c8ec2 is described below commit 39c8ec2cf6cf66cbe371f154208d3074777170d6 Author: Mick Semb Wever AuthorDate: Fri Jul 24 00:06:21 2020 +0200 Increment version to 4.0-beta2 --- build.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build.xml b/build.xml index 89c071e..f0948ae 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + https://gitbox.apache.org/repos/asf/cassandra.git"/> https://gitbox.apache.org/repos/asf/cassandra.git"/> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=tree"/> - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15693:
Status: Ready to Commit (was: Review In Progress)
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163999#comment-17163999
]
Ekaterina Dimitrova commented on CASSANDRA-15693:
-
Confirmed with [~lor...@datastax.com] offline, this is ready for commit.
[~brandon.williams], please assist :)
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163994#comment-17163994
]
Ekaterina Dimitrova commented on CASSANDRA-15693:
-
Tested on my Mac with Python 3.7.7, LGTM +1. I have only one question, I am not
familiar with sphinx.
The following appears when I run ant with target "gen-doc" even if the build is
successful:
{code:java}
gen-doc:
[exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
source/configuration/cassandra_config_file.rst
[exec] python gen-nodetool-docs.py
[exec] sphinx-build -b html -d build/doctrees source build/html
[exec] make: sphinx-build: No such file or directory
[exec] make: *** [html] Error 1
[exec] Result: 2
BUILD SUCCESSFUL
Total time: 1 minute 31 seconds
{code}
[~lor...@datastax.com], [~brandon.williams] any suggestions?
Missing file but the build is successful?
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15665) StreamManager should clearly differentiate between "initiator" and "receiver" sessions
[
https://issues.apache.org/jira/browse/CASSANDRA-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163992#comment-17163992
]
Caleb Rackliffe edited comment on CASSANDRA-15665 at 7/23/20, 9:59 PM:
---
[~jasonstack] Did we introduce a change
[here|https://github.com/apache/cassandra/commit/ebba613b0b34ea338eed508a3ba6cbb235986fd9#diff-85ee7eed2fccf1f9220452e69fbf8ca8R131]
that needs to be guarded by a {{VERSION_40}} check? (nit: I'd also call it
{{sentByFollower}}.)
was (Author: maedhroz):
[~jasonstack] Did we introduce a change
[here|https://github.com/apache/cassandra/commit/ebba613b0b34ea338eed508a3ba6cbb235986fd9#diff-85ee7eed2fccf1f9220452e69fbf8ca8R115]
that needs to be guarded by a {{VERSION_40}} check? (nit: I'd also call it
{{sentByFollower}}.)
> StreamManager should clearly differentiate between "initiator" and "receiver"
> sessions
> --
>
> Key: CASSANDRA-15665
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15665
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta1
>
>
> {{StreamManager}} does currently a suboptimal job in differentiating between
> stream sessions (in form of {{StreamResultFuture}}) which have been either
> initiated or "received", for the following reasons:
> 1) Naming is IMO confusing: a "receiver" session could actually both send and
> receive files, so technically an initiator is also a receiver.
> 2) {{StreamManager#findSession()}} assumes we should first looking into
> "initiator" sessions, then into "receiver" ones: this is a dangerous
> assumptions, in particular for test environments where the same process could
> work as both an initiator and a receiver.
> I would recommend the following changes:
> 1) Rename "receiver" with "follower" everywhere the former is used.
> 2) Introduce a new flag into {{StreamMessageHeader}} to signal if the message
> comes from an initiator or follower session, in order to correctly
> differentiate and look for sessions in {{StreamManager}}.
> While my arguments above might seem trivial, I believe they will improve
> clarity and save from potential bugs/headaches at testing time, and doing
> such changes now that we're revamping streaming for 4.0 seems the right time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15665) StreamManager should clearly differentiate between "initiator" and "receiver" sessions
[
https://issues.apache.org/jira/browse/CASSANDRA-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163992#comment-17163992
]
Caleb Rackliffe commented on CASSANDRA-15665:
-
[~jasonstack] Did we introduce a change
[here|https://github.com/apache/cassandra/commit/ebba613b0b34ea338eed508a3ba6cbb235986fd9#diff-85ee7eed2fccf1f9220452e69fbf8ca8R115]
that needs to be guarded by a {{VERSION_40}} check? (nit: I'd also call it
{{sentByFollower}}.)
> StreamManager should clearly differentiate between "initiator" and "receiver"
> sessions
> --
>
> Key: CASSANDRA-15665
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15665
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta1
>
>
> {{StreamManager}} does currently a suboptimal job in differentiating between
> stream sessions (in form of {{StreamResultFuture}}) which have been either
> initiated or "received", for the following reasons:
> 1) Naming is IMO confusing: a "receiver" session could actually both send and
> receive files, so technically an initiator is also a receiver.
> 2) {{StreamManager#findSession()}} assumes we should first looking into
> "initiator" sessions, then into "receiver" ones: this is a dangerous
> assumptions, in particular for test environments where the same process could
> work as both an initiator and a receiver.
> I would recommend the following changes:
> 1) Rename "receiver" with "follower" everywhere the former is used.
> 2) Introduce a new flag into {{StreamMessageHeader}} to signal if the message
> comes from an initiator or follower session, in order to correctly
> differentiate and look for sessions in {{StreamManager}}.
> While my arguments above might seem trivial, I believe they will improve
> clarity and save from potential bugs/headaches at testing time, and doing
> such changes now that we're revamping streaming for 4.0 seems the right time.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163990#comment-17163990
]
Ekaterina Dimitrova commented on CASSANDRA-15971:
-
Thank you [~jshook], based on your testing and the Slack discussion we had
today, I am going to update and start working on this one tomorrow.
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/fql
>Reporter: Jonathan Shook
>Priority: Normal
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15973) Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic
[ https://issues.apache.org/jira/browse/CASSANDRA-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15973: - Since Version: 4.0-alpha1 Source Control Link: https://github.com/apache/cassandra/commit/57a2a8613d2595b8650c24ef1cf3bb0055202409 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed, thanks! > Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing > logic > --- > > Key: CASSANDRA-15973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15973 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > In 3.0 and earlier, custom versions could have an extra digit and could have > multiple pre release tags separated by -, after upgrading to 4.0 these custom > versions no longer are allowed and fail different parts of the code, any > place which now uses CassandraVersion. > The issue is that version was just a String before, so the only parsing rules > were controlled by the client, now CassandraVersion is used which is more > restrictive. > To help out, we should allow a "hot fix" digit, and preRelease should allow > multiple - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15973) Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic
[ https://issues.apache.org/jira/browse/CASSANDRA-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15973: - Fix Version/s: (was: 4.0-beta) 4.0-beta2 > Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing > logic > --- > > Key: CASSANDRA-15973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15973 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta2 > > > In 3.0 and earlier, custom versions could have an extra digit and could have > multiple pre release tags separated by -, after upgrading to 4.0 these custom > versions no longer are allowed and fail different parts of the code, any > place which now uses CassandraVersion. > The issue is that version was just a String before, so the only parsing rules > were controlled by the client, now CassandraVersion is used which is more > restrictive. > To help out, we should allow a "hot fix" digit, and preRelease should allow > multiple - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15973) Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic
[ https://issues.apache.org/jira/browse/CASSANDRA-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15973: - Status: Ready to Commit (was: Review In Progress) > Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing > logic > --- > > Key: CASSANDRA-15973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15973 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > In 3.0 and earlier, custom versions could have an extra digit and could have > multiple pre release tags separated by -, after upgrading to 4.0 these custom > versions no longer are allowed and fail different parts of the code, any > place which now uses CassandraVersion. > The issue is that version was just a String before, so the only parsing rules > were controlled by the client, now CassandraVersion is used which is more > restrictive. > To help out, we should allow a "hot fix" digit, and preRelease should allow > multiple - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Fix version parsing logic when upgrading from 3.0
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 57a2a86 Fix version parsing logic when upgrading from 3.0
57a2a86 is described below
commit 57a2a8613d2595b8650c24ef1cf3bb0055202409
Author: David Capwell
AuthorDate: Thu Jul 23 10:31:33 2020 -0700
Fix version parsing logic when upgrading from 3.0
Patch by David Capwell, reviewed by Jon Meredith and brandonwilliams for
CASSANDRA-15973
---
CHANGES.txt| 1 +
.../apache/cassandra/utils/CassandraVersion.java | 111 +++---
.../cassandra/utils/CassandraVersionTest.java | 164 +++--
.../org/apache/cassandra/utils/Generators.java | 61
4 files changed, 241 insertions(+), 96 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 0ab4548..a36f379 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-beta2
+ * Fix version parsing logic when upgrading from 3.0 (CASSANDRA-15973)
* Optimize NoSpamLogger use in hot paths (CASSANDRA-15766)
* Verify sstable components on startup (CASSANDRA-15945)
Merged from 3.11:
diff --git a/src/java/org/apache/cassandra/utils/CassandraVersion.java
b/src/java/org/apache/cassandra/utils/CassandraVersion.java
index aed0fe7..b3dca96 100644
--- a/src/java/org/apache/cassandra/utils/CassandraVersion.java
+++ b/src/java/org/apache/cassandra/utils/CassandraVersion.java
@@ -18,10 +18,13 @@
package org.apache.cassandra.utils;
import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
-import com.google.common.base.Objects;
+import com.google.common.annotations.VisibleForTesting;
import org.apache.commons.lang3.StringUtils;
/**
@@ -33,22 +36,35 @@ import org.apache.commons.lang3.StringUtils;
public class CassandraVersion implements Comparable
{
/**
- * note: 3rd group matches to words but only allows number and checked
after regexp test.
+ * note: 3rd/4th groups matches to words but only allows number and
checked after regexp test.
* this is because 3rd and the last can be identical.
**/
-private static final String VERSION_REGEXP =
"(\\d+)\\.(\\d+)(?:\\.(\\w+))?(\\-[.\\w]+)?([.+][.\\w]+)?";
-private static final Pattern PATTERN_WHITESPACE = Pattern.compile("\\w+");
+private static final String VERSION_REGEXP =
"(\\d+)\\.(\\d+)(?:\\.(\\w+))?(?:\\.(\\w+))?(\\-[-.\\w]+)?([.+][.\\w]+)?";
+private static final Pattern PATTERN_WORDS = Pattern.compile("\\w+");
+@VisibleForTesting
+static final int NO_HOTFIX = -1;
-private static final Pattern pattern = Pattern.compile(VERSION_REGEXP);
-private static final Pattern SNAPSHOT = Pattern.compile("-SNAPSHOT");
+private static final Pattern PATTERN = Pattern.compile(VERSION_REGEXP);
public final int major;
public final int minor;
public final int patch;
+public final int hotfix;
private final String[] preRelease;
private final String[] build;
+@VisibleForTesting
+CassandraVersion(int major, int minor, int patch, int hotfix, String[]
preRelease, String[] build)
+{
+this.major = major;
+this.minor = minor;
+this.patch = patch;
+this.hotfix = hotfix;
+this.preRelease = preRelease;
+this.build = build;
+}
+
/**
* Parse a version from a string.
*
@@ -58,8 +74,7 @@ public class CassandraVersion implements
Comparable
*/
public CassandraVersion(String version)
{
-String stripped = SNAPSHOT.matcher(version).replaceFirst("");
-Matcher matcher = pattern.matcher(stripped);
+Matcher matcher = PATTERN.matcher(version);
if (!matcher.matches())
throw new IllegalArgumentException("Invalid version value: " +
version);
@@ -68,12 +83,13 @@ public class CassandraVersion implements
Comparable
this.major = Integer.parseInt(matcher.group(1));
this.minor = Integer.parseInt(matcher.group(2));
this.patch = matcher.group(3) != null ?
Integer.parseInt(matcher.group(3)) : 0;
+this.hotfix = matcher.group(4) != null ?
Integer.parseInt(matcher.group(4)) : NO_HOTFIX;
-String pr = matcher.group(4);
-String bld = matcher.group(5);
+String pr = matcher.group(5);
+String bld = matcher.group(6);
-this.preRelease = pr == null || pr.isEmpty() ? null :
parseIdentifiers(stripped, pr);
-this.build = bld == null || bld.isEmpty() ? null :
parseIdentifiers(stripped, bld);
+this.preRelease = pr == null || pr.isEmpty() ? null :
parseIdentifiers(version, pr);
+this.build = bld == null || bld.isEmpty() ?
[jira] [Updated] (CASSANDRA-15542) In JVM test for repairs on token boundaries
[ https://issues.apache.org/jira/browse/CASSANDRA-15542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15542: -- Fix Version/s: 4.0-beta2 Source Control Link: https://github.com/apache/cassandra/commit/a0debd8f9e8636b69d5ed1752f8e9b1b3c664954 Resolution: Fixed Status: Resolved (was: Ready to Commit) > In JVM test for repairs on token boundaries > > > Key: CASSANDRA-15542 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15542 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Low > Labels: pull-request-available > Fix For: 4.0-beta2 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Putting partitions on each token range +-1 and making sure the logic end to > end with repairs correctly handle inclusive and exclusivity of the bounds. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15973) Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic
[ https://issues.apache.org/jira/browse/CASSANDRA-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15973: - Reviewers: Brandon Williams, Jon Meredith, Brandon Williams (was: Brandon Williams, Brandon Williams, Jon Meredith) Brandon Williams, Jon Meredith, Brandon Williams Status: Review In Progress (was: Patch Available) > Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing > logic > --- > > Key: CASSANDRA-15973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15973 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > In 3.0 and earlier, custom versions could have an extra digit and could have > multiple pre release tags separated by -, after upgrading to 4.0 these custom > versions no longer are allowed and fail different parts of the code, any > place which now uses CassandraVersion. > The issue is that version was just a String before, so the only parsing rules > were controlled by the client, now CassandraVersion is used which is more > restrictive. > To help out, we should allow a "hot fix" digit, and preRelease should allow > multiple - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15973) Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic
[ https://issues.apache.org/jira/browse/CASSANDRA-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15973: - Reviewers: Brandon Williams, Jon Meredith (was: Brandon Williams, Brandon Williams, Jon Meredith) > Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing > logic > --- > > Key: CASSANDRA-15973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15973 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > In 3.0 and earlier, custom versions could have an extra digit and could have > multiple pre release tags separated by -, after upgrading to 4.0 these custom > versions no longer are allowed and fail different parts of the code, any > place which now uses CassandraVersion. > The issue is that version was just a String before, so the only parsing rules > were controlled by the client, now CassandraVersion is used which is more > restrictive. > To help out, we should allow a "hot fix" digit, and preRelease should allow > multiple - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15542) In JVM test for repairs on token boundaries
[ https://issues.apache.org/jira/browse/CASSANDRA-15542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163964#comment-17163964 ] David Capwell commented on CASSANDRA-15542: --- committed, rebased and updated to use nodetool; synced with Chris offline to make sure he was good with the changes. > In JVM test for repairs on token boundaries > > > Key: CASSANDRA-15542 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15542 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Low > Labels: pull-request-available > Fix For: 4.0-beta2 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Putting partitions on each token range +-1 and making sure the logic end to > end with repairs correctly handle inclusive and exclusivity of the bounds. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: In JVM test for repairs on token boundaries
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new a0debd8 In JVM test for repairs on token boundaries
a0debd8 is described below
commit a0debd8f9e8636b69d5ed1752f8e9b1b3c664954
Author: Chris Lohfink
AuthorDate: Tue Feb 4 09:59:25 2020 -0800
In JVM test for repairs on token boundaries
patch by Chris Lohfink; reviewed by David Capwell for CASSANDRA-15542
---
.../apache/cassandra/dht/Murmur3Partitioner.java | 13 ++
.../org/apache/cassandra/utils/ByteBufferUtil.java | 2 +
.../org/apache/cassandra/utils/MurmurHash.java | 113 -
.../distributed/test/RepairBoundaryTest.java | 182 +
.../cassandra/dht/Murmur3PartitionerTest.java | 15 ++
5 files changed, 322 insertions(+), 3 deletions(-)
diff --git a/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
b/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
index 52d0efb..2856f13 100644
--- a/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
+++ b/src/java/org/apache/cassandra/dht/Murmur3Partitioner.java
@@ -36,6 +36,7 @@ import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.MurmurHash;
import org.apache.cassandra.utils.ObjectSizes;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.primitives.Longs;
/**
@@ -207,6 +208,18 @@ public class Murmur3Partitioner implements IPartitioner
{
return new LongToken(token + 1);
}
+
+/**
+ * Reverses murmur3 to find a possible 16 byte key that generates a
given token
+ */
+@VisibleForTesting
+public static ByteBuffer keyForToken(LongToken token)
+{
+ByteBuffer result = ByteBuffer.allocate(16);
+long[] inv = MurmurHash.inv_hash3_x64_128(new long[] {token.token,
0L});
+result.putLong(inv[0]).putLong(inv[1]).position(0);
+return result;
+}
}
/**
diff --git a/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
b/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
index ff3fb3d..5300d9d 100644
--- a/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
+++ b/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
@@ -535,6 +535,8 @@ public class ByteBufferUtil
return ByteBufferUtil.bytes((InetAddress) obj);
else if (obj instanceof String)
return ByteBufferUtil.bytes((String) obj);
+else if (obj instanceof ByteBuffer)
+return (ByteBuffer) obj;
else
throw new IllegalArgumentException(String.format("Cannot convert
value %s of type %s",
obj,
diff --git a/src/java/org/apache/cassandra/utils/MurmurHash.java
b/src/java/org/apache/cassandra/utils/MurmurHash.java
index c02fdcc..80cf5cd 100644
--- a/src/java/org/apache/cassandra/utils/MurmurHash.java
+++ b/src/java/org/apache/cassandra/utils/MurmurHash.java
@@ -18,6 +18,9 @@
package org.apache.cassandra.utils;
import java.nio.ByteBuffer;
+import java.util.BitSet;
+
+import com.google.common.primitives.Longs;
/**
* This is a very fast, non-cryptographic hash suitable for general hash-based
@@ -146,7 +149,7 @@ public class MurmurHash
return h64;
}
-protected static long getblock(ByteBuffer key, int offset, int index)
+protected static long getBlock(ByteBuffer key, int offset, int index)
{
int i_8 = index << 3;
int blockOffset = offset + i_8;
@@ -187,8 +190,8 @@ public class MurmurHash
for(int i = 0; i < nblocks; i++)
{
-long k1 = getblock(key, offset, i*2+0);
-long k2 = getblock(key, offset, i*2+1);
+long k1 = getBlock(key, offset, i * 2 + 0);
+long k2 = getBlock(key, offset, i * 2 + 1);
k1 *= c1; k1 = rotl64(k1,31); k1 *= c2; h1 ^= k1;
@@ -248,4 +251,108 @@ public class MurmurHash
result[1] = h2;
}
+protected static long invRotl64(long v, int n)
+{
+return ((v >>> n) | (v << (64 - n)));
+}
+
+protected static long invRShiftXor(long value, int shift)
+{
+long output = 0;
+long i = 0;
+while (i * shift < 64)
+{
+long c = (0xL << (64 - shift)) >>> (shift * i);
+long partOutput = value & c;
+value ^= partOutput >>> shift;
+output |= partOutput;
+i += 1;
+}
+return output;
+}
+
+protected static long invFmix(long k)
+{
+k = invRShiftXor(k, 33);
+k *= 0x9cb4b2f8129337dbL;
+k = invRShiftXor(k, 33);
+k *= 0x4f74430c22a54005L;
+k = invRShiftXor(k, 33);
+return k;
+}
+
+/**
+
[jira] [Commented] (CASSANDRA-15973) Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic
[ https://issues.apache.org/jira/browse/CASSANDRA-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163924#comment-17163924 ] Jon Meredith commented on CASSANDRA-15973: -- +1 - looks good to me and I really appreciate the QT test. With regards to removing the dead code findSupportingVersion and isSupportedBy, I checked the Cassandra & Python Dtest and did a quick search of GitHub itself and couldn't find anything. Now all we need is a committer. > Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing > logic > --- > > Key: CASSANDRA-15973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15973 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > In 3.0 and earlier, custom versions could have an extra digit and could have > multiple pre release tags separated by -, after upgrading to 4.0 these custom > versions no longer are allowed and fail different parts of the code, any > place which now uses CassandraVersion. > The issue is that version was just a String before, so the only parsing rules > were controlled by the client, now CassandraVersion is used which is more > restrictive. > To help out, we should allow a "hot fix" digit, and preRelease should allow > multiple - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163905#comment-17163905
]
Jonathan Shook commented on CASSANDRA-15971:
I was able to get the server to log FQL data with both Java 8 and Java 11. This
appears to be a docs issue, as I had read that you could configure fql logging
either in yaml or via nodetool. The official docs seem correct, so no issue
there.
However, the other usability issues still apply in my view, and we might want
to triage them separately.
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/fql
>Reporter: Jonathan Shook
>Priority: Normal
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15861) Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure
[
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163844#comment-17163844
]
Caleb Rackliffe commented on CASSANDRA-15861:
-
bq. writing new index summary to a temp file and replacing the old file
atomically; on the streaming side, use hard link to make sure it streams the
same file
I like this a lot. It could also be used in the stats component case, which at
least consolidates our approach to this kind of problem (in general). The
{{ComponentManifest}} becomes more of a {{ComponentStreamer}}, given we'd
delegate the responsibility for putting bytes on the stream to it.
> Mutating sstable component may race with entire-sstable-streaming(ZCS)
> causing checksum validation failure
> --
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Consistency/Streaming,
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator -
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081
> CassandraEntireSSTableStreamReader.java:145 - [Stream
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
> at
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>
> In the above test, it executes "nodetool repair" on node1 and kills node2
> during repair. At the end, node3 reports checksum validation failure on
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be
> transferred to node3.
> 3. Before node1 actually sends the files to node3, node2 is killed and node1
> starts to broadcast repair-failure-message to all participants in
> {{CoordinatorSession#fail}}
> 4. Node1 receives its own repair-failure-message and fails its local repair
> sessions at {{LocalSessions#failSession}} which triggers async background
> compaction.
> 5. Node1's background compaction will mutate sstable's repairAt to 0 and
> pending repair id to null via
> {{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more
> in-progress repair.
> 6. Node1 actually sends the sstable to node3 where the sstable's STATS
> component size is different from the original size recorded in the manifest.
> 7. At the end, node3 reports checksum validation failure when it tries to
> mutate sstable
[jira] [Commented] (CASSANDRA-15502) In Tree Tooling with Java 11
[ https://issues.apache.org/jira/browse/CASSANDRA-15502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163832#comment-17163832 ] David Capwell commented on CASSANDRA-15502: --- +1 > In Tree Tooling with Java 11 > > > Key: CASSANDRA-15502 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15502 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/fuzz, Tool/bulk load, Tool/cqlsh, > Tool/diff, Tool/fql, Tool/nodetool, Tool/sstable, Tool/stress >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > This is to cover testing the various tools running on java 11. > The scope of this testing is manual testing and not automated, different > JIRAs should cover automation testing these tool. > The tools in question are: nodetool, sstableloader, sstablescrub, > sstableupgrade, sstableutil, sstableverify, fqltool, stress, auditlogviewer, > compaction-stress, sstabledump, sstableexpiredblockers, sstablemetadata, > sstableofflinerelevel, sstablesplit, and sstablerepairedset (many of these > may be tested already in dtest) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging
[ https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163823#comment-17163823 ] David Capwell commented on CASSANDRA-15958: --- Failed again: https://app.circleci.com/pipelines/github/dcapwell/cassandra/307/workflows/a5d97246-2f81-489e-81e9-55c311ad65ae/jobs/1542 > org.apache.cassandra.net.ConnectionTest testMessagePurging > -- > > Key: CASSANDRA-15958 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15958 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > Build: > https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/ > Build: > https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/ > java.util.concurrent.TimeoutException > at org.apache.cassandra.net.AsyncPromise.get(AsyncPromise.java:258) > at org.apache.cassandra.net.FutureDelegate.get(FutureDelegate.java:143) > at > org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:268) > at > org.apache.cassandra.net.ConnectionTest.testManual(ConnectionTest.java:236) > at > org.apache.cassandra.net.ConnectionTest.testMessagePurging(ConnectionTest.java:679) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15973) Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic
[ https://issues.apache.org/jira/browse/CASSANDRA-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15973: -- Test and Documentation Plan: unit tests + manual test upgrade Status: Patch Available (was: Open) > Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing > logic > --- > > Key: CASSANDRA-15973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15973 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > In 3.0 and earlier, custom versions could have an extra digit and could have > multiple pre release tags separated by -, after upgrading to 4.0 these custom > versions no longer are allowed and fail different parts of the code, any > place which now uses CassandraVersion. > The issue is that version was just a String before, so the only parsing rules > were controlled by the client, now CassandraVersion is used which is more > restrictive. > To help out, we should allow a "hot fix" digit, and preRelease should allow > multiple - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15953) Support fetching all user tables to compare in Cassandra-diff
[ https://issues.apache.org/jira/browse/CASSANDRA-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163818#comment-17163818 ] Yifan Cai commented on CASSANDRA-15953: --- bq. system.batches uses LocalPartitioner so if you run an auto-discovery diff without excluding system, you get a bunch of errors That is very good point. Thanks for bringing it up. Would you like to take another look at the PR? Now the system keyspaces are always excluded. > Support fetching all user tables to compare in Cassandra-diff > - > > Key: CASSANDRA-15953 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15953 > Project: Cassandra > Issue Type: Improvement > Components: Tool/diff >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > > The spark diff job may fail to launch with kernel error "E2BIG: Argument list > too long", when passing a large list of keyspace table list to compare. > Proposing a mode to fetch all user tables from the clusters to be compared. > When the mode is on, the spark job ignores the parameter "keyspace_tables". -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15974) SASIIndexTest is flaky in cassandra-3.11
[
https://issues.apache.org/jira/browse/CASSANDRA-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-15974:
--
Fix Version/s: 3.0.x
> SASIIndexTest is flaky in cassandra-3.11
>
>
> Key: CASSANDRA-15974
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15974
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Andres de la Peña
>Priority: Normal
> Fix For: 3.0.x
>
>
> The following {{SASIIndexTestSASIIndexTest}} tests seem flaky in 3.11:
> * testIndexMemtableSwitching
> * testPagination
> * testIndexRedistribution
> * testTruncate
> The test failures can be seen in
> [ci-cassandra|https://ci-cassandra.apache.org/job/Cassandra-3.11/76/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/],
> and I have also reproduced in our [internal test
> multiplexer|https://jenkins-cassandra.datastax.lan/job/parameterized-testall/666/].
> They don't seem to happen in trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15814) order by descending on frozen list not working
[
https://issues.apache.org/jira/browse/CASSANDRA-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163812#comment-17163812
]
Andres de la Peña commented on CASSANDRA-15814:
---
Created CASSANDRA-15974 for the flaky SASI tests.
> order by descending on frozen list not working
> --
>
> Key: CASSANDRA-15814
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15814
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Felipe Perez
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 2.2.17, 3.0.22, 3.11.8, 4.0-beta2
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> By creating a table like the following:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> )
> {code}
> It works and version is ordered in an ascending order. But when trying to
> order in descending order:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> ) WITH CLUSTERING ORDER BY (version DESC);
> {code}
> The table is created normally, but when trying to insert a row:
> {code:java}
> insert into software(name, version) values ('t1', [2,10,30,40,50]);
> {code}
> Cassandra throws an error:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid
> list literal for version of type frozen>"
> {code}
> The goal here is that I would like to get the last version of a software.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15974) SASIIndexTest is flaky in cassandra-3.11
Andres de la Peña created CASSANDRA-15974:
-
Summary: SASIIndexTest is flaky in cassandra-3.11
Key: CASSANDRA-15974
URL: https://issues.apache.org/jira/browse/CASSANDRA-15974
Project: Cassandra
Issue Type: Bug
Components: Test/unit
Reporter: Andres de la Peña
The following {{SASIIndexTestSASIIndexTest}} tests seem flaky in 3.11:
* testIndexMemtableSwitching
* testPagination
* testIndexRedistribution
* testTruncate
The test failures can be seen in
[ci-cassandra|https://ci-cassandra.apache.org/job/Cassandra-3.11/76/testReport/org.apache.cassandra.index.sasi/SASIIndexTest/],
and I have also reproduced in our [internal test
multiplexer|https://jenkins-cassandra.datastax.lan/job/parameterized-testall/666/].
They don't seem to happen in trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15973) Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic
[ https://issues.apache.org/jira/browse/CASSANDRA-15973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15973: -- Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear Impact(13164) Complexity: Low Hanging Fruit Discovered By: Workload Replay Fix Version/s: 4.0-beta Severity: Normal Status: Open (was: Triage Needed) > Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing > logic > --- > > Key: CASSANDRA-15973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15973 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > In 3.0 and earlier, custom versions could have an extra digit and could have > multiple pre release tags separated by -, after upgrading to 4.0 these custom > versions no longer are allowed and fail different parts of the code, any > place which now uses CassandraVersion. > The issue is that version was just a String before, so the only parsing rules > were controlled by the client, now CassandraVersion is used which is more > restrictive. > To help out, we should allow a "hot fix" digit, and preRelease should allow > multiple - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15861) Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure
[
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163769#comment-17163769
]
ZhaoYang edited comment on CASSANDRA-15861 at 7/23/20, 5:29 PM:
Updated the patch to load stats component into memory, so that entire-sstable
streaming will not block LCS and incremental repair..
If we want to reduce the blocking time for index summary redistribution, we can
consider:
* writing new index summary to a temp file and replacing the old file
atomically; at the beginning of streaming, open all file channel instances
which still point to the old files (this is file system dependent).
* writing new index summary to a temp file and replacing the old file
atomically; on the streaming side, use hard link to make sure it streams the
same file.
WDYT?
was (Author: jasonstack):
Updated the patch to load stats component into memory, so that entire-sstable
streaming will not block LCS and incremental repair..
If we want to reduce the blocking time for index summary redistribution, we can
consider: writing new index summary to a temp file and replacing the old file
atomically; at the beginning of streaming, open all file channel instances
which still point to the old files (this is file system dependent). WDYT?
> Mutating sstable component may race with entire-sstable-streaming(ZCS)
> causing checksum validation failure
> --
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Consistency/Streaming,
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator -
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081
> CassandraEntireSSTableStreamReader.java:145 - [Stream
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
> at
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>
> In the above test, it executes "nodetool repair" on node1 and kills node2
> during repair. At the end, node3 reports checksum validation failure on
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be
> transferred to node3.
> 3. Before node1 actually sends the files to node3, node2 is killed and node1
> starts to broadcast repair-failure-message to all
[jira] [Created] (CASSANDRA-15973) Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic
David Capwell created CASSANDRA-15973: - Summary: Upgrade of custom 3.0 version fails with custom 4.0 caused by version parsing logic Key: CASSANDRA-15973 URL: https://issues.apache.org/jira/browse/CASSANDRA-15973 Project: Cassandra Issue Type: Bug Components: Build Reporter: David Capwell Assignee: David Capwell In 3.0 and earlier, custom versions could have an extra digit and could have multiple pre release tags separated by -, after upgrading to 4.0 these custom versions no longer are allowed and fail different parts of the code, any place which now uses CassandraVersion. The issue is that version was just a String before, so the only parsing rules were controlled by the client, now CassandraVersion is used which is more restrictive. To help out, we should allow a "hot fix" digit, and preRelease should allow multiple - -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15861) Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure
[
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163769#comment-17163769
]
ZhaoYang commented on CASSANDRA-15861:
--
Updated the patch to load stats component into memory, so that entire-sstable
streaming will not block LCS and incremental repair..
If we want to reduce the blocking time for index summary redistribution, we can
consider: writing new index summary to a temp file and replacing the old file
atomically; at the beginning of streaming, open all file channel instances
which still point to the old files (this is file system dependent). WDYT?
> Mutating sstable component may race with entire-sstable-streaming(ZCS)
> causing checksum validation failure
> --
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Consistency/Streaming,
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator -
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081
> CassandraEntireSSTableStreamReader.java:145 - [Stream
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
> at
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>
> In the above test, it executes "nodetool repair" on node1 and kills node2
> during repair. At the end, node3 reports checksum validation failure on
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be
> transferred to node3.
> 3. Before node1 actually sends the files to node3, node2 is killed and node1
> starts to broadcast repair-failure-message to all participants in
> {{CoordinatorSession#fail}}
> 4. Node1 receives its own repair-failure-message and fails its local repair
> sessions at {{LocalSessions#failSession}} which triggers async background
> compaction.
> 5. Node1's background compaction will mutate sstable's repairAt to 0 and
> pending repair id to null via
> {{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more
> in-progress repair.
> 6. Node1 actually sends the sstable to node3 where the sstable's STATS
> component size is different from the original size recorded in the manifest.
> 7. At the end, node3 reports checksum validation failure when it tries to
> mutate sstable level and "isTransient"
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163743#comment-17163743
]
Olivier Michallat commented on CASSANDRA-15299:
---
Nice to see the performance boost. Is compression enabled in any of those tests?
{quote}Speaking of {{Segment}} in driver, we probably want to use naming that
is consistent with server.
{quote}
Yes. Once we've agreed on the definitive names here, I'll do a global
refactoring.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Alex Petrov
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-alpha
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15693) Generating nodetool docs fails with python 3
[
https://issues.apache.org/jira/browse/CASSANDRA-15693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15693:
Reviewers: Ekaterina Dimitrova, Ekaterina Dimitrova (was: Ekaterina
Dimitrova)
Ekaterina Dimitrova, Ekaterina Dimitrova (was: Ekaterina
Dimitrova)
Status: Review In Progress (was: Patch Available)
> Generating nodetool docs fails with python 3
>
>
> Key: CASSANDRA-15693
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15693
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Labels: patch
> Fix For: 4.0-beta
>
> Attachments: 0001-Fix-file-open-modes-for-python-3.patch
>
>
> Building nodetool docs with the ant target 'gen-doc' fails for python 3.
> Python 3 doesn't allow the file open mode "rw+".
> {noformat}
> gen-doc:
> [exec] python convert_yaml_to_rst.py ../conf/cassandra.yaml
> source/configuration/cassandra_config_file.rst
> [exec] python gen-nodetool-docs.py
> [exec] Makefile:64: recipe for target 'html' failed
> [exec] Traceback (most recent call last):
> [exec] File "gen-nodetool-docs.py", line 75, in
> [exec] with open(helpfilename, "rw+") as helpfile:
> [exec] ValueError: must have exactly one of create/read/write/append mode
> [exec] make: *** [html] Error 1
> [exec] Result: 2
> {noformat}
> Fails on and patch [^0001-Fix-file-open-modes-for-python-3.patch] tested on
> the following platforms: Ubuntu 18.04.4 LTS with Python 3.6.9 and FreeBSD
> 12.1-RELEASE-p1 with Python 3.7.7.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163718#comment-17163718
]
Jonathan Shook commented on CASSANDRA-15971:
My issue is with the actual fql logging on the server not writing logs. I'll
try to look into it today.
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/fql
>Reporter: Jonathan Shook
>Priority: Normal
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15583) 4.0 quality testing: Tooling, Bundled and First Party
[ https://issues.apache.org/jira/browse/CASSANDRA-15583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15583: Description: Reference [doc from NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] for context. *Shepherd: Sam Tunnicliffe* Test plans should cover bundled first-party tooling and CLIs such as nodetool, cqlsh, and new tools supporting full query and audit logging (CASSANDRA-13983, CASSANDRA-12151). ||Tool||UX test||UT coverage||dtest coverage||Comments|| |Nodetool|(x)|(x)|(!)|By no means all the sub commands are tested. Dtest also test nodetool as a side effect| |Cqlsh|(x)|(x) |(!) | | |Cassandra-stress|(x)|(x)|(x) | Some UT| |debug-cql|(x)|(x)|(x) | | |fqltool| | | | | |auditlogviewer| | | | | |*Sstable utilities*| | | | | |sstabledump| | | | | |sstableexpiredblockers| | | | | |sstablelevelreset| | | | | |sstableloader| | | | | |sstablemetadata| | | |Ran in dtests, no dedicated test| |sstableofflinerelevel| | | | | |sstablerepairedset| | | |Ran in dtests, no dedicated test| |sstablescrub| | | | | |sstablesplit| | | | | |sstableupgrade| | | | | |sstableutil| | | | | |sstableverify| | | | | was: Reference [doc from NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] for context. *Shepherd: Sam Tunnicliffe* Test plans should cover bundled first-party tooling and CLIs such as nodetool, cqlsh, and new tools supporting full query and audit logging (CASSANDRA-13983, CASSANDRA-12151). ||Tool||UX test||UT coverage||dtest coverage||Comments|| |Nodetool |(x)|(x)|(!)| By no means all the sub commands are tested. Dtest also test nodetool as a side effect| |Cqlsh | | | | | |Cassandra-stress | | | | | |debug-cql | | | | | |fqltool| | | | | |auditlogviewer | | | | | |*Sstable utilities*| | | | | |sstabledump| | | | | |sstableexpiredblockers | | | | | |sstablelevelreset | | | | | |sstableloader | | | | | |sstablemetadata| | | |Ran in dtests, no dedicated test | |sstableofflinerelevel | | | | | |sstablerepairedset | | | |Ran in dtests, no dedicated test | |sstablescrub | | | | | |sstablesplit | | | | | |sstableupgrade | | | | | |sstableutil| | | | | |sstableverify | | | | | > 4.0 quality testing: Tooling, Bundled and First Party > - > > Key: CASSANDRA-15583 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15583 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/unit >Reporter: Josh McKenzie >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Reference [doc from > NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] > for context. > *Shepherd: Sam Tunnicliffe* > Test plans should cover bundled first-party tooling and CLIs such as > nodetool, cqlsh, and new tools supporting full query and audit logging > (CASSANDRA-13983, CASSANDRA-12151). > ||Tool||UX test||UT coverage||dtest coverage||Comments|| > |Nodetool|(x)|(x)|(!)|By no means all the sub commands are tested. Dtest also > test nodetool as a side effect| > |Cqlsh|(x)|(x) |(!) | | > |Cassandra-stress|(x)|(x)|(x) | Some UT| > |debug-cql|(x)|(x)|(x) | | > |fqltool| | | | | > |auditlogviewer| | | | | > |*Sstable utilities*| | | | | > |sstabledump| | | | | > |sstableexpiredblockers| | | | | > |sstablelevelreset| | | | | > |sstableloader| | | | | > |sstablemetadata| | | |Ran in dtests, no dedicated test| > |sstableofflinerelevel| | | | | > |sstablerepairedset| | | |Ran in dtests, no dedicated test| > |sstablescrub| | | | | > |sstablesplit| | | | | > |sstableupgrade| | | | | > |sstableutil| | | | | > |sstableverify| | | | | -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15946) Communication problem between C* 3 and C* 4 in in-jvm dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-15946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163612#comment-17163612
]
Alex Petrov commented on CASSANDRA-15946:
-
bq. no wonder, why would Cassandra 4.0 need to be able to serialize Cassandra
3.0 payloads
4.0 has to be able to serialize 3.0 payloads since it has to be able to
communicate in mixed mode. However, serializer for request response is set to
null.
Generally, there should not be anything specific to serialization in in-jvm
dtests. If you could describe the communication problem in more details, and
attach a minimal reproduce to the patch, it would be easier to understand
what's going on. Main thing I'm trying to rule out here is that we don't _also_
have the same problem in real clusters. I did try the test from
{{jwest/15833-3.11}} but I'm not sure how that would help since the problem is
between 3.x and 4.0 here.
> Communication problem between C* 3 and C* 4 in in-jvm dtests
>
>
> Key: CASSANDRA-15946
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15946
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
>
> There is a communication problem when testing upgrades using in-JVM dtest
> between Cassandra 3 and 4.
> In a method {{registerInboundFilter}} of {{Instance}}, we get a message which
> was just received and we prepare it for filtering as part of which, we
> serialize the payload again. This is fine when dealing with incoming
> Cassandra 4 message, because we can serialize it. However when we get the
> Cassandra 3 message, which uses a different protocol, and we get something
> like {{REQUEST_RSP}}, we can surely deserialize it through some special
> deserialization path, but we cannot serialize the payload for it as there is
> no serializer defined for {{REQUEST_RSP}} - no wonder, why would Cassandra
> 4.0 need to be able to serialize Cassandra 3.0 payloads?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163602#comment-17163602
]
Benedict Elliott Smith commented on CASSANDRA-15229:
I can, but probably not for a few weeks yet.
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-direct.png, 15229-hit-rate.png,
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png,
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png,
> 15229-unsafe.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
> -
> Since CASSANDRA-5863, chunk cache is implemented to use buffer pool. When
> local pool is full, one of its chunks will be evicted and only put back to
> global pool when all buffers in the evicted chunk are released. But due to
> chunk cache, buffers can be held for long period of time, preventing evicted
> chunk to be recycled even though most of space in the evicted chunk are free.
> There two things need to be improved:
> 1. Evicted chunk with free space should be recycled to global pool, even if
> it's not fully free. It's doable in 4.0.
> 2. Reduce fragmentation caused by different buffer size. With #1, partially
> freed chunk will be available for allocation, but "holes" in the partially
> freed chunk are with different sizes. We should consider allocating fixed
> buffer size which is unlikely to fit in 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15229:
--
Reviewers: Aleksey Yeschenko
Would be great to have a second reviewer here (I volunteer to be one of the
two).
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
> Attachments: 15229-count.png, 15229-direct.png, 15229-hit-rate.png,
> 15229-recirculate-count.png, 15229-recirculate-hit-rate.png,
> 15229-recirculate-size.png, 15229-recirculate.png, 15229-size.png,
> 15229-unsafe.png
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
> -
> Since CASSANDRA-5863, chunk cache is implemented to use buffer pool. When
> local pool is full, one of its chunks will be evicted and only put back to
> global pool when all buffers in the evicted chunk are released. But due to
> chunk cache, buffers can be held for long period of time, preventing evicted
> chunk to be recycled even though most of space in the evicted chunk are free.
> There two things need to be improved:
> 1. Evicted chunk with free space should be recycled to global pool, even if
> it's not fully free. It's doable in 4.0.
> 2. Reduce fragmentation caused by different buffer size. With #1, partially
> freed chunk will be available for allocation, but "holes" in the partially
> freed chunk are with different sizes. We should consider allocating fixed
> buffer size which is unlikely to fit in 4.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15966) Circle CI start_ jobs should not require build
[ https://issues.apache.org/jira/browse/CASSANDRA-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163091#comment-17163091 ] Ekaterina Dimitrova edited comment on CASSANDRA-15966 at 7/23/20, 1:40 PM: --- As far as I understand, we move the requirement for build completion from start_job (which is the approval we give for most of the tests in CircleCI) to the point when we run the actual job. That's great! That way we can approve the tests we need to run right at commit instead of waiting for the build completion to do it. +1 from me. But you need a committer :) was (Author: e.dimitrova): As far as I understand, we move the requirement for build completion from start_job (which is the approval we give for most of the tests in CircleCI) to run the actual job. That's great! That way we can approve the tests we need to run right at commit instead of waiting for the build completion to do it. +1 from me. But you need a committer :) > Circle CI start_ jobs should not require build > -- > > Key: CASSANDRA-15966 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15966 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: 4.0, 4.0-rc > > > The Circle CI start_ jobs all depend on build, so can’t approve them until > the build is done, but their downstream also depends on build. Given this, > we can remove this dependency which will allow someone to approve the > downstream pipelines even if the build isn’t complete yet. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15583) 4.0 quality testing: Tooling, Bundled and First Party
[ https://issues.apache.org/jira/browse/CASSANDRA-15583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15583: Description: Reference [doc from NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] for context. *Shepherd: Sam Tunnicliffe* Test plans should cover bundled first-party tooling and CLIs such as nodetool, cqlsh, and new tools supporting full query and audit logging (CASSANDRA-13983, CASSANDRA-12151). ||Tool||UX test||UT coverage||dtest coverage||Comments|| |Nodetool |(x)|(x)|(!)| By no means all the sub commands are tested. Dtest also test nodetool as a side effect| |Cqlsh | | | | | |Cassandra-stress | | | | | |debug-cql | | | | | |fqltool| | | | | |auditlogviewer | | | | | |*Sstable utilities*| | | | | |sstabledump| | | | | |sstableexpiredblockers | | | | | |sstablelevelreset | | | | | |sstableloader | | | | | |sstablemetadata| | | |Ran in dtests, no dedicated test | |sstableofflinerelevel | | | | | |sstablerepairedset | | | |Ran in dtests, no dedicated test | |sstablescrub | | | | | |sstablesplit | | | | | |sstableupgrade | | | | | |sstableutil| | | | | |sstableverify | | | | | was: Reference [doc from NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] for context. *Shepherd: Sam Tunnicliffe* Test plans should cover bundled first-party tooling and CLIs such as nodetool, cqlsh, and new tools supporting full query and audit logging (CASSANDRA-13983, CASSANDRA-12151). ||Tool||Has utests||Has dtest||J8/J11 coverage||Comments|| |Nodetool |(/)|(/)| | | |Cqlsh |(/)|(/)| | | |Cassandra-stress |(/)|(x)| | | |debug-cql |(x)|(x)| | | |fqltool|(/)|(/)| | | |auditlogviewer |(/)|(/)| | | |*Sstable utilities*| | | | | |sstabledump|(/)|(/)| | | |sstableexpiredblockers |(/)|(/)| | | |sstablelevelreset |(/)|(/)| | | |sstableloader |(/)|(/)| | | |sstablemetadata|(/)|(!)| |Ran in dtests, no dedicated test | |sstableofflinerelevel |(/)|(/)| | | |sstablerepairedset |(/)|(!)| |Ran in dtests, no dedicated test | |sstablescrub |(/)|(/)| | | |sstablesplit |(/)|(/)| | | |sstableupgrade |(/)|(/)| | | |sstableutil|(/)|(/)| | | |sstableverify |(/)|(/)| | | > 4.0 quality testing: Tooling, Bundled and First Party > - > > Key: CASSANDRA-15583 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15583 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/unit >Reporter: Josh McKenzie >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Reference [doc from > NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] > for context. > *Shepherd: Sam Tunnicliffe* > Test plans should cover bundled first-party tooling and CLIs such as > nodetool, cqlsh, and new tools supporting full query and audit logging > (CASSANDRA-13983, CASSANDRA-12151). > ||Tool||UX test||UT coverage||dtest coverage||Comments|| > |Nodetool |(x)|(x)|(!)| By no means all the sub commands are > tested. Dtest also test nodetool as a side effect| > |Cqlsh| | | | | > |Cassandra-stress | | | | | > |debug-cql| | | | | > |fqltool | | | | | > |auditlogviewer | | | | | > |*Sstable utilities* | | | | | > |sstabledump | | | | | > |sstableexpiredblockers | | | | | > |sstablelevelreset| | | | | > |sstableloader| | | | | > |sstablemetadata | | | |Ran in dtests, no dedicated test | > |sstableofflinerelevel| | | | | > |sstablerepairedset | | | |Ran in dtests, no dedicated test | > |sstablescrub | | | | | > |sstablesplit | | | | | > |sstableupgrade | | | | | > |sstableutil | | | | | > |sstableverify| | | | | -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CASSANDRA-15814) order by descending on frozen list not working
[
https://issues.apache.org/jira/browse/CASSANDRA-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-15814:
--
Reviewers: Berenguer Blasi, Andres de la Peña (was: Andres de la Peña,
Berenguer Blasi)
Berenguer Blasi, Andres de la Peña (was: Berenguer Blasi)
Status: Review In Progress (was: Patch Available)
> order by descending on frozen list not working
> --
>
> Key: CASSANDRA-15814
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15814
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Felipe Perez
>Assignee: Andres de la Peña
>Priority: Normal
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> By creating a table like the following:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> )
> {code}
> It works and version is ordered in an ascending order. But when trying to
> order in descending order:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> ) WITH CLUSTERING ORDER BY (version DESC);
> {code}
> The table is created normally, but when trying to insert a row:
> {code:java}
> insert into software(name, version) values ('t1', [2,10,30,40,50]);
> {code}
> Cassandra throws an error:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid
> list literal for version of type frozen>"
> {code}
> The goal here is that I would like to get the last version of a software.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15814) order by descending on frozen list not working
[
https://issues.apache.org/jira/browse/CASSANDRA-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-15814:
--
Fix Version/s: 4.0-beta2
3.11.8
3.0.22
2.2.17
Since Version: 2.1.3
Source Control Link:
https://github.com/apache/cassandra/commit/9f8d5b8d069a1db88e70deafff6c0edc23c896d0
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> order by descending on frozen list not working
> --
>
> Key: CASSANDRA-15814
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15814
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Felipe Perez
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 2.2.17, 3.0.22, 3.11.8, 4.0-beta2
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> By creating a table like the following:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> )
> {code}
> It works and version is ordered in an ascending order. But when trying to
> order in descending order:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> ) WITH CLUSTERING ORDER BY (version DESC);
> {code}
> The table is created normally, but when trying to insert a row:
> {code:java}
> insert into software(name, version) values ('t1', [2,10,30,40,50]);
> {code}
> Cassandra throws an error:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid
> list literal for version of type frozen>"
> {code}
> The goal here is that I would like to get the last version of a software.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (91ccf24 -> fdeb254)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 91ccf24 Merge branch 'cassandra-3.11' into trunk new 9f8d5b8 Fix CQL parsing of collections when the column type is reversed patch by Andrés de la Peña; reviewed by Berenguer Blasi for CASSANDRA-15814 new be1e3de Merge branch 'cassandra-2.2' into cassandra-3.0 new 7451cf0 Merge branch 'cassandra-3.0' into cassandra-3.11 new fdeb254 Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 2 + src/java/org/apache/cassandra/cql3/Lists.java | 17 +- src/java/org/apache/cassandra/cql3/Maps.java | 25 +- src/java/org/apache/cassandra/cql3/Sets.java | 20 +- .../validation/entities/FrozenCollectionsTest.java | 461 + 5 files changed, 440 insertions(+), 85 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15814) order by descending on frozen list not working
[
https://issues.apache.org/jira/browse/CASSANDRA-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-15814:
--
Status: Ready to Commit (was: Review In Progress)
> order by descending on frozen list not working
> --
>
> Key: CASSANDRA-15814
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15814
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Felipe Perez
>Assignee: Andres de la Peña
>Priority: Normal
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> By creating a table like the following:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> )
> {code}
> It works and version is ordered in an ascending order. But when trying to
> order in descending order:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> ) WITH CLUSTERING ORDER BY (version DESC);
> {code}
> The table is created normally, but when trying to insert a row:
> {code:java}
> insert into software(name, version) values ('t1', [2,10,30,40,50]);
> {code}
> Cassandra throws an error:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid
> list literal for version of type frozen>"
> {code}
> The goal here is that I would like to get the last version of a software.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15814) order by descending on frozen list not working
[
https://issues.apache.org/jira/browse/CASSANDRA-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163484#comment-17163484
]
Andres de la Peña commented on CASSANDRA-15814:
---
Thanks for the review.
Committed to {{cassandra-2.2}} as
[9f8d5b8d069a1db88e70deafff6c0edc23c896d0|https://github.com/apache/cassandra/commit/9f8d5b8d069a1db88e70deafff6c0edc23c896d0]
and merged all the way up to {{trunk}}.
> order by descending on frozen list not working
> --
>
> Key: CASSANDRA-15814
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15814
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Felipe Perez
>Assignee: Andres de la Peña
>Priority: Normal
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> By creating a table like the following:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> )
> {code}
> It works and version is ordered in an ascending order. But when trying to
> order in descending order:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> ) WITH CLUSTERING ORDER BY (version DESC);
> {code}
> The table is created normally, but when trying to insert a row:
> {code:java}
> insert into software(name, version) values ('t1', [2,10,30,40,50]);
> {code}
> Cassandra throws an error:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid
> list literal for version of type frozen>"
> {code}
> The goal here is that I would like to get the last version of a software.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (7a554a7 -> be1e3de)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 7a554a7 fix changes in anticipation of release new 9f8d5b8 Fix CQL parsing of collections when the column type is reversed patch by Andrés de la Peña; reviewed by Berenguer Blasi for CASSANDRA-15814 new be1e3de Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 2 + src/java/org/apache/cassandra/cql3/Lists.java | 18 +- src/java/org/apache/cassandra/cql3/Maps.java | 27 +- src/java/org/apache/cassandra/cql3/Sets.java | 20 +- .../validation/entities/FrozenCollectionsTest.java | 468 + 5 files changed, 450 insertions(+), 85 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 7451cf083c1fe9ca1063cc67bc845d9ccf108e38 Merge: 78555b9 be1e3de Author: Andrés de la Peña AuthorDate: Thu Jul 23 12:45:20 2020 +0100 Merge branch 'cassandra-3.0' into cassandra-3.11 # Conflicts: # src/java/org/apache/cassandra/cql3/Maps.java CHANGES.txt| 2 + src/java/org/apache/cassandra/cql3/Lists.java | 17 +- src/java/org/apache/cassandra/cql3/Maps.java | 25 +- src/java/org/apache/cassandra/cql3/Sets.java | 20 +- .../validation/entities/FrozenCollectionsTest.java | 468 + 5 files changed, 447 insertions(+), 85 deletions(-) diff --cc CHANGES.txt index 028b0ff,06a25b8..5f08069 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,15 -1,10 +1,17 @@@ -3.0.22: +3.11.8 + * Frozen RawTuple is not annotated with frozen in the toString method (CASSANDRA-15857) +Merged from 3.0: * Fix empty/null json string representation (CASSANDRA-15896) + Merged from 2.2: + * Fix CQL parsing of collections when the column type is reversed (CASSANDRA-15814) -3.0.21 +3.11.7 + * Fix cqlsh output when fetching all rows in batch mode (CASSANDRA-15905) + * Upgrade Jackson to 2.9.10 (CASSANDRA-15867) + * Fix CQL formatting of read command restrictions for slow query log (CASSANDRA-15503) + * Allow sstableloader to use SSL on the native port (CASSANDRA-14904) +Merged from 3.0: * Backport CASSANDRA-12189: escape string literals (CASSANDRA-15948) * Avoid hinted handoff per-host throttle being arounded to 0 in large cluster (CASSANDRA-15859) * Avoid emitting empty range tombstones from RangeTombstoneList (CASSANDRA-15924) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit be1e3de1c3cefeacbe8c8d8551a00cd9642e32b8
Merge: 7a554a7 9f8d5b8
Author: Andrés de la Peña
AuthorDate: Thu Jul 23 12:40:24 2020 +0100
Merge branch 'cassandra-2.2' into cassandra-3.0
# Conflicts:
# CHANGES.txt
# src/java/org/apache/cassandra/cql3/Lists.java
# src/java/org/apache/cassandra/cql3/Maps.java
# src/java/org/apache/cassandra/cql3/Sets.java
CHANGES.txt| 2 +
src/java/org/apache/cassandra/cql3/Lists.java | 18 +-
src/java/org/apache/cassandra/cql3/Maps.java | 27 +-
src/java/org/apache/cassandra/cql3/Sets.java | 20 +-
.../validation/entities/FrozenCollectionsTest.java | 468 +
5 files changed, 450 insertions(+), 85 deletions(-)
diff --cc CHANGES.txt
index 0bd6421,9034ae1..06a25b8
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,31 -1,5 +1,33 @@@
-2.2.17
+3.0.22:
+ * Fix empty/null json string representation (CASSANDRA-15896)
++Merged from 2.2:
+ * Fix CQL parsing of collections when the column type is reversed
(CASSANDRA-15814)
+
+
+3.0.21
+ * Backport CASSANDRA-12189: escape string literals (CASSANDRA-15948)
+ * Avoid hinted handoff per-host throttle being arounded to 0 in large
cluster (CASSANDRA-15859)
+ * Avoid emitting empty range tombstones from RangeTombstoneList
(CASSANDRA-15924)
+ * Avoid thread starvation, and improve compare-and-swap performance, in the
slab allocators (CASSANDRA-15922)
+ * Fix broken KEYS 2i queries after DROP COMPACT STORAGE (CASSANDRA-15906)
+ * Add token to tombstone warning and error messages (CASSANDRA-15890)
+ * Fixed range read concurrency factor computation and capped as 10 times tpc
cores (CASSANDRA-15752)
+ * Catch exception on bootstrap resume and init native transport
(CASSANDRA-15863)
+ * Fix replica-side filtering returning stale data with CL > ONE
(CASSANDRA-8272, CASSANDRA-8273)
+ * Fix duplicated row on 2.x upgrades when multi-rows range tombstones
interact with collection ones (CASSANDRA-15805)
+ * Rely on snapshotted session infos on StreamResultFuture.maybeComplete to
avoid race conditions (CASSANDRA-15667)
+ * EmptyType doesn't override writeValue so could attempt to write bytes when
expected not to (CASSANDRA-15790)
+ * Fix index queries on partition key columns when some partitions contains
only static data (CASSANDRA-13666)
+ * Avoid creating duplicate rows during major upgrades (CASSANDRA-15789)
+ * liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if
IndexSummaryRedistribution gets interrupted (CASSANDRA-15674)
+ * Fix Debian init start/stop (CASSANDRA-15770)
+ * Fix infinite loop on index query paging in tables with clustering
(CASSANDRA-14242)
+ * Fix chunk index overflow due to large sstable with small chunk length
(CASSANDRA-15595)
+ * cqlsh return non-zero status when STDIN CQL fails (CASSANDRA-15623)
+ * Don't skip sstables in slice queries based only on local min/max/deletion
timestamp (CASSANDRA-15690)
+ * Memtable memory allocations may deadlock (CASSANDRA-15367)
+ * Run evictFromMembership in GossipStage (CASSANDRA-15592)
+Merged from 2.2:
* Fix nomenclature of allow and deny lists (CASSANDRA-15862)
* Remove generated files from source artifact (CASSANDRA-15849)
* Remove duplicated tools binaries from tarballs (CASSANDRA-15768)
diff --cc src/java/org/apache/cassandra/cql3/Lists.java
index 065f74a,c6b78d7..f4b2294
--- a/src/java/org/apache/cassandra/cql3/Lists.java
+++ b/src/java/org/apache/cassandra/cql3/Lists.java
@@@ -29,10 -26,14 +29,12 @@@ import com.google.common.annotations.Vi
import org.apache.cassandra.config.ColumnDefinition;
import org.apache.cassandra.cql3.functions.Function;
-import org.apache.cassandra.db.Cell;
-import org.apache.cassandra.db.ColumnFamily;
-import org.apache.cassandra.db.composites.CellName;
-import org.apache.cassandra.db.composites.Composite;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.rows.*;
+ import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.Int32Type;
import org.apache.cassandra.db.marshal.ListType;
+ import org.apache.cassandra.db.marshal.ReversedType;
import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.serializers.CollectionSerializer;
import org.apache.cassandra.serializers.MarshalException;
@@@ -54,10 -56,20 +56,20 @@@ public abstract class List
public static ColumnSpecification valueSpecOf(ColumnSpecification column)
{
- return new ColumnSpecification(column.ksName, column.cfName, new
ColumnIdentifier("value(" + column.name + ")", true),
((ListType)column.type).getElementsType());
+ return new ColumnSpecification(column.ksName, column.cfName, new
ColumnIdentifier("value(" +
[cassandra] branch cassandra-3.11 updated (78555b9 -> 7451cf0)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 78555b9 fix broken merge new 9f8d5b8 Fix CQL parsing of collections when the column type is reversed patch by Andrés de la Peña; reviewed by Berenguer Blasi for CASSANDRA-15814 new be1e3de Merge branch 'cassandra-2.2' into cassandra-3.0 new 7451cf0 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 2 + src/java/org/apache/cassandra/cql3/Lists.java | 17 +- src/java/org/apache/cassandra/cql3/Maps.java | 25 +- src/java/org/apache/cassandra/cql3/Sets.java | 20 +- .../validation/entities/FrozenCollectionsTest.java | 468 + 5 files changed, 447 insertions(+), 85 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Fix CQL parsing of collections when the column type is reversed patch by Andrés de la Peña; reviewed by Berenguer Blasi for CASSANDRA-15814
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new 9f8d5b8 Fix CQL parsing of collections when the column type is
reversed patch by Andrés de la Peña; reviewed by Berenguer Blasi for
CASSANDRA-15814
9f8d5b8 is described below
commit 9f8d5b8d069a1db88e70deafff6c0edc23c896d0
Author: Andrés de la Peña
AuthorDate: Fri Jun 26 13:09:25 2020 +0100
Fix CQL parsing of collections when the column type is reversed
patch by Andrés de la Peña; reviewed by Berenguer Blasi for CASSANDRA-15814
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/cql3/Lists.java | 21 +-
src/java/org/apache/cassandra/cql3/Maps.java | 27 +-
src/java/org/apache/cassandra/cql3/Sets.java | 21 +-
.../validation/entities/FrozenCollectionsTest.java | 468 +
5 files changed, 450 insertions(+), 88 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 02de7c1..9034ae1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.17
+ * Fix CQL parsing of collections when the column type is reversed
(CASSANDRA-15814)
* Fix nomenclature of allow and deny lists (CASSANDRA-15862)
* Remove generated files from source artifact (CASSANDRA-15849)
* Remove duplicated tools binaries from tarballs (CASSANDRA-15768)
diff --git a/src/java/org/apache/cassandra/cql3/Lists.java
b/src/java/org/apache/cassandra/cql3/Lists.java
index cc75476..c6b78d7 100644
--- a/src/java/org/apache/cassandra/cql3/Lists.java
+++ b/src/java/org/apache/cassandra/cql3/Lists.java
@@ -21,20 +21,19 @@ import static
org.apache.cassandra.cql3.Constants.UNSET_VALUE;
import java.nio.ByteBuffer;
import java.util.ArrayList;
-import java.util.Collections;
import java.util.List;
import java.util.concurrent.atomic.AtomicReference;
-import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.ColumnDefinition;
import org.apache.cassandra.cql3.functions.Function;
import org.apache.cassandra.db.Cell;
import org.apache.cassandra.db.ColumnFamily;
import org.apache.cassandra.db.composites.CellName;
import org.apache.cassandra.db.composites.Composite;
-import org.apache.cassandra.db.composites.CompositesBuilder;
+import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.Int32Type;
import org.apache.cassandra.db.marshal.ListType;
+import org.apache.cassandra.db.marshal.ReversedType;
import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.serializers.CollectionSerializer;
import org.apache.cassandra.serializers.MarshalException;
@@ -57,7 +56,17 @@ public abstract class Lists
public static ColumnSpecification valueSpecOf(ColumnSpecification column)
{
-return new ColumnSpecification(column.ksName, column.cfName, new
ColumnIdentifier("value(" + column.name + ")", true),
((ListType)column.type).getElementsType());
+return new ColumnSpecification(column.ksName, column.cfName, new
ColumnIdentifier("value(" + column.name + ")", true),
elementsType(column.type));
+}
+
+private static AbstractType unwrap(AbstractType type)
+{
+return type.isReversed() ? unwrap(((ReversedType) type).baseType) :
type;
+}
+
+private static AbstractType elementsType(AbstractType type)
+{
+return ((ListType) unwrap(type)).getElementsType();
}
public static class Literal implements Term.Raw
@@ -94,7 +103,9 @@ public abstract class Lists
private void validateAssignableTo(String keyspace, ColumnSpecification
receiver) throws InvalidRequestException
{
-if (!(receiver.type instanceof ListType))
+AbstractType type = unwrap(receiver.type);
+
+if (!(type instanceof ListType))
throw new InvalidRequestException(String.format("Invalid list
literal for %s of type %s", receiver.name, receiver.type.asCQL3Type()));
ColumnSpecification valueSpec = Lists.valueSpecOf(receiver);
diff --git a/src/java/org/apache/cassandra/cql3/Maps.java
b/src/java/org/apache/cassandra/cql3/Maps.java
index 5bb3a48..8d21162 100644
--- a/src/java/org/apache/cassandra/cql3/Maps.java
+++ b/src/java/org/apache/cassandra/cql3/Maps.java
@@ -29,7 +29,9 @@ import org.apache.cassandra.cql3.functions.Function;
import org.apache.cassandra.db.ColumnFamily;
import org.apache.cassandra.db.composites.CellName;
import org.apache.cassandra.db.composites.Composite;
+import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.db.marshal.MapType;
+import org.apache.cassandra.db.marshal.ReversedType;
import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.serializers.CollectionSerializer;
import
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit fdeb254351afdd6efbe460967b8ee181f437b663 Merge: 91ccf24 7451cf0 Author: Andrés de la Peña AuthorDate: Thu Jul 23 12:55:15 2020 +0100 Merge branch 'cassandra-3.11' into trunk # Conflicts: # src/java/org/apache/cassandra/cql3/Lists.java # src/java/org/apache/cassandra/cql3/Maps.java # src/java/org/apache/cassandra/cql3/Sets.java # test/unit/org/apache/cassandra/cql3/validation/entities/FrozenCollectionsTest.java CHANGES.txt| 2 + src/java/org/apache/cassandra/cql3/Lists.java | 17 +- src/java/org/apache/cassandra/cql3/Maps.java | 25 +- src/java/org/apache/cassandra/cql3/Sets.java | 20 +- .../validation/entities/FrozenCollectionsTest.java | 461 + 5 files changed, 440 insertions(+), 85 deletions(-) diff --cc CHANGES.txt index 399b263,5f08069..0ab4548 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -5,67 -2,11 +5,69 @@@ Merged from 3.11 * Frozen RawTuple is not annotated with frozen in the toString method (CASSANDRA-15857) Merged from 3.0: * Fix empty/null json string representation (CASSANDRA-15896) + Merged from 2.2: + * Fix CQL parsing of collections when the column type is reversed (CASSANDRA-15814) -3.11.7 +4.0-beta1 + * Remove BackPressureStrategy (CASSANDRA-15375) + * Improve messaging on indexing frozen collections (CASSANDRA-15908) + * USING_G1 is incorrectly set in cassandra-env.sh if G1 is explicitly disabled with -UseG1GC (CASSANDRA-15931) + * Update compaction_throughput_mb_per_sec throttle default to 64 (CASSANDRA-14902) + * Add option to disable compaction at startup (CASSANDRA-15927) + * FBUtilities.getJustLocalAddress falls back to lo ip on misconfigured nodes (CASSANDRA-15901) + * Close channel and reduce buffer allocation during entire sstable streaming with SSL (CASSANDRA-15900) + * Prune expired messages less frequently in internode messaging (CASSANDRA-15700) + * Fix Ec2Snitch handling of legacy mode for dc names matching both formats, eg "us-west-2" (CASSANDRA-15878) + * Add support for server side DESCRIBE statements (CASSANDRA-14825) + * Fail startup if -Xmn is set when the G1 garbage collector is used (CASSANDRA-15839) + * generateSplits method replaced the generateRandomTokens for ReplicationAwareTokenAllocator. (CASSANDRA-15877) + * Several mbeans are not unregistered when dropping a keyspace and table (CASSANDRA-14888) + * Update defaults for server and client TLS settings (CASSANDRA-15262) + * Differentiate follower/initator in StreamMessageHeader (CASSANDRA-15665) + * Add a startup check to detect if LZ4 uses java rather than native implementation (CASSANDRA-15884) + * Fix missing topology events when running multiple nodes on the same network interface (CASSANDRA-15677) + * Create config.yml.MIDRES (CASSANDRA-15712) + * Fix handling of fully purged static rows in repaired data tracking (CASSANDRA-15848) + * Prevent validation request submission from blocking ANTI_ENTROPY stage (CASSANDRA-15812) + * Add fqltool and auditlogviewer to rpm and deb packages (CASSANDRA-14712) + * Include DROPPED_COLUMNS in schema digest computation (CASSANDRA-15843) + * Fix Cassandra restart from rpm install (CASSANDRA-15830) + * Improve handling of 2i initialization failures (CASSANDRA-13606) + * Add completion_ratio column to sstable_tasks virtual table (CASANDRA-15759) + * Add support for adding custom Verbs (CASSANDRA-15725) + * Speed up entire-file-streaming file containment check and allow entire-file-streaming for all compaction strategies (CASSANDRA-15657,CASSANDRA-15783) + * Provide ability to configure IAuditLogger (CASSANDRA-15748) + * Fix nodetool enablefullquerylog blocking param parsing (CASSANDRA-15819) + * Add isTransient to SSTableMetadataView (CASSANDRA-15806) + * Fix tools/bin/fqltool for all shells (CASSANDRA-15820) + * Fix clearing of legacy size_estimates (CASSANDRA-15776) + * Update port when reconnecting to pre-4.0 SSL storage (CASSANDRA-15727) + * Only calculate dynamicBadnessThreshold once per loop in DynamicEndpointSnitch (CASSANDRA-15798) + * Cleanup redundant nodetool commands added in 4.0 (CASSANDRA-15256) + * Update to Python driver 3.23 for cqlsh (CASSANDRA-15793) + * Add tunable initial size and growth factor to RangeTombstoneList (CASSANDRA-15763) + * Improve debug logging in SSTableReader for index summary (CASSANDRA-15755) + * bin/sstableverify should support user provided token ranges (CASSANDRA-15753) + * Improve logging when mutation passed to commit log is too large (CASSANDRA-14781) + * replace LZ4FastDecompressor with LZ4SafeDecompressor (CASSANDRA-15560) + * Fix buffer pool NPE with concurrent release due to in-progress tiny pool eviction (CASSANDRA-15726) + * Avoid race condition
[jira] [Commented] (CASSANDRA-14813) Crash frequently due to fatal error caused by "StubRoutines::updateBytesCRC32"
[
https://issues.apache.org/jira/browse/CASSANDRA-14813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163419#comment-17163419
]
Romain Anselin commented on CASSANDRA-14813:
Just an FYI we encountered this issue on JDK 8_241. The issue was due to faulty
disk sector so worth checking your syslog and run a SMART diag on the node if
this occur
> Crash frequently due to fatal error caused by "StubRoutines::updateBytesCRC32"
> --
>
> Key: CASSANDRA-14813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14813
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Environment: *OS:*
> {code:java}
> CentOS release 6.9 (Final){code}
> *JAVA:*
> {noformat}
> java version "1.8.0_101"
> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode){noformat}
>
> *Memory:*
> {noformat}
> 256GB{noformat}
> *CPU:*
> {noformat}
> 4* Intel® Xeon® Processor E5-2620 v4{noformat}
> *DISK:*
> {noformat}
> Filesystem Size Used Avail Use% Mounted on
> /dev/sda3 423G 28G 374G 7% /
> tmpfs 126G 68K 126G 1% /dev/shm
> /dev/sda1 240M 40M 188M 18% /boot
> /dev/sdb 3.7T 33M 3.7T 1% /mpp-data/c/cache
> /dev/sdc 3.7T 2.7T 984G 74% /mpp-data/c/data00
> /dev/sdd 3.7T 2.5T 1.2T 68% /mpp-data/c/data01
> /dev/sde 3.7T 2.7T 1.1T 72% /mpp-data/c/data02
> /dev/sdf 3.7T 2.5T 1.2T 68% /mpp-data/c/data03
> /dev/sdg 3.7T 2.4T 1.3T 66% /mpp-data/c/data04
> /dev/sdh 3.7T 2.6T 1.2T 69% /mpp-data/c/data05
> /dev/sdi 3.7T 2.6T 1.2T 70% /mpp-data/c/data06{noformat}
>Reporter: Jinchao Zhang
>Priority: Normal
> Labels: StubRoutines, crash, updateBytesCRC32
> Attachments: f3.png, f4.png, hs1.png, hs2.png, hs_err_pid26350.log
>
>
> Recently, we encountered the same problem described by CASSANDRA-14283 in our
> production system, which runs on Cassandra 3.11.2. We noticed that this issue
> has been resolved in Cassandra 3.11.3 (CASSANDRA-14284), thus we upgrade our
> system to 3.11.3. However, this induce more frequent crash，as shown in the
> following screenshots, and the reduced hs file is posted here as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163404#comment-17163404
]
Berenguer Blasi commented on CASSANDRA-15971:
-
Fqltool works for me under j8, j11, with and without your yaml. This is j11 +
your yaml:
{noformat}
Type: single-query
Query start time: 1595499287786
Protocol version: 4
Generated timestamp:-9223372036854775808
Generated nowInSeconds:1595499287
Query: select * from system.local where key = 'local'
Values:
Type: single-query
Query start time: 1595499287799
Protocol version: 4
Generated timestamp:-9223372036854775808
Generated nowInSeconds:1595499287
Query: SELECT * FROM system_schema.tables;
Values:
bereng@dxpc:~/work/repos/bdpWS/c15583/bin$ ./nodetool disablefullquerylog
bereng@dxpc:~/work/repos/bdpWS/c15583/bin$ java --version
openjdk 11.0.7 2020-04-14
OpenJDK Runtime Environment (build 11.0.7+10-post-Ubuntu-2ubuntu218.04)
OpenJDK 64-Bit Server VM (build 11.0.7+10-post-Ubuntu-2ubuntu218.04, mixed
mode, sharing)
bereng@dxpc:~/work/repos/bdpWS/c15583/bin$
{noformat}
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/fql
>Reporter: Jonathan Shook
>Priority: Normal
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15583) 4.0 quality testing: Tooling, Bundled and First Party
[ https://issues.apache.org/jira/browse/CASSANDRA-15583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15583: Description: Reference [doc from NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] for context. *Shepherd: Sam Tunnicliffe* Test plans should cover bundled first-party tooling and CLIs such as nodetool, cqlsh, and new tools supporting full query and audit logging (CASSANDRA-13983, CASSANDRA-12151). ||Tool||Has utests||Has dtest||J8/J11 coverage||Comments|| |Nodetool |(/)|(/)| | | |Cqlsh |(/)|(/)| | | |Cassandra-stress |(/)|(x)| | | |debug-cql |(x)|(x)| | | |fqltool|(/)|(/)| | | |auditlogviewer |(/)|(/)| | | |*Sstable utilities*| | | | | |sstabledump|(/)|(/)| | | |sstableexpiredblockers |(/)|(/)| | | |sstablelevelreset |(/)|(/)| | | |sstableloader |(/)|(/)| | | |sstablemetadata|(/)|(!)| |Ran in dtests, no dedicated test | |sstableofflinerelevel |(/)|(/)| | | |sstablerepairedset |(/)|(!)| |Ran in dtests, no dedicated test | |sstablescrub |(/)|(/)| | | |sstablesplit |(/)|(/)| | | |sstableupgrade |(/)|(/)| | | |sstableutil|(/)|(/)| | | |sstableverify |(/)|(/)| | | was: Reference [doc from NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] for context. *Shepherd: Sam Tunnicliffe* Test plans should cover bundled first-party tooling and CLIs such as nodetool, cqlsh, and new tools supporting full query and audit logging (CASSANDRA-13983, CASSANDRA-12151). > 4.0 quality testing: Tooling, Bundled and First Party > - > > Key: CASSANDRA-15583 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15583 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/unit >Reporter: Josh McKenzie >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > > Reference [doc from > NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] > for context. > *Shepherd: Sam Tunnicliffe* > Test plans should cover bundled first-party tooling and CLIs such as > nodetool, cqlsh, and new tools supporting full query and audit logging > (CASSANDRA-13983, CASSANDRA-12151). > ||Tool||Has utests||Has dtest||J8/J11 coverage||Comments|| > |Nodetool |(/)|(/)| | | > |Cqlsh|(/)|(/)| | | > |Cassandra-stress |(/)|(x)| | | > |debug-cql|(x)|(x)| | | > |fqltool |(/)|(/)| | | > |auditlogviewer |(/)|(/)| | | > |*Sstable utilities* | | | | | > |sstabledump |(/)|(/)| | | > |sstableexpiredblockers |(/)|(/)| | | > |sstablelevelreset|(/)|(/)| | | > |sstableloader|(/)|(/)| | | > |sstablemetadata |(/)|(!)| |Ran in dtests, no dedicated test | > |sstableofflinerelevel|(/)|(/)| | | > |sstablerepairedset |(/)|(!)| |Ran in dtests, no dedicated test | > |sstablescrub |(/)|(/)| | | > |sstablesplit |(/)|(/)| | | > |sstableupgrade |(/)|(/)| | | > |sstableutil |(/)|(/)| | | > |sstableverify|(/)|(/)| | | -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15502) In Tree Tooling with Java 11
[ https://issues.apache.org/jira/browse/CASSANDRA-15502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163378#comment-17163378 ] Berenguer Blasi commented on CASSANDRA-15502: - [~dcapwell] 100% agree with you. If you read the last messages in CASSANDRA-15583 this is what Sam and I are also coming to think. 15583 so far solves _being able_ to test tooling easily. But as you say coverage in most cases is minimal. I am going to go around tooling and report on coverage (50K foot view) to see what's the best way forward and any LHF. Adding 100% coverage to all tooling would be a mammoth multi-month thing imo, so we need to think how best to navigate that when we get to that bridge :-) > In Tree Tooling with Java 11 > > > Key: CASSANDRA-15502 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15502 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/fuzz, Tool/bulk load, Tool/cqlsh, > Tool/diff, Tool/fql, Tool/nodetool, Tool/sstable, Tool/stress >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > This is to cover testing the various tools running on java 11. > The scope of this testing is manual testing and not automated, different > JIRAs should cover automation testing these tool. > The tools in question are: nodetool, sstableloader, sstablescrub, > sstableupgrade, sstableutil, sstableverify, fqltool, stress, auditlogviewer, > compaction-stress, sstabledump, sstableexpiredblockers, sstablemetadata, > sstableofflinerelevel, sstablesplit, and sstablerepairedset (many of these > may be tested already in dtest) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15502) In Tree Tooling with Java 11
[ https://issues.apache.org/jira/browse/CASSANDRA-15502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163378#comment-17163378 ] Berenguer Blasi edited comment on CASSANDRA-15502 at 7/23/20, 9:27 AM: --- [~dcapwell] 100% agree with you. If you read the last messages in CASSANDRA-15583 this is what Sam and I are also coming to think. 15583 so far solves _being able_ to test tooling easily. But as you say coverage in most cases is minimal. I am going to go around tooling and report on coverage (50K foot view) to see what's the best way forward and any LHF. Adding 100% coverage to all tooling would be a mammoth multi-month thing imo, so we need to think how best to navigate that when we get to that bridge :-) (Edit: I suggest we move the conversation to 15583) was (Author: bereng): [~dcapwell] 100% agree with you. If you read the last messages in CASSANDRA-15583 this is what Sam and I are also coming to think. 15583 so far solves _being able_ to test tooling easily. But as you say coverage in most cases is minimal. I am going to go around tooling and report on coverage (50K foot view) to see what's the best way forward and any LHF. Adding 100% coverage to all tooling would be a mammoth multi-month thing imo, so we need to think how best to navigate that when we get to that bridge :-) > In Tree Tooling with Java 11 > > > Key: CASSANDRA-15502 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15502 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/fuzz, Tool/bulk load, Tool/cqlsh, > Tool/diff, Tool/fql, Tool/nodetool, Tool/sstable, Tool/stress >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > This is to cover testing the various tools running on java 11. > The scope of this testing is manual testing and not automated, different > JIRAs should cover automation testing these tool. > The tools in question are: nodetool, sstableloader, sstablescrub, > sstableupgrade, sstableutil, sstableverify, fqltool, stress, auditlogviewer, > compaction-stress, sstabledump, sstableexpiredblockers, sstablemetadata, > sstableofflinerelevel, sstablesplit, and sstablerepairedset (many of these > may be tested already in dtest) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15971) full query log needs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163373#comment-17163373
]
Berenguer Blasi commented on CASSANDRA-15971:
-
In CASSANDRA-15583 we will be introducing easy tooling testing. Any tooling
improvements could benefit from that #collaborating #justfyi
> full query log needs improvement
>
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/fql
>Reporter: Jonathan Shook
>Priority: Normal
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench
> usage, I ran across many issues which would make it painful for users. Since
> there were several, they will be added to a single issue for now. This issue
> can be broken up if needed.
>
> FQL doesn't work on my system, even though it says it is logging queries.
> With the following configuration in cassandra.yaml:
>
> {noformat}
> full_query_logging_options:
> log_dir: /REDACTED/fullquerylogs
> roll_cycle: HOURLY
> block: true
> max_queue_weight: 268435456 # 256 MiB
> max_log_size: 17179869184 # 16 GiB
> ## archive command is "/path/to/script.sh %path" where %path is replaced
> with the file being rolled:
> # archive_command:
> # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a
> single file `directory-listing.cq4t` is created, which is a 64K sized file of
> zeroes.
>
>
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long)
> -- StackTrace --
> java.lang.NoSuchMethodException:
> sun.nio.ch.FileChannelImpl.map0(int,long,long)
> at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553)
> at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51)
> at
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>
> Subsequent calls produce normal output:
>
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs
> See 'nodetool help' or 'nodetool help '.{noformat}
>
>
> nodetool missing getfullquerylog makes it difficult to verify current
> fullquerylog state without changing it. The conventions for nodetool commands
> should be followed to avoid confusing users.
>
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating
> the default The most commonly used fqltool commands are..
>
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is
> malformatted, mixing the appearance of configuration with comments, which is
> confusing at best.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163372#comment-17163372
]
Alex Petrov commented on CASSANDRA-15299:
-
Thanks Olivier, you're right this one is fixed. I was using other branch to
hunt down the leaks and never switched back.
I did some basic performance tests, and it looks like new v5 outperforms always
outperforms old v4 and v5, and performs either on par with a new v4, or better
than it, in most of scenarios. Scenarios where v4 seems to perform better that
I was able to produce were with a single connected client. There, with a small
number of concurrent async operations, v4 was performing better. However, this
might not be such an important scenario. In general, the more we can use
batching, the better results seem to be. At the same time, checksumming doesn't
seem to negatively contribute to V5 performance.
This is a very basic benchmark, since server and client run on the same
machine, so there's definitely some interference. What might be good about this
test, Cassandra read/write path is mocked out to avoid further interference.
Scenario 1: requestSize = 1197, responseSize = 234, 10 connected clients, 1
async operation per batch:
{code:java}
V5
Variance: 135333.13542032443
Median: 155.0
90p: 159.0
95p: 160.0
99p: 160.0
V4
Variance: 107837.257938666
Median: 153.0
90p: 158.0
95p: 158.0
99p: 158.0
{code}
Scenario 2: requestSize = 1197, responseSize = 234, 10 connected clients, 5
async operation per batch:
{code:java}
Variance: 2.3154951046883754E7
Median: 183.0
90p: 190.0
95p: 191.0
99p: 192.0
Variance: 9145882.574275833
Median: 196.0
90p: 203.0
95p: 204.0
99p: 204.0
{code}
Scenario 3: requestSize = 1197, responseSize = 234, 10 connected clients, 10
async operation per batch:
{code:java}
V5
Variance: 6.080661284308405E7
Median: 182.0
90p: 190.0
95p: 191.0
99p: 191.0
Variance: 7.45228262686024E7
Median: 218.0
90p: 229.0
95p: 230.0
99p: 231.0
{code}
Scenario 4: requestSize = 1515743, responseSize = 415145, 10 connected clients,
1 async operation per batch:
{code:java}
V5
Variance: 3.809988628626683E7
Median: 2442.0
90p: 2482.0
95p: 2487.0
99p: 2489.2995
V4
Variance: 8.865722070719789E7
Median: 3083.0
90p: 3129.0
95p: 3134.0
99p: 3136.0
{code}
Here, we have even a larger boost from V5, which is great, and I think this was
exactly what was aimed for. I haven't tested multiple large ops per batch, but
I also don't see many use-cases for such scenario.
One of the things we probably want to improve is to avoid copying from
{{Frame}} to outgoing buffers. We can use something in the spirit of
{{SegmentBuilder}} that driver uses, and simply write frames directly to the
outgoing buffer. It looks like this part may simplify memory management, since
we'll have one thing less to worry about. Similar improvement can be done on
the incoming path, but here it might be trickier to do so since we're reusing
the code from {{net}} package. It looks like reusing code from {{net}} has
actually made some of the things a bit harder to follow.
Speaking of {{Segment}} in driver, we probably want to use naming that is
consistent with server.
One other comment I wanted to leave is on {{FlushItem}} and releasing things.
Since FlushItem generally has two parts: incoming message (one we're responding
to), and response one, it might make sense to release them together. I haven't
attempted doing this in my patch, since I thought it might be better if we
combine this change with copy-avoiding suggestion mentioned above. But in
general, knowing that two logical parts are released together might make it all
easier to follow.
That said, a simple measurement has shown that this copying does not dominate
the picture. In fact, simple profiling hasn't shown any obvious hotspots. We
can (and probably should) open a separate ticket that could aim at performance
improvements around native protocol.
Some of the things I haven't tested which might be important to check that I
unfortunately didn't have enough time to:
* rapid and frequent disconnects and reconnects
* sending and receiving messages other than QueryMessage (such as Options, etc)
* backpressure on overload
* behaviour in presence of delays, disconnects, and corruption (we also have a
netty Proxy for that)
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>
[jira] [Commented] (CASSANDRA-14200) NullPointerException when dumping sstable with null value for timestamp column
[
https://issues.apache.org/jira/browse/CASSANDRA-14200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163366#comment-17163366
]
Berenguer Blasi commented on CASSANDRA-14200:
-
[~szhou] are you ok closing this one as duplicate of CASSANDRA-15896?
> NullPointerException when dumping sstable with null value for timestamp column
> --
>
> Key: CASSANDRA-14200
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14200
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>Priority: Normal
> Fix For: 3.0.x
>
>
> We have an sstable whose schema has a column of type timestamp and it's not
> part of primary key. When dumping the sstable using sstabledump there is NPE
> like this:
> {code:java}
> Exception in thread "main" java.lang.NullPointerException
> at java.util.Calendar.setTime(Calendar.java:1770)
> at java.text.SimpleDateFormat.format(SimpleDateFormat.java:943)
> at java.text.SimpleDateFormat.format(SimpleDateFormat.java:936)
> at java.text.DateFormat.format(DateFormat.java:345)
> at
> org.apache.cassandra.db.marshal.TimestampType.toJSONString(TimestampType.java:93)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:442)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:376)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:280)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:215)
> at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:104)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242){code}
> The reason is that we use a null Date when there is no value for this column:
> {code}
> public Date deserialize(ByteBuffer bytes)
> {
> return bytes.remaining() == 0 ? null : new
> Date(ByteBufferUtil.toLong(bytes));
> }
> {code}
> It seems that we should not deserialize columns with null values.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15579) 4.0 quality testing: Distributed Read/Write Path: Coordination, Replication, and Read Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-15579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163365#comment-17163365 ] Benedict Elliott Smith commented on CASSANDRA-15579: Just to assist the conversation, here's a (slightly edited) cut/paste from the doc linked in the description, from NGCC, relevant to this ticket: # Upgrade / Diff Testing (cassandra-diff) and Replay Testing # Audit ** IndexInfo changes in 3.7 (source review) ** File Transaction log # Cluster Fuzz / Harry # Targeted Fuzz ** Memtable vs sstables ** Compaction *** +JBOD / CASSANDRA-6696 # Unit tests ** Dropping tombstones # Chunk cache We should probably triage this list in some way, so here's my starting contribution to annotate the above list: # Diff/Replay: My impression is that these are activities for major contributing companies to assert have been done, rather than development work? It's possible Fallout related work at DataStax would be relevant here. # Audit ** IndexInfo: I think this may have been conducted already by [~jwest]? ** File Transaction Log: I'm not certain what is hoped to be achieved by the file transaction log audit, except perhaps some simplification, but perhaps instead we should move that to the targeted fuzz heading, so we can simply build some strong confidence in it as a feature (given it is exceptionally important to correctness)? # Harry: Hopefully should be arriving soon in some form, but either way this can be considered out of scope for current planning # Targeted Fuzz ** Memtable/SSTable: [~bdeggleston] is there some work here that can (or already has been) OSS'd? Or is this better to move under the Cluster Fuzz / Harry heading? ** Compaction: This could be a big piece of work, in which we presumably aim to model compaction correctness generally, or could simply be a variant of the Memtable vs SSTable work in which we verify that compaction always produces the same results as e.g. no compaction *** JBOD: This probably requires its own discussion - I assume this is related to the correctness issues of losing partial data for a partition # Unit tests - any other areas missing? ** Tombstones: Uncertain if this should be a unit test, or integrated into any compaction targeted fuzz test, or both # Chunk Cache: Presumably this wants another targeted fuzz test, but perhaps a relatively easy one given the simple mechanics? It might be nice to also introduce some simple microbenchmarks here too. > 4.0 quality testing: Distributed Read/Write Path: Coordination, Replication, > and Read Repair > > > Key: CASSANDRA-15579 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15579 > Project: Cassandra > Issue Type: Task > Components: Test/unit >Reporter: Josh McKenzie >Assignee: Andres de la Peña >Priority: Normal > Fix For: 4.0-beta > > > Reference [doc from > NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#] > for context. > *Shepherd: Blake Eggleston* > Testing in this area focuses on non-node-local aspects of the read-write > path: coordination, replication, read repair, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15896) NullPointerException in SELECT JSON statement when a UUID field contains an empty string
[
https://issues.apache.org/jira/browse/CASSANDRA-15896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163361#comment-17163361
]
Berenguer Blasi commented on CASSANDRA-15896:
-
meh, probably GH being silly with forward merges again on {{.patch}}. If you
use {{.diff}} you should get what you need next time.
> NullPointerException in SELECT JSON statement when a UUID field contains an
> empty string
>
>
> Key: CASSANDRA-15896
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15896
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter, CQL/Semantics
>Reporter: Ostico
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 3.0.22, 3.11.8, 4.0-beta2
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> It seems that Cassandra accept empty strings "" ( FROM JSON string ) for UUID
> fields but crash when asking for JSON serialization of those fields.
>
> Cassandra version 3.6.11.6 running in docker from official Dockerhub image.
> Java driver:
> {code:java}
>
>
> com.datastax.oss
> java-driver-core
> 4.7.0
>
> {code}
> The attached code is to allow bug reproducibility:
> {code:java}
> package com.foo.bar;
> import com.datastax.oss.driver.api.core.CqlSession;
> import com.datastax.oss.driver.api.core.CqlSessionBuilder;
> import com.datastax.oss.driver.api.core.cql.PreparedStatement;
> import com.datastax.oss.driver.api.core.cql.ResultSet;
> import com.datastax.oss.driver.api.core.cql.Row;
> import com.fasterxml.jackson.databind.ObjectMapper;
> import org.junit.After;
> import org.junit.Before;
> import org.junit.Test;
> import java.net.InetSocketAddress;
> import java.net.URI;
> import java.util.*;
> import static org.junit.Assert.assertFalse;
> import static org.junit.Assert.assertNotNull;
> /**
> * @author Domenico Lupinetti - 23/06/2020
> */
> public class NullPointerExceptionTest {
> protected String uuid;
> protected CqlSession cqlSession;
> @Before
> public void setUp() throws Exception {
> URI node = new URI( "tcp://localhost:9042" );
> final CqlSessionBuilder builder = CqlSession.builder();
> cqlSession = builder.addContactPoint( new InetSocketAddress(
> node.getHost(),
> node.getPort()
> ) ).withLocalDatacenter( "datacenter1" ).build();
> cqlSession.execute( "CREATE KEYSPACE IF NOT EXISTS test_suite WITH
> replication = {'class':'SimpleStrategy','replication_factor':1};" );
> String sb = "CREATE TABLE IF NOT EXISTS test_suite.test ( id uuid
> PRIMARY KEY, another_id uuid, subject text );";
> cqlSession.execute( sb );
> PreparedStatement stm = cqlSession.prepare( "INSERT INTO
> test_suite.test JSON :payload" );
> this.uuid = UUID.randomUUID().toString();
> HashMap payload = new HashMap<>();
> payload.put( "id", this.uuid );
> // *** This exception do not happens if the field is set as NULL
> payload.put( "another_id", "" ); //<-- EMPTY STRING AS UUID
> payload.put( "subject", "Alighieri, Dante. Divina Commedia" );
> ObjectMapper objM = new ObjectMapper();
> cqlSession.execute(
> stm.bind().setString( "payload", objM.writeValueAsString(
> payload ) )
> ); //<-- serialize as JSON
> }
> @After
> public void tearDown() throws Exception {
> cqlSession.execute( "DROP TABLE IF EXISTS test_suite.test;" );
> cqlSession.execute( "DROP KEYSPACE test_suite;" );
> cqlSession.close();
> }
> @Test
> public void testNullPointer() {
> PreparedStatement stmt = cqlSession.prepare( "SELECT JSON id,
> another_id FROM test_suite.test where id = :id;" );
> ResultSet resultSet = cqlSession.execute(
> stmt.bind().setUuid( "id", UUID.fromString( this.uuid ) ) ); // <--
> EXCEPTION
> Row r = resultSet.one();
> assertNotNull( r );
> assertNotNull( r.getString( "[json]" ) );
> assertFalse( Objects.requireNonNull( r.getString( "[json]" )
> ).isEmpty() );
> }
> }
> {code}
> Client stack Trace:
> {code:java}
> com.datastax.oss.driver.api.core.servererrors.ServerError:
> java.lang.NullPointerExceptioncom.datastax.oss.driver.api.core.servererrors.ServerError:
> java.lang.NullPointerException
> at
> com.datastax.oss.driver.api.core.servererrors.ServerError.copy(ServerError.java:54)
> at
> com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
> at
> com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53)
> at
>
[jira] [Commented] (CASSANDRA-15814) order by descending on frozen list not working
[
https://issues.apache.org/jira/browse/CASSANDRA-15814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17163354#comment-17163354
]
Berenguer Blasi commented on CASSANDRA-15814:
-
[~adelapena] thx for putting up with my OCD and your extensive research :-) +1
lgtm
> order by descending on frozen list not working
> --
>
> Key: CASSANDRA-15814
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15814
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Felipe Perez
>Assignee: Andres de la Peña
>Priority: Normal
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> By creating a table like the following:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> )
> {code}
> It works and version is ordered in an ascending order. But when trying to
> order in descending order:
> {code:java}
> CREATE TABLE IF NOT EXISTS software (
> name ascii,
> version frozen>,
> data ascii,
> PRIMARY KEY(name,version)
> ) WITH CLUSTERING ORDER BY (version DESC);
> {code}
> The table is created normally, but when trying to insert a row:
> {code:java}
> insert into software(name, version) values ('t1', [2,10,30,40,50]);
> {code}
> Cassandra throws an error:
> {code:java}
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Invalid
> list literal for version of type frozen>"
> {code}
> The goal here is that I would like to get the last version of a software.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
