[jira] [Commented] (CASSANDRA-16803) jvm-dtest-upgrade failing MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck, ClassNotFoundException: com.vdurmont.semver4j.Semver
[ https://issues.apache.org/jira/browse/CASSANDRA-16803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412952#comment-17412952 ] Michael Semb Wever commented on CASSANDRA-16803: Vote passed: https://lists.apache.org/thread.html/rd37c6b37f196ca080c2647e3b4538ed25ad9ced0980dfe9c9593438b%40%3Cdev.cassandra.apache.org%3E > jvm-dtest-upgrade failing > MixedModeReadTest.mixedModeReadColumnSubsetDigestCheck, > ClassNotFoundException: com.vdurmont.semver4j.Semver > -- > > Key: CASSANDRA-16803 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16803 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x > > > Caused by CASSANDRA-16649 > Reproducible locally. Oddly enough can be reproduced on cassandra-4.0 branch > as well, though CI is not failing. > fyi [~ifesdjeen] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16941) NullPointerException on nodetool status
[
https://issues.apache.org/jira/browse/CASSANDRA-16941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412896#comment-17412896
]
Mohsen Zohrevandi commented on CASSANDRA-16941:
---
and the nodetool status failure occurred around 2021-09-10 00:08:21
> NullPointerException on nodetool status
> ---
>
> Key: CASSANDRA-16941
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16941
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mohsen Zohrevandi
>Priority: Normal
>
> Sometimes {{nodetool status}} fails with the following error:
> {code:java}
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:4871)
> at
> org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The cluster is running Cassandra 3.11.10 and nothing in Cassandra logs
> indicates any errors. The particular node where the error happened was
> running for 7 days. I believe the same NPE happens randomly on other nodes as
> well.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16941) NullPointerException on nodetool status
[
https://issues.apache.org/jira/browse/CASSANDRA-16941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412894#comment-17412894
]
Mohsen Zohrevandi commented on CASSANDRA-16941:
---
Here are the last few lines of logs before the pod was restarted (nodetool
status was used in a Kubernetes livenessProbe that failed due to NPE):
{code:java}
INFO [GossipTasks:1] 2021-09-09 23:59:32,486 Gossiper.java:894 - FatClient
/10.244.47.73 has been silent for 3ms, removing from gossip
INFO [Service Thread] 2021-09-10 00:00:11,050 GCInspector.java:285 - ParNew GC
in 373ms. CMS Old Gen: 2433961072 -> 2495651632; Par Eden Space: 335544320 ->
0; Par Survivor Space: 35273304 -> 41943040
INFO [Service Thread] 2021-09-10 00:00:11,975 GCInspector.java:285 - ParNew GC
in 426ms. CMS Old Gen: 2495652488 -> 2571204104; Par Eden Space: 335544320 ->
0;
INFO [Service Thread] 2021-09-10 00:00:12,880 GCInspector.java:285 - ParNew GC
in 389ms. CMS Old Gen: 2571204264 -> 2643651528; Par Eden Space: 335544320 ->
0;
INFO [Service Thread] 2021-09-10 00:00:13,649 GCInspector.java:285 - ParNew GC
in 339ms. CMS Old Gen: 2643651528 -> 2710161032; Par Eden Space: 333721088 ->
0;
INFO [HintsDispatcher:129] 2021-09-10 00:03:16,564 HintsStore.java:133 -
Deleted hint file 770229d7-1412-4d46-8a2f-50fdf51df82c-1631216119494-1.hints
INFO [HintsDispatcher:129] 2021-09-10 00:03:16,564
HintsDispatchExecutor.java:282 - Finished hinted handoff of file
770229d7-1412-4d46-8a2f-50fdf51df82c-1631216119494-1.hints to endpoint
/10.244.38.27: 770229d7-1412-4d46-8a2f-50fdf51df82c
INFO [HintsDispatcher:128] 2021-09-10 00:04:32,423 HintsStore.java:133 -
Deleted hint file c67cdbd7-194c-48df-b638-7b33f88c173b-1631217661409-1.hints
INFO [HintsDispatcher:128] 2021-09-10 00:04:32,423
HintsDispatchExecutor.java:282 - Finished hinted handoff of file
c67cdbd7-194c-48df-b638-7b33f88c173b-1631217661409-1.hints to endpoint
/10.244.46.145: c67cdbd7-194c-48df-b638-7b33f88c173b
{code}
> NullPointerException on nodetool status
> ---
>
> Key: CASSANDRA-16941
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16941
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mohsen Zohrevandi
>Priority: Normal
>
> Sometimes {{nodetool status}} fails with the following error:
> {code:java}
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:4871)
> at
> org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at
[jira] [Commented] (CASSANDRA-16941) NullPointerException on nodetool status
[
https://issues.apache.org/jira/browse/CASSANDRA-16941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412888#comment-17412888
]
Brandon Williams commented on CASSANDRA-16941:
--
Can you upload some logs from around this time?
> NullPointerException on nodetool status
> ---
>
> Key: CASSANDRA-16941
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16941
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mohsen Zohrevandi
>Priority: Normal
>
> Sometimes {{nodetool status}} fails with the following error:
> {code:java}
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> at
> org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:4871)
> at
> org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
> at sun.rmi.transport.Transport$1.run(Transport.java:200)
> at sun.rmi.transport.Transport$1.run(Transport.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> The cluster is running Cassandra 3.11.10 and nothing in Cassandra logs
> indicates any errors. The particular node where the error happened was
> running for 7 days. I believe the same NPE happens randomly on other nodes as
> well.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16941) NullPointerException on nodetool status
Mohsen Zohrevandi created CASSANDRA-16941:
-
Summary: NullPointerException on nodetool status
Key: CASSANDRA-16941
URL: https://issues.apache.org/jira/browse/CASSANDRA-16941
Project: Cassandra
Issue Type: Bug
Reporter: Mohsen Zohrevandi
Sometimes {{nodetool status}} fails with the following error:
{code:java}
error: null
-- StackTrace --
java.lang.NullPointerException
at
org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:4871)
at
org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:115)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
The cluster is running Cassandra 3.11.10 and nothing in Cassandra logs
indicates any errors. The particular node where the error happened was running
for 7 days. I believe the same NPE happens randomly on other nodes as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16666) Make SSLContext creation pluggable/extensible
[ https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412880#comment-17412880 ] Maulin Vasavada commented on CASSANDRA-1: - Hi [~mck] can you please take a look at the documentation question here by Stefan when you get a chance? I think I am working on one last code enhancement with [~jmeredithco] (about allowing the ssl context cache purge by a custom ssl-context-factory's implementation) but that can be wrapped up soon now since other comments are resolved. > Make SSLContext creation pluggable/extensible > - > > Key: CASSANDRA-1 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Maulin Vasavada >Assignee: Maulin Vasavada >Priority: Normal > Fix For: 4.x > > > Currently Cassandra creates the SSLContext via SSLFactory.java. SSLFactory is > a final class with static methods and not overridable. The SSLFactory loads > the keys and certs from the file based artifacts for the same. While this > works for many, in the industry where security is stricter and contextual, > this approach falls short. Many big organizations need flexibility to load > the SSL artifacts from a custom resource (like custom Key Management > Solution, HashiCorp Vault, Amazon KMS etc). While JSSE SecurityProvider > architecture allows us flexibility to build our custom mechanisms to validate > and process security artifacts, many times all we need is to build upon > Java's existing extensibility that Trust/Key Manager interfaces provide to > load keystores from various resources in the absence of any customized > requirements on the Keys/Certificate formats. > My proposal here is to make the SSLContext creation pluggable/extensible and > have the current SSLFactory.java implement an extensible interface. > I contributed a similar change that is live now in Apache Kafka (2.6.0) - > https://issues.apache.org/jira/browse/KAFKA-8890 > I can spare some time writing the pluggable interface and run by the required > reviewers. > > Created [CEP-9: Make SSLContext creation > pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable] > > > cc: [~dcapwell] [~djoshi] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16909) ☂ Medium Term Repair Improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16909:
--
Description:
This is to track the repair improvement works defined on the dev list.
Email was: [DISCUSS] Repair Improvement Proposal
This JIRA will track the related tasks and be used for documentaiton
{code}
Repair Coordinator State
1. ActiveRepairService.recordRepairStatus
1. Maps JMX command (int) to (status, msg)
2. Get/validate ColumnFamilies
1. Skip if no ColumnFamilies to repair
3. Get replicas for ranges
1. Ignore ranges not covered by this instance IFF
--ignore-unreplicated-keyspaces, else Fail
2. Skip if no ranges to repair
3. If --force filter out non-alive (FailureDetector.isAlive) participants
4. [Not PREVIEW]Update SystemDistributedKeyspace's parent_repair_history
5. ActiveRepairService.prepareForRepair
1. TODO Should this be under PREPARE or still part of validation?
1. If CompactionsPendingThreshold triggered, Fail
2. registerParentRepairSession - update map of UUID ->
ParentRepairSession
2. Send PREPARE_MSG awaiting responses [possible failures: timeout waiting,
participate failure]
1. [improvement] PREPARE_MSG should be idempotent and if no reply
within X, retry Y times
2. Known Failures; all are retryable at this stage
1. Timeout
2. Participate goes from alive to down
3. CompactionsPendingThreshold triggered
1. Not included in
org.apache.cassandra.exceptions.RequestFailureReason#forException, so looks the
same as Unexpected error
1. If updated and in mixed-mode, the update falls back to
UNKNOWN, which then matches the unexpected error behavior
4. Unexpected error (this removes the session)
6. Run RepairTask (normal, IR, preview); see coordinator state for each type
7. On Success
1. Update SystemDistributedKeyspace.parent_repair_history to show the
successful ranges
2. If any sub-session failed, fail the job
3. ActiveRepairService.cleanUp - send CLEANUP_MSG to all participates to
clean up
1. TODO: why is this only on success and not failures as well?
2. [improvement] - does not wait for ACK (though it is sent), we log it
async if we get it (else timeout)
8. On Exception
1. fail
Normal/Preview Repair Coordinator State
1. For each common range
1. ActiveRepairService.submitRepairSession
1. Creates/run a RepairSession for each CommonRange
2. Once all RepairSessions done
1. [not consistent cross each type] handle session errors
2. [Preview Repair] check SyncStatSummary for difference and send a
human readable notification
Incremental Repair Coordinator State
1. org.apache.cassandra.repair.consistent.CoordinatorSessions#registerSession
to create CoordinatorSession
2. CoordinatorSession.execute
1. Send PREPARE_CONSISTENT_REQ and wait for PREPARE_CONSISTENT_RSP
2. Await all success
3. Trigger Normal Repair (see "Normal/Preview Repair Coordinator State")
4. Send FINALIZE_PROPOSE_MSG and wait for FINALIZE_PROMISE_MSG
5. Await all success
6. Send FINALIZE_COMMIT_MSG
1. [bug][improvement] No ACK is done, so if this message is dropped
then some participates will think the IR is still running (which can fail
preview repairs)
RepairSession
1. [Preview Repair - kind=REPAIRED] register with LocalSessions for IR state
changes
2. RepairSession.start
1. [Not Preview Repair] Registering the session into
SystemDistributedKeyspace's table repair_history
2. If endpoints is empty
1. [UNHANDLED - downstream logic does not handle this case] Set future
with empty state (which is later seen as Failed... but not a failed future)
2. [Not Preview Repair] Mark session failed in repair_history
3. Check all endpoints, if any is down and hasSkippedReplicas=false, Fail
the session
4. For each table
1. Create a RepairJob
2. Execute job in RepairTask's executor
3. await all jobs
1. If all success
1. Set session result to include the job results
2. If any fail
1. Fail the session future
2. [Question] why does this NOT update repair_history like
other failures?
RepairJob
1. [parallelism in [SEQUENTIAL, DATACENTER_AWARE] and not IR] send SNAPSHOT_MSG
to all participates
1. [improvement] SNAPSHOT_MSG should be idempotent so coordinator can retry
if no ACK. This calls org.apache.cassandra.repair.TableRepairManager#snapshot
with the RepairRunnable ID; this makes sure to snapshot once unless force=true
(RepairOptions !(dataCenters.isEmpty and hosts.isEmpty()))
2. [improvement] This task is short lived, so rather than running in a
cached pool (which may allocate a thread), inline or add a notation of
cooperative
[jira] [Commented] (CASSANDRA-16909) ☂ Medium Term Repair Improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412879#comment-17412879
]
David Capwell commented on CASSANDRA-16909:
---
Preview and IR updated
> ☂ Medium Term Repair Improvements
> -
>
> Key: CASSANDRA-16909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16909
> Project: Cassandra
> Issue Type: Epic
> Components: Consistency/Repair
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>
> This is to track the repair improvement works defined on the dev list.
> Email was: [DISCUSS] Repair Improvement Proposal
> This JIRA will track the related tasks and be used for documentaiton
> {code}
> Repair Coordinator State
> 1. ActiveRepairService.recordRepairStatus
> 1. Maps JMX command (int) to (status, msg)
> 2. Get/validate ColumnFamilies
> 1. Skip if no ColumnFamilies to repair
> 3. Get replicas for ranges
> 1. Ignore ranges not covered by this instance IFF
> --ignore-unreplicated-keyspaces, else Fail
> 2. Skip if no ranges to repair
> 3. If --force filter out non-alive (FailureDetector.isAlive) participants
> 4. [Not PREVIEW]Update SystemDistributedKeyspace's parent_repair_history
> 5. ActiveRepairService.prepareForRepair
> 1. TODO Should this be under PREPARE or still part of validation?
> 1. If CompactionsPendingThreshold triggered, Fail
> 2. registerParentRepairSession - update map of UUID ->
> ParentRepairSession
> 2. Send PREPARE_MSG awaiting responses [possible failures: timeout
> waiting, participate failure]
> 1. [improvement] PREPARE_MSG should be idempotent and if no reply
> within X, retry Y times
> 2. Known Failures; all are retryable at this stage
> 1. Timeout
> 2. Participate goes from alive to down
> 3. CompactionsPendingThreshold triggered
> 1. Not included in
> org.apache.cassandra.exceptions.RequestFailureReason#forException, so looks
> the same as Unexpected error
> 1. If updated and in mixed-mode, the update falls back to
> UNKNOWN, which then matches the unexpected error behavior
> 4. Unexpected error (this removes the session)
> 6. Run RepairTask (normal, IR, preview); see coordinator state for each type
> 7. On Success
> 1. Update SystemDistributedKeyspace.parent_repair_history to show the
> successful ranges
> 2. If any sub-session failed, fail the job
> 3. ActiveRepairService.cleanUp - send CLEANUP_MSG to all participates to
> clean up
> 1. TODO: why is this only on success and not failures as well?
> 2. [improvement] - does not wait for ACK (though it is sent), we log
> it async if we get it (else timeout)
> 8. On Exception
> 1. fail
> Normal/Preview Repair Coordinator State
> 1. For each common range
> 1. ActiveRepairService.submitRepairSession
> 1. Creates/run a RepairSession for each CommonRange
> 2. Once all RepairSessions done
> 1. [not consistent cross each type] handle session errors
> 2. [Preview Repair] check SyncStatSummary for difference and send a
> human readable notification
> Incremental Repair Coordinator State
> 1. org.apache.cassandra.repair.consistent.CoordinatorSessions#registerSession
> to create CoordinatorSession
> 2. CoordinatorSession.execute
> 1. Send PREPARE_CONSISTENT_REQ and wait for PREPARE_CONSISTENT_RSP
> 2. Await all success
> 3. Trigger Normal Repair (see "Normal/Preview Repair Coordinator State")
> 4. Send FINALIZE_PROPOSE_MSG and wait for FINALIZE_PROMISE_MSG
> 5. Await all success
> 6. Send FINALIZE_COMMIT_MSG
> 1. [bug][improvement] No ACK is done, so if this message is dropped
> then some participates will think the IR is still running (which can fail
> preview repairs)
> RepairSession
> 1. [Preview Repair - kind=REPAIRED] register with LocalSessions for IR state
> changes
> 2. RepairSession.start
> 1. [Not Preview Repair] Registering the session into
> SystemDistributedKeyspace's table repair_history
> 2. If endpoints is empty
> 1. [UNHANDLED - downstream logic does not handle this case] Set
> future with empty state (which is later seen as Failed... but not a failed
> future)
> 2. [Not Preview Repair] Mark session failed in repair_history
> 3. Check all endpoints, if any is down and hasSkippedReplicas=false, Fail
> the session
> 4. For each table
> 1. Create a RepairJob
> 2. Execute job in RepairTask's executor
> 3. await all jobs
> 1. If all success
> 1. Set session result to include the job results
> 2. If any fail
> 1. Fail the session future
> 2. [Question] why does this NOT
[jira] [Commented] (CASSANDRA-16718) Changing listen_address with prefer_local may lead to issues
[ https://issues.apache.org/jira/browse/CASSANDRA-16718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412872#comment-17412872 ] Brandon Williams commented on CASSANDRA-16718: -- What I have discovered is that with all persistence removed and every call relying what information the Gossiper has for the node, this will _still_ use the old address, because only a SYN has been received from the updated node, so checks on the internal state will use the stale information. This is going to be trickier than I suspected. > Changing listen_address with prefer_local may lead to issues > > > Key: CASSANDRA-16718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16718 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Jan Karlsson >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.11.x, 4.0.x > > > Many container based solution function by assigning new listen_addresses when > nodes are stopped. Changing the listen_address is usually as simple as > turning off the node and changing the yaml file. > However, if prefer_local is enabled, I observed that nodes were unable to > join the cluster and fail with 'Unable to gossip with any seeds'. > Trace shows that the changing node will try to communicate with the existing > node but the response is never received. I assume it is because the existing > node attempts to communicate with the local address during the shadow round. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16940) Confusing ProtocolException msg Invalid or unsupported protocol version (4)
[
https://issues.apache.org/jira/browse/CASSANDRA-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad Schoening updated CASSANDRA-16940:
---
Description:
The following warning was seen frequently after upgrading from 3.0.x to 3.11.11
in the cassandra.log:
{noformat}
ProtocolException: Invalid or unsupported protocol version (4); supported
versions are (3/v3, 4/v4, 5/v5-beta){noformat}
It is at best unclear, or maybe a bug in the code throwing this exception
stating version '4' not supported but 4/v4 is.
from org/apache/cassandra/transport/ProtocolVersion.java
public static String invalidVersionMessage(int version)
{ return String.format("Invalid or unsupported protocol version (%d); supported
versions are (%s)", version, String.join(", ",
ProtocolVersion.supportedVersions())); }
We later found invalid IP addresses in the system.peers table and once removed,
this exception went away.
was:
The following warning was seen frequently after upgrading from 3.0.x to 3.11.11
in the cassandra.log:
{noformat}
ProtocolException: Invalid or unsupported protocol version (4); supported
versions are (3/v3, 4/v4, 5/v5-beta){noformat}
It is at best unclear, or maybe a bug in the code throwing this exception
stating version '4' not supported but 4/v4 is.
from org/apache/cassandra/transport/ProtocolVersion.java
public static String invalidVersionMessage(int version) {
return String.format("Invalid or unsupported protocol version (%d); supported
versions are (%s)", version, String.join(", ",
ProtocolVersion.supportedVersions()));
}
We later found invalid IP addresses in the system.peer table and once removed,
this exception went away.
> Confusing ProtocolException msg Invalid or unsupported protocol version (4)
> ---
>
> Key: CASSANDRA-16940
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16940
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Brad Schoening
>Priority: Normal
>
> The following warning was seen frequently after upgrading from 3.0.x to
> 3.11.11 in the cassandra.log:
> {noformat}
> ProtocolException: Invalid or unsupported protocol version (4); supported
> versions are (3/v3, 4/v4, 5/v5-beta){noformat}
> It is at best unclear, or maybe a bug in the code throwing this exception
> stating version '4' not supported but 4/v4 is.
> from org/apache/cassandra/transport/ProtocolVersion.java
> public static String invalidVersionMessage(int version)
> { return String.format("Invalid or unsupported protocol version (%d);
> supported versions are (%s)", version, String.join(", ",
> ProtocolVersion.supportedVersions())); }
> We later found invalid IP addresses in the system.peers table and once
> removed, this exception went away.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16940) Confusing ProtocolException msg Invalid or unsupported protocol version (4)
[
https://issues.apache.org/jira/browse/CASSANDRA-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412861#comment-17412861
]
Brandon Williams commented on CASSANDRA-16940:
--
Hmm, nothing really makes sense on the surface here. Version 4 is obviously
supported, and I don't see anything in the code here that would care about the
peers table. [~samt] can you look?
> Confusing ProtocolException msg Invalid or unsupported protocol version (4)
> ---
>
> Key: CASSANDRA-16940
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16940
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Brad Schoening
>Priority: Normal
>
> The following warning was seen frequently after upgrading from 3.0.x to
> 3.11.11 in the cassandra.log:
> {noformat}
> ProtocolException: Invalid or unsupported protocol version (4); supported
> versions are (3/v3, 4/v4, 5/v5-beta){noformat}
> It is at best unclear, or maybe a bug in the code throwing this exception
> stating version '4' not supported but 4/v4 is.
> from org/apache/cassandra/transport/ProtocolVersion.java
> public static String invalidVersionMessage(int version) {
> return String.format("Invalid or unsupported protocol version (%d); supported
> versions are (%s)", version, String.join(", ",
> ProtocolVersion.supportedVersions()));
> }
> We later found invalid IP addresses in the system.peer table and once
> removed, this exception went away.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16940) Confusing ProtocolException msg Invalid or unsupported protocol version (4)
[
https://issues.apache.org/jira/browse/CASSANDRA-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-16940:
-
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Component/s: Messaging/Client
Discovered By: User Report
Severity: Normal
Status: Open (was: Triage Needed)
> Confusing ProtocolException msg Invalid or unsupported protocol version (4)
> ---
>
> Key: CASSANDRA-16940
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16940
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Brad Schoening
>Priority: Normal
>
> The following warning was seen frequently after upgrading from 3.0.x to
> 3.11.11 in the cassandra.log:
> {noformat}
> ProtocolException: Invalid or unsupported protocol version (4); supported
> versions are (3/v3, 4/v4, 5/v5-beta){noformat}
> It is at best unclear, or maybe a bug in the code throwing this exception
> stating version '4' not supported but 4/v4 is.
> from org/apache/cassandra/transport/ProtocolVersion.java
> public static String invalidVersionMessage(int version) {
> return String.format("Invalid or unsupported protocol version (%d); supported
> versions are (%s)", version, String.join(", ",
> ProtocolVersion.supportedVersions()));
> }
> We later found invalid IP addresses in the system.peer table and once
> removed, this exception went away.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16940) Confusing ProtocolException msg Invalid or unsupported protocol version (4)
Brad Schoening created CASSANDRA-16940:
--
Summary: Confusing ProtocolException msg Invalid or unsupported
protocol version (4)
Key: CASSANDRA-16940
URL: https://issues.apache.org/jira/browse/CASSANDRA-16940
Project: Cassandra
Issue Type: Bug
Reporter: Brad Schoening
The following warning was seen frequently after upgrading from 3.0.x to 3.11.11
in the cassandra.log:
{noformat}
ProtocolException: Invalid or unsupported protocol version (4); supported
versions are (3/v3, 4/v4, 5/v5-beta){noformat}
It is at best unclear, or maybe a bug in the code throwing this exception
stating version '4' not supported but 4/v4 is.
from org/apache/cassandra/transport/ProtocolVersion.java
public static String invalidVersionMessage(int version) {
return String.format("Invalid or unsupported protocol version (%d); supported
versions are (%s)", version, String.join(", ",
ProtocolVersion.supportedVersions()));
}
We later found invalid IP addresses in the system.peer table and once removed,
this exception went away.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16909) ☂ Medium Term Repair Improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16909:
--
Description:
This is to track the repair improvement works defined on the dev list.
Email was: [DISCUSS] Repair Improvement Proposal
This JIRA will track the related tasks and be used for documentaiton
{code}
Repair Coordinator State
1. ActiveRepairService.recordRepairStatus
1. Maps JMX command (int) to (status, msg)
2. Get/validate ColumnFamilies
1. Skip if no ColumnFamilies to repair
3. Get replicas for ranges
1. Ignore ranges not covered by this instance IFF
--ignore-unreplicated-keyspaces, else Fail
2. Skip if no ranges to repair
3. If --force filter out non-alive (FailureDetector.isAlive) participants
4. [Not PREVIEW]Update SystemDistributedKeyspace's parent_repair_history
5. ActiveRepairService.prepareForRepair
1. TODO Should this be under PREPARE or still part of validation?
1. If CompactionsPendingThreshold triggered, Fail
2. registerParentRepairSession - update map of UUID ->
ParentRepairSession
2. Send PREPARE_MSG awaiting responses [possible failures: timeout waiting,
participate failure]
1. [improvement] PREPARE_MSG should be idempotent and if no reply
within X, retry Y times
2. Known Failures; all are retryable at this stage
1. Timeout
2. Participate goes from alive to down
3. CompactionsPendingThreshold triggered
1. Not included in
org.apache.cassandra.exceptions.RequestFailureReason#forException, so looks the
same as Unexpected error
1. If updated and in mixed-mode, the update falls back to
UNKNOWN, which then matches the unexpected error behavior
4. Unexpected error (this removes the session)
6. Run RepairTask (normal, IR, preview); see coordinator state for each type
7. On Success
1. Update SystemDistributedKeyspace.parent_repair_history to show the
successful ranges
2. If any sub-session failed, fail the job
3. ActiveRepairService.cleanUp - message to all participates to clean up
1. TODO: why is this only on success and not failures as well?
8. On Exception
1. fail
Normal/Preview Repair Coordinator State
1. For each common range
1. ActiveRepairService.submitRepairSession
1. Creates/run a RepairSession for each CommonRange
2. Once all RepairSessions done
1. [not consistent cross each type] handle session errors
2. [Preview Repair] check SyncStatSummary for difference and send a
human readable notification
Incremental Repair Coordinator State
1. org.apache.cassandra.repair.consistent.CoordinatorSessions#registerSession
to create CoordinatorSession
2. CoordinatorSession.execute
1. Send PREPARE_CONSISTENT_REQ and wait for PREPARE_CONSISTENT_RSP
2. Await all success
3. Trigger Normal Repair (see "Normal/Preview Repair Coordinator State")
4. Send FINALIZE_PROPOSE_MSG and wait for FINALIZE_PROMISE_MSG
5. Await all success
6. Send FINALIZE_COMMIT_MSG
1. [bug][improvement] No ACK is done, so if this message is dropped
then some participates will think the IR is still running (which can fail
preview repairs)
RepairSession
1. [Preview Repair - kind=REPAIRED] register with LocalSessions for IR state
changes
2. RepairSession.start
1. [Not Preview Repair] Registering the session into
SystemDistributedKeyspace's table repair_history
2. If endpoints is empty
1. [UNHANDLED - downstream logic does not handle this case] Set future
with empty state (which is later seen as Failed... but not a failed future)
2. [Not Preview Repair] Mark session failed in repair_history
3. Check all endpoints, if any is down and hasSkippedReplicas=false, Fail
the session
4. For each table
1. Create a RepairJob
2. Execute job in RepairTask's executor
3. await all jobs
1. If all success
1. Set session result to include the job results
2. If any fail
1. Fail the session future
2. [Question] why does this NOT update repair_history like
other failures?
RepairJob
1. [parallelism in [SEQUENTIAL, DATACENTER_AWARE] and not IR] send SNAPSHOT_MSG
to all participates
1. [improvement] SNAPSHOT_MSG should be idempotent so coordinator can retry
if no ACK. This calls org.apache.cassandra.repair.TableRepairManager#snapshot
with the RepairRunnable ID; this makes sure to snapshot once unless force=true
(RepairOptions !(dataCenters.isEmpty and hosts.isEmpty()))
2. [improvement] This task is short lived, so rather than running in a
cached pool (which may allocate a thread), inline or add a notation of
cooperative sharing if retries are implemented
2. Await snapshot success
3. Send VALIDATION_REQ to all participates (all at once, or batched
[jira] [Created] (CASSANDRA-16939) Shutdown hook is removed after non-fatal OutOfMemoryError
Paulo Motta created CASSANDRA-16939:
---
Summary: Shutdown hook is removed after non-fatal OutOfMemoryError
Key: CASSANDRA-16939
URL: https://issues.apache.org/jira/browse/CASSANDRA-16939
Project: Cassandra
Issue Type: Bug
Reporter: Paulo Motta
On CASSANDRA-7507 the drain shutdown hook is removed when the process hits an
OutOfMemoryError, to avoid trying to do a clean shutdown when the node runs out
of heap space.
On CASSANDRA-13006 cassandra started relying on JVM flags
(ExitOnOutOfMemoryError and CrashOnOutOfMemoryError) to stop the node when
hitting and OutOfMemoryError.
However, there are non-fatal {{OutOfMemoryErrors}} such as {{OutOfMemory:
unable to create new native thread}} or {{OutOfMemory: map failed}} which [do
not cause the process to crash even with the ExitOnOutOfMemoryError or
CrashOnOutOfMemoryError flags|https://bugs.openjdk.java.net/browse/JDK-8155004].
Since the shutdown hook is removed after non-fatal OutOfMemory errors, it's no
longer possible to do a clean shutdown (via {{SIGTERM kill}} or {{nodetool
stopdaemon}}).
I believe the intent of CASSANDRA-7507 was to remove the shutdown hook only on
fatal OutOfMemoryErrors (such as Heap Space Exhausted), those causing the node
to crash. If a node is kept running after an OutOfMemoryError, this should not
prevent the node from being cleanly shutdown afterwards.
We should either make the JVM exit on any OutOfMemory error, or remove the
drain shutdown hook only on fatal OutOfMemoryErrors, those that will cause the
JVM to crash straight away.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412843#comment-17412843
]
Michael Semb Wever commented on CASSANDRA-16938:
bq. I don't see how it matters in a serialized format like json. If people are
trying to read it themselves and specifically chose json output to do
so...well, this is still valid json, heh.
Agreed.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code:java}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. My guess is that the conversion was done trying to work
> around a malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16895) Support Java 17
[ https://issues.apache.org/jira/browse/CASSANDRA-16895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412837#comment-17412837 ] Ekaterina Dimitrova commented on CASSANDRA-16895: - The thing is that if we use the graaljs, according to that page you linked, we also need graalsdk which is under GPL2. I will raise the topic on the mailing list. Thank you for your input and advice! > Support Java 17 > --- > > Key: CASSANDRA-16895 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16895 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > > This ticket is intended to group all issues found to support Java 17 in the > future. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16905) Further restrict schema column drop/recreate conversions
[ https://issues.apache.org/jira/browse/CASSANDRA-16905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16905: Reviewers: Caleb Rackliffe > Further restrict schema column drop/recreate conversions > > > Key: CASSANDRA-16905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16905 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > Fix For: 4.x > > > CASSANDRA-14948 took us a good bit of the way to preventing recreation issues > with columns, however we should take the safety a bit further and disallow > recreation on fixed/variable length mismatches and simple/complex mismatches. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16886) Reduce native_transport_max_frame_size_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-16886: - Fix Version/s: 4.1 Source Control Link: https://github.com/apache/cassandra/commit/08444cbc3f6378f281a811d74c9cb152c8ad19ca Resolution: Fixed Status: Resolved (was: Ready to Commit) I did one more CI run: [!https://ci-cassandra.apache.org/job/Cassandra-devbranch/1103/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1103/pipeline] and committed. Thanks. > Reduce native_transport_max_frame_size_in_mb > > > Key: CASSANDRA-16886 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16886 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1 > > > There is really no point in having this set to 256MB when the commitlog > segment size defaults to 32MB, effectively capping the largest insertable > mutation to 16MB. > The native transport can provide a good first line of defense against large > mutations that would otherwise hit the heap if left to be rejected by the > commitlog. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16886) Reduce native_transport_max_frame_size_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-16886: - Status: Ready to Commit (was: Review In Progress) > Reduce native_transport_max_frame_size_in_mb > > > Key: CASSANDRA-16886 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16886 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > > There is really no point in having this set to 256MB when the commitlog > segment size defaults to 32MB, effectively capping the largest insertable > mutation to 16MB. > The native transport can provide a good first line of defense against large > mutations that would otherwise hit the heap if left to be rejected by the > commitlog. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Reduce max native frame size to 16MB
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 08444cb Reduce max native frame size to 16MB
08444cb is described below
commit 08444cbc3f6378f281a811d74c9cb152c8ad19ca
Author: Brandon Williams
AuthorDate: Thu Aug 26 10:16:42 2021 -0500
Reduce max native frame size to 16MB
Patch by brandonwilliams; reviewed by blerer for CASSANDRA-16886
---
CHANGES.txt| 1 +
conf/cassandra.yaml| 4 ++--
src/java/org/apache/cassandra/config/Config.java | 2 +-
src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 10 ++
.../unit/org/apache/cassandra/transport/CQLConnectionTest.java | 3 +++
5 files changed, 17 insertions(+), 3 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index f0a0dfb..80ee45c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.1
+ * Reduce native transport max frame size to 16MB (CASSANDRA-16886)
* Add support for filtering using IN restrictions (CASSANDRA-14344)
* Provide a nodetool command to invalidate auth caches (CASSANDRA-16404)
* Catch read repair timeout exceptions and add metric (CASSANDRA-16880)
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index ff073da..25f6eb1 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -710,9 +710,9 @@ native_transport_port: 9042
# native_transport_max_threads: 128
#
# The maximum size of allowed frame. Frame (requests) larger than this will
-# be rejected as invalid. The default is 256MB. If you're changing this
parameter,
+# be rejected as invalid. The default is 16MB. If you're changing this
parameter,
# you may want to adjust max_value_size_in_mb accordingly. This should be
positive and less than 2048.
-# native_transport_max_frame_size_in_mb: 256
+# native_transport_max_frame_size_in_mb: 16
# The maximum number of concurrent client connections.
# The default is -1, which means unlimited.
diff --git a/src/java/org/apache/cassandra/config/Config.java
b/src/java/org/apache/cassandra/config/Config.java
index 731b466..77607ed 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -186,7 +186,7 @@ public class Config
public int native_transport_port = 9042;
public Integer native_transport_port_ssl = null;
public int native_transport_max_threads = 128;
-public int native_transport_max_frame_size_in_mb = 256;
+public int native_transport_max_frame_size_in_mb = 16;
public volatile long native_transport_max_concurrent_connections = -1L;
public volatile long native_transport_max_concurrent_connections_per_ip =
-1L;
public boolean native_transport_flush_in_batches_legacy = false;
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 7e80485..7a0eeb0 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -2257,6 +2257,11 @@ public class DatabaseDescriptor
return (int)
ByteUnit.MEBI_BYTES.toBytes(conf.native_transport_max_frame_size_in_mb);
}
+public static void setNativeTransportMaxFrameSize(int bytes)
+{
+conf.native_transport_max_frame_size_in_mb = (int)
ByteUnit.MEBI_BYTES.fromBytes(bytes);
+}
+
public static long getNativeTransportMaxConcurrentConnections()
{
return conf.native_transport_max_concurrent_connections;
@@ -3298,6 +3303,11 @@ public class DatabaseDescriptor
{
return val * multiplier;
}
+
+public long fromBytes(int val)
+{
+return val / multiplier;
+}
}
/**
diff --git a/test/unit/org/apache/cassandra/transport/CQLConnectionTest.java
b/test/unit/org/apache/cassandra/transport/CQLConnectionTest.java
index 73950ae..adc82ea 100644
--- a/test/unit/org/apache/cassandra/transport/CQLConnectionTest.java
+++ b/test/unit/org/apache/cassandra/transport/CQLConnectionTest.java
@@ -59,6 +59,7 @@ import org.apache.cassandra.utils.concurrent.SimpleCondition;
import org.apache.cassandra.utils.concurrent.NonBlockingRateLimiter;
import static
org.apache.cassandra.config.EncryptionOptions.TlsEncryptionPolicy.UNENCRYPTED;
+import static org.apache.cassandra.io.util.FileUtils.ONE_MB;
import static org.apache.cassandra.net.FramingTest.randomishBytes;
import static org.apache.cassandra.transport.Flusher.MAX_FRAMED_PAYLOAD_SIZE;
import static
org.apache.cassandra.utils.concurrent.NonBlockingRateLimiter.NO_OP_LIMITER;
@@ -104,6 +105,8 @@ public class CQLConnectionTest
alloc = GlobalBufferPoolAllocator.instance;
// set connection-local
[jira] [Updated] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16888: Since Version: 4.0.1 Source Control Link: https://github.com/apache/cassandra/commit/bc052fa68f525155246de498cc86bb192f2d479a Resolution: Fixed Status: Resolved (was: Ready to Commit) 4.0: https://github.com/apache/cassandra/commit/bc052fa68f525155246de498cc86bb192f2d479a trunk: https://github.com/apache/cassandra/commit/b6b3be6d8aa79c5e74b0c0f197075fdc48e38287 > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.2, 4.1 > > Time Spent: 20m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.0' into trunk
This is an automated email from the ASF dual-hosted git repository. maedhroz pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b6b3be6d8aa79c5e74b0c0f197075fdc48e38287 Merge: b8f786d bc052fa Author: Caleb Rackliffe AuthorDate: Thu Sep 9 14:09:34 2021 -0500 Merge branch 'cassandra-4.0' into trunk CHANGES.txt| 1 + .../cassandra/io/sstable/SSTableReaderTest.java| 50 +++--- 2 files changed, 27 insertions(+), 24 deletions(-) diff --cc CHANGES.txt index f3b0f76,1a487c4..f0a0dfb --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,31 -1,8 +1,32 @@@ -4.0.2 +4.1 + * Add support for filtering using IN restrictions (CASSANDRA-14344) + * Provide a nodetool command to invalidate auth caches (CASSANDRA-16404) + * Catch read repair timeout exceptions and add metric (CASSANDRA-16880) + * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854) + * Add client warnings and abort to tombstone and coordinator reads which go past a low/high watermark (CASSANDRA-16850) + * Add TTL support to nodetool snapshots (CASSANDRA-16789) + * Allow CommitLogSegmentReader to optionally skip sync marker CRC checks (CASSANDRA-16842) + * allow blocking IPs from updating metrics about traffic (CASSANDRA-16859) + * Request-Based Native Transport Rate-Limiting (CASSANDRA-16663) + * Implement nodetool getauditlog command (CASSANDRA-16725) + * Clean up repair code (CASSANDRA-13720) + * Background schedule to clean up orphaned hints files (CASSANDRA-16815) + * Modify SecondaryIndexManager#indexPartition() to retrieve only columns for which indexes are actually being built (CASSANDRA-16776) + * Batch the token metadata update to improve the speed (CASSANDRA-15291) + * Reduce the log level on "expected" repair exceptions (CASSANDRA-16775) + * Make JMXTimer expose attributes using consistent time unit (CASSANDRA-16760) + * Remove check on gossip status from DynamicEndpointSnitch::updateScores (CASSANDRA-11671) + * Fix AbstractReadQuery::toCQLString not returning valid CQL (CASSANDRA-16510) + * Log when compacting many tombstones (CASSANDRA-16780) + * Display bytes per level in tablestats for LCS tables (CASSANDRA-16799) + * Add isolated flush timer to CommitLogMetrics and ensure writes correspond to single WaitingOnCommit data points (CASSANDRA-16701) + * Add a system property to set hostId if not yet initialized (CASSANDRA-14582) + * GossiperTest.testHasVersion3Nodes didn't take into account trunk version changes, fixed to rely on latest version (CASSANDRA-16651) +Merged from 4.0: + * Remove all the state pollution between tests in SSTableReaderTest (CASSANDRA-16888) * Delay auth setup until after gossip has settled to avoid unavailables on startup (CASSANDRA-16783) - * Fix clustering order logic in CREATE MATERIALIZED VIEW (CASSANDRA-16898) * org.apache.cassandra.db.rows.ArrayCell#unsharedHeapSizeExcludingData includes data twice (CASSANDRA-16900) + * Fix clustering order logic in CREATE MATERIALIZED VIEW (CASSANDRA-16898) * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854) Merged from 3.11: * Make assassinate more resilient to missing tokens (CASSANDRA-16847) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated: Remove all the state pollution between tests in SSTableReaderTest
This is an automated email from the ASF dual-hosted git repository.
maedhroz pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-4.0 by this push:
new bc052fa Remove all the state pollution between tests in
SSTableReaderTest
bc052fa is described below
commit bc052fa68f525155246de498cc86bb192f2d479a
Author: Caleb Rackliffe
AuthorDate: Thu Sep 9 11:55:52 2021 -0500
Remove all the state pollution between tests in SSTableReaderTest
patch by Caleb Rackliffe; reviewed by Marcus Eriksson for CASSANDRA-16888
---
CHANGES.txt| 1 +
.../cassandra/io/sstable/SSTableReaderTest.java| 50 +++---
2 files changed, 27 insertions(+), 24 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 0ff0ffe..1a487c4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0.2
+ * Remove all the state pollution between tests in SSTableReaderTest
(CASSANDRA-16888)
* Delay auth setup until after gossip has settled to avoid unavailables on
startup (CASSANDRA-16783)
* Fix clustering order logic in CREATE MATERIALIZED VIEW (CASSANDRA-16898)
* org.apache.cassandra.db.rows.ArrayCell#unsharedHeapSizeExcludingData
includes data twice (CASSANDRA-16900)
diff --git a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
index c905732..f1fc4cb 100644
--- a/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
+++ b/test/unit/org/apache/cassandra/io/sstable/SSTableReaderTest.java
@@ -30,9 +30,7 @@ import org.junit.BeforeClass;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;
-import org.junit.runner.RunWith;
-import org.apache.cassandra.OrderedJUnit4ClassRunner;
import org.apache.cassandra.SchemaLoader;
import org.apache.cassandra.Util;
import org.apache.cassandra.cql3.Operator;
@@ -66,7 +64,6 @@ import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
-@RunWith(OrderedJUnit4ClassRunner.class)
public class SSTableReaderTest
{
public static final String KEYSPACE1 = "SSTableReaderTest";
@@ -102,6 +99,9 @@ public class SSTableReaderTest
.minIndexInterval(4)
.maxIndexInterval(4)
.bloomFilterFpChance(0.99));
+
+// All tests in this class assume auto-compaction is disabled.
+CompactionManager.instance.disableAutoCompaction();
}
@Test
@@ -109,10 +109,10 @@ public class SSTableReaderTest
{
Keyspace keyspace = Keyspace.open(KEYSPACE1);
ColumnFamilyStore store = keyspace.getColumnFamilyStore(CF_STANDARD2);
+store.discardSSTables(System.currentTimeMillis());
partitioner = store.getPartitioner();
// insert data and compact to a single sstable
-CompactionManager.instance.disableAutoCompaction();
for (int j = 0; j < 10; j++)
{
new RowUpdateBuilder(store.metadata(), j, String.valueOf(j))
@@ -124,7 +124,7 @@ public class SSTableReaderTest
store.forceBlockingFlush();
CompactionManager.instance.performMaximal(store, false);
-List> ranges = new ArrayList>();
+List> ranges = new ArrayList<>();
// 1 key
ranges.add(new Range<>(t(0), t(1)));
// 2 keys
@@ -155,10 +155,10 @@ public class SSTableReaderTest
{
Keyspace keyspace = Keyspace.open(KEYSPACE1);
ColumnFamilyStore store =
keyspace.getColumnFamilyStore(CF_STANDARD);
+store.discardSSTables(System.currentTimeMillis());
partitioner = store.getPartitioner();
// insert a bunch of data and compact to a single sstable
-CompactionManager.instance.disableAutoCompaction();
for (int j = 0; j < 100; j += 2)
{
new RowUpdateBuilder(store.metadata(), j, String.valueOf(j))
@@ -199,6 +199,7 @@ public class SSTableReaderTest
Keyspace keyspace = Keyspace.open(KEYSPACE1);
ColumnFamilyStore store = keyspace.getColumnFamilyStore(CF_STANDARD);
+store.discardSSTables(System.currentTimeMillis());
partitioner = store.getPartitioner();
for (int j = 0; j < 100; j += 2)
@@ -227,6 +228,7 @@ public class SSTableReaderTest
// try to make sure CASSANDRA-8239 never happens again
Keyspace keyspace = Keyspace.open(KEYSPACE1);
ColumnFamilyStore store = keyspace.getColumnFamilyStore(CF_STANDARD);
+store.discardSSTables(System.currentTimeMillis());
partitioner = store.getPartitioner();
for (int j = 0; j < 10; j++)
@@ -256,11 +258,11 @@ public class
[cassandra] branch trunk updated (b8f786d -> b6b3be6)
This is an automated email from the ASF dual-hosted git repository. maedhroz pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b8f786d Merge branch 'cassandra-4.0' into trunk new bc052fa Remove all the state pollution between tests in SSTableReaderTest new b6b3be6 Merge branch 'cassandra-4.0' into trunk The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/io/sstable/SSTableReaderTest.java| 50 +++--- 2 files changed, 27 insertions(+), 24 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412759#comment-17412759 ] Caleb Rackliffe edited comment on CASSANDRA-16888 at 9/9/21, 6:30 PM: -- The multiplexer [likes is|https://app.circleci.com/pipelines/github/maedhroz/cassandra?branch=16888-trunk-with-fix], so I think we're all set. Starting commit... was (Author: maedhroz): The multiplexer [likes is|https://app.circleci.com/pipelines/github/maedhroz/cassandra?branch=16888-trunk-with-fix], so I think we're all set. Staring commit... > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.2, 4.1 > > Time Spent: 10m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16888: Reviewers: Marcus Eriksson (was: Caleb Rackliffe, Marcus Eriksson) > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.x, 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16888: Fix Version/s: (was: 4.0.x) (was: 4.x) 4.1 4.0.2 > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.2, 4.1 > > Time Spent: 10m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16888: Reviewers: Marcus Eriksson, Caleb Rackliffe (was: Caleb Rackliffe, Marcus Eriksson) Marcus Eriksson, Caleb Rackliffe (was: Marcus Eriksson) Status: Review In Progress (was: Patch Available) > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.x, 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16888: Status: Ready to Commit (was: Review In Progress) > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.x, 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412759#comment-17412759 ] Caleb Rackliffe commented on CASSANDRA-16888: - The multiplexer [likes is|https://app.circleci.com/pipelines/github/maedhroz/cassandra?branch=16888-trunk-with-fix], so I think we're all set. Staring commit... > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.x, 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16909) ☂ Medium Term Repair Improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412749#comment-17412749
]
David Capwell commented on CASSANDRA-16909:
---
my focus is on CASSANDRA-16896 atm, get that complete. Once done ill document
preview repair (its mostly done) and IR
> ☂ Medium Term Repair Improvements
> -
>
> Key: CASSANDRA-16909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16909
> Project: Cassandra
> Issue Type: Epic
> Components: Consistency/Repair
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
>
> This is to track the repair improvement works defined on the dev list.
> Email was: [DISCUSS] Repair Improvement Proposal
> This JIRA will track the related tasks and be used for documentaiton
> TODO: document Preview and IR
> {code}
> Repair Coordinator State
> 1. ActiveRepairService.recordRepairStatus
> 1. Maps JMX command (int) to (status, msg)
> 2. Get/validate ColumnFamilies
> 1. Skip if no ColumnFamilies to repair
> 3. Get replicas for ranges
> 1. Ignore ranges not covered by this instance IFF
> --ignore-unreplicated-keyspaces, else Fail
> 2. Skip if no ranges to repair
> 3. If --force filter out non-alive (FailureDetector.isAlive) participants
> 4. [Not PREVIEW]Update SystemDistributedKeyspace's parent_repair_history
> 5. ActiveRepairService.prepareForRepair
> 1. TODO Should this be under PREPARE or still part of validation?
> 1. If CompactionsPendingThreshold triggered, Fail
> 2. registerParentRepairSession - update map of UUID ->
> ParentRepairSession
> 2. Send PREPARE_MSG awaiting responses [possible failures: timeout
> waiting, participate failure]
> 1. [improvement] PREPARE_MSG should be idempotent and if no reply
> within X, retry Y times
> 2. Known Failures; all are retryable at this stage
> 1. Timeout
> 2. Participate goes from alive to down
> 3. CompactionsPendingThreshold triggered
> 1. Not included in
> org.apache.cassandra.exceptions.RequestFailureReason#forException, so looks
> the same as Unexpected error
> 1. If updated and in mixed-mode, the update falls back to
> UNKNOWN, which then matches the unexpected error behavior
> 4. Unexpected error (this removes the session)
> 6. Run RepairTask (normal, IR, preview); see coordinator state for each type
> 7. On Success
> 1. Update SystemDistributedKeyspace.parent_repair_history to show the
> successful ranges
> 2. If any sub-session failed, fail the job
> 3. ActiveRepairService.cleanUp - message to all participates to clean up
> 1. TODO: why is this only on success and not failures as well?
> 8. On Exception
> 1. fail
> Normal/Preview Repair Coordinator State
> 1. For each common range
> 1. ActiveRepairService.submitRepairSession
> 1. Creates/run a RepairSession for each CommonRange
> 2. Once all RepairSessions done
> 1. [not consistent cross each type] handle session errors
> RepairSession
> 1. [Preview Repair - kind=REPAIRED] register with LocalSessions for IR state
> changes
> 2. RepairSession.start
> 1. [Not Preview Repair] Registering the session into
> SystemDistributedKeyspace's table repair_history
> 2. If endpoints is empty
> 1. [UNHANDLED - downstream logic does not handle this case] Set
> future with empty state (which is later seen as Failed... but not a failed
> future)
> 2. [Not Preview Repair] Mark session failed in repair_history
> 3. Check all endpoints, if any is down and hasSkippedReplicas=false, Fail
> the session
> 4. For each table
> 1. Create a RepairJob
> 2. Execute job in RepairTask's executor
> 3. await all jobs
> 1. If all success
> 1. Set session result to include the job results
> 2. If any fail
> 1. Fail the session future
> 2. [Question] why does this NOT update repair_history like
> other failures?
> RepairJob
> 1. [parallelism in [SEQUENTIAL, DATACENTER_AWARE] and not IR] send
> SNAPSHOT_MSG to all participates
> 1. [improvement] SNAPSHOT_MSG should be idempotent so coordinator can
> retry if no ACK. This calls
> org.apache.cassandra.repair.TableRepairManager#snapshot with the
> RepairRunnable ID; this makes sure to snapshot once unless force=true
> (RepairOptions !(dataCenters.isEmpty and hosts.isEmpty()))
> 2. [improvement] This task is short lived, so rather than running in a
> cached pool (which may allocate a thread), inline or add a notation of
> cooperative sharing if retries are implemented
> 2. Await snapshot success
> 3. Send VALIDATION_REQ to all participates (all at once, or batched based off
[jira] [Updated] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16888: Reviewers: Marcus Eriksson > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.x, 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16841) Unexpectedly ignored dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412727#comment-17412727
]
Andres de la Peña commented on CASSANDRA-16841:
---
IIUC the change in the
{{-only-resource-intensive-tests}}/{{-force-resource-intensive-tests}} flags is
reverted but now the message is changed so it also appears when
{{-skip-resource-intensive-tests}} is used, even if there are enough resources
available. Without the patch the message was only emitted when there weren't
enough resources and {{-force-resource-intensive-tests}} wasn't used,
independently of other test selection flags. Is this correct? I think I would
have preferred to separate this in two separate messages in a follow up ticket,
but if you think it's clearer this way I'm fine with it.
> Unexpectedly ignored dtests
> ---
>
> Key: CASSANDRA-16841
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16841
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ruslan Fomkin
>Assignee: Ruslan Fomkin
>Priority: Normal
> Time Spent: 2h
> Remaining Estimate: 0h
>
> An issue, which I was hit:
> When one class in a dtest file is marked as resource intensive, then all
> tests in all classes are treated as resource intensive. For example,
> [repair_tests/repair_test.py|https://github.com/apache/cassandra-dtest/blob/trunk/repair_tests/repair_test.py]
> contains three classes and the last class is marked as resource intensive:
> {code:java}
> @pytest.mark.resource_intensive
> class TestRepairDataSystemTable(Tester):
> {code}
> So if I try to run an unmarked class:
> {code:java}
> pytest --cassandra-dir=../cassandra repair_tests/repair_test.py::TestRepair
> --collect-only --skip-resource-intensive-tests
> {code}
> then all tests are ignored
> {code:java}
> collected 36 items / 36 deselected
> {code}
> This is because a test is treated to be marked if any class in the same file
> has the mark. This bug was introduced in the fix of CASS-16399. Before only
> upgrade tests had such behaviour, i.e., if a class is marked as upgrade test,
> then all tests are upgrade test in the file.
>
> This bug, for example, means that if the same file contains one class marked
> with vnodes and another class with no_vnodes, then no tests will be executed
> in the file.
> I also noticed another issue that If a test run is executed with the argument
> {{-only-resource-intensive-tests}} and there is no sufficient resources for
> resource intensive tests, then no tests were executed. Thus it was necessary
> to provide {{-force-resource-intensive-tests}} in addition.
> Suggestions for the solutions:
> # Require to mark each class and remove the special case of upgrade tests.
> This will simplify the implementation and might be more obvious for new
> comers.
> # Treat {{-only-resource-intensive-tests}} in the same way as
> {{-force-resource-intensive-tests}}, so it will be enough to just specify it
> even with no sufficient resources.
> *Update:* comments were provided to keep only the first suggestion and do not
> implement the second suggestion.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412725#comment-17412725 ] Marcus Eriksson commented on CASSANDRA-16888: - +1 > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.x, 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[
https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412720#comment-17412720
]
Caleb Rackliffe commented on CASSANDRA-16888:
-
The long story short here is that many of the tests in {{SSTableReaderTest}}
share the same table(s), but there was not consistent removal of the SSTables
from previous tests. That combined with partition key reuse, doomed the test in
question here, since it could view an SSTable from
{{testGetPositionsForRangesWithKeyCache()}}. Doing this also happens to remove
the need to use {{OrderedJUnit4ClassRunner}}.
> SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on
> unexpected key cache hit for non-cached key
> ---
>
> Key: CASSANDRA-16888
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16888
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This has started to pop up on 4.0.x and trunk builds in both CircleCI and
> Apache CI:
> [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382]
> [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0]
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/
>
> It passes consistently in local runs, although I have seen one failure in
> about 10 runs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[ https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-16888: Test and Documentation Plan: This patch fixes an existing test class. Status: Patch Available (was: In Progress) patch (4.0): https://github.com/apache/cassandra/pull/1192 CircleCI: https://app.circleci.com/pipelines/github/maedhroz/cassandra?branch=CASSANDRA-16888-4.0 trunk multiplexer run: https://app.circleci.com/pipelines/github/maedhroz/cassandra?branch=16888-trunk-with-fix > SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on > unexpected key cache hit for non-cached key > --- > > Key: CASSANDRA-16888 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16888 > Project: Cassandra > Issue Type: Bug > Components: Local/Caching >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 4.0.x, 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > This has started to pop up on 4.0.x and trunk builds in both CircleCI and > Apache CI: > [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382] > [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0] > https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/ > > It passes consistently in local runs, although I have seen one failure in > about 10 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16907) Fix ivm-upgrade test org.apache.cassandra.distributed.upgrade.CompactStorageUpgradeTest: compactStorageHiddenColumnTest
[ https://issues.apache.org/jira/browse/CASSANDRA-16907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16907: --- Resolution: Duplicate Status: Resolved (was: Open) > Fix ivm-upgrade test > org.apache.cassandra.distributed.upgrade.CompactStorageUpgradeTest: > compactStorageHiddenColumnTest > --- > > Key: CASSANDRA-16907 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16907 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: David Capwell >Priority: Normal > Fix For: 4.x > > > https://app.circleci.com/pipelines/github/dcapwell/cassandra/1017/workflows/3a1caaca-b213-4841-b010-cdcdde73df0b/jobs/6450 > This test has been failing for a while -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16938:
Status: Ready to Commit (was: Review In Progress)
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code:java}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. My guess is that the conversion was done trying to work
> around a malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412701#comment-17412701
]
Ekaterina Dimitrova commented on CASSANDRA-16938:
-
Agreed with all said, the question was I guess about something weird someone
might do but I guess we can't support every weird customization in the world.
+1 on green CI (some tests were still running)
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code:java}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. My guess is that the conversion was done trying to work
> around a malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16841) Unexpectedly ignored dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412648#comment-17412648
]
Ruslan Fomkin edited comment on CASSANDRA-16841 at 9/9/21, 4:40 PM:
[~adelapena] [~e.dimitrova] I addressed your comments and reverted the change
about resource intensive tests, so force execution flag is always required if
not sufficient resources even in the case to execute only resource intensive
tests.
I run [j11_dtests in
CircleCI|https://app.circleci.com/pipelines/github/k-rus/cassandra/10/workflows/e56bdedd-45ba-466f-ae32-a003b9004063]
and
[j8_dtests|https://app.circleci.com/pipelines/github/k-rus/cassandra/10/workflows/846bfcdc-b33e-4e57-9252-56ef445d115a]
was (Author: k-rus):
[~adelapena] [~e.dimitrova] I addressed your comments and reverted the change
about resource intensive tests, so force execution flag is always required if
not sufficient resources even in the case to execute only resource intensive
tests.
I run [j11_dtests in
CircleCI|https://app.circleci.com/pipelines/github/k-rus/cassandra/10/workflows/e56bdedd-45ba-466f-ae32-a003b9004063]
and
[j8_dtests](https://app.circleci.com/pipelines/github/k-rus/cassandra/10/workflows/846bfcdc-b33e-4e57-9252-56ef445d115a)
> Unexpectedly ignored dtests
> ---
>
> Key: CASSANDRA-16841
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16841
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ruslan Fomkin
>Assignee: Ruslan Fomkin
>Priority: Normal
> Time Spent: 2h
> Remaining Estimate: 0h
>
> An issue, which I was hit:
> When one class in a dtest file is marked as resource intensive, then all
> tests in all classes are treated as resource intensive. For example,
> [repair_tests/repair_test.py|https://github.com/apache/cassandra-dtest/blob/trunk/repair_tests/repair_test.py]
> contains three classes and the last class is marked as resource intensive:
> {code:java}
> @pytest.mark.resource_intensive
> class TestRepairDataSystemTable(Tester):
> {code}
> So if I try to run an unmarked class:
> {code:java}
> pytest --cassandra-dir=../cassandra repair_tests/repair_test.py::TestRepair
> --collect-only --skip-resource-intensive-tests
> {code}
> then all tests are ignored
> {code:java}
> collected 36 items / 36 deselected
> {code}
> This is because a test is treated to be marked if any class in the same file
> has the mark. This bug was introduced in the fix of CASS-16399. Before only
> upgrade tests had such behaviour, i.e., if a class is marked as upgrade test,
> then all tests are upgrade test in the file.
>
> This bug, for example, means that if the same file contains one class marked
> with vnodes and another class with no_vnodes, then no tests will be executed
> in the file.
> I also noticed another issue that If a test run is executed with the argument
> {{-only-resource-intensive-tests}} and there is no sufficient resources for
> resource intensive tests, then no tests were executed. Thus it was necessary
> to provide {{-force-resource-intensive-tests}} in addition.
> Suggestions for the solutions:
> # Require to mark each class and remove the special case of upgrade tests.
> This will simplify the implementation and might be more obvious for new
> comers.
> # Treat {{-only-resource-intensive-tests}} in the same way as
> {{-force-resource-intensive-tests}}, so it will be enough to just specify it
> even with no sufficient resources.
> *Update:* comments were provided to keep only the first suggestion and do not
> implement the second suggestion.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412689#comment-17412689
]
Brandon Williams commented on CASSANDRA-16938:
--
bq. at some point we should probably try to get rid of other usages of the
venerable json-simple
I believe that's CASSANDRA-16855
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code:java}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. My guess is that the conversion was done trying to work
> around a malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16841) Unexpectedly ignored dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412648#comment-17412648
]
Ruslan Fomkin edited comment on CASSANDRA-16841 at 9/9/21, 4:40 PM:
[~adelapena] [~e.dimitrova] I addressed your comments and reverted the change
about resource intensive tests, so force execution flag is always required if
not sufficient resources even in the case to execute only resource intensive
tests.
I run [j11_dtests in
CircleCI|https://app.circleci.com/pipelines/github/k-rus/cassandra/10/workflows/e56bdedd-45ba-466f-ae32-a003b9004063]
and
[j8_dtests](https://app.circleci.com/pipelines/github/k-rus/cassandra/10/workflows/846bfcdc-b33e-4e57-9252-56ef445d115a)
was (Author: k-rus):
[~adelapena] [~e.dimitrova] I addressed your comments and reverted the change
about resource intensive tests, so force execution flag is always required if
not sufficient resources even in the case to execute only resource intensive
tests.
I run [j11_dtests in
CircleCI|https://app.circleci.com/pipelines/github/k-rus/cassandra/10/workflows/e56bdedd-45ba-466f-ae32-a003b9004063]
> Unexpectedly ignored dtests
> ---
>
> Key: CASSANDRA-16841
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16841
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ruslan Fomkin
>Assignee: Ruslan Fomkin
>Priority: Normal
> Time Spent: 2h
> Remaining Estimate: 0h
>
> An issue, which I was hit:
> When one class in a dtest file is marked as resource intensive, then all
> tests in all classes are treated as resource intensive. For example,
> [repair_tests/repair_test.py|https://github.com/apache/cassandra-dtest/blob/trunk/repair_tests/repair_test.py]
> contains three classes and the last class is marked as resource intensive:
> {code:java}
> @pytest.mark.resource_intensive
> class TestRepairDataSystemTable(Tester):
> {code}
> So if I try to run an unmarked class:
> {code:java}
> pytest --cassandra-dir=../cassandra repair_tests/repair_test.py::TestRepair
> --collect-only --skip-resource-intensive-tests
> {code}
> then all tests are ignored
> {code:java}
> collected 36 items / 36 deselected
> {code}
> This is because a test is treated to be marked if any class in the same file
> has the mark. This bug was introduced in the fix of CASS-16399. Before only
> upgrade tests had such behaviour, i.e., if a class is marked as upgrade test,
> then all tests are upgrade test in the file.
>
> This bug, for example, means that if the same file contains one class marked
> with vnodes and another class with no_vnodes, then no tests will be executed
> in the file.
> I also noticed another issue that If a test run is executed with the argument
> {{-only-resource-intensive-tests}} and there is no sufficient resources for
> resource intensive tests, then no tests were executed. Thus it was necessary
> to provide {{-force-resource-intensive-tests}} in addition.
> Suggestions for the solutions:
> # Require to mark each class and remove the special case of upgrade tests.
> This will simplify the implementation and might be more obvious for new
> comers.
> # Treat {{-only-resource-intensive-tests}} in the same way as
> {{-force-resource-intensive-tests}}, so it will be enough to just specify it
> even with no sufficient resources.
> *Update:* comments were provided to keep only the first suggestion and do not
> implement the second suggestion.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412686#comment-17412686
]
Andres de la Peña commented on CASSANDRA-16938:
---
{quote}
I don't see how it matters in a serialized format like json. If people are
trying to read it themselves and specifically chose json output to do
so...well, this is still valid json, heh.
{quote}
I agree, the new rearranged json is equivalent to the previous one and any
parser should produce the same output as before.
By the way, at some point we should probably try to get rid of other usages of
the venerable {{json-simple}}. We are using version 1.1 which was released in
2009. The only newer version is 1.1.1, which is from 2012 and doesn't solve the
bug with numeric arrays.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code:java}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. My guess is that the conversion was done trying to work
> around a malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12988) make the consistency level for user-level auth reads and writes configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412674#comment-17412674
]
Josh McKenzie commented on CASSANDRA-12988:
---
Ok, PR updated and squashed w/NEWS.txt entry, and very trivial relaxation of
the python dtests checking for auth failure (removing *which* CL we failed to
achieve and instead just confirming we failed to hit the expected CL server
side). I think this one's ready to go.
Linking re-run to the base tests (unit + in-jvm dtest) since there was a
smattering of unit test failures on the run in the earlier branch for some
reason.
||Item|Link||
|JDK8
base|[Link|https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/79/workflows/eb774443-2edb-4c3c-8652-9d040a353a28]|
|JDK11
base|[Link|https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/79/workflows/a740f309-fab9-4025-8885-b19666d748d4]|
|Squashed
PR|[Link|https://github.com/apache/cassandra/compare/trunk...josh-mckenzie:cassandra-12988?expand=1]|
|dtest
PR|[Link|https://github.com/apache/cassandra-dtest/compare/trunk...josh-mckenzie:cassandra-12988?expand=1]|
> make the consistency level for user-level auth reads and writes configurable
>
>
> Key: CASSANDRA-12988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12988
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Jason Brown
>Assignee: Josh McKenzie
>Priority: Low
> Fix For: 4.x
>
>
> Most reads for the auth-related tables execute at {{LOCAL_ONE}}. We'd like to
> make it configurable, with the default still being {{LOCAL_ONE}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412663#comment-17412663
]
Brandon Williams edited comment on CASSANDRA-16938 at 9/9/21, 4:09 PM:
---
I don't see how it matters in a serialized format like json. If people are
trying to read it themselves and specifically chose json output to do
so...well, this is still valid json, heh.
was (Author: brandon.williams):
I don't see how it matters in a serialized format like json.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code:java}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. My guess is that the conversion was done trying to work
> around a malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412663#comment-17412663
]
Brandon Williams commented on CASSANDRA-16938:
--
I don't see how it matters in a serialized format like json.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code:java}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. My guess is that the conversion was done trying to work
> around a malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16888) SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on unexpected key cache hit for non-cached key
[
https://issues.apache.org/jira/browse/CASSANDRA-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412662#comment-17412662
]
Caleb Rackliffe commented on CASSANDRA-16888:
-
What's interesting is that this test only seems to have been running since
CASSANDRA-12922. There's also {{OrderedJUnit4ClassRunner}} in play here, so
there might be some test state leakage that was papering over that's now a
problem...
> SSTableReaderTest#testGetPositionsKeyCacheStats failing sporadically on
> unexpected key cache hit for non-cached key
> ---
>
> Key: CASSANDRA-16888
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16888
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> This has started to pop up on 4.0.x and trunk builds in both CircleCI and
> Apache CI:
> [https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/60/workflows/15e09ea4-4d35-4035-9afc-ff0d1089041e/jobs/382]
> [https://app.circleci.com/pipelines/github/maedhroz/cassandra/332/workflows/3bd588d9-717f-4877-8e78-7a127180dfee/jobs/2093/tests#failed-test-0]
> https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1067/testReport/junit/org.apache.cassandra.io.sstable/SSTableReaderTest/testGetPositionsKeyCacheStats_cdc/
>
> It passes consistently in local runs, although I have seen one failure in
> about 10 runs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412660#comment-17412660
]
Ekaterina Dimitrova edited comment on CASSANDRA-16938 at 9/9/21, 4:05 PM:
--
Thank you [~adelapena] and [~ddimitrov].
The patch looks great but I think changes of output probably can be introduced
only in minor/major version. I wouldn't do it in a patch version probably.
(even if the old one is ugly, indeed :) )
I think [~brandon.williams] and [~mck] might have some thoughts to share here.
was (Author: e.dimitrova):
Thank you [~adelapena] and [~ddimitrov].
The patch looks great but I think changes of output probably can be introduced
only in minor/major version. I wouldn't do it in a patch version probably.
I think [~brandon.williams] and [~mck] might have some thoughts to share here.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code:java}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. My guess is that the conversion was done trying to work
> around a malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412660#comment-17412660
]
Ekaterina Dimitrova commented on CASSANDRA-16938:
-
Thank you [~adelapena] and [~ddimitrov].
The patch looks great but I think changes of output probably can be introduced
only in minor/major version. I wouldn't do it in a patch version probably.
I think [~brandon.williams] and [~mck] might have some thoughts to share here.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code:java}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. My guess is that the conversion was done trying to work
> around a malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16938:
--
Description:
The output of {{nodetool tpstats -F json}} always prints an empty map for
{{WaitLatencies}}:
{code:java}
$ nodetool tpstats -F json
(...)"WaitLatencies":{},(...)
{code}
The same happens with yaml output:
{code:java}
$ nodetool tpstats -F yaml
(...)
WaitLatencies: {}
(...)
{code}
This happens because [this
cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
silently fails inside a try-catch with an empty catch block:
{code:java}
String[] strValues = (String[])
Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
.map(D -> D.toString())
.toArray();
{code}
When we would need something like:
{code:java}
String[] strValues =
Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
.map(Object::toString)
.toArray(String[]::new);
{code}
This conversion from {{Double[]}} to {{String[]}} was introduced during
CASSANDRA-16230. My guess is that the conversion was done trying to work around
a malformed JSON output detected in [the new
tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
added by that ticket. Without the conversion the output would be something
like:
{code:java}
$ nodetool tpstats -F json
(...)"WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,(...)
{code}
That's because {{json-simple}} doesn't handle well arrays. I think that instead
of converting the array of doubles to an array of strings we can simply use
{{jackson-mapper-asl}} to print the proper array.
was:
The output of {{nodetool tpstats -F json}} always prints an empty map for
{{WaitLatencies}}:
{code}
$ nodetool tpstats -F json
..."WaitLatencies":{},...
{code}
The same happens with yaml output:
{code}
$ nodetool tpstats -F yaml
...
WaitLatencies: {}
...
{code}
This happens because [this
cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
silently fails inside a try-catch with an empty catch block:
{code}
String[] strValues = (String[])
Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
.map(D -> D.toString())
.toArray();
{code}
When we would need something like:
{code}
String[] strValues =
Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
.map(Object::toString)
.toArray(String[]::new);
{code}
This conversion from {{Double[]}} to {{String[]}} was introduced during
CASSANDRA-16230. I think that conversion was done trying to work around a
malformed JSON output detected in [the new
tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
added by that ticket. Without the conversion the output would be something
like:
{code}
$ nodetool tpstats -F json
..."WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,...
{code}
That's because {{json-simple}} doesn't handle well arrays. I think that instead
of converting the array of doubles to an array of strings we can simply use
{{jackson-mapper-asl}} to print the proper array.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code:java}
> $ nodetool tpstats -F json
> (...)"WaitLatencies":{},(...)
> {code}
> The same happens with yaml output:
> {code:java}
> $ nodetool tpstats -F yaml
> (...)
> WaitLatencies: {}
> (...)
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code:java}
> String[] strValues = (String[])
>
[jira] [Comment Edited] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412647#comment-17412647
]
Andres de la Peña edited comment on CASSANDRA-16938 at 9/9/21, 3:46 PM:
Please note that the change to {{jackson-mapper-asl}} will produce some
formatting differences in the output. The original output didn't have any
pretty-printing:
{code:json}
{"ThreadPools":{"ReadStage":{"TotalBlockedTasks":0,"ActiveTasks":0,"PendingTasks":0,"CurrentlyBlockedTasks":0,"CompletedTasks":1},"CompactionExecutor":(...)}
{code}
While the new output does use pretty-printing:
{code:json}
{
"ThreadPools" : {
"CompactionExecutor" : {
"TotalBlockedTasks" : 0,
"ActiveTasks" : 0,
"PendingTasks" : 0,
"CurrentlyBlockedTasks" : 0,
"CompletedTasks" : 41
},
"MemtableReclaimMemory" : {
(...)
}
{code}
Also the order of the attributes is different. I have attached examples of the
output after and before the changes. Of course the old output doesn't contain
the wait latency metrics.
These formatting differences shouldn't be a problem for anyone parsing the
output as proper json, but it might be an issue if someone is trying to parse
the output as regular text. If we think that that can be a problem we can
always keep using the old, ugly compact format.
was (Author: adelapena):
Please note that the change to {{jackson-mapper-asl}} will produce some
formatting differences in the output. The original output didn't have any
pretty-printing:
{code:json}
{"ThreadPools":{"ReadStage":{"TotalBlockedTasks":0,"ActiveTasks":0,"PendingTasks":0,"CurrentlyBlockedTasks":0,"CompletedTasks":1},"CompactionExecutor":(...)}
{code}
While the new output does use pretty-printing:
{code:json}
{
"ThreadPools" : {
"CompactionExecutor" : {
"TotalBlockedTasks" : 0,
"ActiveTasks" : 0,
"PendingTasks" : 0,
"CurrentlyBlockedTasks" : 0,
"CompletedTasks" : 41
},
"MemtableReclaimMemory" : {
(...)
}
{code}
I have attached examples of the output after and before the changes. Of course
the old output doesn't contain the wait latency metrics.
These formatting differences shouldn't be a problem for anyone parsing the
output as proper json, but it might be an issue if someone is trying to parse
the output as regular text. If we think that that can be a problem we can
always keep using the old, ugly compact format.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{},...
> {code}
> The same happens with yaml output:
> {code}
> $ nodetool tpstats -F yaml
> ...
> WaitLatencies: {}
> ...
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. I think that conversion was done trying to work around a
> malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,...
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CASSANDRA-14752) serializers/BooleanSerializer.java is using static bytebuffers which may cause problem for subsequent operations
[
https://issues.apache.org/jira/browse/CASSANDRA-14752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-14752:
---
Reviewers: Benjamin Lerer, Ekaterina Dimitrova (was: Ekaterina Dimitrova)
> serializers/BooleanSerializer.java is using static bytebuffers which may
> cause problem for subsequent operations
>
>
> Key: CASSANDRA-14752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14752
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Varun Barala
>Priority: Normal
> Fix For: 4.x
>
> Attachments: patch, patch-modified
>
>
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/serializers/BooleanSerializer.java#L26]
> It has two static Bytebuffer variables:-
> {code:java}
> private static final ByteBuffer TRUE = ByteBuffer.wrap(new byte[]{1});
> private static final ByteBuffer FALSE = ByteBuffer.wrap(new byte[]{0});{code}
> What will happen if the position of these Bytebuffers is being changed by
> some other operations? It'll affect other subsequent operations. -IMO Using
> static is not a good idea here.-
> A potential place where it can become problematic:
> [https://github.com/apache/cassandra/blob/cassandra-2.1.13/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java#L243]
> Since we are calling *`.remaining()`* It may give wrong results _i.e 0_ if
> these Bytebuffers have been used previously.
> Solution:
>
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/serializers/BooleanSerializer.java#L42]
> Every time we return new bytebuffer object. Please do let me know If there
> is a better way. I'd like to contribute. Thanks!!
> {code:java}
> public ByteBuffer serialize(Boolean value)
> {
> return (value == null) ? ByteBufferUtil.EMPTY_BYTE_BUFFER
> : value ? ByteBuffer.wrap(new byte[] {1}) : ByteBuffer.wrap(new byte[] {0});
> // false
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16841) Unexpectedly ignored dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-16841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412648#comment-17412648
]
Ruslan Fomkin commented on CASSANDRA-16841:
---
[~adelapena] [~e.dimitrova] I addressed your comments and reverted the change
about resource intensive tests, so force execution flag is always required if
not sufficient resources even in the case to execute only resource intensive
tests.
I run [j11_dtests in
CircleCI|https://app.circleci.com/pipelines/github/k-rus/cassandra/10/workflows/e56bdedd-45ba-466f-ae32-a003b9004063]
> Unexpectedly ignored dtests
> ---
>
> Key: CASSANDRA-16841
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16841
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ruslan Fomkin
>Assignee: Ruslan Fomkin
>Priority: Normal
> Time Spent: 2h
> Remaining Estimate: 0h
>
> An issue, which I was hit:
> When one class in a dtest file is marked as resource intensive, then all
> tests in all classes are treated as resource intensive. For example,
> [repair_tests/repair_test.py|https://github.com/apache/cassandra-dtest/blob/trunk/repair_tests/repair_test.py]
> contains three classes and the last class is marked as resource intensive:
> {code:java}
> @pytest.mark.resource_intensive
> class TestRepairDataSystemTable(Tester):
> {code}
> So if I try to run an unmarked class:
> {code:java}
> pytest --cassandra-dir=../cassandra repair_tests/repair_test.py::TestRepair
> --collect-only --skip-resource-intensive-tests
> {code}
> then all tests are ignored
> {code:java}
> collected 36 items / 36 deselected
> {code}
> This is because a test is treated to be marked if any class in the same file
> has the mark. This bug was introduced in the fix of CASS-16399. Before only
> upgrade tests had such behaviour, i.e., if a class is marked as upgrade test,
> then all tests are upgrade test in the file.
>
> This bug, for example, means that if the same file contains one class marked
> with vnodes and another class with no_vnodes, then no tests will be executed
> in the file.
> I also noticed another issue that If a test run is executed with the argument
> {{-only-resource-intensive-tests}} and there is no sufficient resources for
> resource intensive tests, then no tests were executed. Thus it was necessary
> to provide {{-force-resource-intensive-tests}} in addition.
> Suggestions for the solutions:
> # Require to mark each class and remove the special case of upgrade tests.
> This will simplify the implementation and might be more obvious for new
> comers.
> # Treat {{-only-resource-intensive-tests}} in the same way as
> {{-force-resource-intensive-tests}}, so it will be enough to just specify it
> even with no sufficient resources.
> *Update:* comments were provided to keep only the first suggestion and do not
> implement the second suggestion.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412647#comment-17412647
]
Andres de la Peña commented on CASSANDRA-16938:
---
Please note that the change to {{jackson-mapper-asl}} will produce some
formatting differences in the output. The original output didn't have any
pretty-printing:
{code:json}
{"ThreadPools":{"ReadStage":{"TotalBlockedTasks":0,"ActiveTasks":0,"PendingTasks":0,"CurrentlyBlockedTasks":0,"CompletedTasks":1},"CompactionExecutor":(...)}
{code}
While the new output does use pretty-printing:
{code:json}
{
"ThreadPools" : {
"CompactionExecutor" : {
"TotalBlockedTasks" : 0,
"ActiveTasks" : 0,
"PendingTasks" : 0,
"CurrentlyBlockedTasks" : 0,
"CompletedTasks" : 41
},
"MemtableReclaimMemory" : {
(...)
}
{code}
I have attached examples of the output after and before the changes. Of course
the old output doesn't contain the wait latency metrics.
These formatting differences shouldn't be a problem for anyone parsing the
output as proper json, but it might be an issue if someone is trying to parse
the output as regular text. If we think that that can be a problem we can
always keep using the old, ugly compact format.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{},...
> {code}
> The same happens with yaml output:
> {code}
> $ nodetool tpstats -F yaml
> ...
> WaitLatencies: {}
> ...
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. I think that conversion was done trying to work around a
> malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,...
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16938:
--
Attachment: json-after.txt
json-before.txt
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
> Attachments: json-after.txt, json-before.txt
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{},...
> {code}
> The same happens with yaml output:
> {code}
> $ nodetool tpstats -F yaml
> ...
> WaitLatencies: {}
> ...
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. I think that conversion was done trying to work around a
> malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,...
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-16938:
Reviewers: Ekaterina Dimitrova, Ekaterina Dimitrova (was: Ekaterina
Dimitrova)
Ekaterina Dimitrova, Ekaterina Dimitrova
Status: Review In Progress (was: Patch Available)
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{},...
> {code}
> The same happens with yaml output:
> {code}
> $ nodetool tpstats -F yaml
> ...
> WaitLatencies: {}
> ...
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. I think that conversion was done trying to work around a
> malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,...
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16938:
--
Test and Documentation Plan: Some extra tests are included in the PRs
Status: Patch Available (was: In Progress)
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{},...
> {code}
> The same happens with yaml output:
> {code}
> $ nodetool tpstats -F yaml
> ...
> WaitLatencies: {}
> ...
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. I think that conversion was done trying to work around a
> malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,...
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16938:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Transient
Incorrect Response(12987)
Complexity: Normal
Discovered By: User Report
Fix Version/s: 4.0.x
4.1
Severity: Low
Status: Open (was: Triage Needed)
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.1, 4.0.x
>
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{},...
> {code}
> The same happens with yaml output:
> {code}
> $ nodetool tpstats -F yaml
> ...
> WaitLatencies: {}
> ...
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. I think that conversion was done trying to work around a
> malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,...
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
[
https://issues.apache.org/jira/browse/CASSANDRA-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412633#comment-17412633
]
Andres de la Peña commented on CASSANDRA-16938:
---
The proposed patch reverts the conversion from {{Double[]}} to {{String[]}} in
{{TpStatsHolder}} and uses {{jackson-mapper-asl}} instead of {{json-simple}} to
generate the json output:
||PR||CI||
|[4.0|https://github.com/apache/cassandra/pull/1190]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/837/workflows/494afd37-92b7-4127-9626-40cbc279a90c]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/837/workflows/b8fee35e-5b60-4590-80b6-ffa53858b741]|
|[trunk|https://github.com/apache/cassandra/pull/1191]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16938-trunk]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/838/workflows/1ec0b51b-eb8d-4f55-b9d6-7b6d06dcce45]|
All praise to [~dimitarndimitrov] who originally found the problem with
{{json-simple}} and wrote the changes for {{StatsPrinter}}. The changes in
{{TpStatsHolder}} and the new tests are mine.
> Missed wait latencies in the output of `nodetool tpstats -F`
>
>
> Key: CASSANDRA-16938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> The output of {{nodetool tpstats -F json}} always prints an empty map for
> {{WaitLatencies}}:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{},...
> {code}
> The same happens with yaml output:
> {code}
> $ nodetool tpstats -F yaml
> ...
> WaitLatencies: {}
> ...
> {code}
> This happens because [this
> cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
> silently fails inside a try-catch with an empty catch block:
> {code}
> String[] strValues = (String[])
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
> .map(D -> D.toString())
> .toArray();
> {code}
> When we would need something like:
> {code}
> String[] strValues =
> Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
>.map(Object::toString)
>.toArray(String[]::new);
> {code}
> This conversion from {{Double[]}} to {{String[]}} was introduced during
> CASSANDRA-16230. I think that conversion was done trying to work around a
> malformed JSON output detected in [the new
> tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
> added by that ticket. Without the conversion the output would be something
> like:
> {code}
> $ nodetool tpstats -F json
> ..."WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,...
> {code}
> That's because {{json-simple}} doesn't handle well arrays. I think that
> instead of converting the array of doubles to an array of strings we can
> simply use {{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16938) Missed wait latencies in the output of `nodetool tpstats -F`
Andres de la Peña created CASSANDRA-16938:
-
Summary: Missed wait latencies in the output of `nodetool tpstats
-F`
Key: CASSANDRA-16938
URL: https://issues.apache.org/jira/browse/CASSANDRA-16938
Project: Cassandra
Issue Type: Bug
Components: Tool/nodetool
Reporter: Andres de la Peña
Assignee: Andres de la Peña
The output of {{nodetool tpstats -F json}} always prints an empty map for
{{WaitLatencies}}:
{code}
$ nodetool tpstats -F json
..."WaitLatencies":{},...
{code}
The same happens with yaml output:
{code}
$ nodetool tpstats -F yaml
...
WaitLatencies: {}
...
{code}
This happens because [this
cast|https://github.com/apache/cassandra/blob/cassandra-4.0.1/src/java/org/apache/cassandra/tools/nodetool/stats/TpStatsHolder.java#L61-L63]
silently fails inside a try-catch with an empty catch block:
{code}
String[] strValues = (String[])
Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
.map(D -> D.toString())
.toArray();
{code}
When we would need something like:
{code}
String[] strValues =
Arrays.stream(probe.metricPercentilesAsArray(probe.getMessagingQueueWaitMetrics(entry.getKey(
.map(Object::toString)
.toArray(String[]::new);
{code}
This conversion from {{Double[]}} to {{String[]}} was introduced during
CASSANDRA-16230. I think that conversion was done trying to work around a
malformed JSON output detected in [the new
tests|https://github.com/apache/cassandra/blob/cassandra-4.0.1/test/unit/org/apache/cassandra/tools/NodeToolTPStatsTest.java#L158]
added by that ticket. Without the conversion the output would be something
like:
{code}
$ nodetool tpstats -F json
..."WaitLatencies":{"READ_RSP":[Ljava.lang.Double;@398dada8,...
{code}
That's because {{json-simple}} doesn't handle well arrays. I think that instead
of converting the array of doubles to an array of strings we can simply use
{{jackson-mapper-asl}} to print the proper array.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14196) replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails without vnodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412625#comment-17412625 ] Ekaterina Dimitrova commented on CASSANDRA-14196: - Example usages can be found [here|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml#L39-L91] About number of repetitions - I would say we normally use our own judgement based on experience. For more complicated tests I normally raise the repetitions. You can also set the job to [stop|https://github.com/apache/cassandra/blob/trunk/.circleci/config-2_1.yml#L70] in case of failure if you want to. Please feel free to ping me if you are unsure or have issues to run the job. Thank you > replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails > without vnodes > --- > > Key: CASSANDRA-14196 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14196 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/Testing >Reporter: Ariel Weisberg >Assignee: Ruslan Fomkin >Priority: Normal > > Skipping it for now, but it probably should pass without vnodes. > https://circleci.com/gh/aweisberg/cassandra/871#tests/containers/15 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-11245) (windows) dtest failure in commitlog_test.TestCommitLog.die_failure_policy_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-11245:
-
Resolution: Invalid
Status: Resolved (was: Open)
Windows support is now removed.
> (windows) dtest failure in
> commitlog_test.TestCommitLog.die_failure_policy_test
> ---
>
> Key: CASSANDRA-11245
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11245
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/python
>Reporter: Russ Hatch
>Priority: Normal
> Labels: dtest, windows
>
> example failure:
> http://cassci.datastax.com/job/cassandra-2.2_dtest_win32/165/testReport/commitlog_test/TestCommitLog/die_failure_policy_test
> Failed on CassCI build cassandra-2.2_dtest_win32 #165
> flaky test, fails intermittently, error message:
> {noformat}
> Cannot find the commitlog failure message in logs
> {noformat}
> looks like it could be the same cause or related to CASSANDRA-11242
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14196) replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails without vnodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412610#comment-17412610 ] Ruslan Fomkin commented on CASSANDRA-14196: --- [~e.dimitrova] Thank you for brining it. I heard about it but haven't tried. So I am not fully sure how it works. If it will be enough just to enabled repeated_dtest in CI or do I need to do additional adjustments for the number of runs. > replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails > without vnodes > --- > > Key: CASSANDRA-14196 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14196 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/Testing >Reporter: Ariel Weisberg >Assignee: Ruslan Fomkin >Priority: Normal > > Skipping it for now, but it probably should pass without vnodes. > https://circleci.com/gh/aweisberg/cassandra/871#tests/containers/15 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14196) replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails without vnodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412604#comment-17412604 ] Ekaterina Dimitrova commented on CASSANDRA-14196: - Hey [~k-rus], thank you for picking this one! As you mentioned CircleCI and flakiness, just wanted to bring your attention to the point we have there a multiplexer that can be used to run a test in a loop and ensure no flakiness. Excuse me if you already knew about it and I am just making noise. ;) > replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails > without vnodes > --- > > Key: CASSANDRA-14196 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14196 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/Testing >Reporter: Ariel Weisberg >Assignee: Ruslan Fomkin >Priority: Normal > > Skipping it for now, but it probably should pass without vnodes. > https://circleci.com/gh/aweisberg/cassandra/871#tests/containers/15 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14196) replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails without vnodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412604#comment-17412604 ] Ekaterina Dimitrova edited comment on CASSANDRA-14196 at 9/9/21, 2:14 PM: -- Hey [~k-rus], thank you for picking this one! As you mentioned CircleCI and flakiness, just wanted to bring your attention to the point we have multiplexer jobs in CircleCI now that can be used to run a test in a loop and ensure no flakiness. Excuse me if you already knew about it and I am just making noise. ;) was (Author: e.dimitrova): Hey [~k-rus], thank you for picking this one! As you mentioned CircleCI and flakiness, just wanted to bring your attention to the point we have there a multiplexer that can be used to run a test in a loop and ensure no flakiness. Excuse me if you already knew about it and I am just making noise. ;) > replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails > without vnodes > --- > > Key: CASSANDRA-14196 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14196 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/Testing >Reporter: Ariel Weisberg >Assignee: Ruslan Fomkin >Priority: Normal > > Skipping it for now, but it probably should pass without vnodes. > https://circleci.com/gh/aweisberg/cassandra/871#tests/containers/15 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16860) Add --older-than option to nodetool clearsnapshot
[
https://issues.apache.org/jira/browse/CASSANDRA-16860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412579#comment-17412579
]
Stefan Miklosovic commented on CASSANDRA-16860:
---
[~jackcasey-visier] great, let us know how it goes!
> Add --older-than option to nodetool clearsnapshot
> -
>
> Key: CASSANDRA-16860
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16860
> Project: Cassandra
> Issue Type: New Feature
> Components: Local/Snapshots, Tool/nodetool
>Reporter: Jack Casey
>Assignee: Jack Casey
>Priority: Normal
> Fix For: 4.x
>
>
> h1. Summary
> Opening this issue in reference to [this WIP
> PR|https://github.com/apache/cassandra/pull/1148]:
> This functionality allows users of Cassandra to remove snapshots ad-hoc,
> based on a TTL. This is to address the problem of snapshots accumulating. For
> example, an organization I work for aims to keep snapshots for 30 days,
> however we don't have any way to easily clean them after those 30 days are up.
> This is similar to the goals set in:
> https://issues.apache.org/jira/browse/CASSANDRA-16451 however would be
> available for Cassandra 3.x.
> h1. Functionality
> This adds a new command to NodeTool, called {{expiresnapshot}} with the
> following options:
> NAME
> nodetool expiresnapshots - Removes snapshots that are older than a TTL
> in days
> SYNOPSIS
> nodetool [(-h | --host )] [(-p | --port )]
> [(-pw | --password )]
> [(-pwf | --password-file )]
> [(-u | --username )] expiresnapshots [--dry-run]
> (-t | --ttl )
> OPTIONS
> --dry-run
> Run without actually clearing snapshots
> -h , --host
> Node hostname or ip address
> -p , --port
> Remote jmx agent port number
> -pw , --password
> Remote jmx agent password
> -pwf , --password-file
> Path to the JMX password file
> -t , --ttl
> TTL (in days) to expire snapshots
> -u , --username
> Remote jmx agent username
> The snapshot date is taken by converting the default snapshot name timestamps
> (epoch time in miliseconds). For this reason, snapshot names that don't
> contain a timestamp in this format will not be cleared.
> h1. Example Use
> This Cassandra environment has a number of snapshots, a few are recent, and a
> few outdated:
> root@cassandra001:/cassandra# nodetool listsnapshots
> Snapshot Details:
> Snapshot name Keyspace name Column family name True size Size on disk
> 1529173922063 users_keyspace users 362.03 KiB 362.89 KiB
> 1629173909461 users_keyspace users 362.03 KiB 362.89 KiB
> 1629173922063 users_keyspace users 362.03 KiB 362.89 KiB
> 1599173922063 users_keyspace users 362.03 KiB 362.89 KiB
> 1629173916816 users_keyspace users 362.03 KiB 362.89 KiB
> Total TrueDiskSpaceUsed: 1.77 MiB
> To validate the removal runs as expected, we can use the `--dry-run` option:
> root@cassandra001:/cassandra# nodetool expiresnapshots --ttl 30 --dry-run
> Starting simulated cleanup of snapshots older than 30 days
> Clearing (dry run): 1529173922063
> Clearing (dry run): 1599173922063
> Cleared (dry run): 2 snapshots
> Now that we are confident the correct snapshots will be removed, we can omit
> the {{--dry-run}} flag:
> root@cassandra001:/cassandra# nodetool expiresnapshots --ttl 30
> Starting cleanup of snapshots older than 30 days
> Clearing: 1529173922063
> Clearing: 1599173922063
> Cleared: 2 snapshots
> To confirm our changes are successful, we list the snapshots that still
> remain:
> root@cassandra001:/cassandra# nodetool listsnapshots
> Snapshot Details:
> Snapshot name Keyspace name Column family name True size Size on disk
> 1629173909461 users_keyspace users 362.03 KiB 362.89 KiB
> 1629173922063 users_keyspace users 362.03 KiB 362.89 KiB
> 1629173916816 users_keyspace users 362.03 KiB 362.89 KiB
> Total TrueDiskSpaceUsed: 1.06 MiB
> h1. Next Steps
> To be completed:
> - Tests
> - Documentation updates
> I am a new to this repository, and am fuzzy on a few details even after
> reading the contribution guide Any advice on the following would be greatly
> appreciated!
> - What branch would this type of change be merged into? Currently, I'm
> targeting {{apache:trunk}} by default
> - Is there a test strategy/pattern for this type of change? I was not able
> to find any existing tests for similar {{nodetool}} commands
> Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16886) Reduce native_transport_max_frame_size_in_mb
[ https://issues.apache.org/jira/browse/CASSANDRA-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412569#comment-17412569 ] Benjamin Lerer commented on CASSANDRA-16886: The patch looks good to me. > Reduce native_transport_max_frame_size_in_mb > > > Key: CASSANDRA-16886 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16886 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > > There is really no point in having this set to 256MB when the commitlog > segment size defaults to 32MB, effectively capping the largest insertable > mutation to 16MB. > The native transport can provide a good first line of defense against large > mutations that would otherwise hit the heap if left to be rejected by the > commitlog. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14752) serializers/BooleanSerializer.java is using static bytebuffers which may cause problem for subsequent operations
[
https://issues.apache.org/jira/browse/CASSANDRA-14752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-14752:
Status: Changes Suggested (was: Review In Progress)
> serializers/BooleanSerializer.java is using static bytebuffers which may
> cause problem for subsequent operations
>
>
> Key: CASSANDRA-14752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14752
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Varun Barala
>Priority: Normal
> Fix For: 4.x
>
> Attachments: patch, patch-modified
>
>
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/serializers/BooleanSerializer.java#L26]
> It has two static Bytebuffer variables:-
> {code:java}
> private static final ByteBuffer TRUE = ByteBuffer.wrap(new byte[]{1});
> private static final ByteBuffer FALSE = ByteBuffer.wrap(new byte[]{0});{code}
> What will happen if the position of these Bytebuffers is being changed by
> some other operations? It'll affect other subsequent operations. -IMO Using
> static is not a good idea here.-
> A potential place where it can become problematic:
> [https://github.com/apache/cassandra/blob/cassandra-2.1.13/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java#L243]
> Since we are calling *`.remaining()`* It may give wrong results _i.e 0_ if
> these Bytebuffers have been used previously.
> Solution:
>
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/serializers/BooleanSerializer.java#L42]
> Every time we return new bytebuffer object. Please do let me know If there
> is a better way. I'd like to contribute. Thanks!!
> {code:java}
> public ByteBuffer serialize(Boolean value)
> {
> return (value == null) ? ByteBufferUtil.EMPTY_BYTE_BUFFER
> : value ? ByteBuffer.wrap(new byte[] {1}) : ByteBuffer.wrap(new byte[] {0});
> // false
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14752) serializers/BooleanSerializer.java is using static bytebuffers which may cause problem for subsequent operations
[
https://issues.apache.org/jira/browse/CASSANDRA-14752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412564#comment-17412564
]
Ekaterina Dimitrova commented on CASSANDRA-14752:
-
Hey [~varuna], apologize for the very late response.
I think we can simplify a bit by reducing the patch to changing [this
line|https://github.com/Barala/cassandra/commit/2328cda4d4e12e75bc1e9606969f18dd72d4fc27#diff-821957692da5432eb814312397d145fc6445ac35856563b21895e0c1df82ca3aL235]
to
{code:java}
bb.put(component.duplicate()); // it's not ok to consume component as we did
not create it{code}
What do you think?
[~blerer] , do you mind also to review it? I see it is also marked for 4.x but
I think we can port it also to at least 4.0.x.
> serializers/BooleanSerializer.java is using static bytebuffers which may
> cause problem for subsequent operations
>
>
> Key: CASSANDRA-14752
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14752
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Varun Barala
>Priority: Normal
> Fix For: 4.x
>
> Attachments: patch, patch-modified
>
>
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/serializers/BooleanSerializer.java#L26]
> It has two static Bytebuffer variables:-
> {code:java}
> private static final ByteBuffer TRUE = ByteBuffer.wrap(new byte[]{1});
> private static final ByteBuffer FALSE = ByteBuffer.wrap(new byte[]{0});{code}
> What will happen if the position of these Bytebuffers is being changed by
> some other operations? It'll affect other subsequent operations. -IMO Using
> static is not a good idea here.-
> A potential place where it can become problematic:
> [https://github.com/apache/cassandra/blob/cassandra-2.1.13/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java#L243]
> Since we are calling *`.remaining()`* It may give wrong results _i.e 0_ if
> these Bytebuffers have been used previously.
> Solution:
>
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/serializers/BooleanSerializer.java#L42]
> Every time we return new bytebuffer object. Please do let me know If there
> is a better way. I'd like to contribute. Thanks!!
> {code:java}
> public ByteBuffer serialize(Boolean value)
> {
> return (value == null) ? ByteBufferUtil.EMPTY_BYTE_BUFFER
> : value ? ByteBuffer.wrap(new byte[] {1}) : ByteBuffer.wrap(new byte[] {0});
> // false
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16937) cassandra local_quorum query is inconsistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412526#comment-17412526
]
Brandon Williams edited comment on CASSANDRA-16937 at 9/9/21, 11:59 AM:
2.1 is also past EOL, so this jira is the wrong place to seek assistance.
Consider contacting the community via slack or the ML instead:
https://cassandra.apache.org/_/community.html
was (Author: brandon.williams):
2.1 is also past EOL.
> cassandra local_quorum query is inconsistent
> -
>
> Key: CASSANDRA-16937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16937
> Project: Cassandra
> Issue Type: Bug
>Reporter: HUANG DUICAN
>Priority: Normal
>
> The version number of this issue is
> wrong：https://issues.apache.org/jira/browse/CASSANDRA-16919
> update the version: cassandra version: 2.1.15
> Number of nodes: dc1: 80, dc2: 80
> problem:
> Our copy strategy is as follows:
> WITH REPLICATION = \{'class':'NetworkTopologyStrategy','dc1': 3,'dc2': 3};
> We encountered a problem with cassandra, and it was inconsistent when
> querying with local_quorum. We will only read and write in dc1.
> We also use local_quorum for writing, and then use local_quorum for queries.
> But there is a phenomenon, use the following statement:
> select count(*) from table where partitionKey=?
> The results of the query were initially inconsistent and eventually
> consistent.
> Assuming that the first is 1, the second is 9998, and the third is 9997,
> it may remain at 10001 in the end(Maybe it was triggered to read repair,
> which led to the final stabilization) .
> During this period, we have done a large-scale expansion. And make sure that
> every machine is cleaned up. And we also found that the results of using
> getEndpointon different machines are inconsistent.
> In the end, we found that the result of getEndpoint has 4 machines in dc1.
> Then we executed getSstable on the corresponding 4 machines, only 3 machines
> showed the results, and the other machine did not show the results. At the
> same time, we encountered a similar problem with another partitionKey, but
> this partitionKey was only queried once, because we recorded the total number
> of partitionKey in another place, and we can confirm that the total number of
> partitionKey is incorrect.
> After we restarted each machine of dc1 one by one, this problem was solved.
> The total number of partitionKey is consistent with the result recorded by
> us, and if the same query is done multiple times, the result will not change.
> Therefore, I suspect that the gossip synchronization node information is too
> slow, which may lead to inconsistent final results when selecting nodes for
> query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16937) cassandra local_quorum query is inconsistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-16937:
-
Resolution: Duplicate
Status: Resolved (was: Triage Needed)
> cassandra local_quorum query is inconsistent
> -
>
> Key: CASSANDRA-16937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16937
> Project: Cassandra
> Issue Type: Bug
>Reporter: HUANG DUICAN
>Priority: Normal
>
> The version number of this issue is
> wrong：https://issues.apache.org/jira/browse/CASSANDRA-16919
> update the version: cassandra version: 2.1.15
> Number of nodes: dc1: 80, dc2: 80
> problem:
> Our copy strategy is as follows:
> WITH REPLICATION = \{'class':'NetworkTopologyStrategy','dc1': 3,'dc2': 3};
> We encountered a problem with cassandra, and it was inconsistent when
> querying with local_quorum. We will only read and write in dc1.
> We also use local_quorum for writing, and then use local_quorum for queries.
> But there is a phenomenon, use the following statement:
> select count(*) from table where partitionKey=?
> The results of the query were initially inconsistent and eventually
> consistent.
> Assuming that the first is 1, the second is 9998, and the third is 9997,
> it may remain at 10001 in the end(Maybe it was triggered to read repair,
> which led to the final stabilization) .
> During this period, we have done a large-scale expansion. And make sure that
> every machine is cleaned up. And we also found that the results of using
> getEndpointon different machines are inconsistent.
> In the end, we found that the result of getEndpoint has 4 machines in dc1.
> Then we executed getSstable on the corresponding 4 machines, only 3 machines
> showed the results, and the other machine did not show the results. At the
> same time, we encountered a similar problem with another partitionKey, but
> this partitionKey was only queried once, because we recorded the total number
> of partitionKey in another place, and we can confirm that the total number of
> partitionKey is incorrect.
> After we restarted each machine of dc1 one by one, this problem was solved.
> The total number of partitionKey is consistent with the result recorded by
> us, and if the same query is done multiple times, the result will not change.
> Therefore, I suspect that the gossip synchronization node information is too
> slow, which may lead to inconsistent final results when selecting nodes for
> query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16937) cassandra local_quorum query is inconsistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412526#comment-17412526
]
Brandon Williams commented on CASSANDRA-16937:
--
2.1 is also past EOL.
> cassandra local_quorum query is inconsistent
> -
>
> Key: CASSANDRA-16937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16937
> Project: Cassandra
> Issue Type: Bug
>Reporter: HUANG DUICAN
>Priority: Normal
>
> The version number of this issue is
> wrong：https://issues.apache.org/jira/browse/CASSANDRA-16919
> update the version: cassandra version: 2.1.15
> Number of nodes: dc1: 80, dc2: 80
> problem:
> Our copy strategy is as follows:
> WITH REPLICATION = \{'class':'NetworkTopologyStrategy','dc1': 3,'dc2': 3};
> We encountered a problem with cassandra, and it was inconsistent when
> querying with local_quorum. We will only read and write in dc1.
> We also use local_quorum for writing, and then use local_quorum for queries.
> But there is a phenomenon, use the following statement:
> select count(*) from table where partitionKey=?
> The results of the query were initially inconsistent and eventually
> consistent.
> Assuming that the first is 1, the second is 9998, and the third is 9997,
> it may remain at 10001 in the end(Maybe it was triggered to read repair,
> which led to the final stabilization) .
> During this period, we have done a large-scale expansion. And make sure that
> every machine is cleaned up. And we also found that the results of using
> getEndpointon different machines are inconsistent.
> In the end, we found that the result of getEndpoint has 4 machines in dc1.
> Then we executed getSstable on the corresponding 4 machines, only 3 machines
> showed the results, and the other machine did not show the results. At the
> same time, we encountered a similar problem with another partitionKey, but
> this partitionKey was only queried once, because we recorded the total number
> of partitionKey in another place, and we can confirm that the total number of
> partitionKey is incorrect.
> After we restarted each machine of dc1 one by one, this problem was solved.
> The total number of partitionKey is consistent with the result recorded by
> us, and if the same query is done multiple times, the result will not change.
> Therefore, I suspect that the gossip synchronization node information is too
> slow, which may lead to inconsistent final results when selecting nodes for
> query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16937) cassandra local_quorum query is inconsistent
HUANG DUICAN created CASSANDRA-16937:
Summary: cassandra local_quorum query is inconsistent
Key: CASSANDRA-16937
URL: https://issues.apache.org/jira/browse/CASSANDRA-16937
Project: Cassandra
Issue Type: Bug
Reporter: HUANG DUICAN
The version number of this issue is
wrong：https://issues.apache.org/jira/browse/CASSANDRA-16919
update the version: cassandra version: 2.1.15
Number of nodes: dc1: 80, dc2: 80
problem:
Our copy strategy is as follows:
WITH REPLICATION = \{'class':'NetworkTopologyStrategy','dc1': 3,'dc2': 3};
We encountered a problem with cassandra, and it was inconsistent when querying
with local_quorum. We will only read and write in dc1.
We also use local_quorum for writing, and then use local_quorum for queries.
But there is a phenomenon, use the following statement:
select count(*) from table where partitionKey=?
The results of the query were initially inconsistent and eventually consistent.
Assuming that the first is 1, the second is 9998, and the third is 9997, it
may remain at 10001 in the end(Maybe it was triggered to read repair, which led
to the final stabilization) .
During this period, we have done a large-scale expansion. And make sure that
every machine is cleaned up. And we also found that the results of using
getEndpointon different machines are inconsistent. In
the end, we found that the result of getEndpoint has 4 machines in dc1.
Then we executed getSstable on the corresponding 4 machines, only 3 machines
showed the results, and the other machine did not show the results. At the same
time, we encountered a similar problem with another partitionKey, but this
partitionKey was only queried once, because we recorded the total number of
partitionKey in another place, and we can confirm that the total number of
partitionKey is incorrect.
After we restarted each machine of dc1 one by one, this problem was solved.
The total number of partitionKey is consistent with the result recorded by us,
and if the same query is done multiple times, the result will not change.
Therefore, I suspect that the gossip synchronization node information is too
slow, which may lead to inconsistent final results when selecting nodes for
query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14196) replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails without vnodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Fomkin reassigned CASSANDRA-14196: - Assignee: Ruslan Fomkin (was: Ariel Weisberg) > replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails > without vnodes > --- > > Key: CASSANDRA-14196 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14196 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/Testing >Reporter: Ariel Weisberg >Assignee: Ruslan Fomkin >Priority: Normal > > Skipping it for now, but it probably should pass without vnodes. > https://circleci.com/gh/aweisberg/cassandra/871#tests/containers/15 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14196) replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails without vnodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412463#comment-17412463 ] Ruslan Fomkin commented on CASSANDRA-14196: --- [~aweisberg] if you don't mind, I will assign the ticket to me and will run this test without vnodes on CircleCI to see if it is not the issue any more. Let me know if you prefer to keep this ticket on you, or you have comments to add to the ticket. Thank you. > replace_address_test:TestReplaceAddress.test_multi_dc_replace_with_rf1 fails > without vnodes > --- > > Key: CASSANDRA-14196 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14196 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/Testing >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Normal > > Skipping it for now, but it probably should pass without vnodes. > https://circleci.com/gh/aweisberg/cassandra/871#tests/containers/15 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
