[jira] [Comment Edited] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[
https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021835#comment-17021835
]
Gaurav Ashok edited comment on CASSANDRA-15358 at 1/23/20 7:51 AM:
---
Hi,
I also experienced this error while testing out 4.alpha2 version.
I have been testing the incremental repair.
In one of the test a node was missing 30GB of data. (~ 100GB total data). Soon
after starting the repair, I get this exception and the repair process stops
making any progress.
I do get "Maximum memory usage reached (2.000GiB), cannot allocate chunk of
8.000MiB" before the exception.
I am attaching the trace of a repair process and the debug logs.
[^repair_1_trace.txt]
[^debug_logs_during_repair.txt]
Any way I can test out this patch?
was (Author: g-ashok):
Hi,
I also experienced this error while testing out 4.alpha2 version.
I have been testing the incremental repair.
In one of the test a node was missing 30GB of data. (~ 100GB total data). Soon
after starting the repair, I get this exception and the repair process stops
making any progress.
I do get "Maximum memory usage reached (2.000GiB), cannot allocate chunk of
8.000MiB" before the exception.
I am attaching the trace of a repair process and the debug logs.
[^repair_1_trace.txt]
[^debug_logs_during_repair.txt]
Anyway I can test out this patch?
> Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
>
>
> Key: CASSANDRA-15358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15358
> Project: Cassandra
> Issue Type: Bug
> Components: Test/benchmark
>Reporter: Santhosh Kumar Ramalingam
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Labels: 4.0, alpha
> Attachments: all_errors.txt, debug_logs_during_repair.txt,
> repair_1_trace.txt, verbose_logs.diff, verbose_logs.txt
>
>
> Hitting a bug with cassandra 4 alpha version. The same bug is repeated with
> difefrent version of Java(8,11 &12) [~benedict]
>
> Stack trace:
> {code:java}
> INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for
> CQL clients on /1.3.0.6:9042 (unencrypted)...
> WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 -
> 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message
> of type PING_REQ whose timeout expired before reaching the network
> WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 -
> 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..)
> failed: Connection refused: /1.3.0.2:7000
> Caused by: java.net.ConnectException: finishConnect(..) failed: Connection
> refused
> at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124)
> at io.netty.channel.unix.Socket.finishConnect(Socket.java:243)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> WARN [Messaging-EventLoop-3-3] 2019-10-11 16:11:32,639 NoSpamLogger.java:94 -
> 1.3.4.6:7000->1.3.4.5:7000-URGENT_MESSAGES-[no-channel] dropping message of
> type GOSSIP_DIGEST_SYN whose timeout expired before reaching the network
> INFO [Messaging-EventLoop-3-18] 2019-10-11 16:11:33,077 NoSpamLogger.java:91
> - 1.3.4.5:7000->1.3.4.4:7000-URGENT_MESSAGES-[no-channel] failed to connect
>
> ERROR [Messaging-EventLoop-3-11]
[jira] [Commented] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[
https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021835#comment-17021835
]
Gaurav Ashok commented on CASSANDRA-15358:
--
Hi,
I also experienced this error while testing out 4.alpha2 version.
I have been testing the incremental repair.
In one of the test a node was missing 30GB of data. (~ 100GB total data). Soon
after starting the repair, I get this exception and the repair process stops
making any progress.
I do get "Maximum memory usage reached (2.000GiB), cannot allocate chunk of
8.000MiB" before the exception.
I am attaching the trace of a repair process and the debug logs.
[^repair_1_trace.txt]
[^debug_logs_during_repair.txt]
Anyway I can test out this patch?
> Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
>
>
> Key: CASSANDRA-15358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15358
> Project: Cassandra
> Issue Type: Bug
> Components: Test/benchmark
>Reporter: Santhosh Kumar Ramalingam
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Labels: 4.0, alpha
> Attachments: all_errors.txt, debug_logs_during_repair.txt,
> repair_1_trace.txt, verbose_logs.diff, verbose_logs.txt
>
>
> Hitting a bug with cassandra 4 alpha version. The same bug is repeated with
> difefrent version of Java(8,11 &12) [~benedict]
>
> Stack trace:
> {code:java}
> INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for
> CQL clients on /1.3.0.6:9042 (unencrypted)...
> WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 -
> 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message
> of type PING_REQ whose timeout expired before reaching the network
> WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 -
> 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..)
> failed: Connection refused: /1.3.0.2:7000
> Caused by: java.net.ConnectException: finishConnect(..) failed: Connection
> refused
> at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124)
> at io.netty.channel.unix.Socket.finishConnect(Socket.java:243)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> WARN [Messaging-EventLoop-3-3] 2019-10-11 16:11:32,639 NoSpamLogger.java:94 -
> 1.3.4.6:7000->1.3.4.5:7000-URGENT_MESSAGES-[no-channel] dropping message of
> type GOSSIP_DIGEST_SYN whose timeout expired before reaching the network
> INFO [Messaging-EventLoop-3-18] 2019-10-11 16:11:33,077 NoSpamLogger.java:91
> - 1.3.4.5:7000->1.3.4.4:7000-URGENT_MESSAGES-[no-channel] failed to connect
>
> ERROR [Messaging-EventLoop-3-11] 2019-10-10 01:34:34,407
> InboundMessageHandler.java:657 -
> 1.3.4.5:7000->1.3.4.8:7000-LARGE_MESSAGES-0b7d09cd unexpected exception
> caught while processing inbound messages; terminating connection
> java.lang.IllegalArgumentException: initialBuffer is not a direct buffer.
> at io.netty.buffer.UnpooledDirectByteBuf.(UnpooledDirectByteBuf.java:87)
> at
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:59)
> at
> org.apache.cassandra.net.BufferPoolAllocator$Wrapped.(BufferPoolAllocator.java:95)
> at
> org.apache.cassandra.net.BufferPoolAllocator.newDirectBuffer(BufferPoolAllocator.java:56)
> at
>
[jira] [Updated] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[
https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gaurav Ashok updated CASSANDRA-15358:
-
Attachment: debug_logs_during_repair.txt
> Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
>
>
> Key: CASSANDRA-15358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15358
> Project: Cassandra
> Issue Type: Bug
> Components: Test/benchmark
>Reporter: Santhosh Kumar Ramalingam
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Labels: 4.0, alpha
> Attachments: all_errors.txt, debug_logs_during_repair.txt,
> repair_1_trace.txt, verbose_logs.diff, verbose_logs.txt
>
>
> Hitting a bug with cassandra 4 alpha version. The same bug is repeated with
> difefrent version of Java(8,11 &12) [~benedict]
>
> Stack trace:
> {code:java}
> INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for
> CQL clients on /1.3.0.6:9042 (unencrypted)...
> WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 -
> 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message
> of type PING_REQ whose timeout expired before reaching the network
> WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 -
> 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..)
> failed: Connection refused: /1.3.0.2:7000
> Caused by: java.net.ConnectException: finishConnect(..) failed: Connection
> refused
> at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124)
> at io.netty.channel.unix.Socket.finishConnect(Socket.java:243)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> WARN [Messaging-EventLoop-3-3] 2019-10-11 16:11:32,639 NoSpamLogger.java:94 -
> 1.3.4.6:7000->1.3.4.5:7000-URGENT_MESSAGES-[no-channel] dropping message of
> type GOSSIP_DIGEST_SYN whose timeout expired before reaching the network
> INFO [Messaging-EventLoop-3-18] 2019-10-11 16:11:33,077 NoSpamLogger.java:91
> - 1.3.4.5:7000->1.3.4.4:7000-URGENT_MESSAGES-[no-channel] failed to connect
>
> ERROR [Messaging-EventLoop-3-11] 2019-10-10 01:34:34,407
> InboundMessageHandler.java:657 -
> 1.3.4.5:7000->1.3.4.8:7000-LARGE_MESSAGES-0b7d09cd unexpected exception
> caught while processing inbound messages; terminating connection
> java.lang.IllegalArgumentException: initialBuffer is not a direct buffer.
> at io.netty.buffer.UnpooledDirectByteBuf.(UnpooledDirectByteBuf.java:87)
> at
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:59)
> at
> org.apache.cassandra.net.BufferPoolAllocator$Wrapped.(BufferPoolAllocator.java:95)
> at
> org.apache.cassandra.net.BufferPoolAllocator.newDirectBuffer(BufferPoolAllocator.java:56)
> at
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
> at
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
> at
> io.netty.channel.unix.PreferredDirectByteBufAllocator.ioBuffer(PreferredDirectByteBufAllocator.java:53)
> at
> io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:114)
> at
> io.netty.channel.epoll.EpollRecvByteAllocatorHandle.allocate(EpollRecvByteAllocatorHandle.java:75)
> at
>
[jira] [Updated] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[
https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gaurav Ashok updated CASSANDRA-15358:
-
Attachment: repair_1_trace.txt
> Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
>
>
> Key: CASSANDRA-15358
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15358
> Project: Cassandra
> Issue Type: Bug
> Components: Test/benchmark
>Reporter: Santhosh Kumar Ramalingam
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Labels: 4.0, alpha
> Attachments: all_errors.txt, repair_1_trace.txt, verbose_logs.diff,
> verbose_logs.txt
>
>
> Hitting a bug with cassandra 4 alpha version. The same bug is repeated with
> difefrent version of Java(8,11 &12) [~benedict]
>
> Stack trace:
> {code:java}
> INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for
> CQL clients on /1.3.0.6:9042 (unencrypted)...
> WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 -
> 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message
> of type PING_REQ whose timeout expired before reaching the network
> WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343
> - CassandraRoleManager skipped default role setup: some nodes were not ready
> INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369
> - Setup task failed with error, rescheduling
> INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 -
> 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect
> io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..)
> failed: Connection refused: /1.3.0.2:7000
> Caused by: java.net.ConnectException: finishConnect(..) failed: Connection
> refused
> at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124)
> at io.netty.channel.unix.Socket.finishConnect(Socket.java:243)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644)
> at
> io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524)
> at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> WARN [Messaging-EventLoop-3-3] 2019-10-11 16:11:32,639 NoSpamLogger.java:94 -
> 1.3.4.6:7000->1.3.4.5:7000-URGENT_MESSAGES-[no-channel] dropping message of
> type GOSSIP_DIGEST_SYN whose timeout expired before reaching the network
> INFO [Messaging-EventLoop-3-18] 2019-10-11 16:11:33,077 NoSpamLogger.java:91
> - 1.3.4.5:7000->1.3.4.4:7000-URGENT_MESSAGES-[no-channel] failed to connect
>
> ERROR [Messaging-EventLoop-3-11] 2019-10-10 01:34:34,407
> InboundMessageHandler.java:657 -
> 1.3.4.5:7000->1.3.4.8:7000-LARGE_MESSAGES-0b7d09cd unexpected exception
> caught while processing inbound messages; terminating connection
> java.lang.IllegalArgumentException: initialBuffer is not a direct buffer.
> at io.netty.buffer.UnpooledDirectByteBuf.(UnpooledDirectByteBuf.java:87)
> at
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:59)
> at
> org.apache.cassandra.net.BufferPoolAllocator$Wrapped.(BufferPoolAllocator.java:95)
> at
> org.apache.cassandra.net.BufferPoolAllocator.newDirectBuffer(BufferPoolAllocator.java:56)
> at
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
> at
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
> at
> io.netty.channel.unix.PreferredDirectByteBufAllocator.ioBuffer(PreferredDirectByteBufAllocator.java:53)
> at
> io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:114)
> at
> io.netty.channel.epoll.EpollRecvByteAllocatorHandle.allocate(EpollRecvByteAllocatorHandle.java:75)
> at
>
[jira] [Created] (CASSANDRA-15518) build artifact download failed due to http
Dev Leishangthem created CASSANDRA-15518: Summary: build artifact download failed due to http Key: CASSANDRA-15518 URL: https://issues.apache.org/jira/browse/CASSANDRA-15518 Project: Cassandra Issue Type: Bug Components: Build Reporter: Dev Leishangthem Please feel free to close if not apt. The build for cassandra 3.11.4 is failing due to `http` usage in build,properties.default and build.xml. Maven2 Repository Locations (you can override these in "build.properties" to point to a local proxy, e.g. Nexus) artifact.remoteRepository.central:http://repo1.maven.org/maven2 artifact.remoteRepository.apache: http://repo.maven.apache.org/maven2 Changing it to `https` resolves the issue. Also seen that 3.11.5 tag has https. Can this be backported to 3.11.4 ? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021665#comment-17021665 ] David Capwell commented on CASSANDRA-15505: --- Sorry for the delay. At a high level LGTM. The biggest thing I kept hitting while reading the code is "match" means "drop" (where as it used to be "filter" means "drop") and we always have to negate that... this causes me to double take at every call site. Ill try to look closer again tomorrow. > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-15505: --- Labels: pull-request-available (was: ) > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15511) Utilising BTree Improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-15511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019381#comment-17019381
]
Benedict Elliott Smith edited comment on CASSANDRA-15511 at 1/22/20 10:50 PM:
--
I have uploaded the results of several days of performance comparisons, and for
posterity the scripts I used to generate and parse them into a spreadsheet, on
the effect of this change on inserting into a partition.
The results are pretty significant; some highlights:
Collection types are universally and dramatically improved, both contended and
uncontended:
* >99% memory consumption reduction
* ~25x throughput
Non-collection types, uncontended:
* ~25% improvement in throughput
* 50% reduction in memory allocated
Non-collections types, contended:
* vs 3.0 with locking enabled _instantly_ (i.e. upgrade to locking after first
competition)
** With 4 competing threads, across the board lower memory utilisation;
near-equivalent with 16-threads
** With 4 or 16 competing threads, ~200% throughput improvement
* vs 3.0 with default locking
** 100-200% throughput improvement
** 30-60% reduction in memory consumption
Notably, this may justify removing the partition lock entirely. This patch,
but with the lock retained, does see even fewer allocations, but the typical
effect is relatively small, and there is a similarly slight drop in throughput.
I recommend taking a look at the attached (zipped) Excel spreadsheet, which has
some nice visual formatting to see the full breakdown of results. I've also
attached as {{ods}} format for compatibility, but this loses the formatting and
inline graphical representation of improvement/degradation.
was (Author: benedict):
I have uploaded the results of several days of performance comparisons, and for
posterity the scripts I used to generate and parse them into a spreadsheet, on
the effect of this change on inserting into a partition.
The results are pretty significant; some highlights:
Collection types are universally and dramatically improved, both contended and
uncontended:
* >99% memory consumption reduction
* ~25x throughput
Non-collection types, uncontended:
* ~25% improvement in throughput
* 50% reduction in memory allocated
Non-collections types, contended:
* vs 3.0 with locking enabled _instantly_ (i.e. upgrade to locking after first
competition)
** With 4 competing threads, across the board lower memory utilisation;
near-equivalent with 16-threads
** With 4 or 16 competing threads, ~200% throughput improvement
* vs 3.0 with default locking
** 100-200% throughput improvement
** 30-60% reduction in memory consumption
Notably, this may justify removing the partition lock entirely. This patch,
but with the lock retained, does see even fewer allocations, but the typical
effect is relatively small, and there is a similarly slight drop in throughput.
I recommend taking a look at the attached (zipped) Excel spreadsheet, which has
some nice visual formatting to see the full breakdown of results. I've also
attached as {{ods}} format for compatibility, but this loses the formatting.
> Utilising BTree Improvements
>
>
> Key: CASSANDRA-15511
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15511
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Other
>Reporter: Benedict Elliott Smith
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Fix For: 4.x
>
> Attachments: atomicbtreepartition.ods, atomicbtreepartition.xlsx.zip,
> perfsh.tar.gz
>
>
> This patch utilises CASSANDRA-15510 to improve throughput and reduce garbage
> produced by a number of common operations, by employing
> {{transformAndFilter}}, {{transform}} and {{FastBuilder}}
> * {{Row}}, {{Cell}} and {{ComplexColumnData}} cloning are implemented with
> {{BTree.transform}}, so no special builders are necessary;
> ** {{Rows.copy}} removed
> * {{Rows.merge}} implemented using {{BTree.update}} and a {{ColumnData}}
> reconciler
> ** Zero-allocations if result of merge is same as {{existing}}
> ** Fewer comparisons
> * {{ColumnData}} reconciler implemented in same manner
> ** {{Cells.reconcileComplex}} is retired
> ** {{ComplexColumnData}} reconciliation now
> *** Garbage-free if the merge has no effect
> *** Always fewer allocations
> *** Fewer comparisons
> * {{FastBuilder}} employed widely:
> ** {{ClusteringIndexNamesFilter}} deserialization
> ** {{Columns}} deserialization
> ** {{PartitionUpdate}} deserialization
> ** {{AbstractBTreePartition}} construction
> ** Misc others
> The upshot of this work when combined with the proposed patch for
> CASSANDRA-15367 has a dramatic impact on operations over collection types -
> under contention, as much as 100x improved throughput, and hundreds of
>
[jira] [Comment Edited] (CASSANDRA-15392) Pool Merge Iterators
[ https://issues.apache.org/jira/browse/CASSANDRA-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963496#comment-16963496 ] Blake Eggleston edited comment on CASSANDRA-15392 at 1/22/20 10:46 PM: --- [4.0|https://github.com/bdeggleston/cassandra/tree/15392-trunk] was (Author: bdeggleston): [4.0|https://github.com/bdeggleston/cassandra/tree/15392] > Pool Merge Iterators > > > Key: CASSANDRA-15392 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15392 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0 > > > By pooling merge iterators, instead of creating new ones each time we need > them, we can reduce garbage on the compaction and read paths under relevant > workloads by ~4% in many cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15512) Add compaction-related JMH tests
[ https://issues.apache.org/jira/browse/CASSANDRA-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021599#comment-17021599 ] Benedict Elliott Smith commented on CASSANDRA-15512: I attached some simple awk scripts to run the JMH and parse the output into simple CSV format, but there's a degree of laborious work to produce a good spreadsheet. Probably minimised a great deal if you use that one as a template! > Add compaction-related JMH tests > > > Key: CASSANDRA-15512 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15512 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Following up CASSANDRA-15388, we should introduce some isolated JMH tests > covering at least the compaction merge path and partition deserialization, > covering a wide range of partition shapes and characteristics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15512) Add compaction-related JMH tests
[ https://issues.apache.org/jira/browse/CASSANDRA-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021598#comment-17021598 ] Blake Eggleston commented on CASSANDRA-15512: - interesting, ok. Do you have a script that converts the jmh output into a csv file or something? > Add compaction-related JMH tests > > > Key: CASSANDRA-15512 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15512 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Following up CASSANDRA-15388, we should introduce some isolated JMH tests > covering at least the compaction merge path and partition deserialization, > covering a wide range of partition shapes and characteristics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15512) Add compaction-related JMH tests
[ https://issues.apache.org/jira/browse/CASSANDRA-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021589#comment-17021589 ] Benedict Elliott Smith edited comment on CASSANDRA-15512 at 1/22/20 10:33 PM: -- I alluded in [this comment|https://issues.apache.org/jira/browse/CASSANDRA-15388?focusedCommentId=17015376=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17015376] that we probably need to prune the state space a little (you don't need to prune many options to make it more tractable). [This spreadsheet|https://issues.apache.org/jira/secure/attachment/12991371/atomicbtreepartition.xlsx.zip] from CASSANDRA-15511 is how I would approach looking at the data once I have a chance to run some comparisons (though this isn't a complete view). To some extent, simply looking at what patterns occur (that jump out) is informative, as is isolating specific variables and looking at aggregate statistics for them. Obviously, what can be usefully reported to Jira is even more summarised, unfortunately, but it's informative particularly for regressions versus baseline. was (Author: benedict): I alluded in [this comment|https://issues.apache.org/jira/browse/CASSANDRA-15388?focusedCommentId=17015376=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17015376] that we probably need to prune the state space a little (you don't need to prune many options to make it more tractable). [This spreadsheet|https://issues.apache.org/jira/secure/attachment/12991371/atomicbtreepartition.xlsx.zip] from CASSANDRA-15511 is how I would approach looking at the data once I have a chance to run some comparisons (though this isn't a complete view). To some extent, simply looking at what patterns occur (that jump out) is informative, as is looking at isolating specific variables and looking at aggregate statistics for them. Obviously, what can be usefully reported to Jira is even more summarised, unfortunately, but it's informative particularly for regressions versus baseline. > Add compaction-related JMH tests > > > Key: CASSANDRA-15512 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15512 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Following up CASSANDRA-15388, we should introduce some isolated JMH tests > covering at least the compaction merge path and partition deserialization, > covering a wide range of partition shapes and characteristics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15512) Add compaction-related JMH tests
[ https://issues.apache.org/jira/browse/CASSANDRA-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021589#comment-17021589 ] Benedict Elliott Smith commented on CASSANDRA-15512: I alluded in [this comment|https://issues.apache.org/jira/browse/CASSANDRA-15388?focusedCommentId=17015376=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17015376] that we probably need to prune the state space a little (you don't need to prune many options to make it more tractable). [This spreadsheet|https://issues.apache.org/jira/secure/attachment/12991371/atomicbtreepartition.xlsx.zip] from CASSANDRA-15511 is how I would approach looking at the data once I have a chance to run some comparisons (though this isn't a complete view). To some extent, simply looking at what patterns occur (that jump out) is informative, as is looking at isolating specific variables and looking at aggregate statistics for them. Obviously, what can be usefully reported to Jira is even more summarised, unfortunately, but it's informative particularly for regressions versus baseline. > Add compaction-related JMH tests > > > Key: CASSANDRA-15512 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15512 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Following up CASSANDRA-15388, we should introduce some isolated JMH tests > covering at least the compaction merge path and partition deserialization, > covering a wide range of partition shapes and characteristics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15512) Add compaction-related JMH tests
[ https://issues.apache.org/jira/browse/CASSANDRA-15512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021584#comment-17021584 ] Blake Eggleston commented on CASSANDRA-15512: - How are you interpreting the output of these benchmarks? The CompactionIteratorBench will perform over 7700 workload permutations > Add compaction-related JMH tests > > > Key: CASSANDRA-15512 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15512 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Following up CASSANDRA-15388, we should introduce some isolated JMH tests > covering at least the compaction merge path and partition deserialization, > covering a wide range of partition shapes and characteristics. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15392) Pool Merge Iterators
[ https://issues.apache.org/jira/browse/CASSANDRA-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021552#comment-17021552 ] Benedict Elliott Smith edited comment on CASSANDRA-15392 at 1/22/20 9:55 PM: - edit: nevermind, I hadn't properly digested the statement about nesting was (Author: benedict): bq. In cases where there are that many sstables to merge the node is usually also under significant gc pressure, so it would not be a great time to abandon this optimization. Perhaps the size limit should be larger, and an intrusive linked-list stack is certainly a good data structure to use here. But we probably want to impose _some_ coarse size limits? Though you make a very good point that three is much too few in this case. But otherwise we have the problem that infrequent storms of sstable creation slowly causes threads to accumulate "huge" pools - and we can have a lot of these threads, since read-parallelism is gated by the number of threads. It's far from crazy to have a system with > 200 threads, and if they each managed to hold on to 1k iterators at a time (noting we need iterators both for the partitions, their rows and any complex columns, so this seems readily achievable), we'd be looking at 50-100MiB+ of heap utilised by these pools. > Pool Merge Iterators > > > Key: CASSANDRA-15392 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15392 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0 > > > By pooling merge iterators, instead of creating new ones each time we need > them, we can reduce garbage on the compaction and read paths under relevant > workloads by ~4% in many cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15392) Pool Merge Iterators
[ https://issues.apache.org/jira/browse/CASSANDRA-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021552#comment-17021552 ] Benedict Elliott Smith commented on CASSANDRA-15392: bq. In cases where there are that many sstables to merge the node is usually also under significant gc pressure, so it would not be a great time to abandon this optimization. Perhaps the size limit should be larger, and an intrusive linked-list stack is certainly a good data structure to use here. But we probably want to impose _some_ coarse size limits? Though you make a very good point that three is much too few in this case. But otherwise we have the problem that infrequent storms of sstable creation slowly causes threads to accumulate "huge" pools - and we can have a lot of these threads, since read-parallelism is gated by the number of threads. It's far from crazy to have a system with > 200 threads, and if they each managed to hold on to 1k iterators at a time (noting we need iterators both for the partitions, their rows and any complex columns, so this seems readily achievable), we'd be looking at 50-100MiB+ of heap utilised by these pools. > Pool Merge Iterators > > > Key: CASSANDRA-15392 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15392 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0 > > > By pooling merge iterators, instead of creating new ones each time we need > them, we can reduce garbage on the compaction and read paths under relevant > workloads by ~4% in many cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15339) Make available the known JMX endpoints across the cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021543#comment-17021543
]
Chris Lohfink commented on CASSANDRA-15339:
---
{quote}now have to also talk to Cassandra via CQL. {quote}
Maybe can use messaging service queries to other nodes can collect it on demand
instead of broadcasting it in the gossip state.
> Make available the known JMX endpoints across the cluster
> -
>
> Key: CASSANDRA-15339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15339
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Gossip
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Labels: 4.0-feature-freeze-review-requested
>
> With the addition of multiple nodes running on the same server using
> different ports: CASSANDRA-7544 ; it becomes more difficult for third-party
> tools to easily connect to all nodes based on the jmx connection details to
> just one node.
> By adding jmx host and port to gossip, and saving it in {{system.peers_v2}},
> the list of all jmx endpoints in a cluster can be fetch after just the
> initial successful jmx connection and the
> {{StorageServiceMBean.getJmxEndpoints()}} method.
> And example of the difficulty can be illustrated through the potential
> workaround…
> Such a third-party tool could make a native protocol connection, and via the
> driver obtain the list of all possible `host:port` native protocol
> connections, and make a connection to each of these then requesting the
> configuration virtual table, from which the jmx port can be obtained. This is
> a rather cumbersome approach, and can involve third-party tools having to add
> native connection functionality and dependencies. It's also currently not
> possible because CASSANDRA-14573 does not provide the jmx port (it only
> offers the yaml settings).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13917) COMPACT STORAGE queries on dense static tables accept hidden column1 and value columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021531#comment-17021531
]
Alex Petrov commented on CASSANDRA-13917:
-
Committed a follow-up patch to 3.0 with
[b907dc9689dd04ebae1f765570401d1f20a88ebd|https://github.com/apache/cassandra/commit/b907dc9689dd04ebae1f765570401d1f20a88ebd],
and merged up to
[3.11|https://github.com/apache/cassandra/commit/7cd0e92dcac8e3423f2eae08069bd5ebf6a3e236],
and trunk (with {{-s ours}}).
> COMPACT STORAGE queries on dense static tables accept hidden column1 and
> value columns
> --
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Low
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13917-3.0-testall-13.12.2019,
> 13917-3.0-testall-16.01.2020, 13917-3.0-testall-2.png,
> 13917-3.0-testall-20.11.2019.png, 13917-3.0-upgrade-16.01.2020,
> 13917-3.0.png, 13917-3.11-testall-13.12.2019,
> 13917-3.11-testall-16.01.2020.png, 13917-3.11-testall-2.png,
> 13917-3.11-testall-20.11.2019.png, 13917-3.11-upgrade-16.01.2020.png,
> 13917-3.11.png
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?,
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1,
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)",
> 1, 1, 1, ByteBufferUtil.bytes('a'));
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?,
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should
> be as easy as just adding validations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15470) Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB & Bytes
[
https://issues.apache.org/jira/browse/CASSANDRA-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021522#comment-17021522
]
Mallika Kulkarni commented on CASSANDRA-15470:
--
Thanks [~djoshi]. please go ahead
> Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB
> & Bytes
> -
>
> Key: CASSANDRA-15470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15470
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Jordan West
>Assignee: Mallika Kulkarni
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-rc
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{DatabaseDescriptor}} has several functions that convert between user
> supplied sizes in KB/MB and bytes. These are implemented without much
> consistency and, while unlikely, several have the potential to overflow since
> validation on the input is missing. Meanwhile, some widen the number to a
> long correctly. Options include: widening in all places or simply doing
> better validation on start up — currently only the lower bound of the valid
> range is checked for many of these fields.
> List of Affected {{DatabaseDescriptor}} Methods:
> * {{getColumnIndexSize}}
> * {{getColumnIndexCacheSize}}
> * {{getBatchSizeWarnThreshold}}
> * {{getNativeTransportFrameBlockSize}}
> * {{getRepairSessionSpaceInMegabytes}}
> * {{getNativeTransportMaxFrameSize}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15339) Make available the known JMX endpoints across the cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021477#comment-17021477
]
Brandon Williams commented on CASSANDRA-15339:
--
Just a note on this patch: the padding states in ApplicationState.java are not
intended to be consumed, they are there so that the number of padding states
can be added between versions without blowing up the enum in the previous
version (and doing so would be extremely bad.)
> Make available the known JMX endpoints across the cluster
> -
>
> Key: CASSANDRA-15339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15339
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Gossip
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Labels: 4.0-feature-freeze-review-requested
>
> With the addition of multiple nodes running on the same server using
> different ports: CASSANDRA-7544 ; it becomes more difficult for third-party
> tools to easily connect to all nodes based on the jmx connection details to
> just one node.
> By adding jmx host and port to gossip, and saving it in {{system.peers_v2}},
> the list of all jmx endpoints in a cluster can be fetch after just the
> initial successful jmx connection and the
> {{StorageServiceMBean.getJmxEndpoints()}} method.
> And example of the difficulty can be illustrated through the potential
> workaround…
> Such a third-party tool could make a native protocol connection, and via the
> driver obtain the list of all possible `host:port` native protocol
> connections, and make a connection to each of these then requesting the
> configuration virtual table, from which the jmx port can be obtained. This is
> a rather cumbersome approach, and can involve third-party tools having to add
> native connection functionality and dependencies. It's also currently not
> possible because CASSANDRA-14573 does not provide the jmx port (it only
> offers the yaml settings).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15339) Make available the known JMX endpoints across the cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021465#comment-17021465
]
Michael Semb Wever commented on CASSANDRA-15339:
[~cnlwsu],
bq. Can use the setHost() on the Statement from client side to coordinate to
each host vs trying to distribute the data in gossip etc to fill peers table.
Yes, but then third-party tools that talk to Cassandra via JMX: that want to
discover all the JMX ports; now have to *also* talk to Cassandra via CQL.
That's a fair bit of development to push onto third-party tools.
> Make available the known JMX endpoints across the cluster
> -
>
> Key: CASSANDRA-15339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15339
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Gossip
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Labels: 4.0-feature-freeze-review-requested
>
> With the addition of multiple nodes running on the same server using
> different ports: CASSANDRA-7544 ; it becomes more difficult for third-party
> tools to easily connect to all nodes based on the jmx connection details to
> just one node.
> By adding jmx host and port to gossip, and saving it in {{system.peers_v2}},
> the list of all jmx endpoints in a cluster can be fetch after just the
> initial successful jmx connection and the
> {{StorageServiceMBean.getJmxEndpoints()}} method.
> And example of the difficulty can be illustrated through the potential
> workaround…
> Such a third-party tool could make a native protocol connection, and via the
> driver obtain the list of all possible `host:port` native protocol
> connections, and make a connection to each of these then requesting the
> configuration virtual table, from which the jmx port can be obtained. This is
> a rather cumbersome approach, and can involve third-party tools having to add
> native connection functionality and dependencies. It's also currently not
> possible because CASSANDRA-14573 does not provide the jmx port (it only
> offers the yaml settings).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: ninja-fix: Create Jenkins pipeline definition (CASSANDRA-15496)
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new 7827ad5 ninja-fix: Create Jenkins pipeline definition
(CASSANDRA-15496)
7827ad5 is described below
commit 7827ad5430b64bc261cb88443735762434001949
Author: Mick Semb Wever
AuthorDate: Wed Jan 22 20:41:50 2020 +0100
ninja-fix: Create Jenkins pipeline definition (CASSANDRA-15496)
---
.jenkins/Jenkinsfile | 3 +++
1 file changed, 3 insertions(+)
diff --git a/.jenkins/Jenkinsfile b/.jenkins/Jenkinsfile
index adb2a8d..7d54f11 100644
--- a/.jenkins/Jenkinsfile
+++ b/.jenkins/Jenkinsfile
@@ -20,6 +20,8 @@
// Jenkins infrastructure related settings should be kept in
//
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy
//
+// Validate/lint this file using the following command
+// `curl -X POST -F "jenkinsfile=<.jenkins/Jenkinsfile"
https://builds.apache.org/pipeline-model-converter/validate`
pipeline {
agent any
@@ -259,6 +261,7 @@ pipeline {
slackSend channel: '#cassandra-builds', message:
"${currentBuild.fullDisplayName} completed: ${currentBuild.result}. See
${env.BUILD_URL}"
emailext to: 'bui...@cassandra.apache.org', subject: "Build
complete: ${currentBuild.fullDisplayName} [${currentBuild.result}]
${env.GIT_COMMIT}", body: '${CHANGES} ${JELLY_SCRIPT,template="text"}'
}
+}
}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (56843c6 -> 44a15eb)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 56843c6 Merge branch 'cassandra-3.11' into trunk new 7827ad5 ninja-fix: Create Jenkins pipeline definition (CASSANDRA-15496) new b80e146 Merge branch 'cassandra-2.2' into cassandra-3.0 new 7819eb9 Merge branch 'cassandra-3.0' into cassandra-3.11 new 44a15eb Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (b907dc9 -> b80e146)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b907dc9 Make sure that hidden columns are only hidden from CQL new 7827ad5 ninja-fix: Create Jenkins pipeline definition (CASSANDRA-15496) new b80e146 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .jenkins/Jenkinsfile | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (7cd0e92 -> 7819eb9)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 7cd0e92 Merge branch 'cassandra-3.0' into cassandra-3.11 new 7827ad5 ninja-fix: Create Jenkins pipeline definition (CASSANDRA-15496) new b80e146 Merge branch 'cassandra-2.2' into cassandra-3.0 new 7819eb9 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .jenkins/Jenkinsfile | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b80e1469d6af16d073f0b70223937fa67a45da0f Merge: b907dc9 7827ad5 Author: Mick Semb Wever AuthorDate: Wed Jan 22 20:49:35 2020 +0100 Merge branch 'cassandra-2.2' into cassandra-3.0 .jenkins/Jenkinsfile | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 7819eb98744580f5d6b70da85490a572bca92af4 Merge: 7cd0e92 b80e146 Author: Mick Semb Wever AuthorDate: Wed Jan 22 20:50:35 2020 +0100 Merge branch 'cassandra-3.0' into cassandra-3.11 .jenkins/Jenkinsfile | 3 +++ 1 file changed, 3 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 44a15eb01f05b4c5267ada50c72b196cd2802828 Merge: 56843c6 7819eb9 Author: Mick Semb Wever AuthorDate: Wed Jan 22 20:52:54 2020 +0100 Merge branch 'cassandra-3.11' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15339) Make available the known JMX endpoints across the cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021460#comment-17021460
]
David Capwell commented on CASSANDRA-15339:
---
virtual table would be good to expose the details of this host.
One thing I was thinking; would it make sense to write a file to /tmp (delete
on shutdown) which also defines this detail? That way tools on the host can
discover all active clusters (only works with tools on the same node)?
> Make available the known JMX endpoints across the cluster
> -
>
> Key: CASSANDRA-15339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15339
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Gossip
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Labels: 4.0-feature-freeze-review-requested
>
> With the addition of multiple nodes running on the same server using
> different ports: CASSANDRA-7544 ; it becomes more difficult for third-party
> tools to easily connect to all nodes based on the jmx connection details to
> just one node.
> By adding jmx host and port to gossip, and saving it in {{system.peers_v2}},
> the list of all jmx endpoints in a cluster can be fetch after just the
> initial successful jmx connection and the
> {{StorageServiceMBean.getJmxEndpoints()}} method.
> And example of the difficulty can be illustrated through the potential
> workaround…
> Such a third-party tool could make a native protocol connection, and via the
> driver obtain the list of all possible `host:port` native protocol
> connections, and make a connection to each of these then requesting the
> configuration virtual table, from which the jmx port can be obtained. This is
> a rather cumbersome approach, and can involve third-party tools having to add
> native connection functionality and dependencies. It's also currently not
> possible because CASSANDRA-14573 does not provide the jmx port (it only
> offers the yaml settings).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13917) COMPACT STORAGE queries on dense static tables accept hidden column1 and value columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021457#comment-17021457
]
Aleksandr Sorokoumov commented on CASSANDRA-13917:
--
[~ifesdjeen] I reviewed your patch. It covers some cases that I missed and has
better abstraction via {{getColumnDefinitionForCQL}}. +1 to merge your version!
> COMPACT STORAGE queries on dense static tables accept hidden column1 and
> value columns
> --
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Low
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13917-3.0-testall-13.12.2019,
> 13917-3.0-testall-16.01.2020, 13917-3.0-testall-2.png,
> 13917-3.0-testall-20.11.2019.png, 13917-3.0-upgrade-16.01.2020,
> 13917-3.0.png, 13917-3.11-testall-13.12.2019,
> 13917-3.11-testall-16.01.2020.png, 13917-3.11-testall-2.png,
> 13917-3.11-testall-20.11.2019.png, 13917-3.11-upgrade-16.01.2020.png,
> 13917-3.11.png
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?,
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1,
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)",
> 1, 1, 1, ByteBufferUtil.bytes('a'));
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?,
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should
> be as easy as just adding validations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15339) Make available the known JMX endpoints across the cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021454#comment-17021454
]
Chris Lohfink commented on CASSANDRA-15339:
---
Could be a virtual table? or exposed in the settings table. Can use the
setHost() on the Statement from client side to coordinate to each host vs
trying to distribute the data in gossip etc to fill peers table.
> Make available the known JMX endpoints across the cluster
> -
>
> Key: CASSANDRA-15339
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15339
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Gossip
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Labels: 4.0-feature-freeze-review-requested
>
> With the addition of multiple nodes running on the same server using
> different ports: CASSANDRA-7544 ; it becomes more difficult for third-party
> tools to easily connect to all nodes based on the jmx connection details to
> just one node.
> By adding jmx host and port to gossip, and saving it in {{system.peers_v2}},
> the list of all jmx endpoints in a cluster can be fetch after just the
> initial successful jmx connection and the
> {{StorageServiceMBean.getJmxEndpoints()}} method.
> And example of the difficulty can be illustrated through the potential
> workaround…
> Such a third-party tool could make a native protocol connection, and via the
> driver obtain the list of all possible `host:port` native protocol
> connections, and make a connection to each of these then requesting the
> configuration virtual table, from which the jmx port can be obtained. This is
> a rather cumbersome approach, and can involve third-party tools having to add
> native connection functionality and dependencies. It's also currently not
> possible because CASSANDRA-14573 does not provide the jmx port (it only
> offers the yaml settings).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15392) Pool Merge Iterators
[
https://issues.apache.org/jira/browse/CASSANDRA-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021455#comment-17021455
]
Blake Eggleston commented on CASSANDRA-15392:
-
Thanks for the feedback. I just pushed up some commits addressing most points.
Getting rid of linked list queues is a good idea, but don’t think
TinyThreadLocalPool will be sufficient in some cases. If {{MergeIterator#get}}
is called with more than than {{DEFAULT_SIZE}} iterators, we’ll nest pooled
iterators. In cases where there _are_ that many sstables to merge the node is
usually also under significant gc pressure, so it would not be a great time to
abandon this optimization. An intrusive linked stack is the best I could come
up with. It's unbounded, doesn't do any allocations, and has very simple logic,
which was important for preventing oneToOne performance from regressing too
much.
@Jordan, short circuiting the merge iterator in that case is a really good
idea. However, FQL will need a bit of (probably straightforward) rework before
{{MergeIterator#get}} can return a closeable iterator, since it relies on
having separate input and output types. As this relates to SASI, this wouldn’t
be a regression right? I'd guess we're already wasting TrivialOneToOne
iterators?
I’ve also put together a very simple jmh benchmark. After a few iterations of
additional optimization, the manyToOne path saw a 15% increase in throughput in
most cases, and the oneToOne path saw a 2-6% reduction in most cases. Numbers
below:
{code:java}
Baseline:
BenchmarkMode Cnt
ScoreError Units
MergeIteratorGetBench.manyToOnesample 59417991
418.721 ± 22.706 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.00sample
101.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.50sample
148.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.90sample
171.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.95sample
176.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.99sample
188.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.999 sample
1456.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0. sample
9171.213 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p1.00sample
38862848.000 ns/op
MergeIteratorGetBench.one sample 53514991
83.452 ± 4.005 ns/op
MergeIteratorGetBench.one:one·p0.00sample
47.000 ns/op
MergeIteratorGetBench.one:one·p0.50sample
73.000 ns/op
MergeIteratorGetBench.one:one·p0.90sample
90.000 ns/op
MergeIteratorGetBench.one:one·p0.95sample
95.000 ns/op
MergeIteratorGetBench.one:one·p0.99sample
103.000 ns/op
MergeIteratorGetBench.one:one·p0.999 sample
119.000 ns/op
MergeIteratorGetBench.one:one·p0. sample
3830.003 ns/op
MergeIteratorGetBench.one:one·p1.00sample
21987328.000 ns/op
Pooled:
BenchmarkModeCnt
Score Error Units
MergeIteratorGetBench.manyToOnesample 100352304
133.534 ± 1.662 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.00sample
86.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.50sample
128.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.90sample
143.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.95sample
148.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.99sample
157.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0.999 sample
172.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p0. sample
3744.000 ns/op
MergeIteratorGetBench.manyToOne:manyToOne·p1.00sample
13008896.000 ns/op
MergeIteratorGetBench.one sample 97007239
87.340 ± 3.063 ns/op
MergeIteratorGetBench.one:one·p0.00sample
51.000 ns/op
MergeIteratorGetBench.one:one·p0.50sample
78.000 ns/op
MergeIteratorGetBench.one:one·p0.90sample
92.000 ns/op
MergeIteratorGetBench.one:one·p0.95sample
97.000
[jira] [Updated] (CASSANDRA-15517) Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
[
https://issues.apache.org/jira/browse/CASSANDRA-15517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15517:
--
Status: Ready to Commit (was: Review In Progress)
> Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
> ---
>
> Key: CASSANDRA-15517
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15517
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
>
> The assert relies on the order of files in {{File.listFiles}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15517) Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
[
https://issues.apache.org/jira/browse/CASSANDRA-15517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15517:
--
Reviewers: David Capwell, David Capwell (was: David Capwell)
David Capwell, David Capwell (was: David Capwell)
Status: Review In Progress (was: Patch Available)
LGTM +1.
> Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
> ---
>
> Key: CASSANDRA-15517
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15517
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
>
> The assert relies on the order of files in {{File.listFiles}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch master updated: Reduce number of build logs to keep on Jenkins from 50 builds to 25. Disks were running out of space.
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new 2e583ef Reduce number of build logs to keep on Jenkins from 50 builds
to 25. Disks were running out of space.
2e583ef is described below
commit 2e583ef6e7f52123ac39120d9055db493e447e3d
Author: mck
AuthorDate: Wed Jan 22 20:29:30 2020 +0100
Reduce number of build logs to keep on Jenkins from 50 builds to 25.
Disks were running out of space.
---
jenkins-dsl/cassandra_job_dsl_seed.groovy | 18 +-
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/jenkins-dsl/cassandra_job_dsl_seed.groovy
b/jenkins-dsl/cassandra_job_dsl_seed.groovy
index c3b4d86..cd8db6f 100644
--- a/jenkins-dsl/cassandra_job_dsl_seed.groovy
+++ b/jenkins-dsl/cassandra_job_dsl_seed.groovy
@@ -73,7 +73,7 @@ job('Cassandra-template-artifacts') {
jdk(jdkLabel)
label(slaveLabel)
logRotator {
-numToKeep(50)
+numToKeep(25)
artifactNumToKeep(1)
}
wrappers {
@@ -138,7 +138,7 @@ job('Cassandra-template-test') {
jdk(jdkLabel)
label(slaveLabel)
logRotator {
-numToKeep(50)
+numToKeep(25)
artifactNumToKeep(1)
}
wrappers {
@@ -193,7 +193,7 @@ job('Cassandra-template-dtest') {
jdk(jdkLabel)
label(slaveLabel)
logRotator {
-numToKeep(50)
+numToKeep(25)
artifactNumToKeep(1)
}
wrappers {
@@ -243,7 +243,7 @@ matrixJob('Cassandra-template-cqlsh-tests') {
disabled(true)
description(jobDescription)
logRotator {
-numToKeep(50)
+numToKeep(25)
artifactNumToKeep(1)
}
wrappers {
@@ -421,7 +421,7 @@ cassandraBranches.each {
pipelineJob("${jobNamePrefix}") {
description(jobDescription)
logRotator {
-numToKeep(50)
+numToKeep(25)
artifactNumToKeep(1)
}
definition {
@@ -464,7 +464,7 @@ testTargets.each {
jdk(jdkLabel)
label(slaveLabel)
logRotator {
-numToKeep(50)
+numToKeep(25)
artifactNumToKeep(1)
}
wrappers {
@@ -522,7 +522,7 @@ job('Cassandra-devbranch-dtest') {
jdk(jdkLabel)
label(slaveLabel)
logRotator {
-numToKeep(50)
+numToKeep(25)
artifactNumToKeep(1)
}
wrappers {
@@ -581,7 +581,7 @@ matrixJob('Cassandra-devbranch-cqlsh-tests') {
description(jobDescription)
concurrentBuild()
logRotator {
-numToKeep(50)
+numToKeep(25)
artifactNumToKeep(1)
}
wrappers {
@@ -646,7 +646,7 @@ matrixJob('Cassandra-devbranch-cqlsh-tests') {
pipelineJob('Cassandra-devbranch') {
description(jobDescription)
logRotator {
-numToKeep(50)
+numToKeep(25)
artifactNumToKeep(1)
}
parameters {
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 56843c6a0c5aca33b26a0347254a72e24530aff9 Merge: 0e3a906 7cd0e92 Author: Alex Petrov AuthorDate: Wed Jan 22 19:56:46 2020 +0100 Merge branch 'cassandra-3.11' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 7cd0e92dcac8e3423f2eae08069bd5ebf6a3e236
Merge: 6a41aa5 b907dc9
Author: Alex Petrov
AuthorDate: Wed Jan 22 19:53:09 2020 +0100
Merge branch 'cassandra-3.0' into cassandra-3.11
.../org/apache/cassandra/config/CFMetaData.java| 26 +
.../apache/cassandra/config/ColumnDefinition.java | 2 +-
.../cql3/statements/CreateIndexStatement.java | 2 +-
.../cassandra/db/AbstractReadCommandBuilder.java | 5 +-
test/unit/org/apache/cassandra/cql3/ViewTest.java | 63 ++
.../cql3/validation/operations/AlterTest.java | 3 ++
.../cql3/validation/operations/InsertTest.java | 12 ++---
.../cql3/validation/operations/SelectTest.java | 4 ++
.../org/apache/cassandra/db/RowUpdateBuilder.java | 16 +++---
.../index/internal/CassandraIndexTest.java | 9
10 files changed, 90 insertions(+), 52 deletions(-)
diff --cc src/java/org/apache/cassandra/config/ColumnDefinition.java
index c819ae7,6f7f749..4de96a0
--- a/src/java/org/apache/cassandra/config/ColumnDefinition.java
+++ b/src/java/org/apache/cassandra/config/ColumnDefinition.java
@@@ -460,193 -410,4 +460,193 @@@ public class ColumnDefinition extends C
return ((CollectionType) type).valueComparator().isCounter();
return type.isCounter();
}
+
+public Selector.Factory newSelectorFactory(CFMetaData cfm,
AbstractType expectedType, List defs,
VariableSpecifications boundNames) throws InvalidRequestException
+{
+return SimpleSelector.newFactory(this, addAndGetIndex(this, defs));
+}
+
+public AbstractType getExactTypeIfKnown(String keyspace)
+{
+return type;
+}
+
+/**
+ * Because Thrift-created tables may have a non-text comparator, we
cannot determine the proper 'key' until
+ * we know the comparator. ColumnDefinition.Raw is a placeholder that can
be converted to a real ColumnIdentifier
+ * once the comparator is known with prepare(). This should only be used
with identifiers that are actual
+ * column names. See CASSANDRA-8178 for more background.
+ */
+public static abstract class Raw extends Selectable.Raw
+{
+/**
+ * Creates a {@code ColumnDefinition.Raw} from an unquoted identifier
string.
+ */
+public static Raw forUnquoted(String text)
+{
+return new Literal(text, false);
+}
+
+/**
+ * Creates a {@code ColumnDefinition.Raw} from a quoted identifier
string.
+ */
+public static Raw forQuoted(String text)
+{
+return new Literal(text, true);
+}
+
+/**
+ * Creates a {@code ColumnDefinition.Raw} from a pre-existing {@code
ColumnDefinition}
+ * (useful in the rare cases where we already have the column but need
+ * a {@code ColumnDefinition.Raw} for typing purposes).
+ */
+public static Raw forColumn(ColumnDefinition column)
+{
+return new ForColumn(column);
+}
+
+/**
+ * Get the identifier corresponding to this raw column, without
assuming this is an
+ * existing column (unlike {@link #prepare}).
+ */
+public abstract ColumnIdentifier getIdentifier(CFMetaData cfm);
+
+public abstract String rawText();
+
+@Override
+public abstract ColumnDefinition prepare(CFMetaData cfm);
+
+@Override
+public boolean processesSelection()
+{
+return false;
+}
+
+@Override
+public final int hashCode()
+{
+return toString().hashCode();
+}
+
+@Override
+public final boolean equals(Object o)
+{
+if(!(o instanceof Raw))
+return false;
+
+Raw that = (Raw)o;
+return this.toString().equals(that.toString());
+}
+
+private static class Literal extends Raw
+{
+private final String text;
+
+public Literal(String rawText, boolean keepCase)
+{
+this.text = keepCase ? rawText :
rawText.toLowerCase(Locale.US);
+}
+
+public ColumnIdentifier getIdentifier(CFMetaData cfm)
+{
+if (!cfm.isStaticCompactTable())
+return ColumnIdentifier.getInterned(text, true);
+
+AbstractType thriftColumnNameType =
cfm.thriftColumnNameType();
+if (thriftColumnNameType instanceof UTF8Type)
+return ColumnIdentifier.getInterned(text, true);
+
+// We have a Thrift-created table with a non-text comparator.
Check if we have a match column,
[cassandra] branch cassandra-3.0 updated: Make sure that hidden columns are only hidden from CQL
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new b907dc9 Make sure that hidden columns are only hidden from CQL
b907dc9 is described below
commit b907dc9689dd04ebae1f765570401d1f20a88ebd
Author: Alex Petrov
AuthorDate: Thu Jan 16 15:22:57 2020 +0100
Make sure that hidden columns are only hidden from CQL
Patch by Alex Petrov; reviewed by Aleksandr Sorokoumov for CASSANDRA-13917
(follow-up).
---
.../org/apache/cassandra/config/CFMetaData.java| 26 +-
.../apache/cassandra/cql3/ColumnIdentifier.java| 2 +-
src/java/org/apache/cassandra/cql3/Relation.java | 2 +-
.../cassandra/cql3/selection/Selectable.java | 2 +-
.../cql3/statements/CreateIndexStatement.java | 2 +-
.../cql3/statements/ModificationStatement.java | 6 ++---
.../cassandra/cql3/statements/SelectStatement.java | 7 +++---
.../cassandra/db/AbstractReadCommandBuilder.java | 3 +++
.../org/apache/cassandra/db/RowUpdateBuilder.java | 2 +-
test/unit/org/apache/cassandra/cql3/ViewTest.java | 19
.../cql3/validation/operations/AlterTest.java | 3 +++
.../cql3/validation/operations/InsertTest.java | 12 +-
.../cql3/validation/operations/SelectTest.java | 4
.../index/internal/CassandraIndexTest.java | 9
14 files changed, 70 insertions(+), 29 deletions(-)
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 5888f42..68e06be 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -782,11 +782,6 @@ public final class CFMetaData
return compactValueColumn;
}
-private boolean isHiddenColumn(ColumnDefinition def)
-{
-return hiddenColumns.contains(def);
-}
-
public ClusteringComparator getKeyValidatorAsClusteringComparator()
{
boolean isCompound = keyValidator instanceof CompositeType;
@@ -979,7 +974,7 @@ public final class CFMetaData
*/
public ColumnDefinition getColumnDefinition(ColumnIdentifier name)
{
- return getColumnDefinition(name.bytes);
+return getColumnDefinition(name.bytes);
}
// In general it is preferable to work with ColumnIdentifier to make it
@@ -988,10 +983,19 @@ public final class CFMetaData
// for instance) so...
public ColumnDefinition getColumnDefinition(ByteBuffer name)
{
-ColumnDefinition cd = columnMetadata.get(name);
-if (cd == null || isHiddenColumn(cd))
-return null;
-return cd;
+return columnMetadata.get(name);
+}
+
+// Returns only columns that are supposed to be visible through CQL layer
+public ColumnDefinition getColumnDefinitionForCQL(ColumnIdentifier name)
+{
+return getColumnDefinitionForCQL(name.bytes);
+}
+
+public ColumnDefinition getColumnDefinitionForCQL(ByteBuffer name)
+{
+ColumnDefinition cd = getColumnDefinition(name);
+return hiddenColumns.contains(cd) ? null : cd;
}
public static boolean isNameValid(String name)
@@ -1130,7 +1134,7 @@ public final class CFMetaData
public void renameColumn(ColumnIdentifier from, ColumnIdentifier to)
throws InvalidRequestException
{
-ColumnDefinition def = getColumnDefinition(from);
+ColumnDefinition def = getColumnDefinitionForCQL(from);
if (def == null)
throw new InvalidRequestException(String.format("Cannot rename
unknown column %s in keyspace %s", from, cfName));
diff --git a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
index 71e7b9a..5d4e992 100644
--- a/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
+++ b/src/java/org/apache/cassandra/cql3/ColumnIdentifier.java
@@ -222,7 +222,7 @@ public class ColumnIdentifier extends Selectable implements
IMeasurableMemory, C
public Selector.Factory newSelectorFactory(CFMetaData cfm,
List defs) throws InvalidRequestException
{
-ColumnDefinition def = cfm.getColumnDefinition(this);
+ColumnDefinition def = cfm.getColumnDefinitionForCQL(this);
if (def == null)
throw new InvalidRequestException(String.format("Undefined name %s
in selection clause", this));
diff --git a/src/java/org/apache/cassandra/cql3/Relation.java
b/src/java/org/apache/cassandra/cql3/Relation.java
index a88932e..005d984 100644
--- a/src/java/org/apache/cassandra/cql3/Relation.java
+++ b/src/java/org/apache/cassandra/cql3/Relation.java
@@ -263,7 +263,7 @@ public abstract class Relation {
ColumnIdentifier.Raw
entity)
[cassandra] branch trunk updated (0e3a906 -> 56843c6)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 0e3a906 Make it easier to add trace headers to messages new b907dc9 Make sure that hidden columns are only hidden from CQL new 7cd0e92 Merge branch 'cassandra-3.0' into cassandra-3.11 new 56843c6 Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (6a41aa5 -> 7cd0e92)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 6a41aa5 Merge branch 'cassandra-3.0' into cassandra-3.11 new b907dc9 Make sure that hidden columns are only hidden from CQL new 7cd0e92 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/config/CFMetaData.java| 26 + .../apache/cassandra/config/ColumnDefinition.java | 2 +- .../cql3/statements/CreateIndexStatement.java | 2 +- .../cassandra/db/AbstractReadCommandBuilder.java | 5 +- test/unit/org/apache/cassandra/cql3/ViewTest.java | 63 ++ .../cql3/validation/operations/AlterTest.java | 3 ++ .../cql3/validation/operations/InsertTest.java | 12 ++--- .../cql3/validation/operations/SelectTest.java | 4 ++ .../org/apache/cassandra/db/RowUpdateBuilder.java | 16 +++--- .../index/internal/CassandraIndexTest.java | 9 10 files changed, 90 insertions(+), 52 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-15508:
Status: Ready to Commit (was: Review In Progress)
+1, I'll commit tomorrow
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-15508:
Reviewers: Marcus Eriksson, Marcus Eriksson (was: Marcus Eriksson)
Marcus Eriksson, Marcus Eriksson (was: Marcus Eriksson)
Status: Review In Progress (was: Patch Available)
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-15508:
Reviewers: Alex Petrov, Marcus Eriksson (was: Marcus Eriksson)
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021377#comment-17021377
]
Alex Petrov commented on CASSANDRA-15508:
-
+1
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021353#comment-17021353
]
Aleksey Yeschenko commented on CASSANDRA-15499:
---
Committed as
[0e3a90698a94772e57df39e7461efe6b7e09d678|https://github.com/apache/cassandra/commit/0e3a90698a94772e57df39e7461efe6b7e09d678],
cheers.
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15499:
--
Fix Version/s: (was: 4.x)
4.0-alpha
Since Version: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/0e3a90698a94772e57df39e7461efe6b7e09d678
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Make it easier to add trace headers to messages
This is an automated email from the ASF dual-hosted git repository.
aleksey pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 0e3a906 Make it easier to add trace headers to messages
0e3a906 is described below
commit 0e3a90698a94772e57df39e7461efe6b7e09d678
Author: yifan-c
AuthorDate: Sun Jan 12 13:55:54 2020 -0800
Make it easier to add trace headers to messages
patch by Yifan Cai; reviewed by David Capwell for CASSANDRA-15499
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/net/Message.java | 12 +
.../unit/org/apache/cassandra/net/MessageTest.java | 31 ++
3 files changed, 44 insertions(+)
diff --git a/CHANGES.txt b/CHANGES.txt
index 1860e63..85ddd53 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha3
+ * Make it easier to add trace headers to messages (CASSANDRA-15499)
* Fix and optimise partial compressed sstable streaming (CASSANDRA-13938)
* Improve error when JVM 11 can't access required modules (CASSANDRA-15468)
* Better handling of file deletion failures by DiskFailurePolicy
(CASSANDRA-15143)
diff --git a/src/java/org/apache/cassandra/net/Message.java
b/src/java/org/apache/cassandra/net/Message.java
index 05c1bfa..0eb7710 100644
--- a/src/java/org/apache/cassandra/net/Message.java
+++ b/src/java/org/apache/cassandra/net/Message.java
@@ -461,6 +461,18 @@ public class Message
return this;
}
+/**
+ * A shortcut to add tracing params.
+ * Effectively, it is the same as calling {@link #withParam(ParamType,
Object)} with tracing params
+ * If there is already tracing params, calling this method overrides
any existing ones.
+ */
+public Builder withTracingParams()
+{
+if (Tracing.isTracing())
+Tracing.instance.addTraceHeaders(params);
+return this;
+}
+
public Builder withoutParam(ParamType type)
{
params.remove(type);
diff --git a/test/unit/org/apache/cassandra/net/MessageTest.java
b/test/unit/org/apache/cassandra/net/MessageTest.java
index 78eb4c0..6a9d23f 100644
--- a/test/unit/org/apache/cassandra/net/MessageTest.java
+++ b/test/unit/org/apache/cassandra/net/MessageTest.java
@@ -21,6 +21,7 @@ import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
+import java.util.stream.Stream;
import org.junit.AfterClass;
import org.junit.BeforeClass;
@@ -34,8 +35,10 @@ import org.apache.cassandra.io.util.DataInputPlus;
import org.apache.cassandra.io.util.DataOutputBuffer;
import org.apache.cassandra.io.util.DataOutputPlus;
import org.apache.cassandra.locator.InetAddressAndPort;
+import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.tracing.Tracing.TraceType;
import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.utils.UUIDGen;
import static org.apache.cassandra.net.Message.serializer;
import static org.apache.cassandra.net.MessagingService.VERSION_3014;
@@ -195,6 +198,34 @@ public class MessageTest
testCycle(msg);
}
+@Test
+public void testBuilderAddTraceHeaderWhenTraceSessionPresent()
+{
+
Stream.of(TraceType.values()).forEach(this::testAddTraceHeaderWithType);
+}
+
+@Test
+public void testBuilderNotAddTraceHeaderWithNoTraceSession()
+{
+Message msg = Message.builder(Verb._TEST_1,
noPayload).withTracingParams().build();
+assertNull(msg.header.traceSession());
+}
+
+private void testAddTraceHeaderWithType(TraceType traceType)
+{
+try
+{
+UUID sessionId = Tracing.instance.newSession(traceType);
+Message msg = Message.builder(Verb._TEST_1,
noPayload).withTracingParams().build();
+assertEquals(sessionId, msg.header.traceSession());
+assertEquals(traceType, msg.header.traceType());
+}
+finally
+{
+Tracing.instance.stopSession();
+}
+}
+
private void testCycle(Message msg) throws IOException
{
testCycle(msg, VERSION_30);
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15517) Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
[
https://issues.apache.org/jira/browse/CASSANDRA-15517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-15517:
Test and Documentation Plan: cci run
Status: Patch Available (was: Open)
[patch|https://github.com/krummas/cassandra/commits/marcuse/15517],
[tests|https://circleci.com/workflow-run/c083532a-080a-4ab6-8dd2-959834fccfa4]
> Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
> ---
>
> Key: CASSANDRA-15517
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15517
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
>
> The assert relies on the order of files in {{File.listFiles}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15517) Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
[
https://issues.apache.org/jira/browse/CASSANDRA-15517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-15517:
Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear
Impact(13164)
Complexity: Low Hanging Fruit
Component/s: Test/unit
Discovered By: Unit Test
Reviewers: David Capwell
Severity: Low
Status: Open (was: Triage Needed)
> Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
> ---
>
> Key: CASSANDRA-15517
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15517
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
>
> The assert relies on the order of files in {{File.listFiles}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15517) Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
Marcus Eriksson created CASSANDRA-15517:
---
Summary: Fix potentially flaky
ImportTest#testImportCorruptWithoutValidation
Key: CASSANDRA-15517
URL: https://issues.apache.org/jira/browse/CASSANDRA-15517
Project: Cassandra
Issue Type: Bug
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
The assert relies on the order of files in {{File.listFiles}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15499:
--
Status: Ready to Commit (was: Review In Progress)
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021328#comment-17021328
]
David Capwell commented on CASSANDRA-15499:
---
LGTM +1
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021326#comment-17021326
]
David Capwell commented on CASSANDRA-15499:
---
Looking now.
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021248#comment-17021248
]
Aleksey Yeschenko commented on CASSANDRA-15499:
---
+1 from me. When and if David approves, I'll commit for you guys.
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021224#comment-17021224
]
Yifan Cai commented on CASSANDRA-15499:
---
The patch was updated to having a method to add tracing params explicitly.
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[ https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021091#comment-17021091 ] Aleksey Yeschenko commented on CASSANDRA-14781: --- I'm perfectly fine with this level of duplication. Don't mind a separate ticket, either. Although FWIW that other ticket is the important one - we should never ever get to the point when an oversized mutation makes it to the commitlog at all, outside of potentially boundary mixed mode conditions - when a mutation validates for the current messaging version but ends up being slightly over the size for legacy. > Log message when mutation passed to CommitLog#add(Mutation) is too large is > not descriptive enough > -- > > Key: CASSANDRA-14781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14781 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Hints, Local/Commit Log, Messaging/Client >Reporter: Jordan West >Assignee: Tom Petracca >Priority: Normal > Labels: protocolv5 > Fix For: 4.0-beta > > Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch, > CASSANDRA-14781_3.11.patch > > > When hitting > [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257], > the log message produced does not help the operator track down what data is > being written. At a minimum the keyspace and cfIds involved would be useful > (and are available) – more detail might not be reasonable to include. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15509) In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds
[ https://issues.apache.org/jira/browse/CASSANDRA-15509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15509: Fix Version/s: 4.0 3.11.6 3.0.20 2.2.16 Since Version: 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/91f321e096714128b36732313e54975e466315f9 Resolution: Fixed Status: Resolved (was: Ready to Commit) and committed, thanks! > In-jvm upgrade dtest version parsing does not support 4.0 alpha/beta/rc builds > -- > > Key: CASSANDRA-15509 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15509 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0 > > > for example: > https://circleci.com/gh/krummas/cassandra/2686 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 03586bc6ec4080bba9a1ac0ae670fe8c543d84c9 Merge: 14aa561 91f321e Author: Marcus Eriksson AuthorDate: Wed Jan 22 14:18:01 2020 +0100 Merge branch 'cassandra-2.2' into cassandra-3.0 test/distributed/org/apache/cassandra/distributed/impl/Versions.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 6a41aa520b5016be8c8be517c6a50ec0b49c1941 Merge: 6a8e211 03586bc Author: Marcus Eriksson AuthorDate: Wed Jan 22 14:19:35 2020 +0100 Merge branch 'cassandra-3.0' into cassandra-3.11 test/distributed/org/apache/cassandra/distributed/impl/Versions.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Fix in-jvm upgrade dtest version parsing
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new 91f321e Fix in-jvm upgrade dtest version parsing
91f321e is described below
commit 91f321e096714128b36732313e54975e466315f9
Author: Marcus Eriksson
AuthorDate: Thu Jan 16 11:22:48 2020 +0100
Fix in-jvm upgrade dtest version parsing
Patch by marcuse; reviewed by David Capwell for CASSANDRA-15509
---
test/distributed/org/apache/cassandra/distributed/impl/Versions.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/Versions.java
b/test/distributed/org/apache/cassandra/distributed/impl/Versions.java
index dba1c13..8e8e284 100644
--- a/test/distributed/org/apache/cassandra/distributed/impl/Versions.java
+++ b/test/distributed/org/apache/cassandra/distributed/impl/Versions.java
@@ -146,7 +146,7 @@ public class Versions
public static Versions find()
{
logger.info("Looking for dtest jars in " + new
File("build").getAbsolutePath());
-final Pattern pattern = Pattern.compile("dtest-([0-9.]+)\\.jar");
+final Pattern pattern =
Pattern.compile("dtest-(?(\\d+)\\.(\\d+)(\\.\\d+)?(\\.\\d+)?)([~\\-]\\w[.\\w]*(?:\\-\\w[.\\w]*)*)?(\\+[.\\w]+)?\\.jar");
final Map> versions = new HashMap<>();
for (Major major : Major.values())
versions.put(major, new ArrayList<>());
@@ -156,7 +156,7 @@ public class Versions
Matcher m = pattern.matcher(file.getName());
if (!m.matches())
continue;
-String version = m.group(1);
+String version = m.group("fullversion");
Major major = Major.fromFull(version);
versions.get(major).add(new Version(major, version, new URL[] {
toURL(file) }));
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (6a8e211 -> 6a41aa5)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 6a8e211 Merge branch 'cassandra-3.0' into cassandra-3.11 new 91f321e Fix in-jvm upgrade dtest version parsing new 03586bc Merge branch 'cassandra-2.2' into cassandra-3.0 new 6a41aa5 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: test/distributed/org/apache/cassandra/distributed/impl/Versions.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (14aa561 -> 03586bc)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 14aa561 Merge branch 'cassandra-2.2' into cassandra-3.0 new 91f321e Fix in-jvm upgrade dtest version parsing new 03586bc Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: test/distributed/org/apache/cassandra/distributed/impl/Versions.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (3f74990 -> 025a6c7)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 3f74990 Fix in-jvm dtest message serialization new 91f321e Fix in-jvm upgrade dtest version parsing new 03586bc Merge branch 'cassandra-2.2' into cassandra-3.0 new 6a41aa5 Merge branch 'cassandra-3.0' into cassandra-3.11 new 025a6c7 Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: test/distributed/org/apache/cassandra/distributed/impl/Versions.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 025a6c7abbb6b221ae254d99b867bfddb110ef06 Merge: 3f74990 6a41aa5 Author: Marcus Eriksson AuthorDate: Wed Jan 22 14:23:49 2020 +0100 Merge branch 'cassandra-3.11' into trunk test/distributed/org/apache/cassandra/distributed/impl/Versions.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15515) Fix in-jvm message serialization between versions
[ https://issues.apache.org/jira/browse/CASSANDRA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15515: Summary: Fix in-jvm message serialization between versions (was: Fix in-jvm message versioning between versions) > Fix in-jvm message serialization between versions > - > > Key: CASSANDRA-15515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15515 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > trunk in-jvm-dtests currently uses MessagingService.current_version to > serialize messages to other nodes which is wrong in mixed mode -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15515) Fix in-jvm message serialization between versions
[ https://issues.apache.org/jira/browse/CASSANDRA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15515: Fix Version/s: 4.0-alpha Since Version: 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/3f749907f9f6845078aacb25c38a79e9a6fb8bed Resolution: Fixed Status: Resolved (was: Ready to Commit) and committed, thanks! > Fix in-jvm message serialization between versions > - > > Key: CASSANDRA-15515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15515 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-alpha > > > trunk in-jvm-dtests currently uses MessagingService.current_version to > serialize messages to other nodes which is wrong in mixed mode -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15515) Fix in-jvm message serialization between versions
[ https://issues.apache.org/jira/browse/CASSANDRA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15515: Status: Ready to Commit (was: Review In Progress) > Fix in-jvm message serialization between versions > - > > Key: CASSANDRA-15515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15515 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > trunk in-jvm-dtests currently uses MessagingService.current_version to > serialize messages to other nodes which is wrong in mixed mode -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Fix in-jvm dtest message serialization
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 3f74990 Fix in-jvm dtest message serialization
3f74990 is described below
commit 3f749907f9f6845078aacb25c38a79e9a6fb8bed
Author: Marcus Eriksson
AuthorDate: Tue Jan 21 08:58:18 2020 +0100
Fix in-jvm dtest message serialization
Patch by marcuse; reviewed by Alex Petrov for CASSANDRA-15515
---
test/distributed/org/apache/cassandra/distributed/impl/Instance.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/Instance.java
b/test/distributed/org/apache/cassandra/distributed/impl/Instance.java
index f39783d..03cc9b9 100644
--- a/test/distributed/org/apache/cassandra/distributed/impl/Instance.java
+++ b/test/distributed/org/apache/cassandra/distributed/impl/Instance.java
@@ -231,10 +231,11 @@ public class Instance extends IsolatedExecutor implements
IInvokableInstance
{
try (DataOutputBuffer out = new DataOutputBuffer(1024))
{
+int version = MessagingService.instance().versions.get(to);
InetAddressAndPort from = broadcastAddressAndPort();
Tracing.instance.traceOutgoingMessage(messageOut, to);
-Message.serializer.serialize(messageOut, out,
MessagingService.current_version);
-deliver.accept(to, new MessageImpl(messageOut.verb().id,
out.toByteArray(), messageOut.id(), MessagingService.current_version, from));
+Message.serializer.serialize(messageOut, out, version);
+deliver.accept(to, new MessageImpl(messageOut.verb().id,
out.toByteArray(), messageOut.id(), version, from));
}
catch (IOException e)
{
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15515) Fix in-jvm message versioning between versions
[ https://issues.apache.org/jira/browse/CASSANDRA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020966#comment-17020966 ] Alex Petrov commented on CASSANDRA-15515: - +1, lgtm! > Fix in-jvm message versioning between versions > -- > > Key: CASSANDRA-15515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15515 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > trunk in-jvm-dtests currently uses MessagingService.current_version to > serialize messages to other nodes which is wrong in mixed mode -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15515) Fix in-jvm message versioning between versions
[ https://issues.apache.org/jira/browse/CASSANDRA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15515: Reviewers: Alex Petrov, Alex Petrov (was: Alex Petrov) Alex Petrov, Alex Petrov Status: Review In Progress (was: Patch Available) > Fix in-jvm message versioning between versions > -- > > Key: CASSANDRA-15515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15515 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > trunk in-jvm-dtests currently uses MessagingService.current_version to > serialize messages to other nodes which is wrong in mixed mode -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15516) Failed partition updates should not consume memtable space
[ https://issues.apache.org/jira/browse/CASSANDRA-15516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020950#comment-17020950 ] Benedict Elliott Smith commented on CASSANDRA-15516: For those interested, a performance comparison of this work against 3.0 can be found on CASSANDRA-15511, which builds on this patch to be able to remove the partition lock. > Failed partition updates should not consume memtable space > -- > > Key: CASSANDRA-15516 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15516 > Project: Cassandra > Issue Type: Improvement > Components: Local/Memtable >Reporter: Benedict Elliott Smith >Priority: Normal > > Moving my proposed patch for CASSANDRA-15367 here. Contended memtable > updates currently waste memtable memory each time they fail to apply. This > patch avoids this problem by ensuring we only copy data once onto the > memtable, and reconcile this new data for all future attempts to insert. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15516) Failed partition updates should not consume memtable space
[ https://issues.apache.org/jira/browse/CASSANDRA-15516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-15516: --- Description: Moving my proposed patch for CASSANDRA-15367 here. Contended memtable updates currently waste memtable memory each time they fail to apply. This patch avoids this problem by ensuring we only copy data once onto the memtable, and reconcile this new data for all future attempts to insert. > Failed partition updates should not consume memtable space > -- > > Key: CASSANDRA-15516 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15516 > Project: Cassandra > Issue Type: Improvement > Components: Local/Memtable >Reporter: Benedict Elliott Smith >Priority: Normal > > Moving my proposed patch for CASSANDRA-15367 here. Contended memtable > updates currently waste memtable memory each time they fail to apply. This > patch avoids this problem by ensuring we only copy data once onto the > memtable, and reconcile this new data for all future attempts to insert. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15516) Failed partition updates should not consume memtable space
Benedict Elliott Smith created CASSANDRA-15516: -- Summary: Failed partition updates should not consume memtable space Key: CASSANDRA-15516 URL: https://issues.apache.org/jira/browse/CASSANDRA-15516 Project: Cassandra Issue Type: Improvement Components: Local/Memtable Reporter: Benedict Elliott Smith -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15515) Fix in-jvm message versioning between versions
[ https://issues.apache.org/jira/browse/CASSANDRA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15515: Description: trunk in-jvm-dtests currently uses MessagingService.current_version to serialize messages to other nodes which is wrong in mixed mode (was: trunk currently uses MessagingService.current_version to serialize messages to other nodes which is wrong in mixed mode) > Fix in-jvm message versioning between versions > -- > > Key: CASSANDRA-15515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15515 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > trunk in-jvm-dtests currently uses MessagingService.current_version to > serialize messages to other nodes which is wrong in mixed mode -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15515) Fix in-jvm message versioning between versions
[ https://issues.apache.org/jira/browse/CASSANDRA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15515: Test and Documentation Plan: cci Status: Patch Available (was: Open) [patch|https://github.com/krummas/cassandra/commits/marcuse/15515] > Fix in-jvm message versioning between versions > -- > > Key: CASSANDRA-15515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15515 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > trunk currently uses MessagingService.current_version to serialize messages > to other nodes which is wrong in mixed mode -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15515) Fix in-jvm message versioning between versions
[ https://issues.apache.org/jira/browse/CASSANDRA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15515: Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear Impact(13164) Complexity: Low Hanging Fruit Discovered By: Adhoc Test Severity: Low Status: Open (was: Triage Needed) > Fix in-jvm message versioning between versions > -- > > Key: CASSANDRA-15515 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15515 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > trunk currently uses MessagingService.current_version to serialize messages > to other nodes which is wrong in mixed mode -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15515) Fix in-jvm message versioning between versions
Marcus Eriksson created CASSANDRA-15515: --- Summary: Fix in-jvm message versioning between versions Key: CASSANDRA-15515 URL: https://issues.apache.org/jira/browse/CASSANDRA-15515 Project: Cassandra Issue Type: Bug Components: Test/dtest Reporter: Marcus Eriksson Assignee: Marcus Eriksson trunk currently uses MessagingService.current_version to serialize messages to other nodes which is wrong in mixed mode -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
