[jira] [Commented] (CASSANDRA-15120) Nodes that join the ring while another node is MOVING build an invalid view of the token ring
[ https://issues.apache.org/jira/browse/CASSANDRA-15120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839636#comment-16839636 ] Sam Tunnicliffe commented on CASSANDRA-15120: - I share your concerns around making changes to this area in 3.0, but I've spent some time working through this and I believe that this change is correct. I haven't really looked at any of non-test changes made to support gossip/networking in the tests yet, but I'll make another pass at it when you're done with those. > Nodes that join the ring while another node is MOVING build an invalid view > of the token ring > - > > Key: CASSANDRA-15120 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15120 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Membership >Reporter: Benedict >Assignee: Benedict >Priority: Normal > > Gossip only updates the token metadata for nodes in the NORMAL, SHUTDOWN or > LEAVING* statuses. MOVING and REMOVING_TOKEN nodes do not have their ring > information updated (nor do others, but these other states _should_ only be > taken by nodes that are not members of the ring). > If a node missed the most recent token-modifying events because they were not > a member of the ring when they happened (or because Gossip was delayed to > them), they will retain an invalid view of the ring until the node enters the > one of the NORMAL, SHUTDOWN or LEAVING states. > *LEAVING is populated differently, however, and in a probably unsafe manner > that this work will also address. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15126: - Since Version: 3.0.5 Status: Resolved (was: Ready to Commit) Resolution: Fixed Thanks, committed as [c07f3c88a4ba164bf01b0450b2463746b40c0d48|https://github.com/apache/cassandra/commit/c07f3c88a4ba164bf01b0450b2463746b40c0d48] and merged up. > Resource leak when queries apply a RowFilter > > > Key: CASSANDRA-15126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Benedict >Assignee: Benedict >Priority: Normal > Fix For: 3.0.19 > > > RowFilter.CQLFilter optionally removes those partitions that have no matching > results, but fails to close the iterator representing that partition’s > unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 6942c13eebbc18bd3f38ff4566e0a8cc051fc4d6
Merge: aa762c6 a6d32bb
Author: Benedict Elliott Smith
AuthorDate: Tue May 14 15:44:38 2019 +0100
Merge branch 'cassandra-3.11' into trunk
.../apache/cassandra/db/filter/RowFilterTest.java | 130 +
1 file changed, 130 insertions(+)
diff --cc test/unit/org/apache/cassandra/db/filter/RowFilterTest.java
index 000,0e15013..333d3f8
mode 00,100644..100644
--- a/test/unit/org/apache/cassandra/db/filter/RowFilterTest.java
+++ b/test/unit/org/apache/cassandra/db/filter/RowFilterTest.java
@@@ -1,0 -1,130 +1,130 @@@
+ /*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ package org.apache.cassandra.db.filter;
+
+ import java.nio.ByteBuffer;
+ import java.util.ArrayList;
+ import java.util.concurrent.atomic.AtomicBoolean;
+
+ import org.junit.Assert;
+ import org.junit.Test;
+
-import org.apache.cassandra.config.CFMetaData;
-import org.apache.cassandra.config.ColumnDefinition;
+ import org.apache.cassandra.cql3.ColumnIdentifier;
+ import org.apache.cassandra.cql3.Operator;
+ import org.apache.cassandra.db.Clustering;
+ import org.apache.cassandra.db.DecoratedKey;
+ import org.apache.cassandra.db.DeletionTime;
+ import org.apache.cassandra.db.LivenessInfo;
-import org.apache.cassandra.db.PartitionColumns;
++import org.apache.cassandra.db.RegularAndStaticColumns;
+ import org.apache.cassandra.db.marshal.Int32Type;
+ import
org.apache.cassandra.db.partitions.SingletonUnfilteredPartitionIterator;
+ import org.apache.cassandra.db.partitions.UnfilteredPartitionIterator;
+ import org.apache.cassandra.db.rows.BTreeRow;
+ import org.apache.cassandra.db.rows.BufferCell;
+ import org.apache.cassandra.db.rows.Cell;
+ import org.apache.cassandra.db.rows.EncodingStats;
+ import org.apache.cassandra.db.rows.Row;
+ import org.apache.cassandra.db.rows.Rows;
+ import org.apache.cassandra.db.rows.Unfiltered;
+ import org.apache.cassandra.db.rows.UnfilteredRowIterator;
++import org.apache.cassandra.schema.ColumnMetadata;
++import org.apache.cassandra.schema.TableMetadata;
+ import org.apache.cassandra.utils.btree.BTree;
+
+ public class RowFilterTest
+ {
+
+ @Test
+ public void testCQLFilterClose()
+ {
+ // CASSANDRA-15126
-CFMetaData metadata = CFMetaData.Builder.create("testks", "testcf")
-.addPartitionKey("pk",
Int32Type.instance)
-.addStaticColumn("s",
Int32Type.instance)
-.addRegularColumn("r",
Int32Type.instance)
-.build();
-ColumnDefinition s = metadata.getColumnDefinition(new
ColumnIdentifier("s", true));
-ColumnDefinition r = metadata.getColumnDefinition(new
ColumnIdentifier("r", true));
++TableMetadata metadata = TableMetadata.builder("testks", "testcf")
++ .addPartitionKeyColumn("pk",
Int32Type.instance)
++ .addStaticColumn("s",
Int32Type.instance)
++ .addRegularColumn("r",
Int32Type.instance)
++ .build();
++ColumnMetadata s = metadata.getColumn(new ColumnIdentifier("s",
true));
++ColumnMetadata r = metadata.getColumn(new ColumnIdentifier("r",
true));
+
+ ByteBuffer one = Int32Type.instance.decompose(1);
+ RowFilter filter = RowFilter.NONE.withNewExpressions(new
ArrayList<>());
+ filter.add(s, Operator.NEQ, one);
+ AtomicBoolean closed = new AtomicBoolean();
+ UnfilteredPartitionIterator iter = filter.filter(new
SingletonUnfilteredPartitionIterator(new UnfilteredRowIterator()
+ {
+ public DeletionTime partitionLevelDeletion() { return null; }
+ public EncodingStats stats() { return null; }
-public CFMetaData metadata() { return metadata; }
++
[cassandra] branch cassandra-3.0 updated (daf6c85 -> c07f3c8)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from daf6c85 Skip cells with illegal column names when reading legacy sstables add c07f3c8 Resource leak when queries apply a RowFilter No new revisions were added by this update. Summary of changes: .../org/apache/cassandra/db/filter/RowFilter.java | 11 +- .../apache/cassandra/db/filter/RowFilterTest.java | 130 + 2 files changed, 140 insertions(+), 1 deletion(-) create mode 100644 test/unit/org/apache/cassandra/db/filter/RowFilterTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (aa762c6 -> 6942c13)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from aa762c6 Support building Cassandra with JDK 11 add c07f3c8 Resource leak when queries apply a RowFilter add a6d32bb Merge branch 'cassandra-3.0' into cassandra-3.11 new 6942c13 Merge branch 'cassandra-3.11' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../apache/cassandra/db/filter/RowFilterTest.java | 130 + 1 file changed, 130 insertions(+) create mode 100644 test/unit/org/apache/cassandra/db/filter/RowFilterTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (ba01089 -> a6d32bb)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from ba01089 Merge branch 'cassandra-3.0' into cassandra-3.11 add c07f3c8 Resource leak when queries apply a RowFilter add a6d32bb Merge branch 'cassandra-3.0' into cassandra-3.11 No new revisions were added by this update. Summary of changes: .../apache/cassandra/db/filter/RowFilterTest.java | 130 + 1 file changed, 130 insertions(+) create mode 100644 test/unit/org/apache/cassandra/db/filter/RowFilterTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14654) Reduce heap pressure during compactions
[ https://issues.apache.org/jira/browse/CASSANDRA-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839433#comment-16839433 ] Benedict commented on CASSANDRA-14654: -- Unless you made some unrelated changes, this patch is marked Ready to Commit. Just checking you aren't waiting for further action on my or anyone else's part? > Reduce heap pressure during compactions > --- > > Key: CASSANDRA-14654 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14654 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Normal > Labels: Performance, pull-request-available > Fix For: 4.x > > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png > > Time Spent: 40m > Remaining Estimate: 0h > > Small partition compactions are painfully slow with a lot of overhead per > partition. There also tends to be an excess of objects created (ie > 200-700mb/s) per compaction thread. > The EncoderStats walks through all the partitions and with mergeWith it will > create a new one per partition as it walks the potentially millions of > partitions. In a test scenario of about 600byte partitions and a couple 100mb > of data this consumed ~16% of the heap pressure. Changing this to instead > mutably track the min values and create one in a EncodingStats.Collector > brought this down considerably (but not 100% since the > UnfilteredRowIterator.stats() still creates 1 per partition). > The KeyCacheKey makes a full copy of the underlying byte array in > ByteBufferUtil.getArray in its constructor. This is the dominating heap > pressure as there are more sstables. By changing this to just keeping the > original it completely eliminates the current dominator of the compactions > and also improves read performance. > Minor tweak included for this as well for operators when compactions are > behind on low read clusters is to make the preemptive opening setting a > hotprop. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15126: -- Status: Ready to Commit (was: Review In Progress) > Resource leak when queries apply a RowFilter > > > Key: CASSANDRA-15126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Benedict >Assignee: Benedict >Priority: Normal > Fix For: 3.0.19 > > > RowFilter.CQLFilter optionally removes those partitions that have no matching > results, but fails to close the iterator representing that partition’s > unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839370#comment-16839370 ] Aleksey Yeschenko commented on CASSANDRA-15126: --- +1 > Resource leak when queries apply a RowFilter > > > Key: CASSANDRA-15126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Benedict >Assignee: Benedict >Priority: Normal > Fix For: 3.0.19 > > > RowFilter.CQLFilter optionally removes those partitions that have no matching > results, but fails to close the iterator representing that partition’s > unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839348#comment-16839348 ] Sam Tunnicliffe commented on CASSANDRA-15126: - +1 LGTM > Resource leak when queries apply a RowFilter > > > Key: CASSANDRA-15126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Benedict >Assignee: Benedict >Priority: Normal > Fix For: 3.0.19 > > > RowFilter.CQLFilter optionally removes those partitions that have no matching > results, but fails to close the iterator representing that partition’s > unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-15126: Status: Review In Progress (was: Patch Available) > Resource leak when queries apply a RowFilter > > > Key: CASSANDRA-15126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Benedict >Assignee: Benedict >Priority: Normal > Fix For: 3.0.19 > > > RowFilter.CQLFilter optionally removes those partitions that have no matching > results, but fails to close the iterator representing that partition’s > unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15013) Message Flusher queue can grow unbounded, potentially running JVM out of memory
[ https://issues.apache.org/jira/browse/CASSANDRA-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839326#comment-16839326 ] Benedict commented on CASSANDRA-15013: -- Thanks [~sumanth.pasupuleti]. I started reviewing your new changes, and part way through realised we could potentially simplify this all a great deal with a slightly different approach. Namely, if we were to hash endpoints to a specific eventLoop when accepting the connection. If we were to do this, we could have very simple per-thread accounting, and we could even aggregate all of the per-endpoint channels into a single flusher for stopping/starting together once they exceed their limits. Everything would be single threaded, so our logic would be much simpler to reason about. This isn't without its tradeoffs - potentially users might have a setup with a single application node speaking to the cluster, but this would be a very peculiar system design to pair with Cassandra, and a single dedicated eventLoop for this node would still likely suffice for a majority of workloads. We also have the potential issue of endpoint collisions, but if we use a cryptographic hash function this should only be a problem for a very small number of nodes (and if we ever find it is a real problem, we can remedy it) What do you think? I'm sorry for moving the goal posts suddenly, it just hadn't occurred to me until now. My goal is only the best patch, so I'm interested to hear your thoughts. > Message Flusher queue can grow unbounded, potentially running JVM out of > memory > --- > > Key: CASSANDRA-15013 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15013 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Labels: pull-request-available > Fix For: 4.0, 3.0.x, 3.11.x > > Attachments: BlockedEpollEventLoopFromHeapDump.png, > BlockedEpollEventLoopFromThreadDump.png, RequestExecutorQueueFull.png, heap > dump showing each ImmediateFlusher taking upto 600MB.png > > > This is a follow-up ticket out of CASSANDRA-14855, to make the Flusher queue > bounded, since, in the current state, items get added to the queue without > any checks on queue size, nor with any checks on netty outbound buffer to > check the isWritable state. > We are seeing this issue hit our production 3.0 clusters quite often. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15126: - Test and Documentation Plan: unit test included in patch Status: Patch Available (was: Open) > Resource leak when queries apply a RowFilter > > > Key: CASSANDRA-15126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Benedict >Assignee: Benedict >Priority: Normal > > RowFilter.CQLFilter optionally removes those partitions that have no matching > results, but fails to close the iterator representing that partition’s > unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15126: - Fix Version/s: 3.0.19 > Resource leak when queries apply a RowFilter > > > Key: CASSANDRA-15126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Benedict >Assignee: Benedict >Priority: Normal > Fix For: 3.0.19 > > > RowFilter.CQLFilter optionally removes those partitions that have no matching > results, but fails to close the iterator representing that partition’s > unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839313#comment-16839313 ] Benedict commented on CASSANDRA-15126: -- Patch available [here|https://github.com/belliottsmith/cassandra/tree/15126-3.0], with unit test. It turns out this was already fixed in 3.11 and trunk by [~Stefania] in CASSANDRA-13050, but I will merge up the unit test anyway. > Resource leak when queries apply a RowFilter > > > Key: CASSANDRA-15126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Benedict >Assignee: Benedict >Priority: Normal > > RowFilter.CQLFilter optionally removes those partitions that have no matching > results, but fails to close the iterator representing that partition’s > unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
[ https://issues.apache.org/jira/browse/CASSANDRA-15126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15126: - Severity: Normal Complexity: Low Hanging Fruit Discovered By: User Report Bug Category: Parent values: Degradation(12984)Level 1 values: Resource Management(12995) Reviewers: Aleksey Yeschenko, Sam Tunnicliffe Status: Open (was: Triage Needed) > Resource leak when queries apply a RowFilter > > > Key: CASSANDRA-15126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Benedict >Assignee: Benedict >Priority: Normal > > RowFilter.CQLFilter optionally removes those partitions that have no matching > results, but fails to close the iterator representing that partition’s > unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15126) Resource leak when queries apply a RowFilter
Benedict created CASSANDRA-15126: Summary: Resource leak when queries apply a RowFilter Key: CASSANDRA-15126 URL: https://issues.apache.org/jira/browse/CASSANDRA-15126 Project: Cassandra Issue Type: Bug Components: CQL/Interpreter Reporter: Benedict Assignee: Benedict RowFilter.CQLFilter optionally removes those partitions that have no matching results, but fails to close the iterator representing that partition’s unfiltered results, leaking resources when this happens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13292) Replace MessagingService usage of MD5 with something more modern
[
https://issues.apache.org/jira/browse/CASSANDRA-13292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839229#comment-16839229
]
Elliott Sims commented on CASSANDRA-13292:
--
In terms of hash algorithms, a cryptographic hash is one that's expensive to
invert and it doesn't necessarily affect collision probabilities. For digests,
I don't think difficulty of inversion matters at all since it's definitely not
trying to hide the original data or protect against deliberate corruption.
What does matter is output size and distribution. So, any fast 128-bit hash
with good distribution should be equivalent to MD5: Murmur3F (faster than md5
but slower than the rest, well-supported, greenrobot implementation claims to
be much faster than guava),
xxH3 (fast, brand new/unstable, possible collisions),
Farmhash128,
Spookyhash128
Default/reference implementations seem to all be in C/C++ along with most
benchmarks, so "best/fastest" may not be the same as "best/fastest in available
Java libraries with compatible licenses"
> Replace MessagingService usage of MD5 with something more modern
>
>
> Key: CASSANDRA-13292
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13292
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>Priority: Normal
> Attachments: quorum-concurrency-reads-quorum.svg
>
>
> While profiling C* via multiple profilers, I've consistently seen a
> significant amount of time being spent calculating MD5 digests.
> {code}
> Stack Trace Sample CountPercentage(%)
> sun.security.provider.MD5.implCompress(byte[], int) 264 1.566
>sun.security.provider.DigestBase.implCompressMultiBlock(byte[], int, int)
> 200 1.187
> sun.security.provider.DigestBase.engineUpdate(byte[], int, int) 200
> 1.187
> java.security.MessageDigestSpi.engineUpdate(ByteBuffer) 200
> 1.187
> java.security.MessageDigest$Delegate.engineUpdate(ByteBuffer)
> 200 1.187
>java.security.MessageDigest.update(ByteBuffer) 200 1.187
> org.apache.cassandra.db.Column.updateDigest(MessageDigest)
> 193 1.145
>
> org.apache.cassandra.db.ColumnFamily.updateDigest(MessageDigest) 193 1.145
>
> org.apache.cassandra.db.ColumnFamily.digest(ColumnFamily) 193 1.145
>
> org.apache.cassandra.service.RowDigestResolver.resolve() 106 0.629
>
> org.apache.cassandra.service.RowDigestResolver.resolve()106 0.629
>
> org.apache.cassandra.service.ReadCallback.get() 88 0.522
>
> org.apache.cassandra.service.AbstractReadExecutor.get() 88 0.522
>
> org.apache.cassandra.service.StorageProxy.fetchRows(List, ConsistencyLevel)
> 88 0.522
>
> org.apache.cassandra.service.StorageProxy.read(List, ConsistencyLevel)
> 88 0.522
>
> org.apache.cassandra.service.pager.SliceQueryPager.queryNextPage(int,
> ConsistencyLevel, boolean) 88 0.522
>
> org.apache.cassandra.service.pager.AbstractQueryPager.fetchPage(int) 88
> 0.522
>
> org.apache.cassandra.service.pager.SliceQueryPager.fetchPage(int) 88
> 0.522
>
> org.apache.cassandra.cql3.statements.SelectStatement.execute(QueryState,
> QueryOptions) 88 0.522
>
> org.apache.cassandra.cql3.statements.SelectStatement.execute(QueryState,
> QueryOptions) 88 0.522
>
> org.apache.cassandra.cql3.QueryProcessor.processStatement(CQLStatement,
> QueryState, QueryOptions) 88 0.522
>
> org.apache.cassandra.cql3.QueryProcessor.process(String, QueryState,
> QueryOptions) 88 0.522
>
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryState)
> 88 0.522
>
> org.apache.cassandra.transport.Message$Dispatcher.messageReceived(ChannelHandlerContext,
> MessageEvent) 88 0.522
>
>
