[jira] [Comment Edited] (CASSANDRA-15214) OOMs caught and not rethrown
[
https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099546#comment-17099546
]
Joey Lynch edited comment on CASSANDRA-15214 at 5/5/20, 5:57 AM:
-
Quick update on this from the jvmquake side we are now building [architecture
specific artifacts|https://github.com/Netflix-Skunkworks/jvmquake/releases]
that will work with any JVM newer than Java 8, they link only against the
platform specific libc (we're also now testing on Java 8 and 11, on both zulu
and openjdk JVMs). I think this means it would be plausible to include the
{{libjvmquake-linux-x86_64.so}} in {{libs}} and then have a switch on uname -s
-m to determine to pick it up or not. Right now we're only building for linux
amd64 but if there is interest I can generate more architectures (linux arm
probably makes sense, and could do osx). I also still like the idea of having a
agents/available and agents/enabled folder like apache does for modules, users
can just symlink agents from one to the other to include them (and we can
symlink jamm and jvmquake by default).
[~yifanc] I agree that the OutOfMemory conditions that do not result in "true"
JVM OOM (meaning that it would cause a heapdump via {{HeapDumpOnOutOfMemory}})
such as direct buffer allocations will not get caught by jvmquake, my testing
confirms your findings, although the jvmquake GC instability algorithm will
still trigger in various real world scenarios I've run into.
I feel like the right move might be to walk back a small bit of CASSANDRA-13006
where we stopped forcibly killing the JVM ourselves and let the JVM do it.
Specifically if the OOM message contains "Direct buffer memory" we could do
what jvmquake does and force the JVM into a "normal" OOM by [allocating large
long
arrays|https://github.com/Netflix-Skunkworks/jvmquake/blob/master/src/jvmquake.c#L103].
This will then trigger a proper OOM and get us heap dumping. It's relatively
easy to ignore the "sacrificial" long array in a heap dump and we could log
clearly what is happening.
was (Author: jolynch):
Quick update on this from the jvmquake side we are now building [architecture
specific artifacts|https://github.com/Netflix-Skunkworks/jvmquake/releases]
that will work with any JVM newer than Java 8, they link only against the
platform specific libc (we're also now testing on Java 8 and 11, on both zulu
and openjdk JVMs). I think this means it would be plausible to include the
{{libjvmquake-linux-x86_64.so}} in {{libs}} and then have a switch on uname -s
-m to determine to pick it up or not. Right now we're only building for linux
amd64 but if there is interest I can generate more architectures (linux arm
probably makes sense, and could do osx). I also still like the idea of having a
agents/available and agents/enabled folder like apache does for modules, users
can just symlink agents from one to the other to include them (and we can
symlink jamm and jvmquake by default).
[~yifanc] I agree that the OutOfMemory conditions that do not result in "true"
JVM OOM (meaning that it would cause a heapdump via {{HeapDumpOnOutOfMemory}})
will not get caught by jvmquake, my testing confirms your findings, although
the jvmquake GC instability algorithm will still trigger in various real world
scenarios I've run into.
I feel like the right move mightly be to walk back a small bit of
CASSANDRA-13006 where we stopped forcibly killing the JVM ourselves and let the
JVM do it. Specifically if the OOM message contains "Direct buffer memory" we
could do what jvmquake does and force the JVM into a "normal" OOM by
[allocating large long
arrays|https://github.com/Netflix-Skunkworks/jvmquake/blob/master/src/jvmquake.c#L103].
This will then trigger a proper OOM and get us heap dumping. It's relatively
easy to ignore the "sacrificial" long array in a heap dump and we could log
clearly what is happening.
> OOMs caught and not rethrown
>
>
> Key: CASSANDRA-15214
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15214
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client, Messaging/Internode
>Reporter: Benedict Elliott Smith
>Priority: Normal
> Fix For: 4.0, 4.0-rc
>
> Attachments: oom-experiments.zip
>
>
> Netty (at least, and perhaps elsewhere in Executors) catches all exceptions,
> so presently there is no way to ensure that an OOM reaches the JVM handler to
> trigger a crash/heapdump.
> It may be that the simplest most consistent way to do this would be to have a
> single thread spawned at startup that waits for any exceptions we must
> propagate to the Runtime.
> We could probably submit a patch upstream to Netty, but for a guaranteed
> future proof approach, it may be worth paying the cost of a
[jira] [Commented] (CASSANDRA-15214) OOMs caught and not rethrown
[
https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099546#comment-17099546
]
Joey Lynch commented on CASSANDRA-15214:
Quick update on this from the jvmquake side we are now building [architecture
specific artifacts|https://github.com/Netflix-Skunkworks/jvmquake/releases]
that will work with any JVM newer than Java 8, they link only against the
platform specific libc (we're also now testing on Java 8 and 11, on both zulu
and openjdk JVMs). I think this means it would be plausible to include the
{{libjvmquake-linux-x86_64.so}} in {{libs}} and then have a switch on uname -s
-m to determine to pick it up or not. Right now we're only building for linux
amd64 but if there is interest I can generate more architectures (linux arm
probably makes sense, and could do osx). I also still like the idea of having a
agents/available and agents/enabled folder like apache does for modules, users
can just symlink agents from one to the other to include them (and we can
symlink jamm and jvmquake by default).
[~yifanc] I agree that the OutOfMemory conditions that do not result in "true"
JVM OOM (meaning that it would cause a heapdump via {{HeapDumpOnOutOfMemory}})
will not get caught by jvmquake, my testing confirms your findings, although
the jvmquake GC instability algorithm will still trigger in various real world
scenarios I've run into.
I feel like the right move mightly be to walk back a small bit of
CASSANDRA-13006 where we stopped forcibly killing the JVM ourselves and let the
JVM do it. Specifically if the OOM message contains "Direct buffer memory" we
could do what jvmquake does and force the JVM into a "normal" OOM by
[allocating large long
arrays|https://github.com/Netflix-Skunkworks/jvmquake/blob/master/src/jvmquake.c#L103].
This will then trigger a proper OOM and get us heap dumping. It's relatively
easy to ignore the "sacrificial" long array in a heap dump and we could log
clearly what is happening.
> OOMs caught and not rethrown
>
>
> Key: CASSANDRA-15214
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15214
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client, Messaging/Internode
>Reporter: Benedict Elliott Smith
>Priority: Normal
> Fix For: 4.0, 4.0-rc
>
> Attachments: oom-experiments.zip
>
>
> Netty (at least, and perhaps elsewhere in Executors) catches all exceptions,
> so presently there is no way to ensure that an OOM reaches the JVM handler to
> trigger a crash/heapdump.
> It may be that the simplest most consistent way to do this would be to have a
> single thread spawned at startup that waits for any exceptions we must
> propagate to the Runtime.
> We could probably submit a patch upstream to Netty, but for a guaranteed
> future proof approach, it may be worth paying the cost of a single thread.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15753) bin/sstableverify should support user provided token ranges
[ https://issues.apache.org/jira/browse/CASSANDRA-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15753: - Status: Ready to Commit (was: Review In Progress) +1 > bin/sstableverify should support user provided token ranges > --- > > Key: CASSANDRA-15753 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15753 > Project: Cassandra > Issue Type: Improvement > Components: Tool/sstable >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > Verify should allow options for token ranges so the stand alone verifier is > able to include user supplied ranges; this is useful for tools to verify > outside of the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15782) Compression test failure
[ https://issues.apache.org/jira/browse/CASSANDRA-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099495#comment-17099495 ] Joey Lynch commented on CASSANDRA-15782: All cqlsh dtests are now passing. [java 11 cqlsh dtests|https://app.circleci.com/pipelines/github/jolynch/cassandra/12/workflows/3f45cb2b-bb15-44df-a67a-55529fbb9ccf] [java 8 cqlsh dtests|https://app.circleci.com/pipelines/github/jolynch/cassandra/12/workflows/4996b893-228d-4e2f-9ef4-913c849d8cd4] [~brandon.williams] or [~Bereng] have a sec to do a quick CR on the dtest fix [a2fe94ba|https://github.com/apache/cassandra-dtest/commit/a2fe94ba8004a2b7159e58281a8125411895604e]? > Compression test failure > > > Key: CASSANDRA-15782 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15782 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Joey Lynch >Priority: Normal > Fix For: 4.0, 4.0-beta > > > On CASSANDRA-15560 compression test failed. This was bisected to > [9c1bbf3|https://github.com/apache/cassandra/commit/9c1bbf3ac913f9bdf7a0e0922106804af42d2c1e] > from CASSANDRA-15379. > Full details here > CC/ [~jolynch] in case he can spot it quick. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15763) Tunable RangeTombstoneList initial size and resize factor
[ https://issues.apache.org/jira/browse/CASSANDRA-15763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099454#comment-17099454 ] Dinesh Joshi commented on CASSANDRA-15763: -- Thanks for the patch. Are there specific cases that will benefit from this? If so, it would be great to list those cases. > Tunable RangeTombstoneList initial size and resize factor > - > > Key: CASSANDRA-15763 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15763 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > > Cassandra allocates a backing array with a fixed size of one when creating a > new RangeTombstoneList object. The resize factor is fixed to 1.5. > The initial size and resize factor could be made tunable so it could reduce > cost in some cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099447#comment-17099447
]
Olivier Michallat commented on CASSANDRA-15299:
---
I work on the Java driver. Here are a few comments/questions on your branch
(let me know if there is a better medium for this).
1) Name clash:
{quote}The introduction of this new format, brings an unfortunate naming clash
between frames from the previous format [...] and the new outer frames. What
was previously referred to as a Frame could perhaps be better described as an
Envelope or Container [...] Prior to making any conclusion here though, for the
purposes of this document, legacy frames are referred to as CQL frames and the
new enclosures as outer frames
{quote}
You're hinting at renaming at a later point. I think that will be most welcome,
the names as they are now are pretty confusing. And IMHO you should rename the
outer container: even though "frame" is better suited for it, it's more
important to preserve familiarity for people coming from the legacy code.
2) CRC check failures: this will result in either dropping the corrupt frame,
or the whole connection (comments in {{InboundMessageHandler}}).
>From a client point of view, a dropped frame will result in request timeouts.
>We have no way of providing a better error, since the stream ids of the failed
>requests are in the corrupt payload. I'm wondering if it might not be better
>to drop the connection all the time: at least the client gets immediate
>feedback (we could try to propagate a cause), instead of a bunch of requests
>timing out for no apparent reason.
3) When two large messages are sent in a row, how will you tell the last outer
frame of the first message from the first outer frame of the second? Don't you
need some sort of {{is_last_frame}} flag in the header?
4) When compression is enabled, how are you going to test when the payload
reaches the max size? This would have to be added to the logic in
{{V5Flusher}}; but all the messages in the payload are compressed together, so
you can't check the size as you go like you do for the uncompressed version. I
was thinking of maybe compressing all available results and splitting in
multiple uncontained frames if the compressed payload is too big, but that
seems at odds with this invariant in {{InboundMessageHandler}}:
{quote}All the bytes in an uncontained frame always belong to a single
message.{quote}
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-beta
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> particular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32
[jira] [Commented] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[
https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099431#comment-17099431
]
Jon Meredith commented on CASSANDRA-15766:
--
I've pushed up my WIP,
[branch|https://github.com/jonmeredith/cassandra/tree/15766] [and
PR|https://github.com/apache/cassandra/pull/582]
Disassembling generated code, it looks like we trade eagerly constructing a new
object array for an invoke dynamic call. I haven't had a chance to dig any more
to compare the costs. I'm interested what you think.
{code:java}
public void simpleLog();
Code:
0: aload_0
1: getfield #7 // Field mock:Lorg/slf4j/Logger;
4: getstatic #11 // Field
org/apache/cassandra/utils/NoSpamLogger$Level.INFO:Lorg/apache/cassandra/utils/NoSpamLogger$Level;
7: lconst_1
8: getstatic #23 // Field
java/util/concurrent/TimeUnit.NANOSECONDS:Ljava/util/concurrent/TimeUnit;
11: ldc #90 // String {}
13: iconst_1
14: anewarray #26 // class java/lang/Object
17: dup
18: iconst_0
19: ldc #91 // String param
21: aastore
22: invokestatic #28 // Method
org/apache/cassandra/utils/NoSpamLogger.log:(Lorg/slf4j/Logger;Lorg/apache/cassandra/utils/NoSpamLogger$Level;JLjava/util/concurrent/TimeUnit;Ljava/lang/String;[Ljava/lang/Object;)Z
25: pop
26: return
public void paramLog();
Code:
0: aload_0
1: getfield #7 // Field mock:Lorg/slf4j/Logger;
4: getstatic #11 // Field
org/apache/cassandra/utils/NoSpamLogger$Level.INFO:Lorg/apache/cassandra/utils/NoSpamLogger$Level;
7: lconst_1
8: getstatic #23 // Field
java/util/concurrent/TimeUnit.NANOSECONDS:Ljava/util/concurrent/TimeUnit;
11: ldc #90 // String {}
13: invokedynamic #92, 0 // InvokeDynamic
#1:get:()Ljava/util/function/Supplier;
18: invokestatic #82 // Method
org/apache/cassandra/utils/NoSpamLogger.log:(Lorg/slf4j/Logger;Lorg/apache/cassandra/utils/NoSpamLogger$Level;JLjava/util/concurrent/TimeUnit;Ljava/lang/String;Ljava/util/function/Supplier;)Z
21: pop
22: return
{code}
> NoSpamLogger arguments building objects on hot paths
>
>
> Key: CASSANDRA-15766
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15766
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-rc
>
>
> NoSpamLogger is used in hot logging paths to prevent logs being overrun. For
> that to be most effective the arguments to the logger need to be cheap to
> construct. During the internode messaging refactor CASSANDRA-15066,
> performance changes to BufferPool for CASSANDRA-14416
> were accidentally reverted in the merge up from 3.11.
> Reviewing other uses since, it looks like there are a few places where the
> arguments require some form of String building.
> org.apache.cassandra.net.InboundSink#accept
> org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame
> org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize
> org.apache.cassandra.net.OutboundConnection#onOverloaded
> org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks
> Formatting arguments should either be precomputed, or if expensive they
> should be computed after the decision on whether to noSpamLog has been made.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15787) CQLSH doesn't exit with proper code with multiple SOURCE directives.
[
https://issues.apache.org/jira/browse/CASSANDRA-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kaden Luke Nelson updated CASSANDRA-15787:
--
Description:
When I have the following files;
{code:java}
# master.sql
CREATE TABLE asdf ( id text, primary key (id));{code}
{code:java}
# a.sql
SOURCE './master.sql';
CREATE TABLE asdf ( id text, primary key (id));{code}
{code:java}
# b.sql
SOURCE './a.sql';{code}
and execute
{code:java}
cqlsh --keyspace example --file ./b.sql{code}
I don't get returned the proper status code. This happens because when multiple
SOURCE commands are chained together like in the example above, the error
doesn't get passed through to the final 'shell'.
Subshell that needs to pass the statement error value to the parent shell:
https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L1776
final statement error check which returns the parent shell statement error:
[https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L2474]
was:
When I have the following files;
{code:java}
# master.sql
CREATE TABLE asdf ( id text, primary key (id));{code}
{code:java}
# a.sql
SOURCE './master.sql';
CREATE TABLE asdf ( id text, primary key (id));{code}
{code:java}
# b.sql
SOURCE './a.sql';{code}
and execute
{code:java}
cqlsh --keyspace example --file ./b.sql{code}
I don't get returned the proper status code. This happens because when multiple
SOURCE commands are chained together like in the example above, the error
doesn't get passed through to the final 'shell'.
Subshell that needs to pass the statement error value to the parent shell:
[https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L1776
]
final statement error check which returns the parent shell statement error:
[https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L2474]
> CQLSH doesn't exit with proper code with multiple SOURCE directives.
>
>
> Key: CASSANDRA-15787
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15787
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Kaden Luke Nelson
>Assignee: Kaden Luke Nelson
>Priority: Normal
>
> When I have the following files;
>
> {code:java}
> # master.sql
> CREATE TABLE asdf ( id text, primary key (id));{code}
> {code:java}
> # a.sql
> SOURCE './master.sql';
> CREATE TABLE asdf ( id text, primary key (id));{code}
> {code:java}
> # b.sql
> SOURCE './a.sql';{code}
>
> and execute
> {code:java}
> cqlsh --keyspace example --file ./b.sql{code}
>
> I don't get returned the proper status code. This happens because when
> multiple SOURCE commands are chained together like in the example above, the
> error doesn't get passed through to the final 'shell'.
>
> Subshell that needs to pass the statement error value to the parent shell:
> https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L1776
> final statement error check which returns the parent shell statement error:
> [https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L2474]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15787) CQLSH doesn't exit with proper code with multiple SOURCE directives.
Kaden Luke Nelson created CASSANDRA-15787:
-
Summary: CQLSH doesn't exit with proper code with multiple SOURCE
directives.
Key: CASSANDRA-15787
URL: https://issues.apache.org/jira/browse/CASSANDRA-15787
Project: Cassandra
Issue Type: Bug
Components: Tool/cqlsh
Reporter: Kaden Luke Nelson
Assignee: Kaden Luke Nelson
When I have the following files;
{code:java}
# master.sql
CREATE TABLE asdf ( id text, primary key (id));{code}
{code:java}
# a.sql
SOURCE './master.sql';
CREATE TABLE asdf ( id text, primary key (id));{code}
{code:java}
# b.sql
SOURCE './a.sql';{code}
and execute
{code:java}
cqlsh --keyspace example --file ./b.sql{code}
I don't get returned the proper status code. This happens because when multiple
SOURCE commands are chained together like in the example above, the error
doesn't get passed through to the final 'shell'.
Subshell that needs to pass the statement error value to the parent shell:
[https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L1776
]
final statement error check which returns the parent shell statement error:
[https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L2474]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14248) SSTableIndex should not use Ref#globalCount() to determine when to delete index file
[
https://issues.apache.org/jira/browse/CASSANDRA-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099323#comment-17099323
]
Jordan West edited comment on CASSANDRA-14248 at 5/4/20, 8:19 PM:
--
[~jbellis] thanks for the updated link. I will take a look. Do you have a test
that validates the bit about {{discoverForComponents}}. I took one of the
existing \{{SASIIndexTest}}s and added the following:
{code:java}
if (forceFlush)
{
for (SSTable sst: store.getLiveSSTables())
{
Set components =
SSTable.discoverComponentsFor(sst.descriptor);
System.out.println("Components: " + components);
}
}
{code}
When I run it, I see {{Components: [Data.db, Index.db, Filter.db, Digest.crc32,
Statistics.db, TOC.txt, Summary.db, CompressionInfo.db]}}.
Reading {{discoverComponentsFor}}, my take is that
{{Descriptor#filenameFor(Component)}} doesn't properly build the index file
name (since it doesn't have the index name itself and
Component.Type.SECONDARY_INDEX.repr = "SI_*.db" which is not actually doing
glob matching). This results in the exist call failing and the indexes being
excluded.
was (Author: jrwest):
[~jbellis] thanks for the updated link. I will take a look. Do you have a test
that validates the bit about {{discoverForComponents}}. I took one of the
existing \{{SASIIndexTest}}s and added the following:
{code:java}
if (forceFlush)
{
for (SSTable sst: store.getLiveSSTables())
{
Set components =
SSTable.discoverComponentsFor(sst.descriptor);
System.out.println("Components: " + components);
}}
{code}
When I run it, I see {{Components: [Data.db, Index.db, Filter.db, Digest.crc32,
Statistics.db, TOC.txt, Summary.db, CompressionInfo.db]}}.
Reading {{discoverComponentsFor}}, my take is that
{{Descriptor#filenameFor(Component)}} doesn't properly build the index file
name (since it doesn't have the index name itself and
Component.Type.SECONDARY_INDEX.repr = "SI_*.db" which is not actually doing
glob matching).
> SSTableIndex should not use Ref#globalCount() to determine when to delete
> index file
>
>
> Key: CASSANDRA-14248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14248
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.11.x
>
>
> {{SSTableIndex}} instances maintain a {{Ref}} to the underlying
> {{SSTableReader}} instance. When determining whether or not to delete the
> file after the last {{SSTableIndex}} reference is released, the
> implementation uses {{sstableRef.globalCount()}}:
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/sasi/SSTableIndex.java#L135.]
> This is incorrect because {{sstableRef.globalCount()}} returns the number of
> references to the specific instance of {{SSTableReader}}. However, in cases
> like index summary redistribution, there can be more than one instance of
> {{SSTableReader}}. Further, since the reader is shared across multiple
> indexes, not all indexes see the count go to 0. This can lead to cases where
> the {{SSTableIndex}} file is incorrectly deleted or not deleted when it
> should be.
>
> A more correct implementation would be to either:
> * Tie into the existing {{SSTableTidier}}. SASI indexes already are SSTable
> components but are not cleaned up by the {{SSTableTidier}} because they are
> not found with the currently cleanup implementation
> * Revamp {{SSTableIndex}} reference counting to use {{Ref}} and implement a
> new tidier.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14248) SSTableIndex should not use Ref#globalCount() to determine when to delete index file
[
https://issues.apache.org/jira/browse/CASSANDRA-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099323#comment-17099323
]
Jordan West commented on CASSANDRA-14248:
-
[~jbellis] thanks for the updated link. I will take a look. Do you have a test
that validates the bit about {{discoverForComponents}}. I took one of the
existing \{{SASIIndexTest}}s and added the following:
{code:java}
if (forceFlush)
{
for (SSTable sst: store.getLiveSSTables())
{
Set components =
SSTable.discoverComponentsFor(sst.descriptor);
System.out.println("Components: " + components);
}}
{code}
When I run it, I see {{Components: [Data.db, Index.db, Filter.db, Digest.crc32,
Statistics.db, TOC.txt, Summary.db, CompressionInfo.db]}}.
Reading {{discoverComponentsFor}}, my take is that
{{Descriptor#filenameFor(Component)}} doesn't properly build the index file
name (since it doesn't have the index name itself and
Component.Type.SECONDARY_INDEX.repr = "SI_*.db" which is not actually doing
glob matching).
> SSTableIndex should not use Ref#globalCount() to determine when to delete
> index file
>
>
> Key: CASSANDRA-14248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14248
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.11.x
>
>
> {{SSTableIndex}} instances maintain a {{Ref}} to the underlying
> {{SSTableReader}} instance. When determining whether or not to delete the
> file after the last {{SSTableIndex}} reference is released, the
> implementation uses {{sstableRef.globalCount()}}:
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/sasi/SSTableIndex.java#L135.]
> This is incorrect because {{sstableRef.globalCount()}} returns the number of
> references to the specific instance of {{SSTableReader}}. However, in cases
> like index summary redistribution, there can be more than one instance of
> {{SSTableReader}}. Further, since the reader is shared across multiple
> indexes, not all indexes see the count go to 0. This can lead to cases where
> the {{SSTableIndex}} file is incorrectly deleted or not deleted when it
> should be.
>
> A more correct implementation would be to either:
> * Tie into the existing {{SSTableTidier}}. SASI indexes already are SSTable
> components but are not cleaned up by the {{SSTableTidier}} because they are
> not found with the currently cleanup implementation
> * Revamp {{SSTableIndex}} reference counting to use {{Ref}} and implement a
> new tidier.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14248) SSTableIndex should not use Ref#globalCount() to determine when to delete index file
[
https://issues.apache.org/jira/browse/CASSANDRA-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099309#comment-17099309
]
Jonathan Ellis commented on CASSANDRA-14248:
discoverComponentsFor does find it.
Jira is breaking the link by including the close paren, clean link here:
https://github.com/jbellis/cassandra/tree/14248
> SSTableIndex should not use Ref#globalCount() to determine when to delete
> index file
>
>
> Key: CASSANDRA-14248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14248
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.11.x
>
>
> {{SSTableIndex}} instances maintain a {{Ref}} to the underlying
> {{SSTableReader}} instance. When determining whether or not to delete the
> file after the last {{SSTableIndex}} reference is released, the
> implementation uses {{sstableRef.globalCount()}}:
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/sasi/SSTableIndex.java#L135.]
> This is incorrect because {{sstableRef.globalCount()}} returns the number of
> references to the specific instance of {{SSTableReader}}. However, in cases
> like index summary redistribution, there can be more than one instance of
> {{SSTableReader}}. Further, since the reader is shared across multiple
> indexes, not all indexes see the count go to 0. This can lead to cases where
> the {{SSTableIndex}} file is incorrectly deleted or not deleted when it
> should be.
>
> A more correct implementation would be to either:
> * Tie into the existing {{SSTableTidier}}. SASI indexes already are SSTable
> components but are not cleaned up by the {{SSTableTidier}} because they are
> not found with the currently cleanup implementation
> * Revamp {{SSTableIndex}} reference counting to use {{Ref}} and implement a
> new tidier.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14248) SSTableIndex should not use Ref#globalCount() to determine when to delete index file
[
https://issues.apache.org/jira/browse/CASSANDRA-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099300#comment-17099300
]
Jordan West commented on CASSANDRA-14248:
-
[~jbellis] thanks for picking this up. I couldn't access the link you shared.
Looks like a typo. Its been a bit since I filed this so I need to page it back
in but quickly, I think the issue in {{SSTableTidier}} is
{{SSTable.discoverComponentsFor(...)}} does not actually find the index files
([https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L356)|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L356);].
Meanwhile, the check of whether or not `globalCount() == 0` won't always be
true when the index is released for the last time (e.g. when removing an
index).
It would be good to write some tests to validate my guesses above.
> SSTableIndex should not use Ref#globalCount() to determine when to delete
> index file
>
>
> Key: CASSANDRA-14248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14248
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.11.x
>
>
> {{SSTableIndex}} instances maintain a {{Ref}} to the underlying
> {{SSTableReader}} instance. When determining whether or not to delete the
> file after the last {{SSTableIndex}} reference is released, the
> implementation uses {{sstableRef.globalCount()}}:
> [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/index/sasi/SSTableIndex.java#L135.]
> This is incorrect because {{sstableRef.globalCount()}} returns the number of
> references to the specific instance of {{SSTableReader}}. However, in cases
> like index summary redistribution, there can be more than one instance of
> {{SSTableReader}}. Further, since the reader is shared across multiple
> indexes, not all indexes see the count go to 0. This can lead to cases where
> the {{SSTableIndex}} file is incorrectly deleted or not deleted when it
> should be.
>
> A more correct implementation would be to either:
> * Tie into the existing {{SSTableTidier}}. SASI indexes already are SSTable
> components but are not cleaned up by the {{SSTableTidier}} because they are
> not found with the currently cleanup implementation
> * Revamp {{SSTableIndex}} reference counting to use {{Ref}} and implement a
> new tidier.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15786) ChecksummingTransformerTest#corruptionCausesFailure fails for seed 43595190254702
[
https://issues.apache.org/jira/browse/CASSANDRA-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099282#comment-17099282
]
David Capwell commented on CASSANDRA-15786:
---
This is a duplicate of CASSANDRA-15313, which is expected to be resolved by
CASSANDRA-15299
> ChecksummingTransformerTest#corruptionCausesFailure fails for seed
> 43595190254702
> -
>
> Key: CASSANDRA-15786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15786
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> Reproducer:
> * latest trunk (e.g. df8e736700ae2a06675ff50381788d708bc22b96)
> * change 1st line in {{corruptionCausesFailure()}} to
> {{qt().withFixedSeed(43595190254702L)}}
> * run test
> Fails with:
> {code:java}
> java.lang.AssertionError: Property falsified after 617 example(s)
> Smallest found falsifying value(s) :-
>
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Source Control Link:
https://github.com/apache/cassandra/commit/e394dc0bb32f612a476269010930c617dd1ed3cb
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks!
Committed as
[e394dc0bb32f612a476269010930c617dd1ed3cb|https://github.com/apache/cassandra/commit/e394dc0bb32f612a476269010930c617dd1ed3cb]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Reviewers: Eduard Tudenhoefner, Robert Stupp (was: Eduard Tudenhoefner)
Status: Review In Progress (was: Patch Available)
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Status: Ready to Commit (was: Review In Progress)
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Less test stderr spam when running tests w/ java 11
This is an automated email from the ASF dual-hosted git repository.
snazy pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new e394dc0 Less test stderr spam when running tests w/ java 11
e394dc0 is described below
commit e394dc0bb32f612a476269010930c617dd1ed3cb
Author: Robert Stupp
AuthorDate: Wed Apr 1 17:08:55 2020 +0200
Less test stderr spam when running tests w/ java 11
patch by Robert Stupp; reviewed by Eduard Tudenhöfner for CASSANDRA-15780
---
src/java/org/apache/cassandra/utils/FBUtilities.java| 5 +
.../org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java | 5 +
test/unit/org/apache/cassandra/tools/ToolsTester.java | 6 ++
3 files changed, 16 insertions(+)
diff --git a/src/java/org/apache/cassandra/utils/FBUtilities.java
b/src/java/org/apache/cassandra/utils/FBUtilities.java
index 115cd43..9b39f2d 100644
--- a/src/java/org/apache/cassandra/utils/FBUtilities.java
+++ b/src/java/org/apache/cassandra/utils/FBUtilities.java
@@ -72,6 +72,11 @@ import org.apache.cassandra.net.AsyncOneResponse;
public class FBUtilities
{
+static
+{
+preventIllegalAccessWarnings();
+}
+
private static final Logger logger =
LoggerFactory.getLogger(FBUtilities.class);
private static final ObjectMapper jsonMapper = new ObjectMapper(new
JsonFactory());
diff --git
a/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
b/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
index 7c1a0da..c5c3b60 100644
--- a/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
+++ b/test/unit/org/apache/cassandra/io/util/BufferedDataOutputStreamTest.java
@@ -44,10 +44,15 @@ import com.google.common.primitives.UnsignedBytes;
import com.google.common.primitives.UnsignedInteger;
import com.google.common.primitives.UnsignedLong;
+import static
org.apache.cassandra.utils.FBUtilities.preventIllegalAccessWarnings;
import static org.junit.Assert.*;
public class BufferedDataOutputStreamTest
{
+static
+{
+preventIllegalAccessWarnings();
+}
@Test(expected = BufferOverflowException.class)
public void testDataOutputBufferFixedByes() throws Exception
diff --git a/test/unit/org/apache/cassandra/tools/ToolsTester.java
b/test/unit/org/apache/cassandra/tools/ToolsTester.java
index 391c9b9..0bb9beb 100644
--- a/test/unit/org/apache/cassandra/tools/ToolsTester.java
+++ b/test/unit/org/apache/cassandra/tools/ToolsTester.java
@@ -39,6 +39,7 @@ import org.junit.BeforeClass;
import org.slf4j.LoggerFactory;
+import static
org.apache.cassandra.utils.FBUtilities.preventIllegalAccessWarnings;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
@@ -49,6 +50,11 @@ import static org.junit.Assert.fail;
*/
public abstract class ToolsTester
{
+static
+{
+preventIllegalAccessWarnings();
+}
+
private static List initialThreads;
static final String[] EXPECTED_THREADS_WITH_SCHEMA = {
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099251#comment-17099251
]
Robert Stupp commented on CASSANDRA-15780:
--
[https://app.circleci.com/pipelines/github/snazy/cassandra/10/workflows/fa15ded3-6428-426c-be0a-ce0fb87b9d5e]
[https://app.circleci.com/pipelines/github/snazy/cassandra/10/workflows/60eb74ea-d7da-4567-b33d-62da21c78feb]
Those look "as good as trunk".
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[ https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-14781: Since Version: 4.0-beta Source Control Link: https://github.com/apache/cassandra/commit/d3dadcd6f3bbde471e972f8332eb62de0f2d4aae Resolution: Fixed Status: Resolved (was: Ready to Commit) > Log message when mutation passed to CommitLog#add(Mutation) is too large is > not descriptive enough > -- > > Key: CASSANDRA-14781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14781 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Hints, Local/Commit Log, Messaging/Client >Reporter: Jordan West >Assignee: Venkata Harikrishna Nukala >Priority: Normal > Labels: protocolv5 > Fix For: 4.0-beta > > Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch, > CASSANDRA-14781_3.11.patch > > > When hitting > [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257], > the log message produced does not help the operator track down what data is > being written. At a minimum the keyspace and cfIds involved would be useful > (and are available) – more detail might not be reasonable to include. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[ https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-14781: Status: Ready to Commit (was: Changes Suggested) > Log message when mutation passed to CommitLog#add(Mutation) is too large is > not descriptive enough > -- > > Key: CASSANDRA-14781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14781 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Hints, Local/Commit Log, Messaging/Client >Reporter: Jordan West >Assignee: Venkata Harikrishna Nukala >Priority: Normal > Labels: protocolv5 > Fix For: 4.0-beta > > Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch, > CASSANDRA-14781_3.11.patch > > > When hitting > [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257], > the log message produced does not help the operator track down what data is > being written. At a minimum the keyspace and cfIds involved would be useful > (and are available) – more detail might not be reasonable to include. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[ https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099222#comment-17099222 ] Jordan West commented on CASSANDRA-14781: - Tests looked good. Any failures are suspected flaky tests and are unrelated. Committed as d3dadcd6f3bbde471e972f8332eb62de0f2d4aae. > Log message when mutation passed to CommitLog#add(Mutation) is too large is > not descriptive enough > -- > > Key: CASSANDRA-14781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14781 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Hints, Local/Commit Log, Messaging/Client >Reporter: Jordan West >Assignee: Venkata Harikrishna Nukala >Priority: Normal > Labels: protocolv5 > Fix For: 4.0-beta > > Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch, > CASSANDRA-14781_3.11.patch > > > When hitting > [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257], > the log message produced does not help the operator track down what data is > being written. At a minimum the keyspace and cfIds involved would be useful > (and are available) – more detail might not be reasonable to include. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Improve logging when mutation passed to commit log is too large
This is an automated email from the ASF dual-hosted git repository.
jwest pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new d3dadcd Improve logging when mutation passed to commit log is too
large
d3dadcd is described below
commit d3dadcd6f3bbde471e972f8332eb62de0f2d4aae
Author: nvharikrishna
AuthorDate: Wed Jan 8 22:06:06 2020 +0530
Improve logging when mutation passed to commit log is too large
Patch by Venkata Harikrishna Nukala; reviewed by Jordan West for
CASSANDRA-14781
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/batchlog/Batch.java | 4 +-
.../org/apache/cassandra/db/CounterMutation.java | 39 +-
src/java/org/apache/cassandra/db/IMutation.java| 13
src/java/org/apache/cassandra/db/Mutation.java | 36 +
.../db/MutationExceededMaxSizeException.java | 89 ++
.../apache/cassandra/db/commitlog/CommitLog.java | 14 +---
.../cassandra/db/virtual/VirtualMutation.java | 5 ++
src/java/org/apache/cassandra/hints/Hint.java | 2 +-
.../apache/cassandra/schema/MigrationManager.java | 2 +-
.../service/reads/repair/BlockingReadRepairs.java | 61 +++
.../cassandra/test/microbench/MutationBench.java | 2 +-
.../db/MutationExceededMaxSizeExceptionTest.java | 46 +++
.../cassandra/db/commitlog/CommitLogTest.java | 55 +++--
14 files changed, 316 insertions(+), 53 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 89c8d7d..8c4cf35 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha5
+ * Improve logging when mutation passed to commit log is too large
(CASSANDRA-14781)
* replace LZ4FastDecompressor with LZ4SafeDecompressor (CASSANDRA-15560)
* Fix buffer pool NPE with concurrent release due to in-progress tiny pool
eviction (CASSANDRA-15726)
* Avoid race condition when completing stream sessions (CASSANDRA-15666)
diff --git a/src/java/org/apache/cassandra/batchlog/Batch.java
b/src/java/org/apache/cassandra/batchlog/Batch.java
index e91e3ca..fb6c5d5 100644
--- a/src/java/org/apache/cassandra/batchlog/Batch.java
+++ b/src/java/org/apache/cassandra/batchlog/Batch.java
@@ -90,7 +90,7 @@ public final class Batch
size += sizeofUnsignedVInt(batch.decodedMutations.size());
for (Mutation mutation : batch.decodedMutations)
{
-int mutationSize = (int)
Mutation.serializer.serializedSize(mutation, version);
+int mutationSize = mutation.serializedSize(version);
size += sizeofUnsignedVInt(mutationSize);
size += mutationSize;
}
@@ -108,7 +108,7 @@ public final class Batch
out.writeUnsignedVInt(batch.decodedMutations.size());
for (Mutation mutation : batch.decodedMutations)
{
-
out.writeUnsignedVInt(Mutation.serializer.serializedSize(mutation, version));
+out.writeUnsignedVInt(mutation.serializedSize(version));
Mutation.serializer.serialize(mutation, out, version);
}
}
diff --git a/src/java/org/apache/cassandra/db/CounterMutation.java
b/src/java/org/apache/cassandra/db/CounterMutation.java
index bb10a6a..722ad73 100644
--- a/src/java/org/apache/cassandra/db/CounterMutation.java
+++ b/src/java/org/apache/cassandra/db/CounterMutation.java
@@ -45,6 +45,9 @@ import org.apache.cassandra.utils.*;
import org.apache.cassandra.utils.btree.BTreeSet;
import static java.util.concurrent.TimeUnit.*;
+import static org.apache.cassandra.net.MessagingService.VERSION_30;
+import static org.apache.cassandra.net.MessagingService.VERSION_3014;
+import static org.apache.cassandra.net.MessagingService.VERSION_40;
public class CounterMutation implements IMutation
{
@@ -76,6 +79,15 @@ public class CounterMutation implements IMutation
return mutation.getPartitionUpdates();
}
+public void validateSize(int version, int overhead)
+{
+long totalSize = serializedSize(version) + overhead;
+if(totalSize > MAX_MUTATION_SIZE)
+{
+throw new MutationExceededMaxSizeException(this, version,
totalSize);
+}
+}
+
public Mutation getMutation()
{
return mutation;
@@ -308,6 +320,31 @@ public class CounterMutation implements IMutation
return DatabaseDescriptor.getCounterWriteRpcTimeout(unit);
}
+private int serializedSize30;
+private int serializedSize3014;
+private int serializedSize40;
+
+public int serializedSize(int version)
+{
+switch (version)
+{
+case VERSION_30:
+if (serializedSize30 == 0)
+serializedSize30 = (int) serializer.serializedSize(this,
VERSION_30);
+return serializedSize30;
+
[jira] [Commented] (CASSANDRA-15782) Compression test failure
[ https://issues.apache.org/jira/browse/CASSANDRA-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099214#comment-17099214 ] Joey Lynch commented on CASSANDRA-15782: |[patch|https://github.com/apache/cassandra-dtest/commit/a2fe94ba8004a2b7159e58281a8125411895604e]| |[dtest branch|https://github.com/apache/cassandra-dtest/compare/master...jolynch:fix_compression_dtest]| |[cass branch|https://github.com/apache/cassandra/compare/trunk...jolynch:CASSANDRA-15782]| Tests running, will update with result. > Compression test failure > > > Key: CASSANDRA-15782 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15782 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Joey Lynch >Priority: Normal > Fix For: 4.0, 4.0-beta > > > On CASSANDRA-15560 compression test failed. This was bisected to > [9c1bbf3|https://github.com/apache/cassandra/commit/9c1bbf3ac913f9bdf7a0e0922106804af42d2c1e] > from CASSANDRA-15379. > Full details here > CC/ [~jolynch] in case he can spot it quick. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15786) ChecksummingTransformerTest#corruptionCausesFailure fails for seed 43595190254702
[
https://issues.apache.org/jira/browse/CASSANDRA-15786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15786:
-
Fix Version/s: 4.0
> ChecksummingTransformerTest#corruptionCausesFailure fails for seed
> 43595190254702
> -
>
> Key: CASSANDRA-15786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15786
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> Reproducer:
> * latest trunk (e.g. df8e736700ae2a06675ff50381788d708bc22b96)
> * change 1st line in {{corruptionCausesFailure()}} to {{
> qt().withFixedSeed(43595190254702L)}}
> * run test
> Fails with:
> {code}
> java.lang.AssertionError: Property falsified after 617 example(s)
> Smallest found falsifying value(s) :-
>
[jira] [Created] (CASSANDRA-15786) ChecksummingTransformerTest#corruptionCausesFailure fails for seed 43595190254702
Robert Stupp created CASSANDRA-15786:
Summary: ChecksummingTransformerTest#corruptionCausesFailure fails
for seed 43595190254702
Key: CASSANDRA-15786
URL: https://issues.apache.org/jira/browse/CASSANDRA-15786
Project: Cassandra
Issue Type: Bug
Reporter: Robert Stupp
Reproducer:
* latest trunk (e.g. df8e736700ae2a06675ff50381788d708bc22b96)
* change 1st line in {{corruptionCausesFailure()}} to {{
qt().withFixedSeed(43595190254702L)}}
* run test
Fails with:
{code}
java.lang.AssertionError: Property falsified after 617 example(s)
Smallest found falsifying value(s) :-
[jira] [Created] (CASSANDRA-15785) 3.11.6 Image Vulnerabilities
Gil Tohar created CASSANDRA-15785: - Summary: 3.11.6 Image Vulnerabilities Key: CASSANDRA-15785 URL: https://issues.apache.org/jira/browse/CASSANDRA-15785 Project: Cassandra Issue Type: Bug Components: Build Reporter: Gil Tohar Attachments: Screen Shot 2020-05-04 at 1.44.19 PM.png My team has taken the Bitnami Cassandra image, 3.11.6, and scanned it using our image vulnerability scanner, and discovered 4 issues. These are CVE-2019-5436, CVE-2019-5481, CVE-2019-5482, CVE-2020-1967. The solutions entail updating 1) curl and 2) openssl in the image, as described in the attached image. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15784) test_refresh_size_estimates_clears_invalid_entries - nodetool_test.TestNodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15784: Resolution: Fixed Status: Resolved (was: Open) > test_refresh_size_estimates_clears_invalid_entries - > nodetool_test.TestNodetool > --- > > Key: CASSANDRA-15784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15784 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > Dtest failure -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15784) test_refresh_size_estimates_clears_invalid_entries - nodetool_test.TestNodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099181#comment-17099181 ] Ekaterina Dimitrova commented on CASSANDRA-15784: - [~dcapwell] Oh, great, thanks! My search didn't show anything. Closing this one > test_refresh_size_estimates_clears_invalid_entries - > nodetool_test.TestNodetool > --- > > Key: CASSANDRA-15784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15784 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > Dtest failure -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15782) Compression test failure
[
https://issues.apache.org/jira/browse/CASSANDRA-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099172#comment-17099172
]
Joey Lynch commented on CASSANDRA-15782:
[~Bereng] If you are referring to the test_compression_cql_options -
compression_test.TestCompression test failing
https://app.circleci.com/pipelines/github/driftx/cassandra/27/workflows/896fefed-5707-4da0-882d-ca5a29abab58/jobs/177
with the following stack trace:
{noformat}
for sstable_path in sstable_paths:
sstable = os.path.join(sstable_path,
sstables['compression_opts_table'][1])
if os.path.exists(sstable):
> assert 'DEFLATE' == self._get_compression_type(sstable)
E AssertionError: assert 'DEFLATE' == 'UNKNOWN'
E - DEFLATE
E + UNKNOWN
{noformat}
Yeah that looks like my patch, I thought I ran the dtests but guess I missed
the cqlsh dtests (oops). This is an expected change in behavior (deflate should
flush with lz4 now and then compact to deflate). I think an easy fix to the
test would be to compact after the flush. I can fix it.
> Compression test failure
>
>
> Key: CASSANDRA-15782
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15782
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> On CASSANDRA-15560 compression test failed. This was bisected to
> [9c1bbf3|https://github.com/apache/cassandra/commit/9c1bbf3ac913f9bdf7a0e0922106804af42d2c1e]
> from CASSANDRA-15379.
> Full details here
> CC/ [~jolynch] in case he can spot it quick.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15782) Compression test failure
[ https://issues.apache.org/jira/browse/CASSANDRA-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joey Lynch reassigned CASSANDRA-15782: -- Assignee: Joey Lynch > Compression test failure > > > Key: CASSANDRA-15782 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15782 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Joey Lynch >Priority: Normal > Fix For: 4.0, 4.0-beta > > > On CASSANDRA-15560 compression test failed. This was bisected to > [9c1bbf3|https://github.com/apache/cassandra/commit/9c1bbf3ac913f9bdf7a0e0922106804af42d2c1e] > from CASSANDRA-15379. > Full details here > CC/ [~jolynch] in case he can spot it quick. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14801) calculatePendingRanges no longer safe for multiple adjacent range movements
[
https://issues.apache.org/jira/browse/CASSANDRA-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Eggleston updated CASSANDRA-14801:
Reviewers: Blake Eggleston (was: Francisco Fernandez)
> calculatePendingRanges no longer safe for multiple adjacent range movements
> ---
>
> Key: CASSANDRA-14801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14801
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination, Legacy/Distributed Metadata
>Reporter: Benedict Elliott Smith
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Correctness depended upon the narrowing to a {{Set}},
> which we no longer do - we maintain a collection of all {{Replica}}. Our
> {{RangesAtEndpoint}} collection built by {{getPendingRanges}} can as a result
> contain the same endpoint multiple times; and our {{EndpointsForToken}}
> obtained by {{TokenMetadata.pendingEndpointsFor}} may fail to be constructed,
> resulting in cluster-wide failures for writes to the affected token ranges
> for the duration of the range movement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15784) test_refresh_size_estimates_clears_invalid_entries - nodetool_test.TestNodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099145#comment-17099145 ] David Capwell commented on CASSANDRA-15784: --- This is a dup of CASSANDRA-15776, need a reviewer. test modifies size_estiates but nodetool started looking at table_estimates, so switched it so nodetool checks both. > test_refresh_size_estimates_clears_invalid_entries - > nodetool_test.TestNodetool > --- > > Key: CASSANDRA-15784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15784 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > Dtest failure -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15502) In Tree Tooling with Java 11
[ https://issues.apache.org/jira/browse/CASSANDRA-15502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099141#comment-17099141 ] David Capwell commented on CASSANDRA-15502: --- nodetool doesn't have unit tests, but some of the standalone tools do; dtest is currently the test suite for nodetool. Most of the tools don't look to really be tested on java 11, and I keep deprioritizing this so not really any progress here > In Tree Tooling with Java 11 > > > Key: CASSANDRA-15502 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15502 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/fuzz, Tool/bulk load, Tool/cqlsh, > Tool/diff, Tool/fql, Tool/nodetool, Tool/sstable, Tool/stress >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > This is to cover testing the various tools running on java 11. > The scope of this testing is manual testing and not automated, different > JIRAs should cover automation testing these tool. > The tools in question are: nodetool, sstableloader, sstablescrub, > sstableupgrade, sstableutil, sstableverify, fqltool, stress, auditlogviewer, > compaction-stress, sstabledump, sstableexpiredblockers, sstablemetadata, > sstableofflinerelevel, sstablesplit, and sstablerepairedset (many of these > may be tested already in dtest) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13606) Improve handling of 2i initialization failures
[ https://issues.apache.org/jira/browse/CASSANDRA-13606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099127#comment-17099127 ] Andres de la Peña commented on CASSANDRA-13606: --- If I'm understanding the patch, if we create a regular index on a table any write to that base table (including compaction) will fail until the index is completely build. I'm not sure we want this very restrictive behaviour, especially when the index build hasn't even failed. I guess we should have a different behaviour for unfinished and failed index builds, WDYT? I'm not sure about what making the index not available for writes should mean. The patch is making to fail even compactions of the base table, which might be excessive. Perhaps we should just ignore writes to a problematic index, without making the base table operation to fail? > Improve handling of 2i initialization failures > -- > > Key: CASSANDRA-13606 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13606 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Sergio Bossa >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0 > > Time Spent: 40m > Remaining Estimate: 0h > > CASSANDRA-10130 fixes the 2i build management, but initialization failures > are still not properly handled, most notably because: > * Initialization failures make the index non-queryable, but it can still be > written to. > * Initialization failures can be recovered via full rebuilds. > Both points above are probably suboptimal because the initialization logic > could be more complex than just an index build, hence it shouldn't be made > recoverable via a simple rebuild, and could cause the index to be fully > unavailable not just for reads, but for writes as well. > So, we should better handle initialization failures by: > * Allowing the index implementation to specify if unavailable for reads, > writes, or both. > * Providing a proper method to recover, distinct from index rebuilds. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[
https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096595#comment-17096595
]
Jordan West edited comment on CASSANDRA-14781 at 5/4/20, 4:34 PM:
--
Hi [~n.v.harikrishna]. I've picked this back up and am getting it ready to
commit. Thanks for your patience.
I've squashed your branch here:
[https://github.com/jrwest/cassandra/commits/14781-trunk.] I made a few minor
changes along the way (I also re-reviewed since it had been a little bit since
I had read the patch):
* Modified {{CHANGES.txt}}
* Modified {{IMutation#validateSize}} javadoc
* Moved call to {{Keyspace.open}} into the catch block of
{{BlockingReadRepairs#createRepairMutation}}. It was only used if we reached
that block anyways.
* Fixed whitespace formatting in
{{MutationExceededMaxSizeException#prepareMessage}}
I ran a build prior to these changes. The build looked good (better than trunk
actually) and any failures do not seem related:
[https://app.circleci.com/pipelines/github/jrwest/cassandra/4/workflows/e43918eb-40d2-45ad-80c3-dbeaa5ee186b]
I've kicked off a new build with the changes above and with the squash
performed:
[https://app.circleci.com/pipelines/github/jrwest/cassandra/6/workflows/3c3f674e-db89-488a-bbd5-98f04de4fd0d]
EDIT:
I was slightly concerned about the failure in {{read_repair_test.py}}'s
{{test_speculative_data_request}}. Looking closer at the test runs, its flaky
and doesn't look like that flakiness could be related to the changes here
(since the mutation sizes are static).
I've also kicked off a Jenkins build for good measure:
[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/107/]
was (Author: jrwest):
Hi [~n.v.harikrishna]. I've picked this back up and am getting it ready to
commit. Thanks for your patience.
I've squashed your branch here:
[https://github.com/jrwest/cassandra/commits/14781-trunk.] I made a few minor
changes along the way (I also re-reviewed since it had been a little bit since
I had read the patch):
* Modified {{CHANGES.txt}}
* Modified {{IMutation#validateSize}} javadoc
* Moved call to {{Keyspace.open}} into the catch block of
{{BlockingReadRepairs#createRepairMutation}}. It was only used if we reached
that block anyways.
* Fixed whitespace formatting in
{{MutationExceededMaxSizeException#prepareMessage}}
I ran a build prior to these changes. The build looked good (better than trunk
actually) and any failures do not seem related:
[https://app.circleci.com/pipelines/github/jrwest/cassandra/4/workflows/e43918eb-40d2-45ad-80c3-dbeaa5ee186b]
I've kicked off a new build with the changes above and with the squash
performed:
[https://app.circleci.com/pipelines/github/jrwest/cassandra/6/workflows/3c3f674e-db89-488a-bbd5-98f04de4fd0d]
EDIT:
I was slightly concerned about the failure in {{read_repair_test.py}}'s
{{test_speculative_data_request}}. Looking closer at the test runs, its flaky
and doesn't look like that flakiness could be related to the changes here
(since the mutation sizes are static).
I've also kicked off a Jenkins build for good measure:
https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/81/
> Log message when mutation passed to CommitLog#add(Mutation) is too large is
> not descriptive enough
> --
>
> Key: CASSANDRA-14781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14781
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints, Local/Commit Log, Messaging/Client
>Reporter: Jordan West
>Assignee: Venkata Harikrishna Nukala
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-beta
>
> Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch,
> CASSANDRA-14781_3.11.patch
>
>
> When hitting
> [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257],
> the log message produced does not help the operator track down what data is
> being written. At a minimum the keyspace and cfIds involved would be useful
> (and are available) – more detail might not be reasonable to include.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15503) Slow query log indicates opposite LTE when GTE operator
[
https://issues.apache.org/jira/browse/CASSANDRA-15503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-15503:
Reviewers: Berenguer Blasi
> Slow query log indicates opposite LTE when GTE operator
> ---
>
> Key: CASSANDRA-15503
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15503
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Wallace Baggaley
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 3.11.7, 4.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Slow query log is indicating a '<=' when a ">=" operator was sent. This
> appears to be a logging only issue, but it threw off development for a day
> figuring this out. Please fix.
> How to reproduce. Set slow query log timeout to 1 millisecond.
> In cqlsh run
> {noformat}
> CREATE TABLE atable (
> .id text,
> timestamp timestamp,
> PRIMARY KEY ((id), timestamp)
> ) WITH CLUSTERING ORDER BY (timestamp DESC);
> insert into atable (id, timestamp) VALUES ( '1',1);
> insert into atable (id, timestamp) VALUES ( '2',2);
> insert into atable (id, timestamp) VALUES ( '3',3);
> insert into atable (id, timestamp) VALUES ( '4',4);
> insert into atable (id, timestamp) VALUES ( '5',5);
> insert into atable (id, timestamp) VALUES ( '6',6);
> insert into atable (id, timestamp) VALUES ( '7',7);
> insert into atable (id, timestamp) VALUES ( '8',8);
> insert into atable (id, timestamp) VALUES ( '9',9);
> insert into atable (id, timestamp) VALUES ( '10',10);
> insert into atable (id, timestamp) VALUES ( '11',11);
> select * from atable where timestamp >= '1970-01-01 00:00:00.006+' allow
> filtering;
> {noformat}
> In the logs it prints:
> {noformat}
> DEBUG 1 operations were slow in the last 5003 msecs:
> ,
> time 7 msec - slow timeout 1 msec
> {noformat}
> But the query works appropriately and returns
> {noformat}
> id | timestamp
> +-
> 6 | 1970-01-01 00:00:00.006000+
> 7 | 1970-01-01 00:00:00.007000+
> 9 | 1970-01-01 00:00:00.009000+
> 10 | 1970-01-01 00:00:00.01+
> 8 | 1970-01-01 00:00:00.008000+
> 11 | 1970-01-01 00:00:00.011000+
> (6 rows)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Status: Ready to Commit (was: Review In Progress)
Thanks!
Committed as
[df8e736700ae2a06675ff50381788d708bc22b96|https://github.com/apache/cassandra/commit/df8e736700ae2a06675ff50381788d708bc22b96]
to [trunk|https://github.com/apache/cassandra/tree/trunk].
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Source Control Link:
https://github.com/apache/cassandra/commit/df8e736700ae2a06675ff50381788d708bc22b96
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Reviewers: Eduard Tudenhoefner, Robert Stupp (was: Eduard Tudenhoefner)
Status: Review In Progress (was: Patch Available)
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Fix flakiness of org.apache.cassandra.metrics.DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
This is an automated email from the ASF dual-hosted git repository.
snazy pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new df8e736 Fix flakiness of
org.apache.cassandra.metrics.DecayingEstimatedHistogramReservoirTest#testStriping
and apply the same "fix" to other tests
df8e736 is described below
commit df8e736700ae2a06675ff50381788d708bc22b96
Author: Robert Stupp
AuthorDate: Sun May 3 18:22:46 2020 +0200
Fix flakiness of
org.apache.cassandra.metrics.DecayingEstimatedHistogramReservoirTest#testStriping
and apply the same "fix" to other tests
Patch by Robert Stupp; reviewed by Eduard Tudenhöfner for CASSANDRA-15781
---
test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java | 2 +-
.../org/apache/cassandra/db/commitlog/CommitLogUpgradeTestMaker.java | 3 ++-
.../org/apache/cassandra/db/compaction/CompactionExecutorTest.java | 3 ++-
test/unit/org/apache/cassandra/db/monitoring/MonitoringTaskTest.java | 2 +-
test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java | 2 +-
test/unit/org/apache/cassandra/io/util/RandomAccessReaderTest.java | 3 ++-
.../cassandra/metrics/DecayingEstimatedHistogramReservoirTest.java | 3 ++-
test/unit/org/apache/cassandra/net/ConnectionTest.java | 2 +-
test/unit/org/apache/cassandra/utils/StatusLoggerTest.java | 3 ++-
test/unit/org/apache/cassandra/utils/UUIDTests.java| 3 ++-
10 files changed, 16 insertions(+), 10 deletions(-)
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
b/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
index 7f8bcba..71c1d62 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
@@ -1364,7 +1364,7 @@ public class JsonTest extends CQLTester
future.get(30, TimeUnit.SECONDS);
executor.shutdown();
-Assert.assertTrue(executor.awaitTermination(30, TimeUnit.SECONDS));
+Assert.assertTrue(executor.awaitTermination(1, TimeUnit.MINUTES));
}
@Test
diff --git
a/test/unit/org/apache/cassandra/db/commitlog/CommitLogUpgradeTestMaker.java
b/test/unit/org/apache/cassandra/db/commitlog/CommitLogUpgradeTestMaker.java
index d2ad42f..680a0e7 100644
--- a/test/unit/org/apache/cassandra/db/commitlog/CommitLogUpgradeTestMaker.java
+++ b/test/unit/org/apache/cassandra/db/commitlog/CommitLogUpgradeTestMaker.java
@@ -33,6 +33,7 @@ import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import com.google.common.util.concurrent.RateLimiter;
+import org.junit.Assert;
import org.apache.cassandra.SchemaLoader;
import org.apache.cassandra.Util;
@@ -110,7 +111,7 @@ public class CommitLogUpgradeTestMaker
Thread.sleep(runTimeMs);
stop = true;
scheduled.shutdown();
-scheduled.awaitTermination(2, TimeUnit.SECONDS);
+Assert.assertTrue(scheduled.awaitTermination(1, TimeUnit.MINUTES));
int hash = 0;
int cells = 0;
diff --git
a/test/unit/org/apache/cassandra/db/compaction/CompactionExecutorTest.java
b/test/unit/org/apache/cassandra/db/compaction/CompactionExecutorTest.java
index 2f8b5b2..ab3d9e5 100644
--- a/test/unit/org/apache/cassandra/db/compaction/CompactionExecutorTest.java
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionExecutorTest.java
@@ -22,6 +22,7 @@ import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import org.junit.After;
+import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor;
@@ -62,7 +63,7 @@ public class CompactionExecutorTest
public void destroy() throws Exception
{
executor.shutdown();
-executor.awaitTermination(1, TimeUnit.MINUTES);
+Assert.assertTrue(executor.awaitTermination(1, TimeUnit.MINUTES));
}
@Test
diff --git
a/test/unit/org/apache/cassandra/db/monitoring/MonitoringTaskTest.java
b/test/unit/org/apache/cassandra/db/monitoring/MonitoringTaskTest.java
index 454d0b4..dc8c317 100644
--- a/test/unit/org/apache/cassandra/db/monitoring/MonitoringTaskTest.java
+++ b/test/unit/org/apache/cassandra/db/monitoring/MonitoringTaskTest.java
@@ -276,7 +276,7 @@ public class MonitoringTaskTest
}
executorService.shutdown();
-assertTrue(executorService.awaitTermination(30, TimeUnit.SECONDS));
+assertTrue(executorService.awaitTermination(1, TimeUnit.MINUTES));
assertEquals(opCount, operations.size());
waitForOperationsToComplete(operations);
diff --git a/test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java
b/test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java
index c64bec9..3508d76 100644
[jira] [Assigned] (CASSANDRA-15783) test_optimized_primary_range_repair - transient_replication_test.TestTransientReplication
[ https://issues.apache.org/jira/browse/CASSANDRA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova reassigned CASSANDRA-15783: --- Assignee: Ekaterina Dimitrova > test_optimized_primary_range_repair - > transient_replication_test.TestTransientReplication > - > > Key: CASSANDRA-15783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15783 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > Dtest failure. > Example: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517/jobs/585/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15783) test_optimized_primary_range_repair - transient_replication_test.TestTransientReplication
[ https://issues.apache.org/jira/browse/CASSANDRA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15783: Fix Version/s: 4.0-beta 4.0 > test_optimized_primary_range_repair - > transient_replication_test.TestTransientReplication > - > > Key: CASSANDRA-15783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15783 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > Dtest failure. > Example: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517/jobs/585/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[
https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098960#comment-17098960
]
Berenguer Blasi commented on CASSANDRA-15766:
-
+1. Either a supplier approach or a 'shouldLog()' check after getting a
{{NoSpamLogStatement}} where the 2 options I was playing with. I'll be happy to
help with the review. No hurries.
> NoSpamLogger arguments building objects on hot paths
>
>
> Key: CASSANDRA-15766
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15766
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-rc
>
>
> NoSpamLogger is used in hot logging paths to prevent logs being overrun. For
> that to be most effective the arguments to the logger need to be cheap to
> construct. During the internode messaging refactor CASSANDRA-15066,
> performance changes to BufferPool for CASSANDRA-14416
> were accidentally reverted in the merge up from 3.11.
> Reviewing other uses since, it looks like there are a few places where the
> arguments require some form of String building.
> org.apache.cassandra.net.InboundSink#accept
> org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame
> org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize
> org.apache.cassandra.net.OutboundConnection#onOverloaded
> org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks
> Formatting arguments should either be precomputed, or if expensive they
> should be computed after the decision on whether to noSpamLog has been made.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15158) Wait for schema agreement rather then in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098952#comment-17098952 ] Stefan Miklosovic commented on CASSANDRA-15158: --- code for 3.0 [https://github.com/smiklosovic/cassandra/tree/CASSANDRA-15158-cassandra-3] > Wait for schema agreement rather then in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Schema >Reporter: Vincent White >Assignee: Ben Bromhead >Priority: Normal > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15784) test_refresh_size_estimates_clears_invalid_entries - nodetool_test.TestNodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15784: Fix Version/s: 4.0-beta 4.0 > test_refresh_size_estimates_clears_invalid_entries - > nodetool_test.TestNodetool > --- > > Key: CASSANDRA-15784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15784 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > Dtest failure -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15784) test_refresh_size_estimates_clears_invalid_entries - nodetool_test.TestNodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-15784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15784: Bug Category: Parent values: Correctness(12982) Complexity: Normal Component/s: CI Discovered By: User Report Severity: Normal Status: Open (was: Triage Needed) test_refresh_size_estimates_clears_invalid_entries - nodetool_test.TestNodetool Example failure: https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517/jobs/585/tests > test_refresh_size_estimates_clears_invalid_entries - > nodetool_test.TestNodetool > --- > > Key: CASSANDRA-15784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15784 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Priority: Normal > > Dtest failure -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15784) test_refresh_size_estimates_clears_invalid_entries - nodetool_test.TestNodetool
Ekaterina Dimitrova created CASSANDRA-15784: --- Summary: test_refresh_size_estimates_clears_invalid_entries - nodetool_test.TestNodetool Key: CASSANDRA-15784 URL: https://issues.apache.org/jira/browse/CASSANDRA-15784 Project: Cassandra Issue Type: Bug Reporter: Ekaterina Dimitrova Dtest failure -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15783) test_optimized_primary_range_repair - transient_replication_test.TestTransientReplication
[ https://issues.apache.org/jira/browse/CASSANDRA-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15783: Bug Category: Parent values: Correctness(12982)Level 1 values: Test Failure(12990) Complexity: Normal Component/s: Test/dtest Discovered By: User Report Severity: Normal Status: Open (was: Triage Needed) > test_optimized_primary_range_repair - > transient_replication_test.TestTransientReplication > - > > Key: CASSANDRA-15783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15783 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Priority: Normal > > Dtest failure. > Example: > https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517/jobs/585/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15783) test_optimized_primary_range_repair - transient_replication_test.TestTransientReplication
Ekaterina Dimitrova created CASSANDRA-15783: --- Summary: test_optimized_primary_range_repair - transient_replication_test.TestTransientReplication Key: CASSANDRA-15783 URL: https://issues.apache.org/jira/browse/CASSANDRA-15783 Project: Cassandra Issue Type: Bug Reporter: Ekaterina Dimitrova Dtest failure. Example: https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517/jobs/585/tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15782) Compression test failure
[ https://issues.apache.org/jira/browse/CASSANDRA-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15782: Complexity: Normal Fix Version/s: 4.0-beta Severity: Normal Status: Open (was: Triage Needed) > Compression test failure > > > Key: CASSANDRA-15782 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15782 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Priority: Normal > Fix For: 4.0, 4.0-beta > > > On CASSANDRA-15560 compression test failed. This was bisected to > [9c1bbf3|https://github.com/apache/cassandra/commit/9c1bbf3ac913f9bdf7a0e0922106804af42d2c1e] > from CASSANDRA-15379. > Full details here > CC/ [~jolynch] in case he can spot it quick. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[
https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098933#comment-17098933
]
Jon Meredith commented on CASSANDRA-15766:
--
I have a small prototype I was working on when I reported the issue that I
should clean up. I wanted to get some feedback on it before posting the patch,
but it probably doesn't need to wait. I'll try and push it up this morning.
The idea was too check the list of objects passed as arguments to check if they
were {{Supplier}}s and automatically call get on them if they were. The
downside would be anything that was already a supplier might need to be
double-wrapped, but I think that's worth it. I also wanted to work out what the
cost of wrapping things like
{{org.apache.cassandra.net.InboundMessageHandler#id()}} in a {{Supplier}}
versus just the call.
> NoSpamLogger arguments building objects on hot paths
>
>
> Key: CASSANDRA-15766
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15766
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-rc
>
>
> NoSpamLogger is used in hot logging paths to prevent logs being overrun. For
> that to be most effective the arguments to the logger need to be cheap to
> construct. During the internode messaging refactor CASSANDRA-15066,
> performance changes to BufferPool for CASSANDRA-14416
> were accidentally reverted in the merge up from 3.11.
> Reviewing other uses since, it looks like there are a few places where the
> arguments require some form of String building.
> org.apache.cassandra.net.InboundSink#accept
> org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame
> org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize
> org.apache.cassandra.net.OutboundConnection#onOverloaded
> org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks
> Formatting arguments should either be precomputed, or if expensive they
> should be computed after the decision on whether to noSpamLog has been made.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[
https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098933#comment-17098933
]
Jon Meredith edited comment on CASSANDRA-15766 at 5/4/20, 1:36 PM:
---
I have a small prototype I was working on when I reported the issue that I
should clean up. I wanted to get some feedback on it before posting the patch,
but it probably doesn't need to wait. I'll try and push it up this morning.
The idea was too check the list of objects passed as arguments to check if they
were {{Supplier}} s and automatically call get on them if they were. The
downside would be anything that was already a supplier might need to be
double-wrapped, but I think that's worth it. I also wanted to work out what the
cost of wrapping things like
{{org.apache.cassandra.net.InboundMessageHandler#id()}} in a {{Supplier}}
versus just the call.
was (Author: jmeredithco):
I have a small prototype I was working on when I reported the issue that I
should clean up. I wanted to get some feedback on it before posting the patch,
but it probably doesn't need to wait. I'll try and push it up this morning.
The idea was too check the list of objects passed as arguments to check if they
were {{Supplier}}s and automatically call get on them if they were. The
downside would be anything that was already a supplier might need to be
double-wrapped, but I think that's worth it. I also wanted to work out what the
cost of wrapping things like
{{org.apache.cassandra.net.InboundMessageHandler#id()}} in a {{Supplier}}
versus just the call.
> NoSpamLogger arguments building objects on hot paths
>
>
> Key: CASSANDRA-15766
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15766
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-rc
>
>
> NoSpamLogger is used in hot logging paths to prevent logs being overrun. For
> that to be most effective the arguments to the logger need to be cheap to
> construct. During the internode messaging refactor CASSANDRA-15066,
> performance changes to BufferPool for CASSANDRA-14416
> were accidentally reverted in the merge up from 3.11.
> Reviewing other uses since, it looks like there are a few places where the
> arguments require some form of String building.
> org.apache.cassandra.net.InboundSink#accept
> org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame
> org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize
> org.apache.cassandra.net.OutboundConnection#onOverloaded
> org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks
> Formatting arguments should either be precomputed, or if expensive they
> should be computed after the decision on whether to noSpamLog has been made.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098630#comment-17098630
]
Ekaterina Dimitrova edited comment on CASSANDRA-13994 at 5/4/20, 1:33 PM:
--
[Patch|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-13994]
[CI
Java8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517]
[CI
Java11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/4938a2b2-28dd-45e7-971a-cbe5d656f86e]
There are a couple of failures which are flaky tests that exist also on the
trunk, not related.
Two outstanding questions:
- After the user migrates the compact storage, is it enough if we update
the upgrade documentation and indexes defunct properly in case the user still
have them and try to use them? Or do we want to iterate on start through the
indexes again (further to checking for not migrated COMPACT STORAGE tables)
check for old indexes of kind KEYS, and prevent the server from start? For now
there is a warning added to the one during server start if the user should
migrate from compact storage when they haven't done It yet.
- About the system table Built_indexes, I saw there is some
SystemKeyspaceMigrator40 class, I guess on start we can check and migrate this
table there (if it was not already migrated) ?
was (Author: e.dimitrova):
[Patch|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-13994]
[CI
Java8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/9e57522d-52fa-4d44-88d8-5cec0e87f517]
[CI
Java11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/118/workflows/4938a2b2-28dd-45e7-971a-cbe5d656f86e]
There are a couple of failures which are flaky tests that exist also on the
trunk, not related.
Two outstanding questions:
- After the user migrates the compact storage, is it enough if we update
the upgrade documentation and indexes defunct properly in case the user still
have them and try to use them? Or do we want to iterate on start through the
indexes again (further to checking for not migrated COMPACT STORAGE tables)
check for old indexes of kind KEYS, and prevent the server from start? For now
there is a warning added to the one, that the user should migrate from compact
storage on start if they haven't done It yet.
- About the system table Built_indexes, I saw there is some
SystemKeyspaceMigrator40 class, I guess on start we can check and migrate this
table there (if it was not already migrated) ?
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Local Write-Read Paths
>Reporter: Alex Petrov
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-rc
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13606) Improve handling of 2i initialization failures
[ https://issues.apache.org/jira/browse/CASSANDRA-13606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-13606: -- Reviewers: Andres de la Peña > Improve handling of 2i initialization failures > -- > > Key: CASSANDRA-13606 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13606 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Sergio Bossa >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0 > > Time Spent: 40m > Remaining Estimate: 0h > > CASSANDRA-10130 fixes the 2i build management, but initialization failures > are still not properly handled, most notably because: > * Initialization failures make the index non-queryable, but it can still be > written to. > * Initialization failures can be recovered via full rebuilds. > Both points above are probably suboptimal because the initialization logic > could be more complex than just an index build, hence it shouldn't be made > recoverable via a simple rebuild, and could cause the index to be fully > unavailable not just for reads, but for writes as well. > So, we should better handle initialization failures by: > * Allowing the index implementation to specify if unavailable for reads, > writes, or both. > * Providing a proper method to recover, distinct from index rebuilds. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15766) NoSpamLogger arguments building objects on hot paths
[ https://issues.apache.org/jira/browse/CASSANDRA-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098818#comment-17098818 ] Berenguer Blasi commented on CASSANDRA-15766: - Hi [~jmeredithco] I fancy taking this one. Is that ok? > NoSpamLogger arguments building objects on hot paths > > > Key: CASSANDRA-15766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15766 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: Jon Meredith >Assignee: Jon Meredith >Priority: Normal > Fix For: 4.0-rc > > > NoSpamLogger is used in hot logging paths to prevent logs being overrun. For > that to be most effective the arguments to the logger need to be cheap to > construct. During the internode messaging refactor CASSANDRA-15066, > performance changes to BufferPool for CASSANDRA-14416 > were accidentally reverted in the merge up from 3.11. > Reviewing other uses since, it looks like there are a few places where the > arguments require some form of String building. > org.apache.cassandra.net.InboundSink#accept > org.apache.cassandra.net.InboundMessageHandler#processCorruptFrame > org.apache.cassandra.net.InboundMessageHandler.LargeMessage#deserialize > org.apache.cassandra.net.OutboundConnection#onOverloaded > org.apache.cassandra.utils.memory.BufferPool.GlobalPool#allocateMoreChunks > Formatting arguments should either be precomputed, or if expensive they > should be computed after the decision on whether to noSpamLog has been made. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098809#comment-17098809
]
Eduard Tudenhoefner commented on CASSANDRA-15781:
-
changes LGTM
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098808#comment-17098808
]
Eduard Tudenhoefner commented on CASSANDRA-15780:
-
changes LGTM. Can you please link a test run?
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15782) Compression test failure
[ https://issues.apache.org/jira/browse/CASSANDRA-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15782: Bug Category: Parent values: Correctness(12982)Level 1 values: Test Failure(12990) Discovered By: Unit Test > Compression test failure > > > Key: CASSANDRA-15782 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15782 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Priority: Normal > Fix For: 4.0 > > > On CASSANDRA-15560 compression test failed. This was bisected to > [9c1bbf3|https://github.com/apache/cassandra/commit/9c1bbf3ac913f9bdf7a0e0922106804af42d2c1e] > from CASSANDRA-15379. > Full details here > CC/ [~jolynch] in case he can spot it quick. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15757) CustomIndexTest.indexBuildingPagesLargePartitions is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098800#comment-17098800
]
Berenguer Blasi commented on CASSANDRA-15757:
-
Hi [~adelapena] I was looking into this one as well but I didn't go as far as
you did. In any case this makes sense to me so +1 from my side but with a 'I am
a new to the codebase disclaimer' :)
> CustomIndexTest.indexBuildingPagesLargePartitions is flaky
> --
>
> Key: CASSANDRA-15757
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15757
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Jon Meredith
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> CustomIndexTest.indexBuildingPagesLargePartitions is flaky. Failed in CI and
> was able to reproduce failure inside IntelliJ by setting test Repeat to ‘Run
> Until Failure’. Failed after 459 iterations.
> {code}
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.cassandra.index.CustomIndexTest.lambda$indexBuildingPagesLargePartitions$1(CustomIndexTest.java:687)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.cassandra.index.CustomIndexTest.indexBuildingPagesLargePartitions(CustomIndexTest.java:687)
> at jdk.internal.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:53)
> at
> com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15757) CustomIndexTest.indexBuildingPagesLargePartitions is flaky
[
https://issues.apache.org/jira/browse/CASSANDRA-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-15757:
Reviewers: Berenguer Blasi
> CustomIndexTest.indexBuildingPagesLargePartitions is flaky
> --
>
> Key: CASSANDRA-15757
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15757
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Jon Meredith
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> CustomIndexTest.indexBuildingPagesLargePartitions is flaky. Failed in CI and
> was able to reproduce failure inside IntelliJ by setting test Repeat to ‘Run
> Until Failure’. Failed after 459 iterations.
> {code}
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.cassandra.index.CustomIndexTest.lambda$indexBuildingPagesLargePartitions$1(CustomIndexTest.java:687)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.cassandra.index.CustomIndexTest.indexBuildingPagesLargePartitions(CustomIndexTest.java:687)
> at jdk.internal.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:53)
> at
> com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15560) Change io.compressor.LZ4Compressor to LZ4SafeDecompressor
[ https://issues.apache.org/jira/browse/CASSANDRA-15560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098777#comment-17098777 ] Berenguer Blasi commented on CASSANDRA-15560: - CASSANDRA-15782 created > Change io.compressor.LZ4Compressor to LZ4SafeDecompressor > - > > Key: CASSANDRA-15560 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15560 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Compression >Reporter: Jordan West >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0, 4.0-rc > > Time Spent: 0.5h > Remaining Estimate: 0h > > CASSANDRA-15556 and related tickets showed that LZ4FastDecompressor can crash > the JVM and that LZ4SafeDecompressor performs better w/o the crash risk — its > also not deprecated. While we protect ourselves by checksumming the > compressed data but that doesn’t mean we should leave deprecated code that > can segfault the jvm (providing a potential DDOS vector among other things) > in crucial places like io.compress. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15782) Compression test failure
[ https://issues.apache.org/jira/browse/CASSANDRA-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-15782: Fix Version/s: 4.0 > Compression test failure > > > Key: CASSANDRA-15782 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15782 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Priority: Normal > Fix For: 4.0 > > > On CASSANDRA-15560 compression test failed. This was bisected to > [9c1bbf3|https://github.com/apache/cassandra/commit/9c1bbf3ac913f9bdf7a0e0922106804af42d2c1e] > from CASSANDRA-15379. > Full details here > CC/ [~jolynch] in case he can spot it quick. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15782) Compression test failure
Berenguer Blasi created CASSANDRA-15782: --- Summary: Compression test failure Key: CASSANDRA-15782 URL: https://issues.apache.org/jira/browse/CASSANDRA-15782 Project: Cassandra Issue Type: Bug Components: Test/dtest Reporter: Berenguer Blasi On CASSANDRA-15560 compression test failed. This was bisected to [9c1bbf3|https://github.com/apache/cassandra/commit/9c1bbf3ac913f9bdf7a0e0922106804af42d2c1e] from CASSANDRA-15379. Full details here CC/ [~jolynch] in case he can spot it quick. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15502) In Tree Tooling with Java 11
[ https://issues.apache.org/jira/browse/CASSANDRA-15502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098774#comment-17098774 ] Robert Stupp commented on CASSANDRA-15502: -- I've looked into this and all the tools are exercised in the unit-tests and/or dtests. Looks like we're good here? > In Tree Tooling with Java 11 > > > Key: CASSANDRA-15502 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15502 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/fuzz, Tool/bulk load, Tool/cqlsh, > Tool/diff, Tool/fql, Tool/nodetool, Tool/sstable, Tool/stress >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > This is to cover testing the various tools running on java 11. > The scope of this testing is manual testing and not automated, different > JIRAs should cover automation testing these tool. > The tools in question are: nodetool, sstableloader, sstablescrub, > sstableupgrade, sstableutil, sstableverify, fqltool, stress, auditlogviewer, > compaction-stress, sstabledump, sstableexpiredblockers, sstablemetadata, > sstableofflinerelevel, sstablesplit, and sstablerepairedset (many of these > may be tested already in dtest) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Test and Documentation Plan: -
Status: Patch Available (was: Open)
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
[
https://issues.apache.org/jira/browse/CASSANDRA-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15781:
-
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Fix Version/s: 4.0
Reviewers: Eduard Tudenhoefner
Status: Open (was: Triage Needed)
[Github PR here|https://github.com/apache/cassandra/pull/581]
> Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and
> apply the same "fix" to other tests
> -
>
> Key: CASSANDRA-15781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
> flaky, especially when many tests are running concurrently, because the used
> thread-pool can still be running when the assertions are tested (i.e. the
> {{ExecutorService.awaitTermination()}} returned false).
> The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
> return value of {{awaitTermination}} and apply the same change to some other
> tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15781) Fix flakiness of DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix" to other tests
Robert Stupp created CASSANDRA-15781:
Summary: Fix flakiness of
DecayingEstimatedHistogramReservoirTest#testStriping and apply the same "fix"
to other tests
Key: CASSANDRA-15781
URL: https://issues.apache.org/jira/browse/CASSANDRA-15781
Project: Cassandra
Issue Type: Improvement
Components: Test/unit
Reporter: Robert Stupp
Assignee: Robert Stupp
The unit test {{DecayingEstimatedHistogramReservoirTest#testStriping}} is
flaky, especially when many tests are running concurrently, because the used
thread-pool can still be running when the assertions are tested (i.e. the
{{ExecutorService.awaitTermination()}} returned false).
The "fix" is rather a band aid: extend the wait-period to 1 minute, check the
return value of {{awaitTermination}} and apply the same change to some other
tests that _might_ be affected as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Reviewers: Eduard Tudenhoefner
[Github PR here|https://github.com/apache/cassandra/pull/580]
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Test and Documentation Plan: -
Status: Patch Available (was: Open)
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-15780:
-
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Fix Version/s: 4.0
Status: Open (was: Triage Needed)
> Less test stderr spam when running tests w/ java 11
> ---
>
> Key: CASSANDRA-15780
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/unit
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Normal
> Fix For: 4.0
>
>
> There are a bunch of tests that, when run with Java 11, emit the boilerplate
> {{WARNING: An illegal reflective access operation has occurred}} messages.
> The proposed change just adds more invocations to
> {{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to
> get rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15780) Less test stderr spam when running tests w/ java 11
Robert Stupp created CASSANDRA-15780:
Summary: Less test stderr spam when running tests w/ java 11
Key: CASSANDRA-15780
URL: https://issues.apache.org/jira/browse/CASSANDRA-15780
Project: Cassandra
Issue Type: Improvement
Components: Test/unit
Reporter: Robert Stupp
Assignee: Robert Stupp
There are a bunch of tests that, when run with Java 11, emit the boilerplate
{{WARNING: An illegal reflective access operation has occurred}} messages.
The proposed change just adds more invocations to
{{org.apache.cassandra.utils.FBUtilities#preventIllegalAccessWarnings}} to get
rid of many of those.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
