[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032120#comment-17032120
]
David Capwell edited comment on CASSANDRA-15313 at 2/7/20 5:44 AM:
---
Sent patch for CASSANDRA-15556 which makes it so the JVM doesn't crash anymore;
with that in the only remaining issue I know of is
{code}
transformInbound(transformOutbound(input)) != input
{code}
[~samt] would it be possible for you to take a look? I provided a set of seeds
which fail, so can rerun the test with that seed to reproduce the failure.
was (Author: dcapwell):
Sent patch for CASSANDRA-15556 which makes it so the JVM doesn't crash anymore;
with that in the only remaining issue I know of is
{code}
transformInbound(transformOutboun(input)) != input
{code}
[~samt] would it be possible for you to take a look? I provided a set of seeds
which fail, so can rerun the test with that seed to reproduce the failure.
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at org.quicktheories.impl.Precursor.(Precursor.java:17)
> at
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
> at
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
> at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
> at
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
> at
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
> at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
> at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown
> Source)
> at
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
> at
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.impl.Core.generate(Core.java:150)
> at org.quicktheories.impl.Core.shrink(Core.java:103)
> at org.quicktheories.impl.Core.run(Core.java:39)
> at org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:35)
> at org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
> at
> org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
> at
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031849#comment-17031849
]
David Capwell edited comment on CASSANDRA-15313 at 2/6/20 9:46 PM:
---
With my new patch I get the following failure
{code}
java.lang.AssertionError: Property falsified after 4675 example(s)
Smallest found falsifying value(s) :-
{(6364636563646464636365656365636565636564656463636565636464656563656565646364646363646563646363646363656464636465656364636365656564636365636565636463646565636463636364656365646363656563656365636464646464646464646363646464656463636463646564636464646363656464636364646363646365636365646463656465636565646465656464636364646364646565636564656563656563646563636565646563636563646565636363636563656364646363656564656364646463636364646565656363656464636564646463656463656363636565656563646364656363636565636363656363636563636464636364636464636363646564646563646465636564636464636365646465636565636363646564656463646565646364636565636464636365656463636464656364646564636464636465636563656365636563646464646563646564646564656463646563636465656364646565646465656363636363636563656363656563636364646363646464656564656364646364636563656465636563646463646364656365636464656364636564646465646465646564656363656464656563636565646564646565656363636465646463656463646365656463636363646563656365656363636563646465636563636464636363636363656364646365656363636365646364656364636565646363646564656363656464656563656465636365636465656364656364656564636465646464646464646365636365646465646463656363636465636464656564646464646464636464636363646564646365656565636564646564636364636363656565636365656465636363646363656564636563656363636365646564656564646463636364656463636364636464636464646565656463656365636565636363636564636564656463656363646365636363646465646464636464636564636563646463646363646364656364636365646363656465636365646465646563636463646363636465646465646463636463646364646463636464656363636464656565656365656364646565656564636563636565636465636363636364646465646363646463646564646465656464636563636363636363656365656563656365656363656563646364646365636364636463646464646565646463646465656364646463656463646364636465636463656363646364636463656365636364636363636364636365636364646364646465636365646465656463636<...>636364646363636565646365646463636363636563646363656464646364646364636364656363646363656463636363656363636563646563636563656363656363646363636363636363646363646563636363646365636363636465656463636363646363636563636463656363636365656363656364646363646365636463646363636363646363646463636463636363636365636363636363646363636363636363636365636464646363636565636363646363636363636363636364636365656364656363636463636363636363636365636463656563656364646365636564636364656364646363636364636463656463636363636365646364636565656363636363636365636563646363656365636364656364646363636363646564636563636464636363656363646363636363636364636563636363636563636363656463636363636363636363656363636363636463636365656363636364636464656463646363646364636463636363636363636363636465646464636363646363636363646563636463636563636563636463646563656463636463636463636565646363646363636463636363636563636363636463656463656364646363636365636363636364636364636563656364646463646363646363636563636363646364646563636365636364636365636363636565636364646465636363636364646363636365656363656464646363636363646463636464636364656363636464656363636563636364646363636363636463646364636365636363646363646363636463656463646564636364646363636463636463636363636363636364636564636464656363636463646363636364636363636363646363646464636363646363636464636464636363636365646364646563656563656363646563636364636363636364646365636363656463646363636363636364636363646363636365646365646363646364646363636363636565656463646363656363646364636363636365636363646365646363636363636363636564636565646463636563646465646463636463636363656365646364636465636463646563636364646363636463636563646363646365646364646363636363656365636363636363636364636563636363656364656463646363636463636363636563636463656365636363636363646364656363636363636365636563636364646363636463636465636463656464646563646363656564656463646563656363636463656464636364636563,72535),
14, org.apache.cassandra.transport.frame.compress.SnappyCompressor@4b41dd5c,
ADLER32}
Seed was 113149736037976
{code}
and
{code}
java.lang.AssertionError: Property falsified after 21514 example(s)
Smallest found falsifying value(s) :-
[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031849#comment-17031849
]
David Capwell edited comment on CASSANDRA-15313 at 2/6/20 6:59 PM:
---
With my new patch I get the following failure
{code}
java.lang.AssertionError: Property falsified after 4675 example(s)
Smallest found falsifying value(s) :-
{(6364636563646464636365656365636565636564656463636565636464656563656565646364646363646563646363646363656464636465656364636365656564636365636565636463646565636463636364656365646363656563656365636464646464646464646363646464656463636463646564636464646363656464636364646363646365636365646463656465636565646465656464636364646364646565636564656563656563646563636565646563636563646565636363636563656364646363656564656364646463636364646565656363656464636564646463656463656363636565656563646364656363636565636363656363636563636464636364636464636363646564646563646465636564636464636365646465636565636363646564656463646565646364636565636464636365656463636464656364646564636464636465636563656365636563646464646563646564646564656463646563636465656364646565646465656363636363636563656363656563636364646363646464656564656364646364636563656465636563646463646364656365636464656364636564646465646465646564656363656464656563636565646564646565656363636465646463656463646365656463636363646563656365656363636563646465636563636464636363636363656364646365656363636365646364656364636565646363646564656363656464656563656465636365636465656364656364656564636465646464646464646365636365646465646463656363636465636464656564646464646464636464636363646564646365656565636564646564636364636363656565636365656465636363646363656564636563656363636365646564656564646463636364656463636364636464636464646565656463656365636565636363636564636564656463656363646365636363646465646464636464636564636563646463646363646364656364636365646363656465636365646465646563636463646363636465646465646463636463646364646463636464656363636464656565656365656364646565656564636563636565636465636363636364646465646363646463646564646465656464636563636363636363656365656563656365656363656563646364646365636364636463646464646565646463646465656364646463656463646364636465636463656363646364636463656365636364636363636364636365636364646364646465636365646465656463636<...>636364646363636565646365646463636363636563646363656464646364646364636364656363646363656463636363656363636563646563636563656363656363646363636363636363646363646563636363646365636363636465656463636363646363636563636463656363636365656363656364646363646365636463646363636363646363646463636463636363636365636363636363646363636363636363636365636464646363636565636363646363636363636363636364636365656364656363636463636363636363636365636463656563656364646365636564636364656364646363636364636463656463636363636365646364636565656363636363636365636563646363656365636364656364646363636363646564636563636464636363656363646363636363636364636563636363636563636363656463636363636363636363656363636363636463636365656363636364636464656463646363646364636463636363636363636363636465646464636363646363636363646563636463636563636563636463646563656463636463636463636565646363646363636463636363636563636363636463656463656364646363636365636363636364636364636563656364646463646363646363636563636363646364646563636365636364636365636363636565636364646465636363636364646363636365656363656464646363636363646463636464636364656363636464656363636563636364646363636363636463646364636365636363646363646363636463656463646564636364646363636463636463636363636363636364636564636464656363636463646363636364636363636363646363646464636363646363636464636464636363636365646364646563656563656363646563636364636363636364646365636363656463646363636363636364636363646363636365646365646363646364646363636363636565656463646363656363646364636363636365636363646365646363636363636363636564636565646463636563646465646463636463636363656365646364636465636463646563636364646363636463636563646363646365646364646363636363656365636363636363636364636563636363656364656463646363636463636363636563636463656365636363636363646364656363636363636365636563636364646363636463636465636463656464646563646363656564656463646563656363636463656464636364636563,72535),
14, org.apache.cassandra.transport.frame.compress.SnappyCompressor@4b41dd5c,
ADLER32}
Seed was 113149736037976
{code}
and
{code}
java.lang.AssertionError: Property falsified after 21514 example(s)
Smallest found falsifying value(s) :-
[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031849#comment-17031849
]
David Capwell edited comment on CASSANDRA-15313 at 2/6/20 6:54 PM:
---
With my new patch I get the following failure
{code}
java.lang.AssertionError: Property falsified after 4675 example(s)
Smallest found falsifying value(s) :-
{(6364636563646464636365656365636565636564656463636565636464656563656565646364646363646563646363646363656464636465656364636365656564636365636565636463646565636463636364656365646363656563656365636464646464646464646363646464656463636463646564636464646363656464636364646363646365636365646463656465636565646465656464636364646364646565636564656563656563646563636565646563636563646565636363636563656364646363656564656364646463636364646565656363656464636564646463656463656363636565656563646364656363636565636363656363636563636464636364636464636363646564646563646465636564636464636365646465636565636363646564656463646565646364636565636464636365656463636464656364646564636464636465636563656365636563646464646563646564646564656463646563636465656364646565646465656363636363636563656363656563636364646363646464656564656364646364636563656465636563646463646364656365636464656364636564646465646465646564656363656464656563636565646564646565656363636465646463656463646365656463636363646563656365656363636563646465636563636464636363636363656364646365656363636365646364656364636565646363646564656363656464656563656465636365636465656364656364656564636465646464646464646365636365646465646463656363636465636464656564646464646464636464636363646564646365656565636564646564636364636363656565636365656465636363646363656564636563656363636365646564656564646463636364656463636364636464636464646565656463656365636565636363636564636564656463656363646365636363646465646464636464636564636563646463646363646364656364636365646363656465636365646465646563636463646363636465646465646463636463646364646463636464656363636464656565656365656364646565656564636563636565636465636363636364646465646363646463646564646465656464636563636363636363656365656563656365656363656563646364646365636364636463646464646565646463646465656364646463656463646364636465636463656363646364636463656365636364636363636364636365636364646364646465636365646465656463636<...>636364646363636565646365646463636363636563646363656464646364646364636364656363646363656463636363656363636563646563636563656363656363646363636363636363646363646563636363646365636363636465656463636363646363636563636463656363636365656363656364646363646365636463646363636363646363646463636463636363636365636363636363646363636363636363636365636464646363636565636363646363636363636363636364636365656364656363636463636363636363636365636463656563656364646365636564636364656364646363636364636463656463636363636365646364636565656363636363636365636563646363656365636364656364646363636363646564636563636464636363656363646363636363636364636563636363636563636363656463636363636363636363656363636363636463636365656363636364636464656463646363646364636463636363636363636363636465646464636363646363636363646563636463636563636563636463646563656463636463636463636565646363646363636463636363636563636363636463656463656364646363636365636363636364636364636563656364646463646363646363636563636363646364646563636365636364636365636363636565636364646465636363636364646363636365656363656464646363636363646463636464636364656363636464656363636563636364646363636363636463646364636365636363646363646363636463656463646564636364646363636463636463636363636363636364636564636464656363636463646363636364636363636363646363646464636363646363636464636464636363636365646364646563656563656363646563636364636363636364646365636363656463646363636363636364636363646363636365646365646363646364646363636363636565656463646363656363646364636363636365636363646365646363636363636363636564636565646463636563646465646463636463636363656365646364636465636463646563636364646363636463636563646363646365646364646363636363656365636363636363636364636563636363656364656463646363636463636363636563636463656365636363636363646364656363636363636365636563636364646363636463636465636463656464646563646363656564656463646563656363636463656464636364636563,72535),
14, org.apache.cassandra.transport.frame.compress.SnappyCompressor@4b41dd5c,
ADLER32}
Seed was 113149736037976
{code}
and
{code}
java.lang.AssertionError: Property falsified after 21514 example(s)
Smallest found falsifying value(s) :-
[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031759#comment-17031759
]
David Capwell edited comment on CASSANDRA-15313 at 2/6/20 5:00 PM:
---
The output from quick theories doesn't work in isolation; the way the string is
generated then converted to bytes ins't reproducible without relying on the
seed to create the same string. I changed the text to print out the hex of the
bytes and when you run with those bytes the test fails
Here is the hex for the bytes failing
{code}
[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031721#comment-17031721
]
Brandon Williams edited comment on CASSANDRA-15313 at 2/6/20 4:21 PM:
--
bq. Not a huge fan of fixing the seed.
I'm not suggesting that, I only kept it to demonstrate the failure that is
possibly an LZ4 bug.
was (Author: brandon.williams):
bq. Not a huge fan of fixing the seed.
I'm not suggesting that, I only kept it so demonstrate the failure that is
possibly an LZ4 bug.
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at org.quicktheories.impl.Precursor.(Precursor.java:17)
> at
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
> at
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
> at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
> at
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
> at
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
> at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
> at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown
> Source)
> at
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
> at
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.impl.Core.generate(Core.java:150)
> at org.quicktheories.impl.Core.shrink(Core.java:103)
> at org.quicktheories.impl.Core.run(Core.java:39)
> at org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:35)
> at org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
> at
> org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
> at
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031680#comment-17031680
]
Jordan West edited comment on CASSANDRA-15313 at 2/6/20 3:49 PM:
-
[~samt] actually included the test on my behalf. Its not surprising that this
is a memory issue. Thats one motivation for
https://issues.apache.org/jira/browse/CASSANDRA-15348 which we could convert to
when ready. But in the meantime let me see what we can do to reduce the memory
usage of the test.
Interesting its also potentially finding an issue with LZ4.
was (Author: jrwest):
[~samt] actually included the test on my behalf. Its not surprising that this
is a memory issue. Thats one motivation for
https://issues.apache.org/jira/browse/CASSANDRA-15348 which we could convert to
when ready. But in the meantime let me see what we can do to reduce the memory
usage of the test.
Interesting its also potentially finding an issue with LZ4. Have we filed a
separate JIRA for that?
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at org.quicktheories.impl.Precursor.(Precursor.java:17)
> at
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
> at
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
> at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
> at
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
> at
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
> at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
> at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown
> Source)
> at
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
> at
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.impl.Core.generate(Core.java:150)
> at org.quicktheories.impl.Core.shrink(Core.java:103)
> at org.quicktheories.impl.Core.run(Core.java:39)
> at org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:35)
> at org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
> at
> org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
> at
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031680#comment-17031680
]
Jordan West edited comment on CASSANDRA-15313 at 2/6/20 3:40 PM:
-
[~samt] actually included the test on my behalf. Its not surprising that this
is a memory issue. Thats one motivation for
https://issues.apache.org/jira/browse/CASSANDRA-15348 which we could convert to
when ready. But in the meantime let me see what we can do to reduce the memory
usage of the test.
Interesting its also potentially finding an issue with LZ4. Have we filed a
separate JIRA for that?
was (Author: jrwest):
[~samt] actually included the test on my behalf so I can take a closer look here
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at org.quicktheories.impl.Precursor.(Precursor.java:17)
> at
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
> at
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
> at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
> at
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
> at
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
> at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
> at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown
> Source)
> at
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
> at
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.impl.Core.generate(Core.java:150)
> at org.quicktheories.impl.Core.shrink(Core.java:103)
> at org.quicktheories.impl.Core.run(Core.java:39)
> at org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:35)
> at org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
> at
> org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
> at
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031282#comment-17031282
]
David Capwell edited comment on CASSANDRA-15313 at 2/6/20 5:18 AM:
---
"it" is the failing case; sorry for the confusion. By lowering the examples we
will need to run the test more before we see the failing case.
bq. I ran at 4G and it still failed at 200 examples (I didn't try any other
sizes though)
This then sounds like the generators produce too much garbage. I took a stab at
fixing it and attaching a hack patch to the JIRA to show; basically move away
from String since that produces a ton of garbage. With the patch and a failing
test case, the test completes in 200ms (900ms if you run in isolation).
[~samt] don't know the test well but when I run it (not patched) in a debugger
with 1000 examples, I see the call
https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/transport/frame/checksum/ChecksummingTransformerTest.java#L118
2 times; so only 2 out of 1k runs hits that check. My patch doesn't hit it
(after a few runs), but could always tweak the bytes to match the expected
input (always throws ProtocolException).
was (Author: dcapwell):
"it" is the failing case; sorry for the confusion. By lowering the examples we
will need to run the test more before we see the failing case.
bq. I ran at 4G and it still failed at 200 examples (I didn't try any other
sizes though)
This then sounds like the generators produce too much garbage. I took a stab at
fixing it and attaching a hack patch to the JIRA to show; basically move away
from String since that produces a ton of garbage. With the patch and a failing
test case, the test completes in 200ms (900ms if you run in isolation).
[~samt] don't know the test well but when I run it (not patched) in a debugger
with 1000 examples, I see the call
https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/transport/frame/checksum/ChecksummingTransformerTest.java#L118
2 times; so only 2 out of 1k runs hits that check. My patch doesn't hit it
(after a few runs), but could always tweak the bytes to match the expected
input.
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15313-hack.patch
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> at org.quicktheories.impl.Precursor.(Precursor.java:17)
> at
> org.quicktheories.impl.ConcreteDetachedSource.(ConcreteDetachedSource.java:8)
> at
> org.quicktheories.impl.ConcreteDetachedSource.detach(ConcreteDetachedSource.java:23)
> at org.quicktheories.generators.Retry.generate(CodePoints.java:51)
> at
> org.quicktheories.generators.Generate.lambda$intArrays$10(Generate.java:190)
> at
> org.quicktheories.generators.Generate$$Lambda$17/1847008471.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$map$0(Gen.java:36)
> at org.quicktheories.core.Gen$$Lambda$20/71399214.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$mix$10(Gen.java:184)
> at org.quicktheories.core.Gen$$Lambda$45/802243390.generate(Unknown
> Source)
> at org.quicktheories.core.Gen.lambda$flatMap$5(Gen.java:93)
> at org.quicktheories.core.Gen$$Lambda$48/363509958.generate(Unknown
> Source)
> at
> org.quicktheories.dsl.TheoryBuilder4.lambda$prgnToTuple$12(TheoryBuilder4.java:188)
> at
> org.quicktheories.dsl.TheoryBuilder4$$Lambda$40/2003496028.generate(Unknown
> Source)
> at org.quicktheories.core.DescribingGenerator.generate(Gen.java:255)
> at org.quicktheories.core.FilteredGenerator.generate(Gen.java:225)
> at
[jira] [Comment Edited] (CASSANDRA-15313) Fix flaky - ChecksummingTransformerTest - org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030151#comment-17030151
]
Brandon Williams edited comment on CASSANDRA-15313 at 2/4/20 10:01 PM:
---
This is mostly due to the two places where a rather large example size is used,
500. I did many runs lowering this number until I could be sure there were no
timeouts, and that number is 35. However, once the test was not always timing
out (at around 40 examples) it discovered a seed that always fails:
{code:java}
[junit-timeout] Seed was 552252992721215
[junit-timeout] at
org.quicktheories.core.ExceptionReporter.falsify(ExceptionReporter.java:43)
[junit-timeout] at
org.quicktheories.core.ExceptionReporter.falisification(ExceptionReporter.java:37)
[junit-timeout] at
org.quicktheories.impl.TheoryRunner.reportFalsification(TheoryRunner.java:48)
[junit-timeout] at
org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:37)
[junit-timeout] at
org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
[junit-timeout] at
org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
[junit-timeout] at
org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:88)
[junit-timeout]
[junit-timeout]
[junit-timeout] Test
org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest FAILED
{code}
Sometimes instead of failing the test, it will crash the jvm here:
{code:java}
[junit-timeout] # C [liblz4-java4100719362944375598.so+0x5eb7]
LZ4_decompress_fast+0x117
{code}
I've reproduced this against lz4 1.4.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.7.0, and
1.7.1. I think beyond the flakiness which can be solved by reducing the amount
of examples, we've got a legitimate bug in lz4. I pushed the branch to
reproduce here: [https://github.com/driftx/cassandra/tree/CASSANDRA-15313]
You'll need to override the test timeout as this takes around 7 minutes.
WDYT, [~samt]?
was (Author: brandon.williams):
This is mostly due to the two places where a rather large example size is used,
500. I did many runs lowering this number until I could be sure there were no
timeouts, and that number is 35. However, once the test was not always timing
out (at around 40 examples) it discovered a seed that always fails:
{code:java}
[junit-timeout] Seed was 552252992721215
[junit-timeout] at
org.quicktheories.core.ExceptionReporter.falsify(ExceptionReporter.java:43)
[junit-timeout] at
org.quicktheories.core.ExceptionReporter.falisification(ExceptionReporter.java:37)
[junit-timeout] at
org.quicktheories.impl.TheoryRunner.reportFalsification(TheoryRunner.java:48)
[junit-timeout] at
org.quicktheories.impl.TheoryRunner.check(TheoryRunner.java:37)
[junit-timeout] at
org.quicktheories.dsl.TheoryBuilder4.check(TheoryBuilder4.java:150)
[junit-timeout] at
org.quicktheories.dsl.TheoryBuilder4.checkAssert(TheoryBuilder4.java:162)
[junit-timeout] at
org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest.corruptionCausesFailure(ChecksummingTransformerTest.java:88)
[junit-timeout]
[junit-timeout]
[junit-timeout] Test
org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest FAILED
{code}
Sometimes instead of failing the test, it will crash the jvm here:
{code:java}
[junit-timeout] # C [liblz4-java4100719362944375598.so+0x5eb7]
LZ4_decompress_fast+0x117
{code}
I've reproduced this against lz4 1.4.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.7.0, and
1.7.1. I think beyond the flakiness which can be solved by reducing the amount
of examples, we've got a legitimate bug in lz4. I pushed the branch to
reproduce here: https://github.com/driftx/cassandra/tree/CASSANDRA-15313
WDYT, [~samt]?
> Fix flaky - ChecksummingTransformerTest -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> ---
>
> Key: CASSANDRA-15313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15313
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Vinay Chella
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> During the recent runs, this test appears to be flaky.
> Example failure:
> [https://circleci.com/gh/vinaykumarchella/cassandra/459#tests/containers/94]
> corruptionCausesFailure-compression -
> org.apache.cassandra.transport.frame.checksum.ChecksummingTransformerTest
> {code:java}
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
> at
