[jira] [Commented] (CASSANDRA-17298) Test Failure: org.apache.cassandra.cql3.MemtableSizeTest
[ https://issues.apache.org/jira/browse/CASSANDRA-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842927#comment-17842927 ] Josh McKenzie commented on CASSANDRA-17298: --- {quote}when I run locally {quote} Locally in intellij or locally from console? I ask because on my JDK21 work, I have a failure in `MemtableQuickTest` that's passing in IDE and failing from console. Since it looks like they source their runtime env params from different locations I'm starting to suspect drift there. Also seen some other jamm related issues but they're known issues on other JIRA tickets as well, so I wouldn't be surprised if you were wrestling with or seeing something there as well - matches w/what you're seeing in CI. > Test Failure: org.apache.cassandra.cql3.MemtableSizeTest > > > Key: CASSANDRA-17298 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17298 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Josh McKenzie >Priority: Normal > Fix For: 4.0.x, 4.1.x > > > [https://ci-cassandra.apache.org/job/Cassandra-4.0/313/testReport/org.apache.cassandra.cql3/MemtableSizeTest/testTruncationReleasesLogSpace_2/] > Failed 4 times in the last 30 runs. Flakiness: 27%, Stability: 86% > Error Message > Expected heap usage close to 49.930MiB, got 41.542MiB. > {code} > Stacktrace > junit.framework.AssertionFailedError: Expected heap usage close to 49.930MiB, > got 41.542MiB. > at > org.apache.cassandra.cql3.MemtableSizeTest.testSize(MemtableSizeTest.java:130) > at org.apache.cassandra.Util.runCatchingAssertionError(Util.java:644) > at org.apache.cassandra.Util.flakyTest(Util.java:669) > at > org.apache.cassandra.cql3.MemtableSizeTest.testTruncationReleasesLogSpace(MemtableSizeTest.java:61) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > {code} > *UPDATE:* It was discovered that unit tests were running with > memtable_allocation_type: offheap_objects when we ship C* with heap_buffers. > So we changed that in CASSANDRA-19326, now we test with > memtable_allocation_type: heap_buffers. As a result, this test now fails all > the time on 4.0 and 4.1. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17298) Test Failure: org.apache.cassandra.cql3.MemtableSizeTest
[ https://issues.apache.org/jira/browse/CASSANDRA-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842731#comment-17842731 ] Ekaterina Dimitrova commented on CASSANDRA-17298: - I don't know about the local error you get, but this is about the one you see in CI: 4.0 and 4.1 use the old version of jamm where there were issues with the crawling, so I am not surprised measureDeep hit closed module. This cannot be related to your patch. It is jamm issue. > Test Failure: org.apache.cassandra.cql3.MemtableSizeTest > > > Key: CASSANDRA-17298 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17298 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Josh McKenzie >Priority: Normal > Fix For: 4.0.x, 4.1.x > > > [https://ci-cassandra.apache.org/job/Cassandra-4.0/313/testReport/org.apache.cassandra.cql3/MemtableSizeTest/testTruncationReleasesLogSpace_2/] > Failed 4 times in the last 30 runs. Flakiness: 27%, Stability: 86% > Error Message > Expected heap usage close to 49.930MiB, got 41.542MiB. > {code} > Stacktrace > junit.framework.AssertionFailedError: Expected heap usage close to 49.930MiB, > got 41.542MiB. > at > org.apache.cassandra.cql3.MemtableSizeTest.testSize(MemtableSizeTest.java:130) > at org.apache.cassandra.Util.runCatchingAssertionError(Util.java:644) > at org.apache.cassandra.Util.flakyTest(Util.java:669) > at > org.apache.cassandra.cql3.MemtableSizeTest.testTruncationReleasesLogSpace(MemtableSizeTest.java:61) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > {code} > *UPDATE:* It was discovered that unit tests were running with > memtable_allocation_type: offheap_objects when we ship C* with heap_buffers. > So we changed that in CASSANDRA-19326, now we test with > memtable_allocation_type: heap_buffers. As a result, this test now fails all > the time on 4.0 and 4.1. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17298) Test Failure: org.apache.cassandra.cql3.MemtableSizeTest
[ https://issues.apache.org/jira/browse/CASSANDRA-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842726#comment-17842726 ] David Capwell commented on CASSANDRA-17298: --- This fails for me as well, but when I run locally on JDK 11 I get the following error {code} Caused by: FSReadError in /Users/dcapwell/src/github/apache/cassandra/4.0/build/test/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377 at org.apache.cassandra.db.lifecycle.LogReplica.create(LogReplica.java:59) ... Caused by: java.io.IOException: Invalid folder descriptor trying to create log replica /Users/dcapwell/src/github/apache/cassandra/4.0/build/test/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377 ... 26 more {code} > Test Failure: org.apache.cassandra.cql3.MemtableSizeTest > > > Key: CASSANDRA-17298 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17298 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Josh McKenzie >Priority: Normal > Fix For: 4.0.x, 4.1.x > > > [https://ci-cassandra.apache.org/job/Cassandra-4.0/313/testReport/org.apache.cassandra.cql3/MemtableSizeTest/testTruncationReleasesLogSpace_2/] > Failed 4 times in the last 30 runs. Flakiness: 27%, Stability: 86% > Error Message > Expected heap usage close to 49.930MiB, got 41.542MiB. > {code} > Stacktrace > junit.framework.AssertionFailedError: Expected heap usage close to 49.930MiB, > got 41.542MiB. > at > org.apache.cassandra.cql3.MemtableSizeTest.testSize(MemtableSizeTest.java:130) > at org.apache.cassandra.Util.runCatchingAssertionError(Util.java:644) > at org.apache.cassandra.Util.flakyTest(Util.java:669) > at > org.apache.cassandra.cql3.MemtableSizeTest.testTruncationReleasesLogSpace(MemtableSizeTest.java:61) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > {code} > *UPDATE:* It was discovered that unit tests were running with > memtable_allocation_type: offheap_objects when we ship C* with heap_buffers. > So we changed that in CASSANDRA-19326, now we test with > memtable_allocation_type: heap_buffers. As a result, this test now fails all > the time on 4.0 and 4.1. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17298) Test Failure: org.apache.cassandra.cql3.MemtableSizeTest
[ https://issues.apache.org/jira/browse/CASSANDRA-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815741#comment-17815741 ] Ekaterina Dimitrova commented on CASSANDRA-17298: - Hey, [~paulo] - the short answer is this is new - the constant failures since last week. Now, the details :-) It was discovered that unit tests were running with memtable_allocation_type: offheap_objects when we ship C* with heap_buffers. So we changed that in CASSANDRA-19326, now we test with memtable_allocation_type: heap_buffers. As a result, this test now fails all the time. I will now mark tickets 4.0 and 4.1 and this additional information for awareness. I discovered the new failures yesterday but did not find this ticket. We were considering opening a new one with Jacek. > Test Failure: org.apache.cassandra.cql3.MemtableSizeTest > > > Key: CASSANDRA-17298 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17298 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Josh McKenzie >Priority: Normal > Fix For: 4.0.x, 4.1.x > > > [https://ci-cassandra.apache.org/job/Cassandra-4.0/313/testReport/org.apache.cassandra.cql3/MemtableSizeTest/testTruncationReleasesLogSpace_2/] > Failed 4 times in the last 30 runs. Flakiness: 27%, Stability: 86% > Error Message > Expected heap usage close to 49.930MiB, got 41.542MiB. > {code} > Stacktrace > junit.framework.AssertionFailedError: Expected heap usage close to 49.930MiB, > got 41.542MiB. > at > org.apache.cassandra.cql3.MemtableSizeTest.testSize(MemtableSizeTest.java:130) > at org.apache.cassandra.Util.runCatchingAssertionError(Util.java:644) > at org.apache.cassandra.Util.flakyTest(Util.java:669) > at > org.apache.cassandra.cql3.MemtableSizeTest.testTruncationReleasesLogSpace(MemtableSizeTest.java:61) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17298) Test Failure: org.apache.cassandra.cql3.MemtableSizeTest
[ https://issues.apache.org/jira/browse/CASSANDRA-17298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815684#comment-17815684 ] Paulo Motta commented on CASSANDRA-17298: - Looks like this is failing consistently in both 4.0/4.1: * https://butler.cassandra.apache.org/#/ci/upstream/compare/Cassandra-4.0/cassandra-4.0 * [https://butler.cassandra.apache.org/#/ci/upstream/compare/Cassandra-4.0/cassandra-4.1] [~e.dimitrova] I wonder if this should've been addressed by CASSANDRA-16684 or if it's a new issue. I am able to reproduce the failure locally on cassandra-4.1, even after increasing rerunsOnFailure from 2 to 4. {noformat} java.lang.AssertionError: Expected heap usage close to 75.335MiB, got 71.163MiB. at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.cassandra.cql3.MemtableSizeTest.testSizeFlaky(MemtableSizeTest.java:149) at org.apache.cassandra.Util.runCatchingAssertionError(Util.java:696) {noformat} > Test Failure: org.apache.cassandra.cql3.MemtableSizeTest > > > Key: CASSANDRA-17298 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17298 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Josh McKenzie >Priority: Normal > Fix For: 4.0.x > > > [https://ci-cassandra.apache.org/job/Cassandra-4.0/313/testReport/org.apache.cassandra.cql3/MemtableSizeTest/testTruncationReleasesLogSpace_2/] > Failed 4 times in the last 30 runs. Flakiness: 27%, Stability: 86% > Error Message > Expected heap usage close to 49.930MiB, got 41.542MiB. > {code} > Stacktrace > junit.framework.AssertionFailedError: Expected heap usage close to 49.930MiB, > got 41.542MiB. > at > org.apache.cassandra.cql3.MemtableSizeTest.testSize(MemtableSizeTest.java:130) > at org.apache.cassandra.Util.runCatchingAssertionError(Util.java:644) > at org.apache.cassandra.Util.flakyTest(Util.java:669) > at > org.apache.cassandra.cql3.MemtableSizeTest.testTruncationReleasesLogSpace(MemtableSizeTest.java:61) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org