[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787151#comment-17787151
]
Jacek Lewandowski edited comment on CASSANDRA-18710 at 11/17/23 11:48 AM:
--
I've run repeated tests in a couple of configurations (links in the PRs), all
passed, and [~blambov] approved the PR. Merging
was (Author: jlewandowski):
I've run repeated tests in a couple of configurations, all passed, and
[~blambov] approved the PR. Merging
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 5.0-beta, 5.0.x, 5.x
>
> Attachments: org.apache.cassandra.io.DiskSpaceMetricsTest.txt
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786290#comment-17786290
]
Branimir Lambov edited comment on CASSANDRA-18710 at 11/15/23 10:15 AM:
{quote}So perhaps the expected value should be calculated as a moving average
by updating it with subsequent table sizes.
{quote}
This makes sense. Sorting the sstable files by name should give them in the
correct order, so we can easily calculate the moving average from them.
Actually, that would solve the extra flush problem as well, wouldn't it?
was (Author: blambov):
{quote}So perhaps the expected value should be calculated as a moving average
by updating it with subsequent table sizes.
{quote}
This makes sense. Sorting the sstable files by name should give them in the
correct order, so we can easily calculate the moving average from them.
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.1.x, 5.0-beta, 5.0.x, 5.x
>
> Attachments: org.apache.cassandra.io.DiskSpaceMetricsTest.txt
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785915#comment-17785915
]
Jacek Lewandowski edited comment on CASSANDRA-18710 at 11/14/23 2:41 PM:
-
I've managed to reproduce it locally with IntelliJ repeated test runs...
a full fresh log
{noformat}
INFO [main] 2023-11-14 15:39:38,817 CQLTester.java:1004 - CREATE TABLE
cql_test_keyspace_alt.table_26 (pk bigint, PRIMARY KEY (pk))
DEBUG [main] 2023-11-14 15:39:38,818 DefaultSchemaUpdateHandler.java:259 -
Schema updated: SchemaTransformationResult{16db0888-1bd6-360e-a84c-fff0101449fa
--> 6ec911f6-0117-3684-ab6b-1366e985c0fe, diff=KeyspacesDiff{created=[],
dropped=[],
altered=[KeyspaceDiff{before=KeyspaceMetadata{name=cql_test_keyspace_alt,
kind=REGULAR, params=KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}, tables=[], views=[], functions=[], types=[]},
after=KeyspaceMetadata{name=cql_test_keyspace_alt, kind=REGULAR,
params=KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}, tables=[cql_test_keyspace_alt.table_26], views=[],
functions=[], types=[]}, tables=Diff{created=[cql_test_keyspace_alt.table_26],
dropped=[], altered=[]}, views=Diff{created=[], dropped=[], altered=[]},
types=Diff{created=[], dropped=[], altered=[]}, udfs=Diff{created=[],
dropped=[], altered=[]}, udas=Diff{created=[], dropped=[], altered=[]}}]}}
INFO [main] 2023-11-14 15:39:38,819 Keyspace.java:366 - Creating replication
strategy cql_test_keyspace_alt params KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}
INFO [main] 2023-11-14 15:39:38,819 ColumnFamilyStore.java:495 - Initializing
cql_test_keyspace_alt.table_26
DEBUG [main] 2023-11-14 15:39:38,819 DiskBoundaryManager.java:54 - Refreshing
disk boundary cache for cql_test_keyspace_alt.table_26
DEBUG [main] 2023-11-14 15:39:38,819 DiskBoundaryManager.java:93 - Got local
ranges [] (ringVersion = 0)
DEBUG [main] 2023-11-14 15:39:38,819 DiskBoundaryManager.java:57 - Updating
boundaries from null to
DiskBoundaries{directories=[DataDirectory{location=build/test/cassandra/data}],
positions=null, ringVersion=0, directoriesVersion=0} for
cql_test_keyspace_alt.table_26
INFO [main] 2023-11-14 15:39:38,821 ViewManager.java:127 - Not submitting
build tasks for views in keyspace cql_test_keyspace_alt as storage service is
not initialized
INFO [main] 2023-11-14 15:39:38,845 ColumnFamilyStore.java:1021 - Enqueuing
flush of cql_test_keyspace_alt.table_26, Reason: UNIT_TESTS, Usage: 309.570KiB
(0%) on-heap, 15.625KiB (0%) off-heap
INFO [PerDiskMemtableFlushWriter_0:2] 2023-11-14 15:39:38,854
Flushing.java:154 - Writing Memtable-table_26@1116421524(28.320KiB serialized
bytes, 1000 ops, 309.570KiB (0%) on-heap, 15.625KiB (0%) off-heap), flushed
range = [null, null)
INFO [PerDiskMemtableFlushWriter_0:2] 2023-11-14 15:39:38,855
Flushing.java:180 - Completed flushing
/home/jlewandowski/dev/cassandra/cassandra-5.0/build/test/cassandra/data/cql_test_keyspace_alt/table_26-a398db1082fb11eeb194e5f4789e9eaa/nc-1-big-Data.db
(27.743KiB) for commitlog position CommitLogPosition(segmentId=1699972753906,
position=65263)
INFO [MemtableFlushWriter:2] 2023-11-14 15:39:38,888 LogTransaction.java:249 -
Unfinished transaction log, deleting
/home/jlewandowski/dev/cassandra/cassandra-5.0/build/test/cassandra/data/cql_test_keyspace_alt/table_26-a398db1082fb11eeb194e5f4789e9eaa/nc_txn_flush_a39d20d0-82fb-11ee-b194-e5f4789e9eaa.log
DEBUG [MemtableFlushWriter:2] 2023-11-14 15:39:38,895
ColumnFamilyStore.java:1349 - Flushed to
[BigTableReader:big(path='/home/jlewandowski/dev/cassandra/cassandra-5.0/build/test/cassandra/data/cql_test_keyspace_alt/table_26-a398db1082fb11eeb194e5f4789e9eaa/nc-1-big-Data.db')]
(1 sstables, 31.852KiB), biggest 31.852KiB, smallest 31.852KiB
INFO [main] 2023-11-14 15:39:38,913 ColumnFamilyStore.java:1021 - Enqueuing
flush of cql_test_keyspace_alt.table_26, Reason: UNIT_TESTS, Usage: 309.570KiB
(0%) on-heap, 15.625KiB (0%) off-heap
INFO [PerDiskMemtableFlushWriter_0:1] 2023-11-14 15:39:38,921
Flushing.java:154 - Writing Memtable-table_26@1957761258(28.320KiB serialized
bytes, 1000 ops, 309.570KiB (0%) on-heap, 15.625KiB (0%) off-heap), flushed
range = [null, null)
INFO [PerDiskMemtableFlushWriter_0:1] 2023-11-14 15:39:38,923
Flushing.java:180 - Completed flushing
/home/jlewandowski/dev/cassandra/cassandra-5.0/build/test/cassandra/data/cql_test_keyspace_alt/table_26-a398db1082fb11eeb194e5f4789e9eaa/nc-2-big-Data.db
(27.427KiB) for commitlog position CommitLogPosition(segmentId=1699972753906,
position=126263)
INFO [MemtableFlushWriter:1] 2023-11-14 15:39:38,952
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785915#comment-17785915
]
Jacek Lewandowski edited comment on CASSANDRA-18710 at 11/14/23 2:34 PM:
-
I've managed to reproduce it locally with IntelliJ repeated test runs...
a full fresh log
{noformat}
DEBUG [main] 2023-11-14 15:32:22,442 DefaultSchemaUpdateHandler.java:259 -
Schema updated: SchemaTransformationResult{f8993d42-6a31-30e9-a22b-bb9151fe0d47
--> 0bf91067-a88d-31ab-9f12-21fe9470100b,
diff=KeyspacesDiff{created=[KeyspaceMetadata{name=cql_test_keyspace_alt,
kind=REGULAR, params=KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}, tables=[], views=[], functions=[], types=[]}],
dropped=[], altered=[]}}
INFO [main] 2023-11-14 15:32:22,442 Keyspace.java:366 - Creating replication
strategy cql_test_keyspace_alt params KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}
DEBUG [main] 2023-11-14 15:32:22,442 Keyspace.java:370 - New replication
settings for keyspace cql_test_keyspace_alt - invalidating disk boundary caches
DEBUG [main] 2023-11-14 15:32:22,443 TokenMetadata.java:858 - Starting pending
range calculation for cql_test_keyspace_alt
DEBUG [main] 2023-11-14 15:32:22,443 TokenMetadata.java:863 - Pending range
calculation for cql_test_keyspace_alt completed (took: 0ms)
INFO [main] 2023-11-14 15:32:22,443 CQLTester.java:1004 - CREATE TABLE
cql_test_keyspace_alt.table_59 (pk bigint, PRIMARY KEY (pk))
DEBUG [main] 2023-11-14 15:32:22,445 DefaultSchemaUpdateHandler.java:259 -
Schema updated: SchemaTransformationResult{0bf91067-a88d-31ab-9f12-21fe9470100b
--> 50c5fd2a-1feb-349a-bbd5-9bd9ddf90794, diff=KeyspacesDiff{created=[],
dropped=[],
altered=[KeyspaceDiff{before=KeyspaceMetadata{name=cql_test_keyspace_alt,
kind=REGULAR, params=KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}, tables=[], views=[], functions=[], types=[]},
after=KeyspaceMetadata{name=cql_test_keyspace_alt, kind=REGULAR,
params=KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}, tables=[cql_test_keyspace_alt.table_59], views=[],
functions=[], types=[]}, tables=Diff{created=[cql_test_keyspace_alt.table_59],
dropped=[], altered=[]}, views=Diff{created=[], dropped=[], altered=[]},
types=Diff{created=[], dropped=[], altered=[]}, udfs=Diff{created=[],
dropped=[], altered=[]}, udas=Diff{created=[], dropped=[], altered=[]}}]}}
INFO [main] 2023-11-14 15:32:22,445 Keyspace.java:366 - Creating replication
strategy cql_test_keyspace_alt params KeyspaceParams{durable_writes=true,
replication=ReplicationParams{class=org.apache.cassandra.locator.SimpleStrategy,
replication_factor=1}}
INFO [main] 2023-11-14 15:32:22,445 ColumnFamilyStore.java:495 - Initializing
cql_test_keyspace_alt.table_59
DEBUG [main] 2023-11-14 15:32:22,445 DiskBoundaryManager.java:54 - Refreshing
disk boundary cache for cql_test_keyspace_alt.table_59
DEBUG [main] 2023-11-14 15:32:22,445 DiskBoundaryManager.java:93 - Got local
ranges [] (ringVersion = 0)
DEBUG [main] 2023-11-14 15:32:22,445 DiskBoundaryManager.java:57 - Updating
boundaries from null to
DiskBoundaries{directories=[DataDirectory{location=build/test/cassandra/data}],
positions=null, ringVersion=0, directoriesVersion=0} for
cql_test_keyspace_alt.table_59
INFO [main] 2023-11-14 15:32:22,447 ViewManager.java:127 - Not submitting
build tasks for views in keyspace cql_test_keyspace_alt as storage service is
not initialized
INFO [main] 2023-11-14 15:32:22,464 ColumnFamilyStore.java:1021 - Enqueuing
flush of cql_test_keyspace_alt.table_59, Reason: UNIT_TESTS, Usage: 308.594KiB
(0%) on-heap, 15.625KiB (0%) off-heap
INFO [PerDiskMemtableFlushWriter_0:1] 2023-11-14 15:32:22,471
Flushing.java:154 - Writing Memtable-table_59@1863551891(28.320KiB serialized
bytes, 1000 ops, 308.594KiB (0%) on-heap, 15.625KiB (0%) off-heap), flushed
range = [null, null)
INFO [PerDiskMemtableFlushWriter_0:1] 2023-11-14 15:32:22,472
Flushing.java:180 - Completed flushing
/home/jlewandowski/dev/cassandra/cassandra-5.0/build/test/cassandra/data/cql_test_keyspace_alt/table_59-9f7f9ec082fa11eeac18a3d3983bd6d3/nc-1-big-Data.db
(27.334KiB) for commitlog position CommitLogPosition(segmentId=1699972288965,
position=63990)
INFO [MemtableFlushWriter:2] 2023-11-14 15:32:22,504 LogTransaction.java:249 -
Unfinished transaction log, deleting
/home/jlewandowski/dev/cassandra/cassandra-5.0/build/test/cassandra/data/cql_test_keyspace_alt/table_59-9f7f9ec082fa11eeac18a3d3983bd6d3/nc_txn_flush_9f82ac00-82fa-11ee-ac18-a3d3983bd6d3.log
DEBUG
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784223#comment-17784223
]
Brandon Williams edited comment on CASSANDRA-18710 at 11/8/23 10:43 PM:
I noticed that the refactoring around this in CASSANDRA-17071 kept the general
order of events but slightly changed the behavior, which I've restored
[here|https://github.com/driftx/cassandra/commit/b25d1819faf3a963e6e40a2afe6b9e648c4d2aee].
-As one might expect, I haven't been able to reproduce in 5.0 with that patch
([here|https://app.circleci.com/pipelines/github/driftx/cassandra/1362/workflows/ccc0dc0b-7342-4637-a983-160b3f3a6ea5]
or
[here|https://app.circleci.com/pipelines/github/driftx/cassandra/1362/workflows/c35b0923-d293-4197-a58d-7498ab499e09]).
If this looks good and will still work for CASSANDRA-17071's goals, I can
prepare the branches and run full CI.-
This did reproduce after I wrote that,
[here|https://app.circleci.com/pipelines/github/driftx/cassandra/1362/workflows/ccc0dc0b-7342-4637-a983-160b3f3a6ea5].
I'll have to dig into that.
was (Author: brandon.williams):
I noticed that the refactoring around this in CASSANDRA-17071 kept the general
order of events but slightly changed the behavior, which I've restored
[here|https://github.com/driftx/cassandra/commit/b25d1819faf3a963e6e40a2afe6b9e648c4d2aee].
As one might expect, I haven't been able to reproduce in 5.0 with that patch
([here|https://app.circleci.com/pipelines/github/driftx/cassandra/1362/workflows/ccc0dc0b-7342-4637-a983-160b3f3a6ea5]
or
[here|https://app.circleci.com/pipelines/github/driftx/cassandra/1362/workflows/c35b0923-d293-4197-a58d-7498ab499e09]).
If this looks good and will still work for CASSANDRA-17071's goals, I can
prepare the branches and run full CI.
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.1.x, 5.0-beta, 5.0.x, 5.x
>
> Attachments: org.apache.cassandra.io.DiskSpaceMetricsTest.txt
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17784223#comment-17784223
]
Brandon Williams edited comment on CASSANDRA-18710 at 11/8/23 10:22 PM:
I noticed that the refactoring around this in CASSANDRA-17071 kept the general
order of events but slightly changed the behavior, which I've restored
[here|https://github.com/driftx/cassandra/commit/b25d1819faf3a963e6e40a2afe6b9e648c4d2aee].
As one might expect, I haven't been able to reproduce in 5.0 with that patch
([here|https://app.circleci.com/pipelines/github/driftx/cassandra/1362/workflows/ccc0dc0b-7342-4637-a983-160b3f3a6ea5]
or
[here|https://app.circleci.com/pipelines/github/driftx/cassandra/1362/workflows/c35b0923-d293-4197-a58d-7498ab499e09]).
If this looks good and will still work for CASSANDRA-17071's goals, I can
prepare the branches and run full CI.
was (Author: brandon.williams):
I noticed that the refactoring around this in CASSANDRA-17071 kept the general
order of events but slightly changed the behavior, which I've restored
[here|https://github.com/driftx/cassandra/commit/b25d1819faf3a963e6e40a2afe6b9e648c4d2aee].
As one might expect, I haven't been able to reproduce in 5.0 with that patch
([here|https://app.circleci.com/pipelines/github/driftx/cassandra/1362/workflows/ccc0dc0b-7342-4637-a983-160b3f3a6ea5]
or
[here|https://app.circleci.com/pipelines/github/driftx/cassandra/1362/workflows/c35b0923-d293-4197-a58d-7498ab499e09].
If this looks good and will still work for CASSANDRA-17071's goals, I can
prepare the branches and run full CI.
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.1.x, 5.0-beta, 5.0.x, 5.x
>
> Attachments: org.apache.cassandra.io.DiskSpaceMetricsTest.txt
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782331#comment-17782331
]
Ekaterina Dimitrova edited comment on CASSANDRA-18710 at 11/2/23 9:21 PM:
--
{quote}After CASSANDRA-17071 that has moved to CFS.onTableDropped and it seems
the timing of this now does cause the small flushes. I'm not super strong in
this area of the code but it seems like maybe this could be massaged again to
prevent this from happening as before?
{quote}
So if I read correctly, this _may be_ a 4.1+ regression?
[~jlewandowski] , [~blambov] , do you mind to take a look and advise, please?
was (Author: e.dimitrova):
{quote}After CASSANDRA-17071 that has moved to CFS.onTableDropped and it seems
the timing of this now does cause the small flushes. I'm not super strong in
this area of the code but it seems like maybe this could be massaged again to
prevent this from happening as before?
{quote}
So if I read correctly this is 4.1+ regression?
[~jlewandowski] , [~blambov] , do you mind to take a look and advise, please?
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-beta, 5.0.x, 5.x
>
> Attachments: org.apache.cassandra.io.DiskSpaceMetricsTest.txt
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778792#comment-17778792
]
Brandon Williams edited comment on CASSANDRA-18710 at 10/23/23 7:14 PM:
Dropping a CF eventually leads to CFS.onTableDropped, which then [force
recycles|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L3379]
the commit log segments, which leads to our flushes with COMMITLOG_DIRTY.
This was added in CASSANDRA-17071 which is in 4.1, though.
was (Author: brandon.williams):
Dropping a CF eventually leads to CFS.onDropped, which then [force
recycles|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L3379]
the commit log segments, which leads to our flushes with COMMITLOG_DIRTY.
This was added in CASSANDRA-17071 which is in 4.1, though.
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Attachments: org.apache.cassandra.io.DiskSpaceMetricsTest.txt
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778792#comment-17778792
]
Brandon Williams edited comment on CASSANDRA-18710 at 10/23/23 7:11 PM:
Dropping a CF eventually leads to CFS.onDropped, which then [force
recycles|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L3379]
the commit log segments, which leads to our flushes with COMMITLOG_DIRTY.
This was added in CASSANDRA-17071 which is in 4.1, though.
was (Author: brandon.williams):
Dropping a CF eventually leads to CFS.onDropped, which then [force
recycles|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L3379]
the commit log segments, which leads to our flushes with COMMITLOG_DIRTY.
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Attachments: org.apache.cassandra.io.DiskSpaceMetricsTest.txt
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774918#comment-17774918
]
Brandon Williams edited comment on CASSANDRA-18710 at 10/13/23 2:06 PM:
Running with [this
patch|https://github.com/driftx/cassandra/commit/286bb5cd7c36c62e541cc79b025931215a982bc3],
I've managed to reproduce, and it indicates the culprit sstable:
bq. [junit-timeout] INFO [main] 2023-10-12 21:55:12,181
DiskSpaceMetricsTest.java:125 - smallest sstable is
/tmp/cassandra/build/test/cassandra/data/cql_test_keyspace_alt/table_01-03e61210694a11eeb4091bdb4ac3170b/nc-1-big-Data.db
at 2329 bytes
If we grep the log for that sstable:
{quote}
[junit-timeout] INFO [PerDiskMemtableFlushWriter_0:2] 2023-10-12 21:55:11,128
Flushing.java:180 - Completed flushing
/tmp/cassandra/build/test/cassandra/data/cql_test_keyspace_alt/table_01-03e61210694a11eeb4091bdb4ac3170b/nc-1-big-Data.db
(6.839KiB) for commitlog position CommitLogPosition(segmentId=1697147706890,
position=211)
[junit-timeout] DEBUG [MemtableFlushWriter:2] 2023-10-12 21:55:11,177
ColumnFamilyStore.java:1345 - Flushed to
[BigTableReader:big(path='/tmp/cassandra/build/test/cassandra/data/cql_test_keyspace_alt/table_01-03e61210694a11eeb4091bdb4ac3170b/nc-1-big-Data.db')]
(1 sstables, 11.232KiB), biggest 11.232KiB, smallest 11.232KiB
[junit-timeout] INFO [main] 2023-10-12 21:55:12,181
DiskSpaceMetricsTest.java:125 - smallest sstable is
/tmp/cassandra/build/test/cassandra/data/cql_test_keyspace_alt/table_01-03e61210694a11eeb4091bdb4ac3170b/nc-1-big-Data.db
at 2329 bytes
{quote}
It looks like SSTR.onDiskLength() and bytesOnDisk() disagree at some point,
which seems like a bug. [~blambov] can you take a look? I've uploaded the full
log from the failure.
was (Author: brandon.williams):
Running with [this
patch|https://github.com/driftx/cassandra/commit/286bb5cd7c36c62e541cc79b025931215a982bc3],
I've managed to reproduce, and it indicates the culprit sstable:
bq. [junit-timeout] INFO [main] 2023-10-12 21:55:12,181
DiskSpaceMetricsTest.java:125 - smallest sstable is
/tmp/cassandra/build/test/cassandra/data/cql_test_keyspace_alt/table_01-03e61210694a11eeb4091bdb4ac3170b/nc-1-big-Data.db
at 2329 bytes
If we grep the log for that sstable:
{quote}
[junit-timeout] INFO [PerDiskMemtableFlushWriter_0:2] 2023-10-12 21:55:11,128
Flushing.java:180 - Completed flushing
/tmp/cassandra/build/test/cassandra/data/cql_test_keyspace_alt/table_01-03e61210694a11eeb4091bdb4ac3170b/nc-1-big-Data.db
(6.839KiB) for commitlog position CommitLogPosition(segmentId=1697147706890,
position=211)
[junit-timeout] DEBUG [MemtableFlushWriter:2] 2023-10-12 21:55:11,177
ColumnFamilyStore.java:1345 - Flushed to
[BigTableReader:big(path='/tmp/cassandra/build/test/cassandra/data/cql_test_keyspace_alt/table_01-03e61210694a11eeb4091bdb4ac3170b/nc-1-big-Data.db')]
(1 sstables, 11.232KiB), biggest 11.232KiB, smallest 11.232KiB
[junit-timeout] INFO [main] 2023-10-12 21:55:12,181
DiskSpaceMetricsTest.java:125 - smallest sstable is
/tmp/cassandra/build/test/cassandra/data/cql_test_keyspace_alt/table_01-03e61210694a11eeb4091bdb4ac3170b/nc-1-big-Data.db
at 2329 bytes
{quote}
It looks like SSTR.onDiskLength() and bytesOnDisk() disagree at some point,
which seems like a bug. [~blambov] can you take a look?
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Attachments: org.apache.cassandra.io.DiskSpaceMetricsTest.txt
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CASSANDRA-18710) Test failure: org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
[
https://issues.apache.org/jira/browse/CASSANDRA-18710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750466#comment-17750466
]
Ekaterina Dimitrova edited comment on CASSANDRA-18710 at 8/2/23 8:11 PM:
-
I am not able to reproduce the issue in CircleCI even before the patch.
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18710-trunk]
But it seems reasonable to me. Also, I can see how timing in Jenkins could
influence it to fail that way. I think the flush was just forgotten when the
test was written, +1
was (Author: e.dimitrova):
I am not able to reproduce the issue in CircleCI even before the patch.
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18710-trunk]
But it seems reasonable to me. Also, I can see how timing in Jenkins could
influence it to fail that way. I think the flush was just forgotten, +1
> Test failure:
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize-.jdk17 (from
> org.apache.cassandra.io.DiskSpaceMetricsTest-.jdk17)
> --
>
> Key: CASSANDRA-18710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18710
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.x
>
>
> Seen here:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1644/testReport/org.apache.cassandra.io/DiskSpaceMetricsTest/testFlushSize__jdk17/]
> h3.
> {code:java}
> Error Message
> expected:<7200.0> but was:<1367.83970468544>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<7200.0> but
> was:<1367.83970468544> at
> org.apache.cassandra.io.DiskSpaceMetricsTest.testFlushSize(DiskSpaceMetricsTest.java:119)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
