[
https://issues.apache.org/jira/browse/CASSANDRA-19601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847016#comment-17847016
]
Brandon Williams commented on CASSANDRA-19601:
----------------------------------------------
I have found that the flush is actually caused by AbstractCommitLogService
every 5 minutes, regardless of CL mode. In both direct and legacy mode there
is system table data being written on each flush. The difference between them
though is that legacy mode allocates the full segment size on disk before using
it, where direct mode allocates 4k and then it grows with usage, which is what
occasionally trips the test when it is run and intersects with that 5 minute
interval.
bq. one thing we can try is to check that the commit log's dirty regions don't
contain anything from that keyspace
I think that's what I'll have to do, since this discovery shows how measuring
segment size is not a valid way to determine if anything was written at all,
let alone know if it was appropriate or not, and this test relies on that
exclusively.
All of that said, I do think it would be better if direct mode had the same
on-disk semantics as legacy mode, but it's not a huge deal.
> Test failure: test_change_durable_writes
> ----------------------------------------
>
> Key: CASSANDRA-19601
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19601
> Project: Cassandra
> Issue Type: Bug
> Components: CI
> Reporter: Ekaterina Dimitrova
> Assignee: Brandon Williams
> Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> Failing on trunk:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/1880/testReport/junit/dtest-latest.configuration_test/TestConfiguration/Tests___dtest_latest_jdk11_31_64___test_change_durable_writes/]
> [https://app.circleci.com/pipelines/github/blerer/cassandra/400/workflows/893a0edb-9181-4981-b542-77228c8bc975/jobs/10941/tests]
> {code:java}
> AssertionError: Commitlog was written with durable writes disabled
> assert 90112 == 86016
> +90112
> -86016
> self = <configuration_test.TestConfiguration object at 0x7f5eda736550>
> @pytest.mark.timeout(60*30)
> def test_change_durable_writes(self):
> """
> @jira_ticket CASSANDRA-9560
>
> Test that changes to the DURABLE_WRITES option on keyspaces is
> respected in subsequent writes.
>
> This test starts by writing a dataset to a cluster and asserting that
> the commitlogs have been written to. The subsequent test depends on
> the assumption that this dataset triggers an fsync.
>
> After checking this assumption, the test destroys the cluster and
> creates a fresh one. Then it tests that DURABLE_WRITES is respected
> by:
>
> - creating a keyspace with DURABLE_WRITES set to false,
> - using ALTER KEYSPACE to set its DURABLE_WRITES option to true,
> - writing a dataset to this keyspace that is known to trigger a
> commitlog fsync,
> - asserting that the commitlog has grown in size since the data was
> written.
> """
> cluster = self.cluster
> cluster.set_batch_commitlog(enabled=True, use_batch_window =
> cluster.version() < '5.0')
>
> cluster.set_configuration_options(values={'commitlog_segment_size_in_mb': 1})
>
> cluster.populate(1).start()
> durable_node = cluster.nodelist()[0]
>
> durable_init_size = commitlog_size(durable_node)
> durable_session = self.patient_exclusive_cql_connection(durable_node)
>
> # test assumption that write_to_trigger_fsync actually triggers a
> commitlog fsync
> durable_session.execute("CREATE KEYSPACE ks WITH REPLICATION =
> {'class': 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = true")
> durable_session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a
> int, b int, c int)')
> logger.debug('commitlog size diff = ' +
> str(commitlog_size(durable_node) - durable_init_size))
> write_to_trigger_fsync(durable_session, 'ks', 'tab')
> logger.debug('commitlog size diff = ' +
> str(commitlog_size(durable_node) - durable_init_size))
>
> assert commitlog_size(durable_node) > durable_init_size, \
> "This test will not work in this environment;
> write_to_trigger_fsync does not trigger fsync."
>
> durable_session.shutdown()
> cluster.stop()
> cluster.clear()
>
> cluster.set_batch_commitlog(enabled=True, use_batch_window =
> cluster.version() < '5.0')
>
> cluster.set_configuration_options(values={'commitlog_segment_size_in_mb': 1})
> cluster.start()
> node = cluster.nodelist()[0]
> session = self.patient_exclusive_cql_connection(node)
>
> # set up a keyspace without durable writes, then alter it to use them
> session.execute("CREATE KEYSPACE ks WITH REPLICATION = {'class':
> 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = false")
> session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a int, b
> int, c int)')
> init_size = commitlog_size(node)
> write_to_trigger_fsync(session, 'ks', 'tab')
> > assert commitlog_size(node) == init_size, "Commitlog was written with
> > durable writes disabled"
> E AssertionError: Commitlog was written with durable writes disabled
> E assert 90112 == 86016
> E +90112
> E -86016
> configuration_test.py:104: AssertionError
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
