[
https://issues.apache.org/jira/browse/CASSANDRA-21152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18057736#comment-18057736
]
Arvind Kandpal commented on CASSANDRA-21152:
--------------------------------------------
Hi [~dnk],
I have reproduced the issue locally using the provided test case and try to
resolve it and here is what I do:
Findings & Fix: The CursorCompactor was missing the necessary logic to handle
"Shadowable Tombstones" required for Materialized Views, and it also missed a
check for expired liveness info during row merging.
I have implemented the following changes in CursorCompactor.java:
# Disabled Cursor Compaction for MVs: Updated isSupported() to return false if
metadata.isView(). This ensures MVs fall back to the standard
CompactionIterator, which correctly handles shadowing.
# Fixed Expired Row Handling: Updated mergeRows() to explicitly check if a row
is expired (isExpiring() && !isLive()). This prevents "zombie" rows from
persisting in standard tables when using cursor compaction.
Verification: I ran the reproducer test with cursor compaction enabled:
{code}
ant testsome -Dtest.name=org.apache.cassandra.cql3.ViewFlushTest
-Dtest.methods=testView -Dcassandra.cursor_compaction_enabled=true
{code}
Result: ( last lines )
{code:java}
[junit-timeout] DEBUG [MemtablePostFlush:1] 2026-02-11T05:58:48,501
AbstractCommitLogSegmentManager.java:414 - Segment
CommitLogSegment(build/test/cassandra/commitlog/CommitLog-9-1770789510412.log)
is no longer active and will be deleted now
[junit-timeout] INFO [NonPeriodicTasks:1] 2026-02-11T05:58:48,642
BigFormat.java:324 - Deleting sstable:
/mnt/backup_hdd/cassandra-github-code/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testview_00-1b255f4def2540a60000000000000006/pa-1-big
[junit-timeout] INFO [NonPeriodicTasks:1] 2026-02-11T05:58:48,775
LogTransaction.java:266 - Unfinished transaction log, deleting
/mnt/backup_hdd/cassandra-github-code/cassandra/build/test/cassandra/data/cql_test_keyspace/table_testview_00-1b255f4def2540a60000000000000006/pa_txn_unknowncompactiontype_bbbe6630-070e-11f1-b164-a9a31e237e3e.log
[junit-timeout] INFO [GlobalLogFollower] 2026-02-11T05:58:48,853
LocalLog.java:526 - Enacted EXECUTED {ForceSnapshot{epoch=Epoch{epoch=9}}}. New
tail is Epoch{epoch=9}
[junit-timeout] DEBUG [Test Cluster-nio-worker-0] 2026-02-11T05:58:48,863
Cluster.java:2738 - Received event EVENT DROPPED TABLE
cql_test_keyspace.mv_testview_01, scheduling delivery
[junit-timeout] DEBUG [Test Cluster-nio-worker-0] 2026-02-11T05:58:48,867
Cluster.java:2738 - Received event EVENT DROPPED TABLE
cql_test_keyspace.table_testview_00, scheduling delivery
[junit-timeout] DEBUG [GlobalLogFollower] 2026-02-11T05:58:48,871
ClientNotificationListener.java:55 - Maybe notify listeners about []
[junit-timeout] DEBUG [Test Cluster-nio-worker-0] 2026-02-11T05:58:48,872
Cluster.java:2738 - Received event EVENT DROPPED KEYSPACE cql_test_keyspace,
scheduling delivery
[junit-timeout] DEBUG [Test Cluster-nio-worker-0] 2026-02-11T05:58:48,874
Cluster.java:2738 - Received event EVENT DROPPED KEYSPACE
cql_test_keyspace_alt, scheduling delivery
[junit-timeout] DEBUG [main] 2026-02-11T05:58:48,878 Connection.java:894 -
Connection[localhost/127.0.0.1:43085-2, inFlight=0, closed=true] closing
connection
[junit-timeout] DEBUG [main] 2026-02-11T05:58:48,879 ConvictionPolicy.java:93 -
[localhost/127.0.0.1:43085] Connection[localhost/127.0.0.1:43085-2, inFlight=0,
closed=true] closed, remaining = 1
[junit-timeout] DEBUG [main] 2026-02-11T05:58:48,883 Cluster.java:1917 -
Shutting down
[junit-timeout] DEBUG [main] 2026-02-11T05:58:48,884 Connection.java:894 -
Connection[localhost/127.0.0.1:43085-1, inFlight=0, closed=true] closing
connection
[junit-timeout] DEBUG [main] 2026-02-11T05:58:48,884 ConvictionPolicy.java:93 -
[localhost/127.0.0.1:43085] Connection[localhost/127.0.0.1:43085-1, inFlight=0,
closed=true] closed, remaining = 0
[junit-timeout] DEBUG [Test Cluster-nio-worker-1] 2026-02-11T05:58:48,899
PoolThreadCache.java:221 - Freed 7 thread-local buffer(s) from thread: Test
Cluster-nio-worker-1
[junit-timeout] DEBUG [Test Cluster-nio-worker-0] 2026-02-11T05:58:48,900
PoolThreadCache.java:221 - Freed 14 thread-local buffer(s) from thread: Test
Cluster-nio-worker-0
[junit-timeout] INFO [main] 2026-02-11T05:58:48,912 Server.java:220 - Stop
listening for CQL clients
[junit-timeout] ------------- ---------------- ---------------
[delete] Deleting directory
/mnt/backup_hdd/cassandra-github-code/cassandra/build/test/cassandra/commitlog
[delete] Deleting directory
/mnt/backup_hdd/cassandra-github-code/cassandra/build/test/cassandra/data
[delete] Deleting directory
/mnt/backup_hdd/cassandra-github-code/cassandra/build/test/cassandra/saved_caches
[delete] Deleting directory
/mnt/backup_hdd/cassandra-github-code/cassandra/build/test/cassandra/hints
BUILD SUCCESSFUL
Total time: 1 minute 30 seconds
{code}
Can I raise the PR ?
> Test failure: dtest.TestMaterializedViews.test_mv_with_default_ttl_with_flush
> ------------------------------------------------------------------------------
>
> Key: CASSANDRA-21152
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21152
> Project: Apache Cassandra
> Issue Type: Bug
> Reporter: Dmitry Konstantinov
> Assignee: Dmitry Konstantinov
> Priority: Normal
> Fix For: 5.x
>
> Attachments: breaking_point.png
>
>
> It fails consistently now, example:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/2391/testReport/junit/dtest.materialized_views_test/TestMaterializedViews/Tests___dtest_jdk11_1_64___test_mv_with_default_ttl_with_flush/]
> {code:java}
> self = <materialized_views_test.TestMaterializedViews object at
> 0x7f808eb3b3d0>
> @since('3.0')
> def test_mv_with_default_ttl_with_flush(self):
> > self._test_mv_with_default_ttl(True)
> materialized_views_test.py:1333:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> materialized_views_test.py:1368: in _test_mv_with_default_ttl
> assert_none(session, "SELECT k,a,b FROM mv2")
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> session = <cassandra.cluster.Session object at 0x7f808dcc1d60>
> query = 'SELECT k,a,b FROM mv2', cl = None
> def assert_none(session, query, cl=None):
> """
> Assert query returns nothing
> @param session Session to use
> @param query Query to run
> @param cl Optional Consistency Level setting. Default ONE
>
> Examples:
> assert_none(self.session1, "SELECT * FROM test where key=2;")
> assert_none(cursor, "SELECT * FROM test WHERE k=2",
> cl=ConsistencyLevel.SERIAL)
> """
> simple_query = SimpleStatement(query, consistency_level=cl)
> res = session.execute(simple_query)
> list_res = _rows_to_list(res)
> > assert list_res == [], "Expected nothing from {}, but got
> > {}".format(query, list_res)
> E AssertionError: Expected nothing from SELECT k,a,b FROM mv2, but got
> [[1, 1, None]]
> tools/assertions.py:149: AssertionError
> {code}
> !breaking_point.png|width=700!
> it was broken between 2364 and 2367 Cassandra trunk runs.
> [https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-trunk/failure/materialized_views_test/TestMaterializedViews/test_mv_with_default_ttl_with_flush]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
