[
https://issues.apache.org/jira/browse/CASSANDRA-18676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747860#comment-17747860
]
Berenguer Blasi edited comment on CASSANDRA-18676 at 7/27/23 9:30 AM:
----------------------------------------------------------------------
It seems our merges yesterday broke some negative timestamps upgrade tests.
Trunk fails locally for me with:
_pytest -vv --log-cli-level=DEBUG --junit-xml=nosetests.xml
--junit-prefix=dtest-upgrade -s --cassandra-dir=../18648_2
--execute-upgrade-tests-only --upgrade-target-version-only
--upgrade-version-selection all
upgrade_tests/cql_tests.py::TestCQLNodes2RF1_Upgrade_indev_4_1_x_To_indev_trunk::test_negative_timestamp_
{noformat}
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/bereng/.pyenv/versions/3.8.13/lib/python3.8/threading.py", line
932, in _bootstrap_inner
self.run()
File
"/home/bereng/work/repos/bdpWS/dtests/lib/python3.8/site-packages/ccmlib/cluster.py",
line 198, in run
self.scan_and_report()
File
"/home/bereng/work/repos/bdpWS/dtests/lib/python3.8/site-packages/ccmlib/cluster.py",
line 186, in scan_and_report
on_error_call(errordata)
File "/home/bereng/work/repos/bdpWS/dtests/dtest_setup.py", line 177, in
_log_error_handler
pytest.fail("Error details: \n{message}".format(message=message))
File
"/home/bereng/work/repos/bdpWS/dtests/lib/python3.8/site-packages/_pytest/outcomes.py",
line 97, in fail
raise Failed(msg=msg, pytrace=pytrace)
Failed: Error details:
Errors seen in logs for: node1
node1: ERROR [Native-Transport-Requests-1] 2023-07-27 10:53:36,063
ExceptionHandlers.java:229 - Unexpected exception during request; channel =
[id: 0xe1aa274c, L:/127.0.0.1:9042 - R:/127.0.0.1:41778]
java.io.IOError: java.io.IOException: Error building row with data deserialized
from org.apache.cassandra.io.util.DataInputBuffer@90c0103
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:244)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:232)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
at
org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:50)
at
org.apache.cassandra.db.transform.EmptyPartitionsDiscarder.applyToPartition(EmptyPartitionsDiscarder.java:27)
at
org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:95)
at
org.apache.cassandra.service.StorageProxy$6.hasNext(StorageProxy.java:2065)
at
org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:91)
at
org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:931)
at
org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:506)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:481)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:299)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:105)
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:258)
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:351)
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:338)
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116)
at
org.apache.cassandra.transport.Message$Request.execute(Message.java:256)
at
org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:194)
at
org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:213)
at
org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:240)
at
org.apache.cassandra.transport.Dispatcher$RequestProcessor.run(Dispatcher.java:137)
at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.IOException: Error building row with data deserialized from
org.apache.cassandra.io.util.DataInputBuffer@90c0103
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:642)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeOne(UnfilteredSerializer.java:480)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:436)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:239)
... 28 common frames omitted
Caused by: java.lang.AssertionError: Invalid negative timestamp -42
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:592)
... 31 common frames omitted
10:53:36,283 dtest_setup DEBUG removing ccm cluster test at: /tmp/dtest-88aeph3k
10:53:36,527 dtest_setup DEBUG clearing ssl stores from [/tmp/dtest-88aeph3k]
directory
upgrade_tests/cql_tests.py::TestCQLNodes2RF1_Upgrade_indev_4_1_x_To_indev_trunk::test_negative_timestamp
ERROR
{noformat}
was (Author: bereng):
It seems our merges yesterday broke some negative timestamps upgrade tests.
Trunk fails locally for me with:
{noformat}
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/bereng/.pyenv/versions/3.8.13/lib/python3.8/threading.py", line
932, in _bootstrap_inner
self.run()
File
"/home/bereng/work/repos/bdpWS/dtests/lib/python3.8/site-packages/ccmlib/cluster.py",
line 198, in run
self.scan_and_report()
File
"/home/bereng/work/repos/bdpWS/dtests/lib/python3.8/site-packages/ccmlib/cluster.py",
line 186, in scan_and_report
on_error_call(errordata)
File "/home/bereng/work/repos/bdpWS/dtests/dtest_setup.py", line 177, in
_log_error_handler
pytest.fail("Error details: \n{message}".format(message=message))
File
"/home/bereng/work/repos/bdpWS/dtests/lib/python3.8/site-packages/_pytest/outcomes.py",
line 97, in fail
raise Failed(msg=msg, pytrace=pytrace)
Failed: Error details:
Errors seen in logs for: node1
node1: ERROR [Native-Transport-Requests-1] 2023-07-27 10:53:36,063
ExceptionHandlers.java:229 - Unexpected exception during request; channel =
[id: 0xe1aa274c, L:/127.0.0.1:9042 - R:/127.0.0.1:41778]
java.io.IOError: java.io.IOException: Error building row with data deserialized
from org.apache.cassandra.io.util.DataInputBuffer@90c0103
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:244)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:232)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
at
org.apache.cassandra.db.transform.FilteredRows.isEmpty(FilteredRows.java:50)
at
org.apache.cassandra.db.transform.EmptyPartitionsDiscarder.applyToPartition(EmptyPartitionsDiscarder.java:27)
at
org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:95)
at
org.apache.cassandra.service.StorageProxy$6.hasNext(StorageProxy.java:2065)
at
org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:91)
at
org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:931)
at
org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:506)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:481)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:299)
at
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:105)
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:258)
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:351)
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:338)
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116)
at
org.apache.cassandra.transport.Message$Request.execute(Message.java:256)
at
org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:194)
at
org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:213)
at
org.apache.cassandra.transport.Dispatcher.processRequest(Dispatcher.java:240)
at
org.apache.cassandra.transport.Dispatcher$RequestProcessor.run(Dispatcher.java:137)
at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:96)
at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:61)
at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:71)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.IOException: Error building row with data deserialized from
org.apache.cassandra.io.util.DataInputBuffer@90c0103
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:642)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeOne(UnfilteredSerializer.java:480)
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:436)
at
org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer$1.computeNext(UnfilteredRowIteratorSerializer.java:239)
... 28 common frames omitted
Caused by: java.lang.AssertionError: Invalid negative timestamp -42
at
org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeRowBody(UnfilteredSerializer.java:592)
... 31 common frames omitted
10:53:36,283 dtest_setup DEBUG removing ccm cluster test at: /tmp/dtest-88aeph3k
10:53:36,527 dtest_setup DEBUG clearing ssl stores from [/tmp/dtest-88aeph3k]
directory
upgrade_tests/cql_tests.py::TestCQLNodes2RF1_Upgrade_indev_4_1_x_To_indev_trunk::test_negative_timestamp
ERROR
{noformat}
> CorruptedSSTablesCompactionsTest is flaky
> -----------------------------------------
>
> Key: CASSANDRA-18676
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18676
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable, Test/unit
> Reporter: Branimir Lambov
> Assignee: Branimir Lambov
> Priority: Normal
> Fix For: 5.0
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> See e.g. [this
> run|https://app.circleci.com/pipelines/github/blambov/cassandra/510/workflows/fd484f76-b0f0-48c9-8672-d73bdc36b8ec/jobs/13575/tests].
> The test was looked at for CASSANDRA-15879, but it is still failing from time
> to time. One of the failures appears to be introduced by CASSANDRA-14227 and
> the others by CASSANDRA-18134. The failures are genuine problems with
> handling corruption, not just test issues.
> The {{CorruptSSTableException}} paths in {{SSTableIdentityIterator}} should
> likely also catch {{AssertionError}} and {{IllegalArgumentException}}, and
> most probably the tombstone verification should be done on the read path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
