[
https://issues.apache.org/jira/browse/CASSANDRA-18098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846429#comment-17846429
]
Brandon Williams edited comment on CASSANDRA-18098 at 5/14/24 8:32 PM:
-----------------------------------------------------------------------
>From the failure in CASSANDRA-15439:
{noformat}
19:25:25,570 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-19-big-Data.db
19:25:25,570 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-16-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-17-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-18-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-23-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-2-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-5-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-8-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-11-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-14-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-3-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-6-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-24-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-20-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-21-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-22-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-25-big-Data.db
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,329 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-13-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,545 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-7-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,682 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-10-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,841 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-4-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,965 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-15-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,140 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-9-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,286 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-12-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,429 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-1-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,576 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-6-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,661 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-3-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,840 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-14-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:26,039 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-8-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:26,164 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-11-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:26,358 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-5-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:26,479 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-2-big
{noformat}
If we look at the log timestamps, we can see the dtest failing at 19:25:25,570
and sstables 2, 3, 5, 6, 8, 11, and 14 are part of the existing set, but these
are all deleted at a slightly later time in the corresponding system log,
beginning with the deletion of sstable 6 at 19:25:25,576.
>From the failure in CASSANDRA-19557:
{noformat}
20:56:58,522 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-1-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-4-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-7-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-10-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-13-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-2-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-5-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-8-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-11-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-14-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-3-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-6-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-9-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-12-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-15-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-16-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-17-big-Data.db
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:58,521 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-12-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:58,700 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-9-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:58,870 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-15-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,040 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-6-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,176 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-3-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,336 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-10-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,473 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-7-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,600 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-13-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,736 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-4-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,862 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-1-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,948 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-11-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:57:00,041 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-8-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:57:00,122 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-14-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:57:00,206 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-5-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:57:00,293 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-2-big
{noformat}
Here we can observe the same behavior, but more egregious as all the sstables
were deleted after the test failed. I think our removal semantics must have
changed at some point and the test just needs to wait a little longer before
doing its check. I'll work on modifying it to accommodate this.
was (Author: brandon.williams):
>From the failure in CASSANDRA-15439:
{noformat}
19:25:25,570 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-19-big-Data.db
19:25:25,570 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-16-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-17-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-18-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-23-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-2-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-5-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-8-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-11-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-14-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-3-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-6-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-24-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-20-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-21-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-22-big-Data.db
19:25:25,571 bootstrap_test ERROR
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-25-big-Data.db
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,329 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-13-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,545 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-7-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,682 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-10-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,841 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-4-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:24,965 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-15-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,140 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-9-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,286 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-12-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,429 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data0/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-1-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,576 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-6-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,661 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data2/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-3-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:25,840 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-14-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:26,039 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-8-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:26,164 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-11-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:26,358 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-5-big
INFO [NonPeriodicTasks:1] 2024-04-25 19:25:26,479 BigFormat.java:231 -
Deleting sstable:
/tmp/dtest-fezk9v8f/test/node1/data1/keyspace1/standard1-3a213a10033911efaffa37bdbd08956d/nb-2-big
{noformat}
If we look at the log timestamps, we can see the dtest failing at 19:25:25,570
and sstables 2, 3, 5, 6, 8, 11, and 14 are part of the existing set, but these
are all deleted at a slightly later time in the corresponding system log,
beginning with the deletion of sstable 6 at 19:25:25,576.
>From the failure in CASSANDRA-19557:
{noformat}
20:56:58,522 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-1-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-4-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-7-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-10-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-13-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-2-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-5-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-8-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-11-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-14-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-3-big-Data.db
20:56:58,523 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-6-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-9-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-12-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-15-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-16-big-Data.db
20:56:58,524 bootstrap_test ERROR
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-17-big-Data.db
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:58,521 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-12-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:58,700 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-9-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:58,870 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-15-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,040 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-6-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,176 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data2/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-3-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,336 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-10-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,473 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-7-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,600 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-13-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,736 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-4-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,862 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data0/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-1-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:56:59,948 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-11-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:57:00,041 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-8-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:57:00,122 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-14-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:57:00,206 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-5-big
INFO [NonPeriodicTasks:1] 2024-05-03 20:57:00,293 BigFormat.java:324 -
Deleting sstable:
/tmp/dtest-zisjefyg/test/node1/data1/keyspace1/standard1-1b255f4def2540a60000000000000005/oa-2-big
{noformat}
Here we can observe the same behavior, but more egregious as all the sstables
were deleted after the test failed. I think our removal semantics must have
changed at point and the test just needs to wait a little longer before doing
its check. I'll work on modifying it to accommodate this.
> Test failure bootstrap_test.py::test_cleanup
> --------------------------------------------
>
> Key: CASSANDRA-18098
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18098
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
> Reporter: Yifan Cai
> Assignee: Brandon Williams
> Priority: Normal
> Fix For: 4.0.x, 5.0.x, 5.x
>
>
> The test failed a few times in the recent CI runs. For example, this log
> captures a recent failure.
> {code:none}
> 20:02:01,364 ccm INFO node1: using Java 11 for the current invocation
> 20:02:02,679 bootstrap_test ERROR ---
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data0/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-1-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data0/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-4-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data0/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-7-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data0/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-10-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data0/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-13-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data1/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-2-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data1/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-5-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data1/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-8-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data1/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-11-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data1/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-14-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data2/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-3-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data2/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-6-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data2/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-9-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data2/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-12-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data2/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-15-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data2/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-16-big-Data.db
> 20:02:02,679 bootstrap_test ERROR
> /tmp/dtest-8kcle23s/test/node1/data2/keyspace1/standard1-e36da130727b11edb08827a767e354f3/nb-17-big-Data.db
> 20:02:02,679 bootstrap_test ERROR Current count is 17, basecount was 15
> -- generated xml file: /tmp/results/dtests/pytest_result_j11_with_vnodes.xml
> ---
> ===Flaky Test Report===
> test_materialized_views_auth passed 1 out of the required 1 times. Success!
> test_cleanup failed and was not selected for rerun.
> <class 'AssertionError'>
> assert not True
> + where True = <bound method Event.is_set of <threading.Event object at
> 0x7f071d43cba8>>()
> + where <bound method Event.is_set of <threading.Event object at
> 0x7f071d43cba8>> = <threading.Event object at 0x7f071d43cba8>.is_set
> [<TracebackEntry /home/cassandra/cassandra-dtest/bootstrap_test.py:912>]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
