[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835853#comment-17835853 ] Brandon Williams commented on CASSANDRA-18325: -- All that seemed to capture is what we already know, the thread failed. There is no dtest log. > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829370#comment-17829370 ] Caleb Rackliffe commented on CASSANDRA-18325: - I saw this again while testing CASSANDRA-19461. All my logs are attached to that ticket... > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772436#comment-17772436 ] Berenguer Blasi commented on CASSANDRA-18325: - If there's nothing to go about, no logs, no repro... you're stuck. There's nothing else we can do atm. +1 let's close. > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17772393#comment-17772393 ] Brandon Williams commented on CASSANDRA-18325: -- I did one last [2.5k attempt|https://app.circleci.com/pipelines/github/driftx/cassandra/1326/workflows/2675b50f-c81e-4bf2-8723-8cf4e12137f8/jobs/57002/steps] with no luck, and it's possible this was solved elsewhere so let's close. > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767508#comment-17767508 ] Berenguer Blasi commented on CASSANDRA-18325: - Yep the problem comes when you have sthg failing only in jenkins, which seems to be the case now?, you need to cross fingers logs tell you sthg. Otherwise you're stuck. > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767505#comment-17767505 ] Brandon Williams commented on CASSANDRA-18325: -- It actually doesn't repro in 1k runs, at least when I tried. But finding the log that is clearly incorrect makes me think we should give this some time to pop up again. > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767467#comment-17767467 ] Berenguer Blasi commented on CASSANDRA-18325: - If it doesn't repro in 500 repeats I'd say we close it. Just bc of the diminishing returns rule. When/id it raises again then that's a different story. My 2cts > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766846#comment-17766846 ] Brandon Williams commented on CASSANDRA-18325: -- bq. I can confirm I cannot reproduce it 500 times Just realized I misread this and thought it would repro more easily, which it does not. Maybe CASSANDRA-18792 did fix it, though I personally doubt that now. Here's the relevant log I dug out from nightlies: {quote} 02:15:14,90 bootstrap_test ERROR --- 02:15:14,91 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data0/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-10-big-Data.db 02:15:14,91 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data0/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-13-big-Data.db 02:15:14,91 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data0/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-1-big-Data.db 02:15:14,91 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data0/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-4-big-Data.db 02:15:14,92 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data0/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-7-big-Data.db 02:15:14,92 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data1/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-2-big-Data.db 02:15:14,92 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data1/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-21-big-Data.db 02:15:14,92 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data1/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-11-big-Data.db 02:15:14,92 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data1/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-5-big-Data.db 02:15:14,93 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data1/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-22-big-Data.db 02:15:14,93 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data1/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-23-big-Data.db 02:15:14,93 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data1/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-14-big-Data.db 02:15:14,93 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data2/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-19-big-Data.db 02:15:14,93 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data2/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-16-big-Data.db 02:15:14,94 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data2/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-20-big-Data.db 02:15:14,94 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data2/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-17-big-Data.db 02:15:14,94 bootstrap_test ERROR /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-jf8aobcs/test/node1/data2/keyspace1/standard1-dada566051eeb3093f1c24ef6265/nc-18-big-Data.db 02:15:14,94 bootstrap_test ERROR Current count is 17, basecount was 15 FAILED {quote} > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765814#comment-17765814 ] Brandon Williams commented on CASSANDRA-18325: -- CASSANDRA-18792 will fix [this one|https://app.circleci.com/pipelines/github/driftx/cassandra/1298/workflows/5a68ff3e-70d2-44db-afbb-e9fbd093da47/jobs/54273] too. > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764409#comment-17764409 ] Ekaterina Dimitrova commented on CASSANDRA-18325: - I can confirm I cannot reproduce it 500 times on 5.0 in CircleCI (in case you decide to try) > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764398#comment-17764398 ] Brandon Williams commented on CASSANDRA-18325: -- I think I know what this is, I'll take a look. > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764393#comment-17764393 ] Ekaterina Dimitrova commented on CASSANDRA-18325: - I still see it on trunk today - https://ci-cassandra.apache.org/job/Cassandra-trunk/1704/testReport/dtest.bootstrap_test/TestBootstrap/test_cleanup_2/ > Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup > - > > Key: CASSANDRA-18325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18325 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Maxwell Guo >Priority: Normal > Fix For: 5.0.x, 5.x > > > > {code:java} > assert not True > + where True = 0x7f5b16fda070>>() > + where 0x7f5b16fda070>> = .is_set > Stacktrace > self = > def test_cleanup(self): > """ > @jira_ticket CASSANDRA-11179 > Make sure we remove processed files during cleanup > """ > cluster = self.cluster > cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', > 'True') > cluster.set_configuration_options(values= > {'concurrent_compactors': 4} > ) > cluster.populate(1) > cluster.start() > node1, = cluster.nodelist() > for x in range(0, 5): > node1.stress(['write', 'n=100k', 'no-warmup', '-schema', > 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)', > 'replication(factor=1)', '-rate', 'threads=10']) > node1.flush() > node2 = new_node(cluster) > node2.start(wait_for_binary_proto=True) > event = threading.Event() > failed = threading.Event() > jobs = 1 > thread = threading.Thread(target=self._monitor_datadir, args=(node1, event, > len(node1.get_sstables("keyspace1", "standard1")), jobs, failed)) > thread.setDaemon(True) > thread.start() > node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs)) > event.set() > thread.join() > > assert not failed.is_set() > E assert not True > E + where True = 0x7f5b16fda070>>() > E + where 0x7f5b16fda070>> = .is_set > bootstrap_test.py:912: AssertionError > {code} > > failed twice -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org