[ https://issues.apache.org/jira/browse/CASSANDRA-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175902#comment-16175902 ]
Jeff Jirsa commented on CASSANDRA-10659: ---------------------------------------- [~mambocab] / [~JoshuaMcKenzie] / [~philipthompson] - if any of you believe this should still be open, please let me know. Otherwise I'm inclined to close as won't-fix since cassci is no longer generally available , and I suspect any other more general test failures can be addressed elsewhere? > Windows CassCI: Fail on timed-out tests > --------------------------------------- > > Key: CASSANDRA-10659 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10659 > Project: Cassandra > Issue Type: Bug > Reporter: Jim Witschey > Assignee: Jim Witschey > > On our Windows CassCI environments, it looks like some dtests are prone to > hanging, e.g.: > https://cassci.datastax.com/view/Dev/view/josh-mckenzie/job/josh-mckenzie-10641_windows-dtest_win32/1/ > http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/131/ > http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/129/ > http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/128/ > http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/126/ > http://cassci.datastax.com/view/cassandra-2.2/job/cassandra-2.2_dtest_win32/125/ > Ideally these tests wouldn't hang, but regardless, we should figure out a way > to make them fail, rather than timing out Jenkins and botching the rest of > the test run. > The built-in [{{nosetests}} {{multiprocess}} > plugin|http://nose.readthedocs.org/en/latest/plugins/multiprocess.html] would > solve this problem for us -- we could run the tests with {{nosetests > --processes=1 --process-timeout=X}} and it would stop the test and fail if > the test took too long. However, it's broken on Windows. I've filed [a quick > issue on the {{nose}} GitHub|https://github.com/nose-devs/nose/issues/966], > but in the meantime, we should figure out how to avoid this. > Possible solutions: > # [~philipthompson] had a script that would shell out to {{nosetests}} for > each test and kill that process if it took too long. If I understand > correctly, that script is broken, or assumes things that are no longer true. > We can revamp it if we want. > # We could make a patch for {{nose}} to fix the {{multiprocess}} plugin. > # We could hack in some of {{multiprocessing}}'s functionality into the > {{dtest}} suite itself. > 3. may be the best workaround for this problem -- our timeouts aren't caused > just when a tests runs long, but when Jenkins doesn't get any output on > stdout from a hanging test. We may be able to monitor stdout from a second > process and fail the test before Jenkins would time out. > Pinging [~JoshuaMcKenzie] as this is a Windows issue. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org