I just created https://issues.apache.org/jira/browse/CASSANDRA-14134 which includes tons of details (and a patch available for review) with my efforts to migrate dtests from nosetest to pytest (which ultimately ended up also including porting the ode from python 2.7 to python 3).
I'd love if people could pitch in in any way to help get this reviewed and committed so we can reduce the natural drift that will occur with a huge patch like this against the changes going into master. I apologize for sending this so close to the holidays, but I really have been working non-stop trying to get things into a completed and stable state. The latest CircleCI runs I did took roughly 15 minutes to run all the dtests with only 6 failures remaining (when run with vnodes) and 12 failures remaining (when run without vnodes). For comparison the last ASF Jenkins Dtest job to successfully complete took nearly 10 hours (9:51) and we had 36 test failures. Of note, while I was working on this and trying to determine a baseline for the existing tests I found that the ASF Jenkins jobs were incorrectly configured due to a typo. The no-vnodes job is actually running with vnodes (meaning the no-vnodes job is identical to the with-vnodes ASF Jenkins job). There are some bootstrap tests that will 100% reliably hang both nosetest and pytest on test cleanup, however this test only runs in the no-vnodes configuration. I've debugged and fixed a lot of these cases across many test cases over the past few weeks and I no longer know of any tests that can hang CI. Thanks and I'm optimistic about making testing great for the project and most importantly for the OSS C* community! best, kjellman Some highlights that I quickly thought of (in no particular order): {also included in the JIRA} -Migrate dtests from executing using the nosetest framework to pytest -Port the entire code base from Python 2.7 to Python 3.6 -Update run_dtests.py to work with pytest -Add --dtest-print-tests-only option to run_dtests.py to get easily parsable list of all available collected tests -Update README.md for executing the dtests with pytest -Add new debugging tips section to README.md to help with some basics of debugging python3 and pytest -Migrate all existing Enviornment Variable usage as a means to control dtest operation modes to argparse command line options with documented help on each toggles intended usage -Migration of old unitTest and nose based test structure to modern pytest fixture approach -Automatic detection of physical system resources to automatically determine if @pytest.mark.resource_intensive annotated tests should be collected and run on the system where they are being executed -new pytest fixture replacements for @since and @pytest.mark.upgrade_test annotations -Migration to python logging framework -Upgrade thrift bindings to latest version with full python3 compatibility -Remove deprecated cql and pycassa dependencies and migrate any remaining tests to fully remove those dependencies -Fixed dozens of tests that would hang the pytest framework forever when run in CI enviornments -Ran code nearly 300 times in CircleCI during the migration and to find, identify, and fix any tests capable of hanging CI -Upgrade Tests do not yet run in CI and still need additional migration work (although all upgrade test classes compile successfully)