[jira] [Comment Edited] (CASSANDRA-14134) Migrate dtests to use pytest and python3
[ https://issues.apache.org/jira/browse/CASSANDRA-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344046#comment-16344046 ] Ariel Weisberg edited comment on CASSANDRA-14134 at 1/29/18 9:33 PM: - +1 Committed as: Cassandra [e73bc32a940175a21700721bafaa593aa86a1a5f|https://github.com/apache/cassandra/commit/e73bc32a940175a21700721bafaa593aa86a1a5f] dtests [49b2dda4e6643d2b18376d504b5fea4c0b3354a7|https://github.com/apache/cassandra-dtest/commit/49b2dda4e6643d2b18376d504b5fea4c0b3354a7] cassandra-builds [6df93e990a93ea4e422b3b67b55cab49554909eb|https://github.com/apache/cassandra-builds/commit/6df93e990a93ea4e422b3b67b55cab49554909eb] and [63504a1ce2cec70e86bfd2f913325db8190f8f88|https://github.com/apache/cassandra-builds/commit/63504a1ce2cec70e86bfd2f913325db8190f8f88] and [d3cd2e8cec57633f8a064c3620f9f5af442640ca|https://github.com/apache/cassandra-builds/commit/d3cd2e8cec57633f8a064c3620f9f5af442640ca] Thanks! was (Author: aweisberg): +1 Committed as: Cassandra [e73bc32a940175a21700721bafaa593aa86a1a5f|https://github.com/apache/cassandra/commit/e73bc32a940175a21700721bafaa593aa86a1a5f] dtests [49b2dda4e6643d2b18376d504b5fea4c0b3354a7|https://github.com/apache/cassandra-dtest/commit/49b2dda4e6643d2b18376d504b5fea4c0b3354a7] cassandra-builds [6df93e990a93ea4e422b3b67b55cab49554909eb|https://github.com/apache/cassandra-builds/commit/6df93e990a93ea4e422b3b67b55cab49554909eb] and [63504a1ce2cec70e86bfd2f913325db8190f8f88|https://github.com/apache/cassandra-builds/commit/63504a1ce2cec70e86bfd2f913325db8190f8f88] and [d3cd2e8cec57633f8a064c3620f9f5af442640ca|https://github.com/apache/cassandra-builds/commit/d3cd2e8cec57633f8a064c3620f9f5af442640ca] Thanks! > Migrate dtests to use pytest and python3 > > > Key: CASSANDRA-14134 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14134 > Project: Cassandra > Issue Type: Improvement > Components: Testing >Reporter: Michael Kjellman >Assignee: Michael Kjellman >Priority: Major > Fix For: 4.0 > > > h4. Get the C* dtests running on the pytest framework. > C* DTests currently run using the python test framework nosetest. This > framework has been largely abandoned with no releases since 2015 and a > general strong consensus in the python community that pytest is the future. > h4. Why should we do this. > Currently (and historically) dtests have always been difficult to run, flaky > and unpredictable in CI environments, and almost impossible to debug. > On November 28th, 2017, I proposed on the dev@ list that we move the dtests > from nosetests to pytests. I got replies from Jon Haddad, Philip Thompson, > and kurt greaves with really only "+1" like replies to the proposal. > Since then I've been working pretty much non stop to complete the large > refactor of dtests to pytests. As part of this effort (and due to the > migration tools that exist require it) I also ported the code to python3 > (from the current python 2.7 based code-base). > h4. High-level summary of key changes, improvements, and new features. > * Migrate dtests from executing using the nosetest framework to pytest > * Port the entire code base from Python 2.7 to Python 3.6 > * Update run_dtests.py to work with pytest > * Add --dtest-print-tests-only option to run_dtests.py to get easily parsable > list of all available collected tests > * Update README.md for executing the dtests with pytest > * Add new debugging tips section to README.md to help with some basics of > debugging python3 and pytest > * Migrate all existing Enviornment Variable usage as a means to control dtest > operation modes to argparse command line options with documented help on each > toggles intended usage > * Migration of old unitTest and nose based test structure to modern pytest > fixture approach > * Automatic detection of physical system resources to automatically determine > if @pytest.mark.resource_intensive annotated tests should be collected and > run on the system where they are being executed > * new pytest fixture replacements for @since and @pytest.mark.upgrade_test > annotations > * Migration to python logging framework > * Upgrade thrift bindings to latest version with full python3 compatibility > * Remove deprecated cql and pycassa dependencies and migrate any remaining > tests to fully remove those dependencies > * Fixed dozens of tests that would hang the pytest framework forever when run > in CI enviornments > * Ran code nearly 300 times in CircleCI during the migration and to find, > identify, and fix any tests capable of hanging CI > * Upgrade Tests do not yet run in CI and still need additional migration work > (although all upgrade test classes compile successfully) > I started with the *nose2pytest*
[jira] [Comment Edited] (CASSANDRA-14134) Migrate dtests to use pytest and python3
[ https://issues.apache.org/jira/browse/CASSANDRA-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16322569#comment-16322569 ] Ariel Weisberg edited comment on CASSANDRA-14134 at 1/11/18 5:36 PM: - OK, how about we separate the flakey annotation work into a second ticket and the test fixes that consist of adding sleeps into a third ticket. I wouldn't do the work to construct the branches for those other tickets we can discuss what you have here for now. Then the only blockers for a +1 would be [these questions here|https://issues.apache.org/jira/browse/CASSANDRA-14134?focusedCommentId=16314133=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16314133] and it seems like you stopped using execute concurrent with results_generator=true, but left the comment in? I thought you added that in a few places, but now I don't see it and I can't find the commit where you removed it. was (Author: aweisberg): OK, how about we separate the flakey annotation work into a second ticket and the test fixes that consist of adding sleeps into a third ticket. I wouldn't do the work to construct the branches for those other tickets we can discuss what you have here for now. Then the only blockers for a +1 would be [these questions here|https://issues.apache.org/jira/browse/CASSANDRA-14134?focusedCommentId=16314133=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16314133] and It seems like you stopped using execute concurrent with results_generator=true, but left the comment in? I thought you added that in a few places, but now I don't see it and I can't find the commit where you removed it. > Migrate dtests to use pytest and python3 > > > Key: CASSANDRA-14134 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14134 > Project: Cassandra > Issue Type: Improvement > Components: Testing >Reporter: Michael Kjellman >Assignee: Michael Kjellman > > h4. Get the C* dtests running on the pytest framework. > C* DTests currently run using the python test framework nosetest. This > framework has been largely abandoned with no releases since 2015 and a > general strong consensus in the python community that pytest is the future. > h4. Why should we do this. > Currently (and historically) dtests have always been difficult to run, flaky > and unpredictable in CI environments, and almost impossible to debug. > On November 28th, 2017, I proposed on the dev@ list that we move the dtests > from nosetests to pytests. I got replies from Jon Haddad, Philip Thompson, > and kurt greaves with really only "+1" like replies to the proposal. > Since then I've been working pretty much non stop to complete the large > refactor of dtests to pytests. As part of this effort (and due to the > migration tools that exist require it) I also ported the code to python3 > (from the current python 2.7 based code-base). > h4. High-level summary of key changes, improvements, and new features. > * Migrate dtests from executing using the nosetest framework to pytest > * Port the entire code base from Python 2.7 to Python 3.6 > * Update run_dtests.py to work with pytest > * Add --dtest-print-tests-only option to run_dtests.py to get easily parsable > list of all available collected tests > * Update README.md for executing the dtests with pytest > * Add new debugging tips section to README.md to help with some basics of > debugging python3 and pytest > * Migrate all existing Enviornment Variable usage as a means to control dtest > operation modes to argparse command line options with documented help on each > toggles intended usage > * Migration of old unitTest and nose based test structure to modern pytest > fixture approach > * Automatic detection of physical system resources to automatically determine > if @pytest.mark.resource_intensive annotated tests should be collected and > run on the system where they are being executed > * new pytest fixture replacements for @since and @pytest.mark.upgrade_test > annotations > * Migration to python logging framework > * Upgrade thrift bindings to latest version with full python3 compatibility > * Remove deprecated cql and pycassa dependencies and migrate any remaining > tests to fully remove those dependencies > * Fixed dozens of tests that would hang the pytest framework forever when run > in CI enviornments > * Ran code nearly 300 times in CircleCI during the migration and to find, > identify, and fix any tests capable of hanging CI > * Upgrade Tests do not yet run in CI and still need additional migration work > (although all upgrade test classes compile successfully) > I started with the *nose2pytest* [https://github.com/pytest-dev/nose2pytest] > migration tool. As this required python 3 language support I
[jira] [Comment Edited] (CASSANDRA-14134) Migrate dtests to use pytest and python3
[
https://issues.apache.org/jira/browse/CASSANDRA-14134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309483#comment-16309483
]
Stefan Podkowinski edited comment on CASSANDRA-14134 at 1/3/18 11:10 AM:
-
There are a couple of tests left that need renaming to get picked up by pytest.
Class names need to start with {{Test}} now if they have {{test_}} methods that
should be run and are not purely used as superclass.
{noformat}
> find . -name \*.py |xargs egrep '^class [^(]+\(Tester\)' | grep -v 'class
> Test'
./thrift_hsha_test.py:class ThriftHSHATest(Tester):
./cql_test.py:class CQLTester(Tester):
./delete_insert_test.py:class DeleteInsertTest(Tester):
./sstable_generation_loading_test.py:class BaseSStableLoaderTest(Tester):
./replace_address_test.py:class BaseReplaceAddressTest(Tester):
./snapshot_test.py:class SnapshotTester(Tester):
./paging_test.py:class BasePagingTester(Tester):
./replication_test.py:class ReplicationTest(Tester):
./replication_test.py:class SnitchConfigurationUpdateTest(Tester):
./thrift_test.py:class ThriftTester(Tester):
./cqlsh_tests/cqlsh_copy_tests.py:class CqlshCopyTest(Tester):
./cqlsh_tests/cqlsh_tests.py:class CqlshSmokeTest(Tester):
./cqlsh_tests/cqlsh_tests.py:class CqlLoginTest(Tester):
./sstableutil_test.py:class SSTableUtilTest(Tester):
./upgrade_tests/compatibility_flag_test.py:class CompatibilityFlagTest(Tester):
./upgrade_tests/thrift_upgrade_test.py:class UpgradeSuperColumnsThrough(Tester):
./upgrade_tests/upgrade_through_versions_test.py:class UpgradeTester(Tester):
./upgrade_tests/upgrade_compact_storage.py:class
UpgradeSuperColumnsThrough(Tester):
./json_test.py:class ToJsonSelectTests(Tester):
./json_test.py:class FromJsonUpdateTests(Tester):
./json_test.py:class FromJsonSelectTests(Tester):
./json_test.py:class FromJsonInsertTests(Tester):
./json_test.py:class FromJsonDeleteTests(Tester):
./json_test.py:class JsonFullRowInsertSelect(Tester):
./native_transport_ssl_test.py:class NativeTransportSSL(Tester):
./repair_tests/preview_repair_test.py:class PreviewRepairTest(Tester):
./repair_tests/repair_test.py:class BaseRepairTest(Tester):
{noformat}
was (Author: spo...@gmail.com):
There are a couple of tests left that need renaming to get picked up by pytest.
Class names need to start with {{Test}} now.
{noformat}
> egrep '^class [^(]+\(Tester\)' *.py | grep -v 'class Test'
cql_test.py:class CQLTester(Tester):
delete_insert_test.py:class DeleteInsertTest(Tester):
json_test.py:class ToJsonSelectTests(Tester):
json_test.py:class FromJsonUpdateTests(Tester):
json_test.py:class FromJsonSelectTests(Tester):
json_test.py:class FromJsonInsertTests(Tester):
json_test.py:class FromJsonDeleteTests(Tester):
json_test.py:class JsonFullRowInsertSelect(Tester):
native_transport_ssl_test.py:class NativeTransportSSL(Tester):
paging_test.py:class BasePagingTester(Tester):
replace_address_test.py:class BaseReplaceAddressTest(Tester):
replication_test.py:class ReplicationTest(Tester):
replication_test.py:class SnitchConfigurationUpdateTest(Tester):
snapshot_test.py:class SnapshotTester(Tester):
sstable_generation_loading_test.py:class BaseSStableLoaderTest(Tester):
sstableutil_test.py:class SSTableUtilTest(Tester):
thrift_hsha_test.py:class ThriftHSHATest(Tester):
thrift_test.py:class ThriftTester(Tester):
{noformat}
> Migrate dtests to use pytest and python3
>
>
> Key: CASSANDRA-14134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14134
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing
>Reporter: Michael Kjellman
>Assignee: Michael Kjellman
>
> h4. Get the C* dtests running on the pytest framework.
> C* DTests currently run using the python test framework nosetest. This
> framework has been largely abandoned with no releases since 2015 and a
> general strong consensus in the python community that pytest is the future.
> h4. Why should we do this.
> Currently (and historically) dtests have always been difficult to run, flaky
> and unpredictable in CI environments, and almost impossible to debug.
> On November 28th, 2017, I proposed on the dev@ list that we move the dtests
> from nosetests to pytests. I got replies from Jon Haddad, Philip Thompson,
> and kurt greaves with really only "+1" like replies to the proposal.
> Since then I've been working pretty much non stop to complete the large
> refactor of dtests to pytests. As part of this effort (and due to the
> migration tools that exist require it) I also ported the code to python3
> (from the current python 2.7 based code-base).
> h4. High-level summary of key changes, improvements, and new features.
> * Migrate dtests from executing using the nosetest framework to pytest
> * Port the entire code base from Python 2.7 to Python 3.6
> * Update run_dtests.py
