[
https://issues.apache.org/jira/browse/CASSANDRA-16399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270589#comment-17270589
]
Jacek Lewandowski commented on CASSANDRA-16399:
-----------------------------------------------
Summary:
This patch fixes behaviour for both {{run_dtests.py}} and {{pytest}}.
In particular:
- Error handling for invalid parameter values / combinations is in a single
place ({{dtest_config.py}}) and is executed before we actually traverse through
the tests
- We exit with just a clean error message instead of tons of spam
- {{run_dtests.sh}} will not loose the exit code of {{pytest}} any more so we
can clearly detect when test cases collection fails
- removed a bit of boilerplate code from {{run_dtests.py}} - in particular what
it did with xml processing is simply provided with {{-q}} argument of {{pytest}}
- tests filtering has been refactored to be cleaner
- fixed filtering of resource intensive tests
Note that now {{run_dtests.py}} seems to be redundant. If we need it only for
listing dtests, we can simply achieve exactly the same effect using
{{\-\-collect-only -q \-\-ignore=meta_tests}} arguments for {{pytest}} instead
of {{\-\-dtest-print-tests-only}}, plus we need to filter output with {{grep
'.py::'}} (in order to not include the summary line) and pipe stdout to the
target file. Anyway I simplified {{run_dtests.sh}} so that it just use
{{pytest}} with those arguments.
Regarding Cassandra CI, I suppose most of the configurations will benefit with
this fix:
- {{no-vnodes}}, {{vnodes}}, {{off-heap-memtables}} - previously included
resource intensive tests although it was specified to skip them in the build
script
- {{large}} - instead of listing just resource intensive tests it listed all
tests
Some examples of error handling now vs before:
* before:
{noformat}
$ ./run_dtests.py --dtest-print-tests-only
<?xml version="1.0" encoding="UTF-8"?>
<Modules>
</Instance>
</Class>
</Module>
</Modules>
$ pytest --collect-only
==================================================== test session starts
=====================================================
platform darwin -- Python 3.9.1, pytest-3.6.4, py-1.10.0, pluggy-0.7.1
rootdir: /Users/jlewandowski/dev/datastax/cassandra-dtest/trunk, inifile:
pytest.ini
plugins: timeout-1.4.2, flaky-3.7.0
timeout: 900.0s
timeout method: signal
timeout func_only: False
collecting 0 items / 1 errors
INTERNALERROR> Traceback (most
recent call last):
INTERNALERROR> File
"/Users/jlewandowski/dev/datastax/cassandra-dtest/trunk/venv/lib/python3.9/site-packages/_pytest/main.py",
line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
...
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory:
'build.xml'================================================== 1 error in 0.18
seconds ===================================================
$ ./run_dtests.py --cassandra-dir=../../cassandra/ds-trunk
--dtest-print-tests-only --only-resource-intensive-tests
--skip-resource-intensive-tests
<?xml version="1.0" encoding="UTF-8"?>
<Modules>
<Module name="auditlog_test.py">
...
auth_test.py::TestAuthUnavailable::test_permission_cache_background_reload_handle_unavailable
auth_test.py::TestNetworkAuth::test_full_dc_access
auth_test.py::TestNetworkAuth::test_single_dc_access
...
(no error)
{noformat}
* now:
{noformat}
$ ./run_dtests.py --dtest-print-tests-only
ERROR: Required dtest arguments were missing! You must provide either
--cassandra-dir or --cassandra-version. You can also set 'cassandra_dir' in
pytest.ini. Refer to the documentation or invoke the help with --help.
$ pytest --collect-only
ERROR: Required dtest arguments were missing! You must provide either
--cassandra-dir or --cassandra-version. You can also set 'cassandra_dir' in
pytest.ini. Refer to the documentation or invoke the help with --help.
$ ./run_dtests.py --cassandra-dir=../../cassandra/ds-trunk
--dtest-print-tests-only --only-resource-intensive-tests
--skip-resource-intensive-tests
ERROR: --skip-resource-intensive-tests does not make any sense with either
--only-resource-intensive-tests or --force-resource-intensive-tests.
$ pytest --cassandra-dir=../../cassandra/ds-trunk --collect-only
--only-resource-intensive-tests --skip-resource-intensive-tests
ERROR: --skip-resource-intensive-tests does not make any sense with either
--only-resource-intensive-tests or --force-resource-intensive-tests.
{noformat}
> Selecting resource intensive tests is not consistent
> ----------------------------------------------------
>
> Key: CASSANDRA-16399
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16399
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
> Reporter: Jacek Lewandowski
> Assignee: Jacek Lewandowski
> Priority: Normal
> Time Spent: 1h
> Remaining Estimate: 0h
>
> It looks like there are two problems:
> - collection of tests to run fails when there are log messages in stderr
> - collection of resource intensive tests (with
> {code:java}
> --only-resource-intensive-tests{code}
> ) is broken
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
