On Thu, Feb 4, 2021 at 2:54 PM Daniel Latypov wrote:
>
> When using `kunit.py run` to run tests, users must populate a
> `kunitconfig` file to select the options the tests are hidden behind and
> all their dependencies.
>
> The patch [1] to allow specifying a path to kunitconfig promises to make
> this nicer as we can have checked in files corresponding to different
> sets of tests.
>
> But it's still annoying
> 1) when trying to run a subet of tests
> 2) when you want to run tests that don't have such a pre-existing
> kunitconfig and selecting all the necessary options is tricky.
>
> This patch series aims to alleviate both:
> 1) `kunit.py run 'my-suite-*'`
> I.e. use my current kunitconfig, but just run suites that match this glob
> 2) `kunit.py run --alltests 'my-suite-*'`
> I.e. use allyesconfig so I don't have to worry about writing a
> kunitconfig at all.
>
> See the first commit message for more details and discussion about
> future work.
>
> This patch series also includes a bugfix for a latent bug that can't be
> triggered right now but has worse consequences as a result of the
> changes needed to plumb in this suite name glob.
>
> [1]
> https://lore.kernel.org/linux-kselftest/20210201205514.3943096-1-dlaty...@google.com/
I tried applying this series on top of several of your other patches
over the past cycle and running the kunit_tool unit tests
(kunit_tool_test.py) and I got several failures:
==
FAIL: test_exec_builddir (__main__.KUnitMainTest)
--
Traceback (most recent call last):
File "tools/testing/kunit/kunit_tool_test.py", line 417, in test_exec_builddir
self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir,
timeout=300)
File "/usr/lib/python3.8/unittest/mock.py", line 925, in
assert_called_once_with
return self.assert_called_with(*args, **kwargs)
File "/usr/lib/python3.8/unittest/mock.py", line 913, in assert_called_with
raise AssertionError(_error_message()) from cause
AssertionError: expected call not found.
Expected: run_kernel(build_dir='.kunit', timeout=300)
Actual: run_kernel(timeout=300, filter_glob='', build_dir='.kunit')
==
FAIL: test_exec_passes_args_pass (__main__.KUnitMainTest)
--
Traceback (most recent call last):
File "tools/testing/kunit/kunit_tool_test.py", line 337, in
test_exec_passes_args_pass
self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='.kunit',
timeout=300)
File "/usr/lib/python3.8/unittest/mock.py", line 925, in
assert_called_once_with
return self.assert_called_with(*args, **kwargs)
File "/usr/lib/python3.8/unittest/mock.py", line 913, in assert_called_with
raise AssertionError(_error_message()) from cause
AssertionError: expected call not found.
Expected: run_kernel(build_dir='.kunit', timeout=300)
Actual: run_kernel(timeout=300, filter_glob='', build_dir='.kunit')
==
FAIL: test_exec_timeout (__main__.KUnitMainTest)
--
Traceback (most recent call last):
File "tools/testing/kunit/kunit_tool_test.py", line 385, in test_exec_timeout
self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='.kunit',
timeout=timeout)
File "/usr/lib/python3.8/unittest/mock.py", line 925, in
assert_called_once_with
return self.assert_called_with(*args, **kwargs)
File "/usr/lib/python3.8/unittest/mock.py", line 913, in assert_called_with
raise AssertionError(_error_message()) from cause
AssertionError: expected call not found.
Expected: run_kernel(build_dir='.kunit', timeout=3453)
Actual: run_kernel(timeout=3453, filter_glob='', build_dir='.kunit')
==
FAIL: test_run_builddir (__main__.KUnitMainTest)
--
Traceback (most recent call last):
File "tools/testing/kunit/kunit_tool_test.py", line 400, in test_run_builddir
self.linux_source_mock.run_kernel.assert_called_once_with(
File "/usr/lib/python3.8/unittest/mock.py", line 925, in
assert_called_once_with
return self.assert_called_with(*args, **kwargs)
File "/usr/lib/python3.8/unittest/mock.py", line 913, in assert_called_with
raise AssertionError(_error_message()) from cause
AssertionError: expected call not found.
Expected: run_kernel(build_dir='.kunit', timeout=300)
Actual: run_kernel(timeout=300, filter_glob='', build_dir='.kunit')
==
FAIL: test_run_passes_args_pass (__main__.KUnitMainTest)
--
Traceback (most recent call last):
File
