Re: [PATCH v3 00/11] Add Kconfig unit tests
2018-03-13 18:12 GMT+09:00 Masahiro Yamada : > I am applying various cleanups to Kconfig these days. > > However, I fear regressions. I have been thinking of unit-tests. > > There are various cryptic parts in Kconfig and corner cases where > it is difficult to notice breakage. If unit-tests cover those, > I will be able to apply changes more confidently. > > This series introduces a unit-tests. The framework is written based > on pytest. Also, this is written in Python 3. Python 2 will retire > in 2020. So, I believe new python tools should be written in Python 3. > > This is my Python 3 and pytest versions. > > $ python3 --version > Python 3.5.2 > $ python3 -m pytest --version > This is pytest version 3.4.1, imported from > /home/masahiro/.local/lib/python3.5/site-packages/pytest.py > > How to use? > --- > > Please make sure Python3 and pytest are installed on your system. > > Then, run "make testconfig" > > The result looks like as follows: > > $ make testconfig > python3 -B -m pytest ./scripts/kconfig/tests \ > -o > cache_dir=/home/masahiro/workspace/bsp/linux-yamada/scripts/kconfig/tests/.cache > \ > > = test session starts > = > platform linux -- Python 3.5.2, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- > /usr/bin/python3 > cachedir: scripts/kconfig/tests/.cache > rootdir: /home/masahiro/workspace/bsp/linux-yamada/scripts/kconfig/tests, > inifile: pytest.ini > collected 14 items > > scripts/kconfig/tests/auto_submenu/__init__.py::test PASSED > [ 7%] > scripts/kconfig/tests/choice/__init__.py::test_oldask0 PASSED > [ 14%] > scripts/kconfig/tests/choice/__init__.py::test_oldask1 PASSED > [ 21%] > scripts/kconfig/tests/choice/__init__.py::test_allyes PASSED > [ 28%] > scripts/kconfig/tests/choice/__init__.py::test_allmod PASSED > [ 35%] > scripts/kconfig/tests/choice/__init__.py::test_allno PASSED > [ 42%] > scripts/kconfig/tests/choice/__init__.py::test_alldef PASSED > [ 50%] > scripts/kconfig/tests/choice_value_with_m_dep/__init__.py::test PASSED > [ 57%] > scripts/kconfig/tests/err_recursive_inc/__init__.py::test PASSED > [ 64%] > scripts/kconfig/tests/inter_choice/__init__.py::test PASSED > [ 71%] > scripts/kconfig/tests/new_choice_with_dep/__init__.py::test PASSED > [ 78%] > scripts/kconfig/tests/no_write_if_dep_unmet/__init__.py::test PASSED > [ 85%] > scripts/kconfig/tests/rand_nested_choice/__init__.py::test PASSED > [ 92%] > scripts/kconfig/tests/warn_recursive_dep/__init__.py::test PASSED > [100%] > > > Masahiro Yamada (11): > kbuild: add PYTHON2 and PYTHON3 variables > kconfig: tests: add framework for Kconfig unit testing > kconfig: tests: add basic choice tests > kconfig: tests: test automatic submenu creation > kconfig: tests: test if new symbols in choice are asked > kconfig: tests: check unneeded "is not set" with unmet dependency > kconfig: tests: check visibility of tristate choice values in y choice > kconfig: tests: test defconfig when two choices interact > kconfig: tests: test randconfig for choice in choice > kconfig: tests: test if recursive dependencies are detected > kconfig: tests: test if recursive inclusion is detected > All applied to linux-kbuild/kconfig. -- Best Regards Masahiro Yamada
[PATCH v3 00/11] Add Kconfig unit tests
I am applying various cleanups to Kconfig these days. However, I fear regressions. I have been thinking of unit-tests. There are various cryptic parts in Kconfig and corner cases where it is difficult to notice breakage. If unit-tests cover those, I will be able to apply changes more confidently. This series introduces a unit-tests. The framework is written based on pytest. Also, this is written in Python 3. Python 2 will retire in 2020. So, I believe new python tools should be written in Python 3. This is my Python 3 and pytest versions. $ python3 --version Python 3.5.2 $ python3 -m pytest --version This is pytest version 3.4.1, imported from /home/masahiro/.local/lib/python3.5/site-packages/pytest.py How to use? --- Please make sure Python3 and pytest are installed on your system. Then, run "make testconfig" The result looks like as follows: $ make testconfig python3 -B -m pytest ./scripts/kconfig/tests \ -o cache_dir=/home/masahiro/workspace/bsp/linux-yamada/scripts/kconfig/tests/.cache \ = test session starts = platform linux -- Python 3.5.2, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python3 cachedir: scripts/kconfig/tests/.cache rootdir: /home/masahiro/workspace/bsp/linux-yamada/scripts/kconfig/tests, inifile: pytest.ini collected 14 items scripts/kconfig/tests/auto_submenu/__init__.py::test PASSED [ 7%] scripts/kconfig/tests/choice/__init__.py::test_oldask0 PASSED [ 14%] scripts/kconfig/tests/choice/__init__.py::test_oldask1 PASSED [ 21%] scripts/kconfig/tests/choice/__init__.py::test_allyes PASSED [ 28%] scripts/kconfig/tests/choice/__init__.py::test_allmod PASSED [ 35%] scripts/kconfig/tests/choice/__init__.py::test_allno PASSED [ 42%] scripts/kconfig/tests/choice/__init__.py::test_alldef PASSED [ 50%] scripts/kconfig/tests/choice_value_with_m_dep/__init__.py::test PASSED [ 57%] scripts/kconfig/tests/err_recursive_inc/__init__.py::test PASSED [ 64%] scripts/kconfig/tests/inter_choice/__init__.py::test PASSED [ 71%] scripts/kconfig/tests/new_choice_with_dep/__init__.py::test PASSED [ 78%] scripts/kconfig/tests/no_write_if_dep_unmet/__init__.py::test PASSED [ 85%] scripts/kconfig/tests/rand_nested_choice/__init__.py::test PASSED [ 92%] scripts/kconfig/tests/warn_recursive_dep/__init__.py::test PASSED [100%] Masahiro Yamada (11): kbuild: add PYTHON2 and PYTHON3 variables kconfig: tests: add framework for Kconfig unit testing kconfig: tests: add basic choice tests kconfig: tests: test automatic submenu creation kconfig: tests: test if new symbols in choice are asked kconfig: tests: check unneeded "is not set" with unmet dependency kconfig: tests: check visibility of tristate choice values in y choice kconfig: tests: test defconfig when two choices interact kconfig: tests: test randconfig for choice in choice kconfig: tests: test if recursive dependencies are detected kconfig: tests: test if recursive inclusion is detected Makefile | 4 +- scripts/kconfig/Makefile | 8 + scripts/kconfig/tests/auto_submenu/Kconfig | 50 scripts/kconfig/tests/auto_submenu/__init__.py | 12 + scripts/kconfig/tests/auto_submenu/expected_stdout | 10 + scripts/kconfig/tests/choice/Kconfig | 54 scripts/kconfig/tests/choice/__init__.py | 40 +++ .../kconfig/tests/choice/alldef_expected_config| 5 + .../kconfig/tests/choice/allmod_expected_config| 9 + scripts/kconfig/tests/choice/allno_expected_config | 5 + .../kconfig/tests/choice/allyes_expected_config| 9 + .../kconfig/tests/choice/oldask0_expected_stdout | 10 + scripts/kconfig/tests/choice/oldask1_config| 2 + .../kconfig/tests/choice/oldask1_expected_stdout | 15 ++ .../kconfig/tests/choice_value_with_m_dep/Kconfig | 19 ++ .../tests/choice_value_with_m_dep/__init__.py | 15 ++ .../kconfig/tests/choice_value_with_m_dep/config | 2 + .../tests/choice_value_with_m_dep/expected_config | 3 + .../tests/choice_value_with_m_dep/expected_stdout | 4 + scripts/kconfig/tests/conftest.py | 291 + scripts/kconfig/tests/err_recursive_inc/Kconfig| 1 + scripts/kconfig/tests/err_recursive_inc/Kconfig2 | 1 + scripts/kconfig/tests/err_recursive_inc/Kconfig3 | 1 + .../kconfig/tests/err_recursive_inc/__init__.py| 10 + .../tests/err_recursive_inc/expected_stderr| 5 + scripts/kconfig/tests/inter_choice/Kconfig | 23 ++ scripts/kconfig/tests/inter_choice/__init__.py | 14 + scripts/kconfig/tests/inter_choice/defconfig | 1 + scripts/kconfig/tests/inter_choice/
