Re: [PATCH v4 3/9] selftests: Introduce minimal shared logic for running tests
On 03/13/2015 11:19 AM, Shuah Khan wrote: > On 03/10/2015 10:05 PM, Michael Ellerman wrote: >> This adds a Make include file which most selftests can then include to >> get the run_tests logic. >> >> On its own this has the advantage of some reduction in repetition, and >> also means the pass/fail message is defined in fewer places. >> >> However the key advantage is it will allow us to implement install very >> simply in a subsequent patch. >> >> The default implementation just executes each program in $(TEST_PROGS). >> >> We use a variable to hold the default implementation of $(RUN_TESTS) >> because that gives us a clean way to override it if necessary, ie. using >> override. The mount, memory-hotplug and mqueue tests use that to provide >> a different implementation. >> >> Tests are not run via /bin/bash, so if they are scripts they must be >> executable, we add a+x to several. >> >> Signed-off-by: Michael Ellerman > > This patch will be applied to next and queued for 4.1. > This patch is now in linux-kselftest next. Could you please review to make sure, it looks right. I had to drop the shared logic from timers Makefile because, it changed considerably with the additional tests and it wasn't easy to resolve the conflict and keep both changes. So at the moment, timers doesn't use the shared logic. If lib.mk could provide a way to run additional programs that require arguments in addition to RUN_TESTS. In the case of timers, there is one test that requires arguments. In some cases, e.g: memory hotplug, override works well since it is just one executable. In this case, there is a mix. Something that can be addressed in a separate patch. For now, I made the decision to apply with shared logic patch minus the changes to use lib.mk thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Open Source Innovation Group Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 3/9] selftests: Introduce minimal shared logic for running tests
On 03/10/2015 10:05 PM, Michael Ellerman wrote: > This adds a Make include file which most selftests can then include to > get the run_tests logic. > > On its own this has the advantage of some reduction in repetition, and > also means the pass/fail message is defined in fewer places. > > However the key advantage is it will allow us to implement install very > simply in a subsequent patch. > > The default implementation just executes each program in $(TEST_PROGS). > > We use a variable to hold the default implementation of $(RUN_TESTS) > because that gives us a clean way to override it if necessary, ie. using > override. The mount, memory-hotplug and mqueue tests use that to provide > a different implementation. > > Tests are not run via /bin/bash, so if they are scripts they must be > executable, we add a+x to several. > > Signed-off-by: Michael Ellerman This patch will be applied to next and queued for 4.1. -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Open Source Innovation Group Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 3/9] selftests: Introduce minimal shared logic for running tests
On 03/10/2015 10:05 PM, Michael Ellerman wrote: This adds a Make include file which most selftests can then include to get the run_tests logic. On its own this has the advantage of some reduction in repetition, and also means the pass/fail message is defined in fewer places. However the key advantage is it will allow us to implement install very simply in a subsequent patch. The default implementation just executes each program in $(TEST_PROGS). We use a variable to hold the default implementation of $(RUN_TESTS) because that gives us a clean way to override it if necessary, ie. using override. The mount, memory-hotplug and mqueue tests use that to provide a different implementation. Tests are not run via /bin/bash, so if they are scripts they must be executable, we add a+x to several. Signed-off-by: Michael Ellerman m...@ellerman.id.au This patch will be applied to next and queued for 4.1. -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Open Source Innovation Group Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH v4 3/9] selftests: Introduce minimal shared logic for running tests
On 03/13/2015 11:19 AM, Shuah Khan wrote: On 03/10/2015 10:05 PM, Michael Ellerman wrote: This adds a Make include file which most selftests can then include to get the run_tests logic. On its own this has the advantage of some reduction in repetition, and also means the pass/fail message is defined in fewer places. However the key advantage is it will allow us to implement install very simply in a subsequent patch. The default implementation just executes each program in $(TEST_PROGS). We use a variable to hold the default implementation of $(RUN_TESTS) because that gives us a clean way to override it if necessary, ie. using override. The mount, memory-hotplug and mqueue tests use that to provide a different implementation. Tests are not run via /bin/bash, so if they are scripts they must be executable, we add a+x to several. Signed-off-by: Michael Ellerman m...@ellerman.id.au This patch will be applied to next and queued for 4.1. This patch is now in linux-kselftest next. Could you please review to make sure, it looks right. I had to drop the shared logic from timers Makefile because, it changed considerably with the additional tests and it wasn't easy to resolve the conflict and keep both changes. So at the moment, timers doesn't use the shared logic. If lib.mk could provide a way to run additional programs that require arguments in addition to RUN_TESTS. In the case of timers, there is one test that requires arguments. In some cases, e.g: memory hotplug, override works well since it is just one executable. In this case, there is a mix. Something that can be addressed in a separate patch. For now, I made the decision to apply with shared logic patch minus the changes to use lib.mk thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Open Source Innovation Group Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v4 3/9] selftests: Introduce minimal shared logic for running tests
This adds a Make include file which most selftests can then include to get the run_tests logic. On its own this has the advantage of some reduction in repetition, and also means the pass/fail message is defined in fewer places. However the key advantage is it will allow us to implement install very simply in a subsequent patch. The default implementation just executes each program in $(TEST_PROGS). We use a variable to hold the default implementation of $(RUN_TESTS) because that gives us a clean way to override it if necessary, ie. using override. The mount, memory-hotplug and mqueue tests use that to provide a different implementation. Tests are not run via /bin/bash, so if they are scripts they must be executable, we add a+x to several. Signed-off-by: Michael Ellerman --- v3: Rebase onto 4.0-rc2, add +x to more scripts that need it. v4: Rebase onto 4.0-rc3. tools/testing/selftests/breakpoints/Makefile | 5 +++-- tools/testing/selftests/cpu-hotplug/Makefile | 5 +++-- tools/testing/selftests/cpu-hotplug/on-off-test.sh | 0 tools/testing/selftests/efivarfs/Makefile| 5 +++-- tools/testing/selftests/efivarfs/efivarfs.sh | 0 tools/testing/selftests/exec/Makefile| 5 +++-- tools/testing/selftests/firmware/Makefile| 20 ++-- tools/testing/selftests/firmware/fw_filesystem.sh| 0 tools/testing/selftests/firmware/fw_userhelper.sh| 0 tools/testing/selftests/ftrace/Makefile | 5 +++-- tools/testing/selftests/ipc/Makefile | 5 +++-- tools/testing/selftests/kcmp/Makefile| 5 +++-- tools/testing/selftests/lib.mk | 10 ++ tools/testing/selftests/memfd/Makefile | 6 +++--- tools/testing/selftests/memory-hotplug/Makefile | 5 +++-- .../testing/selftests/memory-hotplug/on-off-test.sh | 0 tools/testing/selftests/mount/Makefile | 8 ++-- tools/testing/selftests/mqueue/Makefile | 9 ++--- tools/testing/selftests/net/Makefile | 8 tools/testing/selftests/net/run_afpackettests| 0 tools/testing/selftests/net/run_netsocktests | 0 tools/testing/selftests/ptrace/Makefile | 5 +++-- tools/testing/selftests/size/Makefile| 5 +++-- tools/testing/selftests/sysctl/Makefile | 11 ++- tools/testing/selftests/sysctl/run_numerictests | 0 tools/testing/selftests/sysctl/run_stringtests | 0 tools/testing/selftests/timers/Makefile | 5 +++-- tools/testing/selftests/user/Makefile| 5 +++-- tools/testing/selftests/vm/Makefile | 5 +++-- tools/testing/selftests/vm/run_vmtests | 0 30 files changed, 68 insertions(+), 69 deletions(-) mode change 100644 => 100755 tools/testing/selftests/cpu-hotplug/on-off-test.sh mode change 100644 => 100755 tools/testing/selftests/efivarfs/efivarfs.sh mode change 100644 => 100755 tools/testing/selftests/firmware/fw_filesystem.sh mode change 100644 => 100755 tools/testing/selftests/firmware/fw_userhelper.sh create mode 100644 tools/testing/selftests/lib.mk mode change 100644 => 100755 tools/testing/selftests/memory-hotplug/on-off-test.sh mode change 100644 => 100755 tools/testing/selftests/net/run_afpackettests mode change 100644 => 100755 tools/testing/selftests/net/run_netsocktests mode change 100644 => 100755 tools/testing/selftests/sysctl/run_numerictests mode change 100644 => 100755 tools/testing/selftests/sysctl/run_stringtests mode change 100644 => 100755 tools/testing/selftests/vm/run_vmtests diff --git a/tools/testing/selftests/breakpoints/Makefile b/tools/testing/selftests/breakpoints/Makefile index e18b42b254af..182235640209 100644 --- a/tools/testing/selftests/breakpoints/Makefile +++ b/tools/testing/selftests/breakpoints/Makefile @@ -16,8 +16,9 @@ else echo "Not an x86 target, can't build breakpoints selftests" endif -run_tests: - @./breakpoint_test || echo "breakpoints selftests: [FAIL]" +TEST_PROGS := breakpoint_test + +include ../lib.mk clean: rm -fr breakpoint_test diff --git a/tools/testing/selftests/cpu-hotplug/Makefile b/tools/testing/selftests/cpu-hotplug/Makefile index e9c28d8dc84b..15f02591d22c 100644 --- a/tools/testing/selftests/cpu-hotplug/Makefile +++ b/tools/testing/selftests/cpu-hotplug/Makefile @@ -1,7 +1,8 @@ all: -run_tests: - @/bin/bash ./on-off-test.sh || echo "cpu-hotplug selftests: [FAIL]" +TEST_PROGS := on-off-test.sh + +include ../lib.mk run_full_test: @/bin/bash ./on-off-test.sh -a || echo "cpu-hotplug selftests: [FAIL]" diff --git a/tools/testing/selftests/cpu-hotplug/on-off-test.sh b/tools/testing/selftests/cpu-hotplug/on-off-test.sh old mode 100644 new mode 100755 diff --git a/tools/testing/selftests/efivarfs/Makefile b/tools/testing/selftests/efivarfs/Makefile index
[PATCH v4 3/9] selftests: Introduce minimal shared logic for running tests
This adds a Make include file which most selftests can then include to get the run_tests logic. On its own this has the advantage of some reduction in repetition, and also means the pass/fail message is defined in fewer places. However the key advantage is it will allow us to implement install very simply in a subsequent patch. The default implementation just executes each program in $(TEST_PROGS). We use a variable to hold the default implementation of $(RUN_TESTS) because that gives us a clean way to override it if necessary, ie. using override. The mount, memory-hotplug and mqueue tests use that to provide a different implementation. Tests are not run via /bin/bash, so if they are scripts they must be executable, we add a+x to several. Signed-off-by: Michael Ellerman m...@ellerman.id.au --- v3: Rebase onto 4.0-rc2, add +x to more scripts that need it. v4: Rebase onto 4.0-rc3. tools/testing/selftests/breakpoints/Makefile | 5 +++-- tools/testing/selftests/cpu-hotplug/Makefile | 5 +++-- tools/testing/selftests/cpu-hotplug/on-off-test.sh | 0 tools/testing/selftests/efivarfs/Makefile| 5 +++-- tools/testing/selftests/efivarfs/efivarfs.sh | 0 tools/testing/selftests/exec/Makefile| 5 +++-- tools/testing/selftests/firmware/Makefile| 20 ++-- tools/testing/selftests/firmware/fw_filesystem.sh| 0 tools/testing/selftests/firmware/fw_userhelper.sh| 0 tools/testing/selftests/ftrace/Makefile | 5 +++-- tools/testing/selftests/ipc/Makefile | 5 +++-- tools/testing/selftests/kcmp/Makefile| 5 +++-- tools/testing/selftests/lib.mk | 10 ++ tools/testing/selftests/memfd/Makefile | 6 +++--- tools/testing/selftests/memory-hotplug/Makefile | 5 +++-- .../testing/selftests/memory-hotplug/on-off-test.sh | 0 tools/testing/selftests/mount/Makefile | 8 ++-- tools/testing/selftests/mqueue/Makefile | 9 ++--- tools/testing/selftests/net/Makefile | 8 tools/testing/selftests/net/run_afpackettests| 0 tools/testing/selftests/net/run_netsocktests | 0 tools/testing/selftests/ptrace/Makefile | 5 +++-- tools/testing/selftests/size/Makefile| 5 +++-- tools/testing/selftests/sysctl/Makefile | 11 ++- tools/testing/selftests/sysctl/run_numerictests | 0 tools/testing/selftests/sysctl/run_stringtests | 0 tools/testing/selftests/timers/Makefile | 5 +++-- tools/testing/selftests/user/Makefile| 5 +++-- tools/testing/selftests/vm/Makefile | 5 +++-- tools/testing/selftests/vm/run_vmtests | 0 30 files changed, 68 insertions(+), 69 deletions(-) mode change 100644 = 100755 tools/testing/selftests/cpu-hotplug/on-off-test.sh mode change 100644 = 100755 tools/testing/selftests/efivarfs/efivarfs.sh mode change 100644 = 100755 tools/testing/selftests/firmware/fw_filesystem.sh mode change 100644 = 100755 tools/testing/selftests/firmware/fw_userhelper.sh create mode 100644 tools/testing/selftests/lib.mk mode change 100644 = 100755 tools/testing/selftests/memory-hotplug/on-off-test.sh mode change 100644 = 100755 tools/testing/selftests/net/run_afpackettests mode change 100644 = 100755 tools/testing/selftests/net/run_netsocktests mode change 100644 = 100755 tools/testing/selftests/sysctl/run_numerictests mode change 100644 = 100755 tools/testing/selftests/sysctl/run_stringtests mode change 100644 = 100755 tools/testing/selftests/vm/run_vmtests diff --git a/tools/testing/selftests/breakpoints/Makefile b/tools/testing/selftests/breakpoints/Makefile index e18b42b254af..182235640209 100644 --- a/tools/testing/selftests/breakpoints/Makefile +++ b/tools/testing/selftests/breakpoints/Makefile @@ -16,8 +16,9 @@ else echo Not an x86 target, can't build breakpoints selftests endif -run_tests: - @./breakpoint_test || echo breakpoints selftests: [FAIL] +TEST_PROGS := breakpoint_test + +include ../lib.mk clean: rm -fr breakpoint_test diff --git a/tools/testing/selftests/cpu-hotplug/Makefile b/tools/testing/selftests/cpu-hotplug/Makefile index e9c28d8dc84b..15f02591d22c 100644 --- a/tools/testing/selftests/cpu-hotplug/Makefile +++ b/tools/testing/selftests/cpu-hotplug/Makefile @@ -1,7 +1,8 @@ all: -run_tests: - @/bin/bash ./on-off-test.sh || echo cpu-hotplug selftests: [FAIL] +TEST_PROGS := on-off-test.sh + +include ../lib.mk run_full_test: @/bin/bash ./on-off-test.sh -a || echo cpu-hotplug selftests: [FAIL] diff --git a/tools/testing/selftests/cpu-hotplug/on-off-test.sh b/tools/testing/selftests/cpu-hotplug/on-off-test.sh old mode 100644 new mode 100755 diff --git a/tools/testing/selftests/efivarfs/Makefile b/tools/testing/selftests/efivarfs/Makefile index
