Re: [PATCH 3/3] selftests: silence test output by default

2017-09-18 Thread Shuah Khan 
On 09/18/2017 02:19 PM, Josef Bacik wrote:
> On Mon, Sep 18, 2017 at 01:48:31PM -0600, Shuah Khan wrote:
>> On 09/18/2017 12:24 PM, Josef Bacik wrote:
>>> On Mon, Sep 18, 2017 at 12:13:40PM -0600, Shuah Khan wrote:
 On 09/18/2017 11:52 AM, Josef Bacik wrote:
> On Mon, Sep 18, 2017 at 11:46:18AM -0600, Shuah Khan wrote:
>> On 09/18/2017 11:37 AM, jo...@toxicpanda.com wrote:
>>> From: Josef Bacik 
>>>
>>> Some of the networking tests are very noisy and make it impossible to
>>> see if we actually passed the tests as they run.  Default to suppressing
>>> the output from any tests run in order to make it easier to track what
>>> failed.
>>>
>>> Signed-off-by: Josef Bacik 
>>> --
>>
>> This change suppresses pass/fail wrapper output for all tests, not just 
>> the
>> networking tests.
>>
>> Could you please send me before and after results for what you are trying
>> to fix.
>>
>
> Yeah I wanted to suppress extraneous output from everybody, I just 
> happened to
> notice it because I was testing net.  The default thing already spits out 
> what
> it's running and pass/fail, there's no need to include all of the random 
> output
> unless the user wants to go and run the test manually.  As it is now it's
> _impossible_ to tell what ran and what passed/failed because of all the 
> random
> output.

 Unfortunately kselftests have lots of users that want different things. A 
 recent
 request is to use TAP13 format for output for external parsers to be able 
 to parse.
 That is what this change to add TAP13 header does.

 The output you are seeing is the TAP 13 format to indicate the test has 
 passed.

 The right fix would be to suppress the Pass/Fail from the individual shell 
 script
 and have the shell script exit with error code. kselftest lib.mk will 
 handle the
 error code and print out pass/fail like it is doing now.

 Using the common logic will help avoid duplicate code in tests/test 
 scripts and
 also makes the pass/fail messages consistent.

 In the following output the individual test output can be eliminated since 
 lib.mk
 run_tests does that for you. In addition, you will also get a count of 
 tests at
 the end of the run of all tests in a test directory.

 TAP version 13
 selftests: run_netsocktests
 
 
 running socket test
 
 [PASS]
 ok 1..1 selftests: run_netsocktests [PASS]
 selftests: run_afpackettests
 
 must be run as root
 ok 1..2 selftests: run_afpackettests [PASS]
 selftests: test_bpf.sh
 
 test_bpf: [FAIL]
 not ok 1..3 selftests:  test_bpf.sh [FAIL]
 selftests: netdevice.sh
 
 SKIP: Need root privileges
 ok 1..4 selftests: netdevice.sh [PASS]

 If you eliminate that you will just see the common lib.mk results.

 TAP version 13
 selftests: run_netsocktests
 
 ok 1..1 selftests: run_netsocktests [PASS]
 selftests: run_afpackettests
 
 must be run as root
 ok 1..2 selftests: run_afpackettests [PASS]
 
 selftests: test_bpf.sh
 
 not ok 1..3 selftests:  test_bpf.sh [FAIL]
 selftests: netdevice.sh
 
 SKIP: Need root privileges
 ok 1..4 selftests: netdevice.sh [PASS]


 If you would like to fix the duplicate output, please send me patches
 to remove pass/fail output strings from tests instead. It is on my
 todo to do that this release.

>>>
>>> I'm confused, this is exactly what my patch does, it strips all of the
>>> extraneous output and leaves only the TAP13 output.  Here is the output 
>>> without
>>> my suppression patch
>>>
>>> https://da.gd/pup0
>>>
>>> and here is the output with my suppression patch
>>>
>>> https://da.gd/3olKj
>>>
>>> Unless I'm missing something subtle it appears to be exactly the output you
>>> want, without the random crap from the other tests.  The only thing I'm
>>> redirecting is the output of the _test_ itself, $$BASENAME_TEST from what I 
>>> can
>>> tell is the actual test we're running, not the wrapper, so everything is as 
>>> it
>>> should be.  Thanks,
>>>
>>> Josef
>>>
>>
>> Yes you are right. Yes I see that you are redirecting all
>> output from the test with (./$$BASENAME_TEST > /dev/null 2>&1)
>> My bad.
>>
>> cd `dirname $$TEST` > /dev/null; (./$$BASENAME_TEST > /dev/null 2>&1 && echo 
>> "ok 1..$$test_num selftests:

Re: [PATCH 3/3] selftests: silence test output by default

2017-09-18 Thread Josef Bacik 
On Mon, Sep 18, 2017 at 01:48:31PM -0600, Shuah Khan wrote:
> On 09/18/2017 12:24 PM, Josef Bacik wrote:
> > On Mon, Sep 18, 2017 at 12:13:40PM -0600, Shuah Khan wrote:
> >> On 09/18/2017 11:52 AM, Josef Bacik wrote:
> >>> On Mon, Sep 18, 2017 at 11:46:18AM -0600, Shuah Khan wrote:
>  On 09/18/2017 11:37 AM, jo...@toxicpanda.com wrote:
> > From: Josef Bacik 
> >
> > Some of the networking tests are very noisy and make it impossible to
> > see if we actually passed the tests as they run.  Default to suppressing
> > the output from any tests run in order to make it easier to track what
> > failed.
> >
> > Signed-off-by: Josef Bacik 
> > --
> 
>  This change suppresses pass/fail wrapper output for all tests, not just 
>  the
>  networking tests.
> 
>  Could you please send me before and after results for what you are trying
>  to fix.
> 
> >>>
> >>> Yeah I wanted to suppress extraneous output from everybody, I just 
> >>> happened to
> >>> notice it because I was testing net.  The default thing already spits out 
> >>> what
> >>> it's running and pass/fail, there's no need to include all of the random 
> >>> output
> >>> unless the user wants to go and run the test manually.  As it is now it's
> >>> _impossible_ to tell what ran and what passed/failed because of all the 
> >>> random
> >>> output.
> >>
> >> Unfortunately kselftests have lots of users that want different things. A 
> >> recent
> >> request is to use TAP13 format for output for external parsers to be able 
> >> to parse.
> >> That is what this change to add TAP13 header does.
> >>
> >> The output you are seeing is the TAP 13 format to indicate the test has 
> >> passed.
> >>
> >> The right fix would be to suppress the Pass/Fail from the individual shell 
> >> script
> >> and have the shell script exit with error code. kselftest lib.mk will 
> >> handle the
> >> error code and print out pass/fail like it is doing now.
> >>
> >> Using the common logic will help avoid duplicate code in tests/test 
> >> scripts and
> >> also makes the pass/fail messages consistent.
> >>
> >> In the following output the individual test output can be eliminated since 
> >> lib.mk
> >> run_tests does that for you. In addition, you will also get a count of 
> >> tests at
> >> the end of the run of all tests in a test directory.
> >>
> >> TAP version 13
> >> selftests: run_netsocktests
> >> 
> >> 
> >> running socket test
> >> 
> >> [PASS]
> >> ok 1..1 selftests: run_netsocktests [PASS]
> >> selftests: run_afpackettests
> >> 
> >> must be run as root
> >> ok 1..2 selftests: run_afpackettests [PASS]
> >> selftests: test_bpf.sh
> >> 
> >> test_bpf: [FAIL]
> >> not ok 1..3 selftests:  test_bpf.sh [FAIL]
> >> selftests: netdevice.sh
> >> 
> >> SKIP: Need root privileges
> >> ok 1..4 selftests: netdevice.sh [PASS]
> >>
> >> If you eliminate that you will just see the common lib.mk results.
> >>
> >> TAP version 13
> >> selftests: run_netsocktests
> >> 
> >> ok 1..1 selftests: run_netsocktests [PASS]
> >> selftests: run_afpackettests
> >> 
> >> must be run as root
> >> ok 1..2 selftests: run_afpackettests [PASS]
> >> 
> >> selftests: test_bpf.sh
> >> 
> >> not ok 1..3 selftests:  test_bpf.sh [FAIL]
> >> selftests: netdevice.sh
> >> 
> >> SKIP: Need root privileges
> >> ok 1..4 selftests: netdevice.sh [PASS]
> >>
> >>
> >> If you would like to fix the duplicate output, please send me patches
> >> to remove pass/fail output strings from tests instead. It is on my
> >> todo to do that this release.
> >>
> > 
> > I'm confused, this is exactly what my patch does, it strips all of the
> > extraneous output and leaves only the TAP13 output.  Here is the output 
> > without
> > my suppression patch
> > 
> > https://da.gd/pup0
> > 
> > and here is the output with my suppression patch
> > 
> > https://da.gd/3olKj
> > 
> > Unless I'm missing something subtle it appears to be exactly the output you
> > want, without the random crap from the other tests.  The only thing I'm
> > redirecting is the output of the _test_ itself, $$BASENAME_TEST from what I 
> > can
> > tell is the actual test we're running, not the wrapper, so everything is as 
> > it
> > should be.  Thanks,
> > 
> > Josef
> > 
> 
> Yes you are right. Yes I see that you are redirecting all
> output from the test with (./$$BASENAME_TEST > /dev/null 2>&1)
> My bad.
> 
> cd `dirname $$TEST` > /dev/null; (./$$BASENAME_TEST > /dev/null 2>&1 && echo 
> "ok 1..$$test_num selftests: $$BASENAME_TEST [PASS]") || echo "not ok 
> 1..$$test_num

Re: [PATCH 3/3] selftests: silence test output by default

2017-09-18 Thread Shuah Khan 
On 09/18/2017 12:24 PM, Josef Bacik wrote:
> On Mon, Sep 18, 2017 at 12:13:40PM -0600, Shuah Khan wrote:
>> On 09/18/2017 11:52 AM, Josef Bacik wrote:
>>> On Mon, Sep 18, 2017 at 11:46:18AM -0600, Shuah Khan wrote:
 On 09/18/2017 11:37 AM, jo...@toxicpanda.com wrote:
> From: Josef Bacik 
>
> Some of the networking tests are very noisy and make it impossible to
> see if we actually passed the tests as they run.  Default to suppressing
> the output from any tests run in order to make it easier to track what
> failed.
>
> Signed-off-by: Josef Bacik 
> --

 This change suppresses pass/fail wrapper output for all tests, not just the
 networking tests.

 Could you please send me before and after results for what you are trying
 to fix.

>>>
>>> Yeah I wanted to suppress extraneous output from everybody, I just happened 
>>> to
>>> notice it because I was testing net.  The default thing already spits out 
>>> what
>>> it's running and pass/fail, there's no need to include all of the random 
>>> output
>>> unless the user wants to go and run the test manually.  As it is now it's
>>> _impossible_ to tell what ran and what passed/failed because of all the 
>>> random
>>> output.
>>
>> Unfortunately kselftests have lots of users that want different things. A 
>> recent
>> request is to use TAP13 format for output for external parsers to be able to 
>> parse.
>> That is what this change to add TAP13 header does.
>>
>> The output you are seeing is the TAP 13 format to indicate the test has 
>> passed.
>>
>> The right fix would be to suppress the Pass/Fail from the individual shell 
>> script
>> and have the shell script exit with error code. kselftest lib.mk will handle 
>> the
>> error code and print out pass/fail like it is doing now.
>>
>> Using the common logic will help avoid duplicate code in tests/test scripts 
>> and
>> also makes the pass/fail messages consistent.
>>
>> In the following output the individual test output can be eliminated since 
>> lib.mk
>> run_tests does that for you. In addition, you will also get a count of tests 
>> at
>> the end of the run of all tests in a test directory.
>>
>> TAP version 13
>> selftests: run_netsocktests
>> 
>> 
>> running socket test
>> 
>> [PASS]
>> ok 1..1 selftests: run_netsocktests [PASS]
>> selftests: run_afpackettests
>> 
>> must be run as root
>> ok 1..2 selftests: run_afpackettests [PASS]
>> selftests: test_bpf.sh
>> 
>> test_bpf: [FAIL]
>> not ok 1..3 selftests:  test_bpf.sh [FAIL]
>> selftests: netdevice.sh
>> 
>> SKIP: Need root privileges
>> ok 1..4 selftests: netdevice.sh [PASS]
>>
>> If you eliminate that you will just see the common lib.mk results.
>>
>> TAP version 13
>> selftests: run_netsocktests
>> 
>> ok 1..1 selftests: run_netsocktests [PASS]
>> selftests: run_afpackettests
>> 
>> must be run as root
>> ok 1..2 selftests: run_afpackettests [PASS]
>> 
>> selftests: test_bpf.sh
>> 
>> not ok 1..3 selftests:  test_bpf.sh [FAIL]
>> selftests: netdevice.sh
>> 
>> SKIP: Need root privileges
>> ok 1..4 selftests: netdevice.sh [PASS]
>>
>>
>> If you would like to fix the duplicate output, please send me patches
>> to remove pass/fail output strings from tests instead. It is on my
>> todo to do that this release.
>>
> 
> I'm confused, this is exactly what my patch does, it strips all of the
> extraneous output and leaves only the TAP13 output.  Here is the output 
> without
> my suppression patch
> 
> https://da.gd/pup0
> 
> and here is the output with my suppression patch
> 
> https://da.gd/3olKj
> 
> Unless I'm missing something subtle it appears to be exactly the output you
> want, without the random crap from the other tests.  The only thing I'm
> redirecting is the output of the _test_ itself, $$BASENAME_TEST from what I 
> can
> tell is the actual test we're running, not the wrapper, so everything is as it
> should be.  Thanks,
> 
> Josef
> 

Yes you are right. Yes I see that you are redirecting all
output from the test with (./$$BASENAME_TEST > /dev/null 2>&1)
My bad.

cd `dirname $$TEST` > /dev/null; (./$$BASENAME_TEST > /dev/null 2>&1 && echo 
"ok 1..$$test_num selftests: $$BASENAME_TEST [PASS]") || echo "not ok 
1..$$test_num selftests:  $$BASENAME_TEST [FAIL]"; cd - > /dev/null;\

However, it also suppresses important error messages from individual tests
like in the case of rtnetlink.sh from your before and after output.

User knows the test failed, but doesn't know why. One way to solve
the problem is creating a temp file with the output for reference.

Re: [PATCH 3/3] selftests: silence test output by default

2017-09-18 Thread Josef Bacik 
On Mon, Sep 18, 2017 at 12:13:40PM -0600, Shuah Khan wrote:
> On 09/18/2017 11:52 AM, Josef Bacik wrote:
> > On Mon, Sep 18, 2017 at 11:46:18AM -0600, Shuah Khan wrote:
> >> On 09/18/2017 11:37 AM, jo...@toxicpanda.com wrote:
> >>> From: Josef Bacik 
> >>>
> >>> Some of the networking tests are very noisy and make it impossible to
> >>> see if we actually passed the tests as they run.  Default to suppressing
> >>> the output from any tests run in order to make it easier to track what
> >>> failed.
> >>>
> >>> Signed-off-by: Josef Bacik 
> >>> --
> >>
> >> This change suppresses pass/fail wrapper output for all tests, not just the
> >> networking tests.
> >>
> >> Could you please send me before and after results for what you are trying
> >> to fix.
> >>
> > 
> > Yeah I wanted to suppress extraneous output from everybody, I just happened 
> > to
> > notice it because I was testing net.  The default thing already spits out 
> > what
> > it's running and pass/fail, there's no need to include all of the random 
> > output
> > unless the user wants to go and run the test manually.  As it is now it's
> > _impossible_ to tell what ran and what passed/failed because of all the 
> > random
> > output.
> 
> Unfortunately kselftests have lots of users that want different things. A 
> recent
> request is to use TAP13 format for output for external parsers to be able to 
> parse.
> That is what this change to add TAP13 header does.
> 
> The output you are seeing is the TAP 13 format to indicate the test has 
> passed.
> 
> The right fix would be to suppress the Pass/Fail from the individual shell 
> script
> and have the shell script exit with error code. kselftest lib.mk will handle 
> the
> error code and print out pass/fail like it is doing now.
> 
> Using the common logic will help avoid duplicate code in tests/test scripts 
> and
> also makes the pass/fail messages consistent.
> 
> In the following output the individual test output can be eliminated since 
> lib.mk
> run_tests does that for you. In addition, you will also get a count of tests 
> at
> the end of the run of all tests in a test directory.
> 
> TAP version 13
> selftests: run_netsocktests
> 
> 
> running socket test
> 
> [PASS]
> ok 1..1 selftests: run_netsocktests [PASS]
> selftests: run_afpackettests
> 
> must be run as root
> ok 1..2 selftests: run_afpackettests [PASS]
> selftests: test_bpf.sh
> 
> test_bpf: [FAIL]
> not ok 1..3 selftests:  test_bpf.sh [FAIL]
> selftests: netdevice.sh
> 
> SKIP: Need root privileges
> ok 1..4 selftests: netdevice.sh [PASS]
> 
> If you eliminate that you will just see the common lib.mk results.
> 
> TAP version 13
> selftests: run_netsocktests
> 
> ok 1..1 selftests: run_netsocktests [PASS]
> selftests: run_afpackettests
> 
> must be run as root
> ok 1..2 selftests: run_afpackettests [PASS]
> 
> selftests: test_bpf.sh
> 
> not ok 1..3 selftests:  test_bpf.sh [FAIL]
> selftests: netdevice.sh
> 
> SKIP: Need root privileges
> ok 1..4 selftests: netdevice.sh [PASS]
> 
> 
> If you would like to fix the duplicate output, please send me patches
> to remove pass/fail output strings from tests instead. It is on my
> todo to do that this release.
> 

I'm confused, this is exactly what my patch does, it strips all of the
extraneous output and leaves only the TAP13 output.  Here is the output without
my suppression patch

https://da.gd/pup0

and here is the output with my suppression patch

https://da.gd/3olKj

Unless I'm missing something subtle it appears to be exactly the output you
want, without the random crap from the other tests.  The only thing I'm
redirecting is the output of the _test_ itself, $$BASENAME_TEST from what I can
tell is the actual test we're running, not the wrapper, so everything is as it
should be.  Thanks,

Josef

Re: [PATCH 3/3] selftests: silence test output by default

2017-09-18 Thread Shuah Khan 
On 09/18/2017 11:52 AM, Josef Bacik wrote:
> On Mon, Sep 18, 2017 at 11:46:18AM -0600, Shuah Khan wrote:
>> On 09/18/2017 11:37 AM, jo...@toxicpanda.com wrote:
>>> From: Josef Bacik 
>>>
>>> Some of the networking tests are very noisy and make it impossible to
>>> see if we actually passed the tests as they run.  Default to suppressing
>>> the output from any tests run in order to make it easier to track what
>>> failed.
>>>
>>> Signed-off-by: Josef Bacik 
>>> --
>>
>> This change suppresses pass/fail wrapper output for all tests, not just the
>> networking tests.
>>
>> Could you please send me before and after results for what you are trying
>> to fix.
>>
> 
> Yeah I wanted to suppress extraneous output from everybody, I just happened to
> notice it because I was testing net.  The default thing already spits out what
> it's running and pass/fail, there's no need to include all of the random 
> output
> unless the user wants to go and run the test manually.  As it is now it's
> _impossible_ to tell what ran and what passed/failed because of all the random
> output.

Unfortunately kselftests have lots of users that want different things. A recent
request is to use TAP13 format for output for external parsers to be able to 
parse.
That is what this change to add TAP13 header does.

The output you are seeing is the TAP 13 format to indicate the test has passed.

The right fix would be to suppress the Pass/Fail from the individual shell 
script
and have the shell script exit with error code. kselftest lib.mk will handle the
error code and print out pass/fail like it is doing now.

Using the common logic will help avoid duplicate code in tests/test scripts and
also makes the pass/fail messages consistent.

In the following output the individual test output can be eliminated since 
lib.mk
run_tests does that for you. In addition, you will also get a count of tests at
the end of the run of all tests in a test directory.

TAP version 13
selftests: run_netsocktests


running socket test

[PASS]
ok 1..1 selftests: run_netsocktests [PASS]
selftests: run_afpackettests

must be run as root
ok 1..2 selftests: run_afpackettests [PASS]
selftests: test_bpf.sh

test_bpf: [FAIL]
not ok 1..3 selftests:  test_bpf.sh [FAIL]
selftests: netdevice.sh

SKIP: Need root privileges
ok 1..4 selftests: netdevice.sh [PASS]

If you eliminate that you will just see the common lib.mk results.

TAP version 13
selftests: run_netsocktests

ok 1..1 selftests: run_netsocktests [PASS]
selftests: run_afpackettests

must be run as root
ok 1..2 selftests: run_afpackettests [PASS]

selftests: test_bpf.sh

not ok 1..3 selftests:  test_bpf.sh [FAIL]
selftests: netdevice.sh

SKIP: Need root privileges
ok 1..4 selftests: netdevice.sh [PASS]


If you would like to fix the duplicate output, please send me patches
to remove pass/fail output strings from tests instead. It is on my
todo to do that this release.


thanks,
-- Shuah

Re: [PATCH 3/3] selftests: silence test output by default

2017-09-18 Thread Josef Bacik 
On Mon, Sep 18, 2017 at 11:46:18AM -0600, Shuah Khan wrote:
> On 09/18/2017 11:37 AM, jo...@toxicpanda.com wrote:
> > From: Josef Bacik 
> > 
> > Some of the networking tests are very noisy and make it impossible to
> > see if we actually passed the tests as they run.  Default to suppressing
> > the output from any tests run in order to make it easier to track what
> > failed.
> > 
> > Signed-off-by: Josef Bacik 
> > --
> 
> This change suppresses pass/fail wrapper output for all tests, not just the
> networking tests.
> 
> Could you please send me before and after results for what you are trying
> to fix.
> 

Yeah I wanted to suppress extraneous output from everybody, I just happened to
notice it because I was testing net.  The default thing already spits out what
it's running and pass/fail, there's no need to include all of the random output
unless the user wants to go and run the test manually.  As it is now it's
_impossible_ to tell what ran and what passed/failed because of all the random
output.

Ideally kselftests would work like xfstests does and simply capture the output
to a log so you could go check afterwards, but that's a lot more work.  Making
it easier to tell which tests passed/failed is a good enough first step.
Thanks,

Josef

Re: [PATCH 3/3] selftests: silence test output by default

2017-09-18 Thread Shuah Khan 
On 09/18/2017 11:37 AM, jo...@toxicpanda.com wrote:
> From: Josef Bacik 
> 
> Some of the networking tests are very noisy and make it impossible to
> see if we actually passed the tests as they run.  Default to suppressing
> the output from any tests run in order to make it easier to track what
> failed.
> 
> Signed-off-by: Josef Bacik 
> --

This change suppresses pass/fail wrapper output for all tests, not just the
networking tests.

Could you please send me before and after results for what you are trying
to fix.

>  tools/testing/selftests/lib.mk | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
> index 693616651da5..223234cd98e9 100644
> --- a/tools/testing/selftests/lib.mk
> +++ b/tools/testing/selftests/lib.mk
> @@ -24,7 +24,7 @@ define RUN_TESTS
>   echo "selftests: Warning: file $$BASENAME_TEST is not 
> executable, correct this.";\
>   echo "not ok 1..$$test_num selftests: $$BASENAME_TEST 
> [FAIL]"; \
>   else\
> - cd `dirname $$TEST` > /dev/null; (./$$BASENAME_TEST && 
> echo "ok 1..$$test_num selftests: $$BASENAME_TEST [PASS]") || echo "not ok 
> 1..$$test_num selftests:  $$BASENAME_TEST [FAIL]"; cd - > /dev/null;\
> + cd `dirname $$TEST` > /dev/null; (./$$BASENAME_TEST > 
> /dev/null 2>&1 && echo "ok 1..$$test_num selftests: $$BASENAME_TEST [PASS]") 
> || echo "not ok 1..$$test_num selftests:  $$BASENAME_TEST [FAIL]"; cd - > 
> /dev/null;\
>   fi; \
>   done;
>  endef
> @@ -55,7 +55,7 @@ endif
>  define EMIT_TESTS
>   @for TEST in $(TEST_GEN_PROGS) $(TEST_PROGS); do \
>   BASENAME_TEST=`basename $$TEST`;\
> - echo "(./$$BASENAME_TEST && echo \"selftests: $$BASENAME_TEST 
> [PASS]\") || echo \"selftests: $$BASENAME_TEST [FAIL]\""; \
> + echo "(./$$BASENAME_TEST > /dev/null 2>&1 && echo \"selftests: 
> $$BASENAME_TEST [PASS]\") || echo \"selftests: $$BASENAME_TEST [FAIL]\""; \
>   done;
>  endef
>  
> 

thanks,
-- Shuah

[PATCH 3/3] selftests: silence test output by default

2017-09-18 Thread josef 
From: Josef Bacik 

Some of the networking tests are very noisy and make it impossible to
see if we actually passed the tests as they run.  Default to suppressing
the output from any tests run in order to make it easier to track what
failed.

Signed-off-by: Josef Bacik 
---
 tools/testing/selftests/lib.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 693616651da5..223234cd98e9 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -24,7 +24,7 @@ define RUN_TESTS
echo "selftests: Warning: file $$BASENAME_TEST is not 
executable, correct this.";\
echo "not ok 1..$$test_num selftests: $$BASENAME_TEST 
[FAIL]"; \
else\
-   cd `dirname $$TEST` > /dev/null; (./$$BASENAME_TEST && 
echo "ok 1..$$test_num selftests: $$BASENAME_TEST [PASS]") || echo "not ok 
1..$$test_num selftests:  $$BASENAME_TEST [FAIL]"; cd - > /dev/null;\
+   cd `dirname $$TEST` > /dev/null; (./$$BASENAME_TEST > 
/dev/null 2>&1 && echo "ok 1..$$test_num selftests: $$BASENAME_TEST [PASS]") || 
echo "not ok 1..$$test_num selftests:  $$BASENAME_TEST [FAIL]"; cd - > 
/dev/null;\
fi; \
done;
 endef
@@ -55,7 +55,7 @@ endif
 define EMIT_TESTS
@for TEST in $(TEST_GEN_PROGS) $(TEST_PROGS); do \
BASENAME_TEST=`basename $$TEST`;\
-   echo "(./$$BASENAME_TEST && echo \"selftests: $$BASENAME_TEST 
[PASS]\") || echo \"selftests: $$BASENAME_TEST [FAIL]\""; \
+   echo "(./$$BASENAME_TEST > /dev/null 2>&1 && echo \"selftests: 
$$BASENAME_TEST [PASS]\") || echo \"selftests: $$BASENAME_TEST [FAIL]\""; \
done;
 endef
 
-- 
2.7.4