Re: [PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-04-02 Thread Shuah Khan

On 4/2/21 1:27 PM, Daniel Latypov wrote:

On Fri, Apr 2, 2021 at 11:00 AM Shuah Khan  wrote:


On 4/2/21 3:32 AM, Brendan Higgins wrote:

TL;DR
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit

Per suggestion from Ted [1], we can reduce the amount of typing by
assuming a convention that these files are named '.kunitconfig'.

In the case of [1], we now have
$ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4

Also add in such a fragment for kunit itself so we can give that as an
example more close to home (and thus less likely to be accidentally
broken).

[1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/

Signed-off-by: Daniel Latypov 


Reviewed-by: Brendan Higgins 



Should this be captured in  documentation. Especially since this
is file is .* file.

Do you want to include doc in this patch? Might be better that way.


It definitely should be documented, yes.
The only real example hadn't landed yet when I sent this patch
(fs/ext4/.kunitconfig was going in through the ext4 tree), but now
it's in linus/master.

There's still some uncertainties about what best practices for this
feature should be, i.e.
* how granular should these be?
* how should configs in parent dirs be handled? Should they be
supersets of all the subdirs?
 * E.g. should fs/.kunitconfig be a superset of
fs/ext4/.kunitconfig and any other hypothetical subdir configs?
 * Should we wait on saying "you should do this" until we have
"import" statements/other mechanisms to make this less manual?
* how should we handle non-UML tests, like the KASAN tests?
   * ideally, kunit.py run will eventually support running tests on x86
(using qemu)

If it's fine with you, I was hoping to come back and add a section to
kunit/start.rst when we've had some of those questions more figured
out.



Sound good. I will apply this patch and you can document later.

thanks,
-- Shuah


Re: [PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-04-02 Thread Daniel Latypov
On Fri, Apr 2, 2021 at 11:00 AM Shuah Khan  wrote:
>
> On 4/2/21 3:32 AM, Brendan Higgins wrote:
> >> TL;DR
> >> $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit
> >>
> >> Per suggestion from Ted [1], we can reduce the amount of typing by
> >> assuming a convention that these files are named '.kunitconfig'.
> >>
> >> In the case of [1], we now have
> >> $ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4
> >>
> >> Also add in such a fragment for kunit itself so we can give that as an
> >> example more close to home (and thus less likely to be accidentally
> >> broken).
> >>
> >> [1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/
> >>
> >> Signed-off-by: Daniel Latypov 
> >
> > Reviewed-by: Brendan Higgins 
> >
>
> Should this be captured in  documentation. Especially since this
> is file is .* file.
>
> Do you want to include doc in this patch? Might be better that way.

It definitely should be documented, yes.
The only real example hadn't landed yet when I sent this patch
(fs/ext4/.kunitconfig was going in through the ext4 tree), but now
it's in linus/master.

There's still some uncertainties about what best practices for this
feature should be, i.e.
* how granular should these be?
* how should configs in parent dirs be handled? Should they be
supersets of all the subdirs?
* E.g. should fs/.kunitconfig be a superset of
fs/ext4/.kunitconfig and any other hypothetical subdir configs?
* Should we wait on saying "you should do this" until we have
"import" statements/other mechanisms to make this less manual?
* how should we handle non-UML tests, like the KASAN tests?
  * ideally, kunit.py run will eventually support running tests on x86
(using qemu)

If it's fine with you, I was hoping to come back and add a section to
kunit/start.rst when we've had some of those questions more figured
out.

>
> thanks,
> -- Shuah


Re: [PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-04-02 Thread Shuah Khan

On 4/2/21 3:32 AM, Brendan Higgins wrote:

TL;DR
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit

Per suggestion from Ted [1], we can reduce the amount of typing by
assuming a convention that these files are named '.kunitconfig'.

In the case of [1], we now have
$ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4

Also add in such a fragment for kunit itself so we can give that as an
example more close to home (and thus less likely to be accidentally
broken).

[1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/

Signed-off-by: Daniel Latypov 


Reviewed-by: Brendan Higgins 



Should this be captured in  documentation. Especially since this
is file is .* file.

Do you want to include doc in this patch? Might be better that way.

thanks,
-- Shuah


Re: [PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-04-02 Thread Brendan Higgins
> TL;DR
> $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit
> 
> Per suggestion from Ted [1], we can reduce the amount of typing by
> assuming a convention that these files are named '.kunitconfig'.
> 
> In the case of [1], we now have
> $ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4
> 
> Also add in such a fragment for kunit itself so we can give that as an
> example more close to home (and thus less likely to be accidentally
> broken).
> 
> [1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/
> 
> Signed-off-by: Daniel Latypov 

Reviewed-by: Brendan Higgins 


Re: [PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-02-22 Thread David Gow
On Tue, Feb 23, 2021 at 6:52 AM Daniel Latypov  wrote:
>
> TL;DR
> $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit
>
> Per suggestion from Ted [1], we can reduce the amount of typing by
> assuming a convention that these files are named '.kunitconfig'.
>
> In the case of [1], we now have
> $ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4
>
> Also add in such a fragment for kunit itself so we can give that as an
> example more close to home (and thus less likely to be accidentally
> broken).
>
> [1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/
>
> Signed-off-by: Daniel Latypov 
> ---

Thanks! I really like this.

I'd assumed we'd check if the path exists, and fall back to appending
".kunitconfig", but checking if it's a directory is better.

I tried this out with all the different combinations I could think of,
and it works well.

Reviewed-by: David Gow 

Cheers,
-- David

>  lib/kunit/.kunitconfig | 3 +++
>  tools/testing/kunit/kunit.py   | 4 +++-
>  tools/testing/kunit/kunit_kernel.py| 2 ++
>  tools/testing/kunit/kunit_tool_test.py | 6 ++
>  4 files changed, 14 insertions(+), 1 deletion(-)
>  create mode 100644 lib/kunit/.kunitconfig
>
> diff --git a/lib/kunit/.kunitconfig b/lib/kunit/.kunitconfig
> new file mode 100644
> index ..9235b7d42d38
> --- /dev/null
> +++ b/lib/kunit/.kunitconfig
> @@ -0,0 +1,3 @@
> +CONFIG_KUNIT=y
> +CONFIG_KUNIT_TEST=y
> +CONFIG_KUNIT_EXAMPLE_TEST=y
> diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
> index d5144fcb03ac..5da8fb3762f9 100755
> --- a/tools/testing/kunit/kunit.py
> +++ b/tools/testing/kunit/kunit.py
> @@ -184,7 +184,9 @@ def add_common_opts(parser) -> None:
> help='Run all KUnit tests through allyesconfig',
> action='store_true')
> parser.add_argument('--kunitconfig',
> -help='Path to Kconfig fragment that enables 
> KUnit tests',
> +help='Path to Kconfig fragment that enables 
> KUnit tests.'
> +' If given a directory, (e.g. lib/kunit), 
> "/.kunitconfig" '
> +'will get  automatically appended.',
>  metavar='kunitconfig')
>
>  def add_build_opts(parser) -> None:
> diff --git a/tools/testing/kunit/kunit_kernel.py 
> b/tools/testing/kunit/kunit_kernel.py
> index f309a33256cd..89a7d4024e87 100644
> --- a/tools/testing/kunit/kunit_kernel.py
> +++ b/tools/testing/kunit/kunit_kernel.py
> @@ -132,6 +132,8 @@ class LinuxSourceTree(object):
> return
>
> if kunitconfig_path:
> +   if os.path.isdir(kunitconfig_path):
> +   kunitconfig_path = 
> os.path.join(kunitconfig_path, KUNITCONFIG_PATH)
> if not os.path.exists(kunitconfig_path):
> raise ConfigError(f'Specified kunitconfig 
> ({kunitconfig_path}) does not exist')
> else:
> diff --git a/tools/testing/kunit/kunit_tool_test.py 
> b/tools/testing/kunit/kunit_tool_test.py
> index 1ad3049e9069..2e809dd956a7 100755
> --- a/tools/testing/kunit/kunit_tool_test.py
> +++ b/tools/testing/kunit/kunit_tool_test.py
> @@ -251,6 +251,12 @@ class LinuxSourceTreeTest(unittest.TestCase):
> with tempfile.NamedTemporaryFile('wt') as kunitconfig:
> tree = kunit_kernel.LinuxSourceTree('', 
> kunitconfig_path=kunitconfig.name)
>
> +   def test_dir_kunitconfig(self):
> +   with tempfile.TemporaryDirectory('') as dir:
> +   with open(os.path.join(dir, '.kunitconfig'), 'w') as 
> f:
> +   pass
> +   tree = kunit_kernel.LinuxSourceTree('', 
> kunitconfig_path=dir)
> +
> # TODO: add more test cases.
>
>
>
> base-commit: b12b47249688915e987a9a2a393b522f86f6b7ab
> --
> 2.30.0.617.g56c4b15f3c-goog
>


[PATCH] kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragment

2021-02-22 Thread Daniel Latypov
TL;DR
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit

Per suggestion from Ted [1], we can reduce the amount of typing by
assuming a convention that these files are named '.kunitconfig'.

In the case of [1], we now have
$ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4

Also add in such a fragment for kunit itself so we can give that as an
example more close to home (and thus less likely to be accidentally
broken).

[1] https://lore.kernel.org/linux-ext4/ycnf4yp1db97z...@mit.edu/

Signed-off-by: Daniel Latypov 
---
 lib/kunit/.kunitconfig | 3 +++
 tools/testing/kunit/kunit.py   | 4 +++-
 tools/testing/kunit/kunit_kernel.py| 2 ++
 tools/testing/kunit/kunit_tool_test.py | 6 ++
 4 files changed, 14 insertions(+), 1 deletion(-)
 create mode 100644 lib/kunit/.kunitconfig

diff --git a/lib/kunit/.kunitconfig b/lib/kunit/.kunitconfig
new file mode 100644
index ..9235b7d42d38
--- /dev/null
+++ b/lib/kunit/.kunitconfig
@@ -0,0 +1,3 @@
+CONFIG_KUNIT=y
+CONFIG_KUNIT_TEST=y
+CONFIG_KUNIT_EXAMPLE_TEST=y
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index d5144fcb03ac..5da8fb3762f9 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -184,7 +184,9 @@ def add_common_opts(parser) -> None:
help='Run all KUnit tests through allyesconfig',
action='store_true')
parser.add_argument('--kunitconfig',
-help='Path to Kconfig fragment that enables KUnit 
tests',
+help='Path to Kconfig fragment that enables KUnit 
tests.'
+' If given a directory, (e.g. lib/kunit), 
"/.kunitconfig" '
+'will get  automatically appended.',
 metavar='kunitconfig')
 
 def add_build_opts(parser) -> None:
diff --git a/tools/testing/kunit/kunit_kernel.py 
b/tools/testing/kunit/kunit_kernel.py
index f309a33256cd..89a7d4024e87 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -132,6 +132,8 @@ class LinuxSourceTree(object):
return
 
if kunitconfig_path:
+   if os.path.isdir(kunitconfig_path):
+   kunitconfig_path = 
os.path.join(kunitconfig_path, KUNITCONFIG_PATH)
if not os.path.exists(kunitconfig_path):
raise ConfigError(f'Specified kunitconfig 
({kunitconfig_path}) does not exist')
else:
diff --git a/tools/testing/kunit/kunit_tool_test.py 
b/tools/testing/kunit/kunit_tool_test.py
index 1ad3049e9069..2e809dd956a7 100755
--- a/tools/testing/kunit/kunit_tool_test.py
+++ b/tools/testing/kunit/kunit_tool_test.py
@@ -251,6 +251,12 @@ class LinuxSourceTreeTest(unittest.TestCase):
with tempfile.NamedTemporaryFile('wt') as kunitconfig:
tree = kunit_kernel.LinuxSourceTree('', 
kunitconfig_path=kunitconfig.name)
 
+   def test_dir_kunitconfig(self):
+   with tempfile.TemporaryDirectory('') as dir:
+   with open(os.path.join(dir, '.kunitconfig'), 'w') as f:
+   pass
+   tree = kunit_kernel.LinuxSourceTree('', 
kunitconfig_path=dir)
+
# TODO: add more test cases.
 
 

base-commit: b12b47249688915e987a9a2a393b522f86f6b7ab
-- 
2.30.0.617.g56c4b15f3c-goog