Re: [PATCH v1 14/17] Documentation: kunit: add documentation for KUnit

[Brendan Higgins]

On Wed, Apr 24, 2019 at 11:58 AM Masayoshi Mizuma  wrote:
>
> Hi Brendan,
>
> KUNIT_ASSERT_NOT_ERR_OR_NULL() should be replaced to
> KUNIT_EXPECT_NOT_ERR_OR_NULL(), right?

Generally speaking, no.

There may be places that I have used it improperly, but I think it is
generally okay to use.

If you are refering to the discussion with Frank and Stephen in RFC
v4, my take was that the resolution was that there were some things I
needed to fix in its implementation, but it is conceptually okay, and
that Frank doesn't want me to use it in drivers/of/unittest.c.
Consequently, I fixed the implementation issues that Frank and Stephen
pointed out, and I am holding off on drivers/of/unittest.c for this
patchset. The usage in this file should be fine.

>
> On Thu, Apr 04, 2019 at 03:06:49PM -0700, Brendan Higgins wrote:
> > Add documentation for KUnit, the Linux kernel unit testing framework.
> > - Add intro and usage guide for KUnit
> > - Add API reference
> >
> > Signed-off-by: Felix Guo 
> > Signed-off-by: Brendan Higgins 
> > ---
>
> 



Cheers
___
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

Re: [PATCH v1 14/17] Documentation: kunit: add documentation for KUnit

[Masayoshi Mizuma]

Hi Brendan,

KUNIT_ASSERT_NOT_ERR_OR_NULL() should be replaced to
KUNIT_EXPECT_NOT_ERR_OR_NULL(), right?

On Thu, Apr 04, 2019 at 03:06:49PM -0700, Brendan Higgins wrote:
> Add documentation for KUnit, the Linux kernel unit testing framework.
> - Add intro and usage guide for KUnit
> - Add API reference
> 
> Signed-off-by: Felix Guo 
> Signed-off-by: Brendan Higgins 
> ---



> diff --git a/Documentation/kunit/start.rst b/Documentation/kunit/start.rst
> new file mode 100644
> index 0..5cdba5091905e
> --- /dev/null
> +++ b/Documentation/kunit/start.rst



> +Assertions
> +~~
> +
> +KUnit also has the concept of an *assertion*. An assertion is just like an
> +expectation except the assertion immediately terminates the test case if it 
> is
> +not satisfied.
> +
> +For example:
> +
> +.. code-block:: c
> +
> + static void mock_test_do_expect_default_return(struct kunit *test)
> + {
> + struct mock_test_context *ctx = test->priv;
> + struct mock *mock = ctx->mock;
> + int param0 = 5, param1 = -5;
> + const char *two_param_types[] = {"int", "int"};
> + const void *two_params[] = {, };
> + const void *ret;
> +
> + ret = mock->do_expect(mock,
> +   "test_printk", test_printk,
> +   two_param_types, two_params,
> +   ARRAY_SIZE(two_params));

> + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ret);

KUNIT_EXPECT_NOT_ERR_OR_NULL(test, ret);

> + KUNIT_EXPECT_EQ(test, -4, *((int *) ret));
> + }
> +
> +In this example, the method under test should return a pointer to a value, so
> +if the pointer returned by the method is null or an errno, we don't want to
> +bother continuing the test since the following expectation could crash the 
> test

> +case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us to bail out of the test case if

  +case. `KUNIT_EXPECT_NOT_ERR_OR_NULL(...)` allows us to bail out of the test 
case if

Thanks!
Masa
___
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

[PATCH v1 14/17] Documentation: kunit: add documentation for KUnit

[Brendan Higgins]

Add documentation for KUnit, the Linux kernel unit testing framework.
- Add intro and usage guide for KUnit
- Add API reference

Signed-off-by: Felix Guo 
Signed-off-by: Brendan Higgins 
---
 Documentation/index.rst   |   1 +
 Documentation/kunit/api/index.rst |  16 ++
 Documentation/kunit/api/test.rst  |  15 +
 Documentation/kunit/faq.rst   |  46 +++
 Documentation/kunit/index.rst |  80 ++
 Documentation/kunit/start.rst | 180 
 Documentation/kunit/usage.rst | 447 ++
 7 files changed, 785 insertions(+)
 create mode 100644 Documentation/kunit/api/index.rst
 create mode 100644 Documentation/kunit/api/test.rst
 create mode 100644 Documentation/kunit/faq.rst
 create mode 100644 Documentation/kunit/index.rst
 create mode 100644 Documentation/kunit/start.rst
 create mode 100644 Documentation/kunit/usage.rst

diff --git a/Documentation/index.rst b/Documentation/index.rst
index 80a421cb935e7..264cfd613a774 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -65,6 +65,7 @@ merged much easier.
kernel-hacking/index
trace/index
maintainer/index
+   kunit/index
 
 Kernel API documentation
 
diff --git a/Documentation/kunit/api/index.rst 
b/Documentation/kunit/api/index.rst
new file mode 100644
index 0..c31c530088153
--- /dev/null
+++ b/Documentation/kunit/api/index.rst
@@ -0,0 +1,16 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=
+API Reference
+=
+.. toctree::
+
+   test
+
+This section documents the KUnit kernel testing API. It is divided into 3
+sections:
+
+= 
==
+:doc:`test`   documents all of the standard testing API
+  excluding mocking or mocking related 
features.
+= 
==
diff --git a/Documentation/kunit/api/test.rst b/Documentation/kunit/api/test.rst
new file mode 100644
index 0..7c926014f047c
--- /dev/null
+++ b/Documentation/kunit/api/test.rst
@@ -0,0 +1,15 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+
+Test API
+
+
+This file documents all of the standard testing API excluding mocking or 
mocking
+related features.
+
+.. kernel-doc:: include/kunit/test.h
+   :internal:
+
+.. kernel-doc:: include/kunit/kunit-stream.h
+   :internal:
+
diff --git a/Documentation/kunit/faq.rst b/Documentation/kunit/faq.rst
new file mode 100644
index 0..cb8e4fb2257a0
--- /dev/null
+++ b/Documentation/kunit/faq.rst
@@ -0,0 +1,46 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=
+Frequently Asked Questions
+=
+
+How is this different from Autotest, kselftest, etc?
+
+KUnit is a unit testing framework. Autotest, kselftest (and some others) are
+not.
+
+A `unit test `_ is supposed to
+test a single unit of code in isolation, hence the name. A unit test should be
+the finest granularity of testing and as such should allow all possible code
+paths to be tested in the code under test; this is only possible if the code
+under test is very small and does not have any external dependencies outside of
+the test's control like hardware.
+
+There are no testing frameworks currently available for the kernel that do not
+require installing the kernel on a test machine or in a VM and all require
+tests to be written in userspace and run on the kernel under test; this is true
+for Autotest, kselftest, and some others, disqualifying any of them from being
+considered unit testing frameworks.
+
+What is the difference between a unit test and these other kinds of tests?
+==
+Most existing tests for the Linux kernel would be categorized as an integration
+test, or an end-to-end test.
+
+- A unit test is supposed to test a single unit of code in isolation, hence the
+  name. A unit test should be the finest granularity of testing and as such
+  should allow all possible code paths to be tested in the code under test; 
this
+  is only possible if the code under test is very small and does not have any
+  external dependencies outside of the test's control like hardware.
+- An integration test tests the interaction between a minimal set of 
components,
+  usually just two or three. For example, someone might write an integration
+  test to test the interaction between a driver and a piece of hardware, or to
+  test the interaction between the userspace libraries the kernel provides and
+  the kernel itself; however, one of these tests would probably not test the
+  entire kernel along with hardware interactions and interactions with the
+  userspace.
+- An end-to-end test usually tests the entire system from the