Alright, here's a second version, attempting to address as many concerns as
possible. It's likely I've missed something, though.
Changes from v1:
* Make scripts/get_maintainer.pl survive querying missing files, giving a
warning instead. This is necessary to enable scripts/checkpatch.pl to query
MAINTAINERS about files being deleted.
* Start with the minimal change just documenting the V: entry, which accepts
test commands directly, and tweaking the tools to deal with that.
* However, require the commands accept the -h/--help option so that users have
an easier time getting *some* help. The run_checks.py missing that is the
reason why the patch proposing it for kunit subsystem is marked "DONOTMERGE"
in this version. We can drop that requirement, or soften the language, if
there's opposition.
* Have a *separate* patch documenting 'Tested-with:' as the next (early)
change. Mention that you can add a '#' followed by a results URL, on the
end. Adjust the V: docs/checks to exclude '#'.
* Have a *separate* patch making scripts/checkpatch.pl propose the execution
of the test suite defined in MAINTAINERS whenever the corresponding
subsystem is changed.
* However, use 'CHECK', instead of 'WARNING', to allow submitters specify the
exact (and potentially slightly different) command they used, and not have
checkpatch.pl complain too loudly that they didn't run the (exact
MAINTAINERS-specified) command. This unfortunately means that unless you use
--strict, you won't see the message. We'll try to address that in a new
change at the end.
* Have a *separate* patch introducing the test catalog and accepting
references to that everywhere, with a special syntax to distinguish them
from verbatim/direct commands. The syntax is prepending the test name with a
'*' (just like C pointer dereference). Make checkpatch.pl handle that.
* Drop the recommendation to have the "Docs" and "Sources" fields in test
descriptions, as the description text should focus on giving a good
introduction and not prompt the user to go somewhere else immediately. They
both can be referenced in the text where and how is appropriate.
* Generally keep the previous changes adding V: entries and test suite docs,
and try to accommodate all the requests, but refine the "Summary" fields to
fit the checkpatch.pl messages better.
* Have a separate patch cataloguing the complete kunit suite.
* Finally, add a patch introducing the "proposal strength" keywords
(SUGGESTED/RECOMMENDED/REQUIRED) to the syntax of V: entries, which directly
affect which level of checkpatch.pl message missing 'Tested-with:' tags
would generate: CHECK/WARNING/ERROR respectively. This allows subsystems to
disable checkpatch.pl WARNINGS/ERRORS, and keep their test proposals
inobtrusive, if they so wish. E.g. if they expect people to change their
commands often. At the same time allow stricter workflows for subsystems
with more uniform testing. Or e.g. for subsystems which expect the tests to
explain their parameters in their output, and the submitters to upload and
link their results in their 'Tested-with:' tags.
That seems to be all, but I'm sure I forgot something :D
Anyway, send me more corrections and I'll try to address them, but it's likely
going to happen next year only.
Nick
---
Nikolai Kondrashov (9):
get_maintainer: Survive querying missing files
MAINTAINERS: Introduce V: entry for tests
MAINTAINERS: Propose kunit core tests for framework changes
docs: submitting-patches: Introduce Tested-with:
checkpatch: Propose tests to execute
MAINTAINERS: Support referencing test docs in V:
MAINTAINERS: Propose kvm-xfstests smoke for ext4
docs: tests: Document kunit in general
MAINTAINERS: Add proposal strength to V: entries
Mark Brown (1):
MAINTAINERS: Propose kunit tests for regmap
Documentation/process/index.rst | 1 +
Documentation/process/submitting-patches.rst | 46 +++++++
Documentation/process/tests.rst | 96 +++++++++++++++
MAINTAINERS | 17 +++
scripts/checkpatch.pl | 174 ++++++++++++++++++++++++++-
scripts/get_maintainer.pl | 23 +++-
scripts/parse-maintainers.pl | 3 +-
7 files changed, 355 insertions(+), 5 deletions(-)
---
