Hi Jon,
It is impressive how a single patch became a series with 25 ones ;-)
This version was rebased on next-20260127, which is after docs-mw
got merged with my doc patches, solving some merge conflicts.
It should now apply cleanly on the top of upstream and it will address
the kernel-doc issues with context analysis patches that are already
at linux-next.
On this version, I opted to do something I always wanted when
porting from the venerable kernel-doc perl logic: properly handle
macros without writing really complex regular expressions.
After those changes, the replacement logic becomes a lot more
easier to write, as we can have a cleaner logic to handle
function calls:
function_xforms = [
(KernRe(r"^static +"), ""),
(KernRe(r"^extern +"), ""),
...
(NestedMatch(r"__cond_acquires\s*\("), ""),
(NestedMatch(r"__cond_releases\s*\("), ""),
...
}
It can even be used to replace comma-separated arguments like here:
struct_xforms = [
...
(NestedMatch(r'\bstruct_group\s*\('), r'\2'),
(NestedMatch(r'\bstruct_group_attr\s*\('), r'\3'),
(NestedMatch(r'\bstruct_group_tagged\s*\('), r'struct \1 { \3 }
\2;'),
(NestedMatch(r'\b__struct_group\s*\('), r'\4'),
...
]
As both KernRe and NestedMatch have sub methods, they can be placed
altogether at the same transforms tables.
The regex syntax for them are quite simple: they should end with a
delimiter (usually open parenthesis, but brackets and square
brackets are also supported).
The first 15 patches on this series were co-developed with Randy,
with came up with the original patch. I ended helping identifying
kernel-doc issues and doing some changes to make the parser more
reliable.
Then, I added 10 other patches to improve NestedMatch.
The final two patches on it fixes an issue when parsing
include/net/page_pool/types.h. Right now, it doesn't parse
well struct page_pool_params, as the second struct_group_tagged()
has a /* private */ on it, causing the end parenthesis to be
commented out. After the series, the documentation for it
should be ok.
-
Even NestedMatch being more complex than KernRe, on my machine,
parsing all files is 5% faster than before, because we're not
parsing anymore macro definitions.
Ah, due to the complexity of NestedMatch, I opted to write
some unit tests to verify that the logic there is correct.
We can use it to add other border cases.
Using it is as easy as running:
$ tools/unittests/nested_match.py
(I opted to create a separate directory for it, as this
is not really documentation)
Mauro Carvalho Chehab (23):
docs: kdoc_re: add support for groups()
docs: kdoc_re: don't go past the end of a line
docs: kdoc_parser: move var transformers to the beginning
docs: kdoc_parser: don't mangle with function defines
docs: kdoc_parser: add functions support for NestedMatch
docs: kdoc_parser: use NestedMatch to handle __attribute__ on
functions
docs: kdoc_parser: fix variable regexes to work with size_t
docs: kdoc_parser: fix the default_value logic for variables
docs: kdoc_parser: add some debug for variable parsing
docs: kdoc_parser: don't exclude defaults from prototype
docs: kdoc_parser: fix parser to support multi-word types
docs: kdoc_parser: add support for LIST_HEAD
docs: kdoc_re: properly handle strings and escape chars on it
docs: kdoc_re: better show KernRe() at documentation
docs: kdoc_re: don't recompile NextMatch regex every time
docs: kdoc_re: Change NestedMath args replacement to \0
docs: kdoc_re: make NextedMatch use KernRe
tools: kdoc_re: add support on NestedMatch for argument replacement
tools: python: add helpers to run unit tests
unittests: add tests for NestedMatch class
tools/lib/python/unittest_helper.py
docs: kdoc_parser: better handle struct_group macros
docs: kdoc_re: fix a parse bug on struct page_pool_params
Randy Dunlap (2):
docs: kdoc_parser: ignore context analysis and lock attributes
kdoc_parser: handle struct member macro VIRTIO_DECLARE_FEATURES(name)
tools/lib/python/kdoc/kdoc_parser.py | 109 +++--
tools/lib/python/kdoc/kdoc_re.py | 154 +++++--
tools/lib/python/unittest_helper.py | 279 +++++++++++++
tools/unittests/nested_match.py | 589 +++++++++++++++++++++++++++
4 files changed, 1063 insertions(+), 68 deletions(-)
create mode 100755 tools/lib/python/unittest_helper.py
create mode 100755 tools/unittests/nested_match.py
--
2.52.0
