Re: sequence diagrams in rst documentation
On Sat, 2016-10-22 at 18:37 +0200, Markus Heiser wrote: > Yeah, I thought something similar. But is the import of the extension > a sufficient criteria? > > About ".. math::"; I guess we have to check if math extension AND > pdflatex is installed. > > What do you suppose? TBH, I only considered this briefly, but decided that somebody who went to the effort of installing the sphinx extension would likely not mind to get a subsequent error when it tries to use something that's not installed. I was, in fact, quite surprised that I even could install (on Debian) the plantuml sphinx extension without plantuml, which seems odd. In fact, I think it would be *preferable* to only check the extension ; we should print a message from the dummy plugin to let them know what extensions they need, and if they then go to the effort of installing it they probably don't just not mind getting errors on dependencies, but actually would *prefer* that, since otherwise it can be daunting to try to figure out what *else* you actually have to install. johannes -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
Am 21.10.2016 um 23:19 schrieb Johannes Berg: > On Fri, 2016-10-21 at 18:11 +0200, Markus Heiser wrote: > >> Yes and No. It depends on the tools (toolchains) we want to use. >> As far as I can see from a abstract POV it should by simple for >> math:: / since we already use/need LaTeX for PDF, for other tools >> I have to look. > > Not sure if we were talking past each other - but the pass-through is > actually really simple - example patch below. Yeah, I thought something similar. But is the import of the extension a sufficient criteria? About ".. math::"; I guess we have to check if math extension AND pdflatex is installed. What do you suppose? (ATM my comments on this are superficial, sorry that I haven't not yet fond the time, to look closer into all these generators). --Markus-- > > This makes it output an SVG (with fallback to PNG even) into the HTML, > a PDF into the PDF, and plain pre-formatted text for both when the > plantuml sphinx plugin isn't installed. > > johannes > > > diff --git a/Documentation/conf.py b/Documentation/conf.py > index bf6f310e5170..2c00ab6f0baf 100644 > --- a/Documentation/conf.py > +++ b/Documentation/conf.py > @@ -34,7 +34,8 @@ from load_config import loadConfig > # Add any Sphinx extension module names here, as strings. They can be > # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom > # ones. > -extensions = ['kernel-doc', 'rstFlatTable', 'kernel_include', 'cdomain'] > +extensions = ['kernel-doc', 'rstFlatTable', 'kernel_include', 'cdomain', > + 'plantuml'] > > # The name of the math extension changed on Sphinx 1.4 > if minor > 3: > @@ -494,6 +495,9 @@ pdf_documents = [ > kerneldoc_bin = '../scripts/kernel-doc' > kerneldoc_srctree = '..' > > +plantuml_output_format = "svg" > +plantuml_latex_output_format = "pdf" > + > # > -- > # Since loadConfig overwrites settings from the global namespace, it has to be > # the last statement in the conf.py file > diff --git a/Documentation/driver-api/index.rst > b/Documentation/driver-api/index.rst > index 8e259c5d0322..e0d4f24b6039 100644 > --- a/Documentation/driver-api/index.rst > +++ b/Documentation/driver-api/index.rst > @@ -7,6 +7,12 @@ of device drivers. This document is an only somewhat > organized collection > of some of those interfaces — it will hopefully get better over time! The > available subsections can be seen below. > > +.. uml:: > + > + Alice -> Bob: Hi! > + Alice <- Bob: How are you? > + > + > .. class:: toc-title > > Table of contents > diff --git a/Documentation/sphinx/dummy.py b/Documentation/sphinx/dummy.py > new file mode 100644 > index ..cfac42886ebd > --- /dev/null > +++ b/Documentation/sphinx/dummy.py > @@ -0,0 +1,24 @@ > +from sphinx.util.compat import Directive > +from docutils import nodes > +from docutils.parsers.rst import directives > + > +class IgnoreOptions(object): > +def __getitem__(self, key): > +return directives.unchanged > + > +def create_setup(directives): > +def setup(app): > +for d in directives: > +class TextDirective(Directive): > +has_content = True > +option_spec = IgnoreOptions() > + > +def run(self): > +text = '\n'.join(self.content) > +return [nodes.literal_block('', text, > + classes=['code', > + 'missing-plugin', > + 'missing-plugin-%s' % d])] > + > +app.add_directive(d, TextDirective) > +return setup > diff --git a/Documentation/sphinx/plantuml.py > b/Documentation/sphinx/plantuml.py > new file mode 100644 > index ..0007bc7f24fd > --- /dev/null > +++ b/Documentation/sphinx/plantuml.py > @@ -0,0 +1,7 @@ > +# -*- coding: utf-8 -*- > +from dummy import create_setup > + > +try: > +from sphinxcontrib.plantuml import * > +except: > +setup = create_setup(['uml']) -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
Am 21.10.2016 um 15:04 schrieb Johannes Berg: >> I had the same conclusion for math:: directives pulling in latex >> dependency [1]. Hopefully Markus can help here. > > Yeah, good one. > > Maybe it's actually simple? Depending on where sphinx will look for > plugins first, we could just ship the plugins with a no-op > implementation (pass through the text as pre-formatted text), and if it > finds the plugin first in a system-wide path that version would win for > the better rendering? Yes and No. It depends on the tools (toolchains) we want to use. As far as I can see from a abstract POV it should by simple for math:: / since we already use/need LaTeX for PDF, for other tools I have to look. I general I think, we should not include Java into our toolchains even if it is optional. Thats, why I won't vote for http://plantuml.com/ aafigure: has dependencies [1] to reportlab (which is IMO inapposite) and PIL, which is outdated / and aafigure itself seems outdated [2]. seqdiag: requires blockdiag and this requires Pillow ... until here its seems not bad, but there is also some "optional" dependencies to ImageMagick and reportlab (both smells). IMO a tool which generates SVG would be the best, I have to check if I find some or if some of the above could used in this way. For this I need time ;-) ... ATM I work in a custom project but I hope I will find next week time to dig more deeper. -- Markus -- [1] https://pythonhosted.org/sphinxcontrib-aafig/#requirements [2] http://bazaar.launchpad.net/~aafigure-team/aafigure/trunk/files -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
> I had the same conclusion for math:: directives pulling in latex > dependency [1]. Hopefully Markus can help here. Yeah, good one. Maybe it's actually simple? Depending on where sphinx will look for plugins first, we could just ship the plugins with a no-op implementation (pass through the text as pre-formatted text), and if it finds the plugin first in a system-wide path that version would win for the better rendering? johannes -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
On Fri, 21 Oct 2016, Johannes Bergwrote: >> >https://pythonhosted.org/sphinxcontrib-aafig/ >> > >> > I've not actually played with it at all, but I like the idea that >> > we'd have readable diagrams in the source docs as well... >> >> Well, maybe. I agree having it readable in the source docs as well is >> nice, but for sequence diagrams in particular, I don't think >> >> +---+ +---+ >> | Hello +>+ aafigure! | >> +---+ +---+ >> >> really beats >> >> Hello -> aafigure! > > > I found another one: > > https://pypi.python.org/pypi/sphinxcontrib-plantuml > > That one has really nice output and features, but ends up being a > *java* (of all the things) tool that the thing calls out to ... > > > Perhaps we can have a compromise and embed the raw text when the > tooling isn't all installed, so you can still build useful > documentation, but to get all the "prettiness" you might have to > install more dependencies? I had the same conclusion for math:: directives pulling in latex dependency [1]. Hopefully Markus can help here. BR, Jani. [1] http://lkml.kernel.org/r/877f93qdd2@intel.com -- Jani Nikula, Intel Open Source Technology Center -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
> > https://pythonhosted.org/sphinxcontrib-aafig/ > > > > I've not actually played with it at all, but I like the idea that > > we'd have readable diagrams in the source docs as well... > > Well, maybe. I agree having it readable in the source docs as well is > nice, but for sequence diagrams in particular, I don't think > > +---+ +---+ > | Hello +>+ aafigure! | > +---+ +---+ > > really beats > > Hello -> aafigure! I found another one: https://pypi.python.org/pypi/sphinxcontrib-plantuml That one has really nice output and features, but ends up being a *java* (of all the things) tool that the thing calls out to ... Perhaps we can have a compromise and embed the raw text when the tooling isn't all installed, so you can still build useful documentation, but to get all the "prettiness" you might have to install more dependencies? johannes -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
On Tue, 2016-10-18 at 17:52 -0600, Jonathan Corbet wrote: > In summary, I think we can consider taking on a module if it's what > we need to do the docs right. And if somebody agrees to maintain it! > :) :) I think for the ones that we carry, they're probably specific? > I've heard others say they would like better diagramming support. Do > you think that, maybe, something like aafigure would do the trick? > > https://pythonhosted.org/sphinxcontrib-aafig/ > > I've not actually played with it at all, but I like the idea that > we'd have readable diagrams in the source docs as well... Well, maybe. I agree having it readable in the source docs as well is nice, but for sequence diagrams in particular, I don't think +---+ +---+ | Hello +>+ aafigure! | +---+ +---+ really beats Hello -> aafigure! by much. You could do some alignment even with the latter version, and the above isn't even really a sequence diagram anyway :) Some of the sequence diagrams may also be automatically generated from runtime behaviour observation (e.g. tracing, we've actually done that), and outputting the types of ascii boxes is much more difficult there. For other types of diagrams this may be nice though. Anyway, I guess we'll cross that bridge when we get to it. I'd like to have these types of documentation, perhaps with a script to auto- generate from tracing - such as script and visual display can even serve as a debugging aid :) johannes -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
On Wed, 19 Oct 2016, Markus Heiserwrote: > Am 18.10.2016 um 16:52 schrieb Jani Nikula : > >>> *Only* adding the PNG would be awful, I'd have to keep track of the >>> corresponding source elsewhere, and perhaps couldn't even GPL it >>> because then I couldn't distribute the PNG without corresponding >>> source... >>> >>> Adding the source text would really be the only practical choice, but >>> doing so makes it easy to mismatch things, and also very easy to use >>> proprietary services for it that may go away at any time, etc. >> >> Agreed. And there are other problems with attaching binaries (although >> I'd say we should fix them too) [1]. >> >> [1] http://lkml.kernel.org/r/02a78907-933d-3f61-572e-28154b16b...@redhat.com > > Hmm, I was not briefed that binaries are problematic. I have seen > GIFs e.g. [2] and PDFs with a long history (when I worked with the media > documentation), so I thought binaries are OK. > > Can you give me some more hints to understand in what ways they are > problematic? / sorry if my question seems dump / You can download incremental patches from https://www.kernel.org/ for kernel updates. Seems so 90s, but people apparently still do this. I don't think the traditional diff/patch tools play ball with binaries. The least that could be done is to generate the patches using git diff --binary to include the git binary diff format. I don't see how that would be worse than having just "Binary files foo and bar differ" in the diff. Personally I don't really mind including binaries if they are the *source* format. If they're generated from something else, that something else should be tracked in git instead. And Someone(tm) should fix the tooling to handle binaries... BR, Jani. > > [2] > http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/Documentation/DocBook/v4l/fieldseq_tb.gif?h=v3.0 > > -- Markus -- -- Jani Nikula, Intel Open Source Technology Center -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
Am 18.10.2016 um 16:52 schrieb Jani Nikula: >> *Only* adding the PNG would be awful, I'd have to keep track of the >> corresponding source elsewhere, and perhaps couldn't even GPL it >> because then I couldn't distribute the PNG without corresponding >> source... >> >> Adding the source text would really be the only practical choice, but >> doing so makes it easy to mismatch things, and also very easy to use >> proprietary services for it that may go away at any time, etc. > > Agreed. And there are other problems with attaching binaries (although > I'd say we should fix them too) [1]. > > [1] http://lkml.kernel.org/r/02a78907-933d-3f61-572e-28154b16b...@redhat.com Hmm, I was not briefed that binaries are problematic. I have seen GIFs e.g. [2] and PDFs with a long history (when I worked with the media documentation), so I thought binaries are OK. Can you give me some more hints to understand in what ways they are problematic? / sorry if my question seems dump / [2] http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/Documentation/DocBook/v4l/fieldseq_tb.gif?h=v3.0 -- Markus To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
On Tue, 18 Oct 2016 13:43:41 +0200 Johannes Bergwrote: > > Example here: > > https://johannes.sipsolutions.net/files/80211/mac80211.html#connection-flow > > > > Coming back to this - sadly, it appears that this software (blockdiag, > seqdiag) is completely unmaintained, with open pull requests dating > back to 2012 and the last commit dating back to 2015-08-22. > > I'd want/need feature improvements in it too, but if I can't feed those > back to upstream (since it appears dead), there's little point. > > Perhaps we can ship plugins for this as part of the kernel sources? > Shouldn't be too difficult to reimplement something like this, after > all. OK, I've read through all of this. My thoughts, for whatever it's worth. We already carry a few sphinx plugins in the kernel; there is room for more if we *really* need them. But... - Part of the idea behind switching over to sphinx was to be able to get away from maintaining our own formatting system. Adding plugins to the kernel is a step away from that goal. So I'd like to be sure that there's nothing that's part of standard sphinx that will do the job first. That said, I think that requiring people to install plugins from contrib sites or third-party repos may be even worse. We don't want to put people through misery just to format the docs. In summary, I think we can consider taking on a module if it's what we need to do the docs right. And if somebody agrees to maintain it! :) I've heard others say they would like better diagramming support. Do you think that, maybe, something like aafigure would do the trick? https://pythonhosted.org/sphinxcontrib-aafig/ I've not actually played with it at all, but I like the idea that we'd have readable diagrams in the source docs as well... jon -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
> This could probably be argued either way... Yeah, I guess it could :) > My view has been all along that we should prefer to use existing > extensions written and maintained by others. Perhaps we (the kind of > royal "we" of which I'm personally really not part of) could take on > maintainership of some extensions in the interest of improving kernel > documentation, but I think the goal should be that the extensions are > maintained outside of the kernel tree, that the extensions are > generally usable, and have a chance of attracting attention and > contributions from outside of the kernel community. (Note that this > doesn't preclude us from shipping the extensions in the kernel tree, > as long as it's updated from the upstream, not forked.) Right. I tend to agree, though in the particular case I'm looking at we'd probably have to fork outside the kernel, forming a new upstream, and then ship that version (or perhaps rewrite it, forming a new upstream, and then ship that - doesn't matter all that much) > (This is one part of me being unhappy about making it easy to run > arbitrary scripts to produce documentation; those will never be > generic, and we'll never be able to offload their maintenance outside > of the kernel. We should not think that we have some really special > needs in the kernel.) I agree that we don't necessarily have any special needs (*), but in cases like this (**) it does seem more practical to just ship the plugin with the kernel. Whether or not a separate "upstream" is formed for it could be a secondary question, although it does seem better to do so. (*) although not wanting to ship binary files *is* kinda special :) (**) where the upstream is essentially dead (for all I can tell) and severely limited to the point where a rewrite will be a better choice. Anyway, I'll have to see if we (Intel Linux WiFi team) actually want to do things this way. Using the existing blockdiag/seqdiag is practical since it all exists already. OTOH, a simpler and better-looking solution would also be nice, so if we do go this way I'll investigate more what we can do around this. johannes -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
On Tue, 18 Oct 2016, Johannes Bergwrote: > On Tue, 2016-10-18 at 15:51 +0200, Markus Heiser wrote: >> Here are my thoughts ... >> >> Every extension which is not a part of the sphinx distro brings new >> external dependencies > > I agree. > >> and the development of such extensions is IMO >> far of kernel development's scope. > > Arguably, having good documentation *is* in the scope of kernel > development. > > Also, it could be argued that shipping a module in the kernel sources > would be more reliable than having to require it being externally > installed, like the GCC plugins perhaps. This could probably be argued either way... My view has been all along that we should prefer to use existing extensions written and maintained by others. Perhaps we (the kind of royal "we" of which I'm personally really not part of) could take on maintainership of some extensions in the interest of improving kernel documentation, but I think the goal should be that the extensions are maintained outside of the kernel tree, that the extensions are generally usable, and have a chance of attracting attention and contributions from outside of the kernel community. (Note that this doesn't preclude us from shipping the extensions in the kernel tree, as long as it's updated from the upstream, not forked.) (This is one part of me being unhappy about making it easy to run arbitrary scripts to produce documentation; those will never be generic, and we'll never be able to offload their maintenance outside of the kernel. We should not think that we have some really special needs in the kernel.) >> ATM, there are not many use cases for diagram generators, so why not >> be KISS and creating diagrams with the favorite tool only adding the >> resulting (e.g.) PNG to the Kernel's source? > > *Only* adding the PNG would be awful, I'd have to keep track of the > corresponding source elsewhere, and perhaps couldn't even GPL it > because then I couldn't distribute the PNG without corresponding > source... > > Adding the source text would really be the only practical choice, but > doing so makes it easy to mismatch things, and also very easy to use > proprietary services for it that may go away at any time, etc. Agreed. And there are other problems with attaching binaries (although I'd say we should fix them too) [1]. BR, Jani. [1] http://lkml.kernel.org/r/02a78907-933d-3f61-572e-28154b16b...@redhat.com -- Jani Nikula, Intel Open Source Technology Center -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
On Tue, 2016-10-18 at 15:51 +0200, Markus Heiser wrote: > Here are my thoughts ... > > Every extension which is not a part of the sphinx distro brings new > external dependencies I agree. > and the development of such extensions is IMO > far of kernel development's scope. Arguably, having good documentation *is* in the scope of kernel development. Also, it could be argued that shipping a module in the kernel sources would be more reliable than having to require it being externally installed, like the GCC plugins perhaps. > ATM, there are not many use cases for diagram generators, so why not > be KISS and creating diagrams with the favorite tool only adding the > resulting (e.g.) PNG to the Kernel's source? *Only* adding the PNG would be awful, I'd have to keep track of the corresponding source elsewhere, and perhaps couldn't even GPL it because then I couldn't distribute the PNG without corresponding source... Adding the source text would really be the only practical choice, but doing so makes it easy to mismatch things, and also very easy to use proprietary services for it that may go away at any time, etc. > Before we add new dependencies / complexity, we should get rid > of the DocBook build. That argument is ... completely bogus, politely said. I'm not going to (be able to) do anything about all the docbooks that exist, and have in fact converted the one docbook that I know anything about. Holding back the development of documentation in one subsystem because another subsystem hasn't converted is a garbage argument. johannes -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: sequence diagrams in rst documentation
Am 18.10.2016 um 13:43 schrieb Johannes Berg: > On Tue, 2016-10-11 at 15:53 +0200, Johannes Berg wrote: >>> >>> >>> Related question: I have some sequence diagrams, and just found the >>> seqdiag sphinx plugin. How should we manage adding extensions? Or >>> would you prefer not to add any at all? >> >> Example here: >> https://johannes.sipsolutions.net/files/80211/mac80211.html#connection-flow > > Coming back to this - sadly, it appears that this software (blockdiag, > seqdiag) is completely unmaintained, with open pull requests dating > back to 2012 and the last commit dating back to 2015-08-22. > > I'd want/need feature improvements in it too, but if I can't feed those > back to upstream (since it appears dead), there's little point. > > Perhaps we can ship plugins for this as part of the kernel sources? > Shouldn't be too difficult to reimplement something like this, after > all. Here are my thoughts ... Every extension which is not a part of the sphinx distro brings new external dependencies and the development of such extensions is IMO far of kernel development's scope. ATM, there are not many use cases for diagram generators, so why not be KISS and creating diagrams with the favorite tool only adding the resulting (e.g.) PNG to the Kernel's source? Before we add new dependencies / complexity, we should get rid of the DocBook build. -- Markus -- > johannes > -- > To unsubscribe from this list: send the line "unsubscribe linux-doc" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
sequence diagrams in rst documentation
On Tue, 2016-10-11 at 15:53 +0200, Johannes Berg wrote: > > > > > > Related question: I have some sequence diagrams, and just found the > > seqdiag sphinx plugin. How should we manage adding extensions? Or > > would you prefer not to add any at all? > > Example here: > https://johannes.sipsolutions.net/files/80211/mac80211.html#connection-flow Coming back to this - sadly, it appears that this software (blockdiag, seqdiag) is completely unmaintained, with open pull requests dating back to 2012 and the last commit dating back to 2015-08-22. I'd want/need feature improvements in it too, but if I can't feed those back to upstream (since it appears dead), there's little point. Perhaps we can ship plugins for this as part of the kernel sources? Shouldn't be too difficult to reimplement something like this, after all. johannes -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html