Re: [PATCH 02/22] docs-rst: convert usb docbooks to ReST

[Jonathan Corbet]

On Thu, 30 Mar 2017 11:20:14 +0200
Markus Heiser  wrote:

> @Jon: what do you think about a bulk conversion?

I'm a bit leery of it, to tell the truth.  We're trying to create a
better set of kernel docs, and I'm far from convinced that dumping a
bunch of unloved stuff there in a mechanical way will get us there.

Each of those docs needs to be looked at, and, first of all, we need to
decide whether it's worth keeping or not.  Nobody wants to delete docs,
but old and unmaintained stuff doesn't help our users, IMO.  For the
stuff we want to keep, we need to look at how it fits into the new
scheme, probably split it up, etc.

It's a lot slower, but we've been getting rid of 3-6 template files in
each of the last few cycles, so we are getting there.  I don't think we
need to just give up on the rest.

Thanks,

jon

Re: [PATCH 02/22] docs-rst: convert usb docbooks to ReST

[Mauro Carvalho Chehab]

Em Thu, 30 Mar 2017 13:17:16 +0200
Markus Heiser  escreveu:

> Am 30.03.2017 um 12:12 schrieb Mauro Carvalho Chehab 
> :
> >>> At this point I'd just go with what Mauro has. It's here now, as
> >>> patches. We've seen from the GPU documentation that polishing the
> >>> one-time initial conversion is, after a point, wasted effort. Having the
> >>> documentation in rst attracts more attention and contributions, and any
> >>> remaining issues will get ironed out in rst.
> >> 
> >> I totally agree with you (I have never said something different)
> >>   
> >>> This is also one reason I'm in favor of just bulk converting the rest of
> >>> the .tmpl files using Documentation/sphinx/tmplcvt, get rid of DocBook
> >>> and be done with it, and have the crowds focus on rst.
> >> 
> >> I also agree with that. The tmplcvt script is good enough for this task,
> >> the dbxml2rst tool is more elaborate.  
> > 
> > I like the idea of a bulk conversion. My personal preference here is to
> > use the tmplcvt for such task, at least for simple books like the ones
> > I converted from USB.
> > 
> > The advantage is that it places everything on a single rst file, with,
> > IMHO, works best for books that aren't too complex.
> > Of course, it doesn't hurt to compare the end result with dbxml2rst
> > and see if something could be improved.  
> 
> If it helps ... dbxml2rst also supports single file conversion  ... I updated:
> 
>   
> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated

Ok, I double-checked the results from dbxml2rst with pandoc (via
the script). Those are the differences after running the following commands:

$ wget 
https://raw.githubusercontent.com/return42/sphkerneldoc/master/Documentation/books_migrated/writing_usb_driver/index.rst
$ Documentation/sphinx/tmplcvt 
Documentation/DocBook/writing_usb_driver.tmpl writing_usb_driver.rst
$ diff -uprBw writing_usb_driver.rst index.rst 

1) Author data:

-:Author: Greg Kroah-Hartman
+:author:Kroah-Hartman Greg
+:address:   g...@kroah.com

dbxml2rst inverted the author's name.  It also added author's e-mail.

IMHO, it is better to not have email address there, as it could be
outdated, but this is just my personal preference.

2) dbxml2rst added a copyright information:

+**Copyright** 2001-2002 : Greg Kroah-Hartman

This is a good thing.

3) dbxml2rst added a GPL information.

IMHO, we should add just one GPL information, per hole book
(and not per converted file).

4) dbxml2rst created some references that won't be unique:

+.. _intro:

That's a bad thing, as I bet most converted documents will have "intro"
sections.

5) dbxml2rst use ".. code-block:: c" instead of "::"

I prefer using "::"

6) dbxml2rst appends a commentary at the end:

+.. 
--
+.. This file was automatically converted from DocBook-XML with the dbxml
+.. library (https://github.com/return42/dbxml2rst). The origin XML comes
+.. from the linux kernel:
+..
+..   http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git
+.. 
--

7) dbxml2rst did a worse job with URB conversions:

-USB Home Page: http://www.usb.org
+USB Home Page: `http://www.usb.org `__

So, in summary, at least for this document, the only thing good with
dbxml2rst was that it filled the copyright info.

Maybe for more complex documents, it would do a better job.

Yet, in order to standardize it everywhere, I guess the best would be to
produce copyright data like:

.. include:: 

:Copyright: |copy| 2001-2002 : Greg Kroah-Hartman

Regards,
Mauro

Re: [PATCH 02/22] docs-rst: convert usb docbooks to ReST

[Markus Heiser]

Am 30.03.2017 um 13:17 schrieb Markus Heiser :
> 
> If it helps ... dbxml2rst also supports single file conversion  ... I updated:
> 
>  
> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated
> 
> There you find a folder for each DocBook conversion with only one rst file 
> (index.rst)
> in .. If you like, use it for comparison.

Forget to mentioning one of the main benefits: 

The conversion with dbxml2rst produce tables with directive ".. flat-table::"
instead of building ASCII tables (like pandoc does).

-- Markus --

Re: [PATCH 02/22] docs-rst: convert usb docbooks to ReST

[Markus Heiser]

Am 30.03.2017 um 12:12 schrieb Mauro Carvalho Chehab :
>>> At this point I'd just go with what Mauro has. It's here now, as
>>> patches. We've seen from the GPU documentation that polishing the
>>> one-time initial conversion is, after a point, wasted effort. Having the
>>> documentation in rst attracts more attention and contributions, and any
>>> remaining issues will get ironed out in rst.  
>> 
>> I totally agree with you (I have never said something different)
>> 
>>> This is also one reason I'm in favor of just bulk converting the rest of
>>> the .tmpl files using Documentation/sphinx/tmplcvt, get rid of DocBook
>>> and be done with it, and have the crowds focus on rst.  
>> 
>> I also agree with that. The tmplcvt script is good enough for this task,
>> the dbxml2rst tool is more elaborate.
> 
> I like the idea of a bulk conversion. My personal preference here is to
> use the tmplcvt for such task, at least for simple books like the ones
> I converted from USB.
> 
> The advantage is that it places everything on a single rst file, with,
> IMHO, works best for books that aren't too complex.
> Of course, it doesn't hurt to compare the end result with dbxml2rst
> and see if something could be improved.

If it helps ... dbxml2rst also supports single file conversion  ... I updated:

  
https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated

There you find a folder for each DocBook conversion with only one rst file 
(index.rst)
in .. If you like, use it for comparison.

-- Markus --

Re: [PATCH 02/22] docs-rst: convert usb docbooks to ReST

[Mauro Carvalho Chehab]

Em Thu, 30 Mar 2017 11:20:14 +0200
Markus Heiser  escreveu:

> Am 30.03.2017 um 10:21 schrieb Jani Nikula :
> 
> > On Thu, 30 Mar 2017, Markus Heiser  wrote:  
> >> Hi Mauro,
> >> 
> >> Am 29.03.2017 um 20:54 schrieb Mauro Carvalho Chehab 
> >> :
> >>   
> >>> As we're moving out of DocBook, let's convert the remaining
> >>> USB docbooks to ReST.
> >>> 
> >>> The transformation itself on this patch is a no-brainer
> >>> conversion using pandoc.  
> >> 
> >> right, its a no-brainer ;-) I'am not very happy with this
> >> conversions, some examples see below.
> >> 
> >> I recommend to use a more elaborate conversion as starting point,
> >> e.g. from my sphkerneldoc project:
> >> 
> >> * 
> >> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/gadget
> >> * 
> >> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/writing_musb_glue_layer
> >> * 
> >> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/writing_usb_driver
> >> 
> >> Since these DocBooks hasn't been changed in the last month, the linked reST
> >> should be up to date.  
> > 
> > Markus, I know you've done a lot of work on your conversions, and you
> > like to advocate them, but AFAICT you have never posted the conversions
> > as patches to the list. Your project isn't a clone of the kernel
> > tree. It's a pile of .rst files that nobody knows how to produce from
> > current upstream DocBook .tmpl files. I'm sorry, but this just doesn't
> > work that way.  
> 
> The conversion is done with the dbxml2rst tool:
> 
>   https://github.com/return42/dbxml2rst
> 
> But you are right, the links I send are decoupled from kernel. It is
> a 5 month old snapshot of a DocBook to reST conversion (now updated,
> with no affect to the linked files, since they have not been patched
> in the meantime) ...
> 
> > At this point I'd just go with what Mauro has. It's here now, as
> > patches. We've seen from the GPU documentation that polishing the
> > one-time initial conversion is, after a point, wasted effort. Having the
> > documentation in rst attracts more attention and contributions, and any
> > remaining issues will get ironed out in rst.  
> 
> I totally agree with you (I have never said something different)
> 
> > This is also one reason I'm in favor of just bulk converting the rest of
> > the .tmpl files using Documentation/sphinx/tmplcvt, get rid of DocBook
> > and be done with it, and have the crowds focus on rst.  
> 
> I also agree with that. The tmplcvt script is good enough for this task,
> the dbxml2rst tool is more elaborate.

I like the idea of a bulk conversion. My personal preference here is to
use the tmplcvt for such task, at least for simple books like the ones
I converted from USB.

The advantage is that it places everything on a single rst file, with,
IMHO, works best for books that aren't too complex.

Of course, it doesn't hurt to compare the end result with dbxml2rst
and see if something could be improved.

> 
> If Jonathan also likes to have a bulk conversion of the rest DocBooks,
> we can use tmplcvt or even dbxml2rst for this task. Everything under
> 
>   
> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated
> 
> is just a "make dbxm2rst", I can update every time and if a bulk conversion
> is the way ... I can send such patches or you send a tmplcvt conversion.
> 
> @Jon: what do you think about a bulk conversion?
> 
>  -- Markus --
>   


Thanks,
Mauro

Re: [PATCH 02/22] docs-rst: convert usb docbooks to ReST

[Markus Heiser]

Am 30.03.2017 um 10:21 schrieb Jani Nikula :

> On Thu, 30 Mar 2017, Markus Heiser  wrote:
>> Hi Mauro,
>> 
>> Am 29.03.2017 um 20:54 schrieb Mauro Carvalho Chehab 
>> :
>> 
>>> As we're moving out of DocBook, let's convert the remaining
>>> USB docbooks to ReST.
>>> 
>>> The transformation itself on this patch is a no-brainer
>>> conversion using pandoc.
>> 
>> right, its a no-brainer ;-) I'am not very happy with this
>> conversions, some examples see below.
>> 
>> I recommend to use a more elaborate conversion as starting point,
>> e.g. from my sphkerneldoc project:
>> 
>> * 
>> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/gadget
>> * 
>> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/writing_musb_glue_layer
>> * 
>> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/writing_usb_driver
>> 
>> Since these DocBooks hasn't been changed in the last month, the linked reST
>> should be up to date.
> 
> Markus, I know you've done a lot of work on your conversions, and you
> like to advocate them, but AFAICT you have never posted the conversions
> as patches to the list. Your project isn't a clone of the kernel
> tree. It's a pile of .rst files that nobody knows how to produce from
> current upstream DocBook .tmpl files. I'm sorry, but this just doesn't
> work that way.

The conversion is done with the dbxml2rst tool:

  https://github.com/return42/dbxml2rst

But you are right, the links I send are decoupled from kernel. It is
a 5 month old snapshot of a DocBook to reST conversion (now updated,
with no affect to the linked files, since they have not been patched
in the meantime) ...

> At this point I'd just go with what Mauro has. It's here now, as
> patches. We've seen from the GPU documentation that polishing the
> one-time initial conversion is, after a point, wasted effort. Having the
> documentation in rst attracts more attention and contributions, and any
> remaining issues will get ironed out in rst.

I totally agree with you (I have never said something different)

> This is also one reason I'm in favor of just bulk converting the rest of
> the .tmpl files using Documentation/sphinx/tmplcvt, get rid of DocBook
> and be done with it, and have the crowds focus on rst.

I also agree with that. The tmplcvt script is good enough for this task,
the dbxml2rst tool is more elaborate.

If Jonathan also likes to have a bulk conversion of the rest DocBooks,
we can use tmplcvt or even dbxml2rst for this task. Everything under

  
https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated

is just a "make dbxm2rst", I can update every time and if a bulk conversion
is the way ... I can send such patches or you send a tmplcvt conversion.

@Jon: what do you think about a bulk conversion?

 -- Markus --
  

Re: [PATCH 02/22] docs-rst: convert usb docbooks to ReST

[Jani Nikula]

On Thu, 30 Mar 2017, Markus Heiser  wrote:
> Hi Mauro,
>
> Am 29.03.2017 um 20:54 schrieb Mauro Carvalho Chehab 
> :
>
>> As we're moving out of DocBook, let's convert the remaining
>> USB docbooks to ReST.
>> 
>> The transformation itself on this patch is a no-brainer
>> conversion using pandoc.
>
> right, its a no-brainer ;-) I'am not very happy with this
> conversions, some examples see below.
>
> I recommend to use a more elaborate conversion as starting point,
> e.g. from my sphkerneldoc project:
>
> * 
> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/gadget
> * 
> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/writing_musb_glue_layer
> * 
> https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/writing_usb_driver
>
> Since these DocBooks hasn't been changed in the last month, the linked reST
> should be up to date.

Markus, I know you've done a lot of work on your conversions, and you
like to advocate them, but AFAICT you have never posted the conversions
as patches to the list. Your project isn't a clone of the kernel
tree. It's a pile of .rst files that nobody knows how to produce from
current upstream DocBook .tmpl files. I'm sorry, but this just doesn't
work that way.

At this point I'd just go with what Mauro has. It's here now, as
patches. We've seen from the GPU documentation that polishing the
one-time initial conversion is, after a point, wasted effort. Having the
documentation in rst attracts more attention and contributions, and any
remaining issues will get ironed out in rst.

This is also one reason I'm in favor of just bulk converting the rest of
the .tmpl files using Documentation/sphinx/tmplcvt, get rid of DocBook
and be done with it, and have the crowds focus on rst.


BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center

Re: [PATCH 02/22] docs-rst: convert usb docbooks to ReST

[Markus Heiser]

Hi Mauro,

Am 29.03.2017 um 20:54 schrieb Mauro Carvalho Chehab :

> As we're moving out of DocBook, let's convert the remaining
> USB docbooks to ReST.
> 
> The transformation itself on this patch is a no-brainer
> conversion using pandoc.

right, its a no-brainer ;-) I'am not very happy with this
conversions, some examples see below.

I recommend to use a more elaborate conversion as starting point,
e.g. from my sphkerneldoc project:

* 
https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/gadget
* 
https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/writing_musb_glue_layer
* 
https://github.com/return42/sphkerneldoc/tree/master/Documentation/books_migrated/writing_usb_driver

Since these DocBooks hasn't been changed in the last month, the linked reST
should be up to date.


> +Kernel Mode Gadget API
> +==
> +
> +Gadget drivers declare themselves through a *struct
> +usb\_gadget\_driver*, which is responsible for most parts of enumeration
> +for a *struct usb\_gadget*. The response to a set\_configuration usually
> +involves enabling one or more of the *struct usb\_ep* objects exposed by
> +the gadget, and submitting one or more *struct usb\_request* buffers to

quoting of all underlines is not needed.

> +!Iinclude/linux/usb/composite.h !Edrivers/usb/gadget/composite.c
> +Composite Device Functions
> +--
> +
> +At this writing, a few of the current gadget drivers have been converted
> +to this framework. Near-term plans include converting all of them,
> +except for "gadgetfs".
> +
> +!Edrivers/usb/gadget/function/f\_acm.c
> +!Edrivers/usb/gadget/function/f\_ecm.c
> +!Edrivers/usb/gadget/function/f\_subset.c
> +!Edrivers/usb/gadget/function/f\_obex.c
> +!Edrivers/usb/gadget/function/f\_serial.c
> +Peripheral Controller Drivers
> +=

I guess we miss some documentation here.

-- Markus --