Re: [rsyslog] auto-generating module documentation

[Rainer Gerhards]

2016-11-30 10:27 GMT+01:00 Rainer Gerhards :
> Longer reply follows a bit later. I do not like this idea. It complicates
> things,

As mostolog says, we could use doxygen, but this tool is more for
developer doc than for end user-doc. Writing RST is simple, adding
comments right into the code is usually clumsy, because you need to do
it in a comment block. Doing the doc in an actual string is even more
cumbersome, because you need to escape e.g. the quote characters, need
to have \n for explicit linesfeeds etc. It is so cumbersome that I
would deem it as "unacceptable" for most folks. Pushes the burden for
writing good doc very much up (the shorter the doc I write, the less
formatting pain I have).

Also, the code can also include partial doc, much more needs to be
provided. So we need to be able to merge the sources. Hyperlinking is
also far more complex. Finally, it's a very uncommon process that we
need to teach everyone.

> is a lot of work (at least a couple of days),

We have an existing procedure which cleanly and automatically builds
the doc and places it on the web site.

We would need to find a good way and develop a new process which does
all the extracting and "merge into main doc set" work automatically
and reliably. This includes time for learning new tools and
troubleshooting.

>and boils down to that
> everyone but developers loses the ability to write module doc.

That's actually the main point for me, it again removes users from the
ability to improve (that part of) the doc.

Experience also tells that doc in code is not necessarily more correct
than doc in other files. It is more a matter of discipline. Often
enough, even code comments are not updated when something changes or
code is duplicated. I do not think this would work better with the far
more lengthy doc.

That's why I don't like the idea. Besides, I don't think we could
actually use it in the code at all. So extra effort without any gain -
from my PoV.

Rainer
>
> Rainer
>
> Sent from phone, thus brief.
>
>
> Am 30.11.2016 09:36 schrieb "David Lang" :
>>
>> On Wed, 30 Nov 2016, mosto...@gmail.com wrote:
>>
>>> Probably a stupid idea...will it make sense that information being
>>> populated from rsysloc-doc? (or viceversa, like javadoc)
>>
>>
>> Since this is the code that creates parameter and variable names in the
>> modules, I don't see how the -doc project could push it into the source
>>
>> I'm thinking that rsyslog-doc would create these files from the rsyslog
>> source (or that the rsyslog source would output the data to be picked up by
>> the doc process)
>>
>>> Having everything on one place looks great to me
>>
>>
>> the question is how.
>>
>> Part of this comes down to nuances in C that I don't know. If we add two
>> string pointers to the array, can we declare the array with constants and
>> have the compiler store the constants somewhere and create the appropriate
>> pointers to them
>>
>> Is there some standard tool that can generate docs from C source? I think
>> I've seen people talk about sphinx (or something similar), but I've never
>> followed things very closely, and I don't have any idea if it can deal with
>> things in an array.
>>
>>
>> And then there is the problem of backwards compatibility. Is there a way
>> to make this change a module at a time? or do we have to change all of
>> rsyslog (and break any out-of-tree modules) in a single step?
>>
>> David Lang
>>
>>
>>>
>>> El 28/11/16 a las 17:50, David Lang escribió:

 we have a few cases where the documentation doesn't match the module
 parameters (wrong action parameters shown). While we can go through and fix
 the ones that we find as we find them, I think we should look at finding a
 way to make this more automated

 Is there a reasonable way to expand the array that defines the v6
 parameters, which currently defined as:

 /* the following defines describe the parameter block for puling
  * config parameters. Note that the focus is on ease and saveness of
  * use, not performance. For example, we address parameters by name
  * instead of index, because the former is less error-prone. The
 (severe)
  * performance hit does not matter, as it is a one-time hit during
 config
  * load but never during actual processing. So there is really no reason
  * to care.
  */
 struct cnfparamdescr { /* first the param description */
 const char *name;/**< not a es_str_t to ease definition in code
 */
 ecslCmdHdrlType type;
 unsigned flags;
 };

 to contain the default value and a description.

 Then if we can add a module general description field, we should have
 the ability to create allmost all of the module documenation pages
 automatically (except for the legacy config stuff, and that could be either
 added as another variable in the module, or maintained outside of 

Re: [rsyslog] auto-generating module documentation

[mosto...@gmail.com]

> Is there some standard tool that can generate docs from C source?

doxyfile


Said that, after Rainer's reply, I have nothing to add :)


El 30/11/16 a las 10:27, Rainer Gerhards escribió:

Longer reply follows a bit later. I do not like this idea. It complicates
things, is a lot of work (at least a couple of days), and boils down to
that everyone but developers loses the ability to write module doc.

Rainer

Sent from phone, thus brief.

Am 30.11.2016 09:36 schrieb "David Lang" :


On Wed, 30 Nov 2016, mosto...@gmail.com wrote:

Probably a stupid idea...will it make sense that information being

populated from rsysloc-doc? (or viceversa, like javadoc)


Since this is the code that creates parameter and variable names in the
modules, I don't see how the -doc project could push it into the source

I'm thinking that rsyslog-doc would create these files from the rsyslog
source (or that the rsyslog source would output the data to be picked up by
the doc process)

Having everything on one place looks great to me
the question is how.

Part of this comes down to nuances in C that I don't know. If we add two
string pointers to the array, can we declare the array with constants and
have the compiler store the constants somewhere and create the appropriate
pointers to them

Is there some standard tool that can generate docs from C source? I think
I've seen people talk about sphinx (or something similar), but I've never
followed things very closely, and I don't have any idea if it can deal with
things in an array.


And then there is the problem of backwards compatibility. Is there a way
to make this change a module at a time? or do we have to change all of
rsyslog (and break any out-of-tree modules) in a single step?

David Lang




El 28/11/16 a las 17:50, David Lang escribió:


we have a few cases where the documentation doesn't match the module
parameters (wrong action parameters shown). While we can go through and fix
the ones that we find as we find them, I think we should look at finding a
way to make this more automated

Is there a reasonable way to expand the array that defines the v6
parameters, which currently defined as:

/* the following defines describe the parameter block for puling
  * config parameters. Note that the focus is on ease and saveness of
  * use, not performance. For example, we address parameters by name
  * instead of index, because the former is less error-prone. The (severe)
  * performance hit does not matter, as it is a one-time hit during config
  * load but never during actual processing. So there is really no reason
  * to care.
  */
struct cnfparamdescr { /* first the param description */
 const char *name;/**< not a es_str_t to ease definition in code
*/
 ecslCmdHdrlType type;
 unsigned flags;
};

to contain the default value and a description.

Then if we can add a module general description field, we should have
the ability to create allmost all of the module documenation pages
automatically (except for the legacy config stuff, and that could be either
added as another variable in the module, or maintained outside of the
module)

Thoughts?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC 

Re: [rsyslog] auto-generating module documentation

[Rainer Gerhards]

Longer reply follows a bit later. I do not like this idea. It complicates
things, is a lot of work (at least a couple of days), and boils down to
that everyone but developers loses the ability to write module doc.

Rainer

Sent from phone, thus brief.

Am 30.11.2016 09:36 schrieb "David Lang" :

> On Wed, 30 Nov 2016, mosto...@gmail.com wrote:
>
> Probably a stupid idea...will it make sense that information being
>> populated from rsysloc-doc? (or viceversa, like javadoc)
>>
>
> Since this is the code that creates parameter and variable names in the
> modules, I don't see how the -doc project could push it into the source
>
> I'm thinking that rsyslog-doc would create these files from the rsyslog
> source (or that the rsyslog source would output the data to be picked up by
> the doc process)
>
> Having everything on one place looks great to me
>>
>
> the question is how.
>
> Part of this comes down to nuances in C that I don't know. If we add two
> string pointers to the array, can we declare the array with constants and
> have the compiler store the constants somewhere and create the appropriate
> pointers to them
>
> Is there some standard tool that can generate docs from C source? I think
> I've seen people talk about sphinx (or something similar), but I've never
> followed things very closely, and I don't have any idea if it can deal with
> things in an array.
>
>
> And then there is the problem of backwards compatibility. Is there a way
> to make this change a module at a time? or do we have to change all of
> rsyslog (and break any out-of-tree modules) in a single step?
>
> David Lang
>
>
>
>> El 28/11/16 a las 17:50, David Lang escribió:
>>
>>> we have a few cases where the documentation doesn't match the module
>>> parameters (wrong action parameters shown). While we can go through and fix
>>> the ones that we find as we find them, I think we should look at finding a
>>> way to make this more automated
>>>
>>> Is there a reasonable way to expand the array that defines the v6
>>> parameters, which currently defined as:
>>>
>>> /* the following defines describe the parameter block for puling
>>>  * config parameters. Note that the focus is on ease and saveness of
>>>  * use, not performance. For example, we address parameters by name
>>>  * instead of index, because the former is less error-prone. The (severe)
>>>  * performance hit does not matter, as it is a one-time hit during config
>>>  * load but never during actual processing. So there is really no reason
>>>  * to care.
>>>  */
>>> struct cnfparamdescr { /* first the param description */
>>> const char *name;/**< not a es_str_t to ease definition in code
>>> */
>>> ecslCmdHdrlType type;
>>> unsigned flags;
>>> };
>>>
>>> to contain the default value and a description.
>>>
>>> Then if we can add a module general description field, we should have
>>> the ability to create allmost all of the module documenation pages
>>> automatically (except for the legacy config stuff, and that could be either
>>> added as another variable in the module, or maintained outside of the
>>> module)
>>>
>>> Thoughts?
>>>
>>> David Lang
>>> ___
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>>> DON'T LIKE THAT.
>>>
>>
>> ___
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
>> DON'T LIKE THAT.
>>
>
> ___
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] auto-generating module documentation

[David Lang]

On Wed, 30 Nov 2016, mosto...@gmail.com wrote:

Probably a stupid idea...will it make sense that information being populated 
from rsysloc-doc? (or viceversa, like javadoc)


Since this is the code that creates parameter and variable names in the modules, 
I don't see how the -doc project could push it into the source


I'm thinking that rsyslog-doc would create these files from the rsyslog source 
(or that the rsyslog source would output the data to be picked up by the doc 
process)



Having everything on one place looks great to me


the question is how.

Part of this comes down to nuances in C that I don't know. If we add two string 
pointers to the array, can we declare the array with constants and have the 
compiler store the constants somewhere and create the appropriate pointers to 
them


Is there some standard tool that can generate docs from C source? I think I've 
seen people talk about sphinx (or something similar), but I've never followed 
things very closely, and I don't have any idea if it can deal with things in an 
array.



And then there is the problem of backwards compatibility. Is there a way to make 
this change a module at a time? or do we have to change all of rsyslog (and 
break any out-of-tree modules) in a single step?


David Lang




El 28/11/16 a las 17:50, David Lang escribió:
we have a few cases where the documentation doesn't match the module 
parameters (wrong action parameters shown). While we can go through and fix 
the ones that we find as we find them, I think we should look at finding a 
way to make this more automated


Is there a reasonable way to expand the array that defines the v6 
parameters, which currently defined as:


/* the following defines describe the parameter block for puling
 * config parameters. Note that the focus is on ease and saveness of
 * use, not performance. For example, we address parameters by name
 * instead of index, because the former is less error-prone. The (severe)
 * performance hit does not matter, as it is a one-time hit during config
 * load but never during actual processing. So there is really no reason
 * to care.
 */
struct cnfparamdescr { /* first the param description */
const char *name;/**< not a es_str_t to ease definition in code */
ecslCmdHdrlType type;
unsigned flags;
};

to contain the default value and a description.

Then if we can add a module general description field, we should have the 
ability to create allmost all of the module documenation pages 
automatically (except for the legacy config stuff, and that could be either 
added as another variable in the module, or maintained outside of the 
module)


Thoughts?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
LIKE THAT.
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

Re: [rsyslog] auto-generating module documentation

[mosto...@gmail.com]

Probably a stupid idea...will it make sense that information being 
populated from rsysloc-doc? (or viceversa, like javadoc)


Having everything on one place looks great to me


El 28/11/16 a las 17:50, David Lang escribió:
we have a few cases where the documentation doesn't match the module 
parameters (wrong action parameters shown). While we can go through 
and fix the ones that we find as we find them, I think we should look 
at finding a way to make this more automated


Is there a reasonable way to expand the array that defines the v6 
parameters, which currently defined as:


/* the following defines describe the parameter block for puling
 * config parameters. Note that the focus is on ease and saveness of
 * use, not performance. For example, we address parameters by name
 * instead of index, because the former is less error-prone. The (severe)
 * performance hit does not matter, as it is a one-time hit during config
 * load but never during actual processing. So there is really no reason
 * to care.
 */
struct cnfparamdescr { /* first the param description */
const char *name;/**< not a es_str_t to ease definition in 
code */

ecslCmdHdrlType type;
unsigned flags;
};

to contain the default value and a description.

Then if we can add a module general description field, we should have 
the ability to create allmost all of the module documenation pages 
automatically (except for the legacy config stuff, and that could be 
either added as another variable in the module, or maintained outside 
of the module)


Thoughts?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a 
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST 
if you DON'T LIKE THAT.


___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.

[rsyslog] auto-generating module documentation

[David Lang]

we have a few cases where the documentation doesn't match the module parameters 
(wrong action parameters shown). While we can go through and fix the ones that 
we find as we find them, I think we should look at finding a way to make this 
more automated


Is there a reasonable way to expand the array that defines the v6 parameters, 
which currently defined as:


/* the following defines describe the parameter block for puling
 * config parameters. Note that the focus is on ease and saveness of
 * use, not performance. For example, we address parameters by name
 * instead of index, because the former is less error-prone. The (severe)
 * performance hit does not matter, as it is a one-time hit during config
 * load but never during actual processing. So there is really no reason
 * to care.
 */
struct cnfparamdescr { /* first the param description */
const char *name;/**< not a es_str_t to ease definition in code */
ecslCmdHdrlType type;
unsigned flags;
};

to contain the default value and a description.

Then if we can add a module general description field, we should have the 
ability to create allmost all of the module documenation pages automatically 
(except for the legacy config stuff, and that could be either added as another 
variable in the module, or maintained outside of the module)


Thoughts?

David Lang
___
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE 
THAT.