Re: More flexible POD

2009-08-11 Thread Mark Overmeer

Being on holidays, it is not easy to follow threads closely, so if
I repeat things other people have said already, I apologize beforehand.
My responses may b late as well.

Two years ago, I discussed various options, which compared POD to features
in to other languages and suggested various syntaxes:
http://www.nntp.perl.org/group/perl.perl6.language/2007/07/msg27894.html

What I did not really find back in the (swift) scan of the whole debate
gongin on at the moment, is the need to maintain *error free*, *complete*,
and *consistent* documentation with *minimal effort*.  So: the purpose of
documentation.  This must be clear before syntax is being discussed.

"Error free" is simpler in Perl5, because subs have simple parameter
   prototypes and we have no strict types.  In Perl6, when the user
   has more power in the parameter list, it is harder to document it
   correctly by hand.
   I have experiences converting two large existing modules from
   'POD at the end' into 'intermixing POD and code'.  I found many,
   many mistakes in the POD probably because when changing the code
   later, people do not see where the docs which need to be updated
   are located.
   POD at the end of the code to allow a logical flow of the docs has
   some advantanges.  IMO however, you mix two different needs of use
   for documentation: you wish to teach the user and you need to describe
   the API.
   So, why not seperate those two purposes more clearly?  My code
   always has a rather short DESCRIPTION chapter, then the "use often"
   detailed METHOD or FUNCTION description (interleaved with code)
   and then a DETAILS (maybe a bad name) which is the training bit:
   how everything interacts without going into the disturbing API
   nitbits.

"Complete" is covered only partially by the "POD::Coverage".  Of
   course, it is detectable whether all methods/subs/rules are
   described, but can you also check whether all parameters are
   described in full?

"Consistent" documentation is something Perl5 lacks: have a look at
   the way different (Core) modules describe their API. One of the
   poorest parts is the complete lack of inheritance support for
   documenting Perl.  If you only write distributions with a single
   functional package in them, then POD(5) suffices.  When you do
   more complex stuff (like the IO:: modules with inheritance etc)
   then people start manually maintain lists of inherited methods and
   named parameters in higher level man-pages.  This is very hard to
   maintain an stupid!!

"minimal effort" is crucial: when you want people to write documentation,
   then it must create the most useful/beautiful result with minimal
   effort.  The need to rewrite method call parameters is one the of
   useless and error-prone parts of the current POD design. Of course,
   there must be a way to overrule the prototype when needed.
   As example: my Mail::Box set of modules contain (with depedencies)
   about 140 packages with 1150 documented methods and 200 error messages.
   I use OODoc to improve PODs limitations which saves me the typing
   of 700,000 characters(!) which pure POD would require.

The discussion with Damian, two years ago, is whether POD is a
second class sitizen, where it is a burdon to have it and code the only
important thing, or a first class sitizen.  IMO (which is not generally
accepted in the Perl community, but in common in other languages) it is
the latter. IMO programming is solving a problem in a good way, which
means: it is only solved when "it works and is usable". 
   "it works"
  . it does what we need it to do in our application
  . it has regression tests to guarantee that the API works
   "is usable"
  . algorithmic documentation (where the code is unclear)
  . api documentation

Code is IMHO not more important that docs or tests; they are different
view on the same problem. And most of use spend much more lines of text
on docs and tests than on the code (I guess three times as many lines
on docs as on code) So:we need a very light syntax for POD!

There are a few sources which could be used for POD:
  1) information from the code
  2) information from tests
  3) interleaved documentation
  4) end-of-file documentation
  5) Separate file documentation
  6) author, license, SEE ALSO, distribution, version facts
We need to have a standard way of translating those into a "document
tree".  Then we can have different implementations to serialize that
into MAN, HTML etc.  That latter part does not need an obligatory spec.
I use a template system for it.  The current Perl6 specs do not follow
this model sufficiently.
   
If you agree that DOCs are first class language sitizens, then it must
be clear that the "short notation for often used syntax" rule applies.
   #{{ or #{  are visually screaming far too loud.  I would prefer a
simple  /^\s*\`\s+/, but can think of many alternatives.

The link mentioned before refers to a more details spec with examples
to b

Fwd: More flexible POD

2009-08-10 Thread Matthew Walton
Woops - forgot to reply all (I'm on an irritating mixture of lists
which set reply-to and don't, and I never remember which is which).
Sorry!


-- Forwarded message --
From: Matthew Walton 
Date: Tue, Aug 11, 2009 at 7:10 AM
Subject: Re: More flexible POD
To: Jon Lang 


I'm not sure what it should be, but I do believe that there should be
a solution which allows elegant mixing of code and Pod. I want to
document my APIs by attaching the documentation to the methods in
question, otherwise the documentation won't get updated when the code
does (and if the code at work is anything to go by, won't get updated
at all anyway, but you can at least make it as easy as possible for
the people who do remember).

Attaching blocks of documentation to bits of code is something that
Javadoc's syntax gets very right.

The trouble is, if we just allow arbitrary whitespace before the start
of a Pod block:

class A {
 =head2 foo()
 Frobnicates the widget.
 =end
 method foo() { ... }
}

(modulo accurate Pod directives - I'm a bit hazy on how it works now,
I keep thinking I should be able to say '=method', but maybe that's a
matter for the Pod extractor).

Do we run the risk of causing problems if somebody does this:

my ($several, $lengthily-named, $variables)
 = something-which-produces-a-list();

and the parser thinks 'hang on a sec, that's Pod'. This is
particularly bad for programmers who don't put spaces around their
binary operators (hah! We could enforce it!) and may cause
confuzzlement.

I don't think footnote-like references in the code would help
programmers keep the documentation up to date or help them in reading
it to comprehend the code when they come to maintain it, which I think
are the two key reasons to put your documentation right there in the
code. If you did do it though, you'd have to use named references
(probably valid Perl 6 identifiers), because numbers are just a
nightmare.

Matthew


More flexible POD

2009-08-10 Thread Jon Lang
Masak's Journal[1] identifies a couple of problems involving the
interaction of POD with block-form classes - one in the article
itself, and another in the comments that follow.  The first is that a
mixture of indented block-form classes and non-indented POD is
visually very ugly.  The second is that mixing POD and code is in
general problematic.  In the latter case, a recommendation is made
that good practice would put all of the POD together at the end of the
file, rather than interspersed throughout.

These two issues seem to operate at cross-purposes: on the one hand,
if all of the POD is consolidated at the end of the file, then you
don't have to worry about how ugly it is to mix POD and indented code.
 OTOH, do we really want to punish programmers for trying to mix the
two?

Assuming that we want to make life easier on programmers who like to
intersperse code and POD, one way to do this would be to replace the
"start of line only" rule for POD with an indenting rule not unlike
what is used with block quotes: if there's any leading whitespace on
the opening line of a POD section, then every line in that section
must start with equivalent whitespace.  This complicates the POD
extractor, but not by much.

In terms of placing all of the POD at the end: one of the strongest
reasons for mixing POD with code is that you can easily identify what
code to which a given section of POD refers. Putting all of the POD at
the end lacks that ability.  One possible solution would be to provide
some means for you to specify a "POD footnote" in the code, not unlike
the URL reference that I've got in this email: in the code itself, you
place an anchor (e.g., '#{{=[1]}}') which can then be referenced later
on by a POD section (e.g., '=[1] start ...').  This would provide an
easy way to switch back and forth between the POD and the related
code.  Mind you, I'm not at all enamored with the specific syntax
suggested above.  It's a quick-and-dirty hack intended strictly to
illustrate the concept.

-- 
Jonathan "Dataweaver" Lang

[1] http://use.perl.org/~masak/journal/39334