Re: Project Idea: Perl 6 Syntax Explainer
This looks like it could 2 different things - 1 is a doxygen type markup where you provide extra stuff, the other could just extract the relevent pod or pod6 for a function/method/operator from standard pod. IME, people write the bare minimum documentation - standard perl5 pod is pretty much that, and it works nicely. With only minimal work you can extract function specific pod from a more pod, extracting examples of any method in the synopsis should be trivial. Autodia works without using either PPI or reimplementing the Perl compiler. As an aside I was thinking about updating Autodia to handle perl 6 but didn't have time to work out a budget for a grant (which almost certainly means I don't have time to do it at a below-market-rate). A. -- http://www.aarontrevena.co.uk LAMP System Integration, Development and Hosting
Re: Project Idea: Perl 6 Syntax Explainer
Larry Wall wrote:
On Tue, Jul 10, 2007 at 04:37:13PM -0700, Dave Whipp wrote:
: One approach would simply be to edit Perl-6.0.0-STD.pm and add some
: markup. To pick a token at random:
:
: =p6explain *
: An asterix in a version expression matches any version
: =end
: token whatever { '*' {*} }
Indeed, the example you picked indicates how important context
will be. There are many different uses of '*', and a p6explain that
simply lists all the possible meanings will be very nearly useless.
I hope it's not (nearly) useless.
My intention was the following: in most cases the meaning of an operator
is very obvious (for example * as the multiplication operator), there
are just some cases where the meaning is not obvious:
sub foo([EMAIL PROTECTED])
now instead of grep'ping through 16 Synopsis with 24895 lines (with 248
matches in Operator.pod and 189 in Syntax.pod), I'd like to have 2 or
three result pages that list all possible meanings of *. I don't want to
have a full Perl 6 program explained, just the operators/symbols that I
don't know about right now.
You really need to parse the surrounding text to have any idea what
the * means. And a big reason to make the grammar available (including
all the user-defined bits) is so that nobody ever has to reinvent PPI
for Perl 6. A p6 explainer that is sensitive to grammar mutations
really needs to do its work by annotating an entire compilation unit
and then making that information available to something resembling
an IDE. I realize that's a long term goal.
Which leads me to the question: how should the data be assembled so that
it could be used in such a way?
Would it be enough to store the name of the token, along with the
description?
Moritz
--
Moritz Lenz
http://moritz.faui2k3.org/ | http://perl-6.de/
signature.asc
Description: OpenPGP digital signature
Re: Project Idea: Perl 6 Syntax Explainer
Dave Whipp wrote:
Moritz Lenz wrote:
You're probably right about that, but somebody who writes his own macros
and operators probably doesn't need such a tool anyway. And should write
very good documentation, otherwise he has lost outright.
(I was thinking that your tool would provide a mechanism for them to
provide that very good documentation)
Let's hope it does that some day ;)
It was not my original goal, but it's quite tempting ;)
I think the way to approach it would be to define a
POD6 dialect what provides the necessary markup. The standard grammar
would contain this, as would things like prelude. But anyone would be
able to add it to any code they distribute so as to provide more
comprehensive debug info.
Of course that would be the best solution, but I don't see how I can
implement that soonish. Do you have good idea how to do it?
One approach would simply be to edit Perl-6.0.0-STD.pm and add some
markup. To pick a token at random:
=p6explain *
An asterix in a version expression matches any version
=end
token whatever { '*' {*} }
You convinced me, in-place documentation is probably the best.
This would have the advantage of better documenting the meaning of all
the tokens/rules in the grammar file, which isn't always immediately
obvious from reading it.
Damian's POD Parser can probably do much of the work of actually finding
the p6explain blocks. Establishing a formal link between these and the
token/rule might be more work; but probably isn't necessary, except for
linting purposes.
Which leads me to the question how to do the annotation.
First of all I hope there are no objections against fiddling with STD.pm.
My approach so far (see http://svn.pugscode.org/pugs/misc/perlhints/)
has been to write blocks for each token like so:
key: *
name: Regex Quantifier *
dsl: Regex
syn: token*
desc: matches token zero or more times
ex: 'ab'* # matches the empty string, ab, abab, ababab, ...
link: S05:somwhere
(key: the character(s) that the token stands for; name: a uniq name;
dsl: the domain-specific language the token occurs in (empty if plain
p6; syn: synopsis. desc: description (one to three lines usually); ex:
example; link: reference to the official specs).
It has the advantage of being easily parsable, less character/line noise
and no need for any escapes (expect newlines).
However it is not very PODish, so I'd like to ask for other opinions on
the used markup.
With the link: thing we could actually generate smartlinks that tell you
which part of the synopsis are parsed by which token in STD.pm (Agent
Zhang, are you paying attention? ;-). Do you think that would be desirable?
Moritz
--
Moritz Lenz
http://moritz.faui2k3.org/ | http://perl-6.de/
signature.asc
Description: OpenPGP digital signature
Re: Project Idea: Perl 6 Syntax Explainer
Moritz Lenz wrote:
Dave Whipp wrote:
One approach would simply be to edit Perl-6.0.0-STD.pm and add some
markup. To pick a token at random:
=p6explain *
An asterix in a version expression matches any version
=end
token whatever { '*' {*} }
You convinced me, in-place documentation is probably the best.
This would have the advantage of better documenting the meaning of all
the tokens/rules in the grammar file, which isn't always immediately
obvious from reading it.
Damian's POD Parser can probably do much of the work of actually finding
the p6explain blocks. Establishing a formal link between these and the
token/rule might be more work; but probably isn't necessary, except for
linting purposes.
Which leads me to the question how to do the annotation.
First of all I hope there are no objections against fiddling with STD.pm.
My approach so far (see http://svn.pugscode.org/pugs/misc/perlhints/)
has been to write blocks for each token like so:
key: *
name: Regex Quantifier *
dsl: Regex
syn: token*
desc: matches token zero or more times
ex: 'ab'* # matches the empty string, ab, abab, ababab, ...
link: S05:somwhere
A problem with this might be the lack of multiple language support -
especially for the description.
I'm not sure if you would want docs expanded to huge size to fit every
language in the sun and beyond, but can see real strengths in allowing
it to be inline in order to get it done.
What might be nice ( from my perspective - others may hate it ) is
assigning a URI to each unique instance, so that multiple language
support could be tagged on afterwards, and it could also help with the
link section ( or replace it by reversing the link to many to one rather
than one to one ) - if the synopsis were adjusted to reference these
URIs then the linking info would flow back in the other direction?
The key for your index would then become the URI, and the database could
then be hosted elsewhere ( of course, developers may wish to make there
code contain something similar to the above to help with creating your
data ? )
Steve
Re: Project Idea: Perl 6 Syntax Explainer: Status
A small status update: The name of the project currently is 'perlhints', as suggested by pmurias++ and [particle]++. Some files can be found in the pugs repository in misc/perlhints/ (repository URL: http://svn.pugscode.org/pugs/misc/perlhints/) There is a command line interface called perlhints5.pl (implemented in p5), and a very basic web interface at http://moritz.faui2k3.org/perlhints/, for example http://moritz.faui2k3.org/perlhints/?q=*. What it lacks mostly is the data ;) Currently all data is read from a single flat file. Moritz -- Moritz Lenz http://moritz.faui2k3.org/ | http://perl-6.de/ signature.asc Description: OpenPGP digital signature
Re: Project Idea: Perl 6 Syntax Explainer
On Tue, Jul 10, 2007 at 12:43:05PM +0200, Moritz Lenz wrote: considering the vast number of Operators and the like, I had the idea to implement a tool where you can enter a small piece of p6 syntax, and it explains what that might mean. (like a perldoc -f for operators/syntax elements instead of functions) I proposed a perldoc -f-a-like for operators a coupla years ago on p5p. People thought it was a good idea. I even started work on it, but stopped before I got very far cos other stuff intervened. I should get back on it. -- David Cantrell | Reality Engineer, Ministry of Information Nuke a disabled unborn gay baby whale for JESUS!
Project Idea: Perl 6 Syntax Explainer
Hi, considering the vast number of Operators and the like, I had the idea to implement a tool where you can enter a small piece of p6 syntax, and it explains what that might mean. (like a perldoc -f for operators/syntax elements instead of functions) A short example: $ p6explain '[]' [...] can be * '[$expression]': access to one or more array elements if used as postcircumfix Example: @a[2]; # access the third element of an array @a * '[$operator]': the 'reduce' hyper operator Example: [+] (1, 2, 3); # gives 1 + 2 + 3 == 6 In a regex/rule: * '[$subregex]': groups the content of the brackets without capturing the match result of $subregex $ p6explain '*' * can be * '$exp1 * $epx2' the multiplication operator, forces numerical context onto its arguments. Example: say 2 * 3; * 'multi sub *method_name' exports the sub method_name into the global namespace In a regex/rule: * '$subregex*' is a quantifier meaning that the previous subregex may be repeated an arbitrary number of times, including zero times. Example: m/ [ab]* / matches '', 'ab', 'abab', 'ababab' # END OF EXAMPLE I have a few requirements in mind (not necessarily sorted by importance): * Free software: Artistic License 2, some BSDish or GPLish - I don't mind, as long as it's free ;) * Multiple frontends: I'd like to have at least a command line and a web frontend * Support for multiple (natural) languages. * Perhaps support for other programming languages as well * Should contain links/references to the official docs Now I'd like to hear your opinion: * Anything else that should go into the requirements? * Is it useful at all? * Is it possible to implement it satisfactory without building a p6 compiler? * Do you have a good idea for a project name? Any other ideas and comments are welcome as well. Cheers, Moritz -- Moritz Lenz http://moritz.faui2k3.org/ | http://perl-6.de/ signature.asc Description: OpenPGP digital signature
Re: Project Idea: Perl 6 Syntax Explainer
Moritz Lenz skribis 2007-07-10 12:43 (+0200):
$ p6explain '[]'
[...] can be
* '[$expression]': access to one or more array elements if used as
postcircumfix
Example: @a[2]; # access the third element of an array @a
* '[$operator]': the 'reduce' hyper operator
Example: [+] (1, 2, 3); # gives 1 + 2 + 3 == 6
In a regex/rule:
* '[$subregex]': groups the content of the brackets without
capturing the match result of $subregex
I had the same idea, but in wiki form, with a clickable map of
characters for the curious.
The idea is basically: categorize by unicode symbols, and then describe
how they're used, in any context.
* Multiple frontends: I'd like to have at least a command line and a
web frontend
A command line tool could just do the equivalent of
w3m -dump wiki.perl6.example.com/symbol/*
* Perhaps support for other programming languages as well
Third system syndrome? :)
* Anything else that should go into the requirements?
Ultra easy contribution with revision control.
* Is it useful at all?
Absolutely!
* Is it possible to implement it satisfactory without building a p6
compiler?
Yes. Just match individual literal symbols. Someone looking for * should
encounter all uses of *, including ** and **{}.
* Do you have a good idea for a project name?
I had Decrypt in mind. (http://decrypt.perl6.nl/*)
--
korajn salutojn,
juerd waalboer: perl hacker [EMAIL PROTECTED] http://juerd.nl/sig
convolution: ict solutions and consultancy [EMAIL PROTECTED]
Re: Project Idea: Perl 6 Syntax Explainer
* Anything else that should go into the requirements? It would be even cooler if the command $ p6explain 'some_user_defined_sub' would display prettily-formatted pod for that sub and would work transparently, regardless of whether this sub is defined in the Perl standard library, CPAN module, or user's own code. And if there are several modules that have a sub with this name - print a list of modules (sorted by relevance to context or alphabetically) and let the user select what he wants. An overloaded operator should be the same thing as a sub for this matter. Rationale: Perl 5's perldoc displays the pod for every properly installed and documented module just as if they were perl* manpages and it is transparent to the user. That's one feature of Perl 5 that i absolutely admire. Why not enhance it to smaller units, like subs? And it doesn't break the different things should look different philosophy, as these look different enough to me: $ p6explain 'some_user_defined_sub' # sub $ p6explain Date::Hebrew # an imaginary module $ p6explain := # operator $ p6explain any # builtin $ p6explain perlrules # an imaginary P6 manpage that will replace perlre * Is it useful at all? Very much so, yes. That's what makes Redmond's development tools so attractive - you can put the cursor on any word, press F1 and get all the help. It doesn't work so well with operators, and with user-defined functions it just fails; but for user-defined functions it has the reference feature, which takes the user to the place where the functions is defined (if he's lucky enough to have the code). Actually i see no reason why this couldn't be combined with the help - for me the function's code and its documentation in a human language are usually equally important. Integrate that tool with Emacs, and you've got a self-documenting Perl 6 IDE ;) * Is it possible to implement it satisfactory without building a p6 compiler? Syntax highlighters work even with code that doesn't necessarily compile, so probably the answer is yes. But if this tool could peek into the compiler's syntax tree and see all the namespaces that the compiler knows as an easy-to-navigate data structure, then implementing my wish from above - generically getting help for every token wherever it is defined - would probably be easier. (Too bad i don't really know much about implementing compilers.) * Do you have a good idea for a project name? I really wish that such project just DIDN'T have a name, at least not at the end-user level. If it will have a name, there's a chance that it will join the army of Free Software programs with awful names such as YELP. It can just be a part of perldoc - that's one good name.
Re: Project Idea: Perl 6 Syntax Explainer
(Sorry for personal message previously, now my answer to the list)
Juerd Waalboer wrote:
Moritz Lenz skribis 2007-07-10 12:43 (+0200):
$ p6explain '[]'
[...] can be
* '[$expression]': access to one or more array elements if used as
postcircumfix
Example: @a[2]; # access the third element of an array @a
* '[$operator]': the 'reduce' hyper operator
Example: [+] (1, 2, 3); # gives 1 + 2 + 3 == 6
In a regex/rule:
* '[$subregex]': groups the content of the brackets without
capturing the match result of $subregex
I had the same idea, but in wiki form, with a clickable map of
characters for the curious.
I think the clickable map can easily be generated from the database, but
I'm not really convinced that a wiki is the right idea. (Most wikis tend
to look horrible, and that's a show stopper for me. Of course that can
be dealt with, but it makes me like wikis less ;)
The idea is basically: categorize by unicode symbols, and then describe
how they're used, in any context.
* Multiple frontends: I'd like to have at least a command line and a
web frontend
A command line tool could just do the equivalent of
w3m -dump wiki.perl6.example.com/symbol/*
right, basically ;)
* Perhaps support for other programming languages as well
Third system syndrome? :)
Let's implement the first system first, and make it easily extensible.
For example a different dictionary for ever combination of human and
computer language (perl6-en-base, perl6-en-regex, python-nl-base, ...)
* Anything else that should go into the requirements?
Ultra easy contribution with revision control.
Aye. First place to start with would be the pugs repository (which seems
to be the collection of most perl6 related projects anyway)
* Is it possible to implement it satisfactory without building a p6
compiler?
Yes. Just match individual literal symbols. Someone looking for * should
encounter all uses of *, including ** and **{}.
that's probably the way to go, yes.
* Do you have a good idea for a project name?
I had Decrypt in mind. (http://decrypt.perl6.nl/*)
I'd like the name if there weren't a few disadvantages:
* it is used in many other context and therefore not good googlabel
* it suggests that Perl 6 is cryptic (which it's not most of the time;)
* On my system there is already a `decrypt' executable (from the
`airsnort' package)
I thought about p6explain (or something similar like p6plain (pronounced
pi sixplain)), but both sound witless.
Moritz
--
Moritz Lenz
http://moritz.faui2k3.org/ | http://perl-6.de/
signature.asc
Description: OpenPGP digital signature
Re: Project Idea: Perl 6 Syntax Explainer
Moritz Lenz [EMAIL PROTECTED] writes: Hi, considering the vast number of Operators and the like, I had the idea to implement a tool where you can enter a small piece of p6 syntax, and it explains what that might mean. (like a perldoc -f for operators/syntax elements instead of functions) I really like the idea. Now I'd like to hear your opinion: * Is it useful at all? Yes. * Is it possible to implement it satisfactory without building a p6 compiler? It should ultimately be independent from building yet another compiler subproject. Else you don't find enough comrades to follow. I suggest not starting with an implementation but with the base description itself in any form. That might be a simple text file in pugs svn or a wiki page. That's step 1, IMHO. Do you think it's possible to extract and collect all the operators and descriptions from the synopses into one common place? If we had such a collection we would then format it in a second step into a structured collection and the third step is a frontend to that, the easiest part, IMHO. Step 1 is the real challenge, isn't it? Steffen -- Steffen Schwigon [EMAIL PROTECTED] Dresden Perl Mongers http://dresden-pm.org/ Deutscher Perl-Workshop http://www.perl-workshop.de/
Re: Project Idea: Perl 6 Syntax Explainer
Steffen Schwigon wrote: Moritz Lenz [EMAIL PROTECTED] writes: considering the vast number of Operators and the like, I had the idea to implement a tool where you can enter a small piece of p6 syntax, and it explains what that might mean. (like a perldoc -f for operators/syntax elements instead of functions) I really like the idea. Good ;) * Is it possible to implement it satisfactory without building a p6 compiler? It should ultimately be independent from building yet another compiler subproject. Else you don't find enough comrades to follow. Aye. I think Juerd's suggestion to make it character based is probably the best. I suggest not starting with an implementation but with the base description itself in any form. That might be a simple text file in pugs svn or a wiki page. That's step 1, IMHO. Do you think it's possible to extract and collect all the operators and descriptions from the synopses into one common place? Not automatically, sadly. If we had such a collection we would then format it in a second step into a structured collection and the third step is a frontend to that, the easiest part, IMHO. Since we'll have to skim though the synopsis and compile that file manually, it would be a good idea to actually write it in a form of semantic markup in the first place. Step 1 is the real challenge, isn't it? Yes. Cheers, Moritz -- Moritz Lenz http://moritz.faui2k3.org/ | http://perl-6.de/ signature.asc Description: OpenPGP digital signature
Re: Project Idea: Perl 6 Syntax Explainer
Steffen Schwigon wrote: Do you think it's possible to extract and collect all the operators and descriptions from the synopses into one common place? The problem with that is that it wouldn't scale to user-defined operators/macros. I think the way to approach it would be to define a POD6 dialect what provides the necessary markup. The standard grammar would contain this, as would things like prelude. But anyone would be able to add it to any code they distribute so as to provide more comprehensive debug info.
Re: Project Idea: Perl 6 Syntax Explainer
Dave Whipp wrote: Steffen Schwigon wrote: Do you think it's possible to extract and collect all the operators and descriptions from the synopses into one common place? The problem with that is that it wouldn't scale to user-defined operators/macros. You're probably right about that, but somebody who writes his own macros and operators probably doesn't need such a tool anyway. And should write very good documentation, otherwise he has lost outright. I think the way to approach it would be to define a POD6 dialect what provides the necessary markup. The standard grammar would contain this, as would things like prelude. But anyone would be able to add it to any code they distribute so as to provide more comprehensive debug info. Of course that would be the best solution, but I don't see how I can implement that soonish. Do you have good idea how to do it? Moritz -- Moritz Lenz http://moritz.faui2k3.org/ | http://perl-6.de/ signature.asc Description: OpenPGP digital signature
Re: Project Idea: Perl 6 Syntax Explainer
Moritz Lenz wrote:
You're probably right about that, but somebody who writes his own macros
and operators probably doesn't need such a tool anyway. And should write
very good documentation, otherwise he has lost outright.
(I was thinking that your tool would provide a mechanism for them to
provide that very good documentation)
I think the way to approach it would be to define a
POD6 dialect what provides the necessary markup. The standard grammar
would contain this, as would things like prelude. But anyone would be
able to add it to any code they distribute so as to provide more
comprehensive debug info.
Of course that would be the best solution, but I don't see how I can
implement that soonish. Do you have good idea how to do it?
One approach would simply be to edit Perl-6.0.0-STD.pm and add some
markup. To pick a token at random:
=p6explain *
An asterix in a version expression matches any version
=end
token whatever { '*' {*} }
This would have the advantage of better documenting the meaning of all
the tokens/rules in the grammar file, which isn't always immediately
obvious from reading it.
Damian's POD Parser can probably do much of the work of actually finding
the p6explain blocks. Establishing a formal link between these and the
token/rule might be more work; but probably isn't necessary, except for
linting purposes.
Dave.
