On Tue, 2013-04-23 at 04:23 +0200, Henrik Lindberg wrote: > This is quite difficult. The puppet lexer (with quite a lot of changes > in the 3.x code stream, and with even more changes in the 3.2 --parser > future lexer/parser which will the standard in puppet 4) is designed > primarily for runtime lexing/parsing. Specifically it: > - stops on several types of lexing errors > - does not preserve whitepsace / linebreaks / formatting
That's the reason puppet doc doesn't work with every manifest. > This means that it is quite difficult to reuse it as a library function > for transformation and formatting. Before the new lexer/parser, the > lexer was also intimately tied to a specific puppet version. This means > that you need a specific version of the lexer to lex the puppet > manifests for that particular version of the language. Also not very > good for a more general purpose lexer / transformer. > > When doing cleaning/formatting there are issues that can not really be > resolved at the lexical level - you really do need a parser in order to > apply reasoning based on what the tokens mean (semantics) or may mean > (if ambiguous / in error / not supported in a particular version). > > The same issues (as for the lexer) applies to the puppet parser (this > will get easier with the new parser --parser future in 3.2) but it is > still focusing on parsing with the intent of evaluation. There is no > reverse serialization (from AST to source text) or pretty printing, or > "cleaner". Actually there is a bit in some of the AST leaves for reconstructing puppet doc output. > Although starting with --parser future this would be possible > (still lots and lots of work) to implement. > > As the lexer/parser in puppet evolves it will not maintain backwards > compatibility at the token / parser level - these are considered > internal APIs. (i.e. Different and increasingly more complicated monkey > patches will be required as support for features like heredoc and puppet > templates are added - these are mostly about lexing). > > There are other initiatives; puppet-lint, and Geppetto (IDE) that also > deal with lexing/parsing/validation/formatting/cleanup. They both have > their own puppet DSL lexers / parsers. Geppetto builds a model of the > source and keeps track of all the source text, cross references between > all the files as well, it does validation, pretty printing, provides > quick fixes / clean up in an extensible way. Some of the functionality > is also available from the command line, and some via jenkins, and > travis-ci plugins. It handles Puppet 2.7, 3.x, PE (and in the next > release 3.2 and the future parser as well (as soon as 3.2 is released by > puppetlabs)). Still with puppet doc in mind, would that make sense to base its next version on gepetto? > Lastly, the style guide and reality are not always the same thing :) and > people developing puppet manifests have their own set of styles and > rules they want to follow - one size most certainly does not fit all > here. Quite a lot of flexibility and options are required to > cleanup/format the manifests as well as providing options to fix/resolve > certain problems (hard or stylistic) in different ways that require user > interaction. > > Why not join forces with puppet-lint or Geppetto? As the principal > developer on Geppetto I know I could sure use help :) Do you have any pointers about what you'd want us to help? I mean like the low hanging fruits we could help tackle? -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-dev?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
