Hi Henrik! thanks for your detailed response. I'm aware of the issues you presented and agree with you.
That's the point of my previous message to this mailing list, actually. I believe that we can have our cake and eat it too, with the help of puppet developers. It is not difficult - as the monkey patches prove - to tweak the parser to preserve white spaces. Why not then modify it in a way that it gets more extensible in a sensible way? Then puppet-lint, Geppetto, puppet-cleaner and anything else that anyone can come up with will be able to extend it without monkey patching. No more worries about the new internal changes in the lexer any more. I'm volunteering to make it happen, but obviously puppet developers' will is essential for the success of this modification. P. S. as for the style guide and options to apply parts of them for the user, I wonder if you have seen the options available in puppet-cleaner. I'll take a look at Geppetto as well, thanks for the pointer! El lunes, 22 de abril de 2013 21:23:09 UTC-5, henrik lindberg escribió: > > This is quite difficult. The puppet lexer (with quite a lot of changes > in the 3.x code stream, and with even more changes in the 3.2 --parser > future lexer/parser which will the standard in puppet 4) is designed > primarily for runtime lexing/parsing. Specifically it: > - stops on several types of lexing errors > - does not preserve whitepsace / linebreaks / formatting > > This means that it is quite difficult to reuse it as a library function > for transformation and formatting. Before the new lexer/parser, the > lexer was also intimately tied to a specific puppet version. This means > that you need a specific version of the lexer to lex the puppet > manifests for that particular version of the language. Also not very > good for a more general purpose lexer / transformer. > > When doing cleaning/formatting there are issues that can not really be > resolved at the lexical level - you really do need a parser in order to > apply reasoning based on what the tokens mean (semantics) or may mean > (if ambiguous / in error / not supported in a particular version). > > The same issues (as for the lexer) applies to the puppet parser (this > will get easier with the new parser --parser future in 3.2) but it is > still focusing on parsing with the intent of evaluation. There is no > reverse serialization (from AST to source text) or pretty printing, or > "cleaner". Although starting with --parser future this would be possible > (still lots and lots of work) to implement. > > As the lexer/parser in puppet evolves it will not maintain backwards > compatibility at the token / parser level - these are considered > internal APIs. (i.e. Different and increasingly more complicated monkey > patches will be required as support for features like heredoc and puppet > templates are added - these are mostly about lexing). > > There are other initiatives; puppet-lint, and Geppetto (IDE) that also > deal with lexing/parsing/validation/formatting/cleanup. They both have > their own puppet DSL lexers / parsers. Geppetto builds a model of the > source and keeps track of all the source text, cross references between > all the files as well, it does validation, pretty printing, provides > quick fixes / clean up in an extensible way. Some of the functionality > is also available from the command line, and some via jenkins, and > travis-ci plugins. It handles Puppet 2.7, 3.x, PE (and in the next > release 3.2 and the future parser as well (as soon as 3.2 is released by > puppetlabs)). > > Lastly, the style guide and reality are not always the same thing :) and > people developing puppet manifests have their own set of styles and > rules they want to follow - one size most certainly does not fit all > here. Quite a lot of flexibility and options are required to > cleanup/format the manifests as well as providing options to fix/resolve > certain problems (hard or stylistic) in different ways that require user > interaction. > > Why not join forces with puppet-lint or Geppetto? As the principal > developer on Geppetto I know I could sure use help :) > > Best Regards > - henrik > > On 2013-22-04 20:50, Gerardo Santana G�mez Garrido wrote: > > Hi there, > > > > I've recently made public puppet-cleaner, a tool that makes a manifest > > comply with a subset of the style guide: > > > > https://github..com/santana/puppet-cleaner > > > > It monkey patches puppet 2.7.11, enabling COMMENT, MLCOMMENT and RETURN > > tokens and adding the BLANK token: > > > > > https://github.com/santana/puppet-cleaner/blob/master/lib/puppet-cleaner.rb > > > > Then it loops over the stream of tokens and apply a set of workers that > > transform the input. It's not difficult to extend it to add more > workers. > > > > I've received reports that puppet-cleaner doesn't work with Puppet 3.x, > > which is expected, since I'm tapping puppet internals in a non-reusable > way. > > > > Then I wondered, what if we can work together to make it easy to tap > > puppet's lexer? Then puppet-cleaner and any other tool that anyone can > > imagine may leverage it and create more interesting stuff. > > > > Then it occurred to me that it would be nice to have this cleaner > > included in the set of puppet's face applications and maintained as part > > of puppet's source code tree. Everybody would benefit from having an > > integrated and up to date puppet manifest cleaner. > > > > What do you think? I'm of course volunteering to help to work on it. > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Puppet Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [email protected] <javascript:>. > > To post to this group, send email to > > [email protected]<javascript:>. > > > Visit this group at http://groups.google.com/group/puppet-dev?hl=en. > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > > -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-dev?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
