This is quite difficult. The puppet lexer (with quite a lot of changes
in the 3.x code stream, and with even more changes in the 3.2 --parser
future lexer/parser which will the standard in puppet 4) is designed
primarily for runtime lexing/parsing. Specifically it:
- stops on several types of lexing errors
- does not preserve whitepsace / linebreaks / formatting
This means that it is quite difficult to reuse it as a library function
for transformation and formatting. Before the new lexer/parser, the
lexer was also intimately tied to a specific puppet version. This means
that you need a specific version of the lexer to lex the puppet
manifests for that particular version of the language. Also not very
good for a more general purpose lexer / transformer.
When doing cleaning/formatting there are issues that can not really be
resolved at the lexical level - you really do need a parser in order to
apply reasoning based on what the tokens mean (semantics) or may mean
(if ambiguous / in error / not supported in a particular version).
The same issues (as for the lexer) applies to the puppet parser (this
will get easier with the new parser --parser future in 3.2) but it is
still focusing on parsing with the intent of evaluation. There is no
reverse serialization (from AST to source text) or pretty printing, or
"cleaner". Although starting with --parser future this would be possible
(still lots and lots of work) to implement.
As the lexer/parser in puppet evolves it will not maintain backwards
compatibility at the token / parser level - these are considered
internal APIs. (i.e. Different and increasingly more complicated monkey
patches will be required as support for features like heredoc and puppet
templates are added - these are mostly about lexing).
There are other initiatives; puppet-lint, and Geppetto (IDE) that also
deal with lexing/parsing/validation/formatting/cleanup. They both have
their own puppet DSL lexers / parsers. Geppetto builds a model of the
source and keeps track of all the source text, cross references between
all the files as well, it does validation, pretty printing, provides
quick fixes / clean up in an extensible way. Some of the functionality
is also available from the command line, and some via jenkins, and
travis-ci plugins. It handles Puppet 2.7, 3.x, PE (and in the next
release 3.2 and the future parser as well (as soon as 3.2 is released by
puppetlabs)).
Lastly, the style guide and reality are not always the same thing :) and
people developing puppet manifests have their own set of styles and
rules they want to follow - one size most certainly does not fit all
here. Quite a lot of flexibility and options are required to
cleanup/format the manifests as well as providing options to fix/resolve
certain problems (hard or stylistic) in different ways that require user
interaction.
Why not join forces with puppet-lint or Geppetto? As the principal
developer on Geppetto I know I could sure use help :)
Best Regards
- henrik
On 2013-22-04 20:50, Gerardo Santana Gómez Garrido wrote:
Hi there,
I've recently made public puppet-cleaner, a tool that makes a manifest
comply with a subset of the style guide:
https://github..com/santana/puppet-cleaner
It monkey patches puppet 2.7.11, enabling COMMENT, MLCOMMENT and RETURN
tokens and adding the BLANK token:
https://github.com/santana/puppet-cleaner/blob/master/lib/puppet-cleaner.rb
Then it loops over the stream of tokens and apply a set of workers that
transform the input. It's not difficult to extend it to add more workers.
I've received reports that puppet-cleaner doesn't work with Puppet 3.x,
which is expected, since I'm tapping puppet internals in a non-reusable way.
Then I wondered, what if we can work together to make it easy to tap
puppet's lexer? Then puppet-cleaner and any other tool that anyone can
imagine may leverage it and create more interesting stuff.
Then it occurred to me that it would be nice to have this cleaner
included in the set of puppet's face applications and maintained as part
of puppet's source code tree. Everybody would benefit from having an
integrated and up to date puppet manifest cleaner.
What do you think? I'm of course volunteering to help to work on it.
--
You received this message because you are subscribed to the Google
Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "Puppet
Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
