Re: Thoughts on using Text::Template for our autogenerated code?
>
> Yeah, it's somewhat hard to believe that the cost/benefit ratio would be
> attractive. But maybe you could mock up some examples of what the input
> could look like, and get people on board (or not) before writing any
> code.
>
>
tl;dr - I tried a few things, nothing that persuades myself let alone the
community, but perhaps some ideas for the future.
I borrowed Bertrand's ongoing work for waiteventnames.* because that is
what got me thinking about this in the first place. I considered a few
different templating libraries:
There is no perl implementation of the golang template library (example of
that here: https://blog.gopheracademy.com/advent-2017/using-go-templates/ )
that I could find.
Text::Template does not support loops, and as such it is no better than
here-docs.
Template Toolkit seems to do what we need, but it has a kitchen sink of
dependencies that make it an unattractive option, so I didn't even attempt
it.
HTML::Template has looping and if/then/else constructs, and it is a single
standalone library. It also does a "separation of concerns" wherein you
pass in parameter names and values, and some parameters can be for loops,
which means you pass an arrayref of hashrefs that the template engine loops
over. That's where the advantages stop, however. It is fairly verbose, and
because it is HTML-centric it isn't very good about controlling whitespace,
which leads to piling template directives onto the same line in order to
avoid spurious newlines. As such I cannot recommend it.
My ideal template library would have text something like this:
[% loop events %]
[% $enum_value %]
[% if __first__ +%] = [%+ $inital_value %][% endif %]
[% if ! __last__ %],[% endif +%]
[% end loop %]
[% loop xml_blocks indent: relative,spaces,4 %]
[%element_body%]/>
[% end loop %]
[%+ means "leading whitespace matters", +%] means "trailing whitespace
matters"
That pseudocode is a mix of ASP, HTML::Template. The special variables
__first__ and __last__ refer to which iteration of the loop we are on. You
would pass it a data structure like this:
{events: [ { enum_value: "abc", initial_value: "def"}, ... { enum_value:
"wuv", initial_value: "xyz" } ],
xml_block: [ {attrib_val: "one", element_body: "two"} ]
}
I did one initial pass with just converting printf statements to here-docs,
and the results were pretty unsatisfying. It wasn't really possible to
"see" the output files take shape.
My next attempt was to take the "separation of concerns" code from the
HTML::Template version, constructing the nested data structure of resolved
output values, and then iterating over that once per output file. This
resulted in something cleaner, partly because we're only writing one file
type at a time, partly because the interpolated variables have names much
closer to their output meaning.
In doing this, it occurred to me that a lot of this effort is in getting
the code to conform to our own style guide, at the cost of the generator
code being less readable. What if we wrote the generator and formatted the
code in a way that made sense for the generator, and then pgindented it.
That's not the workflow right now, but perhaps it could be.
Conclusions:
- There is no "good enough" template engine that doesn't require big
changes in dependencies.
- pgindent will not save you from a run-on sentence, like putting all of
a typdef enum values on one line
- There is some clarity value in either separating input processing from
the output processing, or making the input align more closely with the
outputs
- Fiddling with indentation and spacing detracts from legibility no matter
what method is used.
- here docs are basically ok but they necessarily confuse output
indentation with code indentation. it is possible to de-indent them them
with <<~ but that's a 5.25+ feature.
- Any of these principles can be applied at any time, with no overhaul
required.
"sorted-" is the slightly modified version of Bertrand's code.
"eof-as-is-" is a direct conversion of the original but using here-docs.
"heredoc-fone-file-at-a-time-" first generates an output-friendly data
structure
"needs-pgindent-" is what is possible if we format for our own readability
and make pgindent fix the output, though it was not a perfect output match
sorted-generate-waiteventnames.pl
Description: Perl program
eof-as-is-generate-waiteventnames.pl
Description: Perl program
heredoc-one-file-at-a-time-generate-waiteventnames.pl
Description: Perl program
needs-pgindent-generate-waiteventnames.pl
Description: Perl program
Re: Thoughts on using Text::Template for our autogenerated code?
On Fri, Mar 31, 2023 at 4:15 AM Corey Huinker wrote: > For those who already responded, are your concerns limited to the dependency issue? Would you have concerns about a templating library that was developed by us and included in-tree? Libraries (and abstractions in general) require some mental effort to interface with them (that also means debugging when the output fails to match expectations), not to mention maintenance cost. There has to be a compensating benefit in return. The cost-to-benefit ratio here seems unfavorable -- seems like inventing a machine that ties shoelaces, but we usually wear sandals. -- John Naylor EDB: http://www.enterprisedb.com
Re: Thoughts on using Text::Template for our autogenerated code?
Andres Freund writes: > On 2023-03-30 17:15:20 -0400, Corey Huinker wrote: >> For those who already responded, are your concerns limited to the >> dependency issue? Would you have concerns about a templating library that >> was developed by us and included in-tree? I'm not suggesting suggesting we >> write one at this time, at least not until after we make a here-doc-ing >> pass first, but I want to understand the concerns before embarking on any >> refactoring. > The dependency is/was my main issue. But I'm also somewhat doubtful that what > we do warrants the use of a template library in the first place, but I could > be convinced by a patch showing it to be a significant improvement. Yeah, it's somewhat hard to believe that the cost/benefit ratio would be attractive. But maybe you could mock up some examples of what the input could look like, and get people on board (or not) before writing any code. regards, tom lane
Re: Thoughts on using Text::Template for our autogenerated code?
Hi, On 2023-03-30 17:15:20 -0400, Corey Huinker wrote: > For those who already responded, are your concerns limited to the > dependency issue? Would you have concerns about a templating library that > was developed by us and included in-tree? I'm not suggesting suggesting we > write one at this time, at least not until after we make a here-doc-ing > pass first, but I want to understand the concerns before embarking on any > refactoring. The dependency is/was my main issue. But I'm also somewhat doubtful that what we do warrants the use of a template library in the first place, but I could be convinced by a patch showing it to be a significant improvement. Greetings, Andres Freund
Re: Thoughts on using Text::Template for our autogenerated code?
> > I don't think that's remotely a starter. Asking people to install an old > and possibly buggy version of a perl module is not something we should do. > > I think the barrier for this is pretty high. I try to keep module > requirements for the buildfarm client pretty minimal, and this could affect > a much larger group of people. > Those are good reasons. For those who already responded, are your concerns limited to the dependency issue? Would you have concerns about a templating library that was developed by us and included in-tree? I'm not suggesting suggesting we write one at this time, at least not until after we make a here-doc-ing pass first, but I want to understand the concerns before embarking on any refactoring.
Re: Thoughts on using Text::Template for our autogenerated code?
> > I think many of those could just be replaced by multi-line strings and/or > here > documents to get most of the win. > I agree that a pretty good chunk of it can be done with here-docs, but template files do have attractive features (separation of concerns, syntax highlighting, etc) that made it worth asking.
Re: Thoughts on using Text::Template for our autogenerated code?
On 2023-03-30 Th 15:06, Daniel Gustafsson wrote: On 30 Mar 2023, at 19:06, Corey Huinker wrote: Some other digging around shows that the module has been around since 1996 (Perl5 was 1994) and hasn't had a feature update (or any update for that matter) since 2003. So it should meet our baseline 5.14 requirement, which came out in 2011. I assume you then mean tying this to 1.44 (or another version?), since AFAICT there has been both features and bugfixes well after 2003? https://metacpan.org/dist/Text-Template/changes I don't think that's remotely a starter. Asking people to install an old and possibly buggy version of a perl module is not something we should do. I think the barrier for this is pretty high. I try to keep module requirements for the buildfarm client pretty minimal, and this could affect a much larger group of people. cheers andrew -- Andrew Dunstan EDB:https://www.enterprisedb.com
Re: Thoughts on using Text::Template for our autogenerated code?
> On 30 Mar 2023, at 19:06, Corey Huinker wrote: > Some other digging around shows that the module has been around since 1996 > (Perl5 was 1994) and hasn't had a feature update (or any update for that > matter) since 2003. So it should meet our baseline 5.14 requirement, which > came out in 2011. I assume you then mean tying this to 1.44 (or another version?), since AFAICT there has been both features and bugfixes well after 2003? https://metacpan.org/dist/Text-Template/changes -- Daniel Gustafsson
Re: Thoughts on using Text::Template for our autogenerated code?
Hi, On 2023-03-30 13:06:46 -0400, Corey Huinker wrote: > Is there a barrier to us using non-core perl modules, in this case > Text::Template? I don't think we should have a hard build-time dependency on non-core perl modules. On some operating systems having to install such dependencies is quite painful (e.g. windows). > I think it would be a tremendous improvement in readability and > maintainability over our current series of print statements, some > multiline, some not. I think many of those could just be replaced by multi-line strings and/or here documents to get most of the win. Greetings, Andres Freund
Re: Thoughts on using Text::Template for our autogenerated code?
Corey Huinker writes: > Is there a barrier to us using non-core perl modules, in this case > Text::Template? Use for what exactly? I'd be hesitant to require such things to build from a tarball, or to run regression tests. If it's used to build a generated file that we include in tarballs, that might be workable ... but I'd bet a good fraction of the buildfarm falls over (looks like all four of my animals would), and you might get push-back from developers too. > I think it would be a tremendous improvement in readability and > maintainability over our current series of print statements, some > multiline, some not. I suspect it'd have to be quite a remarkable improvement to justify adding a new required build tool ... but show us an example. regards, tom lane
