On Sat, Aug 18, 2018 at 7:30 PM, Daniel Dekany <[email protected]> wrote:
> Saturday, August 18, 2018, 7:19:43 PM, Woonsan Ko wrote:
>
>> On Fri, Aug 17, 2018 at 6:35 PM, Daniel Dekany <[email protected]> wrote:
>>> Friday, August 17, 2018, 7:11:33 PM, Woonsan Ko wrote:
> [snip]
>> Perhaps we can start simply with <sharedDataModel>, <file>, and
>> <mavenProperties> (as Ben and Mark has used it) only in the 1st phase.
>> We can extend it like what you said in the next phases.
>
> Sure. It's just that the original architecture should be compatible
> with the fancier things we are talking about.
Right.
>
> [snip]
>>> That's new approach. Let's see...
>>
>> Yeah, that's something we can consider later, indeed. Not in the 1st phase.
>
> It's better to know fundamental things like this (how generators work)
> ahead.
Right.
Perhaps we can describe the 'variability' in the next phases from
architectural perspective somewhere.
Do we have a wiki? I think wiki is a better place to put all the
product/architecture descriptions than JIRA tickets. What do you
think?
>
>>> What if you have Foo.java.json and Foo.java.xml? I assume that's an
>>> error. So if Foo.java.flt is just a generator, and Foo.java.json is
>>> just a generator, and yet you can pair them, while you can't pair json
>>> with xml. That makes .ftl generators special.
>>>
>>> Also it makes the meaning of templates/ more foggy, since in your
>>> example Foo.java.ftl is exactly the kind of template that you would
>>> put into templates/. As you have stated that too.
>>>
>>> Also, in principle nothing should stop a generator ftl to specify its
>>> own data-model. It could specify to the data-model via <#ftl
>>> attributes={ 'dataModel': ... }>. That's a situation where a json pair
>>> is strange.
>>>
>>> So to me this approach makes the roles of files less clear. But, the
>>> question is if this approach has practical value. Like is there a
>>> known usage scenario where you need a lot of data-models, each with a
>>> specific template that's not used for anything else? What other use
>>> cases are there where you could utilize this generalization?
>>
>> I originally thought this .ftl generator only approach could have a
>> value (like the velocity maven plugin I referred to before), but now I
>> realize that as we already have a different directory structure with
>> separate ones (generator, template, data) already, .ftl generator only
>> approach doesn't make any sense.
>> I'm fine without it.
>
> I'm confused. What do you mean by ".ftl generator only approach"?
What I thought was as follows:
Optionally they can put only .ftl files in the generators directory.
e.g, src/main/freemarker-generator/generators/.../Foo.java.ftl.
Then even if they don't have .json, the system would scan the .ftl
only generator to produce output(s). Just freely write .ftl files
separately to output file(s).
Of course this .ftl file can include or import from others in
*template* directory. As you pointed out, they can define data models
in the .ftl only generator files as well.
I thought this might be helpful to some use cases, and it works
similarly to how the velocity plugin [1] works.
[1] https://github.com/nativelibs4java/maven-velocity-plugin
> Because, I definitely think that ftl files as generators are useful.
> It's a common way of generating files (like when you generate a home
> page that consist of a bunch of files with different structure). So is
> what the json generators do currently, but it's an approach useful in
> different applications. My point is that we can support both and still
> have simple/consistent tool. (Your idea is like if you somehow combine
> together the files with common extensionless-name, to form a single
> generator, which is maybe smart, but I believe it's also harder to
> grasp, so I wanted to see the practical application.)
I described what I was thinking above. Would it be a problematic to support?
While describing it again, I've realized that it is still better to
put the .ftl only generator files under the generators directory while
any included/imported .ftl templates should be in templates directory.
Otherwise, the scanning might be problematic.
Thanks for your elaborations!
Woonsan
>
>> @Ben / @Mark, what do you think? Would these
>> generalization/improvements be acceptable to you?
>>
>> Regards,
>>
>> Woonsan
>>
>>>
>>>> - The output file name is taken after removing the generator file
>>>> extension (.json or .xml), so it can be applied to various cases. e.g,
>>>> .../foo.html.json, .../foo.js.json, or even .../foo.json.json.
>>>>
>>>> ## AFG Core Module
>>>>
>>>> As the features can be useful to any other tools or environments, the
>>>> generic feature should be implemented in a separate core module,
>>>> without depending on Maven directly.
>>>> Maven Plugin submodule or any others depend on this core module.
>>>>
>>>> -----
>>>>
>>>> Perhaps I missed some details and you guys have some different tastes
>>>> or flavors on each, but I've tried to capture all the high level
>>>> features. ;-)
>>>> Please correct anything if I'm wrong in conceptual level or in detail
>>>> level.
>>>> Also, if we come to a consensus in high level about the features,
>>>> perhaps we can move the feature list/descriptions to a JIRA ticket and
>>>> its description.
>>>>
>>>> What do you think?
>>>>
>>>> Regards,
>>>>
>>>> Woonsan
>>>>
>>>>
>>>> On Fri, Aug 17, 2018 at 12:04 PM, Daniel Dekany <[email protected]> wrote:
>>>>> Friday, August 17, 2018, 3:56:27 PM, Ben Jackson wrote:
>>>>>
>>>>>> I think I understand your idea about templates being in both locations,
>>>>>> and
>>>>>> I think it is a reasonable approach.
>>>>>>
>>>>>> My thought is that in the case that you have multiple data models, it
>>>>>> would
>>>>>> probably be clearer to have all of the templates one directory and bind
>>>>>> them to the appropriate data model explicitly.
>>>>>
>>>>> All of the templates? No, I certainly want to keep generator/
>>>>> templates and dependency templates (those in templates/ at the moment)
>>>>> clearly separated. The primary consideration when choosing the
>>>>> directory should be the *role* of the files there, not their format.
>>>>> That generator files can be json-s and ftl-s, and there can be
>>>>> dependencies of generator files which happen to be also ftl-s is just
>>>>> a coincidence. Looking at it from another angle, I think it's good if
>>>>> when in generators/ you see foo.txt.json and bar.txt.ftl and nothing
>>>>> else, then you know that you will have foo.txt and bar.txt in your
>>>>> output directory. If now there you will also see some-dependency.ftl,
>>>>> and it won't generate "some-dependency", that's confusing (plus how
>>>>> will even the plugin know if that's not a generator file).
>>>>>
>>>>> As of the data-model, the most comment case when using FreeMarker is
>>>>> to have a different data-model, per template, and even per execution
>>>>> (as variables change over time). It's also normal that data-models
>>>>> have common parts, like applications-scope variables or commonly
>>>>> utilities. (There's even a Configuration setting for that, called
>>>>> sharedVariables, but you might chose to make you data-model layered
>>>>> otherwise). So, I perceive it as completely normal that there's a
>>>>> shared part of the data-model, and some generator files add their own
>>>>> layer over that. BTW, this is also what the current version does, only
>>>>> the shared part of the data-model is fixed to that variable that holds
>>>>> the Maven properties.
>>>>>
>>>>>> And in the case that you have a single data model, you could provide this
>>>>>> shortcut to not needing to bind it explicitly to all of your templates
>>>>>> through some configuration option. And in this case, there wouldn't be
>>>>>> the
>>>>>> need to have the templates in the generator directory.
>>>>>
>>>>> Ah, I see, you mean there should be a json, that doesn't specify a
>>>>> data-model, and points to the template. That's conceptually the
>>>>> cleanest, but that problem is that if your main use case is like in
>>>>> FMPP, it's annoying to the level that you practically don't support
>>>>> that use case.
>>>>>
>>>>>> The consistent aspect of that approach is that the templates are always
>>>>>> in
>>>>>> the /templates directory,
>>>>>
>>>>> That "rhymes", but I don't think should be a goal in itself... We just
>>>>> low to use ftl-s for all kind of different purposes around here. :)
>>>>>
>>>>>> and a "generator" has a consistent definition of
>>>>>> a configuration file that binds a data model to a template.
>>>>>
>>>>> Yeah, but you can see it as if that binding is sometimes implicit. You
>>>>> can see an quick example of that in the FMPP page linked earlier...
>>>>> where the red lines fork out.
>>>>>
>>>>>> The consistent aspect of the templates in the generator directory
>>>>>> approach
>>>>>> is that things that generate files are always in the same location (the
>>>>>> generators directory), and we could enforce the property that one source
>>>>>> file is generated for every file in the generators directory.
>>>>>
>>>>> I agree with that, but it's not against putting *generator* ftl-s into
>>>>> generators/.
>>>>>
>>>>> Actually... there's a frequent use case where you want to generate
>>>>> output files based on the content of the data-model. See this:
>>>>> http://fmpp.sourceforge.net/qtour.html#sect4. But that the basic
>>>>> approach still remains true.
>>>>>
>>>>>> On Fri, Aug 17, 2018 at 1:32 AM, Daniel Dekany <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> Friday, August 17, 2018, 12:36:51 AM, Ben Jackson wrote:
>>>>>>>
>>>>>>> > Here is my summary of the conversation and my take on it.
>>>>>>> >
>>>>>>> > There are two primary use cases discussed in the thread.
>>>>>>> >
>>>>>>> > 1. Multiple templates instantiated in multiple ways (with multiple
>>>>>>> > "data
>>>>>>> > models"). This was our use case.
>>>>>>> > 2. A bunch of templates being instantiated based on a single shared
>>>>>>> > "data
>>>>>>> > model"
>>>>>>> >
>>>>>>> > So how to best bring together the "data model" and the "template" to
>>>>>>> > fit
>>>>>>> > these use cases with minimal overhead?
>>>>>>> > There is the generic way which is to list out all of the templates and
>>>>>>> data
>>>>>>> > models in separate locations, and then to bring together the templates
>>>>>>> and
>>>>>>> > data models via some configuration. In this world, a structure might
>>>>>>> > be:
>>>>>>> >
>>>>>>> > (@daniel you seem concerend by the default location being
>>>>>>> /src/main/freemarker,
>>>>>>> > do you have a better idea in mind?)
>>>>>>>
>>>>>>> My approach is that this is a product, and products has a somewhat
>>>>>>> unique (basically means searchable) name. Calling this just the
>>>>>>> "Apache FreeMarker Maven Plugin" is problematic for that reason. For
>>>>>>> starters, it's functionality is not very much Maven specific, so it
>>>>>>> might as well will be callable from Gradle, or directly from command
>>>>>>> line (if you modularize it like that). Also there are a few other
>>>>>>> FreeMarker Maven Plugins, which makes searching for related content on
>>>>>>> the web hard. If we have a product name, then it's also obvious what
>>>>>>> the directory name will be. So, let's say, it's "Apache FreeMarker
>>>>>>> FGen" (maybe a stupid name, but just as an example), then, you have
>>>>>>> /src/main/fgen, and when someone sees it and Googles it, he will find
>>>>>>> what that does.
>>>>>>>
>>>>>>> > /src/main/generated/freemarker/dataModels - data models
>>>>>>> > /src/main/generated/freemarker/templates - freemarker templates
>>>>>>> > /src/main/generated/freemarker/generators - json, one per output file
>>>>>>>
>>>>>>> Note that it wasn't the structure I have suggested, just in case you
>>>>>>> (or someone) have missed that. What I said that for case #2, you
>>>>>>> simply put templates into generators/. And of course you can have both
>>>>>>> ftl and json files inside generators/, each generating its output file
>>>>>>> in its own way.
>>>>>>>
>>>>>>> As of dataModel-s, the part of the content the json files is also part
>>>>>>> of the data-model, so it's not the best name. It's rather
>>>>>>> "shared-dataModel", and in case you specify that directly in the
>>>>>>> pom.xml, you don't even need that. And of course, we are talking about
>>>>>>> a layered data-model, where there's a shared data model that's visible
>>>>>>> from everywhere, and it the case of a json generator file it can adds
>>>>>>> another layer on top of that.
>>>>>>>
>>>>>>> > The issue with this more generic structure is that in exchange for
>>>>>>> > this
>>>>>>> > flexibility, you might have a bunch of extra files. For example, in
>>>>>>> > the
>>>>>>> > special case #2, one needs twice as many files in the above structure.
>>>>>>> >
>>>>>>> > So instead of the above structure, the suggestion in the thread is to
>>>>>>> embed
>>>>>>> > both of the common use cases into the plugin in a way that they "play
>>>>>>> nice"
>>>>>>> > together.
>>>>>>> > Create a generators directory and have the plugin interpret how to
>>>>>>> generate
>>>>>>> > the source based on the type of thing in that directory.
>>>>>>> >
>>>>>>> > I think that we could instead ask how we might deviate from the above
>>>>>>> > generic structure that allows a many-to-many mapping while reusing
>>>>>>> > both
>>>>>>> > data models
>>>>>>> > and templates in such a way that the two special cases can be
>>>>>>> > naturally
>>>>>>> > supported.
>>>>>>> >
>>>>>>> > The first enhancement is that a generator could optionally embed the
>>>>>>> > data
>>>>>>> > model. This handles case #1 if you do not want
>>>>>>> > to reuse data models. It also naturally comes from enhancing the
>>>>>>> > json structure in the way that Daniel suggested:
>>>>>>> >
>>>>>>> > {
>>>>>>> > "templateName": "foo.ftl",
>>>>>>> > "otherFutureMavenPluginKey": "something",
>>>>>>> > "dataModel": {
>>>>>>> > /* your application-specific stuff */
>>>>>>> > "myString": "a string",
>>>>>>> > "myNumber": 1,
>>>>>>> > ...
>>>>>>> > }
>>>>>>> > }
>>>>>>> >
>>>>>>> > Here the other option would be for "dataModel" to refer to a file in
>>>>>>> > the
>>>>>>> > "dataModel" directory.
>>>>>>>
>>>>>>> Sure that's a more generic solution. Though usually you don't want to
>>>>>>> use a json generator file if you don't want a data-model that's
>>>>>>> specific to that generator file.
>>>>>>>
>>>>>>> > I would say that instead of overloading the "generators" directory to
>>>>>>> also
>>>>>>> > include templates,
>>>>>>>
>>>>>>> To clarify, there are templates in the templates\ directory, and then
>>>>>>> there can be templates in the generator/ directory. They are
>>>>>>> completely separate matters. In the 2nd case, they are just a type of
>>>>>>> a generator files, like json-s are.
>>>>>>>
>>>>>>> > we allow the plugin to operate in an alternate mode,
>>>>>>> > enabled by configuration, say by setting a "defaultDataModel"
>>>>>>> > parameter.
>>>>>>> In
>>>>>>> > this mode, we generate sources for all templates in the template
>>>>>>> directory
>>>>>>> > from a common data model.
>>>>>>>
>>>>>>> My idea was that the we only ever generate output files for what's in
>>>>>>> the generator directory. There can be files both in generators/ and
>>>>>>> templates/, but hey have a different role. Templates in templates/ are
>>>>>>> just dependencies of some generators/ (the jsons, in this case), or
>>>>>>> maybe they can be #import-ed from templates in generators/.
>>>>>>>
>>>>>>> > In this use case, the project would only use
>>>>>>> > this structure only:
>>>>>>> >
>>>>>>> > /src/main/generated/freemarker/dataModel.xml
>>>>>>>
>>>>>>> NYW, I think for a dataModel it's important that it can come from
>>>>>>> various places, like even from a database. So we should keep that in
>>>>>>> mind.
>>>>>>>
>>>>>>> > /src/main/generated/freemarker/templates/...
>>>>>>> >
>>>>>>> > Having said all of this, I think the changes we would make in the
>>>>>>> > short
>>>>>>> > term would be to update the json structure to include the dataModel
>>>>>>> > as a
>>>>>>> > subtree, and to change the default locations to use the /generated
>>>>>>> > subdirectory rather than /data, and whatever suggestion is
>>>>>>> > appropriate.
>>>>>>> >
>>>>>>> > -Ben
>>>>>>> >
>>>>>>> > On Thu, Aug 16, 2018 at 4:18 PM, Ben Jackson <[email protected]>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> >> Yes, this makes sense.
>>>>>>> >>
>>>>>>> >> -Ben
>>>>>>> >>
>>>>>>> >> On Thu, Aug 16, 2018 at 2:56 PM, Daniel Dekany <[email protected]>
>>>>>>> wrote:
>>>>>>> >>
>>>>>>> >>> Wednesday, August 15, 2018, 10:52:41 PM, Mark Malynn wrote:
>>>>>>> >>>
>>>>>>> >>> [snip]
>>>>>>> >>> > 3) I agree that using the well known json field 'templateName'
>>>>>>> >>> > seems
>>>>>>> >>> > 'hacky' (from a freemarker point of view). However, from the
>>>>>>> >>> > point of
>>>>>>> >>> view
>>>>>>> >>> > of the maven plugin it works nice. It provides the flexibility I
>>>>>>> >>> > need
>>>>>>> >>> with
>>>>>>> >>> > the minimum fuss. I will entertain another mechanism as long as it
>>>>>>> >>> supports
>>>>>>> >>> > my main use case.
>>>>>>> >>> [snip]
>>>>>>> >>>
>>>>>>> >>> The problem with that is that it doesn't work if you happen to need
>>>>>>> >>> a
>>>>>>> >>> "tempalteName" field in your actual data. Yes, that's unlikely, but
>>>>>>> >>> take it as a demonstration of why it's not correct. The "correct"
>>>>>>> solution
>>>>>>> >>> is like this:
>>>>>>> >>>
>>>>>>> >>> {
>>>>>>> >>> "templateName": "foo.ftl",
>>>>>>> >>> "otherFutureMavenPluginKey": "something",
>>>>>>> >>> "dataModel": {
>>>>>>> >>> /* your application-specific stuff */
>>>>>>> >>> "myString": "a string",
>>>>>>> >>> "myNumber": 1,
>>>>>>> >>> ...
>>>>>>> >>> }
>>>>>>> >>> }
>>>>>>> >>>
>>>>>>> >>> Note how it allows adding new keys (like
>>>>>>> >>> "otherFutureMavenPluginKey")
>>>>>>> >>> without breaking backward compatibility.
>>>>>>> >>>
>>>>>>> >>> --
>>>>>>> >>> Thanks,
>>>>>>> >>> Daniel Dekany
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> --
>>>>>>> >> Benjamin Grant Jackson
>>>>>>> >>
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> Daniel Dekany
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Daniel Dekany
>>>>>
>>>>
>>>
>>> --
>>> Thanks,
>>> Daniel Dekany
>>>
>>
>
> --
> Thanks,
> Daniel Dekany
>
