Monday, August 20, 2018, 3:18:20 PM, Woonsan Ko wrote: > On Sat, Aug 18, 2018 at 7:30 PM, Daniel Dekany <[email protected]> wrote: >> Saturday, August 18, 2018, 7:19:43 PM, Woonsan Ko wrote: >> >>> On Fri, Aug 17, 2018 at 6:35 PM, Daniel Dekany <[email protected]> wrote: >>>> Friday, August 17, 2018, 7:11:33 PM, Woonsan Ko wrote: >> [snip] >>> Perhaps we can start simply with <sharedDataModel>, <file>, and >>> <mavenProperties> (as Ben and Mark has used it) only in the 1st phase. >>> We can extend it like what you said in the next phases. >> >> Sure. It's just that the original architecture should be compatible >> with the fancier things we are talking about. > Right. > >> >> [snip] >>>> That's new approach. Let's see... >>> >>> Yeah, that's something we can consider later, indeed. Not in the 1st phase. >> >> It's better to know fundamental things like this (how generators work) >> ahead. > Right. > Perhaps we can describe the 'variability' in the next phases from > architectural perspective somewhere. > Do we have a wiki?
Yes: https://cwiki.apache.org/confluence/display/FREEMARKER/ > I think wiki is a better place to put all the > product/architecture descriptions than JIRA tickets. What do you > think? Without an ASF account the contributors couldn't edit the Wiki page. If *they* open a JIRA issue, then we all can edit it. So that's what I recommend. >>>> What if you have Foo.java.json and Foo.java.xml? I assume that's an >>>> error. So if Foo.java.flt is just a generator, and Foo.java.json is >>>> just a generator, and yet you can pair them, while you can't pair json >>>> with xml. That makes .ftl generators special. >>>> >>>> Also it makes the meaning of templates/ more foggy, since in your >>>> example Foo.java.ftl is exactly the kind of template that you would >>>> put into templates/. As you have stated that too. >>>> >>>> Also, in principle nothing should stop a generator ftl to specify its >>>> own data-model. It could specify to the data-model via <#ftl >>>> attributes={ 'dataModel': ... }>. That's a situation where a json pair >>>> is strange. >>>> >>>> So to me this approach makes the roles of files less clear. But, the >>>> question is if this approach has practical value. Like is there a >>>> known usage scenario where you need a lot of data-models, each with a >>>> specific template that's not used for anything else? What other use >>>> cases are there where you could utilize this generalization? >>> >>> I originally thought this .ftl generator only approach could have a >>> value (like the velocity maven plugin I referred to before), but now I >>> realize that as we already have a different directory structure with >>> separate ones (generator, template, data) already, .ftl generator only >>> approach doesn't make any sense. >>> I'm fine without it. >> >> I'm confused. What do you mean by ".ftl generator only approach"? > What I thought was as follows: Optionally they can put only .ftl > files in the generators directory. e.g, > src/main/freemarker-generator/generators/.../Foo.java.ftl. Then even > if they don't have .json, the system would scan the .ftl only > generator to produce output(s). Just freely write .ftl files > separately to output file(s). Of course this .ftl file can include > or import from others in *template* directory. As you pointed out, > they can define data models in the .ftl only generator files as > well. This was my intent from the beginning. A generator file can be a .json, or an .ftl[h|x] (or whatever we come up with later, like an xml that's transformed by a templates/myxmltransformat.ftl). They are totally unrelated, independent generator files. I have never assumed that they come in pairs, hence there was never a such thing as "ftl only generator" in my head either. All generator files are lonely. > I thought this might be helpful to some use cases, and it works > similarly to how the velocity plugin [1] works. > > [1] https://github.com/nativelibs4java/maven-velocity-plugin > >> Because, I definitely think that ftl files as generators are useful. >> It's a common way of generating files (like when you generate a home >> page that consist of a bunch of files with different structure). So is >> what the json generators do currently, but it's an approach useful in >> different applications. My point is that we can support both and still >> have simple/consistent tool. (Your idea is like if you somehow combine >> together the files with common extensionless-name, to form a single >> generator, which is maybe smart, but I believe it's also harder to >> grasp, so I wanted to see the practical application.) > I described what I was thinking above. Would it be a problematic to support? > While describing it again, I've realized that it is still better to > put the .ftl only generator files under the generators directory while > any included/imported .ftl templates should be in templates directory. > Otherwise, the scanning might be problematic. Each file in "generators/" will be processed to generate output files, so yes, non-generator files don't go into "generators/". > Thanks for your elaborations! > > Woonsan > >> >>> @Ben / @Mark, what do you think? Would these >>> generalization/improvements be acceptable to you? >>> >>> Regards, >>> >>> Woonsan >>> >>>> >>>>> - The output file name is taken after removing the generator file >>>>> extension (.json or .xml), so it can be applied to various cases. e.g, >>>>> .../foo.html.json, .../foo.js.json, or even .../foo.json.json. >>>>> >>>>> ## AFG Core Module >>>>> >>>>> As the features can be useful to any other tools or environments, the >>>>> generic feature should be implemented in a separate core module, >>>>> without depending on Maven directly. >>>>> Maven Plugin submodule or any others depend on this core module. >>>>> >>>>> ----- >>>>> >>>>> Perhaps I missed some details and you guys have some different tastes >>>>> or flavors on each, but I've tried to capture all the high level >>>>> features. ;-) >>>>> Please correct anything if I'm wrong in conceptual level or in detail >>>>> level. >>>>> Also, if we come to a consensus in high level about the features, >>>>> perhaps we can move the feature list/descriptions to a JIRA ticket and >>>>> its description. >>>>> >>>>> What do you think? >>>>> >>>>> Regards, >>>>> >>>>> Woonsan >>>>> >>>>> >>>>> On Fri, Aug 17, 2018 at 12:04 PM, Daniel Dekany <[email protected]> >>>>> wrote: >>>>>> Friday, August 17, 2018, 3:56:27 PM, Ben Jackson wrote: >>>>>> >>>>>>> I think I understand your idea about templates being in both locations, >>>>>>> and >>>>>>> I think it is a reasonable approach. >>>>>>> >>>>>>> My thought is that in the case that you have multiple data models, it >>>>>>> would >>>>>>> probably be clearer to have all of the templates one directory and bind >>>>>>> them to the appropriate data model explicitly. >>>>>> >>>>>> All of the templates? No, I certainly want to keep generator/ >>>>>> templates and dependency templates (those in templates/ at the moment) >>>>>> clearly separated. The primary consideration when choosing the >>>>>> directory should be the *role* of the files there, not their format. >>>>>> That generator files can be json-s and ftl-s, and there can be >>>>>> dependencies of generator files which happen to be also ftl-s is just >>>>>> a coincidence. Looking at it from another angle, I think it's good if >>>>>> when in generators/ you see foo.txt.json and bar.txt.ftl and nothing >>>>>> else, then you know that you will have foo.txt and bar.txt in your >>>>>> output directory. If now there you will also see some-dependency.ftl, >>>>>> and it won't generate "some-dependency", that's confusing (plus how >>>>>> will even the plugin know if that's not a generator file). >>>>>> >>>>>> As of the data-model, the most comment case when using FreeMarker is >>>>>> to have a different data-model, per template, and even per execution >>>>>> (as variables change over time). It's also normal that data-models >>>>>> have common parts, like applications-scope variables or commonly >>>>>> utilities. (There's even a Configuration setting for that, called >>>>>> sharedVariables, but you might chose to make you data-model layered >>>>>> otherwise). So, I perceive it as completely normal that there's a >>>>>> shared part of the data-model, and some generator files add their own >>>>>> layer over that. BTW, this is also what the current version does, only >>>>>> the shared part of the data-model is fixed to that variable that holds >>>>>> the Maven properties. >>>>>> >>>>>>> And in the case that you have a single data model, you could provide >>>>>>> this >>>>>>> shortcut to not needing to bind it explicitly to all of your templates >>>>>>> through some configuration option. And in this case, there wouldn't be >>>>>>> the >>>>>>> need to have the templates in the generator directory. >>>>>> >>>>>> Ah, I see, you mean there should be a json, that doesn't specify a >>>>>> data-model, and points to the template. That's conceptually the >>>>>> cleanest, but that problem is that if your main use case is like in >>>>>> FMPP, it's annoying to the level that you practically don't support >>>>>> that use case. >>>>>> >>>>>>> The consistent aspect of that approach is that the templates are always >>>>>>> in >>>>>>> the /templates directory, >>>>>> >>>>>> That "rhymes", but I don't think should be a goal in itself... We just >>>>>> low to use ftl-s for all kind of different purposes around here. :) >>>>>> >>>>>>> and a "generator" has a consistent definition of >>>>>>> a configuration file that binds a data model to a template. >>>>>> >>>>>> Yeah, but you can see it as if that binding is sometimes implicit. You >>>>>> can see an quick example of that in the FMPP page linked earlier... >>>>>> where the red lines fork out. >>>>>> >>>>>>> The consistent aspect of the templates in the generator directory >>>>>>> approach >>>>>>> is that things that generate files are always in the same location (the >>>>>>> generators directory), and we could enforce the property that one source >>>>>>> file is generated for every file in the generators directory. >>>>>> >>>>>> I agree with that, but it's not against putting *generator* ftl-s into >>>>>> generators/. >>>>>> >>>>>> Actually... there's a frequent use case where you want to generate >>>>>> output files based on the content of the data-model. See this: >>>>>> http://fmpp.sourceforge.net/qtour.html#sect4. But that the basic >>>>>> approach still remains true. >>>>>> >>>>>>> On Fri, Aug 17, 2018 at 1:32 AM, Daniel Dekany <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Friday, August 17, 2018, 12:36:51 AM, Ben Jackson wrote: >>>>>>>> >>>>>>>> > Here is my summary of the conversation and my take on it. >>>>>>>> > >>>>>>>> > There are two primary use cases discussed in the thread. >>>>>>>> > >>>>>>>> > 1. Multiple templates instantiated in multiple ways (with multiple >>>>>>>> > "data >>>>>>>> > models"). This was our use case. >>>>>>>> > 2. A bunch of templates being instantiated based on a single shared >>>>>>>> > "data >>>>>>>> > model" >>>>>>>> > >>>>>>>> > So how to best bring together the "data model" and the "template" to >>>>>>>> > fit >>>>>>>> > these use cases with minimal overhead? >>>>>>>> > There is the generic way which is to list out all of the templates >>>>>>>> > and >>>>>>>> data >>>>>>>> > models in separate locations, and then to bring together the >>>>>>>> > templates >>>>>>>> and >>>>>>>> > data models via some configuration. In this world, a structure might >>>>>>>> > be: >>>>>>>> > >>>>>>>> > (@daniel you seem concerend by the default location being >>>>>>>> /src/main/freemarker, >>>>>>>> > do you have a better idea in mind?) >>>>>>>> >>>>>>>> My approach is that this is a product, and products has a somewhat >>>>>>>> unique (basically means searchable) name. Calling this just the >>>>>>>> "Apache FreeMarker Maven Plugin" is problematic for that reason. For >>>>>>>> starters, it's functionality is not very much Maven specific, so it >>>>>>>> might as well will be callable from Gradle, or directly from command >>>>>>>> line (if you modularize it like that). Also there are a few other >>>>>>>> FreeMarker Maven Plugins, which makes searching for related content on >>>>>>>> the web hard. If we have a product name, then it's also obvious what >>>>>>>> the directory name will be. So, let's say, it's "Apache FreeMarker >>>>>>>> FGen" (maybe a stupid name, but just as an example), then, you have >>>>>>>> /src/main/fgen, and when someone sees it and Googles it, he will find >>>>>>>> what that does. >>>>>>>> >>>>>>>> > /src/main/generated/freemarker/dataModels - data models >>>>>>>> > /src/main/generated/freemarker/templates - freemarker templates >>>>>>>> > /src/main/generated/freemarker/generators - json, one per output file >>>>>>>> >>>>>>>> Note that it wasn't the structure I have suggested, just in case you >>>>>>>> (or someone) have missed that. What I said that for case #2, you >>>>>>>> simply put templates into generators/. And of course you can have both >>>>>>>> ftl and json files inside generators/, each generating its output file >>>>>>>> in its own way. >>>>>>>> >>>>>>>> As of dataModel-s, the part of the content the json files is also part >>>>>>>> of the data-model, so it's not the best name. It's rather >>>>>>>> "shared-dataModel", and in case you specify that directly in the >>>>>>>> pom.xml, you don't even need that. And of course, we are talking about >>>>>>>> a layered data-model, where there's a shared data model that's visible >>>>>>>> from everywhere, and it the case of a json generator file it can adds >>>>>>>> another layer on top of that. >>>>>>>> >>>>>>>> > The issue with this more generic structure is that in exchange for >>>>>>>> > this >>>>>>>> > flexibility, you might have a bunch of extra files. For example, in >>>>>>>> > the >>>>>>>> > special case #2, one needs twice as many files in the above >>>>>>>> > structure. >>>>>>>> > >>>>>>>> > So instead of the above structure, the suggestion in the thread is to >>>>>>>> embed >>>>>>>> > both of the common use cases into the plugin in a way that they "play >>>>>>>> nice" >>>>>>>> > together. >>>>>>>> > Create a generators directory and have the plugin interpret how to >>>>>>>> generate >>>>>>>> > the source based on the type of thing in that directory. >>>>>>>> > >>>>>>>> > I think that we could instead ask how we might deviate from the above >>>>>>>> > generic structure that allows a many-to-many mapping while reusing >>>>>>>> > both >>>>>>>> > data models >>>>>>>> > and templates in such a way that the two special cases can be >>>>>>>> > naturally >>>>>>>> > supported. >>>>>>>> > >>>>>>>> > The first enhancement is that a generator could optionally embed the >>>>>>>> > data >>>>>>>> > model. This handles case #1 if you do not want >>>>>>>> > to reuse data models. It also naturally comes from enhancing the >>>>>>>> > json structure in the way that Daniel suggested: >>>>>>>> > >>>>>>>> > { >>>>>>>> > "templateName": "foo.ftl", >>>>>>>> > "otherFutureMavenPluginKey": "something", >>>>>>>> > "dataModel": { >>>>>>>> > /* your application-specific stuff */ >>>>>>>> > "myString": "a string", >>>>>>>> > "myNumber": 1, >>>>>>>> > ... >>>>>>>> > } >>>>>>>> > } >>>>>>>> > >>>>>>>> > Here the other option would be for "dataModel" to refer to a file in >>>>>>>> > the >>>>>>>> > "dataModel" directory. >>>>>>>> >>>>>>>> Sure that's a more generic solution. Though usually you don't want to >>>>>>>> use a json generator file if you don't want a data-model that's >>>>>>>> specific to that generator file. >>>>>>>> >>>>>>>> > I would say that instead of overloading the "generators" directory to >>>>>>>> also >>>>>>>> > include templates, >>>>>>>> >>>>>>>> To clarify, there are templates in the templates\ directory, and then >>>>>>>> there can be templates in the generator/ directory. They are >>>>>>>> completely separate matters. In the 2nd case, they are just a type of >>>>>>>> a generator files, like json-s are. >>>>>>>> >>>>>>>> > we allow the plugin to operate in an alternate mode, >>>>>>>> > enabled by configuration, say by setting a "defaultDataModel" >>>>>>>> > parameter. >>>>>>>> In >>>>>>>> > this mode, we generate sources for all templates in the template >>>>>>>> directory >>>>>>>> > from a common data model. >>>>>>>> >>>>>>>> My idea was that the we only ever generate output files for what's in >>>>>>>> the generator directory. There can be files both in generators/ and >>>>>>>> templates/, but hey have a different role. Templates in templates/ are >>>>>>>> just dependencies of some generators/ (the jsons, in this case), or >>>>>>>> maybe they can be #import-ed from templates in generators/. >>>>>>>> >>>>>>>> > In this use case, the project would only use >>>>>>>> > this structure only: >>>>>>>> > >>>>>>>> > /src/main/generated/freemarker/dataModel.xml >>>>>>>> >>>>>>>> NYW, I think for a dataModel it's important that it can come from >>>>>>>> various places, like even from a database. So we should keep that in >>>>>>>> mind. >>>>>>>> >>>>>>>> > /src/main/generated/freemarker/templates/... >>>>>>>> > >>>>>>>> > Having said all of this, I think the changes we would make in the >>>>>>>> > short >>>>>>>> > term would be to update the json structure to include the dataModel >>>>>>>> > as a >>>>>>>> > subtree, and to change the default locations to use the /generated >>>>>>>> > subdirectory rather than /data, and whatever suggestion is >>>>>>>> > appropriate. >>>>>>>> > >>>>>>>> > -Ben >>>>>>>> > >>>>>>>> > On Thu, Aug 16, 2018 at 4:18 PM, Ben Jackson <[email protected]> >>>>>>>> wrote: >>>>>>>> > >>>>>>>> >> Yes, this makes sense. >>>>>>>> >> >>>>>>>> >> -Ben >>>>>>>> >> >>>>>>>> >> On Thu, Aug 16, 2018 at 2:56 PM, Daniel Dekany <[email protected]> >>>>>>>> wrote: >>>>>>>> >> >>>>>>>> >>> Wednesday, August 15, 2018, 10:52:41 PM, Mark Malynn wrote: >>>>>>>> >>> >>>>>>>> >>> [snip] >>>>>>>> >>> > 3) I agree that using the well known json field 'templateName' >>>>>>>> >>> > seems >>>>>>>> >>> > 'hacky' (from a freemarker point of view). However, from the >>>>>>>> >>> > point of >>>>>>>> >>> view >>>>>>>> >>> > of the maven plugin it works nice. It provides the flexibility I >>>>>>>> >>> > need >>>>>>>> >>> with >>>>>>>> >>> > the minimum fuss. I will entertain another mechanism as long as >>>>>>>> >>> > it >>>>>>>> >>> supports >>>>>>>> >>> > my main use case. >>>>>>>> >>> [snip] >>>>>>>> >>> >>>>>>>> >>> The problem with that is that it doesn't work if you happen to >>>>>>>> >>> need a >>>>>>>> >>> "tempalteName" field in your actual data. Yes, that's unlikely, but >>>>>>>> >>> take it as a demonstration of why it's not correct. The "correct" >>>>>>>> solution >>>>>>>> >>> is like this: >>>>>>>> >>> >>>>>>>> >>> { >>>>>>>> >>> "templateName": "foo.ftl", >>>>>>>> >>> "otherFutureMavenPluginKey": "something", >>>>>>>> >>> "dataModel": { >>>>>>>> >>> /* your application-specific stuff */ >>>>>>>> >>> "myString": "a string", >>>>>>>> >>> "myNumber": 1, >>>>>>>> >>> ... >>>>>>>> >>> } >>>>>>>> >>> } >>>>>>>> >>> >>>>>>>> >>> Note how it allows adding new keys (like >>>>>>>> >>> "otherFutureMavenPluginKey") >>>>>>>> >>> without breaking backward compatibility. >>>>>>>> >>> >>>>>>>> >>> -- >>>>>>>> >>> Thanks, >>>>>>>> >>> Daniel Dekany >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> -- >>>>>>>> >> Benjamin Grant Jackson >>>>>>>> >> >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> >>>>>>>> -- >>>>>>>> Thanks, >>>>>>>> Daniel Dekany >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Thanks, >>>>>> Daniel Dekany >>>>>> >>>>> >>>> >>>> -- >>>> Thanks, >>>> Daniel Dekany >>>> >>> >> >> -- >> Thanks, >> Daniel Dekany >> > -- Thanks, Daniel Dekany
