Hi Daniel, ad exposing data sources in FTL ==================================================================
I tried that before in https://issues.apache.org/jira/browse/FREEMARKER-148 <https://issues.apache.org/jira/browse/FREEMARKER-148> and http://mail-archives.apache.org/mod_mbox/freemarker-dev/202006.mbox/%3c09810cda-5f60-4fb2-9d24-c00d764e7...@gmail.com%3e <http://mail-archives.apache.org/mod_mbox/freemarker-dev/202006.mbox/%3c09810cda-5f60-4fb2-9d24-c00d764e7...@gmail.com%3E> * The idea was to provide list and map semantic using at once (exposing a beefed up "DataSources") but I ran into shadowing issues I could not resolve * The outcome was reverting the changes and move to a map requiring a name for each data source So yes, if it would be possible to expose a "sequence + hashtype value" that would be awesome * Any links how to implement such a wrapper? * At hindsight I think wrapping the "org.apache.freemarker.generator.base.datasource.DataSources" was the root cause of my problems since it exposes a lot of mwmbwe functions Thanks in advance, Siegfried Goeschl > On 01.10.2021, at 08:21, Daniel Dekany <daniel.dek...@gmail.com> wrote: > > I don't see how exposing DataSources in itself is sufficient. Because, > in the template you primarily just need a list of non-named data > sources, and then a map of named data sources. One only contains the > data sources passed by position, and the other only contains the data > sources passed by name. If we agree on that, then the next question is > naming. It could be dataSources[indexExpression], and > namedDataSource.someName (namedDataSources[nameExpression]). However, > it is probably worth it to pull some more TemplateModel tricks here, > and make dataSources a sequence+hash type value, so we can get away > with a single variable. Then you can write > dataSources[indexExpression], dataSources.someName, and > dataSources[nameExpression]. To access advanced functions, like the > find method, you may expose the DataSources object as is, under a > different name. Like tools.dataSources, or such. > > As for what to do when you have exactly one data source, yes, there > could be a dataSource variable (singular) that only works when you > have exactly one positionally passed data source. However, for > usability reasons, that variable should always exist, so if somebody > tries to use it when they shouldn't, it can give a helpful error > message. > > .dataSource: For that it had to be part of the template language, so > although our unique position makes that achievable, probably not. > Especially as it doesn't really solve the problem, since you have > other variables as well. Like "tools". The best we can do is to keep > the number of such variables low. > > One of my statements was that using URL-s (or filenames) as keys is > not useful for the template author. If the user didn't specify a name > for the data source, then they shouldn't be able to access it by name > either, as there's no stable name to use. What about that? > > On Thu, Sep 30, 2021 at 10:43 PM Siegfried Goeschl > <siegfried.goes...@gmail.com> wrote: >> >> Hi Daniel, >> >> To recap the usage of DataSources ... >> >> * A template does not use any data source, e.g. copying & expanding based on >> environment variables >> * A template uses exactly one data source, e.g. some CSV to HTML >> transformation >> * A template can use one or more named data sources, e.g. >> "users=https://some.url <https://some.url/>" >> * A template aggregating a list of unnamed data sources, e.g parsing and >> extracting stuff from access logs >> >> Over the time I implemented various approaches >> >> * Having a plain list of data sources >> * Wrapping the data sources in custom holder - see >> https://github.com/apache/freemarker-generator/blob/master/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/datasource/DataSources.java >> >> <https://github.com/apache/freemarker-generator/blob/master/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/datasource/DataSources.java> >> * Later one we migrated to a plain map forcing requiring unique keys (so I >> am using HTTP URIs here) >> >> So what about the following approach (you suggested something like this some >> time ago) >> >> * Exposing a "dataSource" when exactly one data source is provided >> * Exposing the org.apache.freemarker.generator.base.datasource.DataSources >> as "dataSources" again? It is not super nice in FreeMarker terms but >> provides convince methods to slice through data sources >> >> Related question - to avoid name clashes with user variables - is it >> possible to use reserved names as well >> >> * ${.dataSource} >> * ${.dataSources} >> >> What do you think?! >> >> Siegfried >> >>> On 28.09.2021, at 13:01, Daniel Dekany <daniel.dek...@gmail.com> wrote: >>> >>> Yeah, I did not answer that, but I did check the feature. It allows >>> using the project in more use-cases, so it's definitely an >>> improvement. >>> >>> That said, my main pain point remains the way data sources are handled >>> (that's regardless of this feature). >>> Like having dataSources?values[0] all over the place is not nice, >>> given it's quite central feature of the project. And that thing is >>> like that because dataSources is a Map where the key is the source >>> URL. But when would someone want to get something by URL? And if >>> somebody has that idea, I would be against it in most cases, as it's >>> fragile (like I would totally pass in a file with a different path or >>> name, believing that doesn't matter). Instead, I think: >>> - dataSources should be just a list of objects. (From that object you >>> can then get file name etc., though normally you shouldn't care about >>> it.) It should only contain the data sources for which no name was >>> explicitly specified by the user. >>> - There should be a separate namedDataSources, which only contains the >>> data sources that the user has explicitly given a name >>> >>> I remember this was a topic back then. I don't know, maybe revisit it. >>> I think it would be much less confusing like above. >>> >>> >>> On Sat, Sep 11, 2021 at 3:55 PM Siegfried Goeschl >>> <siegfried.goes...@gmail.com> wrote: >>>> >>>> Worx now - thx >>>> >>>> Thanks in advance, >>>> >>>> Siegfried Goeschl >>>> >>>> >>>>> On 11.09.2021, at 11:38, Daniel Dekany <daniel.dek...@gmail.com> wrote: >>>>> >>>>> I hopefully fixed that build issue now. >>>>> >>>>> On Sat, Sep 11, 2021 at 9:20 AM Siegfried Goeschl < >>>>> siegfried.goes...@gmail.com> wrote: >>>>> >>>>>> Hi Daniel, >>>>>> >>>>>> Had a quick look at FREEMARKER-154 but document generation fails for me >>>>>> but I have no time to look into it >>>>>> >>>>>> [INFO] --- freemarker-docgen-maven:0.0.2-SNAPSHOT:transform >>>>>> (docgen-transform) @ freemarker-generator-website --- >>>>>> Using FreeMarker 2.3.31 >>>>>> Loading >>>>>> /Users/sgoeschl/work/github/apache/freemarker-generator/freemarker-generator-website/src/main/docgen/book.xml... >>>>>> Output directory: >>>>>> /Users/sgoeschl/work/github/apache/freemarker-generator/freemarker-generator-website/target/website >>>>>> [INFO] >>>>>> ------------------------------------------------------------------------ >>>>>> [INFO] BUILD FAILURE >>>>>> [INFO] >>>>>> ------------------------------------------------------------------------ >>>>>> [INFO] Total time: 2.237 s >>>>>> [INFO] Finished at: 2021-09-11T08:53:17+02:00 >>>>>> [INFO] >>>>>> ------------------------------------------------------------------------ >>>>>> [ERROR] Failed to execute goal >>>>>> org.apache.freemarker.docgen:freemarker-docgen-maven:0.0.2-SNAPSHOT:transform >>>>>> (docgen-transform) on project freemarker-generator-website: Error during >>>>>> document transformation: Insertable file with symbolic name >>>>>> "websitePomGenerated" points to a directory that doesn't exist: >>>>>> "/Users/sgoeschl/work/github/apache/freemarker-generator/freemarker-generator-website/target/docgen-insertable-outputs" >>>>>> -> [Help 1] >>>>>> >>>>>> >>>>>> And XMLMind looks quite nice to use ... >>>>>> >>>>>> Skimmed through comments regarding to documentation >>>>>> >>>>>> * I used the examples in the documentation as mental notes so many of >>>>>> them might be irrelevant and should be removed altogether >>>>>> * And yes, a lot of stuff needs to be rewritten .-) >>>>>> * Gomplate documentation is under MIT licence >>>>>> * Since I don't have the final output - how annoying are all the ASL >>>>>> headers in the sample file? Guess they add a lot of noise? >>>>>> >>>>>> So I suggest to get FREEMARKER-188 out of the door since this is the >>>>>> biggest pain point regarding initial release and move the documentation >>>>>> step by step - what do you think? >>>>>> >>>>>> Thanks in advance, >>>>>> >>>>>> Siegfried Goeschl >>>>>> >>>>>> >>>>>> >>>>>>> On 09.09.2021, at 21:06, Daniel Dekany <daniel.dek...@gmail.com> wrote: >>>>>>> >>>>>>> Maybe it's already too late, but I will... hopefully at the weekend. >>>>>>> >>>>>>> Oh, and the documentation conversion too... what was completed will drag >>>>>>> behind the md-s again. /-: Sorry about that. Have you ever tried using >>>>>> what >>>>>>> we have in that branch? It can save time and improve the quality of the >>>>>>> content, as it inserts the actual example files, runs them, and inserts >>>>>> the >>>>>>> actual output. No copy-pasting. So it's guaranteed that what we tell >>>>>>> is accurate and up to date. XXE is another time saver for me... though >>>>>>> admittedly it needs initial investment to use. (Don't even touch XML >>>>>>> editors where you edit on source level... that's clearly not for >>>>>>> documentation.) >>>>>>> >>>>>>> On Tue, Sep 7, 2021 at 9:38 PM Siegfried Goeschl < >>>>>>> siegfried.goes...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi folks, >>>>>>>> >>>>>>>> After Corona, lockdown, home schooling and endless telcos and finally >>>>>>>> returned to my pet project :) >>>>>>>> >>>>>>>> I'm tackling now the age-old topic of generating output files driven by >>>>>>>> data sources - https://issues.apache.org/jira/browse/FREEMARKER-188 < >>>>>>>> https://issues.apache.org/jira/browse/FREEMARKER-188> >>>>>>>> >>>>>>>> Daniel, since we invested a lot of energy discussing the topic you are >>>>>>>> happily invited to share your thought :) >>>>>>>> >>>>>>>> Thanks in advance, >>>>>>>> >>>>>>>> Siegfried Goeschl >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Best regards, >>>>>>> Daniel Dekany >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> Daniel Dekany >>>> >>> >>> >>> -- >>> Best regards, >>> Daniel Dekany >> > > > -- > Best regards, > Daniel Dekany
