Wow, it's amazing! I'd spend some time testing it. Thanks a lot!
Woonsan On Sat, Feb 23, 2019 at 12:19 PM Daniel Dekany <[email protected]> wrote: > > I have pushed an implementation of the restricted lambdas we were > talking about. Guys, please review/test it. > > I call these "local lambdas" in the source code (but I'm open for > suggestions), as the function they define can only be called in the > same variable scope where the lambda was (as we have no closures in > the template language, nor final variables). Also, they can only be > used as the parameters of the built-ins that explicitly support them. > As the subject shows, the main goal was just support filtering for > #list, without a nested #if, as that breaks #sep, #else, it?hasNext, > it?index, etc. (I will try to make more universal lambdas in FM3... I > guess it would be way too tricky in FM2.) > > For now, I have only added two built-ins that support lambdas: ?filter > and ?map. Any ideas what other such built-ins would be often useful in > templates (with use case, if possible)? > > Examples of using ?filter and ?map: > > <#list products?filter(it -> it.price < 1000) as product> > ${product.name} > </#list> > > <#list products?map(it -> it.name) as name> > ${name} > </#list> > > Of course these built-ins aren't specific to #list, they can be used > anywhere. Naturally, they can be chained as well: > > <#assign chepProdNames = products > ?filter(it -> it.price < 1000) > ?map(it -> it.name) > > > > As a side note, ?filter and ?map also accepts FTL function-s and Java > methods as its parameter, not only lambdas. > > A tricky aspect of this feature is lazy evaluation, in similar sense > as Java 8 Stream intermediate operations are lazy. On most places, > ?filer and ?map are eager, that is, they return a completed sequence > of items. That's because our restricted lambdas only work correctly > "locally". However, there are very common situations where it's clear > that we can use lazy ("streaming") evaluation, as we know where the > resulting stream of elements will be consumed: > > - One such case is when these kind of built-ins are chained. Like in > the last example, ?filter doesn't construct a List in memory, > instead the elements just flow through it (some is dropped, as it's > filtering), into ?map. Only the ?map at the end of the chain will > built a List eagerly. Some other built-ins also allow the left-hand > built-in to stream, like in ?filter(...)?map(...)?join(", ") no List > is built anywhere, instead the elements flow through both ?filter > and ?map, and ?join just appends them to the StringBuilder where it > creates its results. > > - #list also enables lazy evaluation to its 1st parameter. So in the > #list examples above, yet again no List is built in memory. > > - There are other such cases, which I didn't implement yet. For > example the sequence slice operator, like, xs?filter(f)[10..20], > should allow lazy evaluation. > > Feedback is welcome! > > > Monday, December 17, 2018, 11:39:36 AM, Christoph Rüger wrote: > > > Hey Daniel, > > I'm very sorry, but I didn't make any progress with this. I think I was a > > bit over-motivated, but unfortunately I cannot spend more time on this for > > various reasons. > > You can take this over. I'm glad to help out with testing and feedback. > > > > Thanks > > Christoph > > > > > > > > > > Am Mo., 17. Dez. 2018 um 11:04 Uhr schrieb Daniel Dekany <[email protected] > >>: > > > >> Any progress in this? I think I will give it a try in the coming days > >> otherwise. > >> > >> > >> Sunday, November 18, 2018, 10:31:29 PM, Daniel Dekany wrote: > >> > >> > See my answers inline... > >> > > >> > Sunday, November 18, 2018, 8:44:40 PM, Christoph Rüger wrote: > >> > > >> >> Thanks Daniel for your feedback. See my answers below > >> >> > >> >> Am So., 11. Nov. 2018 um 19:14 Uhr schrieb Daniel Dekany < > >> [email protected] > >> >>>: > >> >> > >> >>> Sunday, November 11, 2018, 11:40:50 AM, Christoph Rüger wrote: > >> >>> > >> >>> > Am So., 11. Nov. 2018 um 09:25 Uhr schrieb Daniel Dekany < > >> >>> [email protected] > >> >>> >>: > >> >>> > > >> >>> >> Saturday, November 10, 2018, 3:08:14 PM, Denis Bredelet wrote: > >> >>> >> > >> >>> >> > Hi, > >> >>> >> > > >> >>> >> > Le 9 novembre 2018 à 22:36, Christoph Rüger <[email protected]> > >> a > >> >>> >> écrit : > >> >>> >> > > >> >>> >> > Am Fr., 9. Nov. 2018 um 22:55 Uhr schrieb Daniel Dekany < > >> >>> >> [email protected] > >> >>> >> > : > >> >>> >> > > >> >>> >> > It's certainly tricky, but as far as I see possible (but then, who > >> >>> >> > > >> >>> >> > knows what will one find when actually working on it). It's also a > >> >>> >> > feature missing a lot. It's especially missing for #list (I know > >> that > >> >>> >> > you need it for something else), because if you filter the items > >> >>> >> > inside #list with #if-s, then #sep, ?hasNext, etc. will not be > >> usable. > >> >>> >> > > >> >>> >> > Let me say that I disagree here. > >> >>> >> > > >> >>> >> > I do not think that closures are required for FreeMarker, nor that > >> >>> they > >> >>> >> are a good idea. > >> >>> >> > > >> >>> >> > If we add new features to the FreeMarker *tempate engine* I would > >> >>> >> > rather we focus on multi-part macro body rather than an advanced > >> >>> >> language feature like closures. > >> >>> >> > > >> >>> >> > You can add ?filter and ?map if you want, a simple expression as > >> >>> >> parameter should be enough. > >> >>> >> > >> >>> >> Yes, as I said, we certainly start with only allowing lambdas in > >> >>> >> ?filter/?map, also certainly in ?contains. > >> >>> >> > >> >>> > Would be enough in my opinion and very useful. > >> >>> > > >> >>> > Is it possiblefor you to give some pointers to the code on how this > >> could > >> >>> > be implemented? I would maybe like to wrap my head around this a > >> little > >> >>> bit. > >> >>> > >> >>> Please feel yourself encouraged! (: > >> >>> > >> >>> > I started looking at seq_containsBI ( > >> >>> > > >> >>> > >> https://github.com/apache/freemarker/blob/a03a1473b65d9819674b285a0538fed824f37478/src/main/java/freemarker/core/BuiltInsForSequences.java#L291 > >> >>> ) > >> >>> > and > >> >>> > and reverseBI ( > >> >>> > > >> >>> > >> https://github.com/apache/freemarker/blob/a03a1473b65d9819674b285a0538fed824f37478/src/main/java/freemarker/core/BuiltInsForSequences.java#L264 > >> >>> ) > >> >>> > just to find something related (seq_containsBI checks something) and > >> >>> > reverseBI returns a new sequence. > >> >>> > What I haven't found is a function which takes an Expression as a > >> >>> > parameter. > >> >>> > Is there something similar already or would that be a new thing? > >> >>> > >> >>> It's a new thing in that it will be part of the expression syntax > >> >>> (even if for now we will only allow lambdas as the parameters of a few > >> >>> built-ins, so that we can get away without closures). So it's a new > >> >>> Expression subclass, and has to be part of the parser (ftl.jj) as > >> >>> well. > >> >> > >> >> Hmm, that parser stuff is new for me, it'll take me some time to get > >> into > >> >> it. > >> > > >> > So it's a JavaCC lexer+parser. With a few twists... sorry, but this > >> > code has history... :) > >> > > >> >>> As of lazy evaluation of parameters expressions, that's already > >> >>> done in the built-ins in BuiltInsWithParseTimeParameters, and you will > >> >>> see it's trivial to do, but the situation there is much simpler. > >> >>> > >> >> > >> >> > >> >>> > >> >>> In principle, a LambdaExpression should evaluate to a > >> >>> TemplateMethodModelEx, and then you pass that TemplateMethodModelEx to > >> >>> the called built-in or whatever it is. But with the approach of > >> >>> BuiltInsWithParseTimeParameters we can certainly even skip that, and > >> >>> just bind to the LambdaExpression directly, add a LocalContext that > >> >>> contains the lambda arguments, end evaluate the LambdaExpression right > >> >>> there, in the built-in implementation. Or at least at a very quick > >> >>> glance I think so. > >> >>> > >> >> Not sure I can follow completely but that hint with* > >> >> BuiltInsWithParseTimeParameters* got me started, but at the moment I'm > >> >> stuck as I need to get more familiar with the internals of Freemarker. > >> I am > >> >> also not sure I am on the same page regarding the syntax we are aiming > >> for > >> >> and why I would need to extend the parser when there is something like > >> >> BuiltInsWithParseTimeParameters.... > >> > > >> > I assumed that the syntax will be similar to the Java lambda syntax > >> > (see below why), and that's of course needs lexer/parser changes. > >> > > >> >> Here is an example what I have in mind: > >> >> I started with the ?filter() builtin. I had a syntax like this in mind: > >> >> > >> >> *Example 1: ["a","b","c"]?filter(element, element == "c")* > >> >> *Example 2: ["a","b","c"]?filter(element, element == someOtherVariable, > >> >> someOtherVariable="c")* > >> > > >> > Looking at the above one believe that the value of `element` is passed > >> > in as first argument, and the the value of `element == "c"` as the > >> > second, but that's not the case. It's much better if it's visible that > >> > you got some kind of anonymous function definition there without > >> > knowing about the "filter" built-in. > >> > > >> > Java already has a syntax for expressing this kind of thing, so it > >> > would be better to use that familiar syntax. As far as I know it > >> > doesn't conflict with ours (it kind of it does, but not fatally). > >> > > >> > Also, even if for now we only allow this in said built-ins, we > >> > shouldn't exclude the possibility of making this kind of expression > >> > accessible elsewhere as well. Then, on many places the parsed won't > >> > know what kind of value is expected (consider passing a lambda to an > >> > user defined directive for example), so a syntax like above won't > >> > work. > >> > > >> >> Not sure if that's what you have in mind too, but to me it made sense > >> with > >> >> regards to BuiltInsWithParseTimeParameters and I could start without > >> >> touching parser stuff. > >> > > >> > BuiltInsWithParseTimeParameters doesn't affect the syntax (only a > >> > bit...), as a call to a such built-in looks like a call to any other > >> > built-in. > >> > > >> >> *1st argument 'element'* would just be the iterator variable similar to > >> >> <#list ["a","b","c"] as *element*> > >> >> 2nd argument is the filter lambda expression... aka our filter condition > >> >> 3rd+n argument are optional parameters in case used in the lambda > >> expression > >> > > >> > #list is special, similarly as `for ... in ...` is in most languages. > >> > It's not lambda-ish either; you can't write `as element/2` or such. > >> > > >> >> So at first I was looking how <#list> works and found IteratorBlock, and > >> >> though I could reuse it somehow. > >> >> > >> >> Here is some simple pseudo code I played around for the for Example 1: > >> >> > >> >> static class filter_BI extends BuiltInWithParseTimeParameters { > >> >> > >> >> > >> >> > >> >> TemplateModel _eval(Environment env) throws TemplateException { > >> >> > >> >> // sequence > >> >> > >> >> TemplateModel targetValue = target.evalToNonMissing(env); > >> >> > >> >> > >> >> > >> >> List parameters = this.parameters; > >> >> > >> >> > >> >> > >> >> Expression iteratorAlias = (Expression) parameters.get(0); > >> >> > >> >> Expression conditionExpression = (Expression) > >> parameters.get(1); > >> >> > >> >> > >> >> > >> >> TemplateSequenceModel seq = (TemplateSequenceModel) > >> target.eval > >> >> (env); > >> >> > >> >> for (int i = 0; i < seq.size(); i++) { > >> >> > >> >> TemplateModel cur = seq.get(i); > >> >> > >> >> > >> >> // this is where I am stuck at the moment > >> >> > >> >> // I basically want to evaluate conditionExpression > >> >> where iteratorAlias > >> >> is basically what I passed as 'element' > >> >> > >> >> // I am not sure if or how LocalContext could come into > >> play > >> >> here > >> >> > >> >> // basically for each iteration I would assign the > >> current > >> >> loop element to a context variable with the name 'element' > >> >> > >> >> // and then evaluate conditionExpression with that > >> context. > >> >> > >> >> // if conditionExpression is "true" then I would populate > >> >> add the current sequence element 'cur' > >> >> > >> >> // to a new result-List.... and return that.... something > >> >> > >> >> // I wanted to reuse IteratorBlock here somehow, but > >> didn't > >> >> get it to work yet. > >> >> > >> >> // maybe this is a stupid idea, or we just need something > >> >> similar > >> >> > >> >> > >> >> > >> >> } > >> >> > >> >> } > >> >> > >> >> > >> >> > >> >> Ok so far for my pseudo code.... Maybe you could give some more > >> pointers > >> >> based on that... in case this makes any sense ... > >> > > >> > For LocalContext, Environment has a pushLocalContext method. See the > >> > calls to it, like in visit(TemplateElement[], TemplateDirectiveModel, > >> > Map, List). > >> > > >> > To avoid running into more new things at once than necessary, perhaps > >> > you should start with seq?seq_contains(lambda). The end result should > >> > be like: > >> > > >> > users?contains(u -> u.paying) > >> > > >> >>> Another similarity to BuiltInsWithParseTimeParameters is that we won't > >> >>> allow separating the `?someBI` and `(arg)`. Like, with the example of > >> >>> `cond?then(1, 2)`, you aren't allowed to do this: > >> >>> > >> >>> <#assign t=cond?then> > >> >>> ${t(1, 2)} > >> >>> > >> >>> That maybe looks natural, but most other built-ins allow that. Of > >> >>> course we can't allow that for ?filter etc., because then we need > >> >>> closures again. > >> >>> > >> >>> >> Multi-part macro body is also planned. Means, I know it definitely > >> >>> >> should be added, but who knows when that's done... I mean, it's like > >> >>> >> that for what, a decade? (: It's not even decided what it exactly > >> >>> >> does, as there are many ways of approaching this. (I have my own > >> idea > >> >>> >> about what the right compromise would be, but others has other > >> >>> >> ideas...) > >> >>> >> > >> >>> >> Filtering lists bothers me because the template language should be > >> >>> >> (and somewhat indeed is) specialized on listing things on fancy ways > >> >>> >> that used to come up when generating document-like output. (If it > >> >>> >> doesn't do things like that, you might as well use a general purpose > >> >>> >> language.) Thus, that filter is unsolved (filtering with #if is > >> >>> >> verbose and spoils #sep etc.) bothers me a lot. > >> >>> >> > >> >>> >> BTW, ?filter and ?map is also especially handy in our case as > >> >>> >> FreeMarker doesn't support building new sequences (sequences are > >> >>> >> immutable). Although it has sequence concatenation with `+`, it's > >> not > >> >>> >> good for building a sequence one by one, unless the sequence will be > >> >>> >> quite short. > >> >>> >> > >> >>> > Good point. > >> >>> > > >> >>> > > >> >>> >> > >> >>> >> > Cheers, > >> >>> >> > -- Denis. > >> >>> >> > >> >>> >> -- > >> >>> >> Thanks, > >> >>> >> Daniel Dekany > >> >>> >> > >> >>> >> > >> >>> > > >> >>> > >> >>> -- > >> >>> Thanks, > >> >>> Daniel Dekany > >> >>> > >> >>> > >> >> Thanks > >> >> Christoph > >> >> > >> > > >> > >> -- > >> Thanks, > >> Daniel Dekany > >> > >> > > > > -- > > Christoph Rüger, Geschäftsführer > > Synesty <https://synesty.com/> - Anbinden und Automatisieren ohne > > Programmieren - Automatisierung, Schnittstellen, Datenfeeds > > > > Xing: https://www.xing.com/profile/Christoph_Rueger2 > > LinkedIn: http://www.linkedin.com/pub/christoph-rueger/a/685/198 > > > > -- > Thanks, > Daniel Dekany >
