These optimisations sound great. I will try to run some tests within the next weeks. A bit busy lately. Thanks Christoph
Am Mi., 29. Mai 2019 um 23:55 Uhr schrieb Daniel Dekany <[email protected] >: > Tuesday, April 2, 2019, 12:10:16 PM, Christoph Rüger wrote: > > [snip] > >> Well, if you fear users jumping on ?filter/?map outside #list for no > >> good enough reason, there can be some option to handle that. But I > >> don't think restricting the usage to #list is a good compromise as the > >> default. > > > > I agree. Just keep as it is. > > > >> >> I'm not sure how efficiently could a configuration setting catch > these > >> >> cases, or if it should be addressed on that level. > >> > > >> > Maybe let's postpone configurability discussion a bit until the above > is > >> > more clear. > >> > >> In the light of the above, I think we can start thinking about that > >> now. > > > > On that note on configurability: Would it be possible to programmatically > > influence the Collection (Sequence) which is created under the hood? > > E.g. by specifying a Factory? I ask because we are using something like > > this ( > > > https://dzone.com/articles/a-filebasedcollection-in-java-for-big-collections > ) > > in other places for large collections. I know it is very specific, but > just > > wanted to bring it up. > [snip] > > I think a good approach would be to ban the *implicit* collection of > the result, when the filtered/mapped source is an Iterator, or other > similar stream-like object that's often used for enumerating a huge > number of elements. So for example, let's say you have this: > > <#assign xs2 = xs?filter(f)> > > If xs is List-like, then this will work. Since the xs List fits into > the memory (although a List can be backed by disk, that's rather > rare), hopefully it's not the kind of data amount that can't fit into > the memory again (as xs2). On the other hand, if xs is an > Iterator-like object, then the above statement fails, with the hint > that xs?filter(f)?sequence would work, but might consumes a lot of > memory. > > This is also consistent with how xs[i] works in the existing > FreeMarker versions. That only works if xs is List-like (an FTL > sequence). While xs[i] would be trivial to implement even if xs is > Iterator-like, we don't do that as it's not efficient for a high i, > and so the template author is probably not meant to do that. If he > knows what's he doing though, he can write xs?sequence[i]. Yes, that's > very inefficient if you only use [] once on that sequence, but you see > the logic. map/filter breaks it, as xs?filter(f)[i] works even if xs > is an Iterator, because filter/map currently always returns a > sequence. If xs is Iteartor-like, then I want filter/map to return an > Iterator-like as well, so then [] will fail on it. > > As a side note, I will make ?sequence smarter too, so that > xs?sequence[i] won't actually build a sequence if xs is Iterator-like. > It just have to skip the first i elements after all. (The ?sequence is > still required there. It basically says: "I know what I'm doing, treat > this as a sequence.") > > -- > Thanks, > Daniel Dekany > > -- Synesty GmbH Moritz-von-Rohr-Str. 1a 07745 Jena Tel.: +49 3641 5596493Internet: https://synesty.com <https://synesty.com> Informationen zum Datenschutz: https://synesty.com/datenschutz <https://synesty.com/datenschutz> Geschäftsführer: Christoph Rüger Unternehmenssitz: Jena Handelsregister B beim Amtsgericht: Jena Handelsregister-Nummer: HRB 508766 Ust-IdNr.: DE287564982
