Tuesday, June 5, 2018, 3:28:45 PM, Woonsan Ko wrote: > On Mon, Jun 4, 2018 at 5:04 AM, Daniel Dekany <[email protected]> wrote: >> Sunday, June 3, 2018, 10:13:04 PM, David E Jones wrote: >> >>> You make good points Daniel. At a high level it reminds me of a discussion >>> with a new client a couple months ago on the topic of managing templates >>> for ecommerce sites. They want to use the Liquid template engine partly >>> because they are doing custom ecommerce for some customers and Shopify >>> (which uses Liquid, where Liquid came from) for others, and also because >>> Liquid is more constrained and easier for web developers/designers to use. >>> >>> There is a case for template languages that are less developer oriented and >>> more designer oriented, or maybe better put because visual web designers >>> have no business mucking around with templates server side developer vs >>> web/browser developer (even if this is stuff that generally runs on a >>> server anyway). >>> >>> Right now, ie FM2, FreeMarker strikes a good balance between these (IMO >>> anyway), and the built-ins are an important part of that. Maybe they don't >>> do every little thing everyone in the world might want but they cover a lot >>> of basics and common needs. >>> >>> At a lower level you are correct about template parsing and supporting >>> arbitrary languages for inline expressions. The template parser still needs >>> to be able to figure out where the expression starts and stops. This might >>> not be too bad for C-like languages which current FM expressions are (more >>> or less, mostly the same demarcation characters and all), but is still a >>> challenge and could require something like surrounding expressions with >>> parenthesis (along with nested parenthesis evaluation that FM already does) >>> and that's a little bit ugly. >> >> I wasn't even worried about that yet, but the semantical differences. >> The expression and non-expression language (if they are separate at >> all) build on a common mental model, so you can't, in general, cut out >> the expression language from one language, and put it into another. So >> I can only look at this on a case-by-case basis. >> >> As of the syntactical problems, at least if I write the expression >> parser, I can just exit expression mode if the next token would lead >> to an error, and let the "outer language" process the next character. >> Like you have `a + b n=2`, then `n` would be an error, and the outer >> parser will pick it up and recognize that as a parameter assignment. >> What's tricky is making JavaCC to do this... it doesn't support >> dynamic composing by design. ANTLR, I don't know (nor if it's fast >> enough for use cases like a lot of ?eval/?interpret). For a hand >> written parser, it's certainly not a problem. Though switching to hand >> written is a questionable step, even if I personally find it tempting. >> >>> Anyway, sorry to derail the discussion. >>> >>> On the truncate built-in an optional String type second argument makes >>> sense as a suffix to use if the string is truncated, ie that seems like an >>> easy thing to accommodate without reducing flexibility or having to write >>> that logic to do that inline in the template. >> >> There's also the question of where to cut the text. There's the >> maximum length given as parameter (actually, you subtract the length >> of the cut marker suffix), however if with applying that naively you >> end up with white-space at the end, you probably want to trim that. >> Also certainly if it ends with a ellipsis, dot or a sequence of dots, >> and maybe even if with some other punctuation. There's also the >> typical preference, over a certain maximum length, where you only cut >> at the end of words (unless you have some extremely long word). > > Commons-text's WordUtils#abbreviate(str, lower, upper, appendToEnd) > finds a space from the lower index to find where to cut. > It doesn't care other white spaces nor ending dot(s), but one of > existing simple algorithms at the moment. ;-) > >> >>> On title casing it might be best to defer to what Java does (ie >>> Character.toUpperCase() or .toTitleCase()), or were there issues with that? >> >> They don't convert text to title case, only a single character. In the >> most basic form of title case conversion of text, you convert the >> first letter of each word to title case. You may already have some >> tricky parts there with finding word boundaries. Then, in many >> languages, like in English, you aren't supported to convert words like >> "and", "of", etc. So it's obviously locale dependent. > > WordUtils#capitalizeFully() does. It doesn't recognize "and", "of", etc. > though.
Both WordUtils methods seem to be too simplistic for real world usage, to me at least. The default implementation should be actually usable for at least English content, and beat typical quick in-house solutions in quality. It seems that that's doable (see Jira issue). I guess the other languages will be left for the users (the algorithm is pluggable per language). As far as I know the most common language used for web content is still English, by far. So I guess it's the language FM is used for the most as well. It's quite weird as English isn't the most widely spoken language among the people who use the Internet. But I'm afraid if we pick, let's say, the next 4 most common web content languages, that can be be more offending for those left out, than if we support English only, which people more or less have get used to. > But as you said, if it can be customized through Configuration for > locale specific use cases, I'd say it's perfect. > > Regards, > > Woonsan > >> >>> -David >>> >>> >>> On Thu, May 31, 2018 at 2:03 AM, Daniel Dekany <[email protected]> wrote: >>> >>>> Thursday, May 31, 2018, 1:16:26 AM, David E Jones wrote: >>>> >>>> > My general opinion on this sort of thing is there are lots of libraries >>>> and >>>> > tools around in the Java world for doing string manipulations and such, >>>> > they don't need to be built into a template language. Taking that one >>>> step >>>> > further, I'd actually prefer that the expression syntax in the template >>>> > language makes it easy to call an existing API vs trying to make >>>> everything >>>> > built into the template language. >>>> > >>>> > For example in addition to using FTL I also use Groovy heavily and Groovy >>>> > already has a great API including standard Java API augmentation for all >>>> > sorts of things. Yes, FTL could use the Groovy String and other expanded >>>> > objects as a reference for built in design and have a good set of >>>> commonly >>>> > needed things, but why not just defer to Groovy for those that use it or >>>> > whatever other API/library anyone might prefer. >>>> > >>>> > I know this is a VERY different direction from FM2. What I find most >>>> useful >>>> > in FTL is features like macros and the extensive directives and other >>>> > things that make a template language flexible and powerful. The built-ins >>>> > are handy sometimes, but often don't do what I need anyway so other than >>>> > things like ?has_content I don't use them a lot. >>>> >>>> An important (and as far as I'm concerned, the main) reason for many >>>> built-ins is/was that frameworks and other products drop in FreeMarker >>>> for templates and then often don't give much attention to give tools >>>> to the template authors. Then the poor people who actually has to >>>> create templates come and ask, "I must do X... how can I do that in >>>> FreeMaker?". The less often we have to say "Please pester the >>>> developers (who are often from another company, or are on another >>>> sub-project by now => you have near 0 chance) to expose such >>>> functionality to the templates", the better. Instead we can say "Like >>>> foo?x", and furthermore, ?x is documented, and people can actually >>>> find its documentation, even years later. Surely if you have some >>>> small company with the sharp people, this is not a problem for you. >>>> Actually, then you very often don't want the one-size-fits-all >>>> solutions anyway. >>>> >>>> > I've mentioned this before but what would be a killer feature for FTL is >>>> to >>>> > support pluggable expression evaluators. The FTL expression syntax isn't >>>> > awful >>>> >>>> (Yes it is... but that's besides the point until I have fixed it in >>>> FM3.) >>>> >>>> > but is not nearly as convenient as Groovy, especially when you are >>>> > using it extensively elsewhere. >>>> >>>> The theoretical problem is of course that both languages (the "outer" >>>> template language and the language from where you rip off the >>>> expression language from) have its own ideas, and so there will be >>>> mismatches. The expression language in a language and other parts of >>>> the language aren't separate in general anyway. Which leads to the >>>> question that if you are using Groovy expressions, why not also use >>>> for loops and whatnot from Groovy? (This is fairly common way of >>>> creating template language, just taking an existing language, and add >>>> some convenience syntax to embed it into the static text.) So then we >>>> don't need our own #list, #if, etc. implementation either. But then, >>>> the original idea was that a template language is a domain specific >>>> language, and as such can have more specialized, say, loop constructs >>>> for that domain. (I'm not saying that FM2 did great there, but there >>>> was a lack of determination there as well.) If that's not the case >>>> though... >>>> >>>> Anyway, when and if FM3 will be more mature, we can see if how doable >>>> this is in reality. It's also a question if we are talking about a >>>> syntax, that in 99% looks like Groovy, or we are talking about actual >>>> Groovy. In the last case TemplateModel-s and the particular way null >>>> handling will work will be an obstacle for sure. Not to mention >>>> TemplateMarkupOutputModel-s, that AFAIK don't even have commonly used >>>> equivalent (like a class in the standard Java API). >>>> >>>> > If the default FTL expressions were not used you couldn't use >>>> > built-ins and instead would just have expressions that are expected >>>> > to evaluate to a boolean, string, or arbitrary object and can be >>>> > whatever. >>>> >>>> At least in FM3 built-ins will be just functions you have defined in >>>> the "core" FTL namespace (whose content is specified in template >>>> language Dialect you use) that's "statically linked" to the templates >>>> (allowing some parse-time checks and somewhat faster calls). So >>>> `foo?bar(1)` is basically just a shorthand syntax for `<#import core = >>>> 'freemarker://dialect-or-something'>` and `core.bar(foo, 1)`. There's >>>> a standard/default Dialect of course, which has the big advantage that >>>> it's documented in the official Manual, is used in SO answers, etc. >>>> >>>> > How important is this? Important enough that I've >>>> > considered using a different template language. If FTL wasn't so >>>> > darn convenience for XML transformation using macros I'd probably be >>>> > using Groovy GString templates instead, and just for the far better >>>> > syntax in Groovy. >>>> > >>>> > This also goes back to the object wrapping discussion which is another >>>> can >>>> > of worms... >>>> >>>> Yeah, it's one of the things that I really wanted to get rid of, but >>>> it's a very disrupting transition to do. I don't mean for most of the >>>> users, but for the source code. On the brighter side, what really >>>> happens when you do that transition is that instead of something like: >>>> >>>> if (value is TemplateMapLikeModel) { >>>> ... ((TemplateMapLikeModel) value).get(k) ... >>>> } >>>> >>>> you, in first approach, rewrite it to something like: >>>> >>>> if (mop.isMapLike(value)) { >>>> ... mop.get(value, k) ... >>>> } >>>> >>>> Of course that would be a silly MOP API design, so this below is more >>>> likely, but now it's visible that one comes from the other trivially: >>>> >>>> >>>> MapLike m = mop.tryGetAsMapLike(value); >>>> if (m != null) { >>>> ... m.get(k) ... >>>> } >>>> >>>> Hence when we have our TemplateModel-s cleaned up, it's easier to >>>> migrate to the MOP approach. The "type system" (just a mental model >>>> really) depicted be the TemplateModel still exists, you just use a MOP >>>> to implement it. >>>> >>>> > I don't mean to derail this discussion, but especially these days in a >>>> > world of a huge number of great Java libraries out there it is often >>>> better >>>> > to not try to do everything and instead focus on core strengths that you >>>> > can't get anywhere else, and make it easy for people to use other open >>>> > source projects for their core strengths. This could even go as far as >>>> > FreeMarker builds that have dependencies on certain commonly used open >>>> > source projects, there are many even in the ASF that do various things >>>> that >>>> > FTL built-ins do but with far more options and well known behavior for >>>> the >>>> > many who use them (like various Commons projects). >>>> >>>> It's really the duty of the Dialects feature of FM3. You can put >>>> together your "toolbox" from whatever you want, like call Apache >>>> Commons. I mean it's unrelated to question of expression languages, as >>>> far as I see. (Like, you can call plain Java methods from FM2 as >>>> well.) >>>> >>>> If we want to use Apache Commons etc. in implementing the functions of >>>> the standard dialect... I *guess* we won't benefit from it much at >>>> this point, but I need to see it case-by-case to tell, like, the FM >>>> implementation of function x sucks, while Apache Commons StringUtils.x >>>> is great, etc. In any case, users don't care, as far as they can call >>>> ?x and it does its job. My point in this thread is that we should be >>>> able to tell users to "just write ?x" (see earlier), instead of add >>>> your own solution somehow. (Also, if ?x works, they prefer if we have >>>> less dependencies. Especially if we want to support Andorid.) >>>> >>>> > Would supporting different expression languages cause huge problems for >>>> IDE >>>> > support? Possibly, but IDEs these days have lots of support for language >>>> > injection and layers that are meant for just this sort of thing. >>>> >>>> At least looking from big enough distance it's easy. :) Note that so >>>> far we fail to deliver an editor for Eclipse that integrates the >>>> existing HTML editor functionality with FTL... :-> Please everyone go >>>> and help out Angelo Zerr so that this will change: >>>> >>>> https://github.com/angelozerr/lsp4e-freemarker >>>> https://github.com/angelozerr/freemarker-languageserver >>>> >>>> Anyway, IDE support won't be the obstacle. At worse only the default >>>> expression syntax will have reasonable IDE support. >>>> >>>> > In theory FreeMarker itself could be a heck of a lot smaller and >>>> > simpler and at the same time be far more flexible and powerful. IMO >>>> > that could be a game changer in the world of Java-based template >>>> > languages. Some of this would be more work than not supporting this >>>> > sort of flexibility (ie support Groovy syntax as an option, ie >>>> > without requiring the much larger Groovy jar files as dependencies >>>> > by having a default built-in expression syntax), but by simplifying >>>> > other things it would save a heck of a lot of work (including design >>>> > work). >>>> >>>> So, I'm not sure I follow what exactly are you proposing. To use >>>> vanilla Groovy (the actual implementation) for expressions? How does >>>> that blend with the directives (which mostly also have similar pairs >>>> in Groovy), and other (planned) FM features? (And... why's the result >>>> called FreeMarker at all? ;) ) >>>> >>>> > -David >>>> > >>>> > >>>> > >>>> > On Wed, May 30, 2018 at 5:50 AM, Taher Alkhateeb < >>>> [email protected] >>>> >> wrote: >>>> > >>>> >> I'm not a subject matter expert here, but I always thought sensible >>>> >> defaults with the ability to override using configuration is a wise >>>> >> move because you appeal to many people out of the box and then allow >>>> >> minority to also not feel locked out when they need to override. Also, >>>> >> as a general rule, I found from the projects we worked on that >>>> >> internationalization is always a challenge, but you can also resolve >>>> >> that with the same solution (sensible defaults with configuration >>>> >> overrides); then you simply let the community adopt and improve the >>>> >> code for non-English parts. >>>> >> >>>> >> Of course then the challenge becomes, what's a sensible default? I >>>> >> wish I had a dollar every time I asked that! I'd favor simplicity >>>> >> where possible. In your example above maybe "No space f..." is the >>>> >> simplest, and then people can get fancy if they want to? >>>> >> >>>> >> My 2 cents, sorry for the noise :) >>>> >> >>>> >> On Wed, May 30, 2018 at 11:59 AM, Daniel Dekany <[email protected]> >>>> >> wrote: >>>> >> > There's this old frequently requested built-in, >>>> >> > maybeTooLongText?truncate(n). Like "No space for such a long >>>> >> > title"?truncate(13) would give "No space...". The problem with these >>>> >> > is that there's no single algorithm for this that fits the need of >>>> >> > everyone. Like maybe the result should be "No space f...", or "No >>>> >> > space [...]", etc. This can even change depending on the current value >>>> >> > of the `locale` FreeMarker setting. >>>> >> > >>>> >> > Now there's another similar wish, for "?title_case", which has the >>>> >> > same problem, plus it's much more locale dependent. See: >>>> >> > >>>> >> > https://issues.apache.org/jira/browse/FREEMARKER-97 >>>> >> > >>>> >> > We do provide locale dependent functions, like number and date >>>> >> > formatting. This is pretty much an FM tradition, to treat non-English >>>> >> > users as equals (well, except that the keywords aren't localizable). >>>> >> > However, we push those tasks to the Java platform. But it doesn't >>>> >> > support title case (as far as I know). Besides there can be different >>>> >> > preferences even inside the same country, just as with ?truncate. >>>> >> > >>>> >> > So, it's acceptable to chicken out ;) saying that none of the above >>>> >> > has a widely accepted default, so the correct solution is letting the >>>> >> > user solve this. But I think in practice what happens is that people >>>> >> > need the functionality (especially ?truncate), and then has to come up >>>> >> > with some ad-hoc solution on the spot, which will be almost always >>>> >> > worse than the default we could provide. So we might as well just >>>> >> > provide some decent default, sparing time for lot of users. Then, then >>>> >> > if they are particular about the algorithm (which is certainly rare >>>> >> > nowadays), let them plug their own via the Configurable API. So there >>>> >> > would be something like >>>> >> > cfg.setTruncateImplementation(localToImplementationMap), where the >>>> >> > parameter maps locale-s to Java functions and defines a fallback >>>> >> > function. (I haven't worked out the API details, as you can see.) >>>> >> > >>>> >> > What do you think? >>>> >> > >>>> >> > -- >>>> >> > Thanks, >>>> >> > Daniel Dekany >>>> >> > >>>> >> >>>> >>>> -- >>>> Thanks, >>>> Daniel Dekany >>>> >>>> >> >> -- >> Thanks, >> Daniel Dekany >> > -- Thanks, Daniel Dekany
