I haven't done any extensive testing on the built-in constructs for Arabic support but from my work in OFBiz I can safely say that most of my needs are already available OOTB. I think this is in part due to the java platform as well as using the timezone and locale APIs heavily and all throughout.
On Fri, Jun 8, 2018, 11:48 PM Daniel Dekany <[email protected]> wrote: > Friday, June 8, 2018, 8:41:34 PM, Taher Alkhateeb wrote: > > > One of the absolute hardest things for me to accomplish is to get the > > Arabic language with all the RTL quirks done right. I'm an Arab but even > I > > suffer from the complexity of this language. > > Speaking of which, I never hear about the needs of languages that are > very different (I guess) from the typical western European and US > languages, such as the Arabic, Chinese, Hindi and languages. I only > speak Hungarian and English (and has some very dim past experience > with Russian and German), so I have no idea if FM can be of any help > there. > > > Hoping to achieve true and full internationalization is something that > > might not be very realistic IMHO. So maybe the best decision is not to > > support all languages but rather having the architecture and design of > your > > templating engine done in an extinsible-friendly manner, which seems to > be > > the case here? > > Exactly. > > I wonder how well the existing features work there, where we build on > the the localization features of the Java platform. I'm not sure how > well it works for languages that are very far from the "western > culture". (I had some feedback that suggests that for some such > country the defaults are wrong, but for a while it's possible to > handle that with custom number and date/time formats.) > > > So yeah, perhaps I would prefer to go for English OOTB support and keep a > > flexible architecture in place. > > > > On Thu, Jun 7, 2018, 5:34 PM Daniel Dekany <[email protected]> wrote: > > > >> Tuesday, June 5, 2018, 3:28:45 PM, Woonsan Ko wrote: > >> > >> > On Mon, Jun 4, 2018 at 5:04 AM, Daniel Dekany <[email protected]> > >> wrote: > >> >> Sunday, June 3, 2018, 10:13:04 PM, David E Jones wrote: > >> >> > >> >>> You make good points Daniel. At a high level it reminds me of a > >> discussion > >> >>> with a new client a couple months ago on the topic of managing > >> templates > >> >>> for ecommerce sites. They want to use the Liquid template engine > partly > >> >>> because they are doing custom ecommerce for some customers and > Shopify > >> >>> (which uses Liquid, where Liquid came from) for others, and also > >> because > >> >>> Liquid is more constrained and easier for web developers/designers > to > >> use. > >> >>> > >> >>> There is a case for template languages that are less developer > >> oriented and > >> >>> more designer oriented, or maybe better put because visual web > >> designers > >> >>> have no business mucking around with templates server side > developer vs > >> >>> web/browser developer (even if this is stuff that generally runs on > a > >> >>> server anyway). > >> >>> > >> >>> Right now, ie FM2, FreeMarker strikes a good balance between these > (IMO > >> >>> anyway), and the built-ins are an important part of that. Maybe they > >> don't > >> >>> do every little thing everyone in the world might want but they > cover > >> a lot > >> >>> of basics and common needs. > >> >>> > >> >>> At a lower level you are correct about template parsing and > supporting > >> >>> arbitrary languages for inline expressions. The template parser > still > >> needs > >> >>> to be able to figure out where the expression starts and stops. This > >> might > >> >>> not be too bad for C-like languages which current FM expressions are > >> (more > >> >>> or less, mostly the same demarcation characters and all), but is > still > >> a > >> >>> challenge and could require something like surrounding expressions > with > >> >>> parenthesis (along with nested parenthesis evaluation that FM > already > >> does) > >> >>> and that's a little bit ugly. > >> >> > >> >> I wasn't even worried about that yet, but the semantical differences. > >> >> The expression and non-expression language (if they are separate at > >> >> all) build on a common mental model, so you can't, in general, cut > out > >> >> the expression language from one language, and put it into another. > So > >> >> I can only look at this on a case-by-case basis. > >> >> > >> >> As of the syntactical problems, at least if I write the expression > >> >> parser, I can just exit expression mode if the next token would lead > >> >> to an error, and let the "outer language" process the next character. > >> >> Like you have `a + b n=2`, then `n` would be an error, and the outer > >> >> parser will pick it up and recognize that as a parameter assignment. > >> >> What's tricky is making JavaCC to do this... it doesn't support > >> >> dynamic composing by design. ANTLR, I don't know (nor if it's fast > >> >> enough for use cases like a lot of ?eval/?interpret). For a hand > >> >> written parser, it's certainly not a problem. Though switching to > hand > >> >> written is a questionable step, even if I personally find it > tempting. > >> >> > >> >>> Anyway, sorry to derail the discussion. > >> >>> > >> >>> On the truncate built-in an optional String type second argument > makes > >> >>> sense as a suffix to use if the string is truncated, ie that seems > >> like an > >> >>> easy thing to accommodate without reducing flexibility or having to > >> write > >> >>> that logic to do that inline in the template. > >> >> > >> >> There's also the question of where to cut the text. There's the > >> >> maximum length given as parameter (actually, you subtract the length > >> >> of the cut marker suffix), however if with applying that naively you > >> >> end up with white-space at the end, you probably want to trim that. > >> >> Also certainly if it ends with a ellipsis, dot or a sequence of dots, > >> >> and maybe even if with some other punctuation. There's also the > >> >> typical preference, over a certain maximum length, where you only cut > >> >> at the end of words (unless you have some extremely long word). > >> > > >> > Commons-text's WordUtils#abbreviate(str, lower, upper, appendToEnd) > >> > finds a space from the lower index to find where to cut. > >> > It doesn't care other white spaces nor ending dot(s), but one of > >> > existing simple algorithms at the moment. ;-) > >> > > >> >> > >> >>> On title casing it might be best to defer to what Java does (ie > >> >>> Character.toUpperCase() or .toTitleCase()), or were there issues > with > >> that? > >> >> > >> >> They don't convert text to title case, only a single character. In > the > >> >> most basic form of title case conversion of text, you convert the > >> >> first letter of each word to title case. You may already have some > >> >> tricky parts there with finding word boundaries. Then, in many > >> >> languages, like in English, you aren't supported to convert words > like > >> >> "and", "of", etc. So it's obviously locale dependent. > >> > > >> > WordUtils#capitalizeFully() does. It doesn't recognize "and", "of", > etc. > >> though. > >> > >> Both WordUtils methods seem to be too simplistic for real world usage, > >> to me at least. The default implementation should be actually usable > >> for at least English content, and beat typical quick in-house > >> solutions in quality. It seems that that's doable (see Jira issue). > >> > >> I guess the other languages will be left for the users (the algorithm > >> is pluggable per language). As far as I know the most common language > >> used for web content is still English, by far. So I guess it's the > >> language FM is used for the most as well. It's quite weird as English > >> isn't the most widely spoken language among the people who use the > >> Internet. But I'm afraid if we pick, let's say, the next 4 most common > >> web content languages, that can be be more offending for those left > >> out, than if we support English only, which people more or less have > >> get used to. > >> > >> > But as you said, if it can be customized through Configuration for > >> > locale specific use cases, I'd say it's perfect. > >> > > >> > Regards, > >> > > >> > Woonsan > >> > > >> >> > >> >>> -David > >> >>> > >> >>> > >> >>> On Thu, May 31, 2018 at 2:03 AM, Daniel Dekany <[email protected]> > >> wrote: > >> >>> > >> >>>> Thursday, May 31, 2018, 1:16:26 AM, David E Jones wrote: > >> >>>> > >> >>>> > My general opinion on this sort of thing is there are lots of > >> libraries > >> >>>> and > >> >>>> > tools around in the Java world for doing string manipulations and > >> such, > >> >>>> > they don't need to be built into a template language. Taking that > >> one > >> >>>> step > >> >>>> > further, I'd actually prefer that the expression syntax in the > >> template > >> >>>> > language makes it easy to call an existing API vs trying to make > >> >>>> everything > >> >>>> > built into the template language. > >> >>>> > > >> >>>> > For example in addition to using FTL I also use Groovy heavily > and > >> Groovy > >> >>>> > already has a great API including standard Java API augmentation > >> for all > >> >>>> > sorts of things. Yes, FTL could use the Groovy String and other > >> expanded > >> >>>> > objects as a reference for built in design and have a good set of > >> >>>> commonly > >> >>>> > needed things, but why not just defer to Groovy for those that > use > >> it or > >> >>>> > whatever other API/library anyone might prefer. > >> >>>> > > >> >>>> > I know this is a VERY different direction from FM2. What I find > most > >> >>>> useful > >> >>>> > in FTL is features like macros and the extensive directives and > >> other > >> >>>> > things that make a template language flexible and powerful. The > >> built-ins > >> >>>> > are handy sometimes, but often don't do what I need anyway so > other > >> than > >> >>>> > things like ?has_content I don't use them a lot. > >> >>>> > >> >>>> An important (and as far as I'm concerned, the main) reason for > many > >> >>>> built-ins is/was that frameworks and other products drop in > FreeMarker > >> >>>> for templates and then often don't give much attention to give > tools > >> >>>> to the template authors. Then the poor people who actually has to > >> >>>> create templates come and ask, "I must do X... how can I do that in > >> >>>> FreeMaker?". The less often we have to say "Please pester the > >> >>>> developers (who are often from another company, or are on another > >> >>>> sub-project by now => you have near 0 chance) to expose such > >> >>>> functionality to the templates", the better. Instead we can say > "Like > >> >>>> foo?x", and furthermore, ?x is documented, and people can actually > >> >>>> find its documentation, even years later. Surely if you have some > >> >>>> small company with the sharp people, this is not a problem for you. > >> >>>> Actually, then you very often don't want the one-size-fits-all > >> >>>> solutions anyway. > >> >>>> > >> >>>> > I've mentioned this before but what would be a killer feature for > >> FTL is > >> >>>> to > >> >>>> > support pluggable expression evaluators. The FTL expression > syntax > >> isn't > >> >>>> > awful > >> >>>> > >> >>>> (Yes it is... but that's besides the point until I have fixed it in > >> >>>> FM3.) > >> >>>> > >> >>>> > but is not nearly as convenient as Groovy, especially when you > are > >> >>>> > using it extensively elsewhere. > >> >>>> > >> >>>> The theoretical problem is of course that both languages (the > "outer" > >> >>>> template language and the language from where you rip off the > >> >>>> expression language from) have its own ideas, and so there will be > >> >>>> mismatches. The expression language in a language and other parts > of > >> >>>> the language aren't separate in general anyway. Which leads to the > >> >>>> question that if you are using Groovy expressions, why not also use > >> >>>> for loops and whatnot from Groovy? (This is fairly common way of > >> >>>> creating template language, just taking an existing language, and > add > >> >>>> some convenience syntax to embed it into the static text.) So then > we > >> >>>> don't need our own #list, #if, etc. implementation either. But > then, > >> >>>> the original idea was that a template language is a domain specific > >> >>>> language, and as such can have more specialized, say, loop > constructs > >> >>>> for that domain. (I'm not saying that FM2 did great there, but > there > >> >>>> was a lack of determination there as well.) If that's not the case > >> >>>> though... > >> >>>> > >> >>>> Anyway, when and if FM3 will be more mature, we can see if how > doable > >> >>>> this is in reality. It's also a question if we are talking about a > >> >>>> syntax, that in 99% looks like Groovy, or we are talking about > actual > >> >>>> Groovy. In the last case TemplateModel-s and the particular way > null > >> >>>> handling will work will be an obstacle for sure. Not to mention > >> >>>> TemplateMarkupOutputModel-s, that AFAIK don't even have commonly > used > >> >>>> equivalent (like a class in the standard Java API). > >> >>>> > >> >>>> > If the default FTL expressions were not used you couldn't use > >> >>>> > built-ins and instead would just have expressions that are > expected > >> >>>> > to evaluate to a boolean, string, or arbitrary object and can be > >> >>>> > whatever. > >> >>>> > >> >>>> At least in FM3 built-ins will be just functions you have defined > in > >> >>>> the "core" FTL namespace (whose content is specified in template > >> >>>> language Dialect you use) that's "statically linked" to the > templates > >> >>>> (allowing some parse-time checks and somewhat faster calls). So > >> >>>> `foo?bar(1)` is basically just a shorthand syntax for `<#import > core = > >> >>>> 'freemarker://dialect-or-something'>` and `core.bar(foo, 1)`. > There's > >> >>>> a standard/default Dialect of course, which has the big advantage > that > >> >>>> it's documented in the official Manual, is used in SO answers, etc. > >> >>>> > >> >>>> > How important is this? Important enough that I've > >> >>>> > considered using a different template language. If FTL wasn't so > >> >>>> > darn convenience for XML transformation using macros I'd > probably be > >> >>>> > using Groovy GString templates instead, and just for the far > better > >> >>>> > syntax in Groovy. > >> >>>> > > >> >>>> > This also goes back to the object wrapping discussion which is > >> another > >> >>>> can > >> >>>> > of worms... > >> >>>> > >> >>>> Yeah, it's one of the things that I really wanted to get rid of, > but > >> >>>> it's a very disrupting transition to do. I don't mean for most of > the > >> >>>> users, but for the source code. On the brighter side, what really > >> >>>> happens when you do that transition is that instead of something > like: > >> >>>> > >> >>>> if (value is TemplateMapLikeModel) { > >> >>>> ... ((TemplateMapLikeModel) value).get(k) ... > >> >>>> } > >> >>>> > >> >>>> you, in first approach, rewrite it to something like: > >> >>>> > >> >>>> if (mop.isMapLike(value)) { > >> >>>> ... mop.get(value, k) ... > >> >>>> } > >> >>>> > >> >>>> Of course that would be a silly MOP API design, so this below is > more > >> >>>> likely, but now it's visible that one comes from the other > trivially: > >> >>>> > >> >>>> > >> >>>> MapLike m = mop.tryGetAsMapLike(value); > >> >>>> if (m != null) { > >> >>>> ... m.get(k) ... > >> >>>> } > >> >>>> > >> >>>> Hence when we have our TemplateModel-s cleaned up, it's easier to > >> >>>> migrate to the MOP approach. The "type system" (just a mental model > >> >>>> really) depicted be the TemplateModel still exists, you just use a > MOP > >> >>>> to implement it. > >> >>>> > >> >>>> > I don't mean to derail this discussion, but especially these days > >> in a > >> >>>> > world of a huge number of great Java libraries out there it is > often > >> >>>> better > >> >>>> > to not try to do everything and instead focus on core strengths > >> that you > >> >>>> > can't get anywhere else, and make it easy for people to use other > >> open > >> >>>> > source projects for their core strengths. This could even go as > far > >> as > >> >>>> > FreeMarker builds that have dependencies on certain commonly used > >> open > >> >>>> > source projects, there are many even in the ASF that do various > >> things > >> >>>> that > >> >>>> > FTL built-ins do but with far more options and well known > behavior > >> for > >> >>>> the > >> >>>> > many who use them (like various Commons projects). > >> >>>> > >> >>>> It's really the duty of the Dialects feature of FM3. You can put > >> >>>> together your "toolbox" from whatever you want, like call Apache > >> >>>> Commons. I mean it's unrelated to question of expression > languages, as > >> >>>> far as I see. (Like, you can call plain Java methods from FM2 as > >> >>>> well.) > >> >>>> > >> >>>> If we want to use Apache Commons etc. in implementing the > functions of > >> >>>> the standard dialect... I *guess* we won't benefit from it much at > >> >>>> this point, but I need to see it case-by-case to tell, like, the FM > >> >>>> implementation of function x sucks, while Apache Commons > StringUtils.x > >> >>>> is great, etc. In any case, users don't care, as far as they can > call > >> >>>> ?x and it does its job. My point in this thread is that we should > be > >> >>>> able to tell users to "just write ?x" (see earlier), instead of add > >> >>>> your own solution somehow. (Also, if ?x works, they prefer if we > have > >> >>>> less dependencies. Especially if we want to support Andorid.) > >> >>>> > >> >>>> > Would supporting different expression languages cause huge > problems > >> for > >> >>>> IDE > >> >>>> > support? Possibly, but IDEs these days have lots of support for > >> language > >> >>>> > injection and layers that are meant for just this sort of thing. > >> >>>> > >> >>>> At least looking from big enough distance it's easy. :) Note that > so > >> >>>> far we fail to deliver an editor for Eclipse that integrates the > >> >>>> existing HTML editor functionality with FTL... :-> Please everyone > go > >> >>>> and help out Angelo Zerr so that this will change: > >> >>>> > >> >>>> https://github.com/angelozerr/lsp4e-freemarker > >> >>>> https://github.com/angelozerr/freemarker-languageserver > >> >>>> > >> >>>> Anyway, IDE support won't be the obstacle. At worse only the > default > >> >>>> expression syntax will have reasonable IDE support. > >> >>>> > >> >>>> > In theory FreeMarker itself could be a heck of a lot smaller and > >> >>>> > simpler and at the same time be far more flexible and powerful. > IMO > >> >>>> > that could be a game changer in the world of Java-based template > >> >>>> > languages. Some of this would be more work than not supporting > this > >> >>>> > sort of flexibility (ie support Groovy syntax as an option, ie > >> >>>> > without requiring the much larger Groovy jar files as > dependencies > >> >>>> > by having a default built-in expression syntax), but by > simplifying > >> >>>> > other things it would save a heck of a lot of work (including > design > >> >>>> > work). > >> >>>> > >> >>>> So, I'm not sure I follow what exactly are you proposing. To use > >> >>>> vanilla Groovy (the actual implementation) for expressions? How > does > >> >>>> that blend with the directives (which mostly also have similar > pairs > >> >>>> in Groovy), and other (planned) FM features? (And... why's the > result > >> >>>> called FreeMarker at all? ;) ) > >> >>>> > >> >>>> > -David > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > On Wed, May 30, 2018 at 5:50 AM, Taher Alkhateeb < > >> >>>> [email protected] > >> >>>> >> wrote: > >> >>>> > > >> >>>> >> I'm not a subject matter expert here, but I always thought > sensible > >> >>>> >> defaults with the ability to override using configuration is a > wise > >> >>>> >> move because you appeal to many people out of the box and then > >> allow > >> >>>> >> minority to also not feel locked out when they need to override. > >> Also, > >> >>>> >> as a general rule, I found from the projects we worked on that > >> >>>> >> internationalization is always a challenge, but you can also > >> resolve > >> >>>> >> that with the same solution (sensible defaults with > configuration > >> >>>> >> overrides); then you simply let the community adopt and improve > the > >> >>>> >> code for non-English parts. > >> >>>> >> > >> >>>> >> Of course then the challenge becomes, what's a sensible > default? I > >> >>>> >> wish I had a dollar every time I asked that! I'd favor > simplicity > >> >>>> >> where possible. In your example above maybe "No space f..." is > the > >> >>>> >> simplest, and then people can get fancy if they want to? > >> >>>> >> > >> >>>> >> My 2 cents, sorry for the noise :) > >> >>>> >> > >> >>>> >> On Wed, May 30, 2018 at 11:59 AM, Daniel Dekany < > >> [email protected]> > >> >>>> >> wrote: > >> >>>> >> > There's this old frequently requested built-in, > >> >>>> >> > maybeTooLongText?truncate(n). Like "No space for such a long > >> >>>> >> > title"?truncate(13) would give "No space...". The problem with > >> these > >> >>>> >> > is that there's no single algorithm for this that fits the > need > >> of > >> >>>> >> > everyone. Like maybe the result should be "No space f...", or > "No > >> >>>> >> > space [...]", etc. This can even change depending on the > current > >> value > >> >>>> >> > of the `locale` FreeMarker setting. > >> >>>> >> > > >> >>>> >> > Now there's another similar wish, for "?title_case", which has > >> the > >> >>>> >> > same problem, plus it's much more locale dependent. See: > >> >>>> >> > > >> >>>> >> > https://issues.apache.org/jira/browse/FREEMARKER-97 > >> >>>> >> > > >> >>>> >> > We do provide locale dependent functions, like number and date > >> >>>> >> > formatting. This is pretty much an FM tradition, to treat > >> non-English > >> >>>> >> > users as equals (well, except that the keywords aren't > >> localizable). > >> >>>> >> > However, we push those tasks to the Java platform. But it > doesn't > >> >>>> >> > support title case (as far as I know). Besides there can be > >> different > >> >>>> >> > preferences even inside the same country, just as with > ?truncate. > >> >>>> >> > > >> >>>> >> > So, it's acceptable to chicken out ;) saying that none of the > >> above > >> >>>> >> > has a widely accepted default, so the correct solution is > >> letting the > >> >>>> >> > user solve this. But I think in practice what happens is that > >> people > >> >>>> >> > need the functionality (especially ?truncate), and then has to > >> come up > >> >>>> >> > with some ad-hoc solution on the spot, which will be almost > >> always > >> >>>> >> > worse than the default we could provide. So we might as well > just > >> >>>> >> > provide some decent default, sparing time for lot of users. > >> Then, then > >> >>>> >> > if they are particular about the algorithm (which is certainly > >> rare > >> >>>> >> > nowadays), let them plug their own via the Configurable API. > So > >> there > >> >>>> >> > would be something like > >> >>>> >> > cfg.setTruncateImplementation(localToImplementationMap), where > >> the > >> >>>> >> > parameter maps locale-s to Java functions and defines a > fallback > >> >>>> >> > function. (I haven't worked out the API details, as you can > see.) > >> >>>> >> > > >> >>>> >> > What do you think? > >> >>>> >> > > >> >>>> >> > -- > >> >>>> >> > Thanks, > >> >>>> >> > Daniel Dekany > >> >>>> >> > > >> >>>> >> > >> >>>> > >> >>>> -- > >> >>>> Thanks, > >> >>>> Daniel Dekany > >> >>>> > >> >>>> > >> >> > >> >> -- > >> >> Thanks, > >> >> Daniel Dekany > >> >> > >> > > >> > >> -- > >> Thanks, > >> Daniel Dekany > >> > >> > > -- > Thanks, > Daniel Dekany > >
