Monday, February 13, 2017, 9:00:34 PM, Pedro M. Zamboni wrote:
> This was originally meant to be posted to the Freemarker issue
> tracker, so it’s formatted as such.
Thanks for bringing up this topic. I have responded inline, and then
described what are my current plants regarding the aspects touched
here. I recommend reading the whole thing before responding (unlike I
did it... oh well).
> ----
>
> I’ll refer to “missing values” as just “null” in this post. I hope you
> don’t mind.
OK, but just to be sure you know the reason of this terminology: If
you have an object that doesn't have a `getX()`, and you refer to `x`,
it would be confusing to tell the user that `x` was null. There's
simply no `x`. FM2 treats that as the same as if you do have a getX()
that returns null. So it's too cases but as far as FTL2 cares, they
are the same.
> ----
>
> I was always awkward out by Freemarker’s null handling. This doesn’t
> make sense to me:
>
> {code:none}
> <#if foo.bar.baz.qux.bah??> <#-- Only tests for null in `bah` (error
> if null somewhere else) -->
> <#-- ... -->
> </#if>
> {code}
>
> {code:none}
> <#if (foo.bar.baz.qux.bah)??> <#-- Tests for null all the way through -->
> <#-- ... -->
> </#if>
> {code}
I don't want the "exception catcher" operators in FM3 either. But note
that there's a valid use case behind them, which must be addressed.
It's that you have, say, `user.name`, and it's expected that `user` is
sometimes null. (And you can have such examples with longer chains of
dotted things too of course, where some of the steps are expected to
be null sometimes, while others aren't.) The usual solution is
nowadays (like in Groovy for example) is something like
`user?.name:?'-'`. The existence handler operator is traditionally `!`
in FM though, and `?.` is kind of confusing as `?` is used for
built-ins. So then you end up with `user!.name!'-'` as a IMO good
compromisse. And I plan just that for FM3.
The usual behavior of those operators in most languages is that if in
`e1!.e2` `e1` is null, then `e1!.e2` is null as well. That's fine, but
if it's all to it , that leads to that if in `a.b.c.d.e` you expect
`a` to be null sometimes, but not the others (`.b`, `.c`, etc.), you
still end up with `a!.b!.c!.d!.e`, because all the dot operators can
get a null LHO because of the first `!.`. I belive, it should be
enough to write `a!.b.c.d.e`, because its obvious that if `a` was
null, then the whole chain has to be skipped. More importantly, if you
have to write `a!.b!.c!.d!.e` because of this, and you don't expect
`.b`, `.c`, etc. to be null (I mean as far as their LHO wasn't null),
then now you unwillingly suppressed typos on those names. It's bad
enough that you had to do that for `a`, so, I would allow
`a!.b.c.d.e`. (It wouldn't use any special dot-chain-skipping logic,
it would just happen naturally, and I will tell at the end why.)
There's another problem, which might doesn't worth addressing though.
So I have `user.name`, and yet again, I expect `user` to be `null`
sometimes, but not `name`. That is, if there's a `user` it must have a
`.name`. Problem is, when the result of `user!.name` is `null`, I
can't know if it was `user` that was null or `.name`. So then if I
write `user.naem!'-'` (note the typo), or the model is changed so that
it should be `.fullName` now, I will still only get a gentle "-". It's
easy to address actually, but it would mean that I have a separate
"this one can be null" and a separate "replace null with this"
operator (or built-in, whatever), which I think is just too much to
grok for most user.
Regarding why do we have to be so paranoid about null-s in the first
place... I mean, many languages differentiate between an undefined
variable (or member), which is always an error, and an existing
variable (or member) that holds null. It's the obvious thing to do for
more static languages (like Ceylon). That would be the ideal thing,
but I think the nature of the data-models pushed on FM makes doing
this problematic. Specifically, the frequent usage of Map-s instead of
JavaBean-s (and also the lack of compilation time checking). While in
FM3 I do consider differentiating the two cases (undefined and defined
but null), we still have to deal with those Map-s and the typos in the
key names.
> {{??}} and {{!}} don’t behave like operators generally would in
> programming languages.
Actually they do. They just accept a null LHO, unlike most others. I
know from the Jira issue why you think it's not so, but it's only
`(exp)!exp` and `(exp)??` that do the exception catching trick (and I
want to get rid of that in FM3).
> In my point of view, operators should act similar to functions, and
> there is no way to actually write a function that behaves like
> {{??}} does.
Agreed.
> It seems to me that {{(expr)??}} and {{expr??}} are completely
> different operators with slightly different semantics.
Agreed, those two above should mean exactly the same, and also the
same as `((((exp)))??`, etc.
> Additionally, there is no way to test if only {{qux}} or {{bah}} is
> null (and stop with an error if there is null somewhere else). That
> is, the following doesn’t work:
>
> {code:none}
> <#if foo.bar.baz.(qux.bah)??> <#-- doesn’t work -->
> <#-- ... -->
> </#if>
> {code}
Yip, can't do that because the nesting of that is like:
(((foo.bar).baz).qux).bah), so where do you put the `try` block? My
current idea for FM3 is that there's no try block at all, and you can
write `foo.bar.baz.qux!.bah??`.
> {code:none}
> <#if foo.bar.baz(.qux.bah)??> <#-- doesn’t work -->
> <#-- ... -->
> </#if>
> {code}
>
> h1. My solution
>
> My solution is greatly inspired by [Ceylon|https://ceylon-lang.org/].
>
> h2. Expressions evaluating to null don’t throw exception
>
> It seems pretty stupid since you still wouldn’t be able to do anything
> with null.
Well, when left untreated they will, but I know what you mean.
> You wouldn’t be able to store it in variables, output it,
> access most built‐ins in it, access hash members in it. But this
> feature would allow certain important things to be done: accessing
> certain null‐specific built‐ins in it and most interestingly:
> accessing built‐ins and hash members in it in a safe way.
>
> h2. {{exists}} and {{is_absent}} built‐ins
>
> I think that instead of having an operator exclusive to dealing with
> null is completely unnecessary since Freemarker already has a
> mechanism that could be used to do it: built‐ins. {{exists}} could be
> used to do what {{??}} does today, and {{?is_absent}} would be its
> logical negation.
>
> h2. {{else}} built‐in
>
> Similarly to {{??}}, it feels to me that we could reuse the much more
> general‐purpose built‐ins mechanism in the place of {{!}}. Instead of
> writing {{foo!12}}, one would write {{foo?else(12)}}.
There were ?exists and ?default(exp), but because these are used so
often, they have been deprecated in favor of the dedicated operators.
(FTL tries to specialize on the things that you have to do in this
problem space again and again.)
> h2. {{is_*}} built‐ins
>
> Existing built‐ins that test for type (such as {{is_number}},
> {{is_string}}, {{is_sequence}}) would be able to operate on null and
> would always evaluate to {{false}}.
The problem is that it's not a static language. We receive the
data-model from outside, often through a Map (not even a JavaBean
where you at least know what variables can exist even if they are
null), so it's the worse possible situation to find out if the
template author has made a mistake or not. And we try not to suppress
typos and data-model mismatches. So while it's inconvenient for the
user to do some extra test there, the approach of FM is that an Error
500 is better than showing incorrect information. Yes, we can't catch
these problems on lot of places, like even you you write ${usre!'-'}
(note the typo), but at least wherever we can we try to catch them.
> h2. {{!.}}, {{![]}}, and {{!?}}
>
> These symbols would work as a “null‐safe” hash member access and
> built‐in access. That is, {{foo!.bar}} would access the member {{bar}}
> from {{foo}} unless {{foo}} is null, in which case the whole
> expression will be null. The behavior is analogous for {{![]}} and
> {{!?}}.
So you also thought of `!.` then. That's a good sign, as then it seems
that it's natural enough. And if we have `exp?else(exp)` instead of
`exp!exp`, then `![]` doesn't clash with it, I see that. But I
wouldn't give up `exp!exp` for the sake of `map![key]`, because that's
much rarer then giving a default to something. Well, it can be
`(map!)[]` which is uglier, and kind of strange as it doesn't have
`![` syntactically... I well return to that later.
> The behavior of built‐ins that _can_ operate on null (such as
> {{has_content}} or even {{else}}, {{exists}} and {{is_*}}) would be
> the same as the ones that can’t:
In fact even in FM2 they are the same, but I have mentioned the
confusion regarding that earlier.
> if the operand evaluates to null, the expression will evaluate to
> null, otherwise, it will access the buit‐in.
>
> Then, instead of writing {{(foo.bar.baz.qux.bah)?exists}}, one would
> write {{foo!.bar!.baz!.qux!.bah?exists}}.
(See my problem with that earlier... i.e, why so many `!.`-s?)
> h2. {{exists}} conditions
>
> In Ceylon, one can write {{if(foo, bar, baz)}} to mean {{if(foo && bar
> && baz)}}. What may seem like a trivial syntactic difference actually
> allows for a more powerful feature to exist: the {{exists}} condition.
> The {{exists}} condition is not actually an expression, but it is part
> of the {{if}} syntax. it may be used to declare variables that can be
> used inside the {{if}} body, while also checking for {{null}}. In
> Freemarker, the syntax would look like this:
>
> {code:none}
> <#if exists bah = foo.bar.baz.qux.bah> <#-- Only tests for null in
> `bah` (error if null somewhere else) -->
> ${bah}
> </#if>
> {code}
>
> {code:none}
> <#if exists bah = foo!.bar!.baz!.qux!.bah> <#-- Tests for null all the
way through -->>
> ${bah}
> </#if>
> {code}
Well, `exists` is kind of verbose considering checking if something
exists is frequent in templates, but it's easier to understand when
someone reads the template, so maybe it worths it. I'm not sure yet.
But I don't think it has reason to be bound to `if`. Why is it? One
could argue that it changes the type of `blah` to non-nullable, but I
don't think it's relevant in a language where variables (and members)
has no declared type.
> Multiple {{exists}} condition could be used, separated by a comma ({{,}}):
>
> {code:none}
> <#if exists baz = foo!.bar!.baz, exists bah = baz!.qux!.bah> <#--
> Exposes both `baz` and `bah` -->
> ${baz} ${bah}
> </#if>
> {code}
With a more FM2-ish syntax (but with `!.`) that could be written as:
<#if (baz = foo!.bar!.baz)?? && (bah = baz!.qux!.bah)??>
Do I miss something with the comma operator, considering this is not a
static language? (BTW, an interesting thing above is the scoping of
the assignments. But that's a different topic.)
> {code:none}
> <#-- You still wouldn’t be able to check `null == something`, since
> the abscence of a value (null) doesn’t have a well‐defined value
equality -->>
> <#if exists bah = foo!.bar!.baz!.qux!.bah, bah == "hello"> <#-- Tests
> for null and checks if `bah` is equals to `"hello"` -->
> Hello!
> </#if>
> {code}
Here again, `&&` has a left-to-right ordering as well, who why the
`,`.
> h1. Conclusions
>
> I feel like this will be a great improvement to Freemarker, making it
> feel more intuitive by:
>
> # making its operators’ behaviors more intuitive
> # helping avoid repetition with {{exists}} variables
> # making the language easier to learn by reusing existing mechanisms
> instead of creating completely new operators to learn
Null handling changes is one of the major reasons for FM3 actually, so
we will do something. (And if I had a time machine I would use `?` in
place of `!` on the first place. I'm not sure if back then any well
known language have used `?` to mark something that's optional, even
if only inside operators like `?.`, `?:`, but nowadays it has become a
convention pretty much. As of what to do with that, my personal
opinion is that FTL3 should look and feel like FTL2 where it's
possible, so the *basic* meaning of `!` and `?` stays. But, if things
go well, we may build yet another language on the top of the same
engine, which looks and feel totally differently anyway, like its
top-level-symtax resembles more to Velocity or rather WebMacro, and
there we can put these to their places too.)
Now... What else I do I have in mind regarding FTL3 and nulls:
- I want null-s to "explode" as early as possible, because as far as
see, this is the sensible way to give an error message that explains
why do you get a null there. Like in ${foo.x}, it's obvious that if
`.x` evaluates to `null` you will have an error, because ${exp}
doesn't tolerate null-s. So the `.` operator would be called by
telling it that "and don't you dare to return a null". So then when
when `.` is evaluated, it can ask
the LHO value to give the member "x", and again it can tell to it to
not return a null, instead throw an exception that tells *why* it's
null (like the bean has no such JavaBean property, or the getter
method is coming from a non-public class (FAQ...), or the Map
contains no such key, or it does but it stores a `null`, or the wrapped
CSV has no such column, but has these columns, etc.). This is maybe
a surprisingly high expectation, but I have found that it could
help users considerably to fight those pesky "missing or null"
errors, which is really meaningless to many of them, or at least
rather unhelpful.
- I want FM to be remain as strict/fussy with null-s as it was (to
catch as many typos etc. as we can), but I want an easy and
universal way of telling FM that "I know, this one might evaluates
to null... so deal with it, silently". So that would the postfix `!`
operator, as in `exp!`. How it works is that actually most operators
(and now I count in ${} there too) can deal with null arguments,
only they call the operand expression by saying that they can't
return null (see first point about the better error messages - it's
exactly the same thing). But if there's a `exp!`, the `!` operator
will disobey, and call its own LHO by saying that returning null is
OK, and then despite what the caller has asked, return null if it
has to. This means that there's no `!.` operator, yet `a!.b.c` is
valid and it just means `((a!).b).c`. If the `.` operator gets a
null LHO, it just returns null. It asks its LHO expression to not
return a null, so `missing.b` will fail, but `!` disobeys as I said.
This is also why `(map!)[key]`, which I have mentioned, will just
work. Note again that it's not the caller of the operand expression
who checks for null, but the callee; that's the trick. That's why
you don't have to put `!.` all over the dot-chain once you had one.
- I want to make most built-ins to be null-bypassers, especially those
that just format a value. A null bypasser is a "function" (which can
be built-in, a #function, an operator, etc., I intend to treat
these as the same) which *declares* that if a certain parameter of
it is null (usually the 1st one), then it does nothing but returns
null. (Note that x?foo(y) basically means `core:x(x, y)`, so the
`?` LHO is the first parameter). Thus, if you have
`${maybeMissing?upperCase?trim?upperCase}`, you don't have to worry
about what will `?upperCase` and `?trim` do if maybeMissing is null.
They will just bypass it. Your only concern here is that `${}` will
not like the null. So you can write
${maybeMissing?upperCase?trim?upperCase!'-'}, or you get an error
message which will complain about `maybeMissing` itself (the old
trick again, because I already know when evaluating `maybeMissing`
that it can't return null, because then it will be bypassed twice
and so it reaches the ${} where it would explode.)
- Some built-ins, like `?isXxx` shouldn't be null bypassers, I believe.
They should act similarly to `.`, that is, they demand a non-null
from their LHO, but if they still get a null, they are prepared to
handle it without exception. Like ?isXxx would return `false` then.
So `missing?isString` throws exception, but `missing!?isString`
returns `false` (again, that's just `(missing!)?isString` there,
there's no `!?` operator.)
- I would keep `exp!exp`. I understand the advantages of expressive
names like `?else`, but consider that `exp!exp` is quite natural
once you have learnt about `exp!` (or the other way around... they
are similar, that's my point). Both suppresses the LHO null, but in
one case you tell FreeMarker to do what a lenient language would do,
while in the other you tell FreeMarker to pretend that the value was
the value of the 2nd exp. (BTW, ${x!} would print nothing, and
${x!'none'} would print "none". Because ${varName} is the most basic
and frequent thing people do, it matters that this use case looks
quite natural: If you didn't tell what to print, it prints nothing,
and if you don't like that, you can add after the `!` what to
print.)
OK, so that's part of my plans regarding null handling. (There's more
to it, but I try to not overload a thread.)
--
Thanks,
Daniel Dekany
