Thanks José, I think these are good suggestions. The 2-tuple notation is
indeed confusing, I didn't put much thought into it. Good that with the
:static_atom_encoder option this choice is up to the end user anyway. I'll
be working on this, expect a PR somewhere next week.
Arjan
On Thursday, April 11, 2019 at 12:09:36 PM UTC+2, José Valim wrote:
>
> HI Arjan,
>
> I have some notes after looking a bit further into the tokenizer/parser.
>
> 1. There are two places where we call the binary_to_atom functions: in the
> tokenizer and in the parser. For constructs like :"foo#{bar()}baz", we
> don't convert to atom at tokenizer time but at runtime. So my suggestion is
> to call this new option :static_atoms_encoder. If it is not set, it will
> fallback to the existing_atoms_only behaviour. It is also important to
> document that static_atoms_encoder won't be invoked for operators, syntax
> keywords (fn, etc), and for interpolated atoms (which, as explained above,
> are runtime based).
>
> 2. The :static_atoms_encoder expects a fun with two arguments. The
> function will receive the atom name (as a binary) and a keyword list with
> the line and column. It should return {:ok, token :: term} | {:error,
> reason :: binary}.
>
> 3. Your current replacement token {:":", atom_as_binary} can be confusing.
> [":": "foo"] will have AST of [{:":", "foo"}] with the option disabled and
> [:foo] will have the same AST when enabled. My suggestion to avoid
> ambiguities is to make it a tuple with two elements but the first element
> should either be a PID or a REF.
>
> 4. We should also include a complete reference of where static atoms
> appear. From the top of my head: aliases, remote calls, local calls, var
> names, atoms and keyword lists.
>
>
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/371f1826-8b22-43ea-a1df-cf511f88e60f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
