> This means that:
>
> a. I do not know the variables beforehand, they are user inputed as part
> of the template.
>
And you can and should know that after the parsing.
>
> b. also, it's not an issue of "This is indeed one of the strengths of
> Julia, and it requires neither string manipulation, nor parsing, nor
> creating files nor include.". I can not avoid creating, loading and parsing
> files, because that's what I want to do. I want to allow the users to
> create view files that are basically HTML files with embedded Julia code.
>
String manipulation, parsing are fine, just that you should not try to use
the julia parser to parse other formats. You need to define your format,
specify the syntax of expression (possibly by specifying how/what julia
code can be embedded in it, I highly doubt you want to allow embedding
arbitrary julia code especially if it is going to be used in any web
related context) which also includes parsing/extraction of template
arguments (and possibly global references) which will then give you enough
information to extract necessary values from the user supplied dict of
arguments and pass those to the template.
>
> b. I'm looking for a way to load and interpret multiline template code
> stored in a file. This seems to be tricky by itself as parse(...) works
> line by line (not good for me, I need to interpret the whole block at once,
> for example to handle if/else blocks) and I could not find a functional
> alternative for quote ... end or a way to pass a string into a quote ...
> end block. Like I said, the only thing close to achieving this would be
> include_string()
>
parse() supports this by allowing you to parse expression by expression
within the string. See the full signature of the function.
>
> c. I'd like to do this in a performant manner by avoiding the dreaded
> global scope (but not really by avoiding eval, which I think it's a
> necessary evil in this case).
>
That's what I said, if you actually care about performance, you should
create a function based on the template (i.e. "compile" the template) using
the information you need to extract from the template as mentioned above in
a way similar to what Erik said. You could also play tricks to use
generated function to do codegen by first encode all the information in the
type but it'll unlikely buy you much and will make things much more
complicated.
In either case, keep in mind that the function you creates will never be
freed again.
If you do this, the template instantiation will not need global scope
anymore which I assume is the place you care about performance. After all,
if you want template definition performance doing the parsing and
evaluating at runtime directly will be the fastest.
> ---
>
> Strictly addressing your questions:
> > If the variables are declared in a function, then there must be code
> that uses them; how would this code know which variables exist? Is there
> e.g. a globally known list of variables that will be provided? Or will the
> variables e.g. only be used in strings to generate html code?
>
> I go over the template file line by line and generate Julia code (as a
> string). This code is either logic (if/else, loops, etc) or raw output. For
> instance, the parsing of the above template would generate the following
> Julia code:
>
> "____output = Vector{AbstractString}()\npush!(____output,
> \"<html>\")\npush!(____output, \" <head>\")\npush!(____output, \"
> <title>\")\n____L9OHE6m05ZtELQIOMV8PPH0L = page_title\npush!(____output,
> \"\$(____L9OHE6m05ZtELQIOMV8PPH0L)\")\npush!(____output, \"
> </title>\")\npush!(____output, \" </head>\")\npush!(____output, \"
> <body>\")\npush!(____output, \" <h1>\")\n____wHbq5V7InZ2OJKpatWu7MtUU
> = greeting\npush!(____output,
> \"\$(____wHbq5V7InZ2OJKpatWu7MtUU)\")\npush!(____output,
> \" </h1>\")\npush!(____output, \" <p>\")\nif
> is_logged_in\npush!(____output,
> \" Edit account\")\nelse\npush!(____output, \"
> Login\")\nend\npush!(____output, \" </p>\")\npush!(____output, \"
> </body>\")\npush!(____output, \"</html>\")\npush!(____output, \"\")"
>
> Then this code is eval'd - at this time, the variables should already be
> in scope otherwise Julia won't find them.
>
> Yes, the variables are only used to generate HTML code. So ideally they
> should only be visible in the minimum scope necessary to eval this code.
>
> > Can you elaborate on why you want to access dictionary elements as local
> variables?
>
> I was hoping this could be an approach to eval the code with its vars in a
> non-global scope. Something in the lines of:
>
> function render_tpl(output::AbstractString, vars::Dict{Symbol,Any}) #
> output would be the above generated code and vars the list of vars used in
> the template
> # code here to expand the vars dict into function local variables
> # code here to eval the output (which would be the "____output =
> Vector{..." code above) -- this would automatically have access to the vars
> # return the resulted ____output Vector which now contains the rendered
> template
> end
>
> Thanks,
> Adrian
>
> duminică, 14 august 2016, 17:59:40 UTC+2, Erik Schnetter a scris:
>>
>> Adrian
>>
>> Can you give more details for how the variables would be used? If the
>> variables are declared in a function, then there must be code that uses
>> them; how would this code know which variables exist? Is there e.g. a
>> globally known list of variables that will be provided? Or will the
>> variables e.g. only be used in strings to generate html code?
>>
>> You can generate the syntax tree for function that contains the
>> definitions from the dictionary, and then use `eval` to create the
>> function. (Alternatively, you can use a macro or a generated function.)
>> This is indeed one of the strengths of Julia, and it requires neither
>> string manipulation, nor parsing, nor creating files nor include.
>>
>> Here is an example:
>>
>> fun_expr(var, val) = quote
>> function f(x)
>> $var = $val
>> x + y
>> end
>> end
>>
>> eval(fun_expr(:y, :42))
>>
>> `fun_expr` creates a syntax tree (an `Expr`) that defines a function. The
>> function is quoted. The function arguments `var` and `val` are inserted
>> into that function, forming an assignment. Thus `var` better be an
>> identifier (or something else that can be on the left of an assignment
>> operator), and `val` can be an arbitrary value. Given how the function
>> looks, `var` essentially has to be `y` since it's used in the next line; of
>> course, this was just my arbitrary choice.
>>
>> The call to `eval` then passes the respective arguments, choosing the
>> symbol `y` for the variable name, and the value `42` as value, and defines
>> the function `f`. If you then call `f(2)`, the result is 44.
>>
>> Of course, you can define this function `f` only once. Julia does not
>> allow changing functions. If you want to create many different functions,
>> you would use anonymous functions instead.
>>
>> Having said this -- it's not clear that this is indeed the right way to
>> address your problem; there might be a better solution. Can you elaborate
>> on why you want to access dictionary elements as local variables?
>>
>> -erik
>>
>>
>>
>> On Sun, Aug 14, 2016 at 11:13 AM, Adrian Salceanu <[email protected]>
>> wrote:
>>
>>> Thanks
>>>
>>> Maybe I wasn't clear enough - otherwise, can you please elaborate, I'm
>>> definitely still poking around, any clarifications would be highly
>>> appreciated.
>>>
>>> > creating a new module
>>> -> the module is available at compile time (the users of the templating
>>> system will place the vars in there, by convention).
>>>
>>> > parse julia code
>>> -> it's not really parsing julia code, it has no meaning at the point.
>>> It's simply basic string processing and it's fast - tried with a 10K lines
>>> HTML file, no sweat.
>>>
>>> > defining globals
>>> -> why are they globals? include_string() is used inside a function,
>>> inside a module within the app.
>>>
>>> > eval in a module
>>> -> true, but then what can we do? That's the way of doing
>>> metaprogramming in Julia, and it's widely used, isn't it?
>>> I guess that would be the price for not having to do
>>> print("<html><head>...") like our ancestors used to in PHP or ASP, when not
>>> being chased by tigers (or something around that age).
>>>
>>>
>>> duminică, 14 august 2016, 15:01:47 UTC+2, Yichao Yu a scris:
>>>>
>>>>
>>>>
>>>> On Sun, Aug 14, 2016 at 6:13 PM, Adrian Salceanu <[email protected]>
>>>> wrote:
>>>>
>>>>> Variables contained in a module and then parsed Julia code included
>>>>> within a function using include_string().
>>>>>
>>>>> Any obvious performance issues with this approach?
>>>>>
>>>>
>>>> Everything about it?
>>>> Literally every steps are hitting the slow path that is only meant to
>>>> execute at compile time and not runtime. Including
>>>>
>>>> Creating a new module, parse julia code, eval in a module, defining
>>>> globals.
>>>>
>>>>
>>>>>
>>>>>
>>>>> duminică, 14 august 2016, 12:11:06 UTC+2, Adrian Salceanu a scris:
>>>>>>
>>>>>> OK, actually, that's not nearly half as bad. Variables contained in a
>>>>>> module
>>>>>>
>>>>>> include("src/Ejl_str.jl")
>>>>>> using Ejl
>>>>>>
>>>>>>
>>>>>> module _
>>>>>> couñtry = "España"
>>>>>> lang = "en"
>>>>>> end
>>>>>>
>>>>>>
>>>>>> function render_template()
>>>>>> tpl_data = ejl"""
>>>>>> <% if _.lang == "en" :>
>>>>>> Hello from me, ...
>>>>>> <: else :>
>>>>>> Hola
>>>>>> <: end %>
>>>>>>
>>>>>>
>>>>>> %= _.couñtry == "España" ? "Olé" : "Aye"
>>>>>> moo
>>>>>> """
>>>>>>
>>>>>>
>>>>>> include_string(join(tpl_data, "\n"))
>>>>>> join(____output, "\n")
>>>>>> end
>>>>>>
>>>>>>
>>>>>> render_template() |> println
>>>>>>
>>>>>> Hello from me, ...
>>>>>>
>>>>>>
>>>>>> Olé
>>>>>> moo
>>>>>>
>>>>>>
>>>>>> duminică, 14 august 2016, 11:20:37 UTC+2, Adrian Salceanu a scris:
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> Yes, I've thought about a few ways to mitigate some of these issues:
>>>>>>>
>>>>>>> 1. in the app I can setup a module (like Render) and evaluate into
>>>>>>> this module exclusively.
>>>>>>> Hence, another approach might be to have some helper methods that
>>>>>>> setup the variables in the module and then eval the template itself
>>>>>>> inside
>>>>>>> the module too (must try though). So something in the lines of:
>>>>>>> set(:foo, "foo")
>>>>>>> set(:bar, [1, 2, 3])
>>>>>>> parse_tpl(ejl"""$(foo) and $(bar)""")
>>>>>>> # all the above gets parsed in Render
>>>>>>>
>>>>>>> 2. break the parsing in 2 steps:
>>>>>>> a. reading the template string and parse it to generated Julia code
>>>>>>> (as strings) (an array of Julia code lines) - cache it
>>>>>>> b. (load strings from cache and) eval the code together with the
>>>>>>> vars
>>>>>>>
>>>>>>> ===
>>>>>>>
>>>>>>> Another approach (which is how it's done in one of the Ruby
>>>>>>> templating engine) is to generate a full function definition, whose body
>>>>>>> parses the template and takes the variables as params. And then eval and
>>>>>>> execute the function with its params. However, I'm still struggling with
>>>>>>> the metaprogramming API as for instance parse() chokes on multiple
>>>>>>> lines,
>>>>>>> and I couldn't find a functional equivalent of a quote ... end blocks.
>>>>>>> But
>>>>>>> I'm hoping include_string() will do the trick (must test though).
>>>>>>>
>>>>>>>
>>>>>>> sâmbătă, 13 august 2016, 15:20:01 UTC+2, Yichao Yu a scris:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Aug 13, 2016 at 8:06 PM, Adrian Salceanu <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> That's pretty difficult as my goal is to use embedded Julia as the
>>>>>>>>> templating language. Similar to Ruby's ERB, ex:
>>>>>>>>> http://www.stuartellis.eu/articles/erb/
>>>>>>>>>
>>>>>>>>> So say in the template I have something like
>>>>>>>>>
>>>>>>>>> <% if foo == "bar" %>
>>>>>>>>> Bar
>>>>>>>>> <% else %>
>>>>>>>>> Baz
>>>>>>>>> <% end %>
>>>>>>>>>
>>>>>>>>> The idea is to use Julia itself to parse the code block and Julia
>>>>>>>>> will raise an error is foo is not defined. So I can't really look it
>>>>>>>>> up.
>>>>>>>>>
>>>>>>>>
>>>>>>>> It's ok to use the julia syntax and parser but it's a pretty bad
>>>>>>>> idea to use the julia runtime to actually evaluating the expression,
>>>>>>>> and
>>>>>>>> absolutely not by making them reference to local variables.
>>>>>>>>
>>>>>>>> For a start you are not allowed to reference local variables by
>>>>>>>> names anyway.
>>>>>>>> You also shouldn't allow reference to/overwrite of other local
>>>>>>>> variables (i.e. the template namespace should be fully isolated and
>>>>>>>> independent of any scope in the template engine).
>>>>>>>>
>>>>>>>> Since you want to eval, it seems that efficiency is not an issue,
>>>>>>>> in which case you can create an anonymous module and eval/create
>>>>>>>> globals in
>>>>>>>> that module. This should also be reasonably fast if you are only using
>>>>>>>> the
>>>>>>>> template once.
>>>>>>>>
>>>>>>>> If you want to use it multiple time and compile the template, you
>>>>>>>> should then scan for variable references in the expressions and
>>>>>>>> process it
>>>>>>>> from there.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I can either do
>>>>>>>>>
>>>>>>>>> <% if _[:foo] == "bar" %>
>>>>>>>>>
>>>>>>>>> or
>>>>>>>>>
>>>>>>>>> <% if _(:foo) == "bar" %>
>>>>>>>>>
>>>>>>>>> but it's not that nice.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> sâmbătă, 13 august 2016, 13:24:18 UTC+2, Yichao Yu a scris:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sat, Aug 13, 2016 at 7:13 PM, Adrian Salceanu <
>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>> It's for a templating engine. The user creates the document (a
>>>>>>>>>>> string) which contains interpolated variables placeholders and
>>>>>>>>>>> markup. When
>>>>>>>>>>> the template is rendered, the placeholders must be replaced with the
>>>>>>>>>>> corresponding values from the dict.
>>>>>>>>>>>
>>>>>>>>>>> The lines in the template are eval-ed and so Julia will look for
>>>>>>>>>>> the variables in the scope. So the vars should be already defined.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> You should explicitly look up those variables in the dict instead.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Yes, ultimately I can force the user to use a dict (or rather a
>>>>>>>>>>> function for a bit of semantic sugar) - which is preferable from a
>>>>>>>>>>> performance perspective, but less pretty end error prone from the
>>>>>>>>>>> user
>>>>>>>>>>> perspective.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>
>>
>>
>> --
>> Erik Schnetter <[email protected]> http://www.perimeterinstitute.
>> ca/personal/eschnetter/
>>
>
