Here is the code for the examples: https://github.com/essenciary/ejl/blob/master/2.jl
luni, 15 august 2016, 10:20:18 UTC+2, Adrian Salceanu a scris: > > Erik, thank you very much. > > In a few lines, here is the use case (for a standard MVC web app): > > 1. the user defines some template code which is stored in a view file. > Think HTML with embedded Julia code for interpolating variables, if/else, > loops, etc. > A very basic example would look like this: > > <html> > <head> > <title> > %= page_title > </title> > </head> > <body> > <h1> > %= greeting > </h1> > <p> > <% if is_logged_in :> > Edit account > <: else :> > Login > <: end %> > </p> > </body> > </html> > > 2. there are some very simple parsing rules, such as: > a. "%=" means the line should be interpreted as Julia and outputted, > b. "<% ... %>" defines a multiline Julia block > c. everything else should be considered raw output > > 3. the variables used in the template have to be defined / set beforehand > by the user, usually in a controller file - and somehow be made available > to the template rendering function > > --- > > This means that: > > a. I do not know the variables beforehand, they are user inputed as part > of the template. > > b. also, it's not an issue of "This is indeed one of the strengths of > Julia, and it requires neither string manipulation, nor parsing, nor > creating files nor include.". I can not avoid creating, loading and parsing > files, because that's what I want to do. I want to allow the users to > create view files that are basically HTML files with embedded Julia code. > > b. I'm looking for a way to load and interpret multiline template code > stored in a file. This seems to be tricky by itself as parse(...) works > line by line (not good for me, I need to interpret the whole block at once, > for example to handle if/else blocks) and I could not find a functional > alternative for quote ... end or a way to pass a string into a quote ... > end block. Like I said, the only thing close to achieving this would be > include_string() > > c. I'd like to do this in a performant manner by avoiding the dreaded > global scope (but not really by avoiding eval, which I think it's a > necessary evil in this case). > > --- > > Strictly addressing your questions: > > If the variables are declared in a function, then there must be code > that uses them; how would this code know which variables exist? Is there > e.g. a globally known list of variables that will be provided? Or will the > variables e.g. only be used in strings to generate html code? > > I go over the template file line by line and generate Julia code (as a > string). This code is either logic (if/else, loops, etc) or raw output. For > instance, the parsing of the above template would generate the following > Julia code: > > "____output = Vector{AbstractString}()\npush!(____output, > \"<html>\")\npush!(____output, \" <head>\")\npush!(____output, \" > <title>\")\n____L9OHE6m05ZtELQIOMV8PPH0L = page_title\npush!(____output, > \"\$(____L9OHE6m05ZtELQIOMV8PPH0L)\")\npush!(____output, \" > </title>\")\npush!(____output, \" </head>\")\npush!(____output, \" > <body>\")\npush!(____output, \" <h1>\")\n____wHbq5V7InZ2OJKpatWu7MtUU = > greeting\npush!(____output, > \"\$(____wHbq5V7InZ2OJKpatWu7MtUU)\")\npush!(____output, \" > </h1>\")\npush!(____output, \" <p>\")\nif > is_logged_in\npush!(____output, \" Edit > account\")\nelse\npush!(____output, \" > Login\")\nend\npush!(____output, \" </p>\")\npush!(____output, \" > </body>\")\npush!(____output, \"</html>\")\npush!(____output, \"\")" > > Then this code is eval'd - at this time, the variables should already be > in scope otherwise Julia won't find them. > > Yes, the variables are only used to generate HTML code. So ideally they > should only be visible in the minimum scope necessary to eval this code. > > > Can you elaborate on why you want to access dictionary elements as local > variables? > > I was hoping this could be an approach to eval the code with its vars in a > non-global scope. Something in the lines of: > > function render_tpl(output::AbstractString, vars::Dict{Symbol,Any}) # > output would be the above generated code and vars the list of vars used in > the template > # code here to expand the vars dict into function local variables > # code here to eval the output (which would be the "____output = > Vector{..." code above) -- this would automatically have access to the vars > # return the resulted ____output Vector which now contains the rendered > template > end > > Thanks, > Adrian > > duminică, 14 august 2016, 17:59:40 UTC+2, Erik Schnetter a scris: >> >> Adrian >> >> Can you give more details for how the variables would be used? If the >> variables are declared in a function, then there must be code that uses >> them; how would this code know which variables exist? Is there e.g. a >> globally known list of variables that will be provided? Or will the >> variables e.g. only be used in strings to generate html code? >> >> You can generate the syntax tree for function that contains the >> definitions from the dictionary, and then use `eval` to create the >> function. (Alternatively, you can use a macro or a generated function.) >> This is indeed one of the strengths of Julia, and it requires neither >> string manipulation, nor parsing, nor creating files nor include. >> >> Here is an example: >> >> fun_expr(var, val) = quote >> function f(x) >> $var = $val >> x + y >> end >> end >> >> eval(fun_expr(:y, :42)) >> >> `fun_expr` creates a syntax tree (an `Expr`) that defines a function. The >> function is quoted. The function arguments `var` and `val` are inserted >> into that function, forming an assignment. Thus `var` better be an >> identifier (or something else that can be on the left of an assignment >> operator), and `val` can be an arbitrary value. Given how the function >> looks, `var` essentially has to be `y` since it's used in the next line; of >> course, this was just my arbitrary choice. >> >> The call to `eval` then passes the respective arguments, choosing the >> symbol `y` for the variable name, and the value `42` as value, and defines >> the function `f`. If you then call `f(2)`, the result is 44. >> >> Of course, you can define this function `f` only once. Julia does not >> allow changing functions. If you want to create many different functions, >> you would use anonymous functions instead. >> >> Having said this -- it's not clear that this is indeed the right way to >> address your problem; there might be a better solution. Can you elaborate >> on why you want to access dictionary elements as local variables? >> >> -erik >> >> >> >> On Sun, Aug 14, 2016 at 11:13 AM, Adrian Salceanu <adrian....@gmail.com> >> wrote: >> >>> Thanks >>> >>> Maybe I wasn't clear enough - otherwise, can you please elaborate, I'm >>> definitely still poking around, any clarifications would be highly >>> appreciated. >>> >>> > creating a new module >>> -> the module is available at compile time (the users of the templating >>> system will place the vars in there, by convention). >>> >>> > parse julia code >>> -> it's not really parsing julia code, it has no meaning at the point. >>> It's simply basic string processing and it's fast - tried with a 10K lines >>> HTML file, no sweat. >>> >>> > defining globals >>> -> why are they globals? include_string() is used inside a function, >>> inside a module within the app. >>> >>> > eval in a module >>> -> true, but then what can we do? That's the way of doing >>> metaprogramming in Julia, and it's widely used, isn't it? >>> I guess that would be the price for not having to do >>> print("<html><head>...") like our ancestors used to in PHP or ASP, when not >>> being chased by tigers (or something around that age). >>> >>> >>> duminică, 14 august 2016, 15:01:47 UTC+2, Yichao Yu a scris: >>>> >>>> >>>> >>>> On Sun, Aug 14, 2016 at 6:13 PM, Adrian Salceanu <adrian....@gmail.com> >>>> wrote: >>>> >>>>> Variables contained in a module and then parsed Julia code included >>>>> within a function using include_string(). >>>>> >>>>> Any obvious performance issues with this approach? >>>>> >>>> >>>> Everything about it? >>>> Literally every steps are hitting the slow path that is only meant to >>>> execute at compile time and not runtime. Including >>>> >>>> Creating a new module, parse julia code, eval in a module, defining >>>> globals. >>>> >>>> >>>>> >>>>> >>>>> duminică, 14 august 2016, 12:11:06 UTC+2, Adrian Salceanu a scris: >>>>>> >>>>>> OK, actually, that's not nearly half as bad. Variables contained in a >>>>>> module >>>>>> >>>>>> include("src/Ejl_str.jl") >>>>>> using Ejl >>>>>> >>>>>> >>>>>> module _ >>>>>> couñtry = "España" >>>>>> lang = "en" >>>>>> end >>>>>> >>>>>> >>>>>> function render_template() >>>>>> tpl_data = ejl""" >>>>>> <% if _.lang == "en" :> >>>>>> Hello from me, ... >>>>>> <: else :> >>>>>> Hola >>>>>> <: end %> >>>>>> >>>>>> >>>>>> %= _.couñtry == "España" ? "Olé" : "Aye" >>>>>> moo >>>>>> """ >>>>>> >>>>>> >>>>>> include_string(join(tpl_data, "\n")) >>>>>> join(____output, "\n") >>>>>> end >>>>>> >>>>>> >>>>>> render_template() |> println >>>>>> >>>>>> Hello from me, ... >>>>>> >>>>>> >>>>>> Olé >>>>>> moo >>>>>> >>>>>> >>>>>> duminică, 14 august 2016, 11:20:37 UTC+2, Adrian Salceanu a scris: >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> Yes, I've thought about a few ways to mitigate some of these issues: >>>>>>> >>>>>>> 1. in the app I can setup a module (like Render) and evaluate into >>>>>>> this module exclusively. >>>>>>> Hence, another approach might be to have some helper methods that >>>>>>> setup the variables in the module and then eval the template itself >>>>>>> inside >>>>>>> the module too (must try though). So something in the lines of: >>>>>>> set(:foo, "foo") >>>>>>> set(:bar, [1, 2, 3]) >>>>>>> parse_tpl(ejl"""$(foo) and $(bar)""") >>>>>>> # all the above gets parsed in Render >>>>>>> >>>>>>> 2. break the parsing in 2 steps: >>>>>>> a. reading the template string and parse it to generated Julia code >>>>>>> (as strings) (an array of Julia code lines) - cache it >>>>>>> b. (load strings from cache and) eval the code together with the >>>>>>> vars >>>>>>> >>>>>>> === >>>>>>> >>>>>>> Another approach (which is how it's done in one of the Ruby >>>>>>> templating engine) is to generate a full function definition, whose >>>>>>> body >>>>>>> parses the template and takes the variables as params. And then eval >>>>>>> and >>>>>>> execute the function with its params. However, I'm still struggling >>>>>>> with >>>>>>> the metaprogramming API as for instance parse() chokes on multiple >>>>>>> lines, >>>>>>> and I couldn't find a functional equivalent of a quote ... end blocks. >>>>>>> But >>>>>>> I'm hoping include_string() will do the trick (must test though). >>>>>>> >>>>>>> >>>>>>> sâmbătă, 13 august 2016, 15:20:01 UTC+2, Yichao Yu a scris: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Aug 13, 2016 at 8:06 PM, Adrian Salceanu < >>>>>>>> adrian....@gmail.com> wrote: >>>>>>>> >>>>>>>>> That's pretty difficult as my goal is to use embedded Julia as the >>>>>>>>> templating language. Similar to Ruby's ERB, ex: >>>>>>>>> http://www.stuartellis.eu/articles/erb/ >>>>>>>>> >>>>>>>>> So say in the template I have something like >>>>>>>>> >>>>>>>>> <% if foo == "bar" %> >>>>>>>>> Bar >>>>>>>>> <% else %> >>>>>>>>> Baz >>>>>>>>> <% end %> >>>>>>>>> >>>>>>>>> The idea is to use Julia itself to parse the code block and Julia >>>>>>>>> will raise an error is foo is not defined. So I can't really look it >>>>>>>>> up. >>>>>>>>> >>>>>>>> >>>>>>>> It's ok to use the julia syntax and parser but it's a pretty bad >>>>>>>> idea to use the julia runtime to actually evaluating the expression, >>>>>>>> and >>>>>>>> absolutely not by making them reference to local variables. >>>>>>>> >>>>>>>> For a start you are not allowed to reference local variables by >>>>>>>> names anyway. >>>>>>>> You also shouldn't allow reference to/overwrite of other local >>>>>>>> variables (i.e. the template namespace should be fully isolated and >>>>>>>> independent of any scope in the template engine). >>>>>>>> >>>>>>>> Since you want to eval, it seems that efficiency is not an issue, >>>>>>>> in which case you can create an anonymous module and eval/create >>>>>>>> globals in >>>>>>>> that module. This should also be reasonably fast if you are only using >>>>>>>> the >>>>>>>> template once. >>>>>>>> >>>>>>>> If you want to use it multiple time and compile the template, you >>>>>>>> should then scan for variable references in the expressions and >>>>>>>> process it >>>>>>>> from there. >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> I can either do >>>>>>>>> >>>>>>>>> <% if _[:foo] == "bar" %> >>>>>>>>> >>>>>>>>> or >>>>>>>>> >>>>>>>>> <% if _(:foo) == "bar" %> >>>>>>>>> >>>>>>>>> but it's not that nice. >>>>>>>>> >>>>>>>>> >>>>>>>>> sâmbătă, 13 august 2016, 13:24:18 UTC+2, Yichao Yu a scris: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sat, Aug 13, 2016 at 7:13 PM, Adrian Salceanu < >>>>>>>>>> adrian....@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> It's for a templating engine. The user creates the document (a >>>>>>>>>>> string) which contains interpolated variables placeholders and >>>>>>>>>>> markup. When >>>>>>>>>>> the template is rendered, the placeholders must be replaced with >>>>>>>>>>> the >>>>>>>>>>> corresponding values from the dict. >>>>>>>>>>> >>>>>>>>>>> The lines in the template are eval-ed and so Julia will look for >>>>>>>>>>> the variables in the scope. So the vars should be already defined. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> You should explicitly look up those variables in the dict instead. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yes, ultimately I can force the user to use a dict (or rather a >>>>>>>>>>> function for a bit of semantic sugar) - which is preferable from a >>>>>>>>>>> performance perspective, but less pretty end error prone from the >>>>>>>>>>> user >>>>>>>>>>> perspective. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>> >> >> >> -- >> Erik Schnetter <schn...@gmail.com> >> http://www.perimeterinstitute.ca/personal/eschnetter/ >> >