The `eval` there is redundant with `include`. But there's nothing wrong with running julia code during compile-time to dynamically generate code. The confusion there comes from the fact that Julia has a compile-step between each toplevel statement, which allows you to intermix runtime and compile. However, if you tried to do the same from inside `@generated`, then there is only portion of a compile-step before the runtime, so some constructs can't be handled.
In v0.6, it's possible we could allow the creation of some types (many types are simple enough they could be constructed without needing to actual execute the definition). However, the fix for #265 will soon bar you from calling the constructor of this type, so I'm not sure what would be gained. > This isn't much of an explanation. I think this would seem like less of a "because I say so" if you provided an explanation of what the problem with calling eval within a generated function is, rather than just an assertion that it can't work. The definition of a generated function is "a pure function which computes a function body based solely upon properties of the input argument types". Since `eval` isn't pure, that would violate the definition of what `@generated` is, and therefore it isn't usable. This isn't supposed to be complicated, what an `@generated` function is actually supposed to be allowed to do is just extremely limited, to make sure the system doesn't fall apart on edge cases. On Wed, Aug 10, 2016 at 6:10 PM Tim Holy <[email protected]> wrote: > AFAICT, it remains possible to do dynamic type generation if you (1) print > the > code that would define the type to a file, and (2) `include` the file. > > function create_type_dynamically{T}(::Type{T}) > type_name = string("MyType", T) > isdefined(Main, Symbol(type_name)) && return nothing > filename = joinpath(tempdir(), string(T)) > open(filename, "w") do io > println(io, """ > type $type_name > val::$T > end > """) > end > eval(include(filename)) > nothing > end > > Is this somehow less evil than doing it in a generated function? > > Best, > --Tim > > On Wednesday, August 10, 2016 9:49:23 PM CDT Jameson Nash wrote: > > > Why is it impossible to generate a new type at run time? I surely can > do > > > > this by calling `eval` at module scope. > > > > module scope is compile time != runtime > > > > > Or I could create a type via a macro. > > > > Again, compile time != runtime > > > > > Given this, I can also call `eval` in a function, if I ensure the > > > > function is called only once. > > > > > Note that I've been doing this in Julia 0.4 without any (apparent) > > > > problems. > > > > Sure, I'm just here to tell you why it won't work that way in v0.5 > > > > > I'm not defining thousands of types in my code. I define one type, and > > > > use it all over the place. However, each time my code runs (for days!), > it > > defines a different type, chosen by a set of user parameters. I'm also > not > > adding constraints to type parameters -- the type parameters are just > `Int` > > values. > > > > Right, the basic tradeoff required here is that you just need to provide > a > > convenient way for your user to declare the type at the toplevel that > will > > be used for the run. For example, you can just JIT the code for the whole > > run at the beginning: > > > > function do_run() > > return @eval begin > > lots of function definitions > > do_work() > > end > > end > > > > On Wed, Aug 10, 2016 at 5:14 PM Erik Schnetter <[email protected]> > wrote: > > > On Wed, Aug 10, 2016 at 1:45 PM, Jameson <[email protected]> wrote: > > >> AFAIK, defining an arbitrary new type at runtime is impossible, > sorry. In > > >> v0.4 it was allowed, because we hoped that people understood not to > try. > > >> See also https://github.com/JuliaLang/julia/issues/16806. Note that > it > > >> is insufficient to "handle" the repeat calling via caching in a Dict > or > > >> similar such mechanism. It must always compute the exact final output > > >> from > > >> the input values alone (e.g. it must truly be const pure). > > > > > > The generated function first calculates the name of the type, then > checks > > > (`isdefined`) if this type is defined, and if so, returns it. > Otherwise it > > > is defined and then returned. This corresponds to looking up the type > via > > > `eval(typename)` (a symbol). I assume this is as pure as it gets. > > > > > > Why is it impossible to generate a new type at run time? I surely can > do > > > this by calling `eval` at module scope. Or I could create a type via a > > > macro. Given this, I can also call `eval` in a function, if I ensure > the > > > function is called only once. Note that I've been doing this in Julia > 0.4 > > > without any (apparent) problems. > > > > > > Being able to define types with arbitrary constraints in the type > > > > > >> parameters works OK for toy demos, but it's intentionally rather > > >> difficult > > >> since it causes performance issues at scale. Operations on Array are > > >> likely > > >> to be much faster (including the allocation) than on Tuple (due to the > > >> cost > > >> of *not* allocating) unless that Tuple is very small. > > > > > > I'm not defining thousands of types in my code. I define one type, and > use > > > it all over the place. However, each time my code runs (for days!), it > > > defines a different type, chosen by a set of user parameters. I'm also > not > > > adding constraints to type parameters -- the type parameters are just > > > `Int` > > > values. > > > > > > And yes, I am using a mutable `Vector{T}` as underlying storage, that's > > > not the issue here. The speedup comes from knowing the size of the > array > > > ahead of time, which allows the compiler to optimize indexing > expressions. > > > I've benchmarked it, and examined the generated machine code. There's > no > > > doubt that generating a type is the "right thing" to do in this case. > > > > > > -erik > > > > > > On Wednesday, August 10, 2016 at 1:25:15 PM UTC-4, Erik Schnetter > wrote: > > >>> I want to create a type, and need more flexibility than Julia's > `type` > > >>> definitions offer (see <https://github.com/eschnett/FastArrays.jl>). > > >>> Currently, I have a function that generates the type, and returns the > > >>> type. > > >>> > > >>> I would like to make this a generated function (as it was in Julia > 0.4). > > >>> The advantage is that this leads to type stability: The generated > type > > >>> only > > >>> depends on the types of the arguments pass to the function, and Julia > > >>> would > > >>> be able to infer the type. > > >>> > > >>> In practice, this looks like > > >>> > > >>> using FastArrays > > >>> # A (10x10) fixed-size arraytypealias Arr2d_10x10 FastArray(1:10, > 1:10) > > >>> a2 = Arr2d_10x10{Float64}(:,:) > > >>> > > >>> > > >>> In principle I'd like to write `FastArray{1:10, 1:10}` (with curly > > >>> braces), but Julia doesn't offer sufficient flexibility for this. > Hence > > >>> I > > >>> use a regular function. > > >>> > > >>> To generate the type in the function I need to call `eval`. (Yes, I'm > > >>> aware that the function might be called multiple times, and I'm > handling > > >>> this.) > > >>> > > >>> Do you have a suggestion for a different solution? > > >>> > > >>> -erik > > >>> > > >>> On Wed, Aug 10, 2016 at 11:51 AM, Jameson <[email protected]> wrote: > > >>>> It is tracking the dynamic scope of the code generator, it doesn't > care > > >>>> about what code you emit. The generator function must not cause any > > >>>> side-effects and must be entirely computed from the types of the > inputs > > >>>> and > > >>>> not other global state. Over time, these conditions are likely to be > > >>>> more > > >>>> accurately enforced, as needed to make various optimizations > reliable > > >>>> and/or correct. > > >>>> > > >>>> > > >>>> > > >>>> On Wednesday, August 10, 2016 at 10:48:31 AM UTC-4, Erik Schnetter > > >>>> > > >>>> wrote: > > >>>>> I'm encountering the error "eval cannot be used in a generated > > >>>>> function" in Julia 0.5 for code that is working in Julia 0.4. My > > >>>>> question > > >>>>> is -- what exactly is now disallowed? For example, if a generated > > >>>>> function > > >>>>> `f` calls another (non-generated) function `g`, can `g` then call > > >>>>> `eval`? > > >>>>> Does the word "in" here refer to the code that is generated by the > > >>>>> generated function, or does it refer to the dynamical scope of the > > >>>>> code > > >>>>> generation state of the generated function? > > >>>>> > > >>>>> To avoid the error I have to redesign my code, and I'd like to know > > >>>>> ahead of time what to avoid. A Google search only turned up the C > file > > >>>>> within Julia that emits the respective error message, as well as > the > > >>>>> Travis > > >>>>> build log for my package. > > >>>>> > > >>>>> -erik > > >>>>> > > >>>>> -- > > >>>>> Erik Schnetter <[email protected]> > > >>>>> http://www.perimeterinstitute.ca/personal/eschnetter/ > > >>> > > >>> -- > > >>> Erik Schnetter <[email protected]> > > >>> http://www.perimeterinstitute.ca/personal/eschnetter/ > > > > > > -- > > > Erik Schnetter <[email protected]> > > > http://www.perimeterinstitute.ca/personal/eschnetter/ > > >
