AFAICT, it remains possible to do dynamic type generation if you (1) print the
code that would define the type to a file, and (2) `include` the file.
function create_type_dynamically{T}(::Type{T})
type_name = string("MyType", T)
isdefined(Main, Symbol(type_name)) && return nothing
filename = joinpath(tempdir(), string(T))
open(filename, "w") do io
println(io, """
type $type_name
val::$T
end
""")
end
eval(include(filename))
nothing
end
Is this somehow less evil than doing it in a generated function?
Best,
--Tim
On Wednesday, August 10, 2016 9:49:23 PM CDT Jameson Nash wrote:
> > Why is it impossible to generate a new type at run time? I surely can do
>
> this by calling `eval` at module scope.
>
> module scope is compile time != runtime
>
> > Or I could create a type via a macro.
>
> Again, compile time != runtime
>
> > Given this, I can also call `eval` in a function, if I ensure the
>
> function is called only once.
>
> > Note that I've been doing this in Julia 0.4 without any (apparent)
>
> problems.
>
> Sure, I'm just here to tell you why it won't work that way in v0.5
>
> > I'm not defining thousands of types in my code. I define one type, and
>
> use it all over the place. However, each time my code runs (for days!), it
> defines a different type, chosen by a set of user parameters. I'm also not
> adding constraints to type parameters -- the type parameters are just `Int`
> values.
>
> Right, the basic tradeoff required here is that you just need to provide a
> convenient way for your user to declare the type at the toplevel that will
> be used for the run. For example, you can just JIT the code for the whole
> run at the beginning:
>
> function do_run()
> return @eval begin
> lots of function definitions
> do_work()
> end
> end
>
> On Wed, Aug 10, 2016 at 5:14 PM Erik Schnetter <schnet...@gmail.com> wrote:
> > On Wed, Aug 10, 2016 at 1:45 PM, Jameson <vtjn...@gmail.com> wrote:
> >> AFAIK, defining an arbitrary new type at runtime is impossible, sorry. In
> >> v0.4 it was allowed, because we hoped that people understood not to try.
> >> See also https://github.com/JuliaLang/julia/issues/16806. Note that it
> >> is insufficient to "handle" the repeat calling via caching in a Dict or
> >> similar such mechanism. It must always compute the exact final output
> >> from
> >> the input values alone (e.g. it must truly be const pure).
> >
> > The generated function first calculates the name of the type, then checks
> > (`isdefined`) if this type is defined, and if so, returns it. Otherwise it
> > is defined and then returned. This corresponds to looking up the type via
> > `eval(typename)` (a symbol). I assume this is as pure as it gets.
> >
> > Why is it impossible to generate a new type at run time? I surely can do
> > this by calling `eval` at module scope. Or I could create a type via a
> > macro. Given this, I can also call `eval` in a function, if I ensure the
> > function is called only once. Note that I've been doing this in Julia 0.4
> > without any (apparent) problems.
> >
> > Being able to define types with arbitrary constraints in the type
> >
> >> parameters works OK for toy demos, but it's intentionally rather
> >> difficult
> >> since it causes performance issues at scale. Operations on Array are
> >> likely
> >> to be much faster (including the allocation) than on Tuple (due to the
> >> cost
> >> of *not* allocating) unless that Tuple is very small.
> >
> > I'm not defining thousands of types in my code. I define one type, and use
> > it all over the place. However, each time my code runs (for days!), it
> > defines a different type, chosen by a set of user parameters. I'm also not
> > adding constraints to type parameters -- the type parameters are just
> > `Int`
> > values.
> >
> > And yes, I am using a mutable `Vector{T}` as underlying storage, that's
> > not the issue here. The speedup comes from knowing the size of the array
> > ahead of time, which allows the compiler to optimize indexing expressions.
> > I've benchmarked it, and examined the generated machine code. There's no
> > doubt that generating a type is the "right thing" to do in this case.
> >
> > -erik
> >
> > On Wednesday, August 10, 2016 at 1:25:15 PM UTC-4, Erik Schnetter wrote:
> >>> I want to create a type, and need more flexibility than Julia's `type`
> >>> definitions offer (see <https://github.com/eschnett/FastArrays.jl>).
> >>> Currently, I have a function that generates the type, and returns the
> >>> type.
> >>>
> >>> I would like to make this a generated function (as it was in Julia 0.4).
> >>> The advantage is that this leads to type stability: The generated type
> >>> only
> >>> depends on the types of the arguments pass to the function, and Julia
> >>> would
> >>> be able to infer the type.
> >>>
> >>> In practice, this looks like
> >>>
> >>> using FastArrays
> >>> # A (10x10) fixed-size arraytypealias Arr2d_10x10 FastArray(1:10, 1:10)
> >>> a2 = Arr2d_10x10{Float64}(:,:)
> >>>
> >>>
> >>> In principle I'd like to write `FastArray{1:10, 1:10}` (with curly
> >>> braces), but Julia doesn't offer sufficient flexibility for this. Hence
> >>> I
> >>> use a regular function.
> >>>
> >>> To generate the type in the function I need to call `eval`. (Yes, I'm
> >>> aware that the function might be called multiple times, and I'm handling
> >>> this.)
> >>>
> >>> Do you have a suggestion for a different solution?
> >>>
> >>> -erik
> >>>
> >>> On Wed, Aug 10, 2016 at 11:51 AM, Jameson <vtjn...@gmail.com> wrote:
> >>>> It is tracking the dynamic scope of the code generator, it doesn't care
> >>>> about what code you emit. The generator function must not cause any
> >>>> side-effects and must be entirely computed from the types of the inputs
> >>>> and
> >>>> not other global state. Over time, these conditions are likely to be
> >>>> more
> >>>> accurately enforced, as needed to make various optimizations reliable
> >>>> and/or correct.
> >>>>
> >>>>
> >>>>
> >>>> On Wednesday, August 10, 2016 at 10:48:31 AM UTC-4, Erik Schnetter
> >>>>
> >>>> wrote:
> >>>>> I'm encountering the error "eval cannot be used in a generated
> >>>>> function" in Julia 0.5 for code that is working in Julia 0.4. My
> >>>>> question
> >>>>> is -- what exactly is now disallowed? For example, if a generated
> >>>>> function
> >>>>> `f` calls another (non-generated) function `g`, can `g` then call
> >>>>> `eval`?
> >>>>> Does the word "in" here refer to the code that is generated by the
> >>>>> generated function, or does it refer to the dynamical scope of the
> >>>>> code
> >>>>> generation state of the generated function?
> >>>>>
> >>>>> To avoid the error I have to redesign my code, and I'd like to know
> >>>>> ahead of time what to avoid. A Google search only turned up the C file
> >>>>> within Julia that emits the respective error message, as well as the
> >>>>> Travis
> >>>>> build log for my package.
> >>>>>
> >>>>> -erik
> >>>>>
> >>>>> --
> >>>>> Erik Schnetter <schnet...@gmail.com>
> >>>>> http://www.perimeterinstitute.ca/personal/eschnetter/
> >>>
> >>> --
> >>> Erik Schnetter <schnet...@gmail.com>
> >>> http://www.perimeterinstitute.ca/personal/eschnetter/
> >
> > --
> > Erik Schnetter <schnet...@gmail.com>
> > http://www.perimeterinstitute.ca/personal/eschnetter/
