The upshot of the discussions seems to be "it won't work in 0.5 because the
experts say so, and there are no plans to change that". So I'm going to
accept that statement.
I think I'll use the following work-around:
```Julia
immutable Wrapper{Tag,Types}
data::Types
end
```
where I use `Tag` as `Val{Symbol}` to generate many distinct types, and
`Types` will be a tuple. This allows me to "generate" any immutable type. I
won't be able to access fields via the `.fieldname` syntax, but that is not
important to me.
-erik
On Wed, Aug 10, 2016 at 6:09 PM, Tim Holy <[email protected]> wrote:
> AFAICT, it remains possible to do dynamic type generation if you (1) print
> the
> code that would define the type to a file, and (2) `include` the file.
>
> function create_type_dynamically{T}(::Type{T})
> type_name = string("MyType", T)
> isdefined(Main, Symbol(type_name)) && return nothing
> filename = joinpath(tempdir(), string(T))
> open(filename, "w") do io
> println(io, """
> type $type_name
> val::$T
> end
> """)
> end
> eval(include(filename))
> nothing
> end
>
> Is this somehow less evil than doing it in a generated function?
>
> Best,
> --Tim
>
> On Wednesday, August 10, 2016 9:49:23 PM CDT Jameson Nash wrote:
> > > Why is it impossible to generate a new type at run time? I surely can
> do
> >
> > this by calling `eval` at module scope.
> >
> > module scope is compile time != runtime
> >
> > > Or I could create a type via a macro.
> >
> > Again, compile time != runtime
> >
> > > Given this, I can also call `eval` in a function, if I ensure the
> >
> > function is called only once.
> >
> > > Note that I've been doing this in Julia 0.4 without any (apparent)
> >
> > problems.
> >
> > Sure, I'm just here to tell you why it won't work that way in v0.5
> >
> > > I'm not defining thousands of types in my code. I define one type, and
> >
> > use it all over the place. However, each time my code runs (for days!),
> it
> > defines a different type, chosen by a set of user parameters. I'm also
> not
> > adding constraints to type parameters -- the type parameters are just
> `Int`
> > values.
> >
> > Right, the basic tradeoff required here is that you just need to provide
> a
> > convenient way for your user to declare the type at the toplevel that
> will
> > be used for the run. For example, you can just JIT the code for the whole
> > run at the beginning:
> >
> > function do_run()
> > return @eval begin
> > lots of function definitions
> > do_work()
> > end
> > end
> >
> > On Wed, Aug 10, 2016 at 5:14 PM Erik Schnetter <[email protected]>
> wrote:
> > > On Wed, Aug 10, 2016 at 1:45 PM, Jameson <[email protected]> wrote:
> > >> AFAIK, defining an arbitrary new type at runtime is impossible,
> sorry. In
> > >> v0.4 it was allowed, because we hoped that people understood not to
> try.
> > >> See also https://github.com/JuliaLang/julia/issues/16806. Note that
> it
> > >> is insufficient to "handle" the repeat calling via caching in a Dict
> or
> > >> similar such mechanism. It must always compute the exact final output
> > >> from
> > >> the input values alone (e.g. it must truly be const pure).
> > >
> > > The generated function first calculates the name of the type, then
> checks
> > > (`isdefined`) if this type is defined, and if so, returns it.
> Otherwise it
> > > is defined and then returned. This corresponds to looking up the type
> via
> > > `eval(typename)` (a symbol). I assume this is as pure as it gets.
> > >
> > > Why is it impossible to generate a new type at run time? I surely can
> do
> > > this by calling `eval` at module scope. Or I could create a type via a
> > > macro. Given this, I can also call `eval` in a function, if I ensure
> the
> > > function is called only once. Note that I've been doing this in Julia
> 0.4
> > > without any (apparent) problems.
> > >
> > > Being able to define types with arbitrary constraints in the type
> > >
> > >> parameters works OK for toy demos, but it's intentionally rather
> > >> difficult
> > >> since it causes performance issues at scale. Operations on Array are
> > >> likely
> > >> to be much faster (including the allocation) than on Tuple (due to the
> > >> cost
> > >> of *not* allocating) unless that Tuple is very small.
> > >
> > > I'm not defining thousands of types in my code. I define one type, and
> use
> > > it all over the place. However, each time my code runs (for days!), it
> > > defines a different type, chosen by a set of user parameters. I'm also
> not
> > > adding constraints to type parameters -- the type parameters are just
> > > `Int`
> > > values.
> > >
> > > And yes, I am using a mutable `Vector{T}` as underlying storage, that's
> > > not the issue here. The speedup comes from knowing the size of the
> array
> > > ahead of time, which allows the compiler to optimize indexing
> expressions.
> > > I've benchmarked it, and examined the generated machine code. There's
> no
> > > doubt that generating a type is the "right thing" to do in this case.
> > >
> > > -erik
> > >
> > > On Wednesday, August 10, 2016 at 1:25:15 PM UTC-4, Erik Schnetter
> wrote:
> > >>> I want to create a type, and need more flexibility than Julia's
> `type`
> > >>> definitions offer (see <https://github.com/eschnett/FastArrays.jl>).
> > >>> Currently, I have a function that generates the type, and returns the
> > >>> type.
> > >>>
> > >>> I would like to make this a generated function (as it was in Julia
> 0.4).
> > >>> The advantage is that this leads to type stability: The generated
> type
> > >>> only
> > >>> depends on the types of the arguments pass to the function, and Julia
> > >>> would
> > >>> be able to infer the type.
> > >>>
> > >>> In practice, this looks like
> > >>>
> > >>> using FastArrays
> > >>> # A (10x10) fixed-size arraytypealias Arr2d_10x10 FastArray(1:10,
> 1:10)
> > >>> a2 = Arr2d_10x10{Float64}(:,:)
> > >>>
> > >>>
> > >>> In principle I'd like to write `FastArray{1:10, 1:10}` (with curly
> > >>> braces), but Julia doesn't offer sufficient flexibility for this.
> Hence
> > >>> I
> > >>> use a regular function.
> > >>>
> > >>> To generate the type in the function I need to call `eval`. (Yes, I'm
> > >>> aware that the function might be called multiple times, and I'm
> handling
> > >>> this.)
> > >>>
> > >>> Do you have a suggestion for a different solution?
> > >>>
> > >>> -erik
> > >>>
> > >>> On Wed, Aug 10, 2016 at 11:51 AM, Jameson <[email protected]> wrote:
> > >>>> It is tracking the dynamic scope of the code generator, it doesn't
> care
> > >>>> about what code you emit. The generator function must not cause any
> > >>>> side-effects and must be entirely computed from the types of the
> inputs
> > >>>> and
> > >>>> not other global state. Over time, these conditions are likely to be
> > >>>> more
> > >>>> accurately enforced, as needed to make various optimizations
> reliable
> > >>>> and/or correct.
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Wednesday, August 10, 2016 at 10:48:31 AM UTC-4, Erik Schnetter
> > >>>>
> > >>>> wrote:
> > >>>>> I'm encountering the error "eval cannot be used in a generated
> > >>>>> function" in Julia 0.5 for code that is working in Julia 0.4. My
> > >>>>> question
> > >>>>> is -- what exactly is now disallowed? For example, if a generated
> > >>>>> function
> > >>>>> `f` calls another (non-generated) function `g`, can `g` then call
> > >>>>> `eval`?
> > >>>>> Does the word "in" here refer to the code that is generated by the
> > >>>>> generated function, or does it refer to the dynamical scope of the
> > >>>>> code
> > >>>>> generation state of the generated function?
> > >>>>>
> > >>>>> To avoid the error I have to redesign my code, and I'd like to know
> > >>>>> ahead of time what to avoid. A Google search only turned up the C
> file
> > >>>>> within Julia that emits the respective error message, as well as
> the
> > >>>>> Travis
> > >>>>> build log for my package.
> > >>>>>
> > >>>>> -erik
> > >>>>>
> > >>>>> --
> > >>>>> Erik Schnetter <[email protected]>
> > >>>>> http://www.perimeterinstitute.ca/personal/eschnetter/
> > >>>
> > >>> --
> > >>> Erik Schnetter <[email protected]>
> > >>> http://www.perimeterinstitute.ca/personal/eschnetter/
> > >
> > > --
> > > Erik Schnetter <[email protected]>
> > > http://www.perimeterinstitute.ca/personal/eschnetter/
>
>
>
--
Erik Schnetter <[email protected]>
http://www.perimeterinstitute.ca/personal/eschnetter/
