> Why is it impossible to generate a new type at run time? I surely can do
this by calling `eval` at module scope.
module scope is compile time != runtime
> Or I could create a type via a macro.
Again, compile time != runtime
> Given this, I can also call `eval` in a function, if I ensure the
function is called only once.
> Note that I've been doing this in Julia 0.4 without any (apparent)
problems.
Sure, I'm just here to tell you why it won't work that way in v0.5
> I'm not defining thousands of types in my code. I define one type, and
use it all over the place. However, each time my code runs (for days!), it
defines a different type, chosen by a set of user parameters. I'm also not
adding constraints to type parameters -- the type parameters are just `Int`
values.
Right, the basic tradeoff required here is that you just need to provide a
convenient way for your user to declare the type at the toplevel that will
be used for the run. For example, you can just JIT the code for the whole
run at the beginning:
function do_run()
return @eval begin
lots of function definitions
do_work()
end
end
On Wed, Aug 10, 2016 at 5:14 PM Erik Schnetter <[email protected]> wrote:
> On Wed, Aug 10, 2016 at 1:45 PM, Jameson <[email protected]> wrote:
>
>> AFAIK, defining an arbitrary new type at runtime is impossible, sorry. In
>> v0.4 it was allowed, because we hoped that people understood not to try.
>> See also https://github.com/JuliaLang/julia/issues/16806. Note that it
>> is insufficient to "handle" the repeat calling via caching in a Dict or
>> similar such mechanism. It must always compute the exact final output from
>> the input values alone (e.g. it must truly be const pure).
>>
>
> The generated function first calculates the name of the type, then checks
> (`isdefined`) if this type is defined, and if so, returns it. Otherwise it
> is defined and then returned. This corresponds to looking up the type via
> `eval(typename)` (a symbol). I assume this is as pure as it gets.
>
> Why is it impossible to generate a new type at run time? I surely can do
> this by calling `eval` at module scope. Or I could create a type via a
> macro. Given this, I can also call `eval` in a function, if I ensure the
> function is called only once. Note that I've been doing this in Julia 0.4
> without any (apparent) problems.
>
> Being able to define types with arbitrary constraints in the type
>> parameters works OK for toy demos, but it's intentionally rather difficult
>> since it causes performance issues at scale. Operations on Array are likely
>> to be much faster (including the allocation) than on Tuple (due to the cost
>> of *not* allocating) unless that Tuple is very small.
>>
>
> I'm not defining thousands of types in my code. I define one type, and use
> it all over the place. However, each time my code runs (for days!), it
> defines a different type, chosen by a set of user parameters. I'm also not
> adding constraints to type parameters -- the type parameters are just `Int`
> values.
>
> And yes, I am using a mutable `Vector{T}` as underlying storage, that's
> not the issue here. The speedup comes from knowing the size of the array
> ahead of time, which allows the compiler to optimize indexing expressions.
> I've benchmarked it, and examined the generated machine code. There's no
> doubt that generating a type is the "right thing" to do in this case.
>
> -erik
>
> On Wednesday, August 10, 2016 at 1:25:15 PM UTC-4, Erik Schnetter wrote:
>>>
>>> I want to create a type, and need more flexibility than Julia's `type`
>>> definitions offer (see <https://github.com/eschnett/FastArrays.jl>).
>>> Currently, I have a function that generates the type, and returns the type.
>>>
>>> I would like to make this a generated function (as it was in Julia 0.4).
>>> The advantage is that this leads to type stability: The generated type only
>>> depends on the types of the arguments pass to the function, and Julia would
>>> be able to infer the type.
>>>
>>> In practice, this looks like
>>>
>>> using FastArrays
>>> # A (10x10) fixed-size arraytypealias Arr2d_10x10 FastArray(1:10, 1:10)
>>> a2 = Arr2d_10x10{Float64}(:,:)
>>>
>>>
>>> In principle I'd like to write `FastArray{1:10, 1:10}` (with curly
>>> braces), but Julia doesn't offer sufficient flexibility for this. Hence I
>>> use a regular function.
>>>
>>> To generate the type in the function I need to call `eval`. (Yes, I'm
>>> aware that the function might be called multiple times, and I'm handling
>>> this.)
>>>
>>> Do you have a suggestion for a different solution?
>>>
>>> -erik
>>>
>>>
>>> On Wed, Aug 10, 2016 at 11:51 AM, Jameson <[email protected]> wrote:
>>>
>>>> It is tracking the dynamic scope of the code generator, it doesn't care
>>>> about what code you emit. The generator function must not cause any
>>>> side-effects and must be entirely computed from the types of the inputs and
>>>> not other global state. Over time, these conditions are likely to be more
>>>> accurately enforced, as needed to make various optimizations reliable
>>>> and/or correct.
>>>>
>>>>
>>>>
>>>> On Wednesday, August 10, 2016 at 10:48:31 AM UTC-4, Erik Schnetter
>>>> wrote:
>>>>>
>>>>> I'm encountering the error "eval cannot be used in a generated
>>>>> function" in Julia 0.5 for code that is working in Julia 0.4. My question
>>>>> is -- what exactly is now disallowed? For example, if a generated function
>>>>> `f` calls another (non-generated) function `g`, can `g` then call `eval`?
>>>>> Does the word "in" here refer to the code that is generated by the
>>>>> generated function, or does it refer to the dynamical scope of the code
>>>>> generation state of the generated function?
>>>>>
>>>>> To avoid the error I have to redesign my code, and I'd like to know
>>>>> ahead of time what to avoid. A Google search only turned up the C file
>>>>> within Julia that emits the respective error message, as well as the
>>>>> Travis
>>>>> build log for my package.
>>>>>
>>>>> -erik
>>>>>
>>>>> --
>>>>> Erik Schnetter <[email protected]>
>>>>> http://www.perimeterinstitute.ca/personal/eschnetter/
>>>>>
>>>>
>>>
>>>
>>> --
>>> Erik Schnetter <[email protected]>
>>> http://www.perimeterinstitute.ca/personal/eschnetter/
>>>
>>
>
>
> --
> Erik Schnetter <[email protected]>
> http://www.perimeterinstitute.ca/personal/eschnetter/
>
