Indeed this was my first approach, but due to many small variations the
number of graphs is a bit too big to manage. Currently I precompile a few
trees and for the remainder of variations I do an ifelse variant with
boolean operations which reduces the number of trees at the cost of
computational inefficiency.
But I am most interested in the ifelse status since I could use a single
tree and with lazy evaluations would have full computational efficiency.
That's what the GPUs and theano is all about, right?
On Sunday, 26 March 2017 05:26:37 UTC+2, Jesse Livezey wrote:
>
> I have decided to precompile a general graph in which all the possible
>> graphs are nested. Then during realtime I would set which parts of the
>> general graph to use using the *allowed_branch* variables and *if* nodes.
>> Since afaik ifs are evaluated lazily in each case I would only be using the
>> relevant part of the graph so my computational cost is minimal.
>
>
> Have you considered precompiling all possible graphs individually and then
> just using python conditionals to choose a graph? Maybe this won't work for
> your system, but it might be easier to get right.
>
> On Saturday, March 25, 2017 at 2:21:27 AM UTC-7, Šarūnas S. wrote:
>>
>> Nouiz sorry I understand what you were refering by is constant. I've
>> mislead you with my example.
>>
>> This is a more realistic example:
>>
>> import theano as th
>> import theano.tensor as T
>>
>> allowed_branch = th.shared( np.cast['float32']( 0 ) )
>>
>> x = T.matrix('x')
>> y = T.matrix('y')
>> f = x ** 2 + y ** 2 + 2 * x * y
>>
>> result = th.ifelse.ifelse( T.gt( allowed_branch, T.constant( 0 ) ), f,
>> T.zeros( (2,2) ) )
>>
>>
>>
>> I am working on a realtime system which in a given situation will
>> constructs a relevant computational graph, compute its result and display
>> it.
>> However, the graphs are relatively big and each of their compilation
>> takes too long so I cant compile realtime. Thus I have to somehow
>> precompile.
>>
>> I have decided to precompile a general graph in which all the possible
>> graphs are nested. Then during realtime I would set which parts of the
>> general graph to use using the *allowed_branch* variables and *if*
>> nodes. Since afaik ifs are evaluated lazily in each case I would only be
>> using the relevant part of the graph so my computational cost is minimal.
>>
>>
>> On Saturday, 25 March 2017 10:04:21 UTC+1, Šarūnas S. wrote:
>>>
>>> I suspect that ifelse is running on GPU because this is the profile I get
>>>
>>> ==================
>>> Message: Sum of all(44) printed profiles at exit excluding Scan op
>>> profile.
>>> Time in 95 calls to Function.__call__: 2.309995e-01s
>>> Time in Function.fn.__call__: 2.299995e-01s (99.567%)
>>> Time in thunks: 2.307765e-01s (99.903%)
>>> Total compile time: 1.360100e+01s
>>> Number of Apply nodes: 416
>>> Theano Optimizer time: 6.314001e+00s
>>> Theano validate time: 9.200015e-01s
>>> Theano Linker time (includes C, CUDA code generation/compiling):
>>> 1.169000e+00s
>>> Import time 2.799892e-02s
>>> Node make_thunk time 1.108999e+00s
>>> Node GpuElemwise{Composite{(i0 * ((i1 * i2) + (i1 *
>>> i3)))}}[(0, 2)](CudaNdarrayConstant{0.5},
>>> CudaNdarrayConstant{0.833333313465}, GpuCAReduce{add}{1,1}.0,
>>> GpuCAReduce{add}{1,1}.0) time 6.999969e-03s
>>> Node GpuElemwise{Composite{(-minimum(i0, maximum(minimum(i0,
>>> (maximum((i1 - i2), i3) + i2)), (((i1 + i2) * i4) +
>>> i1))))},no_inplace}(<CudaNdarrayType(float32, scalar)>,
>>> <CudaNdarrayType(float32, scalar)>, <CudaNdarrayType(float32, scalar)>,
>>> CudaNdarrayConstant{120.0}, <CudaNdarrayType(float32, scalar)>) time
>>> 4.999876e-03s
>>> Node GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32,
>>> matrix)>, GpuElemwise{TrueDiv}[(0, 0)].0) time 4.000187e-03s
>>> Node HostFromGpu(<CudaNdarrayType(float32, scalar)>) time
>>> 3.999949e-03s
>>> Node GpuElemwise{Mul}[(0, 1)](GpuDimShuffle{x,x}.0,
>>> GpuDimShuffle{x,0}.0) time 3.999949e-03s
>>>
>>> Time in all call to theano.grad() 0.000000e+00s
>>> Time since theano import 28.959s
>>> Class
>>> ---
>>> <% time> <sum %> <apply time> <time per call> <type> <#call> <#apply>
>>> <Class name>
>>> 55.4% 55.4% 0.128s 8.71e-05s C 1468 301
>>> theano.sandbox.cuda.basic_ops.GpuElemwise
>>> 25.6% 81.0% 0.059s 1.03e-04s C 571 106
>>> theano.sandbox.cuda.basic_ops.GpuCAReduce
>>> 9.1% 90.1% 0.021s 3.72e-05s C 564 150
>>> theano.sandbox.cuda.basic_ops.HostFromGpu
>>> 5.6% 95.7% 0.013s 6.04e-06s Py 2148 168
>>> theano.ifelse.IfElse
>>> 3.5% 99.1% 0.008s 2.16e-04s C 37 4
>>> theano.compile.ops.DeepCopyOp
>>> 0.4% 99.6% 0.001s 1.60e-06s C 623 122
>>> theano.sandbox.cuda.basic_ops.GpuDimShuffle
>>> 0.4% 100.0% 0.001s 1.97e-06s C 506 110
>>> theano.tensor.elemwise.Elemwise
>>> ... (remaining 0 Classes account for 0.00%(0.00s) of the runtime)
>>>
>>> Ops
>>> ---
>>> <% time> <sum %> <apply time> <time per call> <type> <#call> <#apply>
>>> <Op name>
>>> 16.9% 16.9% 0.039s 1.22e-04s C 319 58
>>> GpuElemwise{mul,no_inplace}
>>> 10.0% 26.9% 0.023s 1.49e-04s C 155 30
>>> GpuCAReduce{add}{1,0}
>>> 9.1% 36.0% 0.021s 3.72e-05s C 564 150
>>> HostFromGpu
>>> 8.2% 44.2% 0.019s 1.23e-04s C 154 30
>>> GpuCAReduce{add}{0,1}
>>> 6.9% 51.1% 0.016s 6.61e-05s C 242 44
>>> GpuElemwise{Mul}[(0, 1)]
>>> 6.5% 57.6% 0.015s 6.20e-05s C 242 44
>>> GpuElemwise{maximum,no_inplace}
>>> 6.5% 64.1% 0.015s 6.19e-05s C 242 44
>>> GpuCAReduce{maximum}{1}
>>> 5.6% 69.7% 0.013s 6.04e-06s Py 2148 168
>>> if{inplace,gpu}
>>> 3.5% 73.2% 0.008s 5.59e-05s C 143 26
>>> GpuElemwise{TrueDiv}[(0, 0)]
>>> 3.5% 76.7% 0.008s 2.16e-04s C 37 4
>>> DeepCopyOp
>>> 2.6% 79.3% 0.006s 8.95e-05s C 67 16
>>> GpuElemwise{Mul}[(0, 2)]
>>> 2.2% 81.4% 0.005s 1.25e-04s C 40 4
>>> GpuElemwise{Maximum}[(0, 0)]
>>> 1.7% 83.2% 0.004s 2.00e-04s C 20 2
>>> GpuElemwise{Composite{maximum(i0, maximum(i1, maximum(i2, i3)))}}[(0, 0)]
>>> 1.7% 84.9% 0.004s 4.93e-04s C 8 8
>>> GpuElemwise{neg,no_inplace}
>>> 1.3% 86.2% 0.003s 1.36e-04s C 22 4
>>> GpuElemwise{Composite{((i0 + i1) + i2)},no_inplace}
>>> 1.3% 87.5% 0.003s 2.50e-04s C 12 3
>>> GpuElemwise{Composite{minimum(i0, maximum(minimum(i0, (maximum((i1 - i2),
>>> i3) + i2)), ((i4 * i5) + i1)))}}[(0, 4)]
>>> 1.3% 88.8% 0.003s 9.08e-05s C 33 6
>>> GpuElemwise{Composite{(i0 * (i1 / i2))}}[(0, 0)]
>>> 0.9% 89.6% 0.002s 3.03e-05s C 66 12
>>> GpuElemwise{Composite{(i0 * (i1 / i2))}}[(0, 1)]
>>> 0.9% 90.5% 0.002s 1.00e-04s C 20 2
>>> GpuCAReduce{add}{1,1}
>>> 0.9% 91.4% 0.002s 2.50e-04s C 8 3
>>> GpuElemwise{Composite{minimum(i0, maximum(minimum(i0, (maximum((i1 - i2),
>>> i3) + i2)), (((i2 + i1) * i4) + i1)))},no_inplace}
>>> ... (remaining 28 Ops account for 8.62%(0.02s) of the runtime)
>>>
>>> Apply
>>> ------
>>> <% time> <sum %> <apply time> <time per call> <#call> <id> <Apply name>
>>> 1.7% 1.7% 0.004s 4.00e-04s 10 365
>>> GpuElemwise{Maximum}[(0, 0)](if{inplace,gpu}.0, if{inplace,gpu}.0)
>>> 1.3% 3.0% 0.003s 3.00e-04s 10 105
>>> GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32, matrix)>,
>>> GpuElemwise{TrueDiv}[(0, 0)].0)
>>> 1.3% 4.3% 0.003s 3.00e-04s 10 356
>>> GpuElemwise{Mul}[(0, 1)](GpuDimShuffle{x,x}.0, GpuDimShuffle{0,x}.0)
>>> 1.3% 5.6% 0.003s 3.00e-04s 10 143
>>> GpuCAReduce{add}{1,0}(GpuElemwise{mul,no_inplace}.0)
>>> 1.3% 6.9% 0.003s 3.00e-04s 10 112
>>> GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32, matrix)>,
>>> GpuElemwise{TrueDiv}[(0, 0)].0)
>>> 1.3% 8.2% 0.003s 3.00e-04s 10 169
>>> GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32, matrix)>,
>>> GpuElemwise{Composite{(i0 * (i1 / i2))}}[(0, 1)].0)
>>> 1.3% 9.5% 0.003s 3.00e-04s 10 136
>>> GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32, matrix)>,
>>> GpuElemwise{TrueDiv}[(0, 0)].0)
>>> 1.3% 10.8% 0.003s 3.00e-04s 10 217
>>> GpuCAReduce{add}{0,1}(GpuElemwise{mul,no_inplace}.0)
>>> 1.3% 12.1% 0.003s 3.00e-04s 10 184
>>> GpuElemwise{Composite{(i0 * (i1 / i2))}}[(0, 0)](GpuElemwise{TrueDiv}[(0,
>>> 0)].0, GpuElemwise{maximum,no_inplace}.0, GpuElemwise{add,no_inplace}.0)
>>> 1.3% 13.4% 0.003s 5.96e-04s 5 1
>>> HostFromGpu(GpuElemwise{Composite{minimum(i0, maximum(minimum(i0,
>>> (maximum((i1 - i2), i3) + i2)), (((i2 + i1) * i4) + i1)))},no_inplace}.0)
>>> 0.9% 14.3% 0.002s 1.69e-04s 12 0
>>> DeepCopyOp(<CudaNdarrayType(float32, scalar)>)
>>> 0.9% 15.2% 0.002s 2.00e-04s 10 148
>>> GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32, matrix)>,
>>> GpuElemwise{Composite{(i0 * (i1 / i2))}}[(0, 1)].0)
>>> 0.9% 16.0% 0.002s 2.00e-04s 10 153
>>> GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32, matrix)>,
>>> GpuElemwise{Composite{(i0 * (i1 / i2))}}[(0, 1)].0)
>>> 0.9% 16.9% 0.002s 2.00e-04s 10 126
>>> GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32, matrix)>,
>>> GpuElemwise{TrueDiv}[(0, 0)].0)
>>> 0.9% 17.8% 0.002s 2.00e-04s 10 412
>>> GpuCAReduce{add}{1,1}(GpuElemwise{Composite{(((i0 + i1) + i2) + i3)}}[(0,
>>> 0)].0)
>>> 0.9% 18.6% 0.002s 2.00e-04s 10 103
>>> GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32, matrix)>,
>>> GpuElemwise{TrueDiv}[(0, 0)].0)
>>> 0.9% 19.5% 0.002s 2.00e-04s 10 89
>>> GpuElemwise{TrueDiv}[(0, 0)](GpuElemwise{maximum,no_inplace}.0,
>>> GpuElemwise{Composite{((i0 + i1) + i2)},no_inplace}.0)
>>> 0.9% 20.4% 0.002s 2.00e-04s 10 3
>>> GpuElemwise{maximum,no_inplace}(<CudaNdarrayType(float32, col)>,
>>> CudaNdarrayConstant{[[ 0.001]]})
>>> 0.9% 21.2% 0.002s 2.00e-04s 10 134
>>> GpuElemwise{mul,no_inplace}(<CudaNdarrayType(float32, matrix)>,
>>> GpuElemwise{TrueDiv}[(0, 0)].0)
>>> 0.9% 22.1% 0.002s 2.00e-04s 10 300
>>> GpuElemwise{Mul}[(0, 1)](GpuElemwise{Composite{minimum(i0,
>>> maximum(minimum(i0, (maximum((i1 - i2), i3) + i2)), ((i4 * i5) +
>>> i1)))},no_inplace}.0, GpuDimShuffle{x,0}.0)
>>> ... (remaining 941 Apply instances account for 77.89%(0.18s) of the
>>> runtime)
>>>
>>> Here are tips to potentially make your code run faster
>>> (if you think of new ones, suggest them on the mailing
>>> list).
>>> Test them first, as they are not guaranteed to always
>>> provide a speedup.
>>> Sorry, no tip for today.
>>>
>>> And as you see ifelse is being shown as a PY operation which I would
>>> presume run on CPU. So where does it run? Also, what do you mean by add a
>>> condition is constant?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> P.S In case you need these are my Theano flags
>>>
>>> os.environ['THEANO_FLAGS'] =
>>> ",optimizer=fast_run,floatX=float32,device=gpu,linker=cvm"
>>> os.environ['THEANO_FLAGS'] += ',allow_gc=False,'
>>> os.environ['THEANO_FLAGS'] += ',lib.cnmem=0.3'
>>> os.environ['CUDA_LAUNCH_BLOCKING'] = '1'
>>> os.environ['THEANO_FLAGS'] += ',profile=true'
>>>
>>>
>>> On Friday, 24 March 2017 23:09:11 UTC+1, nouiz wrote:
>>>>
>>>> What tell you the ifelse is on the CPU?
>>>>
>>>> Anyway, add the condition is constant Theano will remove it during the
>>>> compilation.
>>>>
>>>> Fred
>>>>
>>>> Le ven. 24 mars 2017 12:41, Šarūnas S. <[email protected]> a écrit :
>>>>
>>>>> Please find a code example:
>>>>>
>>>>> import theano as th
>>>>> import theano.tensor as T
>>>>>
>>>>> retval = th.ifelse.ifelse( T.gt(T.constant(2.0),T.constant(1.0)), T.
>>>>> ones((500,1)),T.zeros((250,1)))
>>>>>
>>>>> On Friday, 24 March 2017 17:33:59 UTC+1, Šarūnas S. wrote:
>>>>>>
>>>>>> I am using theano version 0.9.0.rc2.dev version.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Friday, 24 March 2017 17:32:33 UTC+1, Šarūnas S. wrote:
>>>>>>>
>>>>>>> In my graph I have a few IfElse nodes and I am wondering how and
>>>>>>> where they are executed.
>>>>>>>
>>>>>>> At first I ran the code with linker=cvm in my THEANO_FLAGS but after
>>>>>>> profiling it looked like the ifelse is being executed on the CPU. Then
>>>>>>> I
>>>>>>> forced the linker=c to check whether the IfElse will go through and I
>>>>>>> got
>>>>>>> the NotImplementedError: if{inplace, gpu} cannot produce C code. Btw
>>>>>>> removing inline optimization did not help as it still gave the same
>>>>>>> error.
>>>>>>>
>>>>>>> So does IfElse have a GPU implementation? If yes how do I use it?
>>>>>>> Also, does it do lazy evaluation or not?
>>>>>>>
>>>>>> --
>>>>>
>>>>> ---
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "theano-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
