First update to Theano dev version. The memory leak is fixed. Second install a good blas. It will speed up your computation. If you use conda
Conda install mkl 3rd profile before you optimize. What takes the most time. Rnn computation? Which rnn? Other stuff? Then try to optimize first the worst bottleneck. Make sure to not compile Theano function in the loops. If the bottleneck is in call to compiled Theano function, you can use Theano profiler. Check Theano tutorial, there is a page about that. Fred Le mer. 12 avr. 2017 05:21, Fufu <[email protected]> a écrit : > > Hey guys, > > I've been trying to make my code faster for a while using parallel and had > limited success, so if someone could help I would be very grateful! > > So, I've 2 different RNNs networks that are not very large (input for one > sample is 2000 time points x 20 features, and I have 2 layers of 30-50 > neurons), and a code that basically looks like this: > > for i in range(3): > > # do some stuff > > for j in range(30): > # predict with RNN_1 > > # do some other stuff > > # predict with RNN_2 > > The whole code takes about 30 seconds to run. > > After some google-research (I'm quite new to parallel computing), I found > a way to make it a bit faster using the multiprocessing like this : > > pool = Pool(processes=3) > multiresults = pool.map(partial(function_for_one_i,arg_a), list_of_arg_b) > > def function_for_one_i(arg_a,arg_b): > > # do some stuff > # > for j in range(30): > # predict with RNN_1 > > # do some other stuff > # > > # predict with RNN_2 > > Now I try to see if there is no other way to make it run faster, but I > found so many different things to try that trying all of them would take me > weeks =) > My problem is that I can't figure out what would be really helpful in my > case. > > So my question is: what would be my best bet to make my code run faster? > - Should I better look at sparse RNN models? > - Multi-theading on the 'j-loop'? > - GPU computations? > - or something else? > > I've been trying to use multithreading on the 'j-loop' (using pool.map() > and a ThreadPool) but without success so far, and I suspect most of the > answers I found on the internet to be outdated as theano is evolving quite > fast. > I use theano 0.8.2 (I've some memory leak problems with theano 0.9) on a > mac book pro. > > Any tip is welcome! =) > > And on another note, many thanks to all theano developers, it's really an > impressive work! > > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
