Re: [julia-users] Re: Euler Maruyama for neural networks, slower than python

[Dupont]

Hi,

I was using anaconda with more than one thread... That may explain the 
differences in timing.

I aopologize for this and thank everybody for your help,

Best regards.

Re: [julia-users] Re: Euler Maruyama for neural networks, slower than python

[Stefan Karpinski]

Compiling from source should be a matter of cloning and running `make`. It
will take a while and use CPU though.

On Thu, Jan 21, 2016 at 2:47 PM, Mauro  wrote:

> My timings are (on linux, 2012 i7 CPU)
> - 0.14s original
> - 0.05 my latest version
> - 0.24 python (2 & 3)
>
> You could try to compile it yourself and see if that is faster.  But I'm
> not sure how much hassle that is on OSX.
>
> Maybe easier/quicker would be to install the 0.3 dmg.  See whether that
> gives better timing.
>
> Maybe someone how makes the .dmg can comment?
>
> On Thu, 2016-01-21 at 19:06, Dupont  wrote:
> > @Mauro,
> >
> > I am using the dmg located at
> >
> https://s3.amazonaws.com/julialang/bin/osx/x64/0.4/julia-0.4.3-osx10.7+.dmg
> > My timings using your code is .330s vs .1s for numpy.
> >
> > I would be happy if one told me that Julia has not caught yet, but the
> fact
> > that you have a speedup is troubling.
> > I have a MacBook Pro (Retina, 15-inch, Mid 2015), on osx 10.11
> >
> > Thank you for your help,
>

Re: [julia-users] Re: Euler Maruyama for neural networks, slower than python

[Mauro]

My timings are (on linux, 2012 i7 CPU)
- 0.14s original
- 0.05 my latest version
- 0.24 python (2 & 3)

You could try to compile it yourself and see if that is faster.  But I'm
not sure how much hassle that is on OSX.

Maybe easier/quicker would be to install the 0.3 dmg.  See whether that
gives better timing.

Maybe someone how makes the .dmg can comment?

On Thu, 2016-01-21 at 19:06, Dupont  wrote:
> @Mauro,
>
> I am using the dmg located at
> https://s3.amazonaws.com/julialang/bin/osx/x64/0.4/julia-0.4.3-osx10.7+.dmg
> My timings using your code is .330s vs .1s for numpy.
>
> I would be happy if one told me that Julia has not caught yet, but the fact
> that you have a speedup is troubling.
> I have a MacBook Pro (Retina, 15-inch, Mid 2015), on osx 10.11
>
> Thank you for your help,

Re: [julia-users] Re: Euler Maruyama for neural networks, slower than python

[Dupont]

@Mauro,

I am using the dmg located at 
https://s3.amazonaws.com/julialang/bin/osx/x64/0.4/julia-0.4.3-osx10.7+.dmg
My timings using your code is .330s vs .1s for numpy.

I would be happy if one told me that Julia has not caught yet, but the fact 
that you have a speedup is troubling.
I have a MacBook Pro (Retina, 15-inch, Mid 2015), on osx 10.11

Thank you for your help,

[julia-users] Re: Euler Maruyama for neural networks, slower than python

[Pablo Zubieta]

I guess that you can also throw a couple of @inbounds before the for loops 
in Mauro's solution to improve things a bit more.

Re: [julia-users] Re: Euler Maruyama for neural networks, slower than python

[Mauro]

On my system Julia is also ~4x faster than Python, so it seems that this
is mainly a problem with the BLAS library.  Did you build Julia
yourself or are you using binaries?

I see some improvements (20%) using an in-place sig function and some
other bits.  Together with Pablo's trick this halfs execution time:

import Base.LinAlg, Base.LinAlg.BlasReal, Base.LinAlg.BlasComplex
Base.disable_threaded_libs()

function sig!(out, x::Vector)
for i=1:length(x)
out[i] =  1.0 ./ (1. + exp(-x[i]))
end
nothing
end

function 
mf_loop(Ndt::Int64,V0::Vector{Float64},V::Vector{Float64},dt::Float64,W::Matrix{Float64},J::Matrix{Float64})
  sv = copy(V0)
  V  = copy(V0)

  for i=1:Ndt

  # sv = sig(V)
  # V = (1-dt)*V +  J*sv * dt + W[:,i]
sig!(sv,V)
BLAS.gemv!('N',dt,J,sv,1.0-dt,V)
  for j=1:length(V)
  V[j] += W[j,i]
  end
  end
  nothing
end

...


On Thu, 2016-01-21 at 17:10, Dupont  wrote:
> Hi,
>
> Thank you for your answer. It is a bit faster for me too but still 3x
> slower than numpy (note that you have to uncomment #sv = sig(V) in your
> code.
>
> Best regards,

[julia-users] Re: Euler Maruyama for neural networks, slower than python

[Dupont]

Hi,

Thank you for your answer. It is a bit faster for me too but still 3x 
slower than numpy (note that you have to uncomment #sv = sig(V) in your 
code.

Best regards,

[julia-users] Re: Euler Maruyama for neural networks, slower than python

[Pablo Zubieta]

I just stumble upon this comment 
 on 
this GitHub issue . It 
seems that the current situation in Julia won't let you match the numpy 
code yet (until that issue is solved). But from the comment in the link, 
the following should be faster (it is at least on my machine).

### Julia code
import Base.LinAlg, Base.LinAlg.BlasReal, Base.LinAlg.BlasComplex

Base.disable_threaded_libs()

function sig(x::Union{Float64,Vector})
  return 1.0 ./ (1. + exp(-x))
end

function mf_loop(Ndt::Int64,V0::Vector{Float64},V::Vector{Float64},dt::
Float64,W::Matrix{Float64},J::Matrix{Float64})
sv = copy(V0)
V  = copy(V0)
for i=1:Ndt
#sv = sig(V)
#V = V + ( -V + J*sv ) * dt + W[:,i]
BLAS.gemv!('N',dt,J,sv,1.0-dt,V)
BLAS.axpy!(1., W[:,i], V)
end
nothing 
end

N = 100
dt = 0.1
Ndt = 1

sigma  = 0.1
W = (randn(N,Ndt)) * sigma * sqrt(dt)
J = (rand(N,N))/N
  
V0 = rand(N)
V = copy(V0)

println( "calcul\n")
mf_loop(1,V0,V,dt,W,J)
@time mf_loop(Ndt,V0,V,dt,W,J)

Cheers.

[julia-users] Re: Euler Maruyama for neural networks, slower than python

[Dupont]

No gain was noticed.

>

[julia-users] Re: Euler Maruyama for neural networks, slower than python

[Dupont]

HI,

here it is:

   _   _ _(_)_ |  A fresh approach to technical computing
  (_) | (_) (_)|  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.2 (2015-12-06 21:47 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/   |  x86_64-apple-darwin13.4.0

julia> versioninfo
versioninfo (generic function with 4 methods)

julia> versioninfo()
Julia Version 0.4.2
Commit bb73f34 (2015-12-06 21:47 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin13.4.0)
  CPU: Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

[julia-users] Re: Euler Maruyama for neural networks, slower than python

[Simon Byrne]

I noticed the commented out BLAS.gemm! and BLAS.axpy! lines: did these help?

-Simon

On Thursday, 21 January 2016 11:12:53 UTC, Viral Shah wrote:
>
> The matrix-vector multiply in there will lose the benefit of BLAS in 
> devectorization. This is one area where we ought to be better, since this 
> code is best not devectorized (from a user's perspective).
>
> On my mac, python is .27 seconds and julia 0.4 is .47 seconds. Python is 
> perhaps not using a fast BLAS, since it is whatever came with pip.
>
> -viral
>
> On Thursday, January 21, 2016 at 4:22:52 PM UTC+5:30, Kristoffer Carlsson 
> wrote:
>>
>> There is no need to annotate your function argument types so tightly, 
>> unless you have a good reason for it.
>>
>> You will generate a lot of temporaries in your V = ...
>>
>> Rewrite it as a loop and it will be a lot faster. You could also take a 
>> look at the Devectorize.jl package.
>>
>>

[julia-users] Re: Euler Maruyama for neural networks, slower than python

[Viral Shah]

The matrix-vector multiply in there will lose the benefit of BLAS in 
devectorization. This is one area where we ought to be better, since this 
code is best not devectorized (from a user's perspective).

On my mac, python is .27 seconds and julia 0.4 is .47 seconds. Python is 
perhaps not using a fast BLAS, since it is whatever came with pip.

-viral

On Thursday, January 21, 2016 at 4:22:52 PM UTC+5:30, Kristoffer Carlsson 
wrote:
>
> There is no need to annotate your function argument types so tightly, 
> unless you have a good reason for it.
>
> You will generate a lot of temporaries in your V = ...
>
> Rewrite it as a loop and it will be a lot faster. You could also take a 
> look at the Devectorize.jl package.
>
>

[julia-users] Re: Euler Maruyama for neural networks, slower than python

[Pablo Zubieta]

I am seeing the opposite on my machine (the Julia version being 4 times 
faster than the one in Python).

It might be a problem with the BLAS library that is being used. What is 
your platform and Julia version?