Unfortunaty for big file no memory :)
julia> F1=load("F.jd","F")
283300x266 Array{Float64,2}:
julia> mapa = Distances.pairwise(Euclidean(), F1')
ERROR: OutOfMemoryError()
in pairwise at C:\Users\SAMSUNG2\.julia\v0.4\Distances\src\generic.jl:132
Do You have any hint for big sets ?
Paul
W dniu 2015-10-12 o 12:23, Vincent Lostanlen pisze:
Dear Paul,
For k=100 and your purpose, parallelization may not be the utmost
performance bottleneck here. I advise you to use the Distances.jl
<https://github.com/JuliaStats/Distances.jl> package.
Since Julia stores contiguous memory in column-major order
<https://julia.readthedocs.org/en/latest/manual/performance-tips/#access-arrays-in-memory-order-along-columns>,
you will first need to transpose the matrix D — or, better, to define
it foremost as a n*k matrix instead of k*n.
Once you've ensured that, calling
mapa = Distances.pairwise(Euclidean(), D)
should give you at least a 100x speedup over the for loop you've
written, so parallelization should no longer be necessary.
Vincent.
Le dimanche 11 octobre 2015 16:34:27 UTC+2, paul analyst a écrit :
Like here , what wrong ?
k=100
mapa=zeros(k,k)
julia> @parallel for i=1:k,j=1:k
mapa[i,j]=sqrt(sum([D[i,:]-D[j,:]].^2))
end
ERROR: syntax: invalid assignment location
Paul
