On Wednesday, January 07, 2015 01:42:24 PM Paul Analyst wrote: > one vector 10^9 it is about 16 GB,
...and that explains it. (Compare against your RAM size.) --Tim > Paul > > W dniu 2015-01-07 o 13:13, Tim Holy pisze: > > Did you try calculating how much memory the right hand side of that > > assignment requires? (Remember that sizeof(Float64) == 8). > > > > --Tim > > > > On Wednesday, January 07, 2015 03:25:54 AM paul analyst wrote: > >> _ _ _(_)_ | A fresh approach to technical computing > >> > >> (_) | (_) (_) | Documentation: http://docs.julialang.org > >> > >> _ _ _| |_ __ _ | Type "help()" for help. > >> > >> | | | | | | |/ _` | | > >> | | | > >> | | |_| | | | (_| | | Version 0.3.3 (2014-10-21 20:18 UTC) > >> > >> _/ |\__'_|_|_|\__'_| | Official http://julialang.org release > >> | > >> |__/ | i686-linux-gnu > >> > >> .... > >> > >> dset[1:10^8,1]=rand(10^8); > >> > >> julia> size(dset) > >> (1000000000,10000000) > >> > >> julia> dset[1:10^9,1]=rand(10^9); > >> ERROR: invalid Array size > >> > >> in rand at random.jl:123 > >> > >> julia> > >> > >> Julia now is using 32% of ram, 2.5 GB of 7.8 GB aviable.. > >> Ubuntu 14.10 > >> Why ? free size of HD is now 30GB > >> Paul > >> > >> > >> W dniu wtorek, 6 stycznia 2015 21:04:53 UTC+1 użytkownik paul analyst > >> > >> napisał: > >>>> 2.if sum k and l > 9 Julia cant work. Is it to big size for hdf5 or for > >>>> > >>>>> system (Win7 64 Home Premium) ? > >>>> > >>>> Not sure. It works for me (Kubuntu Linux 14.04). > >>> > >>>> I checked on Ubuntu : > >>> If sum of "k" or/and "l" is more then 7 I have problem with reading > >>> vectors (cols) > >>> > >>>> samsung2@samsung2:~$ julia > >>>> > >>>> _ > >>>> > >>>> _ _ _(_)_ | A fresh approach to technical computing > >>>> > >>>> (_) | (_) (_) | Documentation: http://docs.julialang.org > >>>> > >>>> _ _ _| |_ __ _ | Type "help()" for help. > >>>> > >>>> | | | | | | |/ _` | | > >>>> | | | > >>>> | | |_| | | | (_| | | Version 0.3.3 (2014-10-21 20:18 UTC) > >>>> > >>>> _/ |\__'_|_|_|\__'_| | Official http://julialang.org release > >>>> | > >>>> |__/ | i686-linux-gnu > >>>> > >>>> julia> using HDF5 > >>>> > >>>> julia> hfi=h5open("F_big.h5","w");close(hfi) > >>>> > >>>> julia> k,l=8,8 ; > >>>> > >>>> julia> fid = h5open("F_big.h5","r+") > >>>> HDF5 data file: F_big.h5 > >>>> > >>>> julia> fid["mygroup/A"]=rand(2)#niepotrzebny > >>>> > >>>> 2-element Array{Float64,1}: > >>>> 0.318459 > >>>> 0.258055 > >>>> > >>>> julia> g = fid["mygroup"] > >>>> HDF5 group: /mygroup (file: F_big.h5) > >>>> > >>>> julia> dset = d_create(g, "F", datatype(Float64), > >>>> dataspace(10^k,1*10^l)) > >>>> HDF5 dataset: /mygroup/F (file: F_big.h5) > >>>> > >>>> julia> h5read("F_big.h5","mygroup/F",(:,1)) > >>>> > >>>> 100000000x1 Array{Float64,2}: > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> ⋮ > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> 0.0 > >>>> > >>>> julia> close(fid) > >>>> > >>>> julia> hfi=h5open("F_big.h5","w");close(hfi) > >>>> > >>>> julia> k,l=9,8 ; > >>>> > >>>> julia> fid = h5open("F_big.h5","r+") > >>>> HDF5 data file: F_big.h5 > >>>> > >>>> julia> fid["mygroup/A"]=rand(2)#niepotrzebny > >>>> > >>>> 2-element Array{Float64,1}: > >>>> 0.129214 > >>>> 0.4785 > >>>> > >>>> julia> g = fid["mygroup"] > >>>> HDF5 group: /mygroup (file: F_big.h5) > >>>> > >>>> julia> dset = d_create(g, "F", datatype(Float64), > >>>> dataspace(10^k,1*10^l)) > >>>> HDF5 dataset: /mygroup/F (file: F_big.h5) > >>>> > >>>> julia> h5read("F_big.h5","mygroup/F",(:,1)) > >>>> ERROR: invalid Array size > >>>> > >>>> in Array at base.jl:223 > >>>> in _getindex at /home/samsung2/.julia/v0.3/HDF5/src/plain.jl:1557 > >>>> in getindex at /home/samsung2/.julia/v0.3/HDF5/src/plain.jl:1550 > >>>> in getindex at /home/samsung2/.julia/v0.3/HDF5/src/plain.jl:1620 > >>>> in h5read at /home/samsung2/.julia/v0.3/HDF5/src/plain.jl:602 > >>>> > >>>> julia> close(fid) > >>>> > >>>> julia> h5read("F_big.h5","mygroup/F",(1:2,:)) > >>>> > >>>> 2x100000000 Array{Float64,2}: > >>>> 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 … 0.0 0.0 0.0 0.0 0.0 > >>>> 0.0 > >>>> > >>>> 0.0 > >>>> > >>>> 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 > >>>> 0.0 > >>>> > >>>> 0.0 > >>>> > >>>> julia> h5read("F_big.h5","mygroup/F",(:,1)) > >>>> ERROR: invalid Array size > >>>> > >>>> in Array at base.jl:223 > >>>> in _getindex at /home/samsung2/.julia/v0.3/HDF5/src/plain.jl:1557 > >>>> in getindex at /home/samsung2/.julia/v0.3/HDF5/src/plain.jl:1550 > >>>> in getindex at /home/samsung2/.julia/v0.3/HDF5/src/plain.jl:1620 > >>>> in h5read at /home/samsung2/.julia/v0.3/HDF5/src/plain.jl:602 > >>>> > >>>> julia> > >>>> > >>>> Paul
