Hi. Actually, the code was using a smaller data set and, in one of the runs, we just set a large number of samples. The obvious approach is to parse directly from a file, but I get a little bit nervous when it took that long and crashed without a warning.
Thanks and sorry for the noise. On Monday, March 21, 2016 at 4:36:55 PM UTC-3, Stefan Karpinski wrote: > > Loading large data by evaluating code that constructs it is not a good > approach. You'd be much better of using a standard data format like JLD > <https://github.com/JuliaLang/JLD.jl>. > > On Mon, Mar 21, 2016 at 3:15 PM, Eduardo Lenz <[email protected] > <javascript:>> wrote: > >> Hi. >> >> I am trying to load a 180 x 1025 matrix of floats, defined in a .jl file, >> like >> >> function Data() >> data = [1E-4 ....... 3.2E-5 >> ..... ..... >> ..... 0.0 ] >> end >> >> The simple act of including this file takes 50s in my computer and, when >> the function >> is called, >> >> data = Data() >> >> it takes more than two hours to finally crash without warning. >> >> I can easily create such matrix with rand(180,1025), such that memory is >> not >> a problem. I can also use it with a smaller number of lines without any >> issues. >> >> It is a known limitation or am I doing something wrong here ? >> >> I am using the last 0.4.3 Windows version (downloaded directly from the >> page) >> >> Thanks. >> > >
