Parsing a data file is the way to go here. In C, you can include large chunks of data since the parsing occurs at compile time. In Julia, parsing happens at run time and it's going to be noticeable. The parser for a simple data format is much more specialized and thus likely to be faster than Julia's (code) parser.
On Sun, Nov 23, 2014 at 4:12 AM, Milan Bouchet-Valat <[email protected]> wrote: > Le samedi 22 novembre 2014 à 14:56 -0800, Jacek Hoła a écrit : > > It seemed pretty handy to have one test in just one file. I used > > readdlm and it works great, but now I have 4 files (3 matrices and one > > jl with readdlms and call to module). Guess I can live with that. > I see. Jeff could give you an informed answer, but I guess the parser > isn't designed to handle very large arrays very efficiently. > > > Regards > > > W dniu sobota, 22 listopada 2014 23:38:10 UTC+1 użytkownik Milan > > Bouchet-Valat napisał: > > Le samedi 22 novembre 2014 à 13:15 -0800, Jacek Hoła a > > écrit : > > > Hi guys, > > > > > > I tried to find some advice here and on google but with no > > luck. I'm > > > trying to initialize a matrix of Float64. Since I have the > > values I > > > just did > > > > > > my_matrix = Float64[3.5 6.7 2 0 lots of numbers...; > > > 3 0 6.4 3.4 and so on...] > > > > > > > > > the matrix was around 800 x 1800. I've got it in a .jl file. > > When I > > > run it it takes quite a long time and uses 2.3 GB of memory. > > Why did > > > it happen and how can I initialize this array explicitly > > (not from > > > external file or through HDF5) without such memory usage? > > Any reason why you really don't want to initialize it from an > > external > > file? Using a simple format readable by readdlm() would be > > quite > > natural. Keeping this data in a .jl file doesn't sound very > > practical. > > > > > > Regards > > > >
