I'll bet you're working within a constrained environment. If you're on a unix
platform, what does 'ulimit -a' say?
Best,
--Tim
On Saturday, April 11, 2015 12:04:03 PM Marc Williams wrote:
> So I get the error when I call mmap_array() as follows:
>
> s = open("matrixfile.bin")
>
> m = read(s, Float64)
>
> weight = mmap_array(Float64, (int64(m),int64(m)), s)
>
> close(s)
>
> When my "matrixfile.bin" is small everything works fine, but when I get to
> a stage where the file size is similar to the amount of RAM available I get
> the following error:
>
> ERROR: memory mapping failed: Cannot allocate memory
>
> in mmap at mmap.jl:35
>
> in mmap_array at mmap.jl:110
>
> in readinfile at none:4
>
> On Friday, 10 April 2015 15:12:14 UTC+1, tshort wrote:
> > More information would help, especially a concise reproducible example.
> >
> > On Fri, Apr 10, 2015 at 8:00 AM, Marc Williams <[email protected]
> >
> > <javascript:>> wrote:
> >> Hi,
> >>
> >> I'm doing some analysis where I need to compute some large matrices (up
> >> to about 50,000 X 50,000) so I quickly run out of RAM, it seems like
> >> using
> >> a memory mapped array could be a useful approach so I compute the
> >> matrices
> >> and save them to a binary file and then read them in using mmap_array().
> >> For the smaller matrices everything works fine but when the size of the
> >> binary file is greater than the amount of RAM available I get the
> >> following
> >> error:
> >>
> >> ERROR: memory mapping failed: Cannot allocate memory
> >>
> >> Once I've read the matrices in using mmap_array() I do some basic
> >> calculation like computing the sum over all the elements, the sum over
> >> rows
> >> and I access every element a couple of times.
> >>
> >> I've not used memory mapping before so am I using it in the right way and
> >> is there anything I'm missing that I need to do to make this a solution
> >> to
> >> my RAM issue?
> >>
> >> Many Thanks
> >> Marc
