So I get the error when I call mmap_array() as follows:
s = open("matrixfile.bin")
m = read(s, Float64)
weight = mmap_array(Float64, (int64(m),int64(m)), s)
close(s)
When my "matrixfile.bin" is small everything works fine, but when I get to
a stage where the file size is similar to the amount of RAM available I get
the following error:
ERROR: memory mapping failed: Cannot allocate memory
in mmap at mmap.jl:35
in mmap_array at mmap.jl:110
in readinfile at none:4
On Friday, 10 April 2015 15:12:14 UTC+1, tshort wrote:
>
> More information would help, especially a concise reproducible example.
>
> On Fri, Apr 10, 2015 at 8:00 AM, Marc Williams <[email protected]
> <javascript:>> wrote:
>
>> Hi,
>>
>> I'm doing some analysis where I need to compute some large matrices (up
>> to about 50,000 X 50,000) so I quickly run out of RAM, it seems like using
>> a memory mapped array could be a useful approach so I compute the matrices
>> and save them to a binary file and then read them in using mmap_array().
>> For the smaller matrices everything works fine but when the size of the
>> binary file is greater than the amount of RAM available I get the following
>> error:
>>
>> ERROR: memory mapping failed: Cannot allocate memory
>>
>> Once I've read the matrices in using mmap_array() I do some basic
>> calculation like computing the sum over all the elements, the sum over rows
>> and I access every element a couple of times.
>>
>> I've not used memory mapping before so am I using it in the right way and
>> is there anything I'm missing that I need to do to make this a solution to
>> my RAM issue?
>>
>> Many Thanks
>> Marc
>>
>
>
