Hi John,
Thanks for writing and for your suggestion. Sorry if my email was not
clear.
I am working with global discrete maps at resolution varying from 0.5x0.5
to 2.5x2.5 degrees per grid point. Those maps are discrete and represent
the presence or absence of suitable habitat, so many of the
Maybe you could use a ragged array, which would only store the half of
the matrix you need: https://bitbucket.org/maurow/ragged.jl
On Fri, 2015-08-14 at 01:03, Charles Novaes de Santana
charles.sant...@gmail.com wrote:
Dear all,
Thank you very much for your help and suggestions! I have
Charles:
Thanks for your response. I understand now that your application is
ecological, but the mathematical nature and the discretization of your
system is still unclear. You might consider a 'lazy' approach, in which you
compute a distance when you need it, rather than computing them all
bigmemory is presumably the same thing as mmap_array.
--Tim
On Friday, August 14, 2015 01:03:10 AM Charles Novaes de Santana wrote:
Dear all,
Thank you very much for your help and suggestions! I have learned a lot
from you. Indeed, I want to use this matrix and not only to read and store
If i read your example correctly, you are asking for a matrix
size=65600x65600 of Float64 which is roughly 32GB. My first order
assumption is, that julia asks the unterlying OS about getting this memory
(btw: as a single block). It would be more intersting what actually your OS
supports as
If you're willing to pay the performance cost associated with disk I/O, a
memory-mapped array (mmap_array) might be what you're looking for.
Memory mapped arrays are going to be a performance nightmare if you
actually need to write to the entire array. If you want a dense 65600 x
65600 matrix, you need at least 4GB times your element size, which by
default is 8 bytes for Float64, which means you need a machine with at
least 32GB of
Dear all,
Thank you very much for your help and suggestions! I have learned a lot
from you. Indeed, I want to use this matrix and not only to read and store
the data.
I should have written in my email that I was wondering if Julia had
something similar to bigmemory and bigalgebra in R (
Charles,
If your work ends with the matrix, then you could write a binary file on
dink, and make your loop calculate and hold a bunch of values each time and
save it to the binary file on disk periodically, until all values are
saved. Try to make the loop hold as many values as possible, so you
What do you mean by distances between sites at different resolution maps?
The word resolution suggests that the N x N size of the matrix results
from the discretization of a continuous function into N data points and the
computation of N^2 distances between those data points. If that's the
10 matches
Mail list logo
