Hi John, Thanks for writing and for your suggestion. Sorry if my email was not clear.
I am working with global discrete maps at resolution varying from 0.5x0.5 to 2.5x2.5 degrees per grid point. Those maps are discrete and represent the presence or absence of suitable habitat, so many of the points are actually 0 (the authors of the maps considered some environmental conditions to define if the sites were suitable or unsuitable habitats for a given species). I would like to know the distance between any pair of suitable habitats to perform a task. My first and simplest try was to use a matrix to record the distances between any pair of points, and to access this matrix whenever I needed to use such a distance in my model. This approach woks fine with lower resolution data, but I understand now that this idea is not that good when I have so many sites. I am thinking in different possibilities now. I see at least three (for sure there are many more :)): 1) to use only the subset of suitable habitats to build the matrix of distances (and then to use sparse matrix as suggested by Stefan) 2) to use a machine with more memory and try to run my models using the matrices with all the sites 3) to try another language/library that might work better with such big amount of data (like python, or R). Thank you all for your feedbacks and your time! Best, Charles On 14 August 2015 at 02:51, John Gibson <johnfgib...@gmail.com> wrote: > What do you mean by "distances between sites at different resolution > maps"? The word "resolution" suggests that the N x N size of the matrix > results from the discretization of a continuous function into N data points > and the computation of N^2 distances between those data points. If that's > the case, there's almost certainly a more compact representation of the > distance function than the N^2 matrix. For example, you can probably > represent those N data points with an expansion over m << N continuous > expansion functions, and the distance function with an expansion over the m > x m tensor product of those function. > > John > > > On Thursday, August 13, 2015 at 6:26:55 AM UTC-4, Charles Santana wrote: >> >> Hi all, >> >> Do you recommend a way to work with bit matrices in Julia. By "big" I >> mean a 65600 x 65600 symmetric matrix (the upper triangular matrix is equal >> to the lower triangular one). >> >> I am studying the distances between sites at different resolution maps. >> For low resolution we have few sites, and for big resolution we have more >> sites (S). >> >> For few sites (small matrices) I was doing something like this: >> >> S = 100;#number of sites >> M = zeros(S,S); >> for i in 1:(S-1) >> for j in (i+1):S >> M[i,j] = dist(i,j);#where dist(i,j) is the distance between sites i >> and j >> end >> end >> >> However, for big matrices I get the following message: >> >> S=65600; >> M = zeros(S,S); >> ERROR: OutOfMemoryError() >> in call at essentials.jl:201 >> in zeros at array.jl:233 >> >> I am using Julia Version 0.4.0-dev+5920 in Ubuntu 14.04. >> >> Thanks for any tip! >> >> Best, >> >> Charles >> -- >> Um axé! :) >> >> -- >> Charles Novaes de Santana, PhD >> http://www.imedea.uib-csic.es/~charles >> > -- Um axé! :) -- Charles Novaes de Santana, PhD http://www.imedea.uib-csic.es/~charles
