Your statements for sample.size=10,000 try to construct a matrix with 3 dense 10,000 x 10,000 blocks. That's approximately 10*10*8 MB each and very likely explains your error message.
Since you have a simple formula for the matrix, why not define a function to implement multiplication by this matrix (and whatever else you want to do with it), rather than use general sparse matrix representations? Reid Huntsinger -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Doran, Harold Sent: Monday, April 18, 2005 4:55 PM To: [email protected] Subject: [R] Construction of a large sparse matrix Dear List: I'm working to construct a very large sparse matrix and have found relief using the SparseM package. I have encountered an issue that is confusing to me and wonder if anyone may be able to suggest a smarter solution. The matrix I'm creating is a covariance matrix for a larger research problem that is subsequently used in a simulation. Below is the latex form of the matrix if anyone wants to see the pattern I am trying to create. The core of my problem seems to localize to the last line of the following portion of code. n<-sample.size*4 k<-n/4 vl.mat <- as.matrix.csr(0, n, n) block <- 1:k #each submatrix size for(i in 1:3) vl.mat[i *k + block, i*k + block] <- LE When the variable LE is 0, the matrix is easily created. For example, when sample.size = 10,000 this matrix was created on my machine in about 1 second. Here is the object size. > object.size(vl.mat) [1] 160692 However, when LE is any number other than 0, the code generates an error. For example, when I try LE <- 2 I get Error: cannot allocate vector of size 781250 Kb In addition: Warning message: Reached total allocation of 1024Mb: see help(memory.size) Error in as.matrix.coo(as.matrix.csr(value, nrow = length(rw), ncol = length(cl))) : Unable to find the argument "x" in selecting a method for function "as.matrix.coo" I'm guessing that single digit integers should occupy the same amount of memory. So, I'm thinking that the matrix is "less sparse" and the problem is related to the introduction of a non-zero element (seems obvious). However, the matrix still retains a very large proportion of zeros. In fact, there are still more zeros than non-zero elements. Can anyone suggest a reason why I am not able to create this matrix? I'm at the limit of my experience and could use a pointer if anyone is able to provide one. Many thanks, Harold P.S. The matrix above is added to another matrix to create the covariance matrix below. The code above is designed to create the portion of the matrix \sigma^2_{vle}\bm{J} . \begin{equation} \label{vert:cov} \bm{\Phi} = var \left [ \begin{array}{c} Y^*_{1}\\ Y^*_{2}\\ Y^*_{3}\\ Y^*_{4}\\ \end{array} \right ] = \left [ \begin{array}{cccc} \sigma^2_{\epsilon}\bm{I}& \sigma^2_{\epsilon}\rho\bm{I} & \bm{0} & \bm{0}\\ \sigma^2_{\epsilon}\rho\bm{I} & \sigma^2_{\epsilon}\bm{I}+\sigma^2_{vle}\bm{J} & \sigma^2_{\epsilon}\rho^2\bm{I} & \bm{0}\\ \bm{0} & \sigma^2_{\epsilon}\rho^2\bm{I} & \sigma^2_{\epsilon}\bm{I}+\sigma^2_{vle}\bm{J}& \sigma^2_{\epsilon}\rho^3\bm{I}\\ \bm{0} & \bm{0} & \sigma^2_{\epsilon}\rho^3\bm{I}& \sigma^2_{\epsilon}\bm{I}+\sigma^2_{vle}\bm{J} \\ \end{array} \right] \end{equation} where $\bm{I}$ is the identity matrix, $\bm{J}$ is the unity matrix, and $\rho$ is the autocorrelation. [[alternative HTML version deleted]] ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
