Re: [R] Error because of large dimension

Thanks all for the reply. I think I need to think of other ways to approach the problem. Hanna 2016-01-24 17:45 GMT-05:00 William Dunlap : > > 28 PiB. Storing such a large matrix even on file is not possible. > > The ads for Amazon Red Shift say it is possible. E.g., >

[R] Error because of large dimension

Hi all, I am doing some calculation with very large dimension. I need to create a matrix with three columns and a very large number of rows (3195*1290*495*35*35*35*15=1.312083e+15) i n order to allocate calculation result from a for loop. R does not allow me to create such a matrix because of

Re: [R] Error because of large dimension

Hey Hanna, nrow and ncol in matrix() are integer-based, for the moment at least; accordingly they have a maximum value. (3195*1290*495*35*35*35*15) is actually larger than an integer can hold - you can test this with: str((3195*1290*495*35*35*35*15)) Which shows that it's stored as a numeric

Re: [R] Error because of large dimension

FYI, the matrix you tried to allocate would hold (3195*1290*495*35*35*35*15) * 3 = 3.936248e+15 values. Each value would occupy 8 bytes of memory (for the double data type). In other words, in order to keep this data matrix in memory you would require a computer with at least 3.148998e+16 bytes

Re: [R] Error because of large dimension

> 28 PiB. Storing such a large matrix even on file is not possible. The ads for Amazon Red Shift say it is possible. E.g., Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse that makes it simple and cost-effective to analyze all your data using your existing business