How are you using transactions?
Benilton Carvalho wrote:
Hi Everyone,
I use R to create tools for analysis of microarrays
(http://www.bioconductor.org).
I'm in a situation where I need to handle 6 matrices with 1M rows and
about 2K columns, ie each matrix takes roughly 15GB RAM.
The procedure I'm working on can be divided in two parts:
1) I read an input file, from which I compute one column for each of
the matrices I mentioned above;
2) After the matrices are ready,all the computations I do can be
performed in batches of rows (say 10K rows at a time), so there's no
need to have all the matrices in memory at the same time.
My (very naive) idea was to use SQLite to store these matrices (via
RSQLite package). So I have the following:
CREATE TABLE alleleA (sample1 REAL, sample2 REAL <all the way to>
sample2000 REAL);
When I have the data for sample1, I use an INSERT statement, which
takes about 4secs.
For all the other columns, I use and UPDATE statement, which is taking
hours (more the 8 now).
What are the obvious things I'm missing here? Or do you have any other
suggestions in order to improve the performance?
Thank you very much,
b
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
