I think your problem is subsetting the raw vector: you are deleteing from the head -- a lot of copying going on. Instead just subset and extract the vector length of interest:
> # within a custom read function: > if (loc == 0) > data <- readBin(bytes, what, n, size, ...) > else if (loc > 0) > data <- readBin(bytes[(1:size) + loc, what, n, size, ...) On 1/31/07, Jon Clayden <[EMAIL PROTECTED]> wrote: > This hasn't generated any feedback after a few days on R-devel, so I'm > forwarding it to R-help in case anyone here has any ideas... > > Thanks, > Jon > > ---------- Forwarded message ---------- > From: Jon Clayden <[EMAIL PROTECTED]> > Date: 26-Jan-2007 11:25 > Subject: readBin is much slower for raw input than for a file > To: [email protected] > > > Dear all, > > I'm trying to write an efficient binary file reader for a file type > that is made up of several fields of variable length, and so requires > many small reads. Doing this on the file directly using a sequence of > readBin() calls is a bit too slow for my needs, so I tried buffering > the file into a raw vector and reading from that ("loc" is the > equivalent of the file pointer): > > fileSize <- file.info(fileName)$size > connection <- file(fileName, "rb") > bytes <- readBin(connection, "raw", n=fileSize) > loc <- 0 > close(connection) > > -- > > # within a custom read function: > if (loc == 0) > data <- readBin(bytes, what, n, size, ...) > else if (loc > 0) > data <- readBin(bytes[-(1:loc)], what, n, size, ...) > > However, this method runs almost 10 times slower for me than the > sequence of file reads did. The initial call to readBin() - for > reading in the file - is very quick, but running Rprof shows that the > vast majority of the run time in doing the full parse is spent in > readBin, so it does seem to be that that's slowing things down. Can > anyone shed any light on why this is? > > I'm not expecting miracles here - and I realise that writing the whole > read routine in C would be much quicker - but surely reading from a > raw vector should work out faster than reading from a file? The system > is R-2.4.1/Linux, Xeon 3.2 GHz, 2 GiB RAM; typical file size is 44 > KiB. > > Thanks in advance, > Jon > > ______________________________________________ > [email protected] mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Jim Holtman Cincinnati, OH +1 513 646 9390 What is the problem you are trying to solve? ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
