On Thu, 20 Jan 2005, Tomas Kalibera wrote:
Dear Prof Ripley,
thanks for your suggestions, it's very nice one can create custom connections
directly in R and I think it is what I need just now.
However, what is wrong with reading a file at a time and combining the
results in R using rbind?
Well, the problem is performance. If I concatenate all those files, they have
around 8MB, can grow to tens of MBs in near future.
Both concatenating and reading from a single file by scan takes 5 seconds
(which is almost OK).
However, reading individual files by read.table and rbinding one by one (
samples=rbind(samples, newSamples ) takes minutes. The same is when I
concatenate lists manually. Scan does not help significantly. I guess there
is some overhead in detecting dimensions of objects in rbind (?) or
re-allocation or copying data ?
rbind is vectorized so you are using it (way) suboptimally.
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
