Not convinced Jeff is completely right about this not concerning R, since I've found that the application language (R,
perl, etc.) makes a difference in how files are accessed by/to OS. He is certainly correct that OS (and versions) are
where the actual reading and writing happens, but sometimes the call to those can be inefficient. (Sorry, I've not got
examples specifically for file reads, but had a case in computation where there was an 800% i.e., 80000 fold difference
in timing with R, which rather took my breath away. That's probably been sorted now.) The difficulty in making general
statements is that a rather full set of comparisons over different commands, datasets, OS and version variants is needed
before the general picture can emerge. Using microbenchmark when you need to find the bottlenecks is how I'd proceed,
which OP is doing.
About 30 years ago, I did write up some preliminary work, never published, on estimating the two halves of a copy, that
is, the reading from file and storing to "memory" or a different storage location. This was via regression with a
singular design matrix, but one can get a minimal length least squares solution via svd. Possibly relevant today to try
to get at slow links on a network.
JN
On 2017-08-22 09:07 AM, Jeff Newmiller wrote:
You need to study how reading files works in your operating system. This
question is not about R.
______________________________________________
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
