Hi all,
I have been looking at lowering the time spent in I/O while using fread on a
csv file. Following a suggestion on the mailing list, I attempted to use fread
on a csv file stored on a ramdisk. It took 5 times longer to read from the Ram
Disk than it did to read from the SSD.
I presume this has to do with the fact that the data is being copied from RAM
to RAM rather than from Disk to RAM.
Any suggestions on if there are alternative methods to read files in faster
would be very helpful. At the moment.
Benchmarks on time taken for reading from SSD vs Ramdisk on the same 416MB file
is given below
####
> timer = proc.time(); z <- fread("testInRAM.csv"); proc.time() - timer
user system elapsed
25.067 0.433 25.485 ##### Read from RAMDisk
> setwd("/Users/xbsd/")
> timer = proc.time(); z <- fread("test.csv"); proc.time() - timer
user system elapsed
5.507 0.177 5.680 ###### Read from SSD
> system("ls -alh test.csv")
-rw-r--r-- 1 xbsd staff 416M Jul 12 19:30 test.csv
> system("ls -alh /Volumes/ramdisk/testInRAM.csv")
-rw-r--r-- 1 xbsd staff 416M Jul 12 19:32 /Volumes/ramdisk/testInRAM.csv
Thanks in advance,
- Raj._______________________________________________
datatable-help mailing list
[email protected]
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/datatable-help
