Michael, The mmap package currently provides an Ops method for comparisons, which returns something like which(i==) does in R - since a vector of logicals the same size would be likely too big to handle in many cases.
At some point I'll implement mmap to mmap operations, though for vectorized ops that result in non-logical output (i.e. numeric), I haven't yet decided on how that should be implemented. Something like a results buffer on disk/memory has been my thinking, but anyone with additional (better!) suggestions please feel free to send me ideas off list. I'll look to add some basic summary statistics as well. Note that you need to have a binary representation on disk (via fwrite in C, or writeBin or as.mmap in R) for this to work. But the package currently supports something like 16 data types, including bit logicals, fixed width character strings (\0 delim vectors), floats (4 byte), and 64 bit ints. The vignette covers a lot of the details. Additionally if you have struct-style data (think row-oriented, with varying types), you can use the struct() feature. This maps to an R list, but allows for very fast access if you are pulling complete rows. example(mmap) example(types) example(struct) The R-forge version has more than the CRAN version at this moment, but I'll be pushing a new one to CRAN soon. Jeff On Fri, Aug 5, 2011 at 8:22 AM, Michael Lachmann <lachm...@eva.mpg.de>wrote: > > On 5 Aug 2011, at 1:20AM, Dirk Eddelbuettel wrote: > > > When you know the (fixed) structure of the data, the CRAN package mmap > can be > > a huge winner. > > Thanks! I didn't know that. > > Is there a package that provides methods for mmap, like sum(x) or maybe > even y=x+z > where x, and z are mmaps? > > I assume that once you mmap to a huge file, you do operations on it by > working on chunks at a time... are there packages for that, or do I have to > write my own code? > > Thanks! > > Michael > ______________________________________________ > R-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > -- Jeffrey Ryan jeffrey.r...@lemnica.com www.lemnica.com www.esotericR.com [[alternative HTML version deleted]] ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
