Thank you very much, Paul, I'll have a look at your code. Francesco
On Thu, Dec 21, 2017 at 1:01 PM, Paul Theodor Pyl <paul.theodor....@embl.de> wrote: > Hi Francesco, > > this is certainly achievable with currently available HDF5 support in > R/Bioconductor. For example the rhdf5 package gives you access to this > functionality > (https://bioconductor.org/packages/release/bioc/html/rhdf5.html). > > rhdf5 is relatively 'low-level', in the sense that it is really close to the > HDF5 library it exposes to R (i.e. you get h5read an h5write functions). For > what you are describing I typically use a small wrapper to make my life a > bit easier, I have something like that on github here: > https://github.com/PaulPyl/h5array > > Please note that this is not an official Bioconductor package so it doesn't > fulfill the strict standards of documentation etc., since it is just a small > wrapper to give you an array-like object that writes/reads its data from > disk though, it should be fairly straightforward to use. > > Best, > Paul > > > On Thu, Dec 21, 2017 at 12:22, Francesco Napolitano <franap...@gmail.com> > wrote: > > Hi, > > I need to deal with very large matrices and I was thinking of using > HDF5-based data models. However, from the documentation and examples > that I have been looking at, I'm not quite sure how to do this. > > My use case is as follows. > I want to build a very large matrix one column at a time, and I need > to write columns directly to disk since I would otherwise run out of > memory. I need a format that, afterwards, will allow me to extract > subsets of rows or columns and rank them. The subsets will be small > enough to be loaded in memory. Can I achieve this with current HDF5 > support in R? > > Any help greatly appreciated. > > than you, > Francesco > > _______________________________________________ > Bioc-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/bioc-devel _______________________________________________ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
