I have a very large, dense matrix (too large to fit in RAM) "A" saved in an HDF5/JLD file. I also have several vectors of the same dimension v[1]..v[n], which *are* able to fit in memory, and I would like to know the result of multiplying each vector by my matrix A.
My current idea is to do this manually -- simply loading each row or column from the matrix dataset, one at a time, and implementing matrix-vector multiplication myself. However, it is possible that my large matrix is already stored is some sort of "chunked"/block form in the HDF5 file, and it would be nice to choose my blocks accordingly so the calculation happens as efficiently as possible. Is there a way to load all blocks of an HDF5 dataset in the most efficient way? In terms of the bigger picture, I've also considered that it might be nice to implement in JLD all general matrix-matrix and matrix-vector operations for JldDataset (which would throw an error when the dimensions of the objects do not match, or when the data types do not make sense for multiplication). But I could also see this being an unwelcome "feature," as it would be quite easy to accidentally call this even when it is possible to load the matrix into memory. (Also, it would not be the most efficient way to handle my problem, as it would involve loading the dataset n times, once for each vector v[i].)
