Jake, You asked a bit ago about strategies for very large SVD's.
I wonder if interpolative decompositions might be an avenue toward that. See, for instance, Less is More: Compact Matrix Decomposition for Large Sparse Graphs <http://www.cs.cmu.edu/~jimeng/papers/SunSDM07.pdf> The idea is that if your basis vectors are sparse, you might do much better in terms of space.
