On Sep 8, 2010, at 4:31 PM, Jake Mannix wrote: > Hey all, > One question I'm wrestling with is whether we should have disk-backed > (local or HDFS) vectors and matrices transparently implement Vector and > Matrix, or whether this will lead to confusion and trick people into > treating them in the same way they think of in-memory versions (ie think > they can do lots of random access operations which just thrash and aren't > performant). > > Any thoughts?
I'd say add the disk backed ones and we'll worry about education separately. Perhaps it's possible for the vector to keep track of thrashing and spit out warnings. Either that or you override the random accessors on the file-based ones to throw exceptions so that it fails early for users. -Grant
