Why not use the Iterable interface then? On Mon, Dec 13, 2010 at 10:57 AM, Dmitriy Lyubimov <[email protected]>wrote:
> No, no new vector types. All that is needed is push-type element handler > that the algorithm can set, something like > > void onNextVectorElement ( int index, double value ) throws IOException > > or something like that. > > IF the handler is set, then the writable uses it during read process, (and > subequently get() is not producing anything) , > > and if the handler is not set , then it works as it does today. > > On Mon, Dec 13, 2010 at 10:18 AM, Ted Dunning <[email protected]> > wrote: > > > Interesting idea. > > > > Would this introduce a new vector type that only allows iterating through > > the elements once? > > > > On Mon, Dec 13, 2010 at 9:49 AM, Dmitriy Lyubimov <[email protected]> > > wrote: > > > > > Hi all, > > > > > > I would like to submit a patch to VectorWritable that allows for > > streaming > > > access to vector elements without having to prebuffer all of them > first. > > > (current code allows for the latter only). > > > > > > That patch would allow to strike down one of the memory usage issues in > > > current Stochastic SVD implementation and effectively open memory bound > > for > > > n of the SVD work. (The value i see is not to open up the the bound > > though > > > but just be more efficient in memory use, thus essentially speeding u p > > the > > > computation. ) > > > > > > If it's ok, i would like to create a JIRA issue and provide a patch for > > it. > > > > > > Another issue is to provide an SSVD patch that depends on that patch > for > > > VectorWritable. > > > > > > Thank you. > > > -Dmitriy > > > > > >
