On Fri, Jul 15, 2016 at 1:49 AM, Alexey Goncharuk < [email protected]> wrote:
> > > > > > > > I know there is IgniteDataStreamer for writing cache, but how about > > > reading cache as stream for iterate all elements with scan performane > > 1-3M > > > tuple/sec? > > > > > > > We already have Scan queries which allow for paginated iteration with > > filters. Are you suggesting something beyond this? > > > I like the idea of DataStreamer approach for scanning a cache. I think it > would be nice to have a way to iterate over cache partitions in parallel, > similar to forEachPartition() method in Spark RDD. > > Benefits compared to current Scan query: > * Parallel execution for different partitions > * Bringing computation to data, not data to client. > > Of course, this can already be implemented by a user with local scan query > + compute task, but having an utility method on an API will cut a lot of > boilerplate code for users. > Got it now. Sounds very useful. I think we should definitely create a ticket for it and see if anyone in the community will pick it up. Sounds like it won’t be too difficult to implement.
