Alexandr, 'local' prefix in Ignite APIs means that the method is invoked only on the current node, while its regular sibling is invoked in distributed fashion. localLoadCache doesn't imply that only local partitions are loaded. it turns out to work this way right now, but it doesn't mean that this can't be change (and I don't suggest to change default behavior, BTW).
Method overhead is decreased with my approach, if used properly. You can call localLoadCache with the data streamer based closure, and the database will be queried only from local node, and the local node will then distribute the data across other nodes. All I did is abstracted this logic of moving an entry from store to cache, because currently user doesn't have an option to override it. If you still believe this doesn't work, can you please elaborate what exactly you propose? What code should we add and/or change in Ignite and how user will use it API wise? -Val On Wed, Nov 16, 2016 at 5:40 AM, Dmitriy Setrakyan <dsetrak...@apache.org> wrote: > On Wed, Nov 16, 2016 at 1:54 PM, Yakov Zhdanov <yzhda...@apache.org> > wrote: > > > > On Wed, Nov 16, 2016 at 11:22 AM, Yakov Zhdanov <yzhda...@apache.org> > > wrote: > > > > > > > Yakov, I agree that such scenario should be avoided. I also think > > that > > > > > > > loadCache(...) method, as it is right now, provides a way to avoid > > it. > > > > > > > > > > > > No, it does not. > > > > > > > > > Yes it does :) > > > > No it doesn't. Load cache should either send a query to DB that filters > all > > the data on server side which, in turn, may result to full-scan of 2 Tb > > data set dozens of times (equal to node count) or send a query that > brings > > the whole dataset to each node which is unacceptable as well. > > > > Why not store the partition ID in the database and query only local > partitions? Whatever approach we design with a DataStreamer will be slower > than this. >
