Partition will not migrate if local or remote iterator is not finished/closed. On Dec 11, 2015 21:05, "Andrey Kornev" <[email protected]> wrote:
> Great suggestion! Thank you, Yakov! > > Just one more question. :) Let's say the scan job is running node A and > processing partition 42. At the same time, a new node B joins and partition > 42 needs to be moved to this node. What will happen to my scan query that > is still running on node A and iterating over the partition's entries? > Would it complete processing the entire partition despite the change of > ownership? Or, would the query terminate at some arbitrary point once the > partition ownership transfer has completed? > > Thanks a lot! > Andrey > > ------------------------------ > Date: Fri, 11 Dec 2015 16:06:16 +0300 > Subject: Re: Computation on NodeEntries > From: [email protected] > To: [email protected] > > Guys, I would do the following: > > 1. Map all my partitions to > nodes: org.apache.ignite.cache.affinity.Affinity#mapPartitionsToNodes > 2. Send jobs (with its list of partitions) to each node using map returned > on step1 > 3. Job may be like: > > new Runnable() { > @Override public void run() { > for (Integer part : parts) { > Iterator<Cache.Entry<Object, Object>> it = cache.query(new > ScanQuery<>(part)).iterator(); > > // do the stuff... > } > > } > }; > > This may result in network calls for some worst cases when topology changes > under your feet, but even in this case this should work. > > > --Yakov > > 2015-12-11 2:13 GMT+03:00 Andrey Kornev <[email protected]>: > > Dmitriy, > > Given the approach you suggested below, what would be your recommendation > for dealing with cluster topology changes while the iteration is in > progress? An obvious one I can think of is to > - somehow detect the change, > - cancel the tasks on all the nodes > - wait until the rebalancing is finished and > - restart the computation. > > Are there any other ways? Ideally, I'd like to have the "exactly-once" > execution semantics. > > Thanks > Andrey > >
