I responded in the ticket: https://issues.apache.org/jira/browse/IGNITE-3222
On Thu, Jun 2, 2016 at 2:10 AM, Pavel Tupitsyn <[email protected]> wrote: > Dmitriy, affinityCall/Run over a partition is useful, I agree. > > But dealing with partitions manually is also error-prone and not obvious at > all. > Invoke over all cache entries is a simple common task, why not provide a > simple method of doing it? > > What kind of memory issues do you expect? > Returning EntryProcessorResult for each entry on a huge cache is an obvious > problem, > same as calling getAll on QueryCursor, for example. We can't protect users > from such things. > > Pavel. > > On Wed, Jun 1, 2016 at 3:18 AM, Dmitriy Setrakyan <[email protected]> > wrote: > > > Pavel, > > > > I actually believe that such method will be error-prone and will cause > all > > sorts of memory issues for users trying to execute this method over large > > caches. > > > > What we need instead is an affinityCall/Run method over a partition, not > a > > key. Why not provide this method instead? > > > > Added my comment to the ticket. > > > > D. > > > > On Tue, May 31, 2016 at 6:18 AM, Pavel Tupitsyn <[email protected]> > > wrote: > > > > > Ok then, looks like there are no obstacles or objections, I've created > a > > > JIRA ticket: > > > https://issues.apache.org/jira/browse/IGNITE-3222 > > > > > > Thanks, > > > Pavel. > > > > > > On Tue, May 31, 2016 at 3:44 PM, Denis Magda <[email protected]> > > wrote: > > > > > > > Pavel, > > > > > > > > > This seems to be a common task, why don't we implement invokeAll > as I > > > > > suggested above? > > > > > > > > I would implement such a method using the approach shown in the > > example. > > > > In my understanding it would be the most efficient way. Data locality > > > will > > > > longer be not an issue when IGNITE-2310 is implemented. > > > > > > > > — > > > > Denis > > > > > > > > > On May 31, 2016, at 1:16 PM, Pavel Tupitsyn < > [email protected]> > > > > wrote: > > > > > > > > > > Denis, thank you, this may work, but: > > > > > * it is too complicated > > > > > * it does not guarantee locality > > > > > > > > > > This seems to be a common task, why don't we implement invokeAll > as I > > > > > suggested above? > > > > > > > > > > On Tue, May 31, 2016 at 1:05 PM, Denis Magda <[email protected]> > > > > wrote: > > > > > > > > > >> Pavel, > > > > >> > > > > >> Here is an example on how to execute scan queries on partitions’ > > > owners > > > > >> and perform some operation for every entry > > > > >> > > > > >> > > > > > > > > > > https://github.com/gridgain/gridgain-advanced-examples/blob/master/src/main/java/org/gridgain/examples/datagrid/query/ScanQueryExample.java > > > > >> < > > > > >> > > > > > > > > > > https://github.com/gridgain/gridgain-advanced-examples/blob/master/src/main/java/org/gridgain/examples/datagrid/query/ScanQueryExample.java > > > > >>> > > > > >> > > > > >> When this feature is implemented [1] it will be possible to > postpone > > > > >> partitions movement until such an operation is in progress. > > Presently > > > if > > > > >> the rebalancing happens the data will be retrieved from a remote > > node > > > > (new > > > > >> partition owner), however the result will be consistent in any > case. > > > > >> > > > > >> [1] https://issues.apache.org/jira/browse/IGNITE-2310 < > > > > >> https://issues.apache.org/jira/browse/IGNITE-2310> > > > > >> > > > > >> — > > > > >> Denis > > > > >> > > > > >>> On May 31, 2016, at 7:42 AM, Yakov Zhdanov < > [email protected]> > > > > >> wrote: > > > > >>> > > > > >>> Vova, even though it operates on single key we will need to "pin" > > > > >> partition > > > > >>> so key does not go to another node. > > > > >>> > > > > >>> Pavel, you can also send closures to all primary nodes to do > local > > > scan > > > > >>> query for each partition. This way you will go over each entry. > > > > >>> > > > > >>> Thanks! > > > > >>> -- > > > > >>> Yakov Zhdanov, Director R&D > > > > >>> *GridGain Systems* > > > > >>> www.gridgain.com > > > > >>> > > > > >>> 2016-05-30 16:05 GMT-04:00 Vladimir Ozerov <[email protected] > >: > > > > >>> > > > > >>>> Affinity run/call operate on a single key AFAIK. > > > > >>>> > > > > >>>> On Mon, May 30, 2016 at 10:55 PM, Dmitriy Setrakyan < > > > > >> [email protected] > > > > >>>>> > > > > >>>> wrote: > > > > >>>> > > > > >>>>> Actually I have seen a ticket to block moving partitions if > > > > >> affinityCall > > > > >>>> or > > > > >>>>> affinityRun are called. I think once these tickets are > > implemented, > > > > the > > > > >>>>> process will become reliable, no? > > > > >>>>> > > > > >>>>> D. > > > > >>>>> > > > > >>>>> On Mon, May 30, 2016 at 9:13 AM, Pavel Tupitsyn < > > > > >> [email protected]> > > > > >>>>> wrote: > > > > >>>>> > > > > >>>>>> Dmitriy, as I understand, there is no reliable way to do that > if > > > > >>>>>> rebalancing happens. > > > > >>>>>> > > > > >>>>>> On Mon, May 30, 2016 at 6:50 PM, Dmitriy Setrakyan < > > > > >>>>> [email protected]> > > > > >>>>>> wrote: > > > > >>>>>> > > > > >>>>>>> I think we do support this use case. Why not send a > computation > > > to > > > > a > > > > >>>>>> server > > > > >>>>>>> and then perform the iteration through the cache entries > > locally > > > on > > > > >>>>> that > > > > >>>>>>> server? > > > > >>>>>>> > > > > >>>>>>> On Mon, May 30, 2016 at 4:44 AM, Pavel Tupitsyn < > > > > >>>>> [email protected]> > > > > >>>>>>> wrote: > > > > >>>>>>> > > > > >>>>>>>> Igniters, > > > > >>>>>>>> > > > > >>>>>>>> Looks like we do not have an efficient way to perform an > > action > > > on > > > > >>>>>> EVERY > > > > >>>>>>>> cache entry. > > > > >>>>>>>> > > > > >>>>>>>> Let's say I want to remove all entries that match a > predicate. > > > > >>>>>>>> My only option is to retrieve these entries via Scan or SQL > > > query, > > > > >>>>> and > > > > >>>>>>> then > > > > >>>>>>>> call removeAll. > > > > >>>>>>>> This involves a lot of unnecessary network trips (send keys > to > > > > >>>> caller > > > > >>>>>>> node, > > > > >>>>>>>> send them back to primary nodes). > > > > >>>>>>>> > > > > >>>>>>>> Would it be possible to implement a method like > > > > >>>>>>>> void IgniteCache.invokeAll(entryProcessor) > > > > >>>>>>>> that invokes the processor on all entries and does not > return > > > > >>>>> anything? > > > > >>>>>>>> There could be more overloads that return results or only > > return > > > > >>>>>> results > > > > >>>>>>>> for changed entries. > > > > >>>>>>>> > > > > >>>>>>>> Thoughts? > > > > >>>>>>>> > > > > >>>>>>>> Pavel. > > > > >>>>>>>> > > > > >>>>>>> > > > > >>>>>> > > > > >>>>> > > > > >>>> > > > > >> > > > > >> > > > > > > > > > > > > > >
