Hi, If EntryProcessor returns null then null is not added in the result map. But I agree that using invokeAll() will have a lot of unnecessary overhead. Perhaps we need add new getAll method on API, otherwise best alternative is use custom ComputeJob or affinityCall.
Thanks, Semyon On Tue, Aug 29, 2017 at 7:20 AM, Dmitriy Setrakyan <[email protected]> wrote: > Andrey, > > I am not sure I understand. According to EntryProcessor API [1] you can > chose to return nothing. > > Also, to my knowledge, you can still do parallel reads while executing the > EntryProcessor. Perhaps other community members can elaborate on this. > > [1] https://static.javadoc.io/javax.cache/cache-api/1.0.0/ > index.html?javax/cache/processor/EntryProcessor.html > > D. > > > On Mon, Aug 28, 2017 at 8:29 PM, Andrey Kornev <[email protected]> > wrote: > >> Dmitriy, >> >> >> It's good to be back! 😃 Glad to find Ignite community as vibrant >> and thriving as ever! >> >> Speaking of invokeAll(), even if we ignore for a moment the overhead >> associated with locking/unlocking a cache entry prior to passing it to the >> EntryProcessor as well as the overhead associated with enlisting the >> touched entries in a transaction, the bigger problem with using >> invokeAll() for filtering is that EntryProcessor must return a value. I'm >> not aware of any way to make EntryProcessor drop the entry from the >> response. The only options is to use a null (or false) to indicate a >> filtered out entry. In my specific case, I'll end up sending back a whole >> bunch of nulls in the result map as I expect most of the keys to be >> rejected by the filter. >> >> Overall, invokeAll() is not what one would call *efficient* (the key word >> in my original question) way of filtering. >> >> Thanks! >> Andrey >> >> ------------------------------ >> *From:* Dmitriy Setrakyan <[email protected]> >> *Sent:* Saturday, August 26, 2017 8:37 AM >> *To:* user >> >> *Subject:* Re: Retrieving multiple keys with filtering >> >> Andrey, >> >> Good to hear from you. Long time no talk. >> >> I don't think invokeAll has only update semantics. You can definitely use >> it just to look at the keys and return a result. Also, as you mentioned, >> Ignite compute is a viable option as well. >> >> The reason that predicates were removed from the get methods is because >> the API was becoming unwary, and also because JCache does not require it. >> >> D. >> >> On Thu, Aug 24, 2017 at 10:50 AM, Andrey Kornev <[email protected] >> > wrote: >> >>> Well, I believe invokeAll() has "update" semantics and using it for >>> read-only filtering of cache entries is probably not going to be efficient >>> or even appropriate. >>> >>> >>> I'm afraid the only viable option I'm left with is to use Ignite's >>> Compute feature: >>> >>> - on the sender, group the keys by affinity. >>> >>> - send each group along with the filter predicate to their >>> affinity nodes using IgniteCompute. >>> >>> - on each node, use getAll() to fetch the local keys and apply the >>> filter. >>> >>> - on the sender node, collect the results of the compute jobs into a map. >>> >>> >>> It's unfortunate that Ignite dropped that original API. What used to be >>> a single API call is now a non-trivial algorithm and one have to worry >>> about things like what happens if the grid topology changes while the >>> compute jobs are executing, etc. >>> >>> Can anyone think of any other less complex/more robust approach? >>> >>> Thanks >>> Andrey >>> >>> ------------------------------ >>> *From:* slava.koptilin <[email protected]> >>> *Sent:* Thursday, August 24, 2017 9:03 AM >>> *To:* [email protected] >>> *Subject:* Re: Retrieving multiple keys with filtering >>> >>> Hi Andrey, >>> >>> Yes, you are right. ScanQuery scans all entries. >>> Perhaps, IgniteCache#invokeAll(keys, cacheEntryProcessor) with custom >>> processor will work for you. >>> https://ignite.apache.org/releases/2.1.0/javadoc/org/apache/ >>> ignite/IgniteCache.html#invokeAll(java.util.Set,%20org. >>> apache.ignite.cache.CacheEntryProcessor,%20java.lang.Object...) >>> >>> Thanks! >>> >>> >>> >>> -- >>> View this message in context: http://apache-ignite-users.705 >>> 18.x6.nabble.com/Retrieving-multiple-keys-with-filtering-tp1 >>> 6391p16400.html >>> Apache Ignite Users - Retrieving multiple keys with filtering >>> <http://apache-ignite-users.70518.x6.nabble.com/Retrieving-multiple-keys-with-filtering-tp16391p16400.html> >>> apache-ignite-users.70518.x6.nabble.com >>> Retrieving multiple keys with filtering. Hello, I have a list of cache >>> keys (up to a few hundred of them) and a filter predicate. I'd like to >>> efficiently retrieve only those values that pass the... >>> >>> >>> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >>> >> >> >
