Hi Anand, Looks like a cluster wide operation cache.size() performed on every iteration. Please, take a look how to iterating over cache entries via ScanQuery [1] .
[1] https://apacheignite.readme.io/v2.2/docs/cache-queries#scan-queries On Tue, Oct 24, 2017 at 7:40 PM, Anand Vijai <[email protected]> wrote: > I have a cache with approx 1MM rows and is partitioned across 3 server > nodes. > I am trying to retrieve all the keys from the cache and then trying to > perform an update operation using cache.invoke. > The cache.invoke itself is very fast but the retrieval of the keys takes a > lot of time - approximately 10-100 keys per second is the throughput I am > getting. > The cache has 15 fields and 4 of them are defined as key fields and are > indexed but since I am using Scanquery i am guessing that the indexes arent > used? > > What is the best/correct way to retrieve all keys and perform an update to > the cache within the server nodes without returning any data back to the > client node. > > Originally i had iterated through the iterator one key at a time and > thought > the update operation was taking time. But i changed the code to create a > keyset first and then did a cache.invoke in separate blocks of code as > below: > > // Execute the query. > Iterator<Cache.Entry<FactKey, Fact>> iterator = > cache.query(scanQuery).iterator(); > Set<FactKey> KEYS_SET = new HashSet<FactKey>(); > FactKey key = new FactKey(); > Fact val = null; > ExecutionTimer t = new ExecutionTimer(); // this is just to track > time taken in seconds > t.start(); > > for (int i = 0; i < cache.size(CachePeekMode.ALL); i++) { > //while (iterator.hasNext()) { > key = iterator.next().getKey(); > > KEYS_SET.add(iterator.next().getKey()); > > if(i%1000 == 0) > { > System.out.println(">>> Update Count: " + i); > t.end(); > System.out.println("Time taken for: " + i + " " + > t.duration()); > t.start(); > cache.invokeAll(KEYS_SET, (entry, args) -> { > val = entry.getValue(); > val.setAmount(val.getAmt1() + val.getAmt2()); > ; > return null; > }); > KEYS_SET.clear(); > System.out.println(">>> Updating ADB: " + i); > t.end(); > System.out.println("Timer for cache invoke: " + t.duration()); > t.start(); > } > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > -- Best regards, Andrey V. Mashenkov
