I have a cache with approx 1MM rows and is partitioned across 3 server nodes.
I am trying to retrieve all the keys from the cache and then trying to
perform an update operation using cache.invoke.
The cache.invoke itself is very fast but the retrieval of the keys takes a
lot of time - approximately 10-100 keys per second is the throughput I am
getting.
The cache has 15 fields and 4 of them are defined as key fields and are
indexed but since I am using Scanquery i am guessing that the indexes arent
used?
What is the best/correct way to retrieve all keys and perform an update to
the cache within the server nodes without returning any data back to the
client node.
Originally i had iterated through the iterator one key at a time and thought
the update operation was taking time. But i changed the code to create a
keyset first and then did a cache.invoke in separate blocks of code as
below:
// Execute the query.
Iterator<Cache.Entry<FactKey, Fact>> iterator =
cache.query(scanQuery).iterator();
Set<FactKey> KEYS_SET = new HashSet<FactKey>();
FactKey key = new FactKey();
Fact val = null;
ExecutionTimer t = new ExecutionTimer(); // this is just to track
time taken in seconds
t.start();
for (int i = 0; i < cache.size(CachePeekMode.ALL); i++) {
//while (iterator.hasNext()) {
key = iterator.next().getKey();
KEYS_SET.add(iterator.next().getKey());
if(i%1000 == 0)
{
System.out.println(">>> Update Count: " + i);
t.end();
System.out.println("Time taken for: " + i + " " + t.duration());
t.start();
cache.invokeAll(KEYS_SET, (entry, args) -> {
val = entry.getValue();
val.setAmount(val.getAmt1() + val.getAmt2());
;
return null;
});
KEYS_SET.clear();
System.out.println(">>> Updating ADB: " + i);
t.end();
System.out.println("Timer for cache invoke: " + t.duration());
t.start();
}
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
