As a side note - can it be so that lambda passed to consume() is executed in 1 thread?
I agree with Sergi that string keys should be replaced with ints. Alex G, can you check how we map invoke in atomic cache? Can it be so that processing 27 keys may end up with sending 13 or 14 messages if keys belong to nodes this way - local node, remote node, local, remote... Can we optimize this to send 1 message and then process local set? On Sep 1, 2015 19:31, "Sergi Vladykin" <[email protected]> wrote: > Sorry, just noticed that you already use field values as cache keys. > But simplifying counters to ints instead of HashMaps of HashSets is > probably still makes sense to try. > > Sergi > > 2015-09-01 19:19 GMT+03:00 Sergi Vladykin <[email protected]>: > >> By the way you can also use field id (just number your fields from 0 to >> 27) instead of storing field names in counter keys, it will give some >> serialization speedup and better memory utilization. >> >> Sergi >> >> 2015-09-01 19:10 GMT+03:00 Sergi Vladykin <[email protected]>: >> >>> As I see you have your field names as keys, you can think of them as >>> locks. >>> I mean that your scalability is limited by these 27 locks and you are >>> acquiring them all for each cache update. >>> How many threads do you use for your latency test? >>> I'd suggest to use something like tuple (fieldName, fieldValue) as key >>> and >>> just an integer (number of such keys) as value which must be incremented >>> or set to 1 if none. >>> >>> Sergi >>> >>> >>> 2015-09-01 17:38 GMT+03:00 javadevmtl <[email protected]>: >>> >>>> Sorry for the confusion... >>>> >>>> So... >>>> >>>> - Just to be clear running 1.3.0 on 2 nodes with total of 64 cores and >>>> 256GB >>>> of RAM >>>> - Cache is configured as off-heap partitioned no backups with 96GB and >>>> each >>>> node is started as -Xmx4g >>>> - Each app is a web server that receives json request that parses the >>>> json >>>> and inserts the json properties as different keys (27 total). >>>> - Requests are load balanced to both nodes. >>>> >>>> All times quoted below includes full business logic and network time. >>>> The >>>> json payload sent always contains the 27 properties. Only the app is >>>> recompiled to either invokeAll() for 1 key 3 keys, 9 keys etc... >>>> >>>> Also time remains constant from 0 records inserted all the way up to >>>> millions (so at least this is good thing) which also means java is >>>> sufficiently warmed up with millions of calls. >>>> >>>> 01 keys: 2ms >>>> 03 keys: 4ms >>>> 09 keys: 9ms >>>> 18 keys: 16ms >>>> 27 keys: 25ms >>>> >>>> I was maybe hopping slightly better latency... Here is the code maybe >>>> you >>>> have suggestions on improvements? >>>> >>>> http://pastebin.com/pR36DwdG >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://apache-ignite-users.70518.x6.nabble.com/Is-invokeAll-considered-a-batch-operation-tp1220p1238.html >>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >>>> >>> >>> >> >
