hi devs
we are migrating an in-memory job platform to Apache Ignite 2.x (2.18) and I thought it would be worth sharing one experience from it, together with a proposal. Our jobs keep small lookup data (settings, thresholds, codes) in REPLICATED caches and read it once per processed record, so these reads run millions of times per job. I believe fast localreads are one of the main reasons to choose a replicated cache in the first place. We configure such caches for this case: on-heap cache enabled and copyOnRead off, so every server holds every value as an object and a read returns the stored instance. Still, a local read costs far more than the lookup it is (again an old implementation). Measured with the attached standalone reproducer (ignite- core only, two server nodes in one JVM, 32 entries, JDK 17, median of 5 rounds of 2 million calls per thread): Read-------------------------------------------- 1 thread --- 3 threads, same keys IgniteCache.get(key) --------------------------- 421 ns ------ 865 ns IgniteCache.localPeek(key, ONHEAP)----------- 507 ns ------ 881 ns ConcurrentHashMap.get(key), same instances-- 2.9 ns ------- 3.9 ns JDK 25 (even though it's officially not supported), run straight after as a cross-check, is faster but shows the same picture (get 258 and 638 ns, the map 3.3 and 4.3 ns). A profile (low level details in README.md) shows that a get takes the entry' s lock four times, builds a future with a random id, and passes two gateway read locks. After some extra analysis (however I'm not sure for 100%), it looks like that with several threads, reading the same keys, these locks and shared counters are contended and the cost doubles, even though nothing writes there. The proposal: an option for REPLICATED caches to keep their values as deserialized objects on every server and serve local reads from them with one map lookup - about 9 ns through our own workaround, some 25 to 50 times faster than get. We had to built such kind of a map ourselves to get there. The standalone reporducer project + README.md (written with AI assistance) is in the attachment. Please read the README.md, which describes the proposal and the measurements in more technical details. Would the community consider such an option for a future Ignite release? It would be nice to hear your view, and which shape, if any, fits Ignite's direction. Thank you in advance, I.P. P.S. Possibly related issues: IGNITE-15989, IGNITE-5574, IGNITE-4932
<<attachment: ignite-replicated-read-reproducer.zip>>
