Guys, Currently when we do cache updates in FULL_SYNC mode we update near readers (near cache entries) synchronously. This is quite big drawback in design, I think. I get each near reader update at cost of 1 extra backup update. I think everyone understands that partitioned cache easily turns to replicated once near readers number increases. In TX cache cost of such updates doubles.
I do not see any benefit on updating near entries in sync way. Outdated reads can still be possible if I don't read from primary or in TX context. So, what I suggest: 1. introduce flag for cahce - withSyncNearUpdates() or extra CacheWriteSynchronizationMode.DHT_SYNC and make it default mode. 2. Near entries are updated in async way 2.1 in atomic mode together with backup updates 2.2 in TX mode after tx is committed on primary. I would also suggest to exclude near readers from lock acquisition/release steps. Only force updates. Updates order will be ensured by single primary node and per-partition striping. 3. Near readers do not respond to primary. Once primary fails near readers get invalidated, if primary is alive then communication recovery ensures that message will be delivered to near. Please share your thoughts. --Yakov
