Re: AI 3.0: writeSynchronizationMode re-thinking

2019-05-15 Thread Anton Vinogradov
Sergey, Sorry for the late response. I'd think twice or more :) and discussed the idea with Alexey G. Seems, we have no reason to check some backups since AI should guarantee it has everything consistent. But, it useful to have some feature to check this guarantee is not bluff. BTW, I'm still

Re: AI 3.0: writeSynchronizationMode re-thinking

2019-04-30 Thread Sergey Kozlov
Anton I'm Ok with your proposal but IMO it should be provided as IEP? On Mon, Apr 29, 2019 at 4:05 PM Anton Vinogradov wrote: > Sergey, > > I'd like to continue the discussion since it closely linked to the problem > I'm currently working on. > > 1) writeSynchronizationMode should not be a

Re: AI 3.0: writeSynchronizationMode re-thinking

2019-04-29 Thread Anton Vinogradov
Sergey, I'd like to continue the discussion since it closely linked to the problem I'm currently working on. 1) writeSynchronizationMode should not be a part of cache configuration, agree. This should be up to the user to decide how strong should be "update guarantee". So, I propose to have a

Re: AI 3.0: writeSynchronizationMode re-thinking

2019-04-25 Thread Sergey Kozlov
There's another point to improve: if *syncPartitions=N* comes as the configurable in run-time it will allow to manage the consistency-performance balance runtime, e.g. switch to full async for preloading and then go to back to full sync for regular operations On Thu, Apr 25, 2019 at 6:48 PM

Re: AI 3.0: writeSynchronizationMode re-thinking

2019-04-25 Thread Sergey Kozlov
Vyacheskav, You're right with the referring to MongoDB doc. In general the idea is very similar. Many vendors use such approach (1). [1] https://dev.mysql.com/doc/refman/8.0/en/replication-options-master.html#sysvar_rpl_semi_sync_master_wait_for_slave_count On Thu, Apr 25, 2019 at 6:40

Re: AI 3.0: writeSynchronizationMode re-thinking

2019-04-25 Thread Vyacheslav Daradur
Hi, Sergey, Makes sense to me in case of performance issues, but may lead to losing data. >> *by the new option *syncPartitions=N* (not best name just for referring) Seems similar to "Write Concern"[1] in MongoDB. It is used in the same way as you described. On the other hand, if you have such

Re: AI 3.0: writeSynchronizationMode re-thinking

2019-04-25 Thread Sergey Kozlov
Ilya See comments inline. On Thu, Apr 25, 2019 at 5:11 PM Ilya Kasnacheev wrote: > Hello! > > When you have 2 backups and N = 1, how will conflicts be resolved? > > Imagine that you had N = 1, and primary node failed immediately after > operation. Now you have one backup that was updated

Re: AI 3.0: writeSynchronizationMode re-thinking

2019-04-25 Thread Anton Vinogradov
Sergey, +1 to your proposal. Looks pretty similar to Kafka's approach. Ilya, >> Will they stay unsynced, or is there any mechanism of re-syncing? Yes, I'm currently working [1] on it. The current implementation allows restoring the latest value. [1]

Re: AI 3.0: writeSynchronizationMode re-thinking

2019-04-25 Thread Ilya Kasnacheev
Hello! When you have 2 backups and N = 1, how will conflicts be resolved? Imagine that you had N = 1, and primary node failed immediately after operation. Now you have one backup that was updated synchronously and one which did not. Will they stay unsynced, or is there any mechanism of

AI 3.0: writeSynchronizationMode re-thinking

2019-04-25 Thread Sergey Kozlov
Igniters I'm working with the wide range of cache configurations and found (from my standpoint) the interesting point for the discussion: Now we have following *writeSynchronizationMode *options: 1. *FULL_ASYNC* - primary partition updated asynchronously - backup partitions