Re: MVCC configuration

[Semyon Boikov]

Yes, we'll add this validation.

On Wed, Sep 20, 2017 at 10:09 AM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> On Tue, Sep 19, 2017 at 11:31 PM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > > Can caches within the same group have different MVCC configuration?
> >
> > Do you think we really need have in the same group caches with different
> > mvcc configuration? for simplicity I would do not allow this.
> >
>
> I agree, let's not allow it. In that case, are you going to have extra
> validation on startup that caches in the same group must have identical
> MVCC config?
>

Re: MVCC configuration

[Dmitriy Setrakyan]

On Tue, Sep 19, 2017 at 11:31 PM, Semyon Boikov <sboi...@gridgain.com>
wrote:

> > Can caches within the same group have different MVCC configuration?
>
> Do you think we really need have in the same group caches with different
> mvcc configuration? for simplicity I would do not allow this.
>

I agree, let's not allow it. In that case, are you going to have extra
validation on startup that caches in the same group must have identical
MVCC config?

Re: MVCC configuration

[Semyon Boikov]

> Can caches within the same group have different MVCC configuration?

I think it is possible to implement, but there are some issues:
- for mvcc we need store mvcc version in hash index item (for now it is 16
bytes), since index items have fixed size then if we store in this index
data for caches with disabled mvcc, then it will have unnecessary 16 bytes
overhead
- for mvcc caches we need create correct hash index in advance, so if group
was created with mvcc disabled, then later it is not possible to add in
this group mvcc enabled cache

Do you think we really need have in the same group caches with different
mvcc configuration? for simplicity I would do not allow this.

Thanks

On Wed, Sep 20, 2017 at 7:30 AM, Dmitriy Setrakyan <dsetrak...@apache.org>
wrote:

> Can caches within the same group have different MVCC configuration?
>
> On Tue, Sep 19, 2017 at 2:34 AM, Vladimir Ozerov <voze...@gridgain.com>
> wrote:
>
> > What I mean is that it might be not applicable for DML by design. E.g.
> may
> > be we will have to fallback to per-memory-policy approach, or to
> > per-cache-group. As we do not know it at the moment and there is no clear
> > demand from users, I would simply put it aside to avoid in mess in public
> > API in future.
> >
> > Moreover, per-cache flag raises additional questions which can be put out
> > of scope otherwise. E.g. is it legal to mix MVCC and non-MVCC caches in a
> > single transaction? If yes, what is the contract? Without fine-grained
> > per-cache approach in the first iteration we can avoid answering it.
> >
> > On Tue, Sep 19, 2017 at 12:25 PM, Semyon Boikov <sboi...@gridgain.com>
> > wrote:
> >
> > > If it is not valid for DML then we can easily detect this situation and
> > > throw exception, but if I do not use DML why non make it configurable
> > > per-cache?
> > >
> > > On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > wrote:
> > >
> > > > I would say that per-cache configuration should be out of scope as
> well
> > > for
> > > > the first iteration. Because we do not know whether it will be valid
> > for
> > > > DML.
> > > >
> > > > On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov <
> sboi...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Folks, thank you for feedback, I want to summarize some decisions:
> > > > >
> > > > > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > > > > per-cache flag - CacheConfiguration.isMvccEnabled, default value
> for
> > > all
> > > > > caches - IgniteConfiguration.isMvccEnabled.
> > > > > 2. For initial implementation mvcc for ATOMIC cache is out of
> scope,
> > it
> > > > can
> > > > > be enabled only for TRANSACTIONAL caches.
> > > > > 3. Mvcc coordinator can be any server node (oldest server node is
> > > > selected
> > > > > automatically). Also I believe we need possibility to have
> > *dedicated*
> > > > mvcc
> > > > > coordinator nodes which will process only internal mvcc messages.
> > Node
> > > > can
> > > > > be marked as dedicated coordinator via new flag
> > > > > IgniteConfiguration.isMvccCoordinator or we can add separate
> > > > > MvccConfiguration bean. But let's skip this decision for now before
> > we
> > > > have
> > > > > benchmarks numbers.
> > > > > 4. Need add some metrics to monitor mvcc coordinator performance.
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <
> > > voze...@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > This could be something like "preferredMvccCoordinator".
> > > > > >
> > > > > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > > > > alexey.goncha...@gmail.com> wrote:
> > > > > >
> > > > > > > >
> > > > > > > > I agree that we need coordinator nodes, but I do not
> understand
> > > why
> > > > > > can't
> > > > > > > > we reuse some cache nodes for it? Why do we need to ask user
> to
> > > > start
> > > > > > up
> > > > > > > > yet another type of node?
> > > > > > > >
> > > > > > >
> > > > > > > Dmitriy,
> > > > > > >
> > > > > > > My understanding is that Semyon does not deny a cache node to
> be
> > > used
> > > > > as
> > > > > > a
> > > > > > > coordinator. This property will allow to optionally have a
> > > > *dedicated*
> > > > > > node
> > > > > > > serving as a coordinator to improve cluster throughput under
> > heavy
> > > > > load.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: MVCC configuration

[Dmitriy Setrakyan]

Can caches within the same group have different MVCC configuration?

On Tue, Sep 19, 2017 at 2:34 AM, Vladimir Ozerov <voze...@gridgain.com>
wrote:

> What I mean is that it might be not applicable for DML by design. E.g. may
> be we will have to fallback to per-memory-policy approach, or to
> per-cache-group. As we do not know it at the moment and there is no clear
> demand from users, I would simply put it aside to avoid in mess in public
> API in future.
>
> Moreover, per-cache flag raises additional questions which can be put out
> of scope otherwise. E.g. is it legal to mix MVCC and non-MVCC caches in a
> single transaction? If yes, what is the contract? Without fine-grained
> per-cache approach in the first iteration we can avoid answering it.
>
> On Tue, Sep 19, 2017 at 12:25 PM, Semyon Boikov <sboi...@gridgain.com>
> wrote:
>
> > If it is not valid for DML then we can easily detect this situation and
> > throw exception, but if I do not use DML why non make it configurable
> > per-cache?
> >
> > On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov <voze...@gridgain.com>
> > wrote:
> >
> > > I would say that per-cache configuration should be out of scope as well
> > for
> > > the first iteration. Because we do not know whether it will be valid
> for
> > > DML.
> > >
> > > On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov <sboi...@gridgain.com>
> > > wrote:
> > >
> > > > Folks, thank you for feedback, I want to summarize some decisions:
> > > >
> > > > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > > > per-cache flag - CacheConfiguration.isMvccEnabled, default value for
> > all
> > > > caches - IgniteConfiguration.isMvccEnabled.
> > > > 2. For initial implementation mvcc for ATOMIC cache is out of scope,
> it
> > > can
> > > > be enabled only for TRANSACTIONAL caches.
> > > > 3. Mvcc coordinator can be any server node (oldest server node is
> > > selected
> > > > automatically). Also I believe we need possibility to have
> *dedicated*
> > > mvcc
> > > > coordinator nodes which will process only internal mvcc messages.
> Node
> > > can
> > > > be marked as dedicated coordinator via new flag
> > > > IgniteConfiguration.isMvccCoordinator or we can add separate
> > > > MvccConfiguration bean. But let's skip this decision for now before
> we
> > > have
> > > > benchmarks numbers.
> > > > 4. Need add some metrics to monitor mvcc coordinator performance.
> > > >
> > > >
> > > > Thanks
> > > >
> > > > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <
> > voze...@gridgain.com>
> > > > wrote:
> > > >
> > > > > This could be something like "preferredMvccCoordinator".
> > > > >
> > > > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > > > alexey.goncha...@gmail.com> wrote:
> > > > >
> > > > > > >
> > > > > > > I agree that we need coordinator nodes, but I do not understand
> > why
> > > > > can't
> > > > > > > we reuse some cache nodes for it? Why do we need to ask user to
> > > start
> > > > > up
> > > > > > > yet another type of node?
> > > > > > >
> > > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > My understanding is that Semyon does not deny a cache node to be
> > used
> > > > as
> > > > > a
> > > > > > coordinator. This property will allow to optionally have a
> > > *dedicated*
> > > > > node
> > > > > > serving as a coordinator to improve cluster throughput under
> heavy
> > > > load.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: MVCC configuration

[Yakov Zhdanov]

Guys, I we should additionally provide ability to manually switch mvcc
coordinator via MBean passing order or ID of a new one. We already have all
the machinery for this.

--Yakov

Re: MVCC configuration

[Vladimir Ozerov]

What I mean is that it might be not applicable for DML by design. E.g. may
be we will have to fallback to per-memory-policy approach, or to
per-cache-group. As we do not know it at the moment and there is no clear
demand from users, I would simply put it aside to avoid in mess in public
API in future.

Moreover, per-cache flag raises additional questions which can be put out
of scope otherwise. E.g. is it legal to mix MVCC and non-MVCC caches in a
single transaction? If yes, what is the contract? Without fine-grained
per-cache approach in the first iteration we can avoid answering it.

On Tue, Sep 19, 2017 at 12:25 PM, Semyon Boikov 
wrote:

> If it is not valid for DML then we can easily detect this situation and
> throw exception, but if I do not use DML why non make it configurable
> per-cache?
>
> On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov 
> wrote:
>
> > I would say that per-cache configuration should be out of scope as well
> for
> > the first iteration. Because we do not know whether it will be valid for
> > DML.
> >
> > On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov 
> > wrote:
> >
> > > Folks, thank you for feedback, I want to summarize some decisions:
> > >
> > > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > > per-cache flag - CacheConfiguration.isMvccEnabled, default value for
> all
> > > caches - IgniteConfiguration.isMvccEnabled.
> > > 2. For initial implementation mvcc for ATOMIC cache is out of scope, it
> > can
> > > be enabled only for TRANSACTIONAL caches.
> > > 3. Mvcc coordinator can be any server node (oldest server node is
> > selected
> > > automatically). Also I believe we need possibility to have *dedicated*
> > mvcc
> > > coordinator nodes which will process only internal mvcc messages. Node
> > can
> > > be marked as dedicated coordinator via new flag
> > > IgniteConfiguration.isMvccCoordinator or we can add separate
> > > MvccConfiguration bean. But let's skip this decision for now before we
> > have
> > > benchmarks numbers.
> > > 4. Need add some metrics to monitor mvcc coordinator performance.
> > >
> > >
> > > Thanks
> > >
> > > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov <
> voze...@gridgain.com>
> > > wrote:
> > >
> > > > This could be something like "preferredMvccCoordinator".
> > > >
> > > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > > alexey.goncha...@gmail.com> wrote:
> > > >
> > > > > >
> > > > > > I agree that we need coordinator nodes, but I do not understand
> why
> > > > can't
> > > > > > we reuse some cache nodes for it? Why do we need to ask user to
> > start
> > > > up
> > > > > > yet another type of node?
> > > > > >
> > > > >
> > > > > Dmitriy,
> > > > >
> > > > > My understanding is that Semyon does not deny a cache node to be
> used
> > > as
> > > > a
> > > > > coordinator. This property will allow to optionally have a
> > *dedicated*
> > > > node
> > > > > serving as a coordinator to improve cluster throughput under heavy
> > > load.
> > > > >
> > > >
> > >
> >
>

Re: MVCC configuration

[Semyon Boikov]

If it is not valid for DML then we can easily detect this situation and
throw exception, but if I do not use DML why non make it configurable
per-cache?

On Tue, Sep 19, 2017 at 12:22 PM, Vladimir Ozerov 
wrote:

> I would say that per-cache configuration should be out of scope as well for
> the first iteration. Because we do not know whether it will be valid for
> DML.
>
> On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov 
> wrote:
>
> > Folks, thank you for feedback, I want to summarize some decisions:
> >
> > 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> > per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
> > caches - IgniteConfiguration.isMvccEnabled.
> > 2. For initial implementation mvcc for ATOMIC cache is out of scope, it
> can
> > be enabled only for TRANSACTIONAL caches.
> > 3. Mvcc coordinator can be any server node (oldest server node is
> selected
> > automatically). Also I believe we need possibility to have *dedicated*
> mvcc
> > coordinator nodes which will process only internal mvcc messages. Node
> can
> > be marked as dedicated coordinator via new flag
> > IgniteConfiguration.isMvccCoordinator or we can add separate
> > MvccConfiguration bean. But let's skip this decision for now before we
> have
> > benchmarks numbers.
> > 4. Need add some metrics to monitor mvcc coordinator performance.
> >
> >
> > Thanks
> >
> > On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov 
> > wrote:
> >
> > > This could be something like "preferredMvccCoordinator".
> > >
> > > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > > alexey.goncha...@gmail.com> wrote:
> > >
> > > > >
> > > > > I agree that we need coordinator nodes, but I do not understand why
> > > can't
> > > > > we reuse some cache nodes for it? Why do we need to ask user to
> start
> > > up
> > > > > yet another type of node?
> > > > >
> > > >
> > > > Dmitriy,
> > > >
> > > > My understanding is that Semyon does not deny a cache node to be used
> > as
> > > a
> > > > coordinator. This property will allow to optionally have a
> *dedicated*
> > > node
> > > > serving as a coordinator to improve cluster throughput under heavy
> > load.
> > > >
> > >
> >
>

Re: MVCC configuration

[Vladimir Ozerov]

I would say that per-cache configuration should be out of scope as well for
the first iteration. Because we do not know whether it will be valid for
DML.

On Tue, Sep 19, 2017 at 12:15 PM, Semyon Boikov 
wrote:

> Folks, thank you for feedback, I want to summarize some decisions:
>
> 1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
> per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
> caches - IgniteConfiguration.isMvccEnabled.
> 2. For initial implementation mvcc for ATOMIC cache is out of scope, it can
> be enabled only for TRANSACTIONAL caches.
> 3. Mvcc coordinator can be any server node (oldest server node is selected
> automatically). Also I believe we need possibility to have *dedicated* mvcc
> coordinator nodes which will process only internal mvcc messages. Node can
> be marked as dedicated coordinator via new flag
> IgniteConfiguration.isMvccCoordinator or we can add separate
> MvccConfiguration bean. But let's skip this decision for now before we have
> benchmarks numbers.
> 4. Need add some metrics to monitor mvcc coordinator performance.
>
>
> Thanks
>
> On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov 
> wrote:
>
> > This could be something like "preferredMvccCoordinator".
> >
> > On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> > alexey.goncha...@gmail.com> wrote:
> >
> > > >
> > > > I agree that we need coordinator nodes, but I do not understand why
> > can't
> > > > we reuse some cache nodes for it? Why do we need to ask user to start
> > up
> > > > yet another type of node?
> > > >
> > >
> > > Dmitriy,
> > >
> > > My understanding is that Semyon does not deny a cache node to be used
> as
> > a
> > > coordinator. This property will allow to optionally have a *dedicated*
> > node
> > > serving as a coordinator to improve cluster throughput under heavy
> load.
> > >
> >
>

Re: MVCC configuration

[Semyon Boikov]

Folks, thank you for feedback, I want to summarize some decisions:

1. Mvcc is disabled by default. We'll add two flags to enable mvcc:
per-cache flag - CacheConfiguration.isMvccEnabled, default value for all
caches - IgniteConfiguration.isMvccEnabled.
2. For initial implementation mvcc for ATOMIC cache is out of scope, it can
be enabled only for TRANSACTIONAL caches.
3. Mvcc coordinator can be any server node (oldest server node is selected
automatically). Also I believe we need possibility to have *dedicated* mvcc
coordinator nodes which will process only internal mvcc messages. Node can
be marked as dedicated coordinator via new flag
IgniteConfiguration.isMvccCoordinator or we can add separate
MvccConfiguration bean. But let's skip this decision for now before we have
benchmarks numbers.
4. Need add some metrics to monitor mvcc coordinator performance.


Thanks

On Tue, Sep 19, 2017 at 10:47 AM, Vladimir Ozerov 
wrote:

> This could be something like "preferredMvccCoordinator".
>
> On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
>
> > >
> > > I agree that we need coordinator nodes, but I do not understand why
> can't
> > > we reuse some cache nodes for it? Why do we need to ask user to start
> up
> > > yet another type of node?
> > >
> >
> > Dmitriy,
> >
> > My understanding is that Semyon does not deny a cache node to be used as
> a
> > coordinator. This property will allow to optionally have a *dedicated*
> node
> > serving as a coordinator to improve cluster throughput under heavy load.
> >
>

Re: MVCC configuration

[Vladimir Ozerov]

This could be something like "preferredMvccCoordinator".

On Tue, Sep 19, 2017 at 10:40 AM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> >
> > I agree that we need coordinator nodes, but I do not understand why can't
> > we reuse some cache nodes for it? Why do we need to ask user to start up
> > yet another type of node?
> >
>
> Dmitriy,
>
> My understanding is that Semyon does not deny a cache node to be used as a
> coordinator. This property will allow to optionally have a *dedicated* node
> serving as a coordinator to improve cluster throughput under heavy load.
>

Re: MVCC configuration

[Alexey Goncharuk]

>
> I agree that we need coordinator nodes, but I do not understand why can't
> we reuse some cache nodes for it? Why do we need to ask user to start up
> yet another type of node?
>

Dmitriy,

My understanding is that Semyon does not deny a cache node to be used as a
coordinator. This property will allow to optionally have a *dedicated* node
serving as a coordinator to improve cluster throughput under heavy load.

Re: MVCC configuration

[Dmitriy Setrakyan]

On Mon, Sep 18, 2017 at 2:39 AM, Semyon Boikov <sboi...@apache.org> wrote:

>
> 1. MVCC will definitely bring some performance overhead, so I think it
> should not be enabled by default, I'm going to add special flag on cache
> configuration: CacheConfiguration.isMvccEnabled.
>

Is it possible for several caches in the same cache group to have different
MVCC configuration?


> 2. In current mvcc architecture there should be some node in cluster
> assigning versions for tx updates and queries (mvcc coordinator). Mvcc
> coordinator is crucial component and it should perform as fast as possible.
> It seems we need introduce special 'dedicated mvcc coordinator' node role:
> it should not be possible to start cache on such node and it should not
> process user's compute jobs. At the same time it should be possible that
> any regular server node can become mvcc coordinator: this can be useful
> during development (no extra setup for mvcc will be needed), or support
> scenario when all dedicated coordinator nodes fail. So we need a way to
> make node a 'dedicated mvcc coordinator', we can add special flag on ignite
> configuration: IgniteConfiguration.isMvccCoordinator.
>

I agree that we need coordinator nodes, but I do not understand why can't
we reuse some cache nodes for it? Why do we need to ask user to start up
yet another type of node?

Re: MVCC configuration

[Dmitriy Setrakyan]

On Mon, Sep 18, 2017 at 4:57 AM, Vladimir Ozerov 
wrote:

> Alex,
>
> With putAll() on ATOMIC cache all bets are off, for sure.
>

Are we all in agreement that MVCC should only be enabled for transactional
caches then?

Re: MVCC configuration

[Alexey Kuznetsov]

Semyon,

How about to have node attribute "COORDINATOR_RANK" or "COORDINATOR_ORDER"?
This attribute can be 1, 2, 3
And node with minimal number will become coordinator.
If it failed, node with next rank/order will be elected as new coordinator.

Make sense?

On Mon, Sep 18, 2017 at 4:39 PM, Semyon Boikov  wrote:

> Hi all,
>
> Currently I'm working on MVCC feature (IGNITE-3478) and need your opinion
> on related configuration options.
>
> 1. MVCC will definitely bring some performance overhead, so I think it
> should not be enabled by default, I'm going to add special flag on cache
> configuration: CacheConfiguration.isMvccEnabled.
>
> 2. In current mvcc architecture there should be some node in cluster
> assigning versions for tx updates and queries (mvcc coordinator). Mvcc
> coordinator is crucial component and it should perform as fast as possible.
> It seems we need introduce special 'dedicated mvcc coordinator' node role:
> it should not be possible to start cache on such node and it should not
> process user's compute jobs. At the same time it should be possible that
> any regular server node can become mvcc coordinator: this can be useful
> during development (no extra setup for mvcc will be needed), or support
> scenario when all dedicated coordinator nodes fail. So we need a way to
> make node a 'dedicated mvcc coordinator', we can add special flag on ignite
> configuration: IgniteConfiguration.isMvccCoordinator.
>
> What do you think?
>
> Thanks
>



-- 
Alexey Kuznetsov

Re: MVCC configuration

[Semyon Boikov]

Vladimir, thanks for comments

> 2) I would also avoid this flag until we clearly understand it is needed.
> All numbers will be assigned from a single thread. For this reason even
> peak load on coordinator should not consume too much resources. I think we
> can assign coordinators automatically in first iteration.

For me need of dedicated coordinator nodes is clear: each mvcc
transaction/query will wait for mvcc coordinator response, if coordinator
will also process cache operations/compute jobs then any user code executed
on coordinator and consuming lot of CPU/heap will slowdown ALL mvcc
transactions/queries. As a user I want to make sure that coordinator node
will process only internal requests related to mvcc.

Also why do you think that all numbers should be assigned from single
thread?

Thanks

On Mon, Sep 18, 2017 at 2:59 PM, Semyon Boikov  wrote:

> Nikolay, thanks for comments
>
>
> > How will Ignite handle "mvcc coordinator" fail?
> > What will happen with if coordinator fails in the middle of a
> transaction?
> > Could tx be committed or rollbacked?
>
> I think coordinator failure will be handled in the same way as failure of
> one of transaction's 'primary' node: if coordinator fails during 'prepare'
> phase then tx is rolledback.
>
> >> Will we have some user notification if coordinator becomes slower?
>
> Now in Ignite we do not have common notion of 'user notification's, but we
> can add some metrics for coordinator performance on public API.
>
> Thanks
>
>
> On Mon, Sep 18, 2017 at 1:01 PM, Николай Ижиков 
> wrote:
>
>> Hello, Semyon!
>>
>> > It seems we need introduce special 'dedicated mvcc coordinator' node
>> role
>>
>> How will Ignite handle "mvcc coordinator" fail?
>>
>> What will happen with if coordinator fails in the middle of a transaction?
>> Could tx be committed or rollbacked?
>>
>> Will we have some user notification if coordinator becomes slower?
>>
>> > IgniteConfiguration.isMvccCoordinator
>>
>> flag name seems OK.
>>
>>
>> 2017-09-18 12:39 GMT+03:00 Semyon Boikov :
>>
>> > Hi all,
>> >
>> > Currently I'm working on MVCC feature (IGNITE-3478) and need your
>> opinion
>> > on related configuration options.
>> >
>> > 1. MVCC will definitely bring some performance overhead, so I think it
>> > should not be enabled by default, I'm going to add special flag on cache
>> > configuration: CacheConfiguration.isMvccEnabled.
>> >
>> > 2. In current mvcc architecture there should be some node in cluster
>> > assigning versions for tx updates and queries (mvcc coordinator). Mvcc
>> > coordinator is crucial component and it should perform as fast as
>> possible.
>> > It seems we need introduce special 'dedicated mvcc coordinator' node
>> role:
>> > it should not be possible to start cache on such node and it should not
>> > process user's compute jobs. At the same time it should be possible that
>> > any regular server node can become mvcc coordinator: this can be useful
>> > during development (no extra setup for mvcc will be needed), or support
>> > scenario when all dedicated coordinator nodes fail. So we need a way to
>> > make node a 'dedicated mvcc coordinator', we can add special flag on
>> ignite
>> > configuration: IgniteConfiguration.isMvccCoordinator.
>> >
>> > What do you think?
>> >
>> > Thanks
>> >
>>
>>
>>
>> --
>> Nikolay Izhikov
>> nizhikov@gmail.com
>>
>
>

Re: MVCC configuration

[Semyon Boikov]

Nikolay, thanks for comments


> How will Ignite handle "mvcc coordinator" fail?
> What will happen with if coordinator fails in the middle of a transaction?
> Could tx be committed or rollbacked?

I think coordinator failure will be handled in the same way as failure of
one of transaction's 'primary' node: if coordinator fails during 'prepare'
phase then tx is rolledback.

>> Will we have some user notification if coordinator becomes slower?

Now in Ignite we do not have common notion of 'user notification's, but we
can add some metrics for coordinator performance on public API.

Thanks


On Mon, Sep 18, 2017 at 1:01 PM, Николай Ижиков 
wrote:

> Hello, Semyon!
>
> > It seems we need introduce special 'dedicated mvcc coordinator' node role
>
> How will Ignite handle "mvcc coordinator" fail?
>
> What will happen with if coordinator fails in the middle of a transaction?
> Could tx be committed or rollbacked?
>
> Will we have some user notification if coordinator becomes slower?
>
> > IgniteConfiguration.isMvccCoordinator
>
> flag name seems OK.
>
>
> 2017-09-18 12:39 GMT+03:00 Semyon Boikov :
>
> > Hi all,
> >
> > Currently I'm working on MVCC feature (IGNITE-3478) and need your opinion
> > on related configuration options.
> >
> > 1. MVCC will definitely bring some performance overhead, so I think it
> > should not be enabled by default, I'm going to add special flag on cache
> > configuration: CacheConfiguration.isMvccEnabled.
> >
> > 2. In current mvcc architecture there should be some node in cluster
> > assigning versions for tx updates and queries (mvcc coordinator). Mvcc
> > coordinator is crucial component and it should perform as fast as
> possible.
> > It seems we need introduce special 'dedicated mvcc coordinator' node
> role:
> > it should not be possible to start cache on such node and it should not
> > process user's compute jobs. At the same time it should be possible that
> > any regular server node can become mvcc coordinator: this can be useful
> > during development (no extra setup for mvcc will be needed), or support
> > scenario when all dedicated coordinator nodes fail. So we need a way to
> > make node a 'dedicated mvcc coordinator', we can add special flag on
> ignite
> > configuration: IgniteConfiguration.isMvccCoordinator.
> >
> > What do you think?
> >
> > Thanks
> >
>
>
>
> --
> Nikolay Izhikov
> nizhikov@gmail.com
>

Re: MVCC configuration

[Vladimir Ozerov]

Alex,

With putAll() on ATOMIC cache all bets are off, for sure.

On Mon, Sep 18, 2017 at 2:53 PM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Vladimir,
>
> I doubt it will be possible to add any meaningful guarantees to ATOMIC
> caches with MVCC. Consider a case when a user does a putAll, not a single
> put. In this case, updates received by multiple primary nodes are not
> connected in any way. Moreover, whenever a primary node fails, the put for
> failed keys will be re-tried, which will lead to all sorts of overlapping
> updates in case of parallel putAll. It is hard to suggest how we should
> handle this, let alone explain this to a user.
>
> -- AG
>
> 2017-09-18 14:50 GMT+03:00 Vladimir Ozerov :
>
> > Yakov,
> >
> > I would say that my example is not about adding transactions to ATOMIC
> > cache, but rather about adding consistent snapshots to it.
> >
> > On Mon, Sep 18, 2017 at 1:59 PM, Yakov Zhdanov 
> > wrote:
> >
> > > Vladimir, I think we can ask user to switch to transactional cache to
> > > support your example. Otherwise, it seems we are turning atomic caches
> to
> > > tx implicitly.
> > >
> > > --Yakov
> > >
> > > 2017-09-18 13:49 GMT+03:00 Vladimir Ozerov :
> > >
> > > > Semen,
> > > >
> > > > Consider use case of some audit table where I log user actions over
> > time.
> > > > Every actions is a put to ATOMIC cache. User interacts with my
> > > application,
> > > > and performs the following set of actions:
> > > > 1. 08:00 MSK -> LOGIN
> > > > 2. 08:10 MSK -> Update something
> > > > 3. 08:20 MSK -> LOGUT
> > > >
> > > > If MVCC is there, whenever I query all actions performed by the
> user, I
> > > > would see either {}, {1}, {1, 2} or {1, 2, 3}
> > > > Without MVCC I can see weird things, such as {1, 3} or {2}, or
> > > whatsoever.
> > > >
> > > > Vladimir.
> > > >
> > > >
> > > > On Mon, Sep 18, 2017 at 1:41 PM, Semyon Boikov  >
> > > > wrote:
> > > >
> > > > > Guys,
> > > > >
> > > > > I do not really understand mvcc for atomic cache, could you please
> > > > provide
> > > > > some real use case.
> > > > >
> > > > > Thank you
> > > > >
> > > > > On Mon, Sep 18, 2017 at 1:37 PM, Yakov Zhdanov <
> yzhda...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Ouch... of course it makes sense for atomic caches. Seems I am
> not
> > > > fully
> > > > > > switched on after weekend =)
> > > > > >
> > > > > > Agree on other points.
> > > > > >
> > > > > > --Yakov
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: MVCC configuration

[Alexey Goncharuk]

Vladimir,

I doubt it will be possible to add any meaningful guarantees to ATOMIC
caches with MVCC. Consider a case when a user does a putAll, not a single
put. In this case, updates received by multiple primary nodes are not
connected in any way. Moreover, whenever a primary node fails, the put for
failed keys will be re-tried, which will lead to all sorts of overlapping
updates in case of parallel putAll. It is hard to suggest how we should
handle this, let alone explain this to a user.

-- AG

2017-09-18 14:50 GMT+03:00 Vladimir Ozerov :

> Yakov,
>
> I would say that my example is not about adding transactions to ATOMIC
> cache, but rather about adding consistent snapshots to it.
>
> On Mon, Sep 18, 2017 at 1:59 PM, Yakov Zhdanov 
> wrote:
>
> > Vladimir, I think we can ask user to switch to transactional cache to
> > support your example. Otherwise, it seems we are turning atomic caches to
> > tx implicitly.
> >
> > --Yakov
> >
> > 2017-09-18 13:49 GMT+03:00 Vladimir Ozerov :
> >
> > > Semen,
> > >
> > > Consider use case of some audit table where I log user actions over
> time.
> > > Every actions is a put to ATOMIC cache. User interacts with my
> > application,
> > > and performs the following set of actions:
> > > 1. 08:00 MSK -> LOGIN
> > > 2. 08:10 MSK -> Update something
> > > 3. 08:20 MSK -> LOGUT
> > >
> > > If MVCC is there, whenever I query all actions performed by the user, I
> > > would see either {}, {1}, {1, 2} or {1, 2, 3}
> > > Without MVCC I can see weird things, such as {1, 3} or {2}, or
> > whatsoever.
> > >
> > > Vladimir.
> > >
> > >
> > > On Mon, Sep 18, 2017 at 1:41 PM, Semyon Boikov 
> > > wrote:
> > >
> > > > Guys,
> > > >
> > > > I do not really understand mvcc for atomic cache, could you please
> > > provide
> > > > some real use case.
> > > >
> > > > Thank you
> > > >
> > > > On Mon, Sep 18, 2017 at 1:37 PM, Yakov Zhdanov 
> > > > wrote:
> > > >
> > > > > Ouch... of course it makes sense for atomic caches. Seems I am not
> > > fully
> > > > > switched on after weekend =)
> > > > >
> > > > > Agree on other points.
> > > > >
> > > > > --Yakov
> > > > >
> > > >
> > >
> >
>

Re: MVCC configuration

[Vladimir Ozerov]

Yakov,

I would say that my example is not about adding transactions to ATOMIC
cache, but rather about adding consistent snapshots to it.

On Mon, Sep 18, 2017 at 1:59 PM, Yakov Zhdanov  wrote:

> Vladimir, I think we can ask user to switch to transactional cache to
> support your example. Otherwise, it seems we are turning atomic caches to
> tx implicitly.
>
> --Yakov
>
> 2017-09-18 13:49 GMT+03:00 Vladimir Ozerov :
>
> > Semen,
> >
> > Consider use case of some audit table where I log user actions over time.
> > Every actions is a put to ATOMIC cache. User interacts with my
> application,
> > and performs the following set of actions:
> > 1. 08:00 MSK -> LOGIN
> > 2. 08:10 MSK -> Update something
> > 3. 08:20 MSK -> LOGUT
> >
> > If MVCC is there, whenever I query all actions performed by the user, I
> > would see either {}, {1}, {1, 2} or {1, 2, 3}
> > Without MVCC I can see weird things, such as {1, 3} or {2}, or
> whatsoever.
> >
> > Vladimir.
> >
> >
> > On Mon, Sep 18, 2017 at 1:41 PM, Semyon Boikov 
> > wrote:
> >
> > > Guys,
> > >
> > > I do not really understand mvcc for atomic cache, could you please
> > provide
> > > some real use case.
> > >
> > > Thank you
> > >
> > > On Mon, Sep 18, 2017 at 1:37 PM, Yakov Zhdanov 
> > > wrote:
> > >
> > > > Ouch... of course it makes sense for atomic caches. Seems I am not
> > fully
> > > > switched on after weekend =)
> > > >
> > > > Agree on other points.
> > > >
> > > > --Yakov
> > > >
> > >
> >
>

Re: MVCC configuration

[Yakov Zhdanov]

Vladimir, I think we can ask user to switch to transactional cache to
support your example. Otherwise, it seems we are turning atomic caches to
tx implicitly.

--Yakov

2017-09-18 13:49 GMT+03:00 Vladimir Ozerov :

> Semen,
>
> Consider use case of some audit table where I log user actions over time.
> Every actions is a put to ATOMIC cache. User interacts with my application,
> and performs the following set of actions:
> 1. 08:00 MSK -> LOGIN
> 2. 08:10 MSK -> Update something
> 3. 08:20 MSK -> LOGUT
>
> If MVCC is there, whenever I query all actions performed by the user, I
> would see either {}, {1}, {1, 2} or {1, 2, 3}
> Without MVCC I can see weird things, such as {1, 3} or {2}, or whatsoever.
>
> Vladimir.
>
>
> On Mon, Sep 18, 2017 at 1:41 PM, Semyon Boikov 
> wrote:
>
> > Guys,
> >
> > I do not really understand mvcc for atomic cache, could you please
> provide
> > some real use case.
> >
> > Thank you
> >
> > On Mon, Sep 18, 2017 at 1:37 PM, Yakov Zhdanov 
> > wrote:
> >
> > > Ouch... of course it makes sense for atomic caches. Seems I am not
> fully
> > > switched on after weekend =)
> > >
> > > Agree on other points.
> > >
> > > --Yakov
> > >
> >
>

Re: MVCC configuration

[Vladimir Ozerov]

Semen,

Consider use case of some audit table where I log user actions over time.
Every actions is a put to ATOMIC cache. User interacts with my application,
and performs the following set of actions:
1. 08:00 MSK -> LOGIN
2. 08:10 MSK -> Update something
3. 08:20 MSK -> LOGUT

If MVCC is there, whenever I query all actions performed by the user, I
would see either {}, {1}, {1, 2} or {1, 2, 3}
Without MVCC I can see weird things, such as {1, 3} or {2}, or whatsoever.

Vladimir.


On Mon, Sep 18, 2017 at 1:41 PM, Semyon Boikov  wrote:

> Guys,
>
> I do not really understand mvcc for atomic cache, could you please provide
> some real use case.
>
> Thank you
>
> On Mon, Sep 18, 2017 at 1:37 PM, Yakov Zhdanov 
> wrote:
>
> > Ouch... of course it makes sense for atomic caches. Seems I am not fully
> > switched on after weekend =)
> >
> > Agree on other points.
> >
> > --Yakov
> >
>

Re: MVCC configuration

[Semyon Boikov]

Guys,

I do not really understand mvcc for atomic cache, could you please provide
some real use case.

Thank you

On Mon, Sep 18, 2017 at 1:37 PM, Yakov Zhdanov  wrote:

> Ouch... of course it makes sense for atomic caches. Seems I am not fully
> switched on after weekend =)
>
> Agree on other points.
>
> --Yakov
>

Re: MVCC configuration

[Yakov Zhdanov]

Ouch... of course it makes sense for atomic caches. Seems I am not fully
switched on after weekend =)

Agree on other points.

--Yakov

Re: MVCC configuration

[Vladimir Ozerov]

Yakov,

MVCC for atomic caches makes sense as well - we will be able to read
consistent data set, which is not possible now. As I explained above,
per-cache configuration might not work when we start working on
transactional SQL design.

Moreover, it looks like an overkill for me at the moment. We will need
global flag anyway - this is convenient, as many application will require
all data to be MVCC-protected. So ideal solution would be to have
IgniteConfiguration.mvccEnabled
+ CacheConfiguration.mvccEnabled, but the latter could be skipped in the
first iteration.

On Mon, Sep 18, 2017 at 1:25 PM, Yakov Zhdanov  wrote:

> Vladimir, should it be on IgniteConfiguration or on CacheConfiguration? I
> think mvcc should be enabled on per cache basis and moreover it makes sense
> only for tx caches.
>
> --Yakov
>

Re: MVCC configuration

[Yakov Zhdanov]

Vladimir, should it be on IgniteConfiguration or on CacheConfiguration? I
think mvcc should be enabled on per cache basis and moreover it makes sense
only for tx caches.

--Yakov

Re: MVCC configuration

[Vladimir Ozerov]

Semen,

My comments:
1) I would propose to have only global flag for now -
IgniteConfiguration.isMvccEnabled.
One key design point we should keep in mind is that MVCC data *MSUT* be
persistent. We can skip it in the first iteration, as we are focused on
key-based cache updates, when typical transaction will update dozens or
hundreds keys. But when transactional SQL is ready, we will have to handle
cases when many thousands and millions rows are updated from concurrent
transactions. Without storing MVCC data on disk we will run out of memory.
Other vendors such as Oracle and Postgres, store MVCC-related data in data
blocks. So there is a risk that we will not be able to manage both MVCC and
non-MVCC caches in a single cache group or single memory policy. So
per-cache configuration looks too dangerous for me at the moment.

2) I would also avoid this flag until we clearly understand it is needed.
All numbers will be assigned from a single thread. For this reason even
peak load on coordinator should not consume too much resources. I think we
can assign coordinators automatically in first iteration.

So my vote is to have single global flag, nothing more -
IgniteConfiguration.isMvccEnabled.

Vladimir.

On Mon, Sep 18, 2017 at 12:39 PM, Semyon Boikov  wrote:

> Hi all,
>
> Currently I'm working on MVCC feature (IGNITE-3478) and need your opinion
> on related configuration options.
>
> 1. MVCC will definitely bring some performance overhead, so I think it
> should not be enabled by default, I'm going to add special flag on cache
> configuration: CacheConfiguration.isMvccEnabled.
>
> 2. In current mvcc architecture there should be some node in cluster
> assigning versions for tx updates and queries (mvcc coordinator). Mvcc
> coordinator is crucial component and it should perform as fast as possible.
> It seems we need introduce special 'dedicated mvcc coordinator' node role:
> it should not be possible to start cache on such node and it should not
> process user's compute jobs. At the same time it should be possible that
> any regular server node can become mvcc coordinator: this can be useful
> during development (no extra setup for mvcc will be needed), or support
> scenario when all dedicated coordinator nodes fail. So we need a way to
> make node a 'dedicated mvcc coordinator', we can add special flag on ignite
> configuration: IgniteConfiguration.isMvccCoordinator.
>
> What do you think?
>
> Thanks
>

Re: MVCC configuration

[Yakov Zhdanov]

1. Agree. Let's disable MVCC by default.
2. Sam, if user wants to have dedicated mvcc-coordinator, then we can use
configuration you suggested. However, I expect more properties will be
needed. How about having MvccConfiguration bean? Once topology has no
dedicated coordinators, topology should pick up some ordinary server (maybe
based on some stats about load and current partition distribution).

One more point - user should have an ability to assign coordinator
manually. I am pretty sure we can do it via custom discovery message.

--Yakov

Re: MVCC configuration

[Николай Ижиков]

Hello, Semyon!

> It seems we need introduce special 'dedicated mvcc coordinator' node role

How will Ignite handle "mvcc coordinator" fail?

What will happen with if coordinator fails in the middle of a transaction?
Could tx be committed or rollbacked?

Will we have some user notification if coordinator becomes slower?

> IgniteConfiguration.isMvccCoordinator

flag name seems OK.


2017-09-18 12:39 GMT+03:00 Semyon Boikov :

> Hi all,
>
> Currently I'm working on MVCC feature (IGNITE-3478) and need your opinion
> on related configuration options.
>
> 1. MVCC will definitely bring some performance overhead, so I think it
> should not be enabled by default, I'm going to add special flag on cache
> configuration: CacheConfiguration.isMvccEnabled.
>
> 2. In current mvcc architecture there should be some node in cluster
> assigning versions for tx updates and queries (mvcc coordinator). Mvcc
> coordinator is crucial component and it should perform as fast as possible.
> It seems we need introduce special 'dedicated mvcc coordinator' node role:
> it should not be possible to start cache on such node and it should not
> process user's compute jobs. At the same time it should be possible that
> any regular server node can become mvcc coordinator: this can be useful
> during development (no extra setup for mvcc will be needed), or support
> scenario when all dedicated coordinator nodes fail. So we need a way to
> make node a 'dedicated mvcc coordinator', we can add special flag on ignite
> configuration: IgniteConfiguration.isMvccCoordinator.
>
> What do you think?
>
> Thanks
>



-- 
Nikolay Izhikov
nizhikov@gmail.com

MVCC configuration

[Semyon Boikov]

Hi all,

Currently I'm working on MVCC feature (IGNITE-3478) and need your opinion
on related configuration options.

1. MVCC will definitely bring some performance overhead, so I think it
should not be enabled by default, I'm going to add special flag on cache
configuration: CacheConfiguration.isMvccEnabled.

2. In current mvcc architecture there should be some node in cluster
assigning versions for tx updates and queries (mvcc coordinator). Mvcc
coordinator is crucial component and it should perform as fast as possible.
It seems we need introduce special 'dedicated mvcc coordinator' node role:
it should not be possible to start cache on such node and it should not
process user's compute jobs. At the same time it should be possible that
any regular server node can become mvcc coordinator: this can be useful
during development (no extra setup for mvcc will be needed), or support
scenario when all dedicated coordinator nodes fail. So we need a way to
make node a 'dedicated mvcc coordinator', we can add special flag on ignite
configuration: IgniteConfiguration.isMvccCoordinator.

What do you think?

Thanks