Re: JCache time-based metrics

[Andrey Gura]

+1 for adding explanation to documentation.

On Thu, Sep 21, 2017 at 10:12 AM, Andrey Mashenkov
 wrote:
> Alex,
> I think we can just add an explanation of  meaningless of time-based metric
> for explicit transactions in documentation.
>
> 21 сент. 2017 г. 9:56 пользователь "Alexey Goncharuk" <
> alexey.goncha...@gmail.com> написал:
>
> Well, a transactional cache still implements the JCache spec, so we need to
> provide these metrics with some meaningful semantics. Agree that mixing
> implicit and explicit txs will mess up the metrics, but I doubt we can do
> anything about it.
>
> 2017-09-20 20:38 GMT+03:00 Andrey Gura :
>
>> Alexey,
>>
>> implicit transactions still can cause problems in metrics interpretation:
>>
>> - locks acquiring is still needed;
>> - metrics that taken in account for implicit transactions and missed
>> metrics for explicit transactions for the same cache still can
>> confuse.
>>
>> On Wed, Sep 20, 2017 at 6:02 PM, Alexey Goncharuk
>>  wrote:
>> > Agree with Andrey here. Probably, we can keep existing JCache metrics
> for
>> > implicit transactions, but we should design a whole new interface to
>> handle
>> > transaction metrics, including various transaction stages, such as lock
>> > acquisition, prepare and commit phases. In my opinion, this is the only
>> way
>> > to have unambiguous metrics in the product.
>> >
>> > 2017-09-20 17:58 GMT+03:00 Andrey Gura :
>> >
>> >> Slava and Igniters,
>> >>
>> >> Do you have any suggestions about described problem?
>> >>
>> >> From my point of view JCache API metrics specification has a number of
>> >> statements that can't be uniquely treated by Ignite user because of
>> >> implementations details. Most obvious problems here are related with
>> >> transactions that are distributed and have some properties that affect
>> >> their behaviour. Pessimistic transactions during entry modification
>> >> will enlist entry into transaction (read operation), acquire lock (it
>> >> can take some time if another transaction is already lock owner) and
>> >> eventually write modified entry into the cache. Where is modification
>> >> operation start? What if after cache.put() and before commit user
>> >> performs some time consuming operation? For optimistic transactions
>> >> still there is lock waiting time that affects metrics.
>> >>
>> >> So JCache API metrics are useless for transactions and could confuse
>> >> user. It seems that we shouldn't support JCache metrics at all in case
>> >> of transactional operations. May be we can offer another metrics that
>> >> could be more useful.
>> >>
>> >> Thoughts?
>> >>
>> >> On Tue, Sep 19, 2017 at 8:19 PM, Вячеслав Коптилин
>> >>  wrote:
>> >> > I'd want to make a new attempt to discuss JCache metrics, especially
>> >> > time-based metrics.
>> >> > As discussed earlier (
>> >> > http://apache-ignite-developers.2346864.n4.nabble.
>> >> com/Cache-Metrics-tt19699.html
>> >> > ),
>> >> > there are two approaches (and at first glance, the second one is
>> >> > preferable):
>> >> > #1 Node that starts some operation is responsible for updating
> the
>> >> > cache metrics.
>> >> > #2 Primary node (node that actually executes a request)
>> >> >
>> >> > I have one question/concern about time-based metrics for
> transactional
>> >> > caches.
>> >> > The JCache specification does not have definition like a transaction,
>> >> > partitioned cache etc,
>> >> > and, therefore, cannot provide convenience and clear understanding
>> about
>> >> > the average time for that case.
>> >> >
>> >> > Let's assume, for example, we have the following code:
>> >> >
>> >> > try (Transaction tx =
>> >> > transactions.txStart(TransactionConcurrency.OPTIMISTIC,
>> >> > TransactionIsolation.READ_COMMITTED)) {
>> >> > value = cache.get(key);
>> >> > // calculate new value for the given key
>> >> > ...
>> >> > cache.put(key, new_value); // (1)
>> >> >
>> >> > // some additional operations
>> >> >
>> >> > // another update
>> >> > value = cache.get(key);// (2)
>> >> > ...
>> >> > cache.put(key, new_value);
>> >> >
>> >> > tx.commit();   // (3)
>> >> > }
>> >> >
>> >> > What is the average time for write operation? Is it a time needed for
>> >> > enlisting an entry in the transaction scope, acquiring locks,
>> committing
>> >> > changes?
>> >> > For that particular case, current implementation accumulates both
>> timings
>> >> > (1)&(2) on the local node during performing put operation, but
> 'write'
>> >> > counter is updated only once on data node during commit phase.
>> >> > I think this behavior is not obvious at least.
>> >> >
>> >> > Thanks,
>> >> > Slava.
>> >>
>>

Re: JCache time-based metrics

[Andrey Mashenkov]

Alex,
I think we can just add an explanation of  meaningless of time-based metric
for explicit transactions in documentation.

21 сент. 2017 г. 9:56 пользователь "Alexey Goncharuk" <
alexey.goncha...@gmail.com> написал:

Well, a transactional cache still implements the JCache spec, so we need to
provide these metrics with some meaningful semantics. Agree that mixing
implicit and explicit txs will mess up the metrics, but I doubt we can do
anything about it.

2017-09-20 20:38 GMT+03:00 Andrey Gura :

> Alexey,
>
> implicit transactions still can cause problems in metrics interpretation:
>
> - locks acquiring is still needed;
> - metrics that taken in account for implicit transactions and missed
> metrics for explicit transactions for the same cache still can
> confuse.
>
> On Wed, Sep 20, 2017 at 6:02 PM, Alexey Goncharuk
>  wrote:
> > Agree with Andrey here. Probably, we can keep existing JCache metrics
for
> > implicit transactions, but we should design a whole new interface to
> handle
> > transaction metrics, including various transaction stages, such as lock
> > acquisition, prepare and commit phases. In my opinion, this is the only
> way
> > to have unambiguous metrics in the product.
> >
> > 2017-09-20 17:58 GMT+03:00 Andrey Gura :
> >
> >> Slava and Igniters,
> >>
> >> Do you have any suggestions about described problem?
> >>
> >> From my point of view JCache API metrics specification has a number of
> >> statements that can't be uniquely treated by Ignite user because of
> >> implementations details. Most obvious problems here are related with
> >> transactions that are distributed and have some properties that affect
> >> their behaviour. Pessimistic transactions during entry modification
> >> will enlist entry into transaction (read operation), acquire lock (it
> >> can take some time if another transaction is already lock owner) and
> >> eventually write modified entry into the cache. Where is modification
> >> operation start? What if after cache.put() and before commit user
> >> performs some time consuming operation? For optimistic transactions
> >> still there is lock waiting time that affects metrics.
> >>
> >> So JCache API metrics are useless for transactions and could confuse
> >> user. It seems that we shouldn't support JCache metrics at all in case
> >> of transactional operations. May be we can offer another metrics that
> >> could be more useful.
> >>
> >> Thoughts?
> >>
> >> On Tue, Sep 19, 2017 at 8:19 PM, Вячеслав Коптилин
> >>  wrote:
> >> > I'd want to make a new attempt to discuss JCache metrics, especially
> >> > time-based metrics.
> >> > As discussed earlier (
> >> > http://apache-ignite-developers.2346864.n4.nabble.
> >> com/Cache-Metrics-tt19699.html
> >> > ),
> >> > there are two approaches (and at first glance, the second one is
> >> > preferable):
> >> > #1 Node that starts some operation is responsible for updating
the
> >> > cache metrics.
> >> > #2 Primary node (node that actually executes a request)
> >> >
> >> > I have one question/concern about time-based metrics for
transactional
> >> > caches.
> >> > The JCache specification does not have definition like a transaction,
> >> > partitioned cache etc,
> >> > and, therefore, cannot provide convenience and clear understanding
> about
> >> > the average time for that case.
> >> >
> >> > Let's assume, for example, we have the following code:
> >> >
> >> > try (Transaction tx =
> >> > transactions.txStart(TransactionConcurrency.OPTIMISTIC,
> >> > TransactionIsolation.READ_COMMITTED)) {
> >> > value = cache.get(key);
> >> > // calculate new value for the given key
> >> > ...
> >> > cache.put(key, new_value); // (1)
> >> >
> >> > // some additional operations
> >> >
> >> > // another update
> >> > value = cache.get(key);// (2)
> >> > ...
> >> > cache.put(key, new_value);
> >> >
> >> > tx.commit();   // (3)
> >> > }
> >> >
> >> > What is the average time for write operation? Is it a time needed for
> >> > enlisting an entry in the transaction scope, acquiring locks,
> committing
> >> > changes?
> >> > For that particular case, current implementation accumulates both
> timings
> >> > (1)&(2) on the local node during performing put operation, but
'write'
> >> > counter is updated only once on data node during commit phase.
> >> > I think this behavior is not obvious at least.
> >> >
> >> > Thanks,
> >> > Slava.
> >>
>

Re: JCache time-based metrics

[Alexey Goncharuk]

Well, a transactional cache still implements the JCache spec, so we need to
provide these metrics with some meaningful semantics. Agree that mixing
implicit and explicit txs will mess up the metrics, but I doubt we can do
anything about it.

2017-09-20 20:38 GMT+03:00 Andrey Gura :

> Alexey,
>
> implicit transactions still can cause problems in metrics interpretation:
>
> - locks acquiring is still needed;
> - metrics that taken in account for implicit transactions and missed
> metrics for explicit transactions for the same cache still can
> confuse.
>
> On Wed, Sep 20, 2017 at 6:02 PM, Alexey Goncharuk
>  wrote:
> > Agree with Andrey here. Probably, we can keep existing JCache metrics for
> > implicit transactions, but we should design a whole new interface to
> handle
> > transaction metrics, including various transaction stages, such as lock
> > acquisition, prepare and commit phases. In my opinion, this is the only
> way
> > to have unambiguous metrics in the product.
> >
> > 2017-09-20 17:58 GMT+03:00 Andrey Gura :
> >
> >> Slava and Igniters,
> >>
> >> Do you have any suggestions about described problem?
> >>
> >> From my point of view JCache API metrics specification has a number of
> >> statements that can't be uniquely treated by Ignite user because of
> >> implementations details. Most obvious problems here are related with
> >> transactions that are distributed and have some properties that affect
> >> their behaviour. Pessimistic transactions during entry modification
> >> will enlist entry into transaction (read operation), acquire lock (it
> >> can take some time if another transaction is already lock owner) and
> >> eventually write modified entry into the cache. Where is modification
> >> operation start? What if after cache.put() and before commit user
> >> performs some time consuming operation? For optimistic transactions
> >> still there is lock waiting time that affects metrics.
> >>
> >> So JCache API metrics are useless for transactions and could confuse
> >> user. It seems that we shouldn't support JCache metrics at all in case
> >> of transactional operations. May be we can offer another metrics that
> >> could be more useful.
> >>
> >> Thoughts?
> >>
> >> On Tue, Sep 19, 2017 at 8:19 PM, Вячеслав Коптилин
> >>  wrote:
> >> > I'd want to make a new attempt to discuss JCache metrics, especially
> >> > time-based metrics.
> >> > As discussed earlier (
> >> > http://apache-ignite-developers.2346864.n4.nabble.
> >> com/Cache-Metrics-tt19699.html
> >> > ),
> >> > there are two approaches (and at first glance, the second one is
> >> > preferable):
> >> > #1 Node that starts some operation is responsible for updating the
> >> > cache metrics.
> >> > #2 Primary node (node that actually executes a request)
> >> >
> >> > I have one question/concern about time-based metrics for transactional
> >> > caches.
> >> > The JCache specification does not have definition like a transaction,
> >> > partitioned cache etc,
> >> > and, therefore, cannot provide convenience and clear understanding
> about
> >> > the average time for that case.
> >> >
> >> > Let's assume, for example, we have the following code:
> >> >
> >> > try (Transaction tx =
> >> > transactions.txStart(TransactionConcurrency.OPTIMISTIC,
> >> > TransactionIsolation.READ_COMMITTED)) {
> >> > value = cache.get(key);
> >> > // calculate new value for the given key
> >> > ...
> >> > cache.put(key, new_value); // (1)
> >> >
> >> > // some additional operations
> >> >
> >> > // another update
> >> > value = cache.get(key);// (2)
> >> > ...
> >> > cache.put(key, new_value);
> >> >
> >> > tx.commit();   // (3)
> >> > }
> >> >
> >> > What is the average time for write operation? Is it a time needed for
> >> > enlisting an entry in the transaction scope, acquiring locks,
> committing
> >> > changes?
> >> > For that particular case, current implementation accumulates both
> timings
> >> > (1)&(2) on the local node during performing put operation, but 'write'
> >> > counter is updated only once on data node during commit phase.
> >> > I think this behavior is not obvious at least.
> >> >
> >> > Thanks,
> >> > Slava.
> >>
>

Re: JCache time-based metrics

[Andrey Gura]

Alexey,

implicit transactions still can cause problems in metrics interpretation:

- locks acquiring is still needed;
- metrics that taken in account for implicit transactions and missed
metrics for explicit transactions for the same cache still can
confuse.

On Wed, Sep 20, 2017 at 6:02 PM, Alexey Goncharuk
 wrote:
> Agree with Andrey here. Probably, we can keep existing JCache metrics for
> implicit transactions, but we should design a whole new interface to handle
> transaction metrics, including various transaction stages, such as lock
> acquisition, prepare and commit phases. In my opinion, this is the only way
> to have unambiguous metrics in the product.
>
> 2017-09-20 17:58 GMT+03:00 Andrey Gura :
>
>> Slava and Igniters,
>>
>> Do you have any suggestions about described problem?
>>
>> From my point of view JCache API metrics specification has a number of
>> statements that can't be uniquely treated by Ignite user because of
>> implementations details. Most obvious problems here are related with
>> transactions that are distributed and have some properties that affect
>> their behaviour. Pessimistic transactions during entry modification
>> will enlist entry into transaction (read operation), acquire lock (it
>> can take some time if another transaction is already lock owner) and
>> eventually write modified entry into the cache. Where is modification
>> operation start? What if after cache.put() and before commit user
>> performs some time consuming operation? For optimistic transactions
>> still there is lock waiting time that affects metrics.
>>
>> So JCache API metrics are useless for transactions and could confuse
>> user. It seems that we shouldn't support JCache metrics at all in case
>> of transactional operations. May be we can offer another metrics that
>> could be more useful.
>>
>> Thoughts?
>>
>> On Tue, Sep 19, 2017 at 8:19 PM, Вячеслав Коптилин
>>  wrote:
>> > I'd want to make a new attempt to discuss JCache metrics, especially
>> > time-based metrics.
>> > As discussed earlier (
>> > http://apache-ignite-developers.2346864.n4.nabble.
>> com/Cache-Metrics-tt19699.html
>> > ),
>> > there are two approaches (and at first glance, the second one is
>> > preferable):
>> > #1 Node that starts some operation is responsible for updating the
>> > cache metrics.
>> > #2 Primary node (node that actually executes a request)
>> >
>> > I have one question/concern about time-based metrics for transactional
>> > caches.
>> > The JCache specification does not have definition like a transaction,
>> > partitioned cache etc,
>> > and, therefore, cannot provide convenience and clear understanding about
>> > the average time for that case.
>> >
>> > Let's assume, for example, we have the following code:
>> >
>> > try (Transaction tx =
>> > transactions.txStart(TransactionConcurrency.OPTIMISTIC,
>> > TransactionIsolation.READ_COMMITTED)) {
>> > value = cache.get(key);
>> > // calculate new value for the given key
>> > ...
>> > cache.put(key, new_value); // (1)
>> >
>> > // some additional operations
>> >
>> > // another update
>> > value = cache.get(key);// (2)
>> > ...
>> > cache.put(key, new_value);
>> >
>> > tx.commit();   // (3)
>> > }
>> >
>> > What is the average time for write operation? Is it a time needed for
>> > enlisting an entry in the transaction scope, acquiring locks, committing
>> > changes?
>> > For that particular case, current implementation accumulates both timings
>> > (1)&(2) on the local node during performing put operation, but 'write'
>> > counter is updated only once on data node during commit phase.
>> > I think this behavior is not obvious at least.
>> >
>> > Thanks,
>> > Slava.
>>

Re: JCache time-based metrics

[Alexey Goncharuk]

Agree with Andrey here. Probably, we can keep existing JCache metrics for
implicit transactions, but we should design a whole new interface to handle
transaction metrics, including various transaction stages, such as lock
acquisition, prepare and commit phases. In my opinion, this is the only way
to have unambiguous metrics in the product.

2017-09-20 17:58 GMT+03:00 Andrey Gura :

> Slava and Igniters,
>
> Do you have any suggestions about described problem?
>
> From my point of view JCache API metrics specification has a number of
> statements that can't be uniquely treated by Ignite user because of
> implementations details. Most obvious problems here are related with
> transactions that are distributed and have some properties that affect
> their behaviour. Pessimistic transactions during entry modification
> will enlist entry into transaction (read operation), acquire lock (it
> can take some time if another transaction is already lock owner) and
> eventually write modified entry into the cache. Where is modification
> operation start? What if after cache.put() and before commit user
> performs some time consuming operation? For optimistic transactions
> still there is lock waiting time that affects metrics.
>
> So JCache API metrics are useless for transactions and could confuse
> user. It seems that we shouldn't support JCache metrics at all in case
> of transactional operations. May be we can offer another metrics that
> could be more useful.
>
> Thoughts?
>
> On Tue, Sep 19, 2017 at 8:19 PM, Вячеслав Коптилин
>  wrote:
> > I'd want to make a new attempt to discuss JCache metrics, especially
> > time-based metrics.
> > As discussed earlier (
> > http://apache-ignite-developers.2346864.n4.nabble.
> com/Cache-Metrics-tt19699.html
> > ),
> > there are two approaches (and at first glance, the second one is
> > preferable):
> > #1 Node that starts some operation is responsible for updating the
> > cache metrics.
> > #2 Primary node (node that actually executes a request)
> >
> > I have one question/concern about time-based metrics for transactional
> > caches.
> > The JCache specification does not have definition like a transaction,
> > partitioned cache etc,
> > and, therefore, cannot provide convenience and clear understanding about
> > the average time for that case.
> >
> > Let's assume, for example, we have the following code:
> >
> > try (Transaction tx =
> > transactions.txStart(TransactionConcurrency.OPTIMISTIC,
> > TransactionIsolation.READ_COMMITTED)) {
> > value = cache.get(key);
> > // calculate new value for the given key
> > ...
> > cache.put(key, new_value); // (1)
> >
> > // some additional operations
> >
> > // another update
> > value = cache.get(key);// (2)
> > ...
> > cache.put(key, new_value);
> >
> > tx.commit();   // (3)
> > }
> >
> > What is the average time for write operation? Is it a time needed for
> > enlisting an entry in the transaction scope, acquiring locks, committing
> > changes?
> > For that particular case, current implementation accumulates both timings
> > (1)&(2) on the local node during performing put operation, but 'write'
> > counter is updated only once on data node during commit phase.
> > I think this behavior is not obvious at least.
> >
> > Thanks,
> > Slava.
>

Re: JCache time-based metrics

[Andrey Gura]

Slava and Igniters,

Do you have any suggestions about described problem?

>From my point of view JCache API metrics specification has a number of
statements that can't be uniquely treated by Ignite user because of
implementations details. Most obvious problems here are related with
transactions that are distributed and have some properties that affect
their behaviour. Pessimistic transactions during entry modification
will enlist entry into transaction (read operation), acquire lock (it
can take some time if another transaction is already lock owner) and
eventually write modified entry into the cache. Where is modification
operation start? What if after cache.put() and before commit user
performs some time consuming operation? For optimistic transactions
still there is lock waiting time that affects metrics.

So JCache API metrics are useless for transactions and could confuse
user. It seems that we shouldn't support JCache metrics at all in case
of transactional operations. May be we can offer another metrics that
could be more useful.

Thoughts?

On Tue, Sep 19, 2017 at 8:19 PM, Вячеслав Коптилин
 wrote:
> I'd want to make a new attempt to discuss JCache metrics, especially
> time-based metrics.
> As discussed earlier (
> http://apache-ignite-developers.2346864.n4.nabble.com/Cache-Metrics-tt19699.html
> ),
> there are two approaches (and at first glance, the second one is
> preferable):
> #1 Node that starts some operation is responsible for updating the
> cache metrics.
> #2 Primary node (node that actually executes a request)
>
> I have one question/concern about time-based metrics for transactional
> caches.
> The JCache specification does not have definition like a transaction,
> partitioned cache etc,
> and, therefore, cannot provide convenience and clear understanding about
> the average time for that case.
>
> Let's assume, for example, we have the following code:
>
> try (Transaction tx =
> transactions.txStart(TransactionConcurrency.OPTIMISTIC,
> TransactionIsolation.READ_COMMITTED)) {
> value = cache.get(key);
> // calculate new value for the given key
> ...
> cache.put(key, new_value); // (1)
>
> // some additional operations
>
> // another update
> value = cache.get(key);// (2)
> ...
> cache.put(key, new_value);
>
> tx.commit();   // (3)
> }
>
> What is the average time for write operation? Is it a time needed for
> enlisting an entry in the transaction scope, acquiring locks, committing
> changes?
> For that particular case, current implementation accumulates both timings
> (1)&(2) on the local node during performing put operation, but 'write'
> counter is updated only once on data node during commit phase.
> I think this behavior is not obvious at least.
>
> Thanks,
> Slava.

JCache time-based metrics

[Вячеслав Коптилин]

I'd want to make a new attempt to discuss JCache metrics, especially
time-based metrics.
As discussed earlier (
http://apache-ignite-developers.2346864.n4.nabble.com/Cache-Metrics-tt19699.html
),
there are two approaches (and at first glance, the second one is
preferable):
#1 Node that starts some operation is responsible for updating the
cache metrics.
#2 Primary node (node that actually executes a request)

I have one question/concern about time-based metrics for transactional
caches.
The JCache specification does not have definition like a transaction,
partitioned cache etc,
and, therefore, cannot provide convenience and clear understanding about
the average time for that case.

Let's assume, for example, we have the following code:

try (Transaction tx =
transactions.txStart(TransactionConcurrency.OPTIMISTIC,
TransactionIsolation.READ_COMMITTED)) {
value = cache.get(key);
// calculate new value for the given key
...
cache.put(key, new_value); // (1)

// some additional operations

// another update
value = cache.get(key);// (2)
...
cache.put(key, new_value);

tx.commit();   // (3)
}

What is the average time for write operation? Is it a time needed for
enlisting an entry in the transaction scope, acquiring locks, committing
changes?
For that particular case, current implementation accumulates both timings
(1)&(2) on the local node during performing put operation, but 'write'
counter is updated only once on data node during commit phase.
I think this behavior is not obvious at least.

Thanks,
Slava.