Client Cache metrics API design discussion.

[Vladimir Ershov]

Igniters!

Here is a handsome moment in our current cache metrics API, that begs for
an improvement and due to it significancy assumed to be discussed
communitywise. Current CacheMetrics interface is confusing for a case, when
it is accessed from a client node.
One of the typical question is:
*what CacheMetrics#getSize should return on a client Node for a non-Near
non-Local cache?*
Here are some options:

   1. Zero. As it works now, it is just 0, since there is no entries on the
   client node.
   2. Amount of all entries for this cache across the cluster.
   3. Or, and here comes an interesting part, - amount of values which were
   fore.x. created through this client node, as it is useful for
   #getAveragePutTime.
   4.  Your variant?

The same for the rest of the API: getCacheHits (0, cluster, client),
getTxDhtCommitQueueSize (0, cluster, for client keys, UnsOpEx?).

Assuming this point can give a good start for our discussion: there are
use-cases, that demands metrics to be gathered for a client node
separately, fore.x. user can measure latency between nodes, by comparing
#getAveragePutTime on client and server side. Thus I consider reasonable to
implement specific ClientCacheMetricsImpl with logic for client, but actual
questions are: what should methods like getSize, getHits return? Is it
necessary to support backward compatibility here for metrics API? Does the
community think that it is reasonable to put our efforts to this task and
that we want to support case for cache metrics on a client node?

Thoughts?

[jira] [Created] (IGNITE-2567) Grouping

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-2567:
--

 Summary: Grouping
 Key: IGNITE-2567
 URL: https://issues.apache.org/jira/browse/IGNITE-2567
 Project: Ignite
  Issue Type: Sub-task
Reporter: Pavel Tupitsyn






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Client Cache metrics API design discussion.

[Valentin Kulichenko]

Vladimir,

As I already suggested in the ticket [1], I think that by default we should
return metrics for the whole cluster. Now we collect them only from local
node, which is confusing, especially on the client. If one needs metrics
for one node or from subset of nodes, metrics(ClusterGroup) method can be
used.

So as for the size, I'm definitely for option 2.

Option 3 is more about 'getCachePuts()', but not 'getSIze()', no? Where do
we increment this counter - on the client or on the primary node? If on the
client, this metric will work just as you described when you get metrics
for a particular client using metrics(ClusterGroup).

Probably it also would be useful to add localMetrics() shortcut method.

[1] https://issues.apache.org/jira/browse/IGNITE-2483

-Val

On Fri, Feb 5, 2016 at 8:44 AM, Vladimir Ershov 
wrote:

> Igniters!
>
> Here is a handsome moment in our current cache metrics API, that begs for
> an improvement and due to it significancy assumed to be discussed
> communitywise. Current CacheMetrics interface is confusing for a case, when
> it is accessed from a client node.
> One of the typical question is:
> *what CacheMetrics#getSize should return on a client Node for a non-Near
> non-Local cache?*
> Here are some options:
>
>1. Zero. As it works now, it is just 0, since there is no entries on the
>client node.
>2. Amount of all entries for this cache across the cluster.
>3. Or, and here comes an interesting part, - amount of values which were
>fore.x. created through this client node, as it is useful for
>#getAveragePutTime.
>4.  Your variant?
>
> The same for the rest of the API: getCacheHits (0, cluster, client),
> getTxDhtCommitQueueSize (0, cluster, for client keys, UnsOpEx?).
>
> Assuming this point can give a good start for our discussion: there are
> use-cases, that demands metrics to be gathered for a client node
> separately, fore.x. user can measure latency between nodes, by comparing
> #getAveragePutTime on client and server side. Thus I consider reasonable to
> implement specific ClientCacheMetricsImpl with logic for client, but actual
> questions are: what should methods like getSize, getHits return? Is it
> necessary to support backward compatibility here for metrics API? Does the
> community think that it is reasonable to put our efforts to this task and
> that we want to support case for cache metrics on a client node?
>
> Thoughts?
>

Re: Proxy serialization issue

[Valentin Kulichenko]

Alex,

I don't know in advance how many bytes I will write because some classes
can be successfully registered, but some not. Therefore I can't extend by
more than 5 bytes at this point.

Makes sense?

-Val

On Fri, Feb 5, 2016 at 4:02 AM, Alexey Goncharuk  wrote:

> Val,
>
> The code looks good to me. The only place that made me wonder was
> out.unsafeEnsure(1 + 4) call which extends the stream by 5 bytes, however
> we can write significantly more bytes. I see that we use the same approach
> in other places, so I was wondering if this is a required call or a
> performance hint?
>

[jira] [Created] (IGNITE-2569) Ignite GridComponent used incorrectly during IgniteStart. Could cause NPE

[Vladimir Ershov (JIRA)]

Vladimir Ershov created IGNITE-2569:
---

 Summary: Ignite GridComponent used incorrectly during IgniteStart. 
Could cause NPE
 Key: IGNITE-2569
 URL: https://issues.apache.org/jira/browse/IGNITE-2569
 Project: Ignite
  Issue Type: Task
  Components: general
Affects Versions: 1.5.0.final
Reporter: Vladimir Ershov
Priority: Minor


IgniteKernal could stop GridComponents during start without calling stopKernal 
before it. Showcase: 
{noformat}
19:09:58,301][ERROR][async-runner-1][IgniteClientReconnectCacheTest3] Failed to 
pre-stop processor: GridProcessorAdapter []
java.lang.AssertionError
 at 
org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.onKernalStop(GridCacheSharedManagerAdapter.java:109)
 at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:884)
 at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:1814)
 at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:1760)
 at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:932)
 at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1687)
 at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1546)
 at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1002)
 at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:533)
 at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:514)
 at org.apache.ignite.Ignition.start(Ignition.java:322)
 at 
org.apache.ignite.internal.IgniteClientReconnectCacheTest$8.call(IgniteClientReconnectCacheTest.java:700)
 at 
org.apache.ignite.internal.IgniteClientReconnectCacheTest$8.call(IgniteClientReconnectCacheTest.java:697)
 at org.apache.ignite.testframework.GridTestUtils$6.run(GridTestUtils.java:758)
 at 
org.apache.ignite.testframework.GridTestUtils$8.call(GridTestUtils.java:1054)
 at org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:86)
{noformat}

This can cause different NPE.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: API doc for .NET

[Valentin Kulichenko]

I uploaded the docs and added links under 'Documentation' menu on the
website.

https://ignite.apache.org/releases/1.5.0.final/dotnetdoc/
https://ignite.apache.org/releases/1.5.0.final/cppdoc/

-Val

On Thu, Feb 4, 2016 at 10:04 PM, Vladimir Ozerov 
wrote:

> We have doxygen plans which generate docs. Though, it appears that these
> docs are not published on the website.
>
> Who can assist us with this?
>
> On Fri, Feb 5, 2016 at 1:57 AM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Igniters,
> >
> > Do we have API doc for .NET? If not, I think we should create it and put
> on
> > the website.
> >
> > -Val
> >
>

Re: API doc for .NET

[Dmitriy Setrakyan]

Great! Now we have to work on adding C++ documentation in readme.

D.

On Fri, Feb 5, 2016 at 12:40 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> I uploaded the docs and added links under 'Documentation' menu on the
> website.
>
> https://ignite.apache.org/releases/1.5.0.final/dotnetdoc/
> https://ignite.apache.org/releases/1.5.0.final/cppdoc/
>
> -Val
>
> On Thu, Feb 4, 2016 at 10:04 PM, Vladimir Ozerov 
> wrote:
>
> > We have doxygen plans which generate docs. Though, it appears that these
> > docs are not published on the website.
> >
> > Who can assist us with this?
> >
> > On Fri, Feb 5, 2016 at 1:57 AM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Igniters,
> > >
> > > Do we have API doc for .NET? If not, I think we should create it and
> put
> > on
> > > the website.
> > >
> > > -Val
> > >
> >
>

Re: Client Cache metrics API design discussion.

[Dmitriy Setrakyan]

Agree. All metrics should return the data for the whole cache, unless
specifically specified otherwise by user.

D.

On Fri, Feb 5, 2016 at 10:56 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Vladimir,
>
> As I already suggested in the ticket [1], I think that by default we should
> return metrics for the whole cluster. Now we collect them only from local
> node, which is confusing, especially on the client. If one needs metrics
> for one node or from subset of nodes, metrics(ClusterGroup) method can be
> used.
>
> So as for the size, I'm definitely for option 2.
>
> Option 3 is more about 'getCachePuts()', but not 'getSIze()', no? Where do
> we increment this counter - on the client or on the primary node? If on the
> client, this metric will work just as you described when you get metrics
> for a particular client using metrics(ClusterGroup).
>
> Probably it also would be useful to add localMetrics() shortcut method.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-2483
>
> -Val
>
> On Fri, Feb 5, 2016 at 8:44 AM, Vladimir Ershov 
> wrote:
>
> > Igniters!
> >
> > Here is a handsome moment in our current cache metrics API, that begs for
> > an improvement and due to it significancy assumed to be discussed
> > communitywise. Current CacheMetrics interface is confusing for a case,
> when
> > it is accessed from a client node.
> > One of the typical question is:
> > *what CacheMetrics#getSize should return on a client Node for a non-Near
> > non-Local cache?*
> > Here are some options:
> >
> >1. Zero. As it works now, it is just 0, since there is no entries on
> the
> >client node.
> >2. Amount of all entries for this cache across the cluster.
> >3. Or, and here comes an interesting part, - amount of values which
> were
> >fore.x. created through this client node, as it is useful for
> >#getAveragePutTime.
> >4.  Your variant?
> >
> > The same for the rest of the API: getCacheHits (0, cluster, client),
> > getTxDhtCommitQueueSize (0, cluster, for client keys, UnsOpEx?).
> >
> > Assuming this point can give a good start for our discussion: there are
> > use-cases, that demands metrics to be gathered for a client node
> > separately, fore.x. user can measure latency between nodes, by comparing
> > #getAveragePutTime on client and server side. Thus I consider reasonable
> to
> > implement specific ClientCacheMetricsImpl with logic for client, but
> actual
> > questions are: what should methods like getSize, getHits return? Is it
> > necessary to support backward compatibility here for metrics API? Does
> the
> > community think that it is reasonable to put our efforts to this task and
> > that we want to support case for cache metrics on a client node?
> >
> > Thoughts?
> >
>

[jira] [Created] (IGNITE-2562) Affix is blinking in some cases

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-2562:


 Summary: Affix is blinking in some cases
 Key: IGNITE-2562
 URL: https://issues.apache.org/jira/browse/IGNITE-2562
 Project: Ignite
  Issue Type: Sub-task
Affects Versions: 1.5.0.final
Reporter: Alexey Kuznetsov
Assignee: Dmitriyff
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Proxy serialization issue

[Alexey Goncharuk]

Val,

The code looks good to me. The only place that made me wonder was
out.unsafeEnsure(1 + 4) call which extends the stream by 5 bytes, however
we can write significantly more bytes. I see that we use the same approach
in other places, so I was wondering if this is a required call or a
performance hint?

[jira] [Created] (IGNITE-2564) CPP: Memory reallocation callback doesn't work.

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-2564:
---

 Summary: CPP: Memory reallocation callback doesn't work.
 Key: IGNITE-2564
 URL: https://issues.apache.org/jira/browse/IGNITE-2564
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 1.5.0.final
Reporter: Vladimir Ozerov
Assignee: Igor Sapego
Priority: Critical
 Fix For: 1.6


*Steps to reproduce*:
1) Start a CPP node.
2) Put large string to cache (say, >1024 chars long)
3) Now try to read this string using Cache.Get() method. 
PlatformNoCallbackException exception is thrown.

*Root cause*: 
External memory reallocation callback is not set.

*Solution*:
Add memory reallocation callback to ignite_environment.cpp



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-2565) TCK suite fail periodically

[Anton Vinogradov (JIRA)]

Anton Vinogradov created IGNITE-2565:


 Summary: TCK suite fail periodically
 Key: IGNITE-2565
 URL: https://issues.apache.org/jira/browse/IGNITE-2565
 Project: Ignite
  Issue Type: Bug
Affects Versions: 1.5.0.final
Reporter: Anton Vinogradov


TCK suite fail periodically fails with assetrions about CacheHitPercentage.
I see that a lot of changes was made last two months where added logic with 
invoking of metrics.onRead(); possible this affects statictic counting.

Typical stacktrace:

java.lang.AssertionError: expected:<100.0> but was:<50.0>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at 
org.jsr107.tck.management.CacheMBStatisticsBeanTest.testCacheStatisticsInvokeEntryProcessorRemove(CacheMBStatisticsBeanTest.java:424)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-2570) LoadCacheClosure and several other classes in GridCacheAdapter are not marked as internal

[Valentin Kulichenko (JIRA)]

Valentin Kulichenko created IGNITE-2570:
---

 Summary: LoadCacheClosure and several other classes in 
GridCacheAdapter are not marked as internal
 Key: IGNITE-2570
 URL: https://issues.apache.org/jira/browse/IGNITE-2570
 Project: Ignite
  Issue Type: Bug
  Components: cache
Reporter: Valentin Kulichenko
 Fix For: 1.6


Need to go through all our internal jobs and closures and make sure they are 
annotation with {{@GridInternal}}.

Probably we can automate this by adding additional check in 
{{ClassesGenerator}} - any class in {{org.apache.ignite.internal}} package that 
implements closure, job, etc., should have this annotation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)