Re: Should Geode stats conform to backwards compatibility constraints?
How about mod MAX_INT? It would wrap back to 0 and make it more consistent with
at least SNMP counters that roll over to 0 when maxed. A monitoring and
graphing system can account for this by recognizing the current value is less
than the previous and typically uses the previous and current values to
recreate the true value.
-Jake
> On Feb 13, 2019, at 8:22 PM, Ryan McMahon wrote:
>
> Sorry that should read “and if the value exceeds MAX_INT”
>
>> On Wed, Feb 13, 2019 at 8:21 PM Ryan McMahon wrote:
>>
>> +1 to introducing a new method which returns the stat as long per Jake’s
>> suggestion. I vote for getInt() to downcast as an int, and the value
>> exceeds MAX_INT, return MAX_INT and possibly add a warning message which
>> points users to the new long version of the method. I think throwing an
>> exception might be a bit much personally.
>>
>> Ryan
>>
>>> On Wed, Feb 13, 2019 at 4:24 PM Owen Nichols wrote:
>>>
>>> Is it possible for the underlying counter to be maintained as a long, and
>>> change the getInt method to return as normal when the value is within
>>> MAX_INT, otherwise throw an exception and/or return MAX_INT when the long
>>> value would overflow an int?
>>>
>>> For the MX Bean, should we keep (but deprecate) the existing attribute,
>>> and add a new attribute TotalNetSearchCompletedAsLong?
>>>
On Feb 13, 2019, at 3:58 PM, Kirk Lund wrote:
Quite a few Geode stats are currently defined as IntCounters which very
easily wrap past Integer.MAX_VALUE. Some users wanted these stats to not
wrap to negative MAX_VALUE, so my team defined the following two tickets
and changed them to LongCounters on the develop branch:
GEODE-6345: StatSamplerStats jvmPauses stat may wrap to negative value
https://issues.apache.org/jira/browse/GEODE-6345
GEODE-6334: CachePerfStats operation count stats may wrap to negative
>>> values
https://issues.apache.org/jira/browse/GEODE-6334
Some people are now concerned that this may break backwards
>>> compatibility
and API for existing users.
There are two potential ways for a user to *experience* this as an API
change:
1) MemberMXBean in JMX
*- int getTotalNetSearchCompleted();*
*+ long getTotalNetSearchCompleted();*
As you can see in GEODE-6334, we changed quite a few stats from integer
>>> to
long in CachePerfStats. The only one directly exposed as an attribute on
MemberMXBean is TotalNetSearchCompleted.
Users will find that this API method changed.
2) Statistics API with undocumented strings
If we assume that users might know or discover that we have a statistics
text id of "cachePerfStats" with an integer stat of name "puts" then
>>> they
could use our Statistics API to get the value:
* 1:CacheFactory cacheFactory = new CacheFactory();*
* 2:Cache cache = cacheFactory.create();*
* 3:*
* 4:Region region = cache.>>> String>createRegionFactory(PARTITION).create("region");*
* 5:*
* 6:Statistics[] statistics =
cache.getDistributedSystem().findStatisticsByTextId("cachePerfStats");*
* 7:int oldPuts = statistics[0].getInt("puts");*
* 8:*
* 9:region.put("some", "value");*
*10:*
*11:int newPuts = statistics[0].getInt("puts");*
*12:*
*13:assertThat(newPuts).isEqualTo(oldPuts + 1);*
The above works in Geode 1.8 and earlier but will begin to throw the
following in Geode 1.9 (when develop branch is released):
*java.lang.IllegalArgumentException: The statistic puts with id 23 is of
type long and it was expected to be an int.*
* at
>>> org.apache.geode.internal.statistics.StatisticDescriptorImpl.checkInt(StatisticDescriptorImpl.java:324)*
* at
>>> org.apache.geode.internal.statistics.StatisticsImpl.getIntId(StatisticsImpl.java:577)*
* at
>>> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:274)*
* at
>>> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:269)*
* at com.user.example.Example.example(Example.java7)*
In order to avoid the above exception, a user would have to change the
>>> code
on lines 7 and 11 to use *getLong* instead of *getInt*.
Should Geode stats be considered a form of API contract and should they
conform to backwards compatibility constraints?
Should we change these from LongCounters back to IntCounters?
Someone did suggest that we could change them back to IntCounters and
>>> then
change our statistics internals to skip to zero anytime a stat attempts
>>> to
increment above MAX_VALUE, however, I think that if we decide that stats
cannot be changed from IntCounters to LongCounters then we should not
>>> make
this change either.
Thanks
Re: Should Geode stats conform to backwards compatibility constraints?
Sorry that should read “and if the value exceeds MAX_INT”
On Wed, Feb 13, 2019 at 8:21 PM Ryan McMahon wrote:
> +1 to introducing a new method which returns the stat as long per Jake’s
> suggestion. I vote for getInt() to downcast as an int, and the value
> exceeds MAX_INT, return MAX_INT and possibly add a warning message which
> points users to the new long version of the method. I think throwing an
> exception might be a bit much personally.
>
> Ryan
>
> On Wed, Feb 13, 2019 at 4:24 PM Owen Nichols wrote:
>
>> Is it possible for the underlying counter to be maintained as a long, and
>> change the getInt method to return as normal when the value is within
>> MAX_INT, otherwise throw an exception and/or return MAX_INT when the long
>> value would overflow an int?
>>
>> For the MX Bean, should we keep (but deprecate) the existing attribute,
>> and add a new attribute TotalNetSearchCompletedAsLong?
>>
>> > On Feb 13, 2019, at 3:58 PM, Kirk Lund wrote:
>> >
>> > Quite a few Geode stats are currently defined as IntCounters which very
>> > easily wrap past Integer.MAX_VALUE. Some users wanted these stats to not
>> > wrap to negative MAX_VALUE, so my team defined the following two tickets
>> > and changed them to LongCounters on the develop branch:
>> >
>> > GEODE-6345: StatSamplerStats jvmPauses stat may wrap to negative value
>> > https://issues.apache.org/jira/browse/GEODE-6345
>> >
>> > GEODE-6334: CachePerfStats operation count stats may wrap to negative
>> values
>> > https://issues.apache.org/jira/browse/GEODE-6334
>> >
>> > Some people are now concerned that this may break backwards
>> compatibility
>> > and API for existing users.
>> >
>> > There are two potential ways for a user to *experience* this as an API
>> > change:
>> >
>> > 1) MemberMXBean in JMX
>> >
>> > *- int getTotalNetSearchCompleted();*
>> > *+ long getTotalNetSearchCompleted();*
>> >
>> > As you can see in GEODE-6334, we changed quite a few stats from integer
>> to
>> > long in CachePerfStats. The only one directly exposed as an attribute on
>> > MemberMXBean is TotalNetSearchCompleted.
>> >
>> > Users will find that this API method changed.
>> >
>> > 2) Statistics API with undocumented strings
>> >
>> > If we assume that users might know or discover that we have a statistics
>> > text id of "cachePerfStats" with an integer stat of name "puts" then
>> they
>> > could use our Statistics API to get the value:
>> >
>> > * 1:CacheFactory cacheFactory = new CacheFactory();*
>> > * 2:Cache cache = cacheFactory.create();*
>> > * 3:*
>> > * 4:Region region = cache.> > String>createRegionFactory(PARTITION).create("region");*
>> > * 5:*
>> > * 6:Statistics[] statistics =
>> > cache.getDistributedSystem().findStatisticsByTextId("cachePerfStats");*
>> > * 7:int oldPuts = statistics[0].getInt("puts");*
>> > * 8:*
>> > * 9:region.put("some", "value");*
>> > *10:*
>> > *11:int newPuts = statistics[0].getInt("puts");*
>> > *12:*
>> > *13:assertThat(newPuts).isEqualTo(oldPuts + 1);*
>> >
>> > The above works in Geode 1.8 and earlier but will begin to throw the
>> > following in Geode 1.9 (when develop branch is released):
>> >
>> > *java.lang.IllegalArgumentException: The statistic puts with id 23 is of
>> > type long and it was expected to be an int.*
>> > * at
>> >
>> org.apache.geode.internal.statistics.StatisticDescriptorImpl.checkInt(StatisticDescriptorImpl.java:324)*
>> > * at
>> >
>> org.apache.geode.internal.statistics.StatisticsImpl.getIntId(StatisticsImpl.java:577)*
>> > * at
>> >
>> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:274)*
>> > * at
>> >
>> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:269)*
>> > * at com.user.example.Example.example(Example.java7)*
>> >
>> > In order to avoid the above exception, a user would have to change the
>> code
>> > on lines 7 and 11 to use *getLong* instead of *getInt*.
>> >
>> > Should Geode stats be considered a form of API contract and should they
>> > conform to backwards compatibility constraints?
>> >
>> > Should we change these from LongCounters back to IntCounters?
>> >
>> > Someone did suggest that we could change them back to IntCounters and
>> then
>> > change our statistics internals to skip to zero anytime a stat attempts
>> to
>> > increment above MAX_VALUE, however, I think that if we decide that stats
>> > cannot be changed from IntCounters to LongCounters then we should not
>> make
>> > this change either.
>> >
>> > Thanks for any input or opinions,
>> > Kirk
>>
>>
Re: Should Geode stats conform to backwards compatibility constraints?
+1 to introducing a new method which returns the stat as long per Jake’s
suggestion. I vote for getInt() to downcast as an int, and the value
exceeds MAX_INT, return MAX_INT and possibly add a warning message which
points users to the new long version of the method. I think throwing an
exception might be a bit much personally.
Ryan
On Wed, Feb 13, 2019 at 4:24 PM Owen Nichols wrote:
> Is it possible for the underlying counter to be maintained as a long, and
> change the getInt method to return as normal when the value is within
> MAX_INT, otherwise throw an exception and/or return MAX_INT when the long
> value would overflow an int?
>
> For the MX Bean, should we keep (but deprecate) the existing attribute,
> and add a new attribute TotalNetSearchCompletedAsLong?
>
> > On Feb 13, 2019, at 3:58 PM, Kirk Lund wrote:
> >
> > Quite a few Geode stats are currently defined as IntCounters which very
> > easily wrap past Integer.MAX_VALUE. Some users wanted these stats to not
> > wrap to negative MAX_VALUE, so my team defined the following two tickets
> > and changed them to LongCounters on the develop branch:
> >
> > GEODE-6345: StatSamplerStats jvmPauses stat may wrap to negative value
> > https://issues.apache.org/jira/browse/GEODE-6345
> >
> > GEODE-6334: CachePerfStats operation count stats may wrap to negative
> values
> > https://issues.apache.org/jira/browse/GEODE-6334
> >
> > Some people are now concerned that this may break backwards compatibility
> > and API for existing users.
> >
> > There are two potential ways for a user to *experience* this as an API
> > change:
> >
> > 1) MemberMXBean in JMX
> >
> > *- int getTotalNetSearchCompleted();*
> > *+ long getTotalNetSearchCompleted();*
> >
> > As you can see in GEODE-6334, we changed quite a few stats from integer
> to
> > long in CachePerfStats. The only one directly exposed as an attribute on
> > MemberMXBean is TotalNetSearchCompleted.
> >
> > Users will find that this API method changed.
> >
> > 2) Statistics API with undocumented strings
> >
> > If we assume that users might know or discover that we have a statistics
> > text id of "cachePerfStats" with an integer stat of name "puts" then they
> > could use our Statistics API to get the value:
> >
> > * 1:CacheFactory cacheFactory = new CacheFactory();*
> > * 2:Cache cache = cacheFactory.create();*
> > * 3:*
> > * 4:Region region = cache. > String>createRegionFactory(PARTITION).create("region");*
> > * 5:*
> > * 6:Statistics[] statistics =
> > cache.getDistributedSystem().findStatisticsByTextId("cachePerfStats");*
> > * 7:int oldPuts = statistics[0].getInt("puts");*
> > * 8:*
> > * 9:region.put("some", "value");*
> > *10:*
> > *11:int newPuts = statistics[0].getInt("puts");*
> > *12:*
> > *13:assertThat(newPuts).isEqualTo(oldPuts + 1);*
> >
> > The above works in Geode 1.8 and earlier but will begin to throw the
> > following in Geode 1.9 (when develop branch is released):
> >
> > *java.lang.IllegalArgumentException: The statistic puts with id 23 is of
> > type long and it was expected to be an int.*
> > * at
> >
> org.apache.geode.internal.statistics.StatisticDescriptorImpl.checkInt(StatisticDescriptorImpl.java:324)*
> > * at
> >
> org.apache.geode.internal.statistics.StatisticsImpl.getIntId(StatisticsImpl.java:577)*
> > * at
> >
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:274)*
> > * at
> >
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:269)*
> > * at com.user.example.Example.example(Example.java7)*
> >
> > In order to avoid the above exception, a user would have to change the
> code
> > on lines 7 and 11 to use *getLong* instead of *getInt*.
> >
> > Should Geode stats be considered a form of API contract and should they
> > conform to backwards compatibility constraints?
> >
> > Should we change these from LongCounters back to IntCounters?
> >
> > Someone did suggest that we could change them back to IntCounters and
> then
> > change our statistics internals to skip to zero anytime a stat attempts
> to
> > increment above MAX_VALUE, however, I think that if we decide that stats
> > cannot be changed from IntCounters to LongCounters then we should not
> make
> > this change either.
> >
> > Thanks for any input or opinions,
> > Kirk
>
>
Re: Geode 1.9 Release Manager
If there are no other takers, I can act as release manager for 1.9 and will cut a release branch this week. On Tue, Jan 29, 2019 at 1:50 PM Alexander Murmann wrote: > Hi everyone! > > February 1st is approaching rapidly which means it's almost time to cut > the 1.9 release. Who is interested in being the release manager for 1.9? > > Thank you! >
Re: Should Geode stats conform to backwards compatibility constraints?
Is it possible for the underlying counter to be maintained as a long, and
change the getInt method to return as normal when the value is within MAX_INT,
otherwise throw an exception and/or return MAX_INT when the long value would
overflow an int?
For the MX Bean, should we keep (but deprecate) the existing attribute, and add
a new attribute TotalNetSearchCompletedAsLong?
> On Feb 13, 2019, at 3:58 PM, Kirk Lund wrote:
>
> Quite a few Geode stats are currently defined as IntCounters which very
> easily wrap past Integer.MAX_VALUE. Some users wanted these stats to not
> wrap to negative MAX_VALUE, so my team defined the following two tickets
> and changed them to LongCounters on the develop branch:
>
> GEODE-6345: StatSamplerStats jvmPauses stat may wrap to negative value
> https://issues.apache.org/jira/browse/GEODE-6345
>
> GEODE-6334: CachePerfStats operation count stats may wrap to negative values
> https://issues.apache.org/jira/browse/GEODE-6334
>
> Some people are now concerned that this may break backwards compatibility
> and API for existing users.
>
> There are two potential ways for a user to *experience* this as an API
> change:
>
> 1) MemberMXBean in JMX
>
> *- int getTotalNetSearchCompleted();*
> *+ long getTotalNetSearchCompleted();*
>
> As you can see in GEODE-6334, we changed quite a few stats from integer to
> long in CachePerfStats. The only one directly exposed as an attribute on
> MemberMXBean is TotalNetSearchCompleted.
>
> Users will find that this API method changed.
>
> 2) Statistics API with undocumented strings
>
> If we assume that users might know or discover that we have a statistics
> text id of "cachePerfStats" with an integer stat of name "puts" then they
> could use our Statistics API to get the value:
>
> * 1:CacheFactory cacheFactory = new CacheFactory();*
> * 2:Cache cache = cacheFactory.create();*
> * 3:*
> * 4:Region region = cache. String>createRegionFactory(PARTITION).create("region");*
> * 5:*
> * 6:Statistics[] statistics =
> cache.getDistributedSystem().findStatisticsByTextId("cachePerfStats");*
> * 7:int oldPuts = statistics[0].getInt("puts");*
> * 8:*
> * 9:region.put("some", "value");*
> *10:*
> *11:int newPuts = statistics[0].getInt("puts");*
> *12:*
> *13:assertThat(newPuts).isEqualTo(oldPuts + 1);*
>
> The above works in Geode 1.8 and earlier but will begin to throw the
> following in Geode 1.9 (when develop branch is released):
>
> *java.lang.IllegalArgumentException: The statistic puts with id 23 is of
> type long and it was expected to be an int.*
> * at
> org.apache.geode.internal.statistics.StatisticDescriptorImpl.checkInt(StatisticDescriptorImpl.java:324)*
> * at
> org.apache.geode.internal.statistics.StatisticsImpl.getIntId(StatisticsImpl.java:577)*
> * at
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:274)*
> * at
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:269)*
> * at com.user.example.Example.example(Example.java7)*
>
> In order to avoid the above exception, a user would have to change the code
> on lines 7 and 11 to use *getLong* instead of *getInt*.
>
> Should Geode stats be considered a form of API contract and should they
> conform to backwards compatibility constraints?
>
> Should we change these from LongCounters back to IntCounters?
>
> Someone did suggest that we could change them back to IntCounters and then
> change our statistics internals to skip to zero anytime a stat attempts to
> increment above MAX_VALUE, however, I think that if we decide that stats
> cannot be changed from IntCounters to LongCounters then we should not make
> this change either.
>
> Thanks for any input or opinions,
> Kirk
Re: Should Geode stats conform to backwards compatibility constraints?
+1 to what Jake said about MemberMXBean - that is definitely a public API
class, so we need to deprecate the old method and introduce a new one.
I'm also not clear about the stats. Technically, they are discoverable
through the public API, so maybe? It seems like they are mix of things
users might want and stuff that is more internal/debugging oriented. If the
downcasting trick Jake mentioned isn't that hard, I'd say we should do
that. That said, I suspect we've renamed some stats in the past.
One other thing to consider - will changing the size of the stats affect
any tooling like VSD or gfsh or ?? that may have to deal with stat data
from multiple geode versions?
-Dan
On Wed, Feb 13, 2019 at 3:59 PM Kirk Lund wrote:
> Quite a few Geode stats are currently defined as IntCounters which very
> easily wrap past Integer.MAX_VALUE. Some users wanted these stats to not
> wrap to negative MAX_VALUE, so my team defined the following two tickets
> and changed them to LongCounters on the develop branch:
>
> GEODE-6345: StatSamplerStats jvmPauses stat may wrap to negative value
> https://issues.apache.org/jira/browse/GEODE-6345
>
> GEODE-6334: CachePerfStats operation count stats may wrap to negative
> values
> https://issues.apache.org/jira/browse/GEODE-6334
>
> Some people are now concerned that this may break backwards compatibility
> and API for existing users.
>
> There are two potential ways for a user to *experience* this as an API
> change:
>
> 1) MemberMXBean in JMX
>
> *- int getTotalNetSearchCompleted();*
> *+ long getTotalNetSearchCompleted();*
>
> As you can see in GEODE-6334, we changed quite a few stats from integer to
> long in CachePerfStats. The only one directly exposed as an attribute on
> MemberMXBean is TotalNetSearchCompleted.
>
> Users will find that this API method changed.
>
> 2) Statistics API with undocumented strings
>
> If we assume that users might know or discover that we have a statistics
> text id of "cachePerfStats" with an integer stat of name "puts" then they
> could use our Statistics API to get the value:
>
> * 1:CacheFactory cacheFactory = new CacheFactory();*
> * 2:Cache cache = cacheFactory.create();*
> * 3:*
> * 4:Region region = cache. String>createRegionFactory(PARTITION).create("region");*
> * 5:*
> * 6:Statistics[] statistics =
> cache.getDistributedSystem().findStatisticsByTextId("cachePerfStats");*
> * 7:int oldPuts = statistics[0].getInt("puts");*
> * 8:*
> * 9:region.put("some", "value");*
> *10:*
> *11:int newPuts = statistics[0].getInt("puts");*
> *12:*
> *13:assertThat(newPuts).isEqualTo(oldPuts + 1);*
>
> The above works in Geode 1.8 and earlier but will begin to throw the
> following in Geode 1.9 (when develop branch is released):
>
> *java.lang.IllegalArgumentException: The statistic puts with id 23 is of
> type long and it was expected to be an int.*
> * at
>
> org.apache.geode.internal.statistics.StatisticDescriptorImpl.checkInt(StatisticDescriptorImpl.java:324)*
> * at
>
> org.apache.geode.internal.statistics.StatisticsImpl.getIntId(StatisticsImpl.java:577)*
> * at
>
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:274)*
> * at
>
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:269)*
> * at com.user.example.Example.example(Example.java7)*
>
> In order to avoid the above exception, a user would have to change the code
> on lines 7 and 11 to use *getLong* instead of *getInt*.
>
> Should Geode stats be considered a form of API contract and should they
> conform to backwards compatibility constraints?
>
> Should we change these from LongCounters back to IntCounters?
>
> Someone did suggest that we could change them back to IntCounters and then
> change our statistics internals to skip to zero anytime a stat attempts to
> increment above MAX_VALUE, however, I think that if we decide that stats
> cannot be changed from IntCounters to LongCounters then we should not make
> this change either.
>
> Thanks for any input or opinions,
> Kirk
>
Re: Should Geode stats conform to backwards compatibility constraints?
I don’t consider the stats API a public API. I think it is a leaked internal
API. Do we document the interactions with this API? Do we document the stat
names? I consider documentation the API. I can discover all sorts of exposed
methods in a library but shouldn’t consider them contracted API.
If however we come to a conclusion that they are API then we could work around
the issues below. We could changes the stat API to be a little forgiving on the
getInt and related calls if the internal storage is a long. We simply downcast
the long to an int. The behavior to the end user does not change but if they
use the getLong they get the new improved stat. For the JMX we just add a new
method, getTotalNetSearchCompletedLong() to get the long version of the stat.
-Jake
> On Feb 13, 2019, at 3:58 PM, Kirk Lund wrote:
>
> Quite a few Geode stats are currently defined as IntCounters which very
> easily wrap past Integer.MAX_VALUE. Some users wanted these stats to not
> wrap to negative MAX_VALUE, so my team defined the following two tickets
> and changed them to LongCounters on the develop branch:
>
> GEODE-6345: StatSamplerStats jvmPauses stat may wrap to negative value
> https://issues.apache.org/jira/browse/GEODE-6345
>
> GEODE-6334: CachePerfStats operation count stats may wrap to negative values
> https://issues.apache.org/jira/browse/GEODE-6334
>
> Some people are now concerned that this may break backwards compatibility
> and API for existing users.
>
> There are two potential ways for a user to *experience* this as an API
> change:
>
> 1) MemberMXBean in JMX
>
> *- int getTotalNetSearchCompleted();*
> *+ long getTotalNetSearchCompleted();*
>
> As you can see in GEODE-6334, we changed quite a few stats from integer to
> long in CachePerfStats. The only one directly exposed as an attribute on
> MemberMXBean is TotalNetSearchCompleted.
>
> Users will find that this API method changed.
>
> 2) Statistics API with undocumented strings
>
> If we assume that users might know or discover that we have a statistics
> text id of "cachePerfStats" with an integer stat of name "puts" then they
> could use our Statistics API to get the value:
>
> * 1:CacheFactory cacheFactory = new CacheFactory();*
> * 2:Cache cache = cacheFactory.create();*
> * 3:*
> * 4:Region region = cache. String>createRegionFactory(PARTITION).create("region");*
> * 5:*
> * 6:Statistics[] statistics =
> cache.getDistributedSystem().findStatisticsByTextId("cachePerfStats");*
> * 7:int oldPuts = statistics[0].getInt("puts");*
> * 8:*
> * 9:region.put("some", "value");*
> *10:*
> *11:int newPuts = statistics[0].getInt("puts");*
> *12:*
> *13:assertThat(newPuts).isEqualTo(oldPuts + 1);*
>
> The above works in Geode 1.8 and earlier but will begin to throw the
> following in Geode 1.9 (when develop branch is released):
>
> *java.lang.IllegalArgumentException: The statistic puts with id 23 is of
> type long and it was expected to be an int.*
> * at
> org.apache.geode.internal.statistics.StatisticDescriptorImpl.checkInt(StatisticDescriptorImpl.java:324)*
> * at
> org.apache.geode.internal.statistics.StatisticsImpl.getIntId(StatisticsImpl.java:577)*
> * at
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:274)*
> * at
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:269)*
> * at com.user.example.Example.example(Example.java7)*
>
> In order to avoid the above exception, a user would have to change the code
> on lines 7 and 11 to use *getLong* instead of *getInt*.
>
> Should Geode stats be considered a form of API contract and should they
> conform to backwards compatibility constraints?
>
> Should we change these from LongCounters back to IntCounters?
>
> Someone did suggest that we could change them back to IntCounters and then
> change our statistics internals to skip to zero anytime a stat attempts to
> increment above MAX_VALUE, however, I think that if we decide that stats
> cannot be changed from IntCounters to LongCounters then we should not make
> this change either.
>
> Thanks for any input or opinions,
> Kirk
Precheckin flakiness
Despite our attempts to make precheckin free of flaky failures, I'm still seeing at least one flaky failure (that does not reproduce and does not correlate to my PR changes) in every single precheckin launched for my PRs. For example, the latest one ( https://concourse.apachegeode-ci.info/builds/37714) is: *org.apache.geode.internal.cache.ha.HAGIIDUnitTest > testGIIRegionQueue FAILED* *java.lang.AssertionError: Suspicious strings were written to the log during this run.* *Fix the strings or use IgnoredException.addIgnoredException to ignore.* * ---* *Found suspect string in log4j at line 2703* *java.net.ConnectException: Connection refused (Connection refused)* I've searched Jira and there is no open ticket for this test. Are we expected to file new Jira tickets for every single failure I see in precheckin? And if so, are other developers filing tickets like I am? Or do we go back to ignoring all of these flaky failures (I don't really consider this a good option)? I'm worried that I'm wasting a lot of time filing all of these Jira tickets if no one is ever going to work on them and then they get bulk closed by someone 12 months from now...
Should Geode stats conform to backwards compatibility constraints?
Quite a few Geode stats are currently defined as IntCounters which very
easily wrap past Integer.MAX_VALUE. Some users wanted these stats to not
wrap to negative MAX_VALUE, so my team defined the following two tickets
and changed them to LongCounters on the develop branch:
GEODE-6345: StatSamplerStats jvmPauses stat may wrap to negative value
https://issues.apache.org/jira/browse/GEODE-6345
GEODE-6334: CachePerfStats operation count stats may wrap to negative values
https://issues.apache.org/jira/browse/GEODE-6334
Some people are now concerned that this may break backwards compatibility
and API for existing users.
There are two potential ways for a user to *experience* this as an API
change:
1) MemberMXBean in JMX
*- int getTotalNetSearchCompleted();*
*+ long getTotalNetSearchCompleted();*
As you can see in GEODE-6334, we changed quite a few stats from integer to
long in CachePerfStats. The only one directly exposed as an attribute on
MemberMXBean is TotalNetSearchCompleted.
Users will find that this API method changed.
2) Statistics API with undocumented strings
If we assume that users might know or discover that we have a statistics
text id of "cachePerfStats" with an integer stat of name "puts" then they
could use our Statistics API to get the value:
* 1:CacheFactory cacheFactory = new CacheFactory();*
* 2:Cache cache = cacheFactory.create();*
* 3:*
* 4:Region region = cache.createRegionFactory(PARTITION).create("region");*
* 5:*
* 6:Statistics[] statistics =
cache.getDistributedSystem().findStatisticsByTextId("cachePerfStats");*
* 7:int oldPuts = statistics[0].getInt("puts");*
* 8:*
* 9:region.put("some", "value");*
*10:*
*11:int newPuts = statistics[0].getInt("puts");*
*12:*
*13:assertThat(newPuts).isEqualTo(oldPuts + 1);*
The above works in Geode 1.8 and earlier but will begin to throw the
following in Geode 1.9 (when develop branch is released):
*java.lang.IllegalArgumentException: The statistic puts with id 23 is of
type long and it was expected to be an int.*
* at
org.apache.geode.internal.statistics.StatisticDescriptorImpl.checkInt(StatisticDescriptorImpl.java:324)*
* at
org.apache.geode.internal.statistics.StatisticsImpl.getIntId(StatisticsImpl.java:577)*
* at
org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:274)*
* at
org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:269)*
* at com.user.example.Example.example(Example.java7)*
In order to avoid the above exception, a user would have to change the code
on lines 7 and 11 to use *getLong* instead of *getInt*.
Should Geode stats be considered a form of API contract and should they
conform to backwards compatibility constraints?
Should we change these from LongCounters back to IntCounters?
Someone did suggest that we could change them back to IntCounters and then
change our statistics internals to skip to zero anytime a stat attempts to
increment above MAX_VALUE, however, I think that if we decide that stats
cannot be changed from IntCounters to LongCounters then we should not make
this change either.
Thanks for any input or opinions,
Kirk
Re: Apache Geode PMC quarterly report: DRAFT for your review
+1 - Thanks for the mention of the native client! On Wed, Feb 13, 2019 at 9:05 AM Dave Barnes wrote: > I've incorporated Anthony's additions and an addition of my own (Karen > succeeded Mark as PMC chair). > Any other suggestions? Review closes at noon PST. > > On Wed, Feb 13, 2019 at 7:22 AM Anthony Baker wrote: > > > Under activity I would add: > > > > - Added benchmarks to baseline performance > > - Explored the use of micrometer for exposing metrics of cache operations > > > > Anthony > > > > > > > On Feb 12, 2019, at 3:34 PM, Dave Barnes wrote: > > > > > > Please respond by noon tomorrow. > > > Pretty complete, as far as I know, except for public events and > > > presentations. > > > Thanks, > > > Dave > > > > > > Description: > > > > > > Apache Geode provides a database-like consistency model, reliable > > > transaction processing and a shared-nothing architecture to maintain > very > > > low latency performance with high concurrency processing. > > > Issues: > > > > > > There are no issues requiring board attention at this time. > > > Activity: > > > > > > - We released v1.8.0, containing 124 improvements and new features, > > > resolving 83 bugs and a total of 380 JIRA tickets > > > - The v1.8.0 release includes the first release of the geode-native > > > source, a C++/C# client API for Geode. > > > - Security enhancements include support for Trust and Keystore > > rotation, > > > endpoint validation during SSL handshake, and dynamic function > > security. > > > > > > Health report: > > > > > > - Mailing lists remain active and productive. > > > - JIRA tickets show that issues continue to be identified and > resolved. > > > - We’re continuing to work on attracting new contributors and making > it > > > easier to participate in the community. > > > > > > PMC changes: > > > > > > - Currently 46 PMC members. > > > - No new PMC members added in the last 3 months > > > - Last PMC addition was Dick Cavender on Tue Feb 20 2018 > > > > > > Committer base changes: > > > > > > - Currently 98 committers. > > > - No new committers added in the last 3 months, but several > invitations > > > in the pipeline > > > - Last committer addition was Robert Houghton at Mon Oct 01 2018 > > > > > > Releases: > > > > > > - 1.8.0 was released on Wed Dec 12 2018 > > > > > > Mailing list activity: > > > > > > Mailing lists have remained active and have maintained consistent usage > > > levels. > > > > > > - > > > > > > dev@geode.apache.org: > > > - 193 subscribers (up 3 in the last 3 months): > > > - 465 emails sent to list (825 in previous quarter) > > > - > > > > > > iss...@geode.apache.org: > > > - 55 subscribers (up 0 in the last 3 months): > > > - 3342 emails sent to list (4259 in previous quarter) > > > - > > > > > > notificati...@geode.apache.org: > > > - 7 subscribers (up 0 in the last 3 months): > > > - 2500 emails sent to list (1865 in previous quarter) > > > - > > > > > > u...@geode.apache.org: > > > - 250 subscribers (up 0 in the last 3 months): > > > - 94 emails sent to list (212 in previous quarter) > > > > > > JIRA activity: > > > > > > - 357 JIRA tickets created in the last 3 months > > > - 314 JIRA tickets closed/resolved in the last 3 months > > > > > -- Charlie Black | cbl...@pivotal.io
Re: [DISCUSS] Static analysis of statics
On Wed, Feb 13, 2019 at 9:03 AM Dan Smith wrote: … > I can switch them to @MakeReferentImmutable if that makes more sense. > > -Dan I think you understand my concerns. I trust you to decide what's best.
Re: Apache Geode PMC quarterly report: DRAFT for your review
I've incorporated Anthony's additions and an addition of my own (Karen succeeded Mark as PMC chair). Any other suggestions? Review closes at noon PST. On Wed, Feb 13, 2019 at 7:22 AM Anthony Baker wrote: > Under activity I would add: > > - Added benchmarks to baseline performance > - Explored the use of micrometer for exposing metrics of cache operations > > Anthony > > > > On Feb 12, 2019, at 3:34 PM, Dave Barnes wrote: > > > > Please respond by noon tomorrow. > > Pretty complete, as far as I know, except for public events and > > presentations. > > Thanks, > > Dave > > > > Description: > > > > Apache Geode provides a database-like consistency model, reliable > > transaction processing and a shared-nothing architecture to maintain very > > low latency performance with high concurrency processing. > > Issues: > > > > There are no issues requiring board attention at this time. > > Activity: > > > > - We released v1.8.0, containing 124 improvements and new features, > > resolving 83 bugs and a total of 380 JIRA tickets > > - The v1.8.0 release includes the first release of the geode-native > > source, a C++/C# client API for Geode. > > - Security enhancements include support for Trust and Keystore > rotation, > > endpoint validation during SSL handshake, and dynamic function > security. > > > > Health report: > > > > - Mailing lists remain active and productive. > > - JIRA tickets show that issues continue to be identified and resolved. > > - We’re continuing to work on attracting new contributors and making it > > easier to participate in the community. > > > > PMC changes: > > > > - Currently 46 PMC members. > > - No new PMC members added in the last 3 months > > - Last PMC addition was Dick Cavender on Tue Feb 20 2018 > > > > Committer base changes: > > > > - Currently 98 committers. > > - No new committers added in the last 3 months, but several invitations > > in the pipeline > > - Last committer addition was Robert Houghton at Mon Oct 01 2018 > > > > Releases: > > > > - 1.8.0 was released on Wed Dec 12 2018 > > > > Mailing list activity: > > > > Mailing lists have remained active and have maintained consistent usage > > levels. > > > > - > > > > dev@geode.apache.org: > > - 193 subscribers (up 3 in the last 3 months): > > - 465 emails sent to list (825 in previous quarter) > > - > > > > iss...@geode.apache.org: > > - 55 subscribers (up 0 in the last 3 months): > > - 3342 emails sent to list (4259 in previous quarter) > > - > > > > notificati...@geode.apache.org: > > - 7 subscribers (up 0 in the last 3 months): > > - 2500 emails sent to list (1865 in previous quarter) > > - > > > > u...@geode.apache.org: > > - 250 subscribers (up 0 in the last 3 months): > > - 94 emails sent to list (212 in previous quarter) > > > > JIRA activity: > > > > - 357 JIRA tickets created in the last 3 months > > - 314 JIRA tickets closed/resolved in the last 3 months > >
Re: [DISCUSS] Static analysis of statics
Regarding @Immutable - yes it's intentionally a field annotation as well as a class annotation. The reason to make it a field annotation is that the static analysis tools aren't quite cool enough to figure out if a field is really immutable so we have to manually tell the tool that the field is immutable. For example "public static final List = Collections.unmodifiableList(X)" might immutable but it's hard to deduce that. I agree that an @Immutable field should be final and the referent is immutable. Also agreed - @MakeImmutable should also apply to classes. I'll fix that. I did an earlier pass where I made as many fields as I could final - so most of the remaining cases would be your @MakeReferentImmutable case. I can switch them to @MakeReferentImmutable if that makes more sense. -Dan On Tue, Feb 12, 2019 at 3:18 PM Bill Burcham wrote: > I think the @Immutable anno in *Java Concurrency and Practice* is a class > annotation—not a field one. > > Looking at that PR, it looks like this @Immutable anno is usable both on a > type (class) and on a field. > > Is that an oversight? If not, then what does it mean? Does @Immutable on a > field mean both: > > • the field is final > • the object referenced by the field is immutable > > ? > > Looks like in the current PR @MakeImmutable applies only to fields—not > classes. I imagine that's an oversight. > > My quick thought is that these two annotations, in the spirit of *JCP*, > should be type/class-level and not field-level, > > Immutable > MakeImmutable > > and that perhaps we could have some field-level annos like: > > MakeNonStatic > MakeFinal > MakeReferentImmutable - change the type referenced by this field to be an > immutable one > > In this way your MakeImmutable field anno is teased apart into two: > MakeFinal, MakeReferentImmutable. The possible benefit is that the two > annos can be used either separately or together to cover 3 situations > instead of just one. >
Re: Apache Geode PMC quarterly report: DRAFT for your review
+1 - Looks good to me. -Dan On Tue, Feb 12, 2019 at 3:34 PM Dave Barnes wrote: > Please respond by noon tomorrow. > Pretty complete, as far as I know, except for public events and > presentations. > Thanks, > Dave > > Description: > > Apache Geode provides a database-like consistency model, reliable > transaction processing and a shared-nothing architecture to maintain very > low latency performance with high concurrency processing. > Issues: > > There are no issues requiring board attention at this time. > Activity: > >- We released v1.8.0, containing 124 improvements and new features, >resolving 83 bugs and a total of 380 JIRA tickets >- The v1.8.0 release includes the first release of the geode-native >source, a C++/C# client API for Geode. >- Security enhancements include support for Trust and Keystore rotation, >endpoint validation during SSL handshake, and dynamic function security. > > Health report: > >- Mailing lists remain active and productive. >- JIRA tickets show that issues continue to be identified and resolved. >- We’re continuing to work on attracting new contributors and making it >easier to participate in the community. > > PMC changes: > >- Currently 46 PMC members. >- No new PMC members added in the last 3 months >- Last PMC addition was Dick Cavender on Tue Feb 20 2018 > > Committer base changes: > >- Currently 98 committers. >- No new committers added in the last 3 months, but several invitations >in the pipeline >- Last committer addition was Robert Houghton at Mon Oct 01 2018 > > Releases: > >- 1.8.0 was released on Wed Dec 12 2018 > > Mailing list activity: > > Mailing lists have remained active and have maintained consistent usage > levels. > >- > >dev@geode.apache.org: >- 193 subscribers (up 3 in the last 3 months): > - 465 emails sent to list (825 in previous quarter) >- > >iss...@geode.apache.org: >- 55 subscribers (up 0 in the last 3 months): > - 3342 emails sent to list (4259 in previous quarter) >- > >notificati...@geode.apache.org: >- 7 subscribers (up 0 in the last 3 months): > - 2500 emails sent to list (1865 in previous quarter) >- > >u...@geode.apache.org: >- 250 subscribers (up 0 in the last 3 months): > - 94 emails sent to list (212 in previous quarter) > > JIRA activity: > >- 357 JIRA tickets created in the last 3 months >- 314 JIRA tickets closed/resolved in the last 3 months >
Re: Apache Geode PMC quarterly report: DRAFT for your review
Under activity I would add: - Added benchmarks to baseline performance - Explored the use of micrometer for exposing metrics of cache operations Anthony > On Feb 12, 2019, at 3:34 PM, Dave Barnes wrote: > > Please respond by noon tomorrow. > Pretty complete, as far as I know, except for public events and > presentations. > Thanks, > Dave > > Description: > > Apache Geode provides a database-like consistency model, reliable > transaction processing and a shared-nothing architecture to maintain very > low latency performance with high concurrency processing. > Issues: > > There are no issues requiring board attention at this time. > Activity: > > - We released v1.8.0, containing 124 improvements and new features, > resolving 83 bugs and a total of 380 JIRA tickets > - The v1.8.0 release includes the first release of the geode-native > source, a C++/C# client API for Geode. > - Security enhancements include support for Trust and Keystore rotation, > endpoint validation during SSL handshake, and dynamic function security. > > Health report: > > - Mailing lists remain active and productive. > - JIRA tickets show that issues continue to be identified and resolved. > - We’re continuing to work on attracting new contributors and making it > easier to participate in the community. > > PMC changes: > > - Currently 46 PMC members. > - No new PMC members added in the last 3 months > - Last PMC addition was Dick Cavender on Tue Feb 20 2018 > > Committer base changes: > > - Currently 98 committers. > - No new committers added in the last 3 months, but several invitations > in the pipeline > - Last committer addition was Robert Houghton at Mon Oct 01 2018 > > Releases: > > - 1.8.0 was released on Wed Dec 12 2018 > > Mailing list activity: > > Mailing lists have remained active and have maintained consistent usage > levels. > > - > > dev@geode.apache.org: > - 193 subscribers (up 3 in the last 3 months): > - 465 emails sent to list (825 in previous quarter) > - > > iss...@geode.apache.org: > - 55 subscribers (up 0 in the last 3 months): > - 3342 emails sent to list (4259 in previous quarter) > - > > notificati...@geode.apache.org: > - 7 subscribers (up 0 in the last 3 months): > - 2500 emails sent to list (1865 in previous quarter) > - > > u...@geode.apache.org: > - 250 subscribers (up 0 in the last 3 months): > - 94 emails sent to list (212 in previous quarter) > > JIRA activity: > > - 357 JIRA tickets created in the last 3 months > - 314 JIRA tickets closed/resolved in the last 3 months
