Re: Default branch for geode-examples

[Dan Smith]

Good  question! I think we intentionally left the default branch as master
so that users cloning the examples from github will get examples that work
against a released version of geode.

That said, I don't feel too strongly if we would rather make things more
consistent with the geode repo - it seems like we also try to keep develop
stable.

-Dan

On Wed, Dec 12, 2018 at 9:07 AM Anthony Baker  wrote:

> Alexander noticed that some recent PR’s against the geode-examples repo
> made against the master branch.  That breaks the gitflow approach where
> only released code is on master.  Should we update the default branch to be
> develop?
>
>
> Anthony
>
>

Re: JIRA Access

[Dan Smith]

Hi Charlie,

Sure, what's your user name in the apache JIRA?

-Dan

On Thu, Jan 3, 2019 at 3:02 PM Charlie Black  wrote:

> I am currently acting as a product manager for geode native client
> component.   I was wondering if I could have edit privileges for the Geode
> JIRA?
>
> Thanks,
>
> Charlie
>
> --
> Charlie Black | cbl...@pivotal.io
>

Re: JIRA Access

[Dan Smith]

Done! You should have access now.

-Dan

On Thu, Jan 3, 2019 at 3:53 PM Charlie Black  wrote:

> Dan,
>
> Awesome and thanks!   My username is charliemblack
>
> Charlie
>
> On Thu, Jan 3, 2019 at 3:17 PM Dan Smith  wrote:
>
> > Hi Charlie,
> >
> > Sure, what's your user name in the apache JIRA?
> >
> > -Dan
> >
> > On Thu, Jan 3, 2019 at 3:02 PM Charlie Black  wrote:
> >
> > > I am currently acting as a product manager for geode native client
> > > component.   I was wondering if I could have edit privileges for the
> > Geode
> > > JIRA?
> > >
> > > Thanks,
> > >
> > > Charlie
> > >
> > > --
> > > Charlie Black | cbl...@pivotal.io
> > >
> >
>
>
> --
> Charlie Black | cbl...@pivotal.io
>

Re: Defining public SPIs in Geode

[Dan Smith]

What sort of services are you thinking of adding?

Technically, things like a CacheLoader or a SecurityManager are SPIs in the
sense that they places where you can plug your code into geode. Using a
ServiceLoader might ok in cases where we want something to be plugged in if
the jar is on the classpath, without any additional configuration. So it
probably depends on what the SPI is for.

-Dan

On Tue, Jan 8, 2019 at 3:59 PM Dale Emery  wrote:

> We are exploring adding one or more public Service Provider Interfaces
> (SPIs) for Geode, and would like some guidance about standards,
> conventions, precedent, and such.
>
> Do we have standards or conventions for creating SPIs? Good examples? Bad
> examples?
>
> Are there standards or conventions described elsewhere (e.g. Spring) that
> we should consider when defining SPIs?
>
> Is it as simple(!) as defining a service interface, then using Java's
> ServiceLoader mechanism to discover and load instances of it?
>
> Any additional considerations for adding a public SPI, compared to a
> private one?
>
> Cheers,
> Dale
>
>

Re: [Proposal] Adding Micrometer to Apache Geode

[Dan Smith]

+1 Micrometer looks awesome.

I'd love to see more details on how this will work. How will you hook
micrometer in? Will all of the stats that are currently exposed in VSD be
exposed in micrometer, or just some subset? How does a user configure which
monitoring system(s) micrometer is sending data too? Will micrometer stats
also be enabled on clients?

-Dan

On Tue, Jan 15, 2019 at 11:58 AM Dale Emery  wrote:

> Hi Udo and all,
>
> > On Jan 15, 2019, at 10:06 AM, Udo Kohlmeyer  wrote:
> >
> > It would be good to see the new Micrometer stats have a logical
> grouping, that makes it easier for users to search for metrics.
>
> Do you know of any useful guidelines or conventions for creating a
> hierarchy of metrics, and criteria for evaluating the goodness of the
> hierarchy?
>
> And for which details to represent in the meter name hierarchy vs
> tags/dimensions?
>
> Dale
>
> —
> Dale Emery
> dem...@pivotal.io
>
>

Re: [Proposal] adding a new type of PdxInstance

[Dan Smith]

If I understand this right, you are talking about a way to create a
PdxInstance that has no corresponding java class. How about just a
RegionService.createPdxInstanceFactory() method that doesn't take a
classname, and therefore has no corresponding java class? It seems a
PdxInstances without a class is a more fundamental PdxInstance. A
PdxInstance with a java classname on it is just an extension of the
classless version.

I agree what Udo is talking about - giving the user better control of
*when* there value is deserialized to a java object - is also valuable, but
a separate feature.

-Dan

On Tue, Jan 15, 2019 at 1:09 PM Darrel Schneider 
wrote:

> Even before the JSON pdx support we had internal support for a PdxInstance
> that deserializes as a PdxInstance.
> This is just adding an external api for that already existing internal
> feature. So it is pretty simple to do if we can figure out how to name it.
>
>
> On Tue, Jan 15, 2019 at 11:18 AM Galen O'Sullivan 
> wrote:
>
> > I suspect Udo is remembering something we both had to deal with, which is
> > that the lack of values to get/put PDXInstances on Regions make some
> > patterns difficult. In internal code, we have to set some thread-locals
> to
> > get serialized values out, and in general, I think that setting
> > pdx-read-serialized is a violation of the contract you'd expect from the
> > type signature of get, put, etc. Having a separate API for serialized
> > objects, and possibly region-level configuration, makes a lot more sense.
> > You could even have the non-PDX get fail on regions that are set to only
> > use PDX-serialized objects for everything.
> >
> > We already have something like a PdxInstance that always deserializes to
> a
> > PdxInstance -- have a look at the __GEMFIRE_JSON mess that we use for
> JSON.
> > However you end up doing the new PDXInstance stuff, I strongly suggest
> > using the new solution for JSON objects.
> >
> > -Galen
> >
> > On Tue, Jan 15, 2019 at 10:49 AM Darrel Schneider  >
> > wrote:
> >
> > > I like the idea of adding support to the region configuration that lets
> > > users control how it stores the data. But even if we did that, and you
> > are
> > > correct that it would be much more work, I don't think it would address
> > > this issue or remove the value of a PdxInstance that always
> deserializes
> > to
> > > a PdxInstance. So I'd like this proposal to stay focused on PdxInstance
> > and
> > > not get side tracked. PdxInstances can be used outside of regions (for
> > > example arguments to functions).
> > >
> > > I'd like to see a separate proposal about being able to configure how a
> > > region stores its data. I could be wrong, but I think that proposal
> would
> > > focus on the values, not the keys. Storing keys as serialized data is
> > > tricky because you need to come up with a equals and hashCode and if
> > those
> > > are going to be done based on a sequence of serialized bytes then you
> > > really need to understand your serialization code and make sure that
> > > "equal" objects always have the same serialized form.
> > >
> > >
> > > On Tue, Jan 15, 2019 at 10:38 AM Udo Kohlmeyer  wrote:
> > >
> > > > Darrel, thank you for this.
> > > >
> > > > I would like to propose a counter-proposal.
> > > >
> > > > Instead of introducing another PDXInstance type, why don't we improve
> > > > the serialization framework itself? I know my proposal is most likely
> > > > going to take a little more effort than adding a new type, but I
> > believe
> > > > it is less of a work around.
> > > >
> > > > MY proposal is to have the PDX serialization configuration be a
> little
> > > > more explicit. In the sense that a user can define serialization
> > details
> > > > down to the Region.Key or Region.Value level.
> > > >
> > > > Why would we possibly have a "one size fits all" approach? Could one
> > > > have a setup where serialization configuration is stored on a per
> > region
> > > > basis. Maybe in some cases we want to deserialize the key and in some
> > > > cases we don't want to. In some regions we want to leave the value in
> > > > serialized form and in others we don't. The point is, why limit to a
> > > > single flag.
> > > >
> > > > --Udo
> > > >
> > > > On 1/15/19 10:17, Darrel Schneider wrote:
> > > > > As part of GEODE-6272 we realized we need a way to use a
> PdxInstance
> > as
> > > > key
> > > > > for a Region entry. The problem with the current PdxInstance
> behavior
> > > is
> > > > > that in some members the key may be seen as a PdxInstance and in
> > others
> > > > > seen as an instance of a domain class. This inconsistency can lead
> to
> > > > > problems, in particular with partitioned regions because of the
> key's
> > > > hash
> > > > > code being used to determine the bucket. You can read more about
> this
> > > > here:
> > > > >
> > > >
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__geode.apache.org_docs_guide_17_developing_data-5Fserialization_using-5Fpdx-5Fregion

Re: Preventing new build warnings

[Dan Smith]

Sounds good. +1 to failing the build if new warnings are introduced.

-Dan

On Tue, Jan 15, 2019 at 12:59 PM Galen O'Sullivan 
wrote:

> I'm for failing CI on warnings. It would be nice to reduce or eliminate our
> existing build warnings as well.
>
> Thanks,
> Galen
>
>
> On Tue, Jan 15, 2019 at 12:33 PM Peter Tran  wrote:
>
> > Hello!
> >
> > I've noticed that there is no mechanism in which we prevent new PRs from
> > introduce new build warnings. In our PR template we ask people to self
> > report that they have a "clean build" but nothing more to ensure we're
> not
> > adding new warnings.
> >
> > Has there been an initiative to address this in the past? Would it be too
> > restrictive if CI fails if new warnings are introduced in a PR?
> >
> > Thanks
> > --
> > Peter Tran
> >
>

Re: dunit hang in ClusterConfigLocatorRestartDUnitTest

[Dan Smith]

Thanks Kirk - this is GEODE-5676. I'm glad the callstacks are finally
working again, that should help us track this down! I'll update that ticket
with anything I can figure out from this failure.

-Dan

On Wed, Jan 23, 2019 at 10:05 AM Kirk Lund  wrote:

> Looks like there's a Test worker thread that keeps running and changing.
> Maybe the dunit test is pushing the dunit job over its time limit:
>
> "Test worker" #27 prio=5 os_prio=0 cpu=1332.79ms elapsed=4871.91s
> tid=0x7f67c4a78800 nid=0x1c runnable  [0x7f675fbf3000]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(java.base@11.0.1/Native
> Method)
> at java.net.SocketInputStream.socketRead(java.base@11.0.1
> /SocketInputStream.java:115)
> at java.net.SocketInputStream.read(java.base@11.0.1
> /SocketInputStream.java:168)
> at java.net.SocketInputStream.read(java.base@11.0.1
> /SocketInputStream.java:140)
> at java.io.BufferedInputStream.fill(java.base@11.0.1
> /BufferedInputStream.java:252)
> at java.io.BufferedInputStream.read(java.base@11.0.1
> /BufferedInputStream.java:271)
> - locked <0xd131b270> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readByte(java.base@11.0.1
> /DataInputStream.java:270)
> at sun.rmi.transport.StreamRemoteCall.executeCall(java.rmi@11.0.1
> /StreamRemoteCall.java:222)
> at sun.rmi.server.UnicastRef.invoke(java.rmi@11.0.1
> /UnicastRef.java:161)
> at
>
> java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(java.rmi@11.0.1
> /RemoteObjectInvocationHandler.java:209)
> at
> java.rmi.server.RemoteObjectInvocationHandler.invoke(java.rmi@11.0.1
> /RemoteObjectInvocationHandler.java:161)
> at com.sun.proxy.$Proxy32.executeMethodOnObject(Unknown Source)
> at
> org.apache.geode.test.dunit.VM.executeMethodOnObject(VM.java:530)
> at org.apache.geode.test.dunit.VM.invoke(VM.java:390)
> at
>
> org.apache.geode.test.dunit.rules.ClusterStartupRule.startServerVM(ClusterStartupRule.java:239)
> at
>
> org.apache.geode.test.dunit.rules.ClusterStartupRule.startServerVM(ClusterStartupRule.java:232)
> at
>
> org.apache.geode.test.dunit.rules.ClusterStartupRule.startServerVM(ClusterStartupRule.java:218)
> at
>
> org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest.serverRestartsAfterOneLocatorDies(ClusterConfigLocatorRestartDUnitTest.java:98)
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.1
> /Native
> Method)
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.1
> /NativeMethodAccessorImpl.java:62)
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.1
> /DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(java.base@11.0.1
> /Method.java:566)
> at
>
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
>
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
>
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
>
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
>
> org.apache.geode.test.junit.rules.DescribedExternalResource$1.evaluate(DescribedExternalResource.java:40)
> at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
>
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
>
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
> at
>
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
> at
>
> org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
> at
>
> org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66)
> at
>
> org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.jav

[DISCUSS] Static analysis of statics

[Dan Smith]

Hi devs,

We've expressed interest in getting rid of singletons and allowing multiple
copies of cache to run in the same JVM.

I'd like to get a handle on what static state we have. As a first step I'd
like to introduce a few annotations and some static analysis to find all of
our mutable static fields.

I intend to add a PMD rule to look for mutable static state, and some new
annotations to the fields we have right now to indicate what we want to do
with them.

More details are in this PR -  https://github.com/apache/geode/pull/3178

Are people ok with adding these additional annotations? You will see static
fields be marked with things like @MakeNotStatic or @MakeImmutable until we
get them all cleaned up.

-Dan

Re: Apache Geode PMC quarterly report: DRAFT for your review

[Dan Smith]

+1 - Looks good to me.

-Dan

On Tue, Feb 12, 2019 at 3:34 PM Dave Barnes  wrote:

> Please respond by noon tomorrow.
> Pretty complete, as far as I know, except for public events and
> presentations.
> Thanks,
> Dave
>
> Description:
>
> Apache Geode provides a database-like consistency model, reliable
> transaction processing and a shared-nothing architecture to maintain very
> low latency performance with high concurrency processing.
> Issues:
>
> There are no issues requiring board attention at this time.
> Activity:
>
>- We released v1.8.0, containing 124 improvements and new features,
>resolving 83 bugs and a total of 380 JIRA tickets
>- The v1.8.0 release includes the first release of the geode-native
>source, a C++/C# client API for Geode.
>- Security enhancements include support for Trust and Keystore rotation,
>endpoint validation during SSL handshake, and dynamic function security.
>
> Health report:
>
>- Mailing lists remain active and productive.
>- JIRA tickets show that issues continue to be identified and resolved.
>- We’re continuing to work on attracting new contributors and making it
>easier to participate in the community.
>
> PMC changes:
>
>- Currently 46 PMC members.
>- No new PMC members added in the last 3 months
>- Last PMC addition was Dick Cavender on Tue Feb 20 2018
>
> Committer base changes:
>
>- Currently 98 committers.
>- No new committers added in the last 3 months, but several invitations
>in the pipeline
>- Last committer addition was Robert Houghton at Mon Oct 01 2018
>
> Releases:
>
>- 1.8.0 was released on Wed Dec 12 2018
>
> Mailing list activity:
>
> Mailing lists have remained active and have maintained consistent usage
> levels.
>
>-
>
>dev@geode.apache.org:
>- 193 subscribers (up 3 in the last 3 months):
>   - 465 emails sent to list (825 in previous quarter)
>-
>
>iss...@geode.apache.org:
>- 55 subscribers (up 0 in the last 3 months):
>   - 3342 emails sent to list (4259 in previous quarter)
>-
>
>notificati...@geode.apache.org:
>- 7 subscribers (up 0 in the last 3 months):
>   - 2500 emails sent to list (1865 in previous quarter)
>-
>
>u...@geode.apache.org:
>- 250 subscribers (up 0 in the last 3 months):
>   - 94 emails sent to list (212 in previous quarter)
>
> JIRA activity:
>
>- 357 JIRA tickets created in the last 3 months
>- 314 JIRA tickets closed/resolved in the last 3 months
>

Re: [DISCUSS] Static analysis of statics

[Dan Smith]

Regarding @Immutable - yes it's intentionally a field annotation as well as
a class annotation. The reason to make it a field annotation is that the
static analysis tools aren't quite cool enough to figure out if a field is
really immutable so we have to manually tell the tool that the field is
immutable. For example "public static final List =
Collections.unmodifiableList(X)" might immutable but it's hard to deduce
that.

I agree that an @Immutable field should be final and the referent is
immutable.

Also agreed - @MakeImmutable should also apply to classes. I'll fix that.

I did an earlier pass where I made as many fields as I could final - so
most of the remaining cases would be your @MakeReferentImmutable case. I
can switch them to @MakeReferentImmutable if that makes more sense.

-Dan

On Tue, Feb 12, 2019 at 3:18 PM Bill Burcham  wrote:

> I think the @Immutable anno in *Java Concurrency and Practice* is a class
> annotation—not a field one.
>
> Looking at that PR, it looks like this @Immutable anno is usable both on a
> type (class) and on a field.
>
> Is that an oversight? If not, then what does it mean? Does @Immutable on a
> field mean both:
>
> • the field is final
> • the object referenced by the field is immutable
>
> ?
>
> Looks like in the current PR @MakeImmutable applies only to fields—not
> classes. I imagine that's an oversight.
>
> My quick thought is that these two annotations, in the spirit of *JCP*,
> should be type/class-level and not field-level,
>
> Immutable
> MakeImmutable
>
> and that perhaps we could have some field-level annos like:
>
> MakeNonStatic
> MakeFinal
> MakeReferentImmutable - change the type referenced by this field to be an
> immutable one
>
> In this way your MakeImmutable field anno is teased apart into two:
> MakeFinal, MakeReferentImmutable. The possible benefit is that the two
> annos can be used either separately or together to cover 3 situations
> instead of just one.
>

Re: Should Geode stats conform to backwards compatibility constraints?

[Dan Smith]

+1 to what Jake said about MemberMXBean - that is definitely a public API
class, so we need to deprecate the old method and introduce a new one.

I'm also not clear about the stats. Technically, they are discoverable
through the public API, so maybe? It seems like they are mix of things
users might want and stuff that is more internal/debugging oriented. If the
downcasting trick Jake mentioned isn't that hard, I'd say we should do
that. That said, I suspect we've renamed some stats in the past.

One other thing to consider - will changing the size of the stats affect
any tooling like VSD or gfsh or ?? that may have to deal with stat data
from multiple geode versions?

-Dan

On Wed, Feb 13, 2019 at 3:59 PM Kirk Lund  wrote:

> Quite a few Geode stats are currently defined as IntCounters which very
> easily wrap past Integer.MAX_VALUE. Some users wanted these stats to not
> wrap to negative MAX_VALUE, so my team defined the following two tickets
> and changed them to LongCounters on the develop branch:
>
> GEODE-6345: StatSamplerStats jvmPauses stat may wrap to negative value
> https://issues.apache.org/jira/browse/GEODE-6345
>
> GEODE-6334: CachePerfStats operation count stats may wrap to negative
> values
> https://issues.apache.org/jira/browse/GEODE-6334
>
> Some people are now concerned that this may break backwards compatibility
> and API for existing users.
>
> There are two potential ways for a user to *experience* this as an API
> change:
>
> 1) MemberMXBean in JMX
>
> *-  int getTotalNetSearchCompleted();*
> *+  long getTotalNetSearchCompleted();*
>
> As you can see in GEODE-6334, we changed quite a few stats from integer to
> long in CachePerfStats. The only one directly exposed as an attribute on
> MemberMXBean is TotalNetSearchCompleted.
>
> Users will find that this API method changed.
>
> 2) Statistics API with undocumented strings
>
> If we assume that users might know or discover that we have a statistics
> text id of "cachePerfStats" with an integer stat of name "puts" then they
> could use our Statistics API to get the value:
>
> * 1:CacheFactory cacheFactory = new CacheFactory();*
> * 2:Cache cache = cacheFactory.create();*
> * 3:*
> * 4:Region region = cache. String>createRegionFactory(PARTITION).create("region");*
> * 5:*
> * 6:Statistics[] statistics =
> cache.getDistributedSystem().findStatisticsByTextId("cachePerfStats");*
> * 7:int oldPuts = statistics[0].getInt("puts");*
> * 8:*
> * 9:region.put("some", "value");*
> *10:*
> *11:int newPuts = statistics[0].getInt("puts");*
> *12:*
> *13:assertThat(newPuts).isEqualTo(oldPuts + 1);*
>
> The above works in Geode 1.8 and earlier but will begin to throw the
> following in Geode 1.9 (when develop branch is released):
>
> *java.lang.IllegalArgumentException: The statistic puts with id 23 is of
> type long and it was expected to be an int.*
> * at
>
> org.apache.geode.internal.statistics.StatisticDescriptorImpl.checkInt(StatisticDescriptorImpl.java:324)*
> * at
>
> org.apache.geode.internal.statistics.StatisticsImpl.getIntId(StatisticsImpl.java:577)*
> * at
>
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:274)*
> * at
>
> org.apache.geode.internal.statistics.StatisticsImpl.getInt(StatisticsImpl.java:269)*
> * at com.user.example.Example.example(Example.java7)*
>
> In order to avoid the above exception, a user would have to change the code
> on lines 7 and 11 to use *getLong* instead of *getInt*.
>
> Should Geode stats be considered a form of API contract and should they
> conform to backwards compatibility constraints?
>
> Should we change these from LongCounters back to IntCounters?
>
> Someone did suggest that we could change them back to IntCounters and then
> change our statistics internals to skip to zero anytime a stat attempts to
> increment above MAX_VALUE, however, I think that if we decide that stats
> cannot be changed from IntCounters to LongCounters then we should not make
> this change either.
>
> Thanks for any input or opinions,
> Kirk
>

Re: Release Managers

[Dan Smith]

I added a 1.10.0 release to JIRA.

-Dan

On Wed, Feb 20, 2019 at 9:29 AM Sai Boorlagadda 
wrote:

> Do we need to create an infra ticket to create a new release?
> I don't seem to have permissions to create releases in Jira.
>
> On Wed, Feb 20, 2019 at 8:42 AM Jacob Barrett  wrote:
>
> > Release manager need to add the next version number to JIRA when they
> > branch the release. Changes going into develop need to be marked finished
> > with a version but may not be merged to the current release branch.
> >
> > Also, after release isn’t the release branch supposed to be deleted? So
> > what gives with release/1.7.0 and release/1.8.0?
> >
> > -Jake
> >
> >
>

Re: Merging GEODE-6424 into release/1.9.0

[Dan Smith]

+1

On Wed, Feb 20, 2019 at 12:24 PM Jacob Barrett  wrote:

> Anyone have issue with merging
> https://issues.apache.org/jira/browse/GEODE-6424 <
> https://issues.apache.org/jira/browse/GEODE-6424> into release/1.9.0?
>
> Without it we will have to wait for the next release before we can have
> meaningful baselines for function and query benchmarks. Without this fix
> baselines will continue to vary by as much as 45% making them useless.
>
> It’s also a big performance boost. Concurrent local cache gets see about a
> 50% bump in throughput due to reduced contention for stats, even with timed
> stats enabled. Other operations haven’t been benchmarked but should see
> similar improvements where stats were the bottleneck.
>
> -Jake
>
>

Re: next steps on release/1.9.0

[Dan Smith]

I removed 1.9.0 from GEODE-6816.

-Dan

On Wed, Feb 27, 2019 at 10:26 AM Sai Boorlagadda 
wrote:

> By end of this week, I am planning to create the first release candidate.
> Are there any other issues other than this last one?
>
> - GEODE-6359 - Bruce is looking into it.
> - geode-examples using https in build scripts.
>
> There are about 6[1] issues in JIRA that are in open/in-progress/re-opened
> status for 1.9.0.
> Can I request all the devs to reflect JIRA with current status?
>
> GEODE-6067 - add gfsh list data-source command - Darrel Schneider
> GEODE-5817 - CI Failure: StopServerAccepta... - Kenneth Howe
> GEODE-5816 - ClusterStartupRule fails to ...   - Dan Smith
> GEODE-5711 - create jndi-binding gfsh ... - Karen Smoler
> Miller
> GEODE-4794 - ConfigurePDXCommand    - Joey McAllister
> GEODE-2113 - Implement SSL over NIO - Joey McAllister
>
> [1]https://issues.apache.org
>
> /jira/browse/GEODE-6107?jql=project%20%3D%20GEODE%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.9.0
>

Re: Dependency review for release 1.9.0

[Dan Smith]

I see that geo, grumpy-core, and commons math came from adding geospatial
support to redis -
https://github.com/apache/geode/commit/7bf02251fd047cb1cf575c01b80a9807108618da

-Dan

On Thu, Feb 28, 2019 at 9:41 AM Anthony Baker  wrote:

> Looks a number of the new dependencies came in transitively with the guava
> version bump.
>
> > On Feb 27, 2019, at 5:32 PM, Anthony Baker  wrote:
> >
> > I was reviewing the release branch and noticed a number of new
> dependencies have been added since the last release.  When you add a new
> dependency, please review and follow the project license guide [1].  In
> particular, update the LICENSE file in geode-assembly/src/main/dist
> depending on the license type.
> >
> > Currently we need to update the LICENSE file with the additional
> MIT/BSD/CDDL dependencies.  We may also need to update NOTICE files.
> There’s also a version conflict with multiple versions of Jackson in use
> (2.9.6 / 2.9.8).
> >
> > @Sai - these need to be fixed on release/1.9.0
> >
> > Here’s the list of additions:
> >
> >   animal-sniffer-annotations-1.17.jar
> >   checker-qual-2.5.2.jar
> >   commons-math3-3.2.jar
> >   error_prone_annotations-2.2.0.jar
> >   failureaccess-1.0.jar
> >   geo-0.7.1.jar
> >   grumpy-core-0.2.2.jar
> >   istack-commons-runtime-2.2.jar
> >   j2objc-annotations-1.1.jar
> >   javax.activation-1.2.0.jar
> >   javax.activation-api-1.2.0.jar
> >   jsr305-3.0.2.jar
> >   listenablefuture-.0-empty-to-avoid-conflict-with-guava.jar
> >
> > Removed:
> >
> >   activation-1.1.1
> >   jaxb-core-2.2.11.jar
> >
> > Anthony
> >
> > [1]
> https://cwiki.apache.org/confluence/display/GEODE/License+Guide+for+Contributors
> <
> https://cwiki.apache.org/confluence/display/GEODE/License+Guide+for+Contributors
> >
> >
>
>

Re: Dependency review for release 1.9.0

[Dan Smith]

Currently, geode servers just have a flat classpath with all of the
dependencies of all of the modules. Having the ability to add optional
modules sounds like a good feature, though.

John - what you are describing applies to maven dependencies - and I do
think that we should isolate optional features into separate maven
dependencies like geode-lucene. But that doesn't help with servers launched
through gfsh start server unless we provide a way to configure which geode
modules are present on the server's classpath.

-Dan

On Thu, Feb 28, 2019 at 10:03 AM John Blum  wrote:

> Well, that just requires that you appropriately declare dependencies with
> the "optionality" and "scope" (e.g. "compile", "test", "provided", etc).
>
> Additionally, Geode modules could selectively pull in the required deps as
> needed.  For example, `geode-lucene` would only pull in the Apache Lucene
> dependencies if the `geode-lucene` module is used.
>
> No brainer.
>
> On Thu, Feb 28, 2019 at 9:47 AM Charlie Black  wrote:
>
> > Hopefully, we are thinking about classpath of the server and lazily
> adding
> > these jars only when a feature is turned on.
> >
> > On Thu, Feb 28, 2019 at 9:45 AM Dan Smith  wrote:
> >
> > > I see that geo, grumpy-core, and commons math came from adding
> geospatial
> > > support to redis -
> > >
> > >
> >
> https://github.com/apache/geode/commit/7bf02251fd047cb1cf575c01b80a9807108618da
> > >
> > > -Dan
> > >
> > > On Thu, Feb 28, 2019 at 9:41 AM Anthony Baker 
> wrote:
> > >
> > > > Looks a number of the new dependencies came in transitively with the
> > > guava
> > > > version bump.
> > > >
> > > > > On Feb 27, 2019, at 5:32 PM, Anthony Baker 
> > wrote:
> > > > >
> > > > > I was reviewing the release branch and noticed a number of new
> > > > dependencies have been added since the last release.  When you add a
> > new
> > > > dependency, please review and follow the project license guide [1].
> In
> > > > particular, update the LICENSE file in geode-assembly/src/main/dist
> > > > depending on the license type.
> > > > >
> > > > > Currently we need to update the LICENSE file with the additional
> > > > MIT/BSD/CDDL dependencies.  We may also need to update NOTICE files.
> > > > There’s also a version conflict with multiple versions of Jackson in
> > use
> > > > (2.9.6 / 2.9.8).
> > > > >
> > > > > @Sai - these need to be fixed on release/1.9.0
> > > > >
> > > > > Here’s the list of additions:
> > > > >
> > > > >   animal-sniffer-annotations-1.17.jar
> > > > >   checker-qual-2.5.2.jar
> > > > >   commons-math3-3.2.jar
> > > > >   error_prone_annotations-2.2.0.jar
> > > > >   failureaccess-1.0.jar
> > > > >   geo-0.7.1.jar
> > > > >   grumpy-core-0.2.2.jar
> > > > >   istack-commons-runtime-2.2.jar
> > > > >   j2objc-annotations-1.1.jar
> > > > >   javax.activation-1.2.0.jar
> > > > >   javax.activation-api-1.2.0.jar
> > > > >   jsr305-3.0.2.jar
> > > > >
>  listenablefuture-.0-empty-to-avoid-conflict-with-guava.jar
> > > > >
> > > > > Removed:
> > > > >
> > > > >   activation-1.1.1
> > > > >   jaxb-core-2.2.11.jar
> > > > >
> > > > > Anthony
> > > > >
> > > > > [1]
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GEODE/License+Guide+for+Contributors
> > > > <
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/GEODE/License+Guide+for+Contributors
> > > > >
> > > > >
> > > >
> > > >
> > >
> >
> >
> > --
> > Charlie Black | cbl...@pivotal.io
> >
>
>
> --
> -John
> john.blum10101 (skype)
>

Re: apply to get edit permission of wiki

[Dan Smith]

Sure! What is your username on the apache wiki?

-Dan

On Thu, Mar 7, 2019 at 12:19 PM Gang Yan  wrote:

> Hi Geode dev
>
> could you help to get edit permission of wiki page  :
>
> https://cwiki.apache.org/confluence/display/GEODE/Cluster+Management+Service
>
> thanks.
>
> Thanks and regards
>
> Gang Yan（闫钢）
> GemFire product management team, Beaverton
> g...@pivotal.io
>

Re: apply to get edit permission of wiki

[Dan Smith]

Hi Gang,

I've added you to the wiki. You may want to subscribe to the dev list
(email dev-subscr...@geode.apache.org).

Thanks!
-Dan

On Thu, Mar 7, 2019 at 2:12 PM Anthony Baker  wrote:

> What’s your username?
>
> > On Mar 7, 2019, at 12:15 PM, Gang Yan  wrote:
> >
> > Hi Geode dev
> >
> > could you help to get edit permission of wiki page  :
> >
> https://cwiki.apache.org/confluence/display/GEODE/Cluster+Management+Service
> >
> > thanks.
> >
> > Thanks and regards
> >
> > Gang Yan（闫钢）
> > GemFire product management team, Beaverton
> > g...@pivotal.io
>
>

Re: Apache Jira access

[Dan Smith]

Hi Matthew,

What's your JIRA username?

-Dan

On Fri, Mar 8, 2019 at 11:21 AM Matthew Reddington 
wrote:

> Hello,
>
> I am requesting access to Jira so that I may be assigned tickets.
>
> Matthew Reddington
>

Re: Request to assign tickets to myself

[Dan Smith]

Hi Michael,

What's your JIRA username?

-Dan

On Fri, Mar 8, 2019 at 11:28 AM Michael Martell  wrote:

> Hello,
>
> I can already create tickets.
> I am requesting access to Jira so that I may be assigned tickets.
>
> Thanks,
> Mike
>

[DISCUSS] Moving redis to a separate module

[Dan Smith]

I created a PR to move our redis support to a separate module. Let me know
what you think:

https://github.com/apache/geode/pull/3284

Geode servers will still include redis on the classpath, so the only effect
of this is that if you are launching a server based on the maven
dependencies, you will need geode-core and geode-redis to launch a server
with redis.

In addition to making it easier to find the redis specific code this also
removes 4 dependencies from geode-core.

-Dan

Re: [DISCUSS] Moving redis to a separate module

[Dan Smith]

Feedback seems to be positive, I'll go ahead and merge this. I'll create a
separate PR to split out memcached - I think that is a good idea.

Thanks!
-Dan

On Tue, Mar 12, 2019 at 5:56 PM Anilkumar Gingade 
wrote:

> +1
>
> On Tue, Mar 12, 2019 at 5:32 PM John Blum  wrote:
>
> > Definitely a reasonable change.  Perhaps, for consistency sake, the same
> > should be applied to Geode's Memcached support? (in another PR).
> >
> >
> > On Tue, Mar 12, 2019 at 4:23 PM Dan Smith  wrote:
> >
> > > I created a PR to move our redis support to a separate module. Let me
> > know
> > > what you think:
> > >
> > > https://github.com/apache/geode/pull/3284
> > >
> > > Geode servers will still include redis on the classpath, so the only
> > effect
> > > of this is that if you are launching a server based on the maven
> > > dependencies, you will need geode-core and geode-redis to launch a
> server
> > > with redis.
> > >
> > > In addition to making it easier to find the redis specific code this
> also
> > > removes 4 dependencies from geode-core.
> > >
> > > -Dan
> > >
> >
> >
> > --
> > -John
> > john.blum10101 (skype)
> >
>

[DISCUSS] Changing many geode-core dependencies from compile to runtime

[Dan Smith]

Hi,

We would like to start using gradle's new implementation dependency
notation in our build files.

This will affect downstream consumers of geode-core, hopefully in a good
way, in that many of our dependencies will now be marked runtime
dependencies in the pom instead of compile. That means it is easier for
users to avoid accidentally using one of these dependencies in their code.
But it also means if they are using one of these dependencies directly,
they will have to explicitly add it to their maven/gradle build going
forward.

Here is a PR for this:
https://github.com/apache/geode/pull/3314

Are there any concerns with making this switch?

These are the dependencies that are being flipped to runtime instead of
compile in the pom:

com.github.stephenc.findbugs:findbugs-annotations
org.jgroups:jgroups
antlr:antlr
com.fasterxml.jackson.core:jackson-annotations
com.fasterxml.jackson.core:jackson-databind
commons-io:commons-io
commons-validator:commons-validator
javax.xml.bind:jaxb-api
com.sun.xml.bind:jaxb-impl
org.apache.commons:commons-lang3
it.unimi.dsi:fastutil
net.java.dev.jna:jna
net.sf.jopt-simple:jopt-simple
org.apache.logging.log4j:log4j-api
org.apache.logging.log4j:log4j-core
org.eclipse.jetty:jetty-server
io.github.classgraph:classgraph
com.healthmarketscience.rmiio:rmiio

Re: [DISCUSS] removal of geode-json module

[Dan Smith]

Here's the original legal ticket -
https://issues.apache.org/jira/browse/LEGAL-349. It does seem kinda fuzzy.

What error are you getting if you remove geode-json? I don't see org.json
anywhere in the dependenies of geode-web-api:
./gradlew geode-web-api:dependencies

I also found this thing - which is the android version of org.json packaged
up:
https://mvnrepository.com/artifact/com.vaadin.external.google/android-json

On Fri, Mar 15, 2019 at 2:12 PM Bruce Schuchardt 
wrote:

> I've removed use of geode-json in non-test code and I'd like to remove
> it completely and just add a dependency on a org.json package in a Maven
> repository.  The only one available is org.json though, so here's the
> question: Is acceptable to use org.json with it's silly license (see
> below) if we're not including it in our distribution?
>
>
> Copyright (c) 2002 JSON.org
>
> Permission is hereby granted, free of charge, to any person
> obtaining a copy of this software and associated documentation files
> (the "Software"), to deal in the Software without restriction,
> including without limitation the rights to use, copy, modify, merge,
> publish, distribute, sublicense, and/or sell copies of the Software,
> and to permit persons to whom the Software is furnished to do so,
> subject to the following conditions:
>
> The above copyright notice and this permission notice shall be
> included in all copies or substantial portions of the Software.
>
> _*The Software shall be used for Good, not Evil.*_
>
> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
> NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
> BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
> ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
> CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
> SOFTWARE.
>
> Note: We can't use openjson, which is what geode-json is based on,
> because it's packaged as com.github.openjson instead of org.json.
>
>

Re: [DISCUSS] removal of geode-json module

[Dan Smith]

Huh, it looks like org.skyscreamer.jsonassert.JSONAssert is actually
dependent that vaadin artifact anyway, but we *excluded* it our geode-junit
build file! That's why you are getting a runtime error. Maybe just remove
the exclusion.

>From geode-junit:

compile('org.skyscreamer:jsonassert') {
  exclude module: 'android-json'
}


On Mon, Mar 18, 2019 at 11:31 AM Bruce Schuchardt 
wrote:

> I don't want to spend a lot more time on the removal of the geode-json
> module.  I'm going to delete it and leave the geode-web-api test
> dependent on the vaadin implementation of org.json.
>
> I still haven't been able to run the geode-pulse uiTests.  I'm going to
> leave that module dependent on the vaadin implementation as well since,
> again, it's only using org.json for tests.
>
>

Re: [DISCUSS] Proposal to re-cut Geode 1.9.0 release branch

[Dan Smith]

Is the geode-managability sub-project and the new micrometer API in a place
where we can cut a release branch? I know a bunch of changes have gone in
since the release branch, are we comfortable releasing these new
experimental features as they are right now?

-Dan

On Tue, Mar 19, 2019 at 2:38 PM Dick Cavender  wrote:

> +1 to re-cutting the 1.9 release branch off a more stable develop sha
> within the last couple days.
>
> On Tue, Mar 19, 2019 at 1:14 PM Bruce Schuchardt 
> wrote:
>
> > If we recut the release branch we need to update JIRA tickets marked
> > fixed in 1.10
> >
> > On 3/19/19 12:48 PM, Sai Boorlagadda wrote:
> > > > It was known at the time that develop was not as stable as desired,
> > > so we planned to cherry-pick fixes from develop until the release
> > > branch was stable enough to ship.
> > > I want to clarify that we decided to cut the release branch not that
> > > develop was not stable. But really that it is desirable to cut the
> > > branch sooner to avoid any regression risk that can be introduced by
> > > on-going work on develop.
> > >
> > > Nevertheless looks like develop is more stable than release branch due
> > > to some test fixes that were not cherry-picked into the release branch.
> > > I think its a good idea to re-cut the branch as our current position
> > > to stabilize release branch before releasing.
> > >
> > > +1 to re-cut.
> > >
> > > Sai
> > >
> > > On Tue, Mar 19, 2019 at 12:19 PM Owen Nichols  > > > wrote:
> > >
> > > The Geode 1.9.0 release branch was originally cut 4 weeks ago on
> > > Feb 19.  It was known at the time that develop was not as stable
> > > as desired, so we planned to cherry-pick fixes from develop until
> > > the release branch was stable enough to ship.  While this is a
> > > good strategy when starting from a fairly good baseline, it seems
> > > in this case it has only added complexity without leading to
> > > stability.
> > >
> > > Looking at the pipelines over the last week (see attached
> > > metrics), it appears we have been far more successful at
> > > stabilizing /develop/ than /release/1.9.0/. Rather than trying to
> > > cherry-pick more and more fixes to the release branch, I propose
> > > we RE-CUT the 1.9.0 release branch later this week in order to
> > > start from a much more stable baseline.
> > >
> > > -Owen
> > >
> > >
> > >
> >
>

Re: [DISCUSS] TTL setting on WAN

[Dan Smith]

Sounds like a good feature. I'm interested to see what ordering guarantees
we want to implement - if we can expire things in the order they were added
to the queue that seems like a good way to go.

As Anil pointed out - do you really want to let the user pass in an
ExpirationAttributes object? That allows the user to control the
ExpirationAction (destroy, invalidate, etc.). I don't think the
ExpirationAction makes sense for a GatewaySender?

-Dan

On Wed, Mar 20, 2019 at 10:26 AM Anilkumar Gingade 
wrote:

> +1. Will the expiration (destroy) be applied on local queues or the
> expiration will be replicated (for both serial and parallel)?
>
> -Anil.
>
>
> On Wed, Mar 20, 2019 at 8:59 AM Bruce Schuchardt 
> wrote:
>
> > We've seen situations where the receiving side of a WAN gateway is slow
> > to accept data or is not accepting any data.  This can cause queues to
> > fill up on the sending side.  If disk-overflow is being used this can
> > even lead to an outage.  Some users are concerned more with the latest
> > data and don't really care if old data is thrown away in this
> > situation.  They may have set a TTL on their Regions and would like to
> > be able to do the same thing with their GatewaySenders.
> >
> > With that in mind I'd like to add this method to GatewaySenderFactory:
> >
> > /** * Sets the timeToLive expiration attribute for queue entries for the
> > next * {@code GatewaySender} created. * * @param timeToLive the
> > timeToLive ExpirationAttributes for entries in this region * @return a
> > reference to this GatewaySenderFactory object * @throws
> > IllegalArgumentException if timeToLive is null * @see
> > RegionFactory#setEntryTimeToLive */ public GatewaySenderFactory
> > setEntryTimeToLive(ExpirationAttributes timeToLive);
> >
> > The exact implementation may not be the same as for Regions since we
> > probably want to expire the oldest entries first and make sure we do so
> > in their order in the queue.
> >
> >
>

Re: [DISCUSS] TTL setting on WAN

[Dan Smith]

> 2) The developer wants to replicate _state_.  This means that implicit
> state changes (expiration or eviction w/ destroy) could allow us to
> optimize the queue size.  This is very similar to conflation, just a
> different kind of optimization.
>
> For this second case, does it make sense to allow the user to specify a
> different TTL than the underlying region?  It seems like what the user
> wants is to not replicate stale data and having an extra TTL attribute
> would just be another value to mis-configure.  What do you think about just
> providing a boolean flag?
>
>
This kinda jogged my memory. AsyncEventQueues actually *do* have a boolean
flag to allow you to forward expiration events to the queue. I have no idea
how this interacts with conflation though -
https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventQueueFactory.html#setForwardExpirationDestroy-boolean-

Re: [DISCUSS] Proposal to re-cut Geode 1.9.0 release branch

[Dan Smith]

I'm fine releasing with just the service loader API for micrometer.

I'd just like the folks working on these new public APIs agree that they
are ok with these APIs getting released to end users and put to use in the
state they are right now.

-Dan

On Wed, Mar 20, 2019 at 12:43 PM Dale Emery  wrote:

> > On Mar 20, 2019, at 11:25 AM, Alexander Murmann 
> wrote:
> >
> > Dale, is there any downside to making these changes in 1.10 other than
> > plainly having them later?
>
> I don’t think so, but I’d want to hear Dan’s opinion on that, given that
> his approval of our PR was contingent on our promise to do it.
>
> FYI, the additional work to improve usability is non-trivial, which is why
> we haven’t done it already.
>
> —
> Dale Emery
> dem...@pivotal.io
>
>
>
> > On Mar 20, 2019, at 11:25 AM, Alexander Murmann 
> wrote:
> >
> > Dale, is there any downside to making these changes in 1.10 other than
> > plainly having them later?
> >
> > On Wed, Mar 20, 2019 at 11:15 AM Dave Barnes  wrote:
> >
> >> geode-native is ready to into the 1.9 release candidate build.
> >>
> >> On Wed, Mar 20, 2019 at 10:42 AM Dave Barnes 
> wrote:
> >>
> >>> The geode-native PR will be ready to check in momentarily. Just waiting
> >>> for Travis to do its diligence.
> >>>
> >>> On Wed, Mar 20, 2019 at 9:47 AM Alexander Murmann  >
> >>> wrote:
> >>>
> >>>> Dale, do I understand correctly that the only concern around the
> >>>> Micrometer
> >>>> work right now it that it's not useful yet, however it's not harmful
> >>>> either?
> >>>>
> >>>> Dave, is it correct that if that PR doesn't make it into the newly cut
> >>>> branch, we'd be shipping with a older version of geode-native? What
> are
> >>>> the
> >>>> two versions and what would be the implications of this not making it
> >> into
> >>>> this release?
> >>>>
> >>>> On Tue, Mar 19, 2019 at 5:29 PM Dave Barnes 
> wrote:
> >>>>
> >>>>> The Geode 1.9.0 release includes a source-only release of the
> >>>> geode-native
> >>>>> repo. There's a pull-request in process to update version numbers and
> >>>> the
> >>>>> doc build environment in that repo; should be ready to merge tomorrow
> >>>>> morning.
> >>>>>
> >>>>> On Tue, Mar 19, 2019 at 5:20 PM Dale Emery 
> wrote:
> >>>>>
> >>>>>> The Micrometer API is in, and marked as experimental. But we have
> >> not
> >>>> yet
> >>>>>> updated CacheFactory to allow injecting a meter registry (or metrics
> >>>>>> publishing service) there. So currently the only way to publish is
> >> to
> >>>> add
> >>>>>> metrics publishing service via the ServiceLoader mechanism.
> >>>>>>
> >>>>>> —
> >>>>>> Dale Emery
> >>>>>> dem...@pivotal.io
> >>>>>>
> >>>>>>
> >>>>>>> On Mar 19, 2019, at 3:29 PM, Dan Smith  wrote:
> >>>>>>>
> >>>>>>> Is the geode-managability sub-project and the new micrometer API
> >> in
> >>>> a
> >>>>>> place
> >>>>>>> where we can cut a release branch? I know a bunch of changes have
> >>>> gone
> >>>>> in
> >>>>>>> since the release branch, are we comfortable releasing these new
> >>>>>>> experimental features as they are right now?
> >>>>>>>
> >>>>>>> -Dan
> >>>>>>>
> >>>>>>> On Tue, Mar 19, 2019 at 2:38 PM Dick Cavender 
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> +1 to re-cutting the 1.9 release branch off a more stable develop
> >>>> sha
> >>>>>>>> within the last couple days.
> >>>>>>>>
> >>>>>>>> On Tue, Mar 19, 2019 at 1:14 PM Bruce Schuchardt <
> >>>>>> bschucha...@pivotal.io>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> If we recut the release branch we need to update JIRA 

Re: [DISCUSS] TTL setting on WAN

[Dan Smith]

Following up on the conflation thing - Anil and I did an experiment.
Conflation definitely *does* happen on everything in the queue, not just
the last batch. But we didn't see destroys get conflated with updates.

So one thing that might make this use case work is to conflate the destroys
with the updates. Then the disk space would be freed up when the expiration
events are conflated in the queue.

-Dan

On Tue, Mar 26, 2019 at 8:19 AM Bruce Schuchardt 
wrote:

> I've been thinking along those lines as well Suranjan.  Since conflation
> and expiry-forwarding don't solve the problem of running out of disk
> space the solution needs to involve the dispatch thread.
>
> For the session-state caching scenario that raised this whole issue I
> think what you've described will work.  Looking at it with a wider lens
> I'm a little concerned about a TTL on the queue because multiple regions
> can feed into the same queue and you might not have the same TTL
> settings on all of those regions.
>
> On 3/25/19 4:53 PM, Suranjan Kumar wrote:
> > Hi,
> >   I think the one approach for a user would be to 'filter' the events
> while
> > dispatching. If I remember correctly, we can attach a filter at dispatch
> > time and filter the events based on creationTime of the GatewayEvent. We
> > can provide a pre created filter and use it based on some so that user
> > doesn't have to write his/her own.
> >
> > Something like,
> > /**
> > All the events which spend timeToLive or more time in queue will be
> deleted
> > from the queue
> > and will not be sent to remote site.
> > Possible consequence is that two sites can be inconsistent in case
> > */
> > public GatewaySenderFactory setEntryTimeToLive(long timeToLive);
> >
> > As queues will be read in LRU way this would be faster too. Only drawback
> > is that there will be only one thread (not sure if we have concurrent
> > dispatcher yet) clearing the queue.
> >
> > As Udo/Dan mentioned above, user needs to be aware of the consequences.
> >
> >
> > On Tue, Mar 26, 2019 at 3:09 AM Bruce Schuchardt  >
> > wrote:
> >
> >> I've walked through the code to forward expiration actions to async
> >> event listeners & don't see how to apply it to removal of queue entries
> >> for WAN.  The current implementation just queues the expiration
> >> actions.  If we wanted to remove any queued events associated with the
> >> expired region entry we'd have to scan the whole queue, which would be
> >> too slow if we're overflowing the queue to disk.
> >>
> >> I've also walked through the conflation code.  It applies only to the
> >> current batch being processed by the gateway sender.  The data structure
> >> used to perform conflation is just a Map that is created in the sender's
> >> batch processing method and then thrown away.
> >>
> >> On 3/20/19 11:15 AM, Dan Smith wrote:
> >>>> 2) The developer wants to replicate _state_.  This means that implicit
> >>>> state changes (expiration or eviction w/ destroy) could allow us to
> >>>> optimize the queue size.  This is very similar to conflation, just a
> >>>> different kind of optimization.
> >>>>
> >>>> For this second case, does it make sense to allow the user to specify
> a
> >>>> different TTL than the underlying region?  It seems like what the user
> >>>> wants is to not replicate stale data and having an extra TTL attribute
> >>>> would just be another value to mis-configure.  What do you think about
> >> just
> >>>> providing a boolean flag?
> >>>>
> >>>>
> >>> This kinda jogged my memory. AsyncEventQueues actually *do* have a
> >> boolean
> >>> flag to allow you to forward expiration events to the queue. I have no
> >> idea
> >>> how this interacts with conflation though -
> >>>
> >>
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cache/asyncqueue/AsyncEventQueueFactory.html#setForwardExpirationDestroy-boolean-
>

Re: Weird intermittent build error caused by spotlessJava

[Dan Smith]

I have had several issues running clean in combination with other gradle
commands - I think because of our parallel build. Maybe do this?

$ ./gradlew clean; ./gradlew build -xtest

On Tue, Mar 26, 2019 at 10:43 AM Kirk Lund  wrote:

> Intermittent build error caused by spotlessJava. My checkout of Geode
> intermittently gets into a state that then fails to build. I've been seeing
> this come and go for the last month or two. I'm not sure what puts it into
> this state but executing:
>
> $ ./gradlew clean build -x test
>
> ...will repeatedly fail in junit:spotlessJava with a
> java.lang.reflect.InvocationTargetException and a root cause of
> ava.lang.NoClassDefFoundError:
> org/openjdk/tools/javac/comp/InferenceContext.
>
> Stopping the gradle daemon(s) does not clear the problem up. Cleaning does
> not clear it up. Changing JDK version does not clear it up. But I just
> discovered this morning that if I attempt to execute spotless directly that
> finally clears it up:
>
> $ ./gradlew spotlessApply
> or
> $ ./gradlew spotlessCheck
>
> After either spotlessApply or spotlessCheck, the build will stop failing
> when I execute:
>
> $ ./gradlew clean build -x test
>
> The stack thrown by spotlessJava and printed by gradle when it's stuck in
> this state is below...
>
> > Task :geode-junit:spotlessJava FAILED
> Step 'removeUnusedImports' found problem in
>
> 'geode-junit/src/main/java/org/apache/geode/test/junit/rules/gfsh/DebuggableCommand.java':
> null
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
>
> com.diffplug.spotless.java.GoogleJavaFormatStep$State.lambda$createRemoveUnusedImportsOnly$1(GoogleJavaFormatStep.java:153)
> at
>
> com.diffplug.spotless.FormatterStepImpl$Standard.format(FormatterStepImpl.java:78)
> at
> com.diffplug.spotless.FormatterStep$Strict.format(FormatterStep.java:76)
> at com.diffplug.spotless.Formatter.compute(Formatter.java:230)
> at com.diffplug.spotless.Formatter.isClean(Formatter.java:167)
> at
> com.diffplug.gradle.spotless.SpotlessTask.check(SpotlessTask.java:263)
> at
>
> com.diffplug.gradle.spotless.SpotlessTask.performAction(SpotlessTask.java:205)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:103)
> at
>
> org.gradle.api.internal.project.taskfactory.IncrementalTaskAction.doExecute(IncrementalTaskAction.java:73)
> at
>
> org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:41)
> at
>
> org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:28)
> at
>
> org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$4.run(ExecuteActionsTaskExecuter.java:338)
> at
>
> org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:402)
> at
>
> org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:394)
> at
>
> org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
> at
>
> org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
> at
>
> org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
> at
>
> org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:92)
> at
>
> org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
> at
>
> org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:327)
> at
>
> org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:312)
> at
>
> org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.access$200(ExecuteActionsTaskExecuter.java:75)
> at
>
> org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.execute(ExecuteActionsTaskExecuter.java:158)
> at
>
> org.gradle.internal.execution.impl.steps.ExecuteStep.execute(ExecuteStep.java:46)
> 

Re: need edit access to ciwiki

[Dan Smith]

Hi Bill,

You should have access now.

-Dan

On Tue, Mar 26, 2019 at 3:06 PM Bill Burcham  wrote:

> I need edit access to https://cwiki.apache.org for:
>
> bill.burc...@gmail.com
>
> The immediate need is to update some UML here:
>
> https://cwiki.apache.org/confluence/display/GEODE/HA+Client+Event+Queues
>
> Thanks!
>

Re: Request access to JIRA

[Dan Smith]

Hi Mario,

You should have access to assign geode tickets now.

Thanks!
-Dan

On Fri, Mar 29, 2019 at 1:29 AM Mario Ivanac  wrote:

> Hi Geode Dev,
>
>
> Can you give me access to JIRA, so I can assign to tickets.
>
> My JIRA username is "mivanac".
>
>
> Thanks,
>
> Mario
>

Re: Request access to Jira

[Dan Smith]

Hi Mario,

You should have access to assign geode tickets now.

Thanks!
-Dan

On Fri, Mar 29, 2019 at 1:12 AM Mario Kevo  wrote:

> Hi all,
>
> Can you give me access to Jira so I can assigne on tickets?
> My Jira Username is 'mkevo'.
>
> Thanks and BR,
> Mario
>

[DISCUSS] Move or remove org.apache.geode.admin

[Dan Smith]

Hi devs,

The org.apache.geode.admin package has been deprecated for about 7 years
[1].

I'd like to remove it, or at least move it to a separate module. I actually
started some work to move it to a separate module [2]. However in the
course of this process, I've found that this module has extremely little
test coverage, so I'm not confident in the move. For example, it has a
whole JMX manager feature (different than our current JMX manager) which
does not appear to have any tests. This JMX manager feature won't work as
shipped unless you find and add some mx4j jars to your classpath.

I think the best course of action is probably to remove it entirely.
However, this does bring up a couple of questions:

1) Is it ok in general to remove deprecated API in a non X.0 release (eg
geode 1.10 instead of geode 2.0)?

2) How about in this case, when this feature has been deprecated for so
long, and may or may not work?

-Dan

[1]
https://geode.apache.org/releases/latest/javadoc/org/apache/geode/admin/package-summary.html
[2] Draft PR for moving the org.apache.geode.admin to a separate module -
https://github.com/apache/geode/pull/3392

Re: [DISCUSS] Move or remove org.apache.geode.admin

[Dan Smith]

> But this "hidden" feature, what is that? Is this something that we would
> like to support or is this something that would be replaced with our
> existing efforts in the ConfigurationService and Metrics area.
>

The old JMX agent [1] was superseded by the newer JMX manager [2] sometime
before geode 1.0 was ever shipped. The agent script mentioned in the
javadocs for the old jmx agent was never shipped with geode as far as I can
tell, but you can launch an agent programmatically. Also the link to the
descriptions of the mbeans is broken, but you can get an idea of what
mbeans it had by looking the mbeans-descriptors.xml file in the source.

[1]
https://geode.apache.org/releases/latest/javadoc/org/apache/geode/admin/jmx/Agent.html
[2]
https://geode.apache.org/docs/guide/17/managing/management/list_of_mbeans.html

Re: [Geode Build] Are the idea and eclipse plugins being used by anyone?

[Dan Smith]

I was under the impression that intellij's import actually used some of the
information in the idea{} blocks in the build. See
https://docs.gradle.org/current/userguide/idea_plugin.html.

But if the import works well without that extra configuration, I think we
should get rid of it.

-Dan

On Wed, Apr 3, 2019 at 2:32 PM Michael Oleske  wrote:

> I tend to import into Intellij the way you have described.  I've never
> used ./gradlew
> idea to configure.
>
> -michael
>
> On Wed, Apr 3, 2019 at 2:24 PM Patrick Rhomberg 
> wrote:
>
> > I'm eyeballing the ide.gradle file we have and am wondering if it is
> cruft
> > from many iterations of both Gradle and IntelliJ/Eclipse ago.
> >
> > In my own IntelliJ workflow, I have always just asked IntelliJ to open
> the
> > gradle project via the build.gradle itself, and it's easy to import other
> > modules into my workspace via Import Module from Existing Sources...
> I've
> > never had to deal with running ./gradlew idea to configure a project IML
> > which I needed to merge against another project file.
> >
> > Is this the workflow others are using?  Or is the idea plugin here a
> > hanger-on that should be removed as no longer necessary?
> >
> > Also, as an IntelliJ user, I have zero experience on how the workflow for
> > Eclipse is similar or different.  The closures in our ide.gradle appear
> to
> > be similar, but I don't have the Eclipse-side context to confirm or deny
> > that.
> >
> > Thanks in advance for any additional insight.
> >
> > Imagination is Change.
> > ~Patrick
> >
>

Re: [Geode Build] Are the idea and eclipse plugins being used by anyone?

[Dan Smith]

+1 to removing the idea plugin and the eclipse plugin as well. There is an
eclipse plugin <https://projects.eclipse.org/projects/tools.buildship> that
lets you import gradle projects similar to the intellij one. I think that
is what our eclipse developers are using now.

I still think the intellij import interacts with the idea plugin in some
way, but maybe we no longer need to configure the import. This is what the
gradle docs say about the intellij import:

If you simply want to load a Gradle project into IntelliJ IDEA, then use
the IDE’s import facility
<https://www.jetbrains.com/help/idea/gradle.html#gradle_import>.
You do not need to apply this plugin to import your project into IDEA,
although if you do,
the import will take account of any extra IDEA configuration you have that
doesn’t directly modify
the generated files — see the Configuration
<https://docs.gradle.org/current/userguide/idea_plugin.html#sec:idea_configuration>
section for more details.


-Dan

On Fri, Apr 5, 2019 at 9:23 AM Robert Houghton  wrote:

> @upthewaterspout The plugin generates .iml, .ipr files used by old versions
> of IntelliJ. The plugin does not give a way to create or modify the
> currently-used XML files that live inside of the /.idea directory.
>
> +1 to removal of the idea plugin.
>
> On Wed, Apr 3, 2019 at 3:22 PM Dan Smith  wrote:
>
> > I was under the impression that intellij's import actually used some of
> the
> > information in the idea{} blocks in the build. See
> > https://docs.gradle.org/current/userguide/idea_plugin.html.
> >
> > But if the import works well without that extra configuration, I think we
> > should get rid of it.
> >
> > -Dan
> >
> > On Wed, Apr 3, 2019 at 2:32 PM Michael Oleske 
> wrote:
> >
> > > I tend to import into Intellij the way you have described.  I've never
> > > used ./gradlew
> > > idea to configure.
> > >
> > > -michael
> > >
> > > On Wed, Apr 3, 2019 at 2:24 PM Patrick Rhomberg 
> > > wrote:
> > >
> > > > I'm eyeballing the ide.gradle file we have and am wondering if it is
> > > cruft
> > > > from many iterations of both Gradle and IntelliJ/Eclipse ago.
> > > >
> > > > In my own IntelliJ workflow, I have always just asked IntelliJ to
> open
> > > the
> > > > gradle project via the build.gradle itself, and it's easy to import
> > other
> > > > modules into my workspace via Import Module from Existing Sources...
> > > I've
> > > > never had to deal with running ./gradlew idea to configure a project
> > IML
> > > > which I needed to merge against another project file.
> > > >
> > > > Is this the workflow others are using?  Or is the idea plugin here a
> > > > hanger-on that should be removed as no longer necessary?
> > > >
> > > > Also, as an IntelliJ user, I have zero experience on how the workflow
> > for
> > > > Eclipse is similar or different.  The closures in our ide.gradle
> appear
> > > to
> > > > be similar, but I don't have the Eclipse-side context to confirm or
> > deny
> > > > that.
> > > >
> > > > Thanks in advance for any additional insight.
> > > >
> > > > Imagination is Change.
> > > > ~Patrick
> > > >
> > >
> >
>

Re: Fixing Awaitility await().untilAsserted(new WaitCriterion

[Dan Smith]

WaitCriterion extends ThrowingRunnable. So this pattern should still work
as WaitCriterion did before. But just using an assertion inside of a lambda
is the better option.

-Dan

On Thu, Apr 11, 2019 at 12:02 PM Kirk Lund  wrote:

> Just a quick heads up... I'm seeing an Awaitility usage pattern that is
> broken and does nothing. Specifically, it's any uses of dunit WaitCriterion
> with untilAsserted:
>
>   GeodeAwaitility.*await().untilAsserted(new WaitCriterion*() {
>
> @Override
> public boolean done() {
>   return region.isDestroyed();
> }
>
> @Override
> public String description() {
>   return "Region was not destroyed : " + region.isDestroyed();
> }
>   });
>
> The above is broken. It will not await anything. Invoking "new
> WaitCriterion()" will not throw anything so untilAsserted returns
> immediately. This may cause some flaky tests to be flakier because we're no
> longer waiting for the WaitCriterion done to be true.
>
> To fix this, just change it to use await().until with a lambda containing
> the contents of done():
>
>   GeodeAwaitility.*await().until(() -> *region.isDestroyed());
>

Re: JIRA and Wiki permissions

[Dan Smith]

Hi Alberto,

You should have access to both now.

Thanks!
-Dan

On Thu, Apr 11, 2019 at 7:53 AM Alberto Gomez 
wrote:

> Hi,
>
>
> Could you please grant me permissions to edit the wiki and assign JIRA
> tickets?
>
>
> My id is alberto.gomez
>
>
> Thanks!
>
>
> Alberto
>
>
>

Re: [VOTE] Apache Geode 1.9.0 RC2

[Dan Smith]

Hi Sai,

It looks like you left some signing properties in gradle.properties in the
source distribution. Building the source distribution is failing with this
error to this. Maybe you could make a clean source distribution and
re-upload it?

1: Task failed with an exception.
---
* What went wrong:
Execution failed for task ':geode-assembly:signDistTar'.
> Cannot perform signing task ':geode-assembly:signDistTar' because it has
no configured signatory

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or
--debug option to get more log output. Run with --scan to get full insights.
==

-Dan

On Fri, Apr 12, 2019 at 4:34 PM Sai Boorlagadda 
wrote:

> Hello, Geode dev community,
>
>
> This is the second release candidate for Apache Geode, version 1.9.0.
>
> Thanks to all the community members for their contributions to this
> release!
>
>
> Please do a review and give your feedback. The deadline is before 5 PM PST
> Wed April 17th, 2019.
>
>
> This release resolves 295 issues on Apache Geode JIRA system. Release notes
> can be found at:
>
>
> https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.9.0
>
>
> Please note that we are voting upon the source tags: rel/v1.9.0.RC2
>
>
> Apache Geode:
>
> https://github.com/apache/geode/tree/rel/v1.9.0.RC2
>
> Apache Geode examples:
>
> https://github.com/apache/geode-examples/tree/rel/v1.9.0.RC2
>
> Apache Geode Native:
>
> https://github.com/apache/geode-native/tree/rel/v1.9.0.RC2
>
>
> Commit ID:
>
> Apache Geode: 4a868075e59a26753485958bf86900881b794934
>
> Apache Geode Examples: 461f03614d389325496a82874cd5cf8b3329afdf
>
> Apache Geode Native: cef8078392f51d9a2d4b6a8b828a17359004b279
>
>
> Source and binary files:
>
> https://dist.apache.org/repos/dist/dev/geode/1.9.0.RC2/
>
>
> Maven staging repo:
>
> https://repository.apache.org/content/repositories/orgapachegeode-1052
>
>
> Geode's KEYS file containing PGP keys we use to sign the release:
>
> https://github.com/apache/geode/blob/develop/KEYS
>
>
> Signed the release with fingerprint:
>
> rsa4096 2018-01-14 [SC]
>
> 0889380187C20785C2A7168045936212F0FF55B0
>
>
> PS: Command to run geode-examples: ./gradlew -PgeodeReleaseUrl=
> https://dist.apache.org/repos/dist/dev/geode/1.9.0.RC2
> -PgeodeRepositoryUrl=
> https://repository.apache.org/content/repositories/orgapachegeode-1052
> build
> runAll
>
>
> Regards,
>
> Sai Boorlagadda
>

Re: [VOTE] Apache Geode 1.9.0 RC3

[Dan Smith]

+1

The java bits look good to me. I didn't look too much at geode-native -
Maybe someone who knows more about the geode-native codebase can take a
look at part?

Thanks,
-Dan

On Mon, Apr 15, 2019 at 3:06 PM Sai Boorlagadda 
wrote:

> Hello, Geode dev community,
>
>
> This is the second release candidate for Apache Geode, version 1.9.0.
>
> Thanks to all the community members for their contributions to this
> release!
>
>
> Please do a review and give your feedback. The deadline is before 3 PM PST
> Thur April 18th, 2019.
>
>
> This release resolves 296 issues on Apache Geode JIRA system. Release notes
> can be found at:
>
>
> https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.9.0
>
>
> Please note that we are voting upon the source tags: rel/v1.9.0.RC3
>
>
> Apache Geode:
>
> https://github.com/apache/geode/tree/rel/v1.9.0.RC3
>
> Apache Geode examples:
>
> https://github.com/apache/geode-examples/tree/rel/v1.9.0.RC3
>
> Apache Geode Native:
>
> https://github.com/apache/geode-native/tree/rel/v1.9.0.RC3
>
>
> Commit ID:
>
> Apache Geode: 6b05caec551fb04af18b3d6579d443fcb81aa1a0
>
> Apache Geode Examples: 461f03614d389325496a82874cd5cf8b3329afdf
>
> Apache Geode Native: cef8078392f51d9a2d4b6a8b828a17359004b279
>
>
> Source and binary files:
>
> https://dist.apache.org/repos/dist/dev/geode/1.9.0.RC3/
>
>
> Maven staging repo:
>
> https://repository.apache.org/content/repositories/orgapachegeode-1053
>
>
> Geode's KEYS file containing PGP keys we use to sign the release:
>
> https://github.com/apache/geode/blob/develop/KEYS
>
>
> Signed the release with fingerprint:
>
> rsa4096 2018-01-14 [SC]
>
> 0889380187C20785C2A7168045936212F0FF55B0
>
>
> PS: Command to run geode-examples: ./gradlew -PgeodeReleaseUrl=
> https://dist.apache.org/repos/dist/dev/geode/1.9.0.RC3
> -PgeodeRepositoryUrl=
> https://repository.apache.org/content/repositories/orgapachegeode-1053
> build
> runAll
>
>
> PS: I have cherry-picked a commit to fix GEODE-3948
>
>
> Regards,
>
> Sai Boorlagadda
>

Re: How to publish client stats on server

[Dan Smith]

As Anthony pointed out, each member (client, server, locator, etc.) writes
it's own statistics to it's own stats file.

There is are some server side stats that might have the information you are
looking for - see CacheServerStats. That has information about how many
operations the server has received, etc.

-Dan

On Tue, Apr 16, 2019 at 8:36 AM Anthony Baker  wrote:

> The client stats are written to a file on the client. They don’t get
> published to the server.
>
> Anthony
>
> > On Apr 16, 2019, at 6:19 AM, Alberto Bustamante Reyes
>  wrote:
> >
> > Hi Geode community,
> >
> > Im trying to run a simple test to check how the client stats are
> published on the server, but I have not been able to do it.
> >
> > The server is started with the statistic sampling enabled, and in the
> client I set the sending interval with setPoolStatisticInterval, but when I
> open the stats file with VSD, I cannot see any "ClientStat" there. I was
> expecting to see the stats "Client-to-Server Messaging Performance
> (ClientStats)".
> >
> > I have checked the code and these two actions (setting time interval and
> stats sampling) are the two conditions that enable the publishing of the
> client stats, if Im not wrong.
> >
> > What am I missing? Thanks in advance.
> >
> > BR/
> >
> > Alberto
>

Re: defunct branches

[Dan Smith]

You just need to do git remote prune origin. Git doesn't remove remote
branches from your local copy automatically.

-Dan

On Thu, Apr 18, 2019 at 8:08 AM Bruce Schuchardt 
wrote:

> Sorry to spam everyone.  "git branch -r" seems to be a local thing.  I
> made a fresh clone of the apache repo and now only see the branches
> mentioned in the UI.
>
> On 4/17/19 5:06 PM, Jason Huynh wrote:
> > Hi Bruce,
> >
> > I am unable to see the same branches on geode repo.  I do see these
> > branches on my personal fork but that's because I haven't updated my own
> > personal fork in some time...
> >
> > Is there a chance that your origin is pointing to your personal fork and
> > not the Apache Geode Repo?
> >
> > I am also unable to see these branches through the ui:
> > https://github.com/apache/geode/branches/all
> >
> >
> >
> > On Wed, Apr 17, 2019 at 4:17 PM Bruce Schuchardt  >
> > wrote:
> >
> >> We have nearly 400 branches in the repo right now.  Most of them are for
> >> efforts that have been merged to develop long ago.  Don't forget to
> >> delete your branches when you're done with them.
> >>
> >>
> >>
>

Re: GEODE-6630 fix for release 1.9.0

[Dan Smith]

+1 - this looks like a good fix to get in if it was introduced in 1.9.0.

-Dan

On Thu, Apr 18, 2019 at 1:28 PM Eric Shu  wrote:

> I'd like to include the fix for the NPE.
> It is new in 1.9.
>
> Regards,
> Eric
>

Re: Propose GEODE-6544 fix for release 1.9.0

[Dan Smith]

Owen and I talked to Kirk and Aaron. It sounds like this issue does not
need to hold up release 1.9.0 because it's been in all previous versions.
If we need to create another RC we will get this fix in there.

-Dan

On Fri, Apr 19, 2019 at 1:50 PM Owen Nichols  wrote:

> Hi Aaron, Kirk and Geode community,
>
> FYI, 1.9.0.RC4 has already been tagged and is being prepared for a VOTE
> email to go out later today.
>
> [As usual] if you or anyone else feels that 1.9.0 should not ship without
> this or other changes, you may cast a “-1” vote once you receive the
> 1.9.0.RC4 VOTE email later today.
>
> -Owen
>
> > On Apr 19, 2019, at 12:35 PM, Aaron Lindsey  wrote:
> >
> > We'd like to propose including this fix into 1.9.0 as well:
> > https://github.com/apache/geode/pull/3362
> > https://jira.apache.org/jira/browse/GEODE-6544
> >
> > This fixes a memory leak caused by a thread leak in Pulse. Every time a
> > user signs out, a thread is left running. The fix for GEODE-6544 stops
> that
> > thread when the user signs out.
> >
> > - Aaron and Kirk
>
>

Geode release steps

[Dan Smith]

Hi devs,

I'm looking at trying to automate some of the geode release steps, but one
thing I noticed is that we are inconsistent about when or if we create a
release branch in our different repos that we release (geode,
geode-examples, geode-native). For geode, we create a branch when we start
the release process. For geode-examples and native, we just grab the latest
develop when we create a release candidate.

Is there any reason why we shouldn't consistently create a release branch
in all of these repos at the same time?

-Dan

Re: [VOTE] Apache Geode 1.9.0 RC4

[Dan Smith]

+1

Looks good to me. I ran the geode-release-check against the repo.

-Dan

On Fri, Apr 19, 2019 at 5:00 PM Owen Nichols  wrote:

> Hello, Geode dev community,
>
>
> This is the fourth release candidate for Apache Geode, version 1.9.0.
>
> Thanks to all the community members for their contributions to this
> release!
>
>
> Please do a review and give your feedback. The deadline is before 3 PM PST
> Wed April 24th, 2019.
>
>
> 1.9.0 resolves 296 issues on Apache Geode JIRA system. Release notes
> can be found at:
>
>
> https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.9.0
> <
> https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.9.0
> >
>
>
> Please note that we are voting upon the source tags: rel/v1.9.0.RC4
>
>
> Apache Geode:
>
> https://github.com/apache/geode/tree/rel/v1.9.0.RC4 <
> https://github.com/apache/geode/tree/rel/v1.9.0.RC4>
>
> Apache Geode examples:
>
> https://github.com/apache/geode-examples/tree/rel/v1.9.0.RC4 <
> https://github.com/apache/geode-examples/tree/rel/v1.9.0.RC4>
>
> Apache Geode Native:
>
> https://github.com/apache/geode-native/tree/rel/v1.9.0.RC4 <
> https://github.com/apache/geode-native/tree/rel/v1.9.0.RC4>
>
>
> Commit ID:
>
> Apache Geode: c0a73d1cb84986d432003bd12e70175520e63597
>
> Apache Geode Examples: b58b2720e9cb1fae9e2cbca7053bc9c748dab79d
>
> Apache Geode Native: add53da376c3044feb2fb22dc37a30989c271f19
>
>
> Source and binary files:
>
> https://dist.apache.org/repos/dist/dev/geode/1.9.0.RC4 <
> https://dist.apache.org/repos/dist/dev/geode/1.9.0.RC4>
>
> Maven staging repo:
>
> https://repository.apache.org/content/repositories/orgapachegeode-1054 <
> https://repository.apache.org/content/repositories/orgapachegeode-1054>
>
> Geode's KEYS file containing PGP keys we use to sign the release:
>
> https://github.com/apache/geode/blob/develop/KEYS <
> https://github.com/apache/geode/blob/develop/KEYS>
>
>
> Signed the release with GPG fingerprint:
>
> DB5476815A475574577D442B468A4800EAFB2498
>
> PS: Command to run geode-examples: ./gradlew -PgeodeReleaseUrl=
> https://dist.apache.org/repos/dist/dev/geode/1.9.0.RC4 <
> https://dist.apache.org/repos/dist/dev/geode/1.9.0.RC4>
> -PgeodeRepositoryUrl=
> https://repository.apache.org/content/repositories/orgapachegeode-1054 <
> https://repository.apache.org/content/repositories/orgapachegeode-1054>
> build
> runAll
>
>
> Regards,
> - Owen & Dan

Re: Pulse - Non-Embedded Mode

[Dan Smith]

Option 2 does sound like a good way to go. It does seem like if you are
making changes to fix non-embedded mode, you probably need to add an
acceptance test for that mode since there is non already, regardless of
whether you deprecate non-embedded mode.

I have no issues with deprecating either embedded or non-embedded mode. I
don't think we've put a lot of energy into pulse recently.

-Dan


On Tue, Apr 30, 2019 at 2:12 PM Jens Deppe  wrote:

> More accurately, I think geode-core is only required when TLS is enabled on
> the locator and Pulse needs to make JMX/RMI calls over TLS.
>
> I would vote for option 2 in this scenario.
>
> --Jens
>
> On Tue, Apr 30, 2019 at 1:44 PM Jinmei Liao  wrote:
>
> > I believe to run pulse in non-embedded mode, you just need to install the
> > war in a web server and some configuration changes, you don't need
> > geode-core at all.
> >
> > We do lack the acceptance test to run pulse in non-embedded mode though.
> We
> > have a few unit tests that touches some aspect of it.
> >
> > On Tue, Apr 30, 2019 at 12:10 PM Michael Oleske 
> > wrote:
> >
> > > Hi Geode Community!
> > >
> > > Some colleagues and I were looking at GEODE-6683 (
> > > https://issues.apache.org/jira/browse/GEODE-6683) and noticed that we
> do
> > > not have test coverage for running Pulse in non-embedded mode.  We were
> > > wondering what our strategy is around Pulse in non-embedded mode. In
> > order
> > > to fully fix the issue, we would prefer to have a high-level acceptance
> > > test that actually tries to run Pule in non-embedded mode (we could not
> > > find an existing acceptance test that performs this).   However, this
> > > non-embedded mode seems a bit odd, as the instructions for it (
> > >
> > >
> >
> https://geode.apache.org/docs/guide/19/tools_modules/pulse/pulse-hosted.html
> > > )
> > > are slightly confusing and need some updating for geode (such as making
> > > sure geode-core is on the class path). It seems strange to try and
> host a
> > > web app in this way, especially with the extra configuration needed
> > (cannot
> > > just plop the Pulse war file in my web server with some config and have
> > it
> > > work).  So there's some questions about the best path forward.
> > >
> > > 1.  Should we continue supporting non-embedded mode for Pulse?  It
> seems
> > > like it may be useful to allow Pulse to run outside of a member, but
> not
> > as
> > > it currently does.  If it was deprecated, I wouldn't be as insistent on
> > an
> > > acceptance test for it.
> > >
> > > 2.  Should we try to make a separate artifact that is intended to be
> > > deployed on a web server?  This would have a new artifact that could
> run
> > > elsewhere then (with maybe a user provided config file for properties.)
> > >
> > > 3.  For the issue that brought up these questions (GEODE-6683), we have
> > > currently only written some unit tests to add the properties. So the
> > > current question is what type of path forward should we take?
> > >
> > >
> > > -michael
> > >
> >
> >
> > --
> > Cheers
> >
> > Jinmei
> >
>

Re: Request to access Jira

[Dan Smith]

You should have access now.

Thanks!
-Dan

On Fri, May 3, 2019 at 1:15 PM Donal Evans  wrote:

> Hi,
>
> Could I please be granted access to the Jira for Geode? My username is
> DonalEvans,
>
> Thanks,
> Donal
>

Re: Permission for working on Jira

[Dan Smith]

Hi Geet,

Welcome! I think I already gave you access to work with geode JIRAs
earlier? I see you listed as having access in JIRA. Is there something you
are trying to do that isn't working?

-Dan

On Mon, May 13, 2019 at 2:23 PM Geetanjali rawat 
wrote:

> Hello Devs
>
> I would like to request for the permission for working on Geode Jira.
>
> Thanks
> Geet
>

Backwards compatibility issue with JSONFormatter

[Dan Smith]

I think the changes for GEODE-6327 broke backwards compatibility in
JSONFormatter with the change from fromJSON(String jsonString) to
fromJSON(String jsonString, String... identityFields)

Adding an additional varargs parameter to the method breaks code that was
compiled against the non-varargs version. I think we need to overload the
method instead.

Thanks to Gester for discovering this with his test!

-Dan

Re: Backwards compatibility issue with JSONFormatter

[Dan Smith]

Sounds good! Yeah, this is a bit of an interesting case where code will
still compile against the new API without change, but I think maintaining
compatibility of the binary API is also important. Thanks for looking into
it.

-Dan

On Tue, May 14, 2019 at 11:22 AM Ryan McMahon  wrote:

> Hi all,
>
> This is my mistake - it was a misunderstanding of what constitutes a
> breaking public API change.  If a client app were to recompile using the
> new bits with the new method signature, there wouldn't be an issue because
> the new signature would work with 0 varargs.  The problem arises when you
> hot swap the geode dependencies for that client app without recompiling, as
> the bytecode no longer matches.  Since we do support the hot swap use case
> for CVEs etc, I see now this is considered a breaking API change.
>
> I'll change this to use a method overload instead, it should be a pretty
> simple fix.  Luckily, this issue didn't make it into any Geode releases.
>
> Thanks,
> Ryan
>
>
> On Tue, May 14, 2019 at 10:47 AM Udo Kohlmeyer  wrote:
>
> > How did this slip the review process that the signature of a public API
> > was changed?
> >
> > Well done Gester for finding this!!
> >
> > --Udo
> >
> > On 5/14/19 10:40, Dan Smith wrote:
> > > I think the changes for GEODE-6327 broke backwards compatibility in
> > > JSONFormatter with the change from fromJSON(String jsonString) to
> > > fromJSON(String jsonString, String... identityFields)
> > >
> > > Adding an additional varargs parameter to the method breaks code that
> was
> > > compiled against the non-varargs version. I think we need to overload
> the
> > > method instead.
> > >
> > > Thanks to Gester for discovering this with his test!
> > >
> > > -Dan
> > >
> >
>

Re: [DISCUSS] is it time to make Windows tests gating?

[Dan Smith]

Looking at the metrics for the windows jobs, it looks like the windows
tests are mostly red due to a few specific tests. The acceptance and gfsh
distributed tests jobs seem to be ok, it's just the unit tests and
integration tests that have problems. It also looks like
ExportConfigCommandTest stopped failing recently. Anyone want to take the
credit?

I'd be in favor of putting them in the main pipeline but not the PR
pipeline, since they take so long. Based on the metrics it looks like we
are pretty close to green with these jobs in terms of effort. Anyone want
to volunteer to fix the remaining failures (or see if they have been fixed
recently)?

-Dan

https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-metrics/jobs/GeodeWindowsUnitTestOpenJDK8Metrics/builds/209

CreateDiskStoreCommandTest:  2 failures
ExportConfigCommandTest:  34 failures

https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-metrics/jobs/GeodeWindowsIntegrationTestOpenJDK8Metrics/builds/209

LauncherMemberMXBeanIntegrationTest:  4 failures
AvailablePortHelperIntegrationTest:  17 failures




On Thu, May 16, 2019 at 5:32 PM Owen Nichols  wrote:

> I’ve created a PR for this: https://github.com/apache/geode/pull/3597
>
> > On May 16, 2019, at 3:06 PM, Blake Bender  wrote:
> >
> > +1 this needs to happen.  I hope that doesn't cause too much pain for the
> > dev team, but the native client team has a hard requirement that all our
> > stuff works properly on Windows at all times, and it can cause trouble if
> > random builds of the server can break us on Windows.
> >
> > I would hesitate to run these per-commit if they're taking that long, but
> > daily is a thing that can easily happen.
> >
> > On Thu, May 16, 2019 at 2:23 PM Bruce Schuchardt  >
> > wrote:
> >
> >> big +1, as long as artifacts of failed runs can be downloaded
> >>
> >> On 5/15/19 6:28 PM, Owen Nichols wrote:
> >>> For a very long time we’ve had Windows tests in the main pipeline
> >> (hidden away, not in the default view), but the pipeline proceeds to
> >> publish regardless of whether Windows tests fail or even run at all.
> >>>
> >>> Now seems like a good time to review whether to:
> >>> a) treat Windows tests as first-class tests and prevent the pipeline
> >> from proceeding if any test fails on Windows
> >>> b) keep as-is
> >>> c) change Windows tests to trigger only once a week rather than on
> every
> >> commit, if they are going to remain "informational only"
> >>>
> >>> One disadvantage to making Windows tests gating is that they currently
> >> take much longer to run (around 5 hours, vs 2 hours for Linux tests).
> >>
>
>

Re: [DISCUSS] is it time to make Windows tests gating?

[Dan Smith]

You're awesome Jens!

-Dan

On Fri, May 17, 2019 at 9:48 AM Jens Deppe  wrote:

> I fixed the first two and started ignoring
> AvailablePortHelperIntegrationTest for Windows (Windows has a default
> active port range wildly different than Linux which makes this test very
> flaky, and since it's just testing test code it seemed reasonable to ignore
> it).
>
> LauncherMemberMXBeanIntegration test still needs some love and I can look
> at that since I've worked on it in the past.
>
> --Jens
>
> On Fri, May 17, 2019 at 9:33 AM Dan Smith  wrote:
>
> > Looking at the metrics for the windows jobs, it looks like the windows
> > tests are mostly red due to a few specific tests. The acceptance and gfsh
> > distributed tests jobs seem to be ok, it's just the unit tests and
> > integration tests that have problems. It also looks like
> > ExportConfigCommandTest stopped failing recently. Anyone want to take the
> > credit?
> >
> > I'd be in favor of putting them in the main pipeline but not the PR
> > pipeline, since they take so long. Based on the metrics it looks like we
> > are pretty close to green with these jobs in terms of effort. Anyone want
> > to volunteer to fix the remaining failures (or see if they have been
> fixed
> > recently)?
> >
> > -Dan
> >
> >
> >
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-metrics/jobs/GeodeWindowsUnitTestOpenJDK8Metrics/builds/209
> >
> > CreateDiskStoreCommandTest:  2 failures
> > ExportConfigCommandTest:  34 failures
> >
> >
> >
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-metrics/jobs/GeodeWindowsIntegrationTestOpenJDK8Metrics/builds/209
> >
> > LauncherMemberMXBeanIntegrationTest:  4 failures
> > AvailablePortHelperIntegrationTest:  17 failures
> >
> >
> >
> >
> > On Thu, May 16, 2019 at 5:32 PM Owen Nichols 
> wrote:
> >
> > > I’ve created a PR for this: https://github.com/apache/geode/pull/3597
> > >
> > > > On May 16, 2019, at 3:06 PM, Blake Bender 
> wrote:
> > > >
> > > > +1 this needs to happen.  I hope that doesn't cause too much pain for
> > the
> > > > dev team, but the native client team has a hard requirement that all
> > our
> > > > stuff works properly on Windows at all times, and it can cause
> trouble
> > if
> > > > random builds of the server can break us on Windows.
> > > >
> > > > I would hesitate to run these per-commit if they're taking that long,
> > but
> > > > daily is a thing that can easily happen.
> > > >
> > > > On Thu, May 16, 2019 at 2:23 PM Bruce Schuchardt <
> > bschucha...@pivotal.io
> > > >
> > > > wrote:
> > > >
> > > >> big +1, as long as artifacts of failed runs can be downloaded
> > > >>
> > > >> On 5/15/19 6:28 PM, Owen Nichols wrote:
> > > >>> For a very long time we’ve had Windows tests in the main pipeline
> > > >> (hidden away, not in the default view), but the pipeline proceeds to
> > > >> publish regardless of whether Windows tests fail or even run at all.
> > > >>>
> > > >>> Now seems like a good time to review whether to:
> > > >>> a) treat Windows tests as first-class tests and prevent the
> pipeline
> > > >> from proceeding if any test fails on Windows
> > > >>> b) keep as-is
> > > >>> c) change Windows tests to trigger only once a week rather than on
> > > every
> > > >> commit, if they are going to remain "informational only"
> > > >>>
> > > >>> One disadvantage to making Windows tests gating is that they
> > currently
> > > >> take much longer to run (around 5 hours, vs 2 hours for Linux
> tests).
> > > >>
> > >
> > >
> >
>

Re: [DISCUSS] Propose new committer and PMC member - Peter Tran

[Dan Smith]

Folks, this thread is dead. Please stop voting here!

>From Anthony:

Discussions related to committers and PMC members should be held on the
private@geode mailing list not the dev@ list.


On Mon, May 20, 2019 at 1:43 PM Xiaojian Zhou  wrote:

> +1
>
> On Mon, May 20, 2019 at 11:50 AM Mike Stolz  wrote:
>
> > This has the heading [DISCUSS] instead of [VOTE]
> >
> > I'm +1 anyway.
> >
> > On Mon, May 20, 2019 at 2:15 PM Jinmei Liao  wrote:
> >
> > > I'd like to discuss the proposal to add Peter Tran as a new Geode
> > > committer and PMC member. Peter has been working with Manageability
> > > team for a while now and has contributed to the team immensely. Before
> > > that he was a key contributor to PCC team.
> > >
> > >
> > > Please cast your vote. Voting ends one week from today (Monday, May
> 27th,
> > > 2019).
> > >
> > > [   ] +1  Approve
> > > [   ] +0  No opinion
> > > [   ] -1   Disapprove (and reason why)
> > >
> > >
> > > Pull requests and reviews:
> > >
> > > https://github.com/apache/geode/pulls?q=author%3Apetahhh
> > >
> > >
> > >
> >
> https://github.com/apache/geode/pulls?utf8=%E2%9C%93&q=commenter%3Apetahhh%20
> > >
> > >
> > > Mailing list:
> > >
> > > https://geode.markmail.org/search/?q=from%3A%22ptran%22
> > >
> > >
> > > JIRA:
> > >
> > >
> > >
> >
> https://issues.apache.org/jira/browse/GEODE-1910?jql=project%20%3D%20GEODE%20AND%20(reporter%20%3D%20%22peter%20tran%22%20or%20assignee%20%3D%20%22peter%20tran%22)%20ORDER%20BY%20summary%20DESC
> > >
> > >
> > > Thanks!
> > >
> > > --
> > > Cheers
> > >
> > > Jinmei
> > >
> >
>

Re: Changing external methods to no longer throw UnsupportedOperationException

[Dan Smith]

+1 to implementing this method in a minor release.

I'm with Jake on this one. Every bug we fix changes the behavior of the
product in some small way. This seems like a behavior change for the better
- I can't picture a use case where someone is actually *relying* on this
method throwing UnsupportedOperationException.

I suppose someone might write a test that this feature is not supported -
but it seems like the only reason to do that would be to detect when geode
starts supporting getStatistics, so they'd probably be happy to see
getStatistics start working!


-Dan

On Thu, May 23, 2019 at 10:31 AM Anilkumar Gingade 
wrote:

> I agree, this may not look like the usecase that one would be using or
> depending. Going with the backward compatibility requirement this will be
> breaking that contract.
> Again, based on the scenario and usecases, there could be exceptions. I am
> trying to see if the versioning support that's used to keep the backward
> compatibility contract can be used here.
>
> On Thu, May 23, 2019 at 10:17 AM Jacob Barrett 
> wrote:
>
> > But what application is going to legitimately call this method and expect
> > that it throw an exception? What would be the function of that usage?
> >
> > If you assume that calling this method under these conditions had no
> value
> > and would therefor never have been called then one could argue that
> > implementing this method is adding a feature. It adds a case where one
> > could legitimately call this method under new conditions.
> >
> > > On May 23, 2019, at 10:06 AM, Anilkumar Gingade 
> > wrote:
> > >
> > > As this changes the behavior on the existing older application; it
> seems
> > to
> > > break the backward compatibility requirements.
> > > We use client versions to keep the contracts/behavior same for older
> > > client; can we do the same here.
> > >
> > > -Anil.
> > >
> > >
> > > On Thu, May 23, 2019 at 8:33 AM Darrel Schneider <
> dschnei...@pivotal.io>
> > > wrote:
> > >
> > >> Is it okay, in a minor release, to implement Region.getStatistics for
> > >> partitioned regions? See GEODE-2685. The current behavior is for it to
> > >> always throw UnsupportedOperationException. I doubt that any
> > application is
> > >> depending on that behavior but it could be. I know we have seen
> changes
> > >> like this in the past break the tests of other products that are
> > layered on
> > >> top of Geode.
> > >> Should this type of change be considered one that breaks backwards
> > >> compatibility?
> > >>
> >
> >
>

Re: Could you add my username to Geode JIRA

[Dan Smith]

Hi Jack,

Done! You should have access now.

Thanks,
-Dan

On Thu, May 23, 2019 at 1:18 PM Jack Weissburg 
wrote:

> Hello,
>
> Could you please add jackw26 to Geode JIRA so that I can be assigned to
> tickets.
>
> Thanks,
>
> Jack
>

Re: [DISCUSS] Remove exception.getMessage() error handling

[Dan Smith]

I think the right thing to do in those 100+ cases may be different in each
case, a blanket search and replace might not be the best idea.

-Dan



On Fri, May 24, 2019 at 3:05 PM Jinmei Liao  wrote:

> does exception.toString() print out the entire stack trace? If so, I don't
> think we should use that to replace exception.getMessage().
>
> On Fri, May 24, 2019 at 1:18 PM Jack Weissburg 
> wrote:
>
> > Hi All,
> >
> > Owen and I investigated a strange error message caused because Geode
> > previously handled an error exception with exception.getMessage() instead
> > of
> >  exception.toString().
> >
> > The issue is that the exception message from exception.getMessage() could
> > be unhelpful or empty. Instead, we should return the whole exception via
> > exception.toString().
> >
> > exception.getMessage()is used widely throughout the Geode code base (100+
> > times) and could be prone to cause more issues similar to
> > https://issues.apache.org/jira/browse/GEODE-6796
> > I would like to change the remaining instances of
> exception.getMessage()to
> > avoid future issues of this variety.
> >
> > Thoughts? Comments?
> >
> > Best,
> > Jack Weissburg
> >
>
>
> --
> Cheers
>
> Jinmei
>

Re: Permission for wiki

[Dan Smith]

Added - you should have access now!

Thanks,
-Dan

On Tue, May 28, 2019 at 3:28 PM Jack Weissburg 
wrote:

> Hello,
>
> I would like to have permission to edit the wiki:
>
> https://cwiki.apache.org/confluence/display/GEODE/Guidelines+for+Cluster+Management+service
>
> Thank you,
>
> Jack Weissburg
>

Re: [DISCUSS] require reviews before merging a PR

[Dan Smith]

As others pointed out - I think it's really important to remember that
people and community are much more important than process. That is a key
part of "The Apache Way" - it's not a set very specific rules, but more of
a philosophy of building an open community. I think this page has a good
take on the apache way - http://theapacheway.com/.

The page I linked also mentions "Getting good enough consensus is often
better than voting." Apache is about consensus building, *not* about
voting. Ideally, the only things we should formally vote on are
irreversible decisions such as a release or new committer.

Regarding code reviews, I think having a really strict process implies that
we don't trust our committers. Rather than that, maybe have some guidelines
for committers who aren't sure how much of a review to get. Here's what I
feel like I've been trying to follow:
1. Fixing a typo, broken spotless, etc. - just push it!
2. Straightforward change - get at least one reviewer. Maybe a second
author on the commit should count here?
3. Technically challenging, complicated change - get multiple reviewers
4. New Public API, large features - Write up a wiki page and have a
discussion on the dev list to get feedback

For all of these, if someone rejects your PR/feature, fix the issues or
talk with them. A good rule of thumb is if you are frustrated or annoyed by
what they said - talk to them one-on-one first instead of answering
directly in the PR/thread. If you a really pissed, wait a day and then talk
to them one-on-one.

-Dan

On Fri, May 31, 2019 at 11:00 AM Helena Bales  wrote:

> Thanks for the filter, Jinmei!
>
> +1 to Naba and Bruce.
>
> I will add that I think we should have a formal policy of getting *a*
> review (and more for large changes), and that PRs that are merged without
> one should include (in the PR) a comment providing a justification for this
> merge, so that committers can review the reasoning later on if needed. This
> has the benefits of being low impact on our current workflow, but also
> increasing transparency, accountability, and thoughtfulness around the
> proposed changes and their impact.
>
> On Fri, May 31, 2019 at 10:42 AM Jinmei Liao  wrote:
>
> > I was told that screenshot that I sent earlier got filtered out by the
> dev
> > list. Basically, the filter puts "notificati...@github.com" in the
> "From"
> > section, and put "review_reques...@noreply.github.com" in the "Doesn't
> > have" section of the form.
> >
> > On Fri, May 31, 2019 at 10:36 AM Anthony Baker 
> wrote:
> >
> > >
> > >
> > > > On May 31, 2019, at 10:01 AM, Owen Nichols 
> > wrote:
> > > >
> > > > We chose to make Geode an Apache open source project for a reason.
> If
> > > we no longer wish to embrace The Apache Way <
> > > https://www.apache.org/theapacheway/index.html>, perhaps we should
> > > reconsider.
> > >
> > > I strongly disagree with the assertion that we are not following the
> > > Apache Way because we aren’t doing RTC.  Please take a look around
> other
> > > ASF communities and compare that to our approach.  I think you’ll see a
> > lot
> > > of similarities in the way we review GitHub pull requests.
> > >
> > > >
> > > > If we believe that reviewing each other’s code changes is an onerous
> > > burden of no value, we should question why.   The long-term success of
> > > Geode depends on sharing of knowledge, not “cowboy coders”.  3 reviews
> > > means now 3 other people are more familiar with that part of the code…
> > >
> > > Yes of course:  community >> code.  Can you point me to cases of
> “cowboy
> > > coding” in Geode?  I’m not seeing it but happy to be convinced
> otherwise.
> > >
> > > >
> > > > If apathy is our thing, Apache does allows for “lazy consensus”, but
> > you
> > > have to declare that you will be using it, e.g. “This PR fixes
> > > GEODE-123456; if no-one objects within three days, I'll assume lazy
> > > consensus and merge it.”
> > >
> > > IMO lazy consensus does not imply apathy.
> > >
> > >
> > >
> > > Anthony
> > >
> > >
> >
> > --
> > Cheers
> >
> > Jinmei
> >
>

Re: what is the best way to update a geode pull request

[Dan Smith]

+1 to pushing PR changes as separate commits. Also +1 to creating PRs with
multiple commits where it makes sense.

-Dan

On Fri, May 31, 2019 at 1:28 PM Owen Nichols  wrote:

> Personally, I do not force-push to my PRs once any review comments have
> been accumulated, for the reasons you mention.
>
> Not sure if some people just force-push out of habit, or if the
> requirement for initial commit to be squashed creates some fear.
>
> I would go a step further and suggest that there is no reason to even
> require the initial PR to be a single, squashed commit.  If you have made a
> small fix and also done some refactoring or cleanup, as a reviewer I would
> much rather you submit that PR with multiple distinct commits, so I don’t
> have to comb through a gigantic diff trying to spot what is the real change
> and what is just renames.
>
> GitHub already very nicely presents the combined delta by default, as if
> it was a single squashed commit, so to me there is only negative value in
> encouraging/requiring pre-squashing.
>
> > On May 31, 2019, at 1:20 PM, Darrel Schneider 
> wrote:
> >
> > Something I have noticed is that often when I have requested changes be
> > made to a pull request is that the changes are force pushed ask a single
> > commit to the pr. I would actually prefer that the changes show up as a
> new
> > commit on the pr instead of everything being rebased into one commit.
> That
> > makes the history of the pr easier to follow and make it easy to see what
> > has changed since the previous review. What do others think? Have we done
> > something that makes contributors think the pull request has to be single
> > commit? I know the initial pull request is supposed to be but from then
> on
> > I'd prefer that we wait to squash when we merge it to develop.
>
>

Re: Static Analysis Tools such as SonarQube or others?

[Dan Smith]

We're currently running PMD as part of the gradle build. PMD is just
running a couple of rules specifically to look for mutable statics. We've
also enabled integration with lgtm to get a report -
https://lgtm.com/projects/g/apache/geode/.


I think added more static analysis is a good idea. I'm not that particular
about which tool(s) we are using - although maybe we should focus on open
source tools? I do think that in order to be valuable, the static analysis
rules need to fail the build like we're doing with spotless and PMD. So I
think an approach of cleaning up and enforcing one rule at a time is better
than just generating a report with a bunch of rule violations.

-Dan


On Tue, Jun 4, 2019 at 6:56 AM Peter Tran  wrote:

> Hi all,
>
> Has anyone had experience using static analysis tools such as SonarQube?
> Were there helpful? And favourites that worked well?
>
> Thanks
>

Re: [PROPOSAL] Add hostname-for-clients to ConfigurationProperties

[Dan Smith]

A user can configure the hostname-for-clients for locators and gateway
receivers - they are part of the respective gfsh commands. Are you
suggesting deprecating those settings as well, or just having a global
property that is the default value if it is not overridden for a specific
locator, cache-server, or gateway-receiver?

-Dan

On Tue, Jun 4, 2019 at 12:02 PM Jinmei Liao  wrote:

> We have "jmx-manager-hostname-for-clients" in the geode properties, we
> think it would be a good idea to deprecate that and use
> "hostname-for-clients" for the entire server. Currently we already need
> this property when launching a locator and start a gateway-receiver, and we
> have no way to pre-configure them.
>
> This is first step towards fixing our current problem of cluster
> configuration prescribing specific hostname-for-senders for gateway
> receivers.
>
> Thoughts/comments?
>

Re: [DISCUSS] Criteria for PMC, committers

[Dan Smith]

One concern I have about not bundling committer and PMC membership is that
there is not much incentive to nominate someone to become a PMC member.
When someone is an active contributor but not a committer it's visible in
the fact that they have to ask others to merge their PRs, which also
provides incentive to nominate them. But once they are a committer it's not
really obvious that someone is not also a PMC member, so there is no
reminder to nominate them for the next step.

For that reason I think we should continue bundling the two.

I'm also concerned with the "mateship bias" that Udo described. I do think
evidence of good judgement and a sense of responsibility is much more
important than the code that a nominee writes, and that's much easier to
judge if you have more contact with them. But maybe we can look for that
evidence specifically in the mailing list and PR comments?

-Dan

On Tue, Jun 4, 2019 at 2:59 PM Mark Bretl  wrote:

> I think in any point of view we are looking at Committer or PMC membership,
> it does come down to merit. Does the person have the merit, which can also
> be trust, to be a Committer and/or PMC member?
>
> The difference between Committer and PMC member is not simply being
> 'gatekeepers' of the codebase, PMC members provide oversight to the entire
> project [1], which I cannot put the same responsibility on a Committer. I
> don't think there has to a promotion path in terms of Committer -> PMC,
> however, I do believe there is a distinction between the two, especially
> since it is the Apache Board which ultimately makes the decision for PMC
> membership. A candidate be nominated for both at one time, but then it
> would be an all or nothing vote.
>
> --Mark
>
> [1]: https://www.apache.org/foundation/governance/pmcs.html
>
> On Fri, May 31, 2019 at 3:17 PM Jacob Barrett  wrote:
>
> > I think it's a lot of what is under the contributing section, but I think
> > it needs to be cleaned up. A lot of the what should be done is lost under
> > the screen shots of things. I nice clear bullet point, with maybe links
> to
> > the wiki for details, would be nice. If we collected all of this in the
> > CONTRIBUTING.md its right there in your source and baked into GitHub,
> which
> > is the primary place developers go, not the wiki. Then our PR template
> > could just say something like “[ ] complies with contributing
> guidelines”.
> > Eh??
> >
> > -Jake
> >
> >
> > > On May 31, 2019, at 3:12 PM, Anthony Baker  wrote:
> > >
> > > Are you thinking in terms of something like this?
> > > https://cwiki.apache.org/confluence/display/GEODE/Code+of+Conduct <
> > https://cwiki.apache.org/confluence/display/GEODE/Code+of+Conduct>
> > >
> > > Or something more specific to coding tasks?
> > >
> > >
> > > Thanks,
> > > Anthony
> > >
> > >
> > >> On May 31, 2019, at 2:41 AM, Owen Nichols 
> wrote:
> > >>
> > >> I think it might be helpful to first define clearly what code of
> > conduct a committer is expected to follow.  That exercise would help
> frame
> > exactly what we are trusting new (and existing) committers to adhere to.
> > >
> >
> >
>

Re: [DISCUSS] require reviews before merging a PR

[Dan Smith]

>
> Would it be possible to allow people who do not have committer status to
> request reviewers on a pull request.


I'm not sure it's possible to do this with the "reviewers" field - if
someone can figure out how to do this with the github IU, we can at least
try filing an ticket with apache infrastructure.

In the meantime, you can also just add a comment mentioning the people
you'd like to review the request. That works for requesting reviews from
non-committers as well.

-Dan

On Thu, Jun 6, 2019 at 9:05 AM Joris Melchior  wrote:

> Would it be possible to allow people who do not have committer status to
> request reviewers on a pull request. In some cases we may know who should
> take a look at it and in that case making it official by adding these
> people to the pull request would be good IMO.
>
> On Thu, Jun 6, 2019 at 10:26 AM Jens Deppe  wrote:
>
> > As reviewers we should also feel empowered to request additional
> reviewers
> > on a PR (perhaps beyond whomever the original submitter may already have
> > requested).
> >
> > I think that, sometimes the complexity of a change prevents someone from
> > commenting on just a portion of the change if they do not feel
> comfortable
> > understanding the scope of the whole change.
> >
> > Having said that though, once you have 'touched' a PR you should also be
> > tracking the PR for additional commits or feedback until it is merged.
> >
> > --Jens
> >
> > On Wed, Jun 5, 2019 at 11:37 AM Alexander Murmann 
> > wrote:
> >
> > > >
> > > > If we trust committers, why review at all? Just commit... and we
> might
> > > > catch a problem, we might not.
> > >
> > > Honestly that to me would be the ideal state. However, our test
> coverage
> > > and code quality is nowhere near to allow for that.
> > >
> > > What I was referring to is different though. I didn't say "trust them
> to
> > > write perfect code", but trust " to decide how much review they require
> > to
> > > feel comfortable".  In some cases this might mean one review and in
> > others
> > > maybe two, three or even more and maybe even by very specific people.
> > >
> > > On Wed, Jun 5, 2019 at 11:31 AM Udo Kohlmeyer  wrote:
> > >
> > > > Alexander, thank you for your response. And yes, change is
> > uncomfortable
> > > > and in some cases more reviewers would not have caught issues. BUT,
> > more
> > > > people would have seen the code, maybe become more familiar with it,
> > > etc...
> > > >
> > > > I don't say don't trust committers, as I am one. But I also know
> that I
> > > > mistakes are made regardless of intent. If we trust committers, why
> > > > review at all? Just commit... and we might catch a problem, we might
> > not.
> > > >
> > > > --Udo
> > > >
> > > > On 6/5/19 11:20, Alexander Murmann wrote:
> > > > > Udo, I agree with many of the pains you are addressing, but am
> > > > pessimistic
> > > > > that having more reviewers will solve them.
> > > > >
> > > > > You are absolutely correct in calling out that the code is ugly
> > complex
> > > > and
> > > > > missing coverage. The best way to address this is to clean up the
> > code
> > > > and
> > > > > improve coverage. You say yourself "In the past single small
> changes
> > > have
> > > > > caused failures the were completely unforeseen by anyone". I don't
> > > think
> > > > > more eyeballs will go a long way in making someone see complex bugs
> > > > > introduced by seemingly safe changes.
> > > > >
> > > > > I also am concerned that introducing a hurdle like this will make
> > > > > committers more excited to review PRs with care, but rather might
> > lead
> > > to
> > > > > less care. It  would be great of our committers were more
> passionate
> > > > about
> > > > > PR reviews and do them more often, but forcing it rarely
> accomplishes
> > > > that
> > > > > goal.
> > > > >
> > > > > I'd rather see us trust our committers to decide how much review
> they
> > > > > require to feel comfortable about their work and use the time saved
> > to
> > > > > address the root of the problem (accidental complexity & lack of
> test
> > > > > coverage)
> > > > >
> > > > > On Wed, Jun 5, 2019 at 11:03 AM Udo Kohlmeyer 
> > wrote:
> > > > >
> > > > >> @Kirk, I totally understand the pain that you speak of. I too
> agree
> > > that
> > > > >> every line of changed code should have a test confirming that
> > behavior
> > > > >> was not changed.
> > > > >>
> > > > >> I don't believe that we need to go as far as revoking committer
> > rights
> > > > >> and reviewing each committer again, BUT it would be AMAZING that
> out
> > > of
> > > > >> our 100 committers, 80% of them would be more active in PR
> reviews,
> > > > >> mailing lists and in the end active on the project outside of
> their
> > > > >> focus area.
> > > > >>
> > > > >> I do want to remind all Geode committers, it IS your
> responsibility
> > to
> > > > >> be part of the PR review cycle. I will hold myself just as
> > accountable
> > > > >> to this than what I hold every committer t

Re: [DISCUSS] changing geode 32-bit counter stats to 64-bit

[Dan Smith]

+1 - this sounds like a good change to me.

-Dan

On Fri, Jun 7, 2019 at 1:47 PM Jacob Barrett  wrote:

> I like this!
>
> I’d go ahead and change all the usage of the int methods to the long
> methods. I’d deprecate the int methods to make it very clear.
>
> If some consumer is using the int methods they will still work with the
> same rollover issues but perhaps with the deprecated warning they will
> update their code. Without the warning they may never know.
>
> -jake
>
>
> > On Jun 7, 2019, at 1:32 PM, Darrel Schneider 
> wrote:
> >
> > We have had a couple of tickets that have problems with 32-bit counters
> > changing too fast and causing them to be hard to understand when they
> wrap
> > around (see GEODE-6425 and GEODE-6834). We have also had problems with
> some
> > stats being broken because they were changing the 32-bit one when they
> > should have been changing the 64-bit one (see GEODE-6776).
> > The current implementation has one array of values for the 32-bit stats
> and
> > another array of values for the 64-bit stats. We use indexes into these
> > arrays when changing a stat. Given an int "id" used to index these
> arrays,
> > we can not tell if we should be indexing the 32-bit array or 64-bit
> array.
> > The first 32-bit stat for a type will have an id of 0 and the first
> 64-bit
> > stat on that type will also have an id of 0. But our current
> implementation
> > has the same type of value in both arrays (LongAdder
> > see: StripedStatisticsImpl fields intAdders and longAdders). So if we
> > simply change our implementation to have a single array, then the ids
> will
> > no longer be overloaded.
> >
> > Changing this internal implementation also improves backward
> compatibility.
> > Currently when we change one of our counters from 32-bit to 64-bit it is
> > possible we would break existing applications that are using the
> Statistics
> > interface. It has methods on it that allow stats to be read given an it:
> > getInt(int id) and getLong(int id). If they are currently reading a
> 32-bit
> > stat using getInt(id) and we change that stat to be 64-bit (like we did
> in
> > GEODE-6425) then they will now be reading the wrong stat when they call
> > getInt(id). If they used getInt(String) or getInt(StatisticDescriptor)
> they
> > will be okay. But if we make this simple change to have 32-bit and 64-bit
> > stats stored in the same array then getInt(id) will do the right thing
> when
> > we change a stat from 32-bit to 64-bit.
> >
> > Does anyone see a problem with making this change?
> >
> > After we do it I think we should change all of our counters to 64-bit
> since
> > they are always stored in a LongAdder anyway and we should deprecate the
> > methods that support 32-bit stats.
>

Re: Permissions to apache-geode jira dashboard

[Dan Smith]

You should have access now.

Thanks,
-Dan

On Mon, Jun 10, 2019 at 10:31 AM Murtuza Boxwala 
wrote:

> Can I please have permissions to edit the Apache-Geode JIRA dashboard.  My
> username is mboxwala.  I would like to be able to add new tickets, edit
> ticket, assign tickets to myself…
>
> Thanks,
> Murtuza

Re: [PROPOSAL] Instrumenting Geode Code

[Dan Smith]

+1

I like it!

I like that we are not having to maintain duplicate meter and stat code.
Also, I think keeping hiding metrics and stats inside an instrumentation
class with methods like regionAdded makes it much easier to unit test
classes that record metrics.

-Dan
On Mon, Jun 10, 2019, 5:13 PM Aaron Lindsey  wrote:

> +1
>
> I like this approach compared to the previous proposals because it's
> simpler (doesn't require a custom registry) and makes it more
> straightforward to replace stats with Micrometer meters in the future.
>
> - Aaron
>
>
> On Fri, Jun 7, 2019 at 4:58 PM Dale Emery  wrote:
>
>> Proposal for instrumenting Geode
>>
>>
>> *Goals*
>>
>> - Allow routing measurements to statistics, Micrometer meters, or both as
>> appropriate, to allow publishing each measurement to the appropriate places.
>>
>> - Minimize the intrusiveness of new and existing metrics instrumentation.
>>
>> - Minimize the amount of code that must know how to route measurements to
>> stats and meters.
>>
>> - Maintain Geode performance.
>>
>> - Not preclude any options for deprecating and removing Geode’s
>> statistics mechanism in the future. And take steps to make deprecating and
>> removing the existing statistics mechanism easier.
>>
>>
>> *Proposal*
>>
>> - Continue to use Geode’s existing style of instrumentation class (e.g.
>> CachePerfStats) to instrument code.
>>
>> - Enhance the instrumentation classes to create and register meters when
>> appropriate, and to route each measurement to the appropriate stats and
>> meters.
>>
>> - When we want to route a given measurement to both a statistic and a
>> meter, use the "legacy meter" wrapper mechanism (described below).
>>
>> - Incrementally improve the instrumentation classes to focus more on
>> reporting domain events (e.g. regionAdded()) and less on reporting
>> measurements (e.g. incRegions()).
>>
>>
>> *Legacy Meter Wrappers*
>>
>> To route a given measurement to both a Micrometer meter and a Geode
>> statistic, do the following in the instrumentation class's constructor:
>>
>> - Create and register a normal Micrometer meter in the cache's meter
>> registry.
>>
>> - Wrap the registered meter in a custom "legacy meter" that reads and
>> writes the stat, and also writes to the registered meter.
>>
>> In the instrumentation methods (e.g. endPut()):
>>
>> - Use the legacy meter to report measurements.
>>
>> I've attached two diagrams below to show how the wrapper mechanism works
>> for "counter" style stats and meters. The first is a class diagram, showing
>> how the parts relate in general. The second is a sequence diagram that
>> shows in some detail how the parts interact to route a given measurement to
>> both a meter and a stat.
>>
>> If you want even more details, Jake Barrett has created a nice "proof of
>> concept" implementation:
>> https://github.com/apache/geode/compare/develop...pivotal-jbarrett:wip/LegacyMeterBuilder
>>
>>
>>
>> Please let us know if you have questions, comments, or concerns.
>>
>>
>> Cheers,
>> Dale
>>
>> —
>> Dale Emery
>> dem...@pivotal.io
>>
>>
>>
>>
>>
>>
>>
>>
>>

Re: Disk dir size units in javadoc comments

[Dan Smith]

I think the comment is correct, but what that test is doing is weird. I see
it's setting this property, which leads be to believe that there is some
code that changes the units based on this property.

DirectoryHolder.SET_DIRECTORY_SIZE_IN_BYTES_FOR_TESTING_PURPOSES = true;

Maybe a better change would be to add another setDiskDirsAndSizes
method that actually takes a unit and remove this static property?


On Tue, Jun 18, 2019 at 8:32 AM Alberto Bustamante Reyes
 wrote:

> Hi all,
>
>
> I have observed that in the javadoc comments of setDiskDirsAndSizes method
> in DiskStoreFactory class, it is stated that dir sizes are expected to be
> in megabytes:
>
>   /**
>* Sets the directories to which this disk store's data is written and
> also set the sizes in
>* megabytes of each directory.
>*
>* @param diskDirs directories to put the oplog files.
>* @param diskDirSizes sizes of disk directories in megabytes
>* @return a reference to this
>*
>* @throws IllegalArgumentException if length of the size array does not
> match to the length of
>* the dir array
>*/
>   DiskStoreFactory setDiskDirsAndSizes(File[] diskDirs, int[]
> diskDirSizes);
>
>
> But I was taking a look at DiskRegionJUnitTest, and I have seen that the
> values introduced to create the dirs in the tests, are treated as bytes.
> Check "testDiskFullExcep()", for example.
>
> Also, the comment in the default disk dir size in the DiskStoreFactory
> class, talks also about megabytes:
>
>   /**
>* The default disk directory size in megabytes.
>* 
>* Current value: 2,147,483,647 which is two petabytes.
>*/
>   int DEFAULT_DISK_DIR_SIZE = Integer.MAX_VALUE; // unlimited for bug 41863
>
> I supose these comments about "megabytes" are wrong and they should be
> changed to "bytes". Could someone confirm that? In that case I can change
> it.
>
> Thanks in advance!
>
> Alberto B.
>
>
>

Re: Confluence access

[Dan Smith]

Done! You should have access now.

Thanks,
-Dan

On Tue, Jun 18, 2019 at 4:03 PM Murtuza Boxwala  wrote:

> Can I get access to the confluence wiki, please?
>
> Email: mboxw...@pivotal.io 
> Username: mboxwala

[DISCUSS] Add a test dependency to geode-core - ArchUnit

[Dan Smith]

Hi all,

Bill, Ernie, and I would like to add a new (apache licensed) test
dependency to geode-core - https://github.com/TNG/ArchUnit. This is a tool
that lets you write tests that make assertions about the interdependencies
of your code - for example enforcing that package A does not depend on
package B.

Initially we intend to add some tests about what parts of the system the
org.apache.geode.distributed.internal.membership package depends on, with
an eye towards making that code more independently testable (proposal on
that coming soon!).

Does anyone have an issue with adding this test dependency?

-Dan

Re: [PROPOSAL]: Improve OQL Method Invocation Security

[Dan Smith]

+1

This looks really good!

I put a couple of comments inline, and I have a few more general questions
here:
1. Is the RegionQueryInvocationAuthorizer different than our existing shiro
permissions? I thought users can already grant permissions for specific
regions. What does this add in addition to that?
2. I'm a little unclear on if your
MethodInvocationAuthorizer.authorizeMethodInvocation is supposed to take a
region or the target object. If it is really accepting a region, do we
actually have a region available in all cases? We could be invoking methods
on an object in lots of places in the query tree.
3. The DataAwareBasedMethodAuthorizer seems a bit vague on how it's
actually going to work. It also might be a security risk, since it will
allow users with DATA:READ permission to invoke any method on these objects.

-Dan

On Wed, Jun 19, 2019 at 11:34 AM Jacob Barrett  wrote:

> Thanks Juan!
>
> > On Jun 19, 2019, at 9:55 AM, Juan José Ramos  wrote:
> >
> > Hello all,
> >
> > I've removed all "biased" words I could find from the original document
> so
> > the *Proposal [1]* is ready for review and discussion now. All feedback
> is
> > welcome.
> > Best regards.
> >
> > [1]:
> >
> https://cwiki.apache.org/confluence/display/GEODE/OQL+Method+Invocation+Security
> >
> >> On Fri, Jun 14, 2019 at 8:39 PM Juan José Ramos 
> wrote:
> >>
> >> Hey Jake,
> >>
> >> Thanks for bringing this up. As you might have found out already,
> english
> >> is not my native language, I actually had to do some research to find
> out
> >> *exactly what you meant* regarding the bias around the "whitelist" word
> >> :-|... It was an honest mistake and I sincerely apologize in advance if
> >> anyone got offended in any way.
> >> That said, I won't have time to go through the proposal and make the
> >> required changes until next week, so I'll keep the document hidden until
> >> all biased words are replaced.
> >> Cheers.
> >>
> >>
> >> On Sat, Jun 15, 2019 at 12:25 AM Jacob Barrett 
> >> wrote:
> >>
>  As part of GEODE-3247 <
> https://issues.apache.org/jira/browse/GEODE-3247>,
> >>> several options were analysed and, after considering the wealth of
> security
> >>> holes and the difficulty of determining which methods deployed by the
> >>> developer were intended to be available for queries and which were
> not, the
> >>> decision was made to tighten up the Security and, by default, disallow
> any
> >>> method call not explicitly whitelisted.
> >>>
> >>> Please avoid biased words, like whitelist, in source and proposals.
> There
> >>> are several other places in this document that use these terms. Can you
> >>> please update the document without them.
> >>>
> >>> Thanks,
> >>> Jake
> >>>
> >>>
> >>
> >> --
> >> Juan José Ramos Cassella
> >> Senior Technical Support Engineer
> >> Email: jra...@pivotal.io
> >> Office#: +353 21 4238611
> >> Mobile#: +353 87 2074066
> >> After Hours Contact#: +1 877 477 2269
> >> Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT
> >> How to upload artifacts:
> >> https://support.pivotal.io/hc/en-us/articles/204369073
> >> How to escalate a ticket:
> >> https://support.pivotal.io/hc/en-us/articles/203809556
> >>
> >> [image: support]  [image: twitter]
> >>  [image: linkedin]
> >>  [image: facebook]
> >>  [image: google plus]
> >>  [image: youtube]
> >> <
> https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
> >>
> >
> >
> > --
> > Juan José Ramos Cassella
> > Senior Technical Support Engineer
> > Email: jra...@pivotal.io
> > Office#: +353 21 4238611
> > Mobile#: +353 87 2074066
> > After Hours Contact#: +1 877 477 2269
> > Office Hours: Mon - Thu 08:30 - 17:00 GMT. Fri 08:30 - 16:00 GMT
> > How to upload artifacts:
> > https://support.pivotal.io/hc/en-us/articles/204369073
> > How to escalate a ticket:
> > https://support.pivotal.io/hc/en-us/articles/203809556
> >
> > [image: support]  [image: twitter]
> >  [image: linkedin]
> >  [image: facebook]
> >  [image: google plus]
> >  [image: youtube]
> > <
> https://www.youtube.com/playlist?list=PLAdzTan_eSPScpj2J50ErtzR9ANSzv3kl>
>

Re: [DISCUSS] Add a test dependency to geode-core - ArchUnit

[Dan Smith]

We don't have a membership gradle module, just a package. We're adding this
to geode-core.

For a little more context - we are thinking about refactoring membership
(and/or maybe some other pieces) into separate gradle modules - proposal
forthcoming! However, as a first step we need to untangle those pieces of
code from the rest of geode-core. Rather than creating some long lived
branch we can incrementally untangle the code a piece at a time, on
develop. Having a way to track progress and enforce the direction of
dependencies on the way to a separate gradle module will help with that.

-Dan

On Thu, Jun 20, 2019 at 4:23 PM Jacob Barrett  wrote:

> Are you adding this dependency to just the membership module? I am cool
> with that.
>
> > On Jun 20, 2019, at 2:39 PM, Dan Smith  wrote:
> >
> > Hi all,
> >
> > Bill, Ernie, and I would like to add a new (apache licensed) test
> > dependency to geode-core - https://github.com/TNG/ArchUnit. This is a
> tool
> > that lets you write tests that make assertions about the
> interdependencies
> > of your code - for example enforcing that package A does not depend on
> > package B.
> >
> > Initially we intend to add some tests about what parts of the system the
> > org.apache.geode.distributed.internal.membership package depends on, with
> > an eye towards making that code more independently testable (proposal on
> > that coming soon!).
> >
> > Does anyone have an issue with adding this test dependency?
> >
> > -Dan
>
>

Re: [DISCUSS] RFC 0: Lightweight RFC Process

[Dan Smith]

+1

Looks good to me!

A couple of minor thoughts - it might be nice to have separate
subdirectories on the wiki for the different proposal states, to easily see
what state the proposals are in.

One thing that isn't visible in these states - is the proposal actively
under development?  It might be nice to clearly distinguish between
proposals that are under development vs. finished.

-Dan

On Mon, Jun 24, 2019 at 2:31 PM Alexander Murmann 
wrote:

> Having the RFC discussion in a pull request was by far the most
> controversial aspect of this proposal. Because we were unable to come to an
> agreement, we should stick with the smallest change to what we are doing
> already. Therefore I moved the proposal to the wiki where all existing
> proposals are.
>
> Please take a look there
>  >
> and let's discuss here if any changes should be made. Otherwise I intent to
> move the proposal to `active` tomorrow.
>
> Thank you everyone!
>
> On Thu, Jun 13, 2019 at 4:26 PM Alexander Murmann 
> wrote:
>
> > Hi everyone,
> >
> > I am proposing a new process that is aimed to address some of the issues
> > we've recently encountered in making collective decisions. The process I
> am
> > proposing would use pull request to discuss proposals.
> >
> > To demonstrate the process, I submitted my proposal as a pull request
> > . While this email has the
> > [DISCUSS] label, I ask to *please keep all discussions on the PR
> > * to test drive the proposed
> > process. There already is a healthy discussion on the PR, including if
> the
> > process in general should take place via a PR.
> >
> > Thanks!
> >
>

Re: [DISCUSS] RFC 0: Lightweight RFC Process

[Dan Smith]

>
> Just to make sure I got this 100% right, you mean the work related as part
> of the proposal would be under development, correct?


Yes! And I like your suggestion to just create a couple of buckets on the
wiki, rather than one for each state.

-Dan

On Mon, Jun 24, 2019 at 3:37 PM Alexander Murmann 
wrote:

> > it might be nice to have separate subdirectories on the wiki for the
> different proposal states, to easily see what state the proposals are in.
>
> I think that's useful. I wonder if it would make sense though to bucket
> some of these. Maybe we could make do with just a "current" and "old"
> directory. I like hiding RFCs that are no longer relevant, but don't care
> much if they are dropped or superseded. Likewise, I don't care much if they
> are active, under discussion or under development. Thoughts?
>
> > One thing that isn't visible in these states - is the proposal actively
> under development?  It might be nice to clearly distinguish between
> proposals that are under development vs. finished.
>
> Just to make sure I got this 100% right, you mean the work related as part
> of the proposal would be under development, correct?
>
> On Mon, Jun 24, 2019 at 3:30 PM Dan Smith  wrote:
>
> > +1
> >
> > Looks good to me!
> >
> > A couple of minor thoughts - it might be nice to have separate
> > subdirectories on the wiki for the different proposal states, to easily
> see
> > what state the proposals are in.
> >
> > One thing that isn't visible in these states - is the proposal actively
> > under development?  It might be nice to clearly distinguish between
> > proposals that are under development vs. finished.
> >
> > -Dan
> >
> > On Mon, Jun 24, 2019 at 2:31 PM Alexander Murmann 
> > wrote:
> >
> >> Having the RFC discussion in a pull request was by far the most
> >> controversial aspect of this proposal. Because we were unable to come to
> >> an
> >> agreement, we should stick with the smallest change to what we are doing
> >> already. Therefore I moved the proposal to the wiki where all existing
> >> proposals are.
> >>
> >> Please take a look there
> >> <
> >>
> https://cwiki.apache.org/confluence/display/GEODE/Lightweight+RFC+Process
> >> >
> >> and let's discuss here if any changes should be made. Otherwise I intent
> >> to
> >> move the proposal to `active` tomorrow.
> >>
> >> Thank you everyone!
> >>
> >> On Thu, Jun 13, 2019 at 4:26 PM Alexander Murmann 
> >> wrote:
> >>
> >> > Hi everyone,
> >> >
> >> > I am proposing a new process that is aimed to address some of the
> issues
> >> > we've recently encountered in making collective decisions. The process
> >> I am
> >> > proposing would use pull request to discuss proposals.
> >> >
> >> > To demonstrate the process, I submitted my proposal as a pull request
> >> > <https://github.com/apache/geode/pull/3706>. While this email has the
> >> > [DISCUSS] label, I ask to *please keep all discussions on the PR
> >> > <https://github.com/apache/geode/pull/3706>* to test drive the
> proposed
> >> > process. There already is a healthy discussion on the PR, including if
> >> the
> >> > process in general should take place via a PR.
> >> >
> >> > Thanks!
> >> >
> >>
> >
>

Re: [PROPOSAL]: Improve OQL Method Invocation Security

[Dan Smith]

+1

This proposal looks good to me!

On Mon, Jun 24, 2019 at 4:15 PM Udo Kohlmeyer  wrote:

> +1, Count me in
>
> On 6/24/19 13:06, Juan José Ramos wrote:
> > Hey Jake,
> >
> > Sure, I guess we could do a live session if there's enough interest after
> > people have reviewed the proposal.
> > Best regards.
> >
> > On Mon, Jun 24, 2019 at 4:17 PM Jacob Barrett 
> wrote:
> >
> >>
> >>> On Jun 24, 2019, at 11:49 AM, Juan José Ramos 
> wrote:
> >>>
> >>>   I’d rather get feedback in any way and aggregate everything on my own
> >> than
> >>> maybe not getting anything because I'm explicitly limiting the options
> to
> >>> provide it.
> >> Dealers choice so both it is!
> >>
> >> Could you also consider public live session on some medium, like Zoom,
> >> where you can walk through the proposal and take like feedback and
> >> questions?
> >>
> >> Thanks,
> >> Jake
> >>
> >>
> >>
>

Re: [DISCUSS] RFC 0: Lightweight RFC Process

[Dan Smith]

>
> Will moving the page around on the wiki result in dead links to the draft
> version?
>

No. If you use the share button in the wiki, you get a permanent link to
the page. Even if you just copy the URL from the address bar it doesn't
include the folder the page is in.

-Dan

Re: [PROPOSAL]: Improve OQL Method Invocation Security

[Dan Smith]

A couple of other things to keep in mind to make sure you aren't
introducing security holes

1) Queries can be executed by users with read privileges. So if it needs to
be really clear to the operator whether the MethodInvocationAuthorizer they
are configuring is going to let users call methods that modify the data or
not.
2) OQL lets you invoke methods on objects that are passed as parameters to
the query. So that means that any class that exists on the server and is
serializable could potentially have methods invoked in it through a query
when someone opens up access with one of these MethodInvocationAuthorizers.
So the RegexBasedMethodAuthorizer needs to be used with care.

-Dan

On Tue, Jun 25, 2019 at 11:53 AM Anthony Baker  wrote:

> Here are the things I think are important:
>
> 1) I shouldn’t have to change my domain classes in order to run a query.
> 2) I shouldn’t have to configure anything to run a “normal” query that
> uses classes deployed into the cluster and stored in the region.
> 3) By default the cluster is secure from malicious users trying to execute
> untrusted code*.
>
>
> Anthony
>
> * if a user chooses to deploy code into the cluster that does bad things
> that’s on them
>
>
> > On Jun 25, 2019, at 11:07 AM, Aaron Lindsey  wrote:
> >
> > +1 to the proposal
> >
> > Some comments:
> >
> >   - There is almost always trade-off between security and ease-of-use.
> The
> >   proposed implementation of JavaBeanAccessorBasedMethodAuthorizer makes
> me
> >   feel uneasy because it would be super easy to create a method that
> begins
> >   with "get" or "is" but is not actually a JavaBean accessor method.
> However,
> >   I can also see how nice this would be in terms of ease-of-use.
> >   - From a security standpoint I prefer AnnotationBasedMethodAuthorizer
> >   because it's very explicit on which methods may be executed. To remove
> the
> >   coupling between the configuration and domain classes, you could
> separate
> >   out the mapping of which methods are allowed into a different class
> and not
> >   use annotations.
> >   - How have other projects solved this problem? Can we add a "related
> >   work" section to the proposal? If you've already looked and didn't
> really
> >   find anything relevant you could also mention that in the proposal.
> >
> > - Aaron
> >
> >
> > On Mon, Jun 24, 2019 at 4:31 PM Jason Huynh  wrote:
> >
> >> +1
> >>
> >> I have some concerns about all of the different ways we configure geode
> to
> >> be secure, but that's a different issue ;-)
> >> Overall, very thorough proposal Juan!
> >>
> >>
> >>
> >> On Mon, Jun 24, 2019 at 4:22 PM Dan Smith  wrote:
> >>
> >>> +1
> >>>
> >>> This proposal looks good to me!
> >>>
> >>> On Mon, Jun 24, 2019 at 4:15 PM Udo Kohlmeyer  wrote:
> >>>
> >>>> +1, Count me in
> >>>>
> >>>> On 6/24/19 13:06, Juan José Ramos wrote:
> >>>>> Hey Jake,
> >>>>>
> >>>>> Sure, I guess we could do a live session if there's enough interest
> >>> after
> >>>>> people have reviewed the proposal.
> >>>>> Best regards.
> >>>>>
> >>>>> On Mon, Jun 24, 2019 at 4:17 PM Jacob Barrett 
> >>>> wrote:
> >>>>>
> >>>>>>
> >>>>>>> On Jun 24, 2019, at 11:49 AM, Juan José Ramos 
> >>>> wrote:
> >>>>>>>
> >>>>>>>  I’d rather get feedback in any way and aggregate everything on my
> >>> own
> >>>>>> than
> >>>>>>> maybe not getting anything because I'm explicitly limiting the
> >>> options
> >>>> to
> >>>>>>> provide it.
> >>>>>> Dealers choice so both it is!
> >>>>>>
> >>>>>> Could you also consider public live session on some medium, like
> >> Zoom,
> >>>>>> where you can walk through the proposal and take like feedback and
> >>>>>> questions?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Jake
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>
> >>
>
>

Re: [DISCUSS] Add a test dependency to geode-core - ArchUnit

[Dan Smith]

Following up on this, it looks like there is consensus to go ahead and add
ArchUnit. Thanks all!

-Dan

On Fri, Jun 21, 2019 at 3:49 PM Charlie Black  wrote:

> I like the idea especially if its only for compile/test time and doesn't
> make it into the binary bits.
>
> On Fri, Jun 21, 2019 at 1:04 PM Udo Kohlmeyer  wrote:
>
> > Thank you for the response...
> >
> > #1 - But isn't cyclical package dependent code not a smell and practice,
> > whilst at the same time and uni-directional dependency is preferred.
> >
> > Soo... I think I see the benefit to be more, that ArchUnit allows the
> > untangling of code into a modular way WITHOUT a big bang approach of
> > moving the code into modules and then having to be concerned about the
> > fallout. But also it allows for the managing of package dependencies
> > WITHOUT breaking the code out into different separate modules.
> >
> > I really like ArchUnit :).. We should prioritize adoption :)
> >
> > --Udo
> >
> > On 6/21/19 12:48, Murtuza Boxwala wrote:
> > > Two things come to mind:
> > >
> > > 1) uni-directional dependency
> > > Packages can be dependent on each other, because the classes inside of
> > them can use each other. e.g.  let’s say package A has class A1 and class
> > A2 and package B has class B1 and B2.  A1 can depend on B1, and B2 can
> > depend on A2. Hence, the packages are dependent on each other.
> > >
> > > Modules can only have uni-directional dependency. If Module A depends
> on
> > Module B, then no class in Module B can reference a class in Module A.
> > This prevents tangling, i.e. spaghetti
> > >
> > > 2) Incremental compilation
> > > This lack of tangling helps not only developers, but the compiler too.
> > In the packages example above, if I change any of the classes, all the
> code
> > has to get recompiled because the dependency lines can go in any
> direction,
> > and the compiler won’t attempt to optimize.  In the modules case, if
> Module
> > A changes, Module B will not recompile, because the dependency guarantees
> > that nothing about Module B could have been affected.
> > >
> > >> On Jun 21, 2019, at 2:14 PM, Udo Kohlmeyer  wrote:
> > >>
> > >> I know that I'm missing the benefit of physically moving the code from
> > the package into its own module.
> > >>
> > >> Could you possibly explain to me what it is?
> > >>
> > >> On 6/21/19 07:37, Murtuza Boxwala wrote:
> > >>> I think that’s a really clever way to increment toward splitting
> > geode-core into more modules. I am excited to see what it looks like 👍
> > >>>
> > >>>> On Jun 20, 2019, at 7:45 PM, Jacob Barrett 
> > wrote:
> > >>>>
> > >>>> Gotcha! Sounds good.
> > >>>>
> > >>>>> On Jun 20, 2019, at 4:35 PM, Dan Smith  wrote:
> > >>>>>
> > >>>>> We don't have a membership gradle module, just a package. We're
> > adding this
> > >>>>> to geode-core.
> > >>>>>
> > >>>>> For a little more context - we are thinking about refactoring
> > membership
> > >>>>> (and/or maybe some other pieces) into separate gradle modules -
> > proposal
> > >>>>> forthcoming! However, as a first step we need to untangle those
> > pieces of
> > >>>>> code from the rest of geode-core. Rather than creating some long
> > lived
> > >>>>> branch we can incrementally untangle the code a piece at a time, on
> > >>>>> develop. Having a way to track progress and enforce the direction
> of
> > >>>>> dependencies on the way to a separate gradle module will help with
> > that.
> > >>>>>
> > >>>>> -Dan
> > >>>>>
> > >>>>>> On Thu, Jun 20, 2019 at 4:23 PM Jacob Barrett <
> jbarr...@pivotal.io>
> > wrote:
> > >>>>>>
> > >>>>>> Are you adding this dependency to just the membership module? I am
> > cool
> > >>>>>> with that.
> > >>>>>>
> > >>>>>>> On Jun 20, 2019, at 2:39 PM, Dan Smith 
> wrote:
> > >>>>>>>
> > >>>>>>> Hi all,
> > >>>>>>>
> > >>>>>>> Bill, Ernie, and I would like to add a new (apache licensed) test
> > >>>>>>> dependency to geode-core - https://github.com/TNG/ArchUnit. This
> > is a
> > >>>>>> tool
> > >>>>>>> that lets you write tests that make assertions about the
> > >>>>>> interdependencies
> > >>>>>>> of your code - for example enforcing that package A does not
> > depend on
> > >>>>>>> package B.
> > >>>>>>>
> > >>>>>>> Initially we intend to add some tests about what parts of the
> > system the
> > >>>>>>> org.apache.geode.distributed.internal.membership package depends
> > on, with
> > >>>>>>> an eye towards making that code more independently testable
> > (proposal on
> > >>>>>>> that coming soon!).
> > >>>>>>>
> > >>>>>>> Does anyone have an issue with adding this test dependency?
> > >>>>>>>
> > >>>>>>> -Dan
> > >
> >
>
>
> --
> Charlie Black | cbl...@pivotal.io
>

Re: Access to update JIRA

[Dan Smith]

Done! You should have access now.

Thanks,
-Dan

On Wed, Jun 26, 2019 at 6:24 PM Kamilla Aslami  wrote:

> Hi Helena,
>
> My username is kaslami.
>
> Thanks!
> Kamilla
>
> On Wed, Jun 26, 2019 at 6:21 PM Helena Bales  wrote:
>
> > Hi Kamilla,
> >
> > What is your username for Jira? We'll need that to give you access.
> >
> > ~Helena
> >
> >
> > On Wed, Jun 26, 2019, 6:05 PM Kamilla Aslami  wrote:
> >
> > > Hello,
> > >
> > > Can I get access to update issues in JIRA, please? Right now I can't
> > change
> > > status of issues I'm working on.
> > >
> > > Thank you,
> > > Kamilla
> > >
> >
>

Re: [PROPOSAL]: Improve OQL Method Invocation Security

[Dan Smith]

With Juan's proposal, the user still has the option to use java's security
manager. They can install a MethodAuthorizor that allows all methods and
use a java SecurityManager to restrict method access to the methods they
want.

Regarding the question of "How far should we go to prevent users from
shooting themselves in the foot?" I think we do need to honor the security
policies that administrators have configured for geode. Since we allow the
administrators to configure the system to give certain users READ but not
WRITE access, it's on us to make sure those users are not allowed to modify
the data. If users with READ access can modify data, that is a security
vulnerability in Geode.

-Dan


On Wed, Jul 3, 2019 at 8:33 AM Juan José Ramos  wrote:

> Hello,
>
> Thanks for the feedback provided so far.
> I'm still resilient to add the *Java Security Manager* as an option, maybe
> it's due to my lack of experience around the subject but I still believe it
> is killing mosquitoes with a bazooka. We need to prevent this "bad things"
> only from within the OQL execution, and also provide our users with the
> means to customize the behaviour on their own desire, meaning that the
> configurable *Authorizer* stuff is still needed (it's up to them if they
> want to re-introduce CVE-2017-9795).
> Either way, this is not my decision to make, it's up to the community to
> decide. I'll go ahead and update the proposal, we can have another feedback
> round (1 week) and cast votes afterwards.
> Cheers.
>
>
> On Tue, Jul 2, 2019 at 5:23 PM Jason Huynh  wrote:
>
> > Are security manager policies modifiable on the fly?  Just wondering if
> > someone decides they want to disallow or allow something, will they need
> to
> > restart their vms/geode node?
> >
> > I think Dan pointed this out earlier in the thread, but just wanted to
> have
> > us consider the original cve that led to the heavy handed deny all method
> > invocations:
> >
> >   CVE-2017-9795 Apache Geode OQL method invocation vulnerability
> >
> >   Description:
> >   A malicious user with read access to specific regions within a Geode
> >   cluster may execute OQL queries that allow read and write access to
> >   objects within unauthorized regions.  In addition a user could invoke
> >   methods that allow remote code execution
> >
> > I think Juan's proposal would still allow us to provide multiple
> solutions
> > that may or may not reopen that hole, but it would be up to the user to
> > decide what they are willing to accept.  The choice for what should be
> > default would still be up for debate...
> >
> >
> >
> > On Tue, Jul 2, 2019 at 1:07 PM Jacob Barrett 
> wrote:
> >
> > >
> > >
> > > > On Jul 2, 2019, at 11:58 AM, Juan José Ramos 
> > wrote:
> > > >
> > > > Hello Jake,
> > > >
> > > > I've been doing some reading about the *Java Security Manager* and,
> > even
> > > > when it might work for our use case, I don't think is a good fit due
> to
> > > the
> > > > following reasons:
> > > > 1). We already have chosen *Shiro* for authentication and
> > authorization,
> > > > adding yet another security framework (and mapping between roles and
> > > > permissions between the two of them) for OQL method invocation
> security
> > > > seems overkilling to me. I'd rather spend some time improving the
> > > > *ResourcePermissionBasedMethodAuthorizer
> > >
> > > The security manager doesn’t have to be as fined grained as individual
> > > users. Do we really intend to support actions on objects base on
> current
> > > user too?
> > >
> > > > [1] *and/or adding *Permissions*/*Roles* to our current security
> > > framework
> > > > than introducing a new security framework altogether.
> > >
> > > Again, we don’t have to add anything. If the user wants to restrict
> > access
> > > to certain activities they could add a policy file to the JVM to do so.
> > It
> > > doesn’t have to be user based. We could simply have two hard coded
> roles,
> > > UserCode and SystemCode. SystemCode has AllPermissions, UserCode has
> > none.
> > > Now we run all queries under the context of UserCode and it can’t
> execute
> > > any File, Runtime, Socket, etc.
> > >
> > > > 2). There can only be one *Security Manager* per JVM at a given time
> > > Eh, sorta but no, you can use thread contexts and class loader
> isolation,
> > > but there really doesn’t need to be more than one.
> > > .
> > > > 3). Customers already using a custom *Security Manager* on their own
> > will
> > > > be in trouble... we can certainly wrote our own implementation as a
> > > > composite and parse multiple policy files, but this will probably
> break
> > > the
> > > > backward compatibility for these customers (and requires yet more
> > > > configuration and things to keep in mind when upgrading).
> > > Their custom SecurityManager would conform to the same rules as the
> > > default file based one in the JRE. They would be able to control code
> > > access using their existing framework. Seems like win to me.
> > >
>

Re: Proposal: For PR reviews and change requests can we have a 7 day turn around on re-reviews?

[Dan Smith]

+1

What do you think about assigning someone to each PR to make sure it gets
through the process? We don't currently seem to be using github's
"assignee" field. Committers can make themselves the assignee, but for
contributors we could assign a committer who will make sure the PR gets
reviewed and merged in a timely fashion.

-Dan

On Tue, Jul 9, 2019 at 10:34 AM Mark Hanson  wrote:

> Hi All,
>
> TL;DR
>
> Can we have a norm( preferred, but not required ) of providing feedback
> within seven days of the last checkin to a PR?
>
> Long version
>
> I have just spent a bit of time reviewing PRs that have been open for a
> while and sent some emails to reviewers of the ones that are open the
> longest. In my humble opinion, it would be very nice if we could close out
> some of the older PRs where the requester has made changes to, but
> reviewers have not re-reviewed. An ideal norm would seem to be 7 days. One
> might notice that I have a PR that I requested a change on, that I have not
> provided feedback on, so I am in the same boat...
>
> Thoughts?
>
> Thanks,
> Mark

Re: [Proposal] Refactor the Cache and Region perf stats structure.

[Dan Smith]

Seems reasonable. I'm guessing that CachePerfImpl contains many
RegionStats. Does PartitionRegionStatsImpl just contain a single
RegionStats?

On Wed, Jul 10, 2019 at 4:49 PM Mark Hanson  wrote:

> Hi All,
>
> As many of you may know our structure for our perf stats is not great. I
> would like to propose we refactor the code to have the following
> inheritance model, which Kirk and I came up with.
>
> It is my belief that fixing this will allow future features to be
> implemented in a much less painful way.
>
> Thoughts?
>
> Thanks,
> Mark
>

Re: [DISCUSS] RFC 0: Lightweight RFC Process

[Dan Smith]

Following up on this, I took a stab at organizing our old proposals into
the buckets on the wiki. We now have:

Under Discussion - Draft and In Discussion proposals
In Development - proposals under active development
Active - Proposals that are completely implemented
Dropped - Proposals that were not approved or development stalled out.

If I moved your proposal to "Dropped" erroneously, please feel free to move
it back! I moved things there that did not appear to have been implemented
or have any recent activity.

I put a few things in "Unknown State." If you know what state these
proposals are in, please move them!

https://cwiki.apache.org/confluence/display/GEODE/Project+Proposals+and+Specifications

On Wed, Jun 26, 2019 at 11:20 AM Alexander Murmann 
wrote:

> Given discussion here and previous discussion on the PR, I consider this
> proposal approved and updated its state accordingly.
>
> I also incorporated Dan's suggestion of moving deprecated proposals and
> added a reference to the new process at the top of the Project Proposals
> and Specifications page
> <
> https://cwiki.apache.org/confluence/display/GEODE/Project+Proposals+and+Specifications
> >
> .
>
> Thank you all for you great feedback throughout this process!
>
> On Tue, Jun 25, 2019 at 10:07 AM Dan Smith  wrote:
>
> > >
> > > Will moving the page around on the wiki result in dead links to the
> draft
> > > version?
> > >
> >
> > No. If you use the share button in the wiki, you get a permanent link to
> > the page. Even if you just copy the URL from the address bar it doesn't
> > include the folder the page is in.
> >
> > -Dan
> >
>
>
> --
> Alexander J. Murmann
> (650) 283-1933
>

Re: some pages in the wiki may be not on the right place

[Dan Smith]

+1

-Dan

On Fri, Jul 12, 2019 at 9:44 AM Gang Yan  wrote:

> seems no disagree response( and no agree response), I am planning to do
> this.
>
> Thanks
>
> Gang Yan（闫钢）
>
>
>
> On Fri, May 17, 2019 at 12:06 PM Gang Yan  wrote:
>
> > Hi Geode Dev
> >
> > updated table, to add 4 sub pages:
> >
> > original title original page traslated title translated page status last
> > updated
> > Replacing JGroups 2.2.9
> >
> https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9
> 替换
> > JGroups 2.2.9
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146172
> unfinished
> > translated last modified on Jul 16, 2016
> > 关于 Geode
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146511
> translated
> > topic last modified on Jul 17, 2016
> > Developing Applications for Geode
> >
> https://cwiki.apache.org/confluence/display/GEODE/Developing+Applications+for+Geode
> > 应用开发
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146968
> unfinished
> > translated, outdated  last modified on Jul 22, 2016
> > Design https://cwiki.apache.org/confluence/display/GEODE/Design 设计
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146976
> translated
> > topic last modified on Jul 30, 2016
> > Index https://cwiki.apache.org/confluence/display/GEODE/Index 概览
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146019
> translated,
> > outdated last modified  on Oct 28, 2016
> > Project Proposals and Specifications
> >
> https://cwiki.apache.org/confluence/display/GEODE/Project+Proposals+and+Specifications
> > 工程草案和规范
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146488
> translated
> > topic, only 3 topics last modified on Jul 16, 2016
> > Geode Redis Adapter Proposal
> >
> https://cwiki.apache.org/confluence/display/GEODE/Geode+Redis+Adapter+Proposal
> Geode
> > Redis Adaptor适配器
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146495
> > translated last modified on Jul 16, 2016
> > OQL Aggregate functions and UDA
> >
> https://cwiki.apache.org/confluence/display/GEODE/OQL+Aggregate+functions+and+UDA
> OQL
> > 聚集函数和 UDA
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65148037
> > translated last modified on Jul 30, 2016
> > Automated Rebalancing
> > https://cwiki.apache.org/confluence/display/GEODE/Automated+Rebalancing
> > 自动重平衡
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146493
> just
> > translated the title Created on Jul 16, 2016
> > Geode Security Framework
> >
> https://cwiki.apache.org/confluence/display/GEODE/Geode+Security+Framework
> Geode
> > 安全框架
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65148044
> > translated last modified on Jul 30, 2016
> > Moving from memcached to gemcached
> >
> https://cwiki.apache.org/confluence/display/GEODE/Moving+from+memcached+to+gemcached
> 从
> > memcached 迁移到 gemcached
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65148029
> > translated last modified on Jul 30, 2016
> > Network Configuration Best Practices
> >
> https://cwiki.apache.org/confluence/display/GEODE/Network+Configuration+Best+Practices
> > 网络配置最佳实践
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146974
> > translated  last modified on Jul 26, 2016
> > Roadmap https://cwiki.apache.org/confluence/display/GEODE/Roadmap 技术路线
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146514
> > translated last modified on Jul 16, 2016
> >
> >
> >
> > thanks
> >
> > Gang Yan（闫钢）
> >
> >
> >
> > On Fri, May 17, 2019 at 11:42 AM Gang Yan  wrote:
> >
> >> Hi Geode Dev
> >>
> >> some pages in wiki , are just a translation not a proposal.
> >>
> >> the pages are listed as follows, please focus on the column of "status"
> >> and "last updated":
> >> original title original page traslated title translated page status last
> >> updated
> >> Replacing JGroups 2.2.9
> >>
> https://cwiki.apache.org/confluence/display/GEODE/Replacing+JGroups+2.2.9
> 替换
> >> JGroups 2.2.9
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146172
> unfinished
> >> translated last modified on Jul 16, 2016
> >> 关于 Geode
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146511
> translated
> >> topic last modified on Jul 17, 2016
> >> Developing Applications for Geode
> >>
> https://cwiki.apache.org/confluence/display/GEODE/Developing+Applications+for+Geode
> >> 应用开发
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146968
> unfinished
> >> translated, outdated last modified on Jul 22, 2016
> >> Design https://cwiki.apache.org/confluence/display/GEODE/Design 设计
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65146976
> translated
> >> topic last modified on Jul 30, 2016
> >> Index https://cwiki.apache.org/confluence/display/GEODE/Index 概览
> >>
> http

Re: [DISCUSS] RFC 0: Lightweight RFC Process

[Dan Smith]

@Udo

I like both of your suggestions. Most of the proposals that I put in
"Dropped" still seemed like good ideas, perhaps even things we'd already
agreed to on the mailing list, but hadn't seen any recent development.

If no one objects, I'll go ahead and rename Active->Completed and
Dropped->Icebox. Should we change the name of the state in the Lightweight
RFC Process
<https://cwiki.apache.org/confluence/display/GEODE/Lightweight+RFC+Process>?

-Dan

On Mon, Jul 15, 2019 at 3:57 PM Udo Kohlmeyer  wrote:

> @Dan,
>
> Thank you for your first attempt at this.
>
> Maybe we should be a rename "Active" to "Completed". "Active" to me
> means that we are currently working on them, rather having completed
> them. I don't view these proposals as features that can be toggled
> on/off (or active/inactive).
>
> Also, I disagree with the approach that proposals that are not actively
> worked on are "Dropped". Which in itself is incorrect as well. Maybe
> there should be an "Icebox" area, that lists a set of proposals that
> have not yet been approved, but also not yet rejected.
>
> I think it is ok to have an "Icebox" of proposals that lists areas of
> improvement we want to target, but as of yet, no concrete proposal has
> yet been submitted. Modularity comes to mind. It is not that we don't
> want to do it, it is just that there is no proposal that has been
> accepted/completed.
>
> --Udo
>
> On 7/12/19 12:57, Dan Smith wrote:
> > Following up on this, I took a stab at organizing our old proposals into
> > the buckets on the wiki. We now have:
> >
> > Under Discussion - Draft and In Discussion proposals
> > In Development - proposals under active development
> > Active - Proposals that are completely implemented
> > Dropped - Proposals that were not approved or development stalled out.
> >
> > If I moved your proposal to "Dropped" erroneously, please feel free to
> move
> > it back! I moved things there that did not appear to have been
> implemented
> > or have any recent activity.
> >
> > I put a few things in "Unknown State." If you know what state these
> > proposals are in, please move them!
> >
> >
> https://cwiki.apache.org/confluence/display/GEODE/Project+Proposals+and+Specifications
> >
> > On Wed, Jun 26, 2019 at 11:20 AM Alexander Murmann 
> > wrote:
> >
> >> Given discussion here and previous discussion on the PR, I consider this
> >> proposal approved and updated its state accordingly.
> >>
> >> I also incorporated Dan's suggestion of moving deprecated proposals and
> >> added a reference to the new process at the top of the Project Proposals
> >> and Specifications page
> >> <
> >>
> https://cwiki.apache.org/confluence/display/GEODE/Project+Proposals+and+Specifications
> >> .
> >>
> >> Thank you all for you great feedback throughout this process!
> >>
> >> On Tue, Jun 25, 2019 at 10:07 AM Dan Smith  wrote:
> >>
> >>>> Will moving the page around on the wiki result in dead links to the
> >> draft
> >>>> version?
> >>>>
> >>> No. If you use the share button in the wiki, you get a permanent link
> to
> >>> the page. Even if you just copy the URL from the address bar it doesn't
> >>> include the folder the page is in.
> >>>
> >>> -Dan
> >>>
> >>
> >> --
> >> Alexander J. Murmann
> >> (650) 283-1933
> >>
>

Re: [DISCUSS] Time to cut Geode 1.10.0?

[Dan Smith]

+1 for cutting a 1.10.0 release branch.



On Fri, Jul 26, 2019 at 3:55 PM Nabarun Nag  wrote:

> Hi,
> I believe the original authors of the feature has done their due diligence
> and followed all steps, we can get this feature in under the Experimental
> flag and let the community improve on it, if there is anything else that
> needs to be done.
>
> We have done this before for Lucene re-index feature, where we involved the
> entire community in its development, phase by phase. The wiki is up and
> running, if someone has any concerns can raise it as a JIRA or comment in
> the wiki and the community as a whole can decide if it is a valid
> concern or not and act upon it.
>
> Regards
> Nabarun Nag
>
>
>
> On Fri, Jul 26, 2019 at 3:40 PM Udo Kohlmeyer  wrote:
>
> > @Alexander + @Jared,
> >
> > So maybe that was my misunderstanding on the RFC (not being optional on
> > new feature work). Given that this is a new feature, there is
> > significant risk to getting it "wrong".
> >
> > I was expecting more discussion around this. I have some objections to
> > the current approach/design. Given that my day job does not allow me to
> > respond in a timely manner, I would have not been able to get all my
> > concerns raised. In addition, throwing something onto the wiki, and then
> > a few weeks before we'd like to cut a version raising a discussion
> > thread on work that has been going on for months already does not help
> > with the community being able to read, digest, think, reason and respond
> > BEFORE it is released.
> >
> > I know `@Experimental` is non-binding on API's or usage, BUT I prefer
> > some of the ground work to have been discussed, API's validated BEFORE
> > it is released into the wild. I mean this is a PUBLIC API, so we'd
> > prefer to get it more correct than the previous one.
> >
> > Maybe it is just me, taking it too serious... Where I prefer not release
> > something as close to 95% correct (and discussed).
> >
> > Anyway.. If we want to cut 1.10... and we should... Let's do so.. but
> > I'd prefer that more on the correctness on the approach.
> >
> > --Udo
> >
> > On 7/25/19 11:08 AM, Alexander Murmann wrote:
> > >> I don't believe we should be including anything into the Geode release
> > >> that has not gone through the correct process of feature proposal.
> > >>
> > >> All work under the experimental cluster management service has not yet
> > >> been approved by the agreed upon RFC process.
> > >>
> > > Udo, I didn't take the RFC process that we agreed on to be a gate
> keeper,
> > > but rather a way to de-risk work before making a PR.
> > >
> > >  From the RFC doc in the wiki:
> > >
> > >> When to write an RFC?
> > >>
> > >> Writing an RFC should be entirely voluntary. There is always the
> option
> > of
> > >> going straight to a pull request. However, for larger changes, it
> might
> > be
> > >> wise to de-risk the risk of rejection of the pull request by first
> > >> gathering input from the community. Therefore it’s up to every member
> of
> > >> our community to decide themselves when they want to reach for this
> > tool.
> > >>
> > > If we want to change the role of the RFC process, that's fine with me,
> > but
> > > we should have that discussion first.
> > >
> > > On Tue, Jul 23, 2019 at 10:30 AM Jared Stewart <
> stewart.ja...@gmail.com>
> > > wrote:
> > >
> > >> What was missing from the RFC process for the cluster management
> > service?
> > >> I saw a [Discuss] thread for it, as well as a proposal at
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/GEODE/Cluster+Management+Service
> > >>
> > >> On Tue, Jul 23, 2019 at 10:02 AM Udo Kohlmeyer 
> wrote:
> > >>
> > >>> I don't believe we should be including anything into the Geode
> release
> > >>> that has not gone through the correct process of feature proposal.
> > >>>
> > >>> All work under the experimental cluster management service has not
> yet
> > >>> been approved by the agreed upon RFC process.
> > >>>
> > >>> I don't believe we should be including this work, experimental or
> > >>> otherwise.
> > >>>
> > >>> --Udo
> > >>>
> > >>> On 7/22/19 4:51 PM, Alexander Murmann wrote:
> >  Udo, do you mind explaining how the RFC process comes into this? Are
> > >> you
> >  suggesting that we should wait if an RFC had a target release
> > attached?
> > 
> >  On Mon, Jul 22, 2019 at 4:47 PM Udo Kohlmeyer 
> wrote:
> > 
> > > I don't think we need to wait for this, as there has been no RFC
> > >> process
> > > followed.
> > >
> > > --Udo
> > >
> > > On 7/22/19 3:38 PM, Jinmei Liao wrote:
> > >> Work is still being planned to move the cluster management rest
> > >> service
> > >> under an experimental version flag and use a geode property to
> turn
> > >> it
> > >> on/off. I would say we are ready to cut the geode 1.10.0 after
> that
> > >>> work
> > > is
> > >> complete.
> > >>
> > >> On Mon, Jul 22, 2019 at 3:24 PM Alexander Murmann <
> > >> 

Another change for 1.10.0 release

[Dan Smith]

Hi all,

I'd like to get the fix for GEODE-7055 (Don't send failure replies from a
P2P reader thread) into the 1.10.0 release branch.

This bug was causing a hang on startup for users of the session replication
module that didn't put the session jars on the classpath of the locator.
The hang doesn't happen with 1.9.0, so I'd I think we should make sure it
won't happen with 1.10.0 as well.

-Dan

Re: Unit tests are hanging?

[Dan Smith]

> With all that, I propose we permanently bump the timeouts on UnitTestX jobs
> across the board (main pipeline, PR pipeline, etc) from 10 to 20 minutes to
> be more tolerant of these types of degradations.
>

+1

-Dan

Re: Wiki edit permissions

[Dan Smith]

Done! You should have access now.

Thanks,
-Dan

On Fri, Aug 9, 2019 at 10:02 AM Helena Bales  wrote:

> Hello,
> I am trying to update a wiki doc and need permissions for Confluence. My
> LDAP username is heybales.
> Thank you!
> ~Helena
>

Re: Proposal to Include GEODE-7079 in 1.10.0

[Dan Smith]

+1 to merging Juan's fix for GEODE-7079. I've seen systems taken down by
rapidly filling up the logs in the past, this does seem to be a critical
fix from the perspective of system stability.

Also, this is the change, which doesn't seem particularly risky to me.

-  ConflationKey key = new
ConflationKey(gsEvent.getRegion().getFullPath(),
+  ConflationKey key = new ConflationKey(gsEvent.getRegionPath(),

-Dan

On Thu, Aug 15, 2019 at 12:23 PM Udo Kohlmeyer  wrote:

> Whilst I agree with "*finish* when we believe the quality of the release
> branch is sufficient", I disagree that we should have cut a branch and
> continue to patch that branch with non-critical fixes. i.e this issue
> has been around for a while and has no averse side effects. Issues like
> GEODE-7081, which is new due to a new commit, AND it has critical
> stability implications on the server, that I can agree we should include
> in a potential release branch.
>
> Otherwise we can ALWAYS argue that said release branch is not of
> "sufficient" quality, especially if there are numerous existing JIRA's
> pertaining to bugs already in the system.
>
> To quote Juan's original email:
>
> /"Note: *no events are lost (even without the fix)* but, if the region
> takes//
> //a while to recover, the logs  for the member can grow pretty quickly
> due to//
> //the continuously thrown *NPEs.*"/
>
> In addition to this, if there is a commit in a cut release branch, which
> is requiring us to continuously patching the release branch, in order to
> stabilize that feature/fix, maybe we should consider reverting that fix
> and release it at a later stage, when it is believed that this fix is
> more stable and have better, more comprehensive test coverage.
>
> So far, GEODE-7081, does not have me convinced that it is critical. OR
> maybe it is the latter of my options, where it is a stabilization commit
> to a new feature, which begs the question, should we have accepted the
> original feature commit if there are all manner of side effects which we
> are only discovering.
>
> --Udo
>
> On 8/15/19 11:08 AM, Anthony Baker wrote:
> > While we can’t fix *all known bugs*, I think where we do have a fix for
> an important issue we should think hard about the cost of not including
> that in a release.
> >
> > IMO, the fixed time approach to releases means that we *start* the
> release effort (including stabilization and bug fixing if needed) on a
> known date and we *finish* when new believe the quality of the release
> branch is sufficient.  Given the number of important fixes being requested,
> I’m not sure we are there yet.
> >
> > I think the release branch concept has merit because it allows us to
> isolate ongoing work from the changes needed for a release.
> >
> > +1 for including GEODE-7079.
> >
> > Anthony
> >
> >
> >> On Aug 15, 2019, at 10:51 AM, Udo Kohlmeyer 
> wrote:
> >>
> >> Seems everyone is in favor or including a /*non-critical*/ fix to an
> already cut branch of the a potential release...
> >>
> >> Am I missing something?
> >>
> >> Why cut a release at all... just have a perpetual cycle of fixes added
> to develop and users can chose what nightly snapshot build they would want
> to use..
> >>
> >> I'm voting -1 on a non-critical issue, which is existing and worst
> effect is to fill logs will NPE logs... (yes, not something we want).
> >>
> >> I believed that we (as a Geode community) agreed that once a release
> has been cut, only critical issue fixes will be included. If we continue
> just continually adding to the ALREADY CUT 1.10 release, where do we stop
> and when do we release...
> >>
> >> --Udo
> >>
> >> On 8/15/19 10:19 AM, Nabarun Nag wrote:
> >>> +1
> >>>
> >>> On Thu, Aug 15, 2019 at 10:15 AM Alexander Murmann <
> amurm...@apache.org>
> >>> wrote:
> >>>
>  +1
> 
>  Agreed to fixing this. It's impossible for a user to discover they
> hit an
>  edge case that we fail to support till they are in prod and restart.
> 
>  On Thu, Aug 15, 2019 at 10:09 AM Juan José Ramos 
>  wrote:
> 
> > Hello Udo,
> >
> > Even if it is an existing issue I'd still consider it critical for
> those
> > cases on which there are unprocessed events on the persistent queue
>  after a
> > restart and the region takes long to recover... you can actually see
> > millions of *NPEs* flooding the member's logs.
> > My two cents anyway, it's up to the community to make the final
> decision.
> > Cheers.
> >
> >
> > On Thu, Aug 15, 2019 at 5:58 PM Udo Kohlmeyer 
> wrote:
> >
> >> Juan,
> >>
> >>   From your explanation, it seems this issue is existing and not
> >> critical. Could we possibly hold this for 1.11?
> >>
> >> --Udo
> >>
> >> On 8/15/19 5:29 AM, Ju@N wrote:
> >>> Hello team,
> >>>
> >>> I'd like to propose including the *fix [1]* for *GEODE-7079 [2]* in
> >> release
> >>> 1.10.0.
> >>> Long story short: a *Nu

Re: Propose fix for 1.10 release: Prevent NPE in getLocalSize()

[Dan Smith]

Normally cherry-picking to the release branch is the release managers job
(Dick in this case) [1]. He asked me to help out while he was on vacation,
so I will go ahead and cherry-pick it over.

I kinda like the process Jake proposed though - creating a PR against the
release branch. My only concern would be that maybe it would be too easy
for someone to accidentally merge to the release branch without realizing
that was the target?  Still might be better than making the release manager
do the cherry-pick.

[1] https://cwiki.apache.org/confluence/display/GEODE/Releasing+Apache+Geode

-Dan

On Thu, Aug 15, 2019 at 2:53 PM Jacob Barrett  wrote:

> You should be able to do the cherry-pick on your fork and then open a PR
> against the release branch.
>
> > On Aug 15, 2019, at 2:04 PM, Aaron Lindsey  wrote:
> >
> > It sounds like there is consensus on adding this fix. Could someone
> please cherry-pick this for me?
> >
> > Thanks,
> > Aaron
> >
> >> On Aug 14, 2019, at 1:13 PM, Udo Kohlmeyer  wrote:
> >>
> >> @Aaron,Kirk - thank you for the clarification.
> >>
> >> +1 to include the fix, as reverting GEODE-7001 would be more effort :)
> >>
> >> --Udo
> >>
> >> On 8/14/19 9:25 AM, Aaron Lindsey wrote:
> >>> @Udo, I think Kirk explained it well — This issue was introduced very
> recently (right before we cut the release branch) and it has serious
> consequences (requires restarting the server).
> >>>
> >>> - Aaron
> >>>
>  On Aug 14, 2019, at 9:06 AM, Kirk Lund  wrote:
> 
>  +1 to include this fix in 1.10.0
> 
>  FYI: The race condition for this code path to throw NPE (which is
>  catastrophic and requires restarting the server) was introduced by
> commit
>  279fa0 on July 31 for GEODE-7001.
> 
>  On Tue, Aug 13, 2019 at 6:22 PM Anthony Baker 
> wrote:
> 
> > Given that we’re trying to stabilize the release branch and this fix
> seems
> > to *help* that I’m in favor of merging it.
> >
> > Anthony
> >
> >
> >> On Aug 13, 2019, at 5:32 PM, Udo Kohlmeyer  wrote:
> >>
> >> @Aaron, is this an existing issue (i.e this was not introduced in a
> > current refactor)?
> >> If the answer is anything other that "This will make the system stop
> > working", I would vote: -1
> >> If this is an existing issue and has been around for a while, I
> think we
> > hold off including this.
> >> I think the boat has sailed on the inclusion of issues into the 1.10
> > release. Sorry...
> >> --Udo
> >>
> >> On 8/13/19 4:58 PM, Aaron Lindsey wrote:
> >>> I’d like to propose including
> >
> https://github.com/apache/geode/pull/3913/commits/6f1814d1f719cc06b13769c40a9d6d01f99f927c
> > <
> >
> https://github.com/apache/geode/pull/3913/commits/6f1814d1f719cc06b13769c40a9d6d01f99f927c
> >
> > in the Geode 1.10 release.
> >>> This commit fixes an issue where a NullPointerException is thrown
> from
> > PartitionedRegion.getLocalSize() when the statistics callback
> sampler is
> > invoked before a PartitionedRegion is initialized.
> >>> - Aaron
> >>>
> >>>
> >>>
> >
> >
>
>

Re: Propose fix for 1.10 release: Prevent NPE in getLocalSize()

[Dan Smith]

@kirk - go ahead and push it.

-Dan

On Thu, Aug 15, 2019 at 3:13 PM Kirk Lund  wrote:

> I have the cherry-pick ready to push or file a PR. Let me know what you
> prefer...
>
> On Thu, Aug 15, 2019 at 3:01 PM Dan Smith  wrote:
>
> > Normally cherry-picking to the release branch is the release managers job
> > (Dick in this case) [1]. He asked me to help out while he was on
> vacation,
> > so I will go ahead and cherry-pick it over.
> >
> > I kinda like the process Jake proposed though - creating a PR against the
> > release branch. My only concern would be that maybe it would be too easy
> > for someone to accidentally merge to the release branch without realizing
> > that was the target?  Still might be better than making the release
> manager
> > do the cherry-pick.
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/GEODE/Releasing+Apache+Geode
> >
> > -Dan
> >
> > On Thu, Aug 15, 2019 at 2:53 PM Jacob Barrett 
> wrote:
> >
> > > You should be able to do the cherry-pick on your fork and then open a
> PR
> > > against the release branch.
> > >
> > > > On Aug 15, 2019, at 2:04 PM, Aaron Lindsey 
> > wrote:
> > > >
> > > > It sounds like there is consensus on adding this fix. Could someone
> > > please cherry-pick this for me?
> > > >
> > > > Thanks,
> > > > Aaron
> > > >
> > > >> On Aug 14, 2019, at 1:13 PM, Udo Kohlmeyer  wrote:
> > > >>
> > > >> @Aaron,Kirk - thank you for the clarification.
> > > >>
> > > >> +1 to include the fix, as reverting GEODE-7001 would be more effort
> :)
> > > >>
> > > >> --Udo
> > > >>
> > > >> On 8/14/19 9:25 AM, Aaron Lindsey wrote:
> > > >>> @Udo, I think Kirk explained it well — This issue was introduced
> very
> > > recently (right before we cut the release branch) and it has serious
> > > consequences (requires restarting the server).
> > > >>>
> > > >>> - Aaron
> > > >>>
> > > >>>> On Aug 14, 2019, at 9:06 AM, Kirk Lund  wrote:
> > > >>>>
> > > >>>> +1 to include this fix in 1.10.0
> > > >>>>
> > > >>>> FYI: The race condition for this code path to throw NPE (which is
> > > >>>> catastrophic and requires restarting the server) was introduced by
> > > commit
> > > >>>> 279fa0 on July 31 for GEODE-7001.
> > > >>>>
> > > >>>> On Tue, Aug 13, 2019 at 6:22 PM Anthony Baker 
> > > wrote:
> > > >>>>
> > > >>>>> Given that we’re trying to stabilize the release branch and this
> > fix
> > > seems
> > > >>>>> to *help* that I’m in favor of merging it.
> > > >>>>>
> > > >>>>> Anthony
> > > >>>>>
> > > >>>>>
> > > >>>>>> On Aug 13, 2019, at 5:32 PM, Udo Kohlmeyer 
> > wrote:
> > > >>>>>>
> > > >>>>>> @Aaron, is this an existing issue (i.e this was not introduced
> in
> > a
> > > >>>>> current refactor)?
> > > >>>>>> If the answer is anything other that "This will make the system
> > stop
> > > >>>>> working", I would vote: -1
> > > >>>>>> If this is an existing issue and has been around for a while, I
> > > think we
> > > >>>>> hold off including this.
> > > >>>>>> I think the boat has sailed on the inclusion of issues into the
> > 1.10
> > > >>>>> release. Sorry...
> > > >>>>>> --Udo
> > > >>>>>>
> > > >>>>>> On 8/13/19 4:58 PM, Aaron Lindsey wrote:
> > > >>>>>>> I’d like to propose including
> > > >>>>>
> > >
> >
> https://github.com/apache/geode/pull/3913/commits/6f1814d1f719cc06b13769c40a9d6d01f99f927c
> > > >>>>> <
> > > >>>>>
> > >
> >
> https://github.com/apache/geode/pull/3913/commits/6f1814d1f719cc06b13769c40a9d6d01f99f927c
> > > >
> > > >>>>> in the Geode 1.10 release.
> > > >>>>>>> This commit fixes an issue where a NullPointerException is
> thrown
> > > from
> > > >>>>> PartitionedRegion.getLocalSize() when the statistics callback
> > > sampler is
> > > >>>>> invoked before a PartitionedRegion is initialized.
> > > >>>>>>> - Aaron
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>
> > > >
> > >
> > >
> >
>

Re: [DISCUSS] Controlling event dispatch to AsyncEventListener (review by Aug 22)

[Dan Smith]

Hi Anil,

While I like the idea of matching the API of GatewaySender, I'm not sure I
see how this solves the problem. Is it required of the user to call pause
on the AsyncEventQueue as soon as it is created? How would someone do that
when creating AEQs with xml or cluster configuration? Maybe it would be
better to not dispatch any events until we are done creating all regions?

-Dan

On Fri, Aug 16, 2019 at 2:31 PM Anilkumar Gingade 
wrote:

> Proposal to support controlling capability with event dispatch to
> AsyncEventQueue Listener.
>
> Wiki proposal page:
>
> https://cwiki.apache.org/confluence/display/GEODE/%5BDraft%5D+Controlling+event+dispatch+to+AsyncEventListener
>
> Here is the details from the wiki page:
> *Problem*
>
> *The Geode system requires AEQs to be configured before regions are
> created. If an AEQ listener is operating on a secondary region, this could
> cause listener to operate on a region which is not yet created or fully
> initialized (for region with co-located regions) which could result in
> missing events or dead-lock scenario between region (co-located region)
> creation threads. This scenario is likely to happen during persistence
> recovery; when AEQs are created in the start, the recovered AEQ events are
> dispatched immediately, thus invoking the AEQ listeners.*
> Anti-Goals
>
> None
> *Solution*
>
> *The proposed solution is to provide a way to control dispatching AEQ
> events to the AEQ Listeners, this could be done by adding "pause"  and
> "resume" capability to the AEQ, which will allow application to decide when
> to dispatch events to the listeners. *
>
>
> *The proposal is similar to existing "pause" and "resume" behavior on the
> GatewaySender, on which the AEQ is based on (AEQ implementation is a
> wrapper around GatewaySender). *
> Changes and Additions to Public Interfaces
>
> *The proposed APIs are:*
>
> *On "AsyncEventQueueFactory" interface - *
>
> *AsyncEventQueue pauseEventDispatchToListener();*
>
> *On "AsyncEventQueue" interface -*
>
> *boolean resumeEventDispatchToListener(); **returns true or false if the
> event dispatch is resumed successfully.*
>
>
> *The constraints on the pauseEventDispatchToListener() will remain similar
> to as in "GatewaySender.pause()" :*
>
> "It should be kept in mind that the events will still be getting queued
> into the queue. The scope of this operation is the VM on which it is
> invoked. In case the AEQ is parallel, the AEQ will be paused on individual
> node where this API is called and the AEQ on other VM's can still dispatch
> events. In case the AEQ is not parallel, and the running AEQ on which this
> API is invoked is not primary then primary AEQ will still continue
> dispatching events."
> Performance Impact
>
>
> *This will have similar performance and resource implication as with the
> "GatewaySender.pause()" functionality. If the AEQ is not resumed or kept in
> "pause" state for long, it may start consuming the configured memory and
> overflow it into disk and may cause disk full scenario.*
> Backwards Compatibility and Upgrade Path
>
> *Impact with rolling upgrade: *
>
> *As the api is applicable at individual VM level, there is no message
> serialization changes involved. And only applicable to the events getting
> dispatched to the listeners on that VM. And the AEQ which are replicated
> (for redundancy) continues to work as before.*
>
> *Backward compatibility requirements: *
>
> *None. The AEQs are configured and managed at the server side. There is no
> messaging involved between client/server.*
>
> *Disk formatting changes:*
>
> *None.*
>
> *Deprecation and Application Changes:*
>
>
> *None. If needed, the existing application can be modified to control event
> dispatch with AEQ listener.*
> Prior Art
>
> *Without this, the AEQ listeners operating on other regions could
> experience missing events or dead lock, if there are co-located regions.*
>
> *This approach is simple and can take advantage of the existing
> functionality that is already supported in GatewaySender on which AEQ is
> based on.*
>

Re: Rat check reporting error in travis-ci for geode-native

[Dan Smith]

Hi Helena,

I think the problem is that rat is currently broken on geode-native
develop.

-Dan

On Wed, Aug 21, 2019 at 2:55 PM Helena Bales  wrote:

> Hi Alberto,
>
> If you have added a file as part of your PR, please add the header as a
> commit to your working branch and add that as part of the same PR. Your
> branch should be able to pass a build, spotlessCheck, and unit tests as a
> prerequisite for making a pull request.
>
> Please follow up here if you have more questions.
> Thank you,
> Helena Bales
>
> On Tue, Aug 20, 2019 at 7:40 AM Alberto Gomez 
> wrote:
>
> > Hi,
> >
> > I sent a PR for the geode-native repo and the travis-ci is reporting a
> > Rat error on the following file:
> >
> > ./packer/windows/install-doxygen.ps1
> >
> > The reason is that the copyright header is not included in it. I could
> > add it in my PR but I guess it would be more convenient if this is fixed
> > on a independent commit by someone working on it.
> >
> > Could anyone take care of it?
> >
> > Thanks,
> >
> > - Alberto G.
> >
> >
> >
>

Fwd: [DISCUSS] Improvements on client function execution API

[Dan Smith]

For the two new execute() methods you are proposing, are you also
suggesting that they be implemented on the server?

I took a look at the C++ API. The C++ API also has a
ResultCollector.getResult(timeout) method. Does that timeout do anything,
given that execute(timeout) already waited for the given timeout?

Long term, are you thinking that we should remove execute() with no
parameters and getResult(timeout) and just have these two blocking
execute(timeout) methods on the server and clients? I think you are right
that we shouldn't change the behavior of the existing execute() to avoid
breaking our users. But I also want to understand what our long term plan
is for a consistent and sensible API.

-Dan

Propose including GEODE-7085 in 1.10

[Dan Smith]

Hi,

I'd like to propose including the fixes for GEODE-7085 into 1.10 (SHA's
below). This is not a new issue, but it does result not being able to
recover from disk without this fix if a cluster has more than 2 billion
updates to a single bucket from a single member.

SHAs: f587101  ,  f17931bf

Thanks,
-Dan

Re: [DISCUSS] Pulling the current proposed 1.10 release until we can agree on develop being stable

[Dan Smith]

+1 to creating RC1 with the current release/1.10 branch this week.

I don't see a fundamental problem with cherry-picking some targeted and
tested fixes to release/1.10, based on our assessment of the risk to
customers vs. the risk of destabilizing the branch. I think release/1.10 is
in a good state, and we should go ahead with the release.

-Dan


On Tue, Aug 27, 2019 at 9:28 AM Bruce Schuchardt 
wrote:

> The "develop" branch has a refactoring of membership code that should
> not be included in 1.10.  I waited until the release branch was cut to
> push these changes.
>
> On 8/26/19 4:06 PM, Udo Kohlmeyer wrote:
> > Hi there Apache Geode devs,
> >
> > It has been some weeks since the proposed 1.10 release was cut. We've
> > gone through a few cycles where we keep on submitting "please include
> > ticket GEODE-XXX" because it is critical and will break the system.
> > WHICH in reality tells me that current develop is broken and unstable.
> >
> > I'm going to suggest that we abandon the current 1.10 release branch.
> > I cannot shake the feeling that our continuous cherry picking into a
> > branch will result in either the branch becoming unmaintainable, given
> > we have only select fixes in the branch OR we end up with a branch
> > that is more stable than our current development branch, which would
> > result in us having to rebase the develop branch onto the 1.10 branch.
> >
> > I propose that once we see the pipeline is clean and green for a solid
> > we then again attempt to cut 1.10 branch.
> >
> > We CANNOT continue adding to a branch in order to stabilize it.. It
> > just means the branch we cut from is unstable. If we cannot cut a
> > branch from develop without having to have weeks of stabilization
> > cycles, then our main branch is broken...
> >
> > Either way, not a good spot to be in.
> >
> > Thoughts?
> >
> > --Udo
> >
>