Build failed in Jenkins: Geode-nightly-flaky #183

2017-11-30 Thread Apache Jenkins Server 
See 


Changes:

[github] GEODE-3539: add ability to stop a vm without clean out the working dir

[jiliao] GEODE-3788: add alter async-event-queue command and tests

[github] GEODE-4000: The serializerClass is actually introduced in 1.4 not 1.3.

[github] GEODE-4011: Ensure that LogWrapper is closed correctly in

[boglesby] GEODE-3987: enforce GatewayReceiver uniqueness per member

[github] GEODE-1897 Docs:  configure eviction through gfsh (#1098)

[github] GEODE-3788: alter async event queue on a locator with no cluster config

[github] GEODE-3539: add test coverage for "create async-event-queue" and "lis…

[github] GEODE-3539: enhance rule to start locator joining other locators 
(#1104)

[metatype] GEODE-4023: Add precheckin tests to pipeline.

[github] GEODE-1683: fix ClientAuthorizationDUnit test failures (#1106)

[dbarnes] GEODE-1897 Docs for gfsh eviction, minor correction.

[bschuchardt] GEODE-3923 Provide whitelist/blacklist capability for java 
serialization

[bschuchardt] GEODE-4000: The serializerClass is actually introduced in 1.4 not 
1.3

[bschuchardt] GEODE-3923 Provide whitelist/blacklist capability for java 
serialization

--
[...truncated 139.55 KB...]
Download 
https://repo1.maven.org/maven2/org/springframework/plugin/spring-plugin-core/1.2.0.RELEASE/spring-plugin-core-1.2.0.RELEASE.jar
Download 
https://repo1.maven.org/maven2/org/springframework/plugin/spring-plugin-metadata/1.2.0.RELEASE/spring-plugin-metadata-1.2.0.RELEASE.jar
Download 
https://repo1.maven.org/maven2/org/mapstruct/mapstruct/1.0.0.Final/mapstruct-1.0.0.Final.jar
Download 
https://repo1.maven.org/maven2/com/thoughtworks/paranamer/paranamer/2.8/paranamer-2.8.jar
Download 
https://repo1.maven.org/maven2/io/springfox/springfox-core/2.6.1/springfox-core-2.6.1.jar
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:geode-web-api:processResources
:geode-web-api:classes
:geode-assembly:docs
:geode-assembly:gfshDepsJar
:geode-client-protocol:javadoc
:geode-client-protocol:javadocJar
:geode-client-protocol:sourcesJar
:geode-client-protocol:signArchives SKIPPED
:geode-common:javadocJar
:geode-common:sourcesJar
:geode-common:signArchives SKIPPED
:geode-core:javadocJar
:geode-core:raJar
:geode-core:jcaJar
:geode-core:sourcesJar
:geode-core:signArchives SKIPPED
:geode-core:webJar
:geode-cq:jar
:geode-cq:javadoc
:geode-cq:javadocJar
:geode-cq:sourcesJar
:geode-cq:signArchives SKIPPED
:geode-json:javadocJar
:geode-json:sourcesJar
:geode-json:signArchives SKIPPED
:geode-lucene:jar
:geode-lucene:javadoc
:geode-lucene:javadocJar
:geode-lucene:sourcesJar
:geode-lucene:signArchives SKIPPED
:geode-old-client-support:jar
:geode-old-client-support:javadoc
:geode-old-client-support:javadocJar
:geode-old-client-support:sourcesJar
:geode-old-client-support:signArchives SKIPPED
:geode-protobuf:jar
:geode-protobuf:javadoc
:geode-protobuf:javadocJar
:geode-protobuf:sourcesJar
:geode-protobuf:signArchives SKIPPED
:geode-protobuf:zip
:geode-pulse:javadoc
:geode-pulse:javadocJar
:geode-pulse:sourcesJar
:geode-pulse:war
:geode-pulse:signArchives SKIPPED
:geode-rebalancer:jar
:geode-rebalancer:javadoc
:geode-rebalancer:javadocJar
:geode-rebalancer:sourcesJar
:geode-rebalancer:signArchives SKIPPED
:geode-wan:jar
:geode-wan:javadoc
:geode-wan:javadocJar
:geode-wan:sourcesJar
:geode-wan:signArchives SKIPPED
:geode-web:javadoc NO-SOURCE
:geode-web:javadocJar
:geode-web:sourcesJar
:geode-web:war
:geode-web:signArchives SKIPPED
:geode-web-api:javadoc
:geode-web-api:javadocJar
:geode-web-api:sourcesJar
:geode-web-api:war
:geode-web-api:signArchives SKIPPED
:geode-assembly:installDist
:geode-pulse:jar
:geode-assembly:compileTestJava
Download 
https://repo1.maven.org/maven2/org/codehaus/cargo/cargo-core-uberjar/1.6.3/cargo-core-uberjar-1.6.3.pom
Download 
https://repo1.maven.org/maven2/org/codehaus/cargo/cargo-core/1.6.3/cargo-core-1.6.3.pom
Download 
https://repo1.maven.org/maven2/org/codehaus/cargo/codehaus-cargo/1.6.3/codehaus-cargo-1.6.3.pom
Download 
https://repo1.maven.org/maven2/commons-discovery/commons-discovery/0.5/commons-discovery-0.5.pom
Download https://repo1.maven.org/maven2/org/apache/ant/ant/1.7.1/ant-1.7.1.pom
Download 
https://repo1.maven.org/maven2/org/apache/ant/ant-parent/1.7.1/ant-parent-1.7.1.pom
Download 
https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.7.1/ant-launcher-1.7.1.pom
Download 
https://repo1.maven.org/maven2/org/codehaus/cargo/cargo-core-uberjar/1.6.3/cargo-core-uberjar-1.6.3.jar
Download 
https://repo1.maven.org/maven2/commons-discovery/commons-discovery/0.5/commons-discovery-0.5.jar
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unche

Build failed in Jenkins: Geode-nightly #1029

2017-11-30 Thread Apache Jenkins Server 
See 


Changes:

[github] GEODE-3539: add ability to stop a vm without clean out the working dir

[jiliao] GEODE-3788: add alter async-event-queue command and tests

[github] GEODE-4000: The serializerClass is actually introduced in 1.4 not 1.3.

[github] GEODE-4011: Ensure that LogWrapper is closed correctly in

[boglesby] GEODE-3987: enforce GatewayReceiver uniqueness per member

[github] GEODE-1897 Docs:  configure eviction through gfsh (#1098)

[github] GEODE-3788: alter async event queue on a locator with no cluster config

[github] GEODE-3539: add test coverage for "create async-event-queue" and "lis…

--
[...truncated 244.31 KB...]
:geode-cq:processTestResources
:geode-cq:testClasses
:geode-cq:checkMissedTests
:geode-cq:spotlessJavaCheck
:geode-cq:spotlessCheck
:geode-cq:test
:geode-cq:check
:geode-cq:build
:geode-cq:distributedTest
:geode-cq:integrationTest
:geode-json:assemble
:geode-json:compileTestJava NO-SOURCE
:geode-json:processTestResources
:geode-json:testClasses
:geode-json:checkMissedTests NO-SOURCE
:geode-json:spotlessJavaCheck
:geode-json:spotlessCheck
:geode-json:test NO-SOURCE
:geode-json:check
:geode-json:build
:geode-json:distributedTest NO-SOURCE
:geode-json:integrationTest NO-SOURCE
:geode-junit:javadoc
:geode-junit:javadocJar
:geode-junit:sourcesJar
:geode-junit:signArchives SKIPPED
:geode-junit:assemble
:geode-junit:compileTestJavaNote: 

 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: 

 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-junit:processTestResources
:geode-junit:testClasses
:geode-junit:checkMissedTests
:geode-junit:spotlessJavaCheck
:geode-junit:spotlessCheck
:geode-junit:test
:geode-junit:check
:geode-junit:build
:geode-junit:distributedTest
:geode-junit:integrationTest
:geode-lucene:assemble
:geode-lucene:compileTestJavaNote: Some input files use or override a 
deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-lucene:processTestResources
:geode-lucene:testClasses
:geode-lucene:checkMissedTests
:geode-lucene:spotlessJavaCheck
:geode-lucene:spotlessCheck
:geode-lucene:test
:geode-lucene:check
:geode-lucene:build
:geode-lucene:distributedTest
:geode-lucene:integrationTest
:geode-old-client-support:assemble
:geode-old-client-support:compileTestJavaNote: 

 uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

:geode-old-client-support:processTestResources NO-SOURCE
:geode-old-client-support:testClasses
:geode-old-client-support:checkMissedTests
:geode-old-client-support:spotlessJavaCheck
:geode-old-client-support:spotlessCheck
:geode-old-client-support:test
:geode-old-client-support:check
:geode-old-client-support:build
:geode-old-client-support:distributedTest
:geode-old-client-support:integrationTest
:geode-old-versions:distributedTest NO-SOURCE
:geode-old-versions:integrationTest NO-SOURCE
:geode-protobuf:assemble
:geode-protobuf:extractIncludeTestProto
:geode-protobuf:extractTestProto UP-TO-DATE
:geode-protobuf:generateTestProto NO-SOURCE
:geode-protobuf:compileTestJavaNote: Some input files use or override a 
deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-protobuf:processTestResources
:geode-protobuf:testClasses
:geode-protobuf:checkMissedTests
:geode-protobuf:spotlessJavaCheck
:geode-protobuf:spotlessCheck
:geode-protobuf:test
:geode-protobuf:check
:geode-protobuf:build
:geode-protobuf:distributedTest
:geode-protobuf:integrationTest
:geode-pulse:assemble
:geode-pulse:compileTestJavaNote: Some input files use or override a deprecated 
API.
Note: Recompile with -Xlint:deprecation for details.
Note: 

 uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

:geode-pulse:processTestResources
:geode-pulse:testClasses
:geode-pulse:checkMissedTests
:geode-pulse:spotlessJavaCheck
:geode-pulse:spotlessCheck
:geode-pulse:test
:geode-pulse:check
:geode-pulse:build
:geode-pulse:distributedTest
:geode-pulse:integrationTest
:geode-rebalancer:assemble
:geode-rebalancer:compileTestJav

Re: [DISCUSS] changes to registerInterest API

2017-11-30 Thread Dan Smith 
I think it should be registerInterestAll(Iterablekeys) to mirror
Collection.addAll.

I could easily see a user creating their own Tuple class or using one that
happens to also implement Iterable as their key.

We're not doing a very good job of discouraging people to use iterable
objects as their key if they work everywhere else and then break with this
one API.

-Dan

On Thu, Nov 30, 2017 at 2:45 PM, John Blum  wrote:

> No, no... good question.
>
> I just think it would be wiser if users created a single, CompositeKey
> class type, with properly implements equals and hashCode methods, as I
> pointed out.
>
> I don't see any advantage in using a java.util.Collection as a key over
> implementing a CompositeKey type.
>
> As such, anything we can do to discourage users from using Collection types
> as a key, I think is a good thing.
>
>
> On Thu, Nov 30, 2017 at 2:35 PM, Jason Huynh  wrote:
>
> > Yeah I am not sure if anyone does it,and I don't think it would be a good
> > idea to use a collection as a key but just thought I'd ask the
> question...
> >
> > On Thu, Nov 30, 2017 at 2:33 PM John Blum  wrote:
> >
> > > For instance, this came up recently...
> > >
> > >
> > > https://stackoverflow.com/questions/46551278/gemfire-
> > composite-key-pojo-as-gemfire-key
> > >
> > > I have seen other similar posts too!
> > >
> > >
> > >
> > > On Thu, Nov 30, 2017 at 2:30 PM, John Blum  wrote:
> > >
> > > > Does anyone actually do this in practice?  If so, yikes!
> > > >
> > > > Even if the List is immutable, the elements may not be, so using a
> List
> > > as
> > > > a key starts to open 1 up to a lot of problems.
> > > >
> > > > As others have pointed out in SO and other channels, information
> should
> > > > not be kept in the key.
> > > >
> > > > It is perfect fine to have a "Composite" Key, but then define a
> > > > CompositeKey class type with properly implemented equals(:Object) and
> > > > hashCode():int methods.
> > > >
> > > > For the most part, Keys should really only ever be simple Scalar
> values
> > > > (e.g. Long, String, etc).
> > > >
> > > > -j
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Nov 30, 2017 at 2:25 PM, Jason Huynh 
> > wrote:
> > > >
> > > >> I started work on the following plan:
> > > >> - deprecate current "ALL_KEYS" and List passing behavior in
> > > >> registerInterest
> > > >> ()
> > > >> - add registerInterestForAllKeys();
> > > >> - add registerInterest(T... keys)
> > > >> - add registerInterest(Iterablekeys)
> > > >>
> > > >> I might be missing something here but:
> > > >> With the addition of registerInterest(Iterable keys), I think we
> > > would
> > > >> not be able to register interest a List as the key itself.  A list
> > would
> > > >> be
> > > >> iterated over due to the addition of registerInterest(Iterable
> > keys).
> > > >> A
> > > >> list in a list would be passed into registerInterest and again be
> > > iterated
> > > >> over.  I could change the newly created registerInterest call and
> > > >> explicitly name it something else or are we ok with Iterables not
> > being
> > > >> able to be registered as individual keys.
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> On Mon, Nov 20, 2017 at 9:05 AM Kirk Lund  wrote:
> > > >>
> > > >> > John's approach looks best for when you need to specify keys.
> > > >> >
> > > >> > For ALL_KEYS, what about an API that doesn't require a token or
> all
> > > >> keys:
> > > >> >
> > > >> > public void registerInterestForAllKeys();
> > > >> >
> > > >> > On Fri, Nov 17, 2017 at 1:24 PM, Jason Huynh 
> > > wrote:
> > > >> >
> > > >> > > Thanks John for the clarification!
> > > >> > >
> > > >> > > On Fri, Nov 17, 2017 at 1:12 PM John Blum 
> > wrote:
> > > >> > >
> > > >> > > > This...
> > > >> > > >
> > > >> > > > > The Iterable version would handle any collection type by
> > having
> > > >> the
> > > >> > > user
> > > >> > > > pass
> > > >> > > > in the iterator for the collection.
> > > >> > > >
> > > >> > > > Is not correct.
> > > >> > > >
> > > >> > > > The Collection interface itself "extends" the
> > > >> java.lang.Iterable
> > > >> > > > interface (see here...
> > > >> > > >
> > > https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html
> > > >> > > under
> > > >> > > > "*All
> > > >> > > > Superinterfaces*").
> > > >> > > >
> > > >> > > > Therefore a user can simply to this...
> > > >> > > >
> > > >> > > > *List* keys = ...
> > > >> > > >
> > > >> > > > region.registerInterest(keys); *// calls the
> > > >> > > > Region.registerInterest(:Iterable) method.*
> > > >> > > >
> > > >> > > > Alternatively, this would also be allowed...
> > > >> > > >
> > > >> > > > *Set* keys = ...
> > > >> > > >
> > > >> > > > region.registerInterest(keys);
> > > >> > > >
> > > >> > > >
> > > >> > > > On Fri, Nov 17, 2017 at 11:44 AM, Jason Huynh <
> > jhu...@pivotal.io>
> > > >> > wrote:
> > > >> > > >
> > > >> > > > > Current idea is to:
> > > >> > > > > - deprecate current "ALL_KEYS" and List passing behavior in
> > > >> > > > > regi

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #752 was SUCCESSFUL (with 2264 tests). Change made by Oliver Gierke.

2017-11-30 Thread Spring CI 

---
Spring Data GemFire > Nightly-ApacheGeode > #752 was successful.
---
Scheduled with changes by Oliver Gierke.
2266 tests in total.

https://build.spring.io/browse/SGF-NAG-752/




--
Code Changes
--
Oliver Gierke (db0977f8eb01b7a4db7641ad9f66429420f938f7):

>DATAGEODE-70 - Reduced documentation build time.
>We now don't include the namespace schema files into the reference 
>documentation anymore as they span multiple pages in the PDF and thus cause 
>heavy load in the PDF conversion trying to lay those out properly [0]. 
>Switched to a simple link pointing to the schemas for now.
>
>[0] https://twitter.com/olivergierke/status/935818047385948160



--
This message is automatically generated by Atlassian Bamboo

Re: [DISCUSS] changes to registerInterest API

2017-11-30 Thread John Blum 
No, no... good question.

I just think it would be wiser if users created a single, CompositeKey
class type, with properly implements equals and hashCode methods, as I
pointed out.

I don't see any advantage in using a java.util.Collection as a key over
implementing a CompositeKey type.

As such, anything we can do to discourage users from using Collection types
as a key, I think is a good thing.


On Thu, Nov 30, 2017 at 2:35 PM, Jason Huynh  wrote:

> Yeah I am not sure if anyone does it,and I don't think it would be a good
> idea to use a collection as a key but just thought I'd ask the question...
>
> On Thu, Nov 30, 2017 at 2:33 PM John Blum  wrote:
>
> > For instance, this came up recently...
> >
> >
> > https://stackoverflow.com/questions/46551278/gemfire-
> composite-key-pojo-as-gemfire-key
> >
> > I have seen other similar posts too!
> >
> >
> >
> > On Thu, Nov 30, 2017 at 2:30 PM, John Blum  wrote:
> >
> > > Does anyone actually do this in practice?  If so, yikes!
> > >
> > > Even if the List is immutable, the elements may not be, so using a List
> > as
> > > a key starts to open 1 up to a lot of problems.
> > >
> > > As others have pointed out in SO and other channels, information should
> > > not be kept in the key.
> > >
> > > It is perfect fine to have a "Composite" Key, but then define a
> > > CompositeKey class type with properly implemented equals(:Object) and
> > > hashCode():int methods.
> > >
> > > For the most part, Keys should really only ever be simple Scalar values
> > > (e.g. Long, String, etc).
> > >
> > > -j
> > >
> > >
> > >
> > >
> > > On Thu, Nov 30, 2017 at 2:25 PM, Jason Huynh 
> wrote:
> > >
> > >> I started work on the following plan:
> > >> - deprecate current "ALL_KEYS" and List passing behavior in
> > >> registerInterest
> > >> ()
> > >> - add registerInterestForAllKeys();
> > >> - add registerInterest(T... keys)
> > >> - add registerInterest(Iterablekeys)
> > >>
> > >> I might be missing something here but:
> > >> With the addition of registerInterest(Iterable keys), I think we
> > would
> > >> not be able to register interest a List as the key itself.  A list
> would
> > >> be
> > >> iterated over due to the addition of registerInterest(Iterable
> keys).
> > >> A
> > >> list in a list would be passed into registerInterest and again be
> > iterated
> > >> over.  I could change the newly created registerInterest call and
> > >> explicitly name it something else or are we ok with Iterables not
> being
> > >> able to be registered as individual keys.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On Mon, Nov 20, 2017 at 9:05 AM Kirk Lund  wrote:
> > >>
> > >> > John's approach looks best for when you need to specify keys.
> > >> >
> > >> > For ALL_KEYS, what about an API that doesn't require a token or all
> > >> keys:
> > >> >
> > >> > public void registerInterestForAllKeys();
> > >> >
> > >> > On Fri, Nov 17, 2017 at 1:24 PM, Jason Huynh 
> > wrote:
> > >> >
> > >> > > Thanks John for the clarification!
> > >> > >
> > >> > > On Fri, Nov 17, 2017 at 1:12 PM John Blum 
> wrote:
> > >> > >
> > >> > > > This...
> > >> > > >
> > >> > > > > The Iterable version would handle any collection type by
> having
> > >> the
> > >> > > user
> > >> > > > pass
> > >> > > > in the iterator for the collection.
> > >> > > >
> > >> > > > Is not correct.
> > >> > > >
> > >> > > > The Collection interface itself "extends" the
> > >> java.lang.Iterable
> > >> > > > interface (see here...
> > >> > > >
> > https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html
> > >> > > under
> > >> > > > "*All
> > >> > > > Superinterfaces*").
> > >> > > >
> > >> > > > Therefore a user can simply to this...
> > >> > > >
> > >> > > > *List* keys = ...
> > >> > > >
> > >> > > > region.registerInterest(keys); *// calls the
> > >> > > > Region.registerInterest(:Iterable) method.*
> > >> > > >
> > >> > > > Alternatively, this would also be allowed...
> > >> > > >
> > >> > > > *Set* keys = ...
> > >> > > >
> > >> > > > region.registerInterest(keys);
> > >> > > >
> > >> > > >
> > >> > > > On Fri, Nov 17, 2017 at 11:44 AM, Jason Huynh <
> jhu...@pivotal.io>
> > >> > wrote:
> > >> > > >
> > >> > > > > Current idea is to:
> > >> > > > > - deprecate current "ALL_KEYS" and List passing behavior in
> > >> > > > > registerInterest()
> > >> > > > > - add registerInterestAllKeys();
> > >> > > > > - add registerInterest(T... keys) and
> > registerInterest(Iterable
> > >> > > keys)
> > >> > > > > and
> > >> > > > > not have one specifically for List or specific collections.
> > >> > > > >
> > >> > > > > The Iterable version would handle any collection type by
> having
> > >> the
> > >> > > user
> > >> > > > > pass in the iterator for the collection.
> > >> > > > >
> > >> > > > > On Fri, Nov 17, 2017 at 11:32 AM Jacob Barrett <
> > >> jbarr...@pivotal.io>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > I am failing to see where registerInterest(List keys) is
> an
> > >> > issue
> > >> > > > for
> > >> > > > > > the

Re: [DISCUSS] changes to registerInterest API

2017-11-30 Thread Jason Huynh 
Yeah I am not sure if anyone does it,and I don't think it would be a good
idea to use a collection as a key but just thought I'd ask the question...

On Thu, Nov 30, 2017 at 2:33 PM John Blum  wrote:

> For instance, this came up recently...
>
>
> https://stackoverflow.com/questions/46551278/gemfire-composite-key-pojo-as-gemfire-key
>
> I have seen other similar posts too!
>
>
>
> On Thu, Nov 30, 2017 at 2:30 PM, John Blum  wrote:
>
> > Does anyone actually do this in practice?  If so, yikes!
> >
> > Even if the List is immutable, the elements may not be, so using a List
> as
> > a key starts to open 1 up to a lot of problems.
> >
> > As others have pointed out in SO and other channels, information should
> > not be kept in the key.
> >
> > It is perfect fine to have a "Composite" Key, but then define a
> > CompositeKey class type with properly implemented equals(:Object) and
> > hashCode():int methods.
> >
> > For the most part, Keys should really only ever be simple Scalar values
> > (e.g. Long, String, etc).
> >
> > -j
> >
> >
> >
> >
> > On Thu, Nov 30, 2017 at 2:25 PM, Jason Huynh  wrote:
> >
> >> I started work on the following plan:
> >> - deprecate current "ALL_KEYS" and List passing behavior in
> >> registerInterest
> >> ()
> >> - add registerInterestForAllKeys();
> >> - add registerInterest(T... keys)
> >> - add registerInterest(Iterablekeys)
> >>
> >> I might be missing something here but:
> >> With the addition of registerInterest(Iterable keys), I think we
> would
> >> not be able to register interest a List as the key itself.  A list would
> >> be
> >> iterated over due to the addition of registerInterest(Iterable keys).
> >> A
> >> list in a list would be passed into registerInterest and again be
> iterated
> >> over.  I could change the newly created registerInterest call and
> >> explicitly name it something else or are we ok with Iterables not being
> >> able to be registered as individual keys.
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Nov 20, 2017 at 9:05 AM Kirk Lund  wrote:
> >>
> >> > John's approach looks best for when you need to specify keys.
> >> >
> >> > For ALL_KEYS, what about an API that doesn't require a token or all
> >> keys:
> >> >
> >> > public void registerInterestForAllKeys();
> >> >
> >> > On Fri, Nov 17, 2017 at 1:24 PM, Jason Huynh 
> wrote:
> >> >
> >> > > Thanks John for the clarification!
> >> > >
> >> > > On Fri, Nov 17, 2017 at 1:12 PM John Blum  wrote:
> >> > >
> >> > > > This...
> >> > > >
> >> > > > > The Iterable version would handle any collection type by having
> >> the
> >> > > user
> >> > > > pass
> >> > > > in the iterator for the collection.
> >> > > >
> >> > > > Is not correct.
> >> > > >
> >> > > > The Collection interface itself "extends" the
> >> java.lang.Iterable
> >> > > > interface (see here...
> >> > > >
> https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html
> >> > > under
> >> > > > "*All
> >> > > > Superinterfaces*").
> >> > > >
> >> > > > Therefore a user can simply to this...
> >> > > >
> >> > > > *List* keys = ...
> >> > > >
> >> > > > region.registerInterest(keys); *// calls the
> >> > > > Region.registerInterest(:Iterable) method.*
> >> > > >
> >> > > > Alternatively, this would also be allowed...
> >> > > >
> >> > > > *Set* keys = ...
> >> > > >
> >> > > > region.registerInterest(keys);
> >> > > >
> >> > > >
> >> > > > On Fri, Nov 17, 2017 at 11:44 AM, Jason Huynh 
> >> > wrote:
> >> > > >
> >> > > > > Current idea is to:
> >> > > > > - deprecate current "ALL_KEYS" and List passing behavior in
> >> > > > > registerInterest()
> >> > > > > - add registerInterestAllKeys();
> >> > > > > - add registerInterest(T... keys) and
> registerInterest(Iterable
> >> > > keys)
> >> > > > > and
> >> > > > > not have one specifically for List or specific collections.
> >> > > > >
> >> > > > > The Iterable version would handle any collection type by having
> >> the
> >> > > user
> >> > > > > pass in the iterator for the collection.
> >> > > > >
> >> > > > > On Fri, Nov 17, 2017 at 11:32 AM Jacob Barrett <
> >> jbarr...@pivotal.io>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > I am failing to see where registerInterest(List keys) is an
> >> > issue
> >> > > > for
> >> > > > > > the key type in the region. If our region is Region
> >> then I
> >> > > > would
> >> > > > > > expect registerInterest(List). If the keys are unknown
> >> or a
> >> > > mix
> >> > > > > > then you should have Region and thus
> >> > > > > registerInterest(List >> > > > > >
> >> > > > > > I echo John's statements on VarArgs and type erasure as well
> as
> >> his
> >> > > > > > argument for Iterable.
> >> > > > > >
> >> > > > > > Also, List does not restrict you from List indexes. The
> >> region
> >> > > would
> >> > > > > be
> >> > > > > > Region> with registerInterest >> ing>>().
> >> > > > > >
> >> > > > > > -Jake
> >> > > > > >
> >> > > > > >
> >> > > > > > On Fri, Nov 17, 2017 at 10:04 AM John Blum 
> >> > wrote:
> >> > > > > >
> >> > > > > > > Personally, I pref

please read: geode-3923

2017-11-30 Thread Bruce Schuchardt 
GEODE-3923 changes have been merged to develop life as you know it will 
change a bit.  The merge implements serialization 
white-listing/blacklisting and to insure that we have white-listed the 
correct set of classes, and that no new Geode classes are introduced 
that aren't properly registered, *all unit tests will be run with 
white-listing enabled*.


*What is white-listing?*  It's basically telling Geode to only 
deserialize instances of known classes.  You turn it on with a new cache 
property, validate-serializable-objects.  You white-list a 
java-serializable class by creating a pattern that matches your class 
using the new cache property serializable-object-filter.  There are 
numerous examples of this in the merged test code.


DataSerializable classes do not need to be whitelisted unless they are 
sometimes java-serialized, such as being a value in an array of Objects 
that is serialized with DataSerializer.writeObject().  I've run into a 
few of those.


*Example:*

validate-serializable-objects=true  (default is false except for 
distributedTests)


serializable-object-filter=org.apache.geode.test.query.TestObject;org.apache.geode.test.functions.*


*What happens if you neglect to whitelist your class?*  Your test will 
fail and you will see an InvalidClassException in your test output.  You 
should search for "rejecting" in the output to find the classes that you 
need to whitelist.


*How are Geode classes white-listed?*  Everything that is in 
sanctionedSerializables.txt is automatically white-listed.  The 
AnalyzeSerializablesJUnitTest has new test methods that ensure that all 
classes in the "sanctioned" file can be serialized and deserialized.  It 
also ensures that classes listed in the excludedClasses.txt file can not 
be serialized and deserialized and that none of them are also in the 
sanctionedSerializables.txt white-list.


*Which commit did this to me?*

a2bd5788d0b0728d91d551ddaac878b336d7892a was the merge to develop.  The 
commit message has additional details:


commit a2bd5788d0b0728d91d551ddaac878b336d7892a
Author: Bruce Schuchardt 
Date:   Thu Nov 30 14:25:48 2017 -0800

    GEODE-3923 Provide whitelist/blacklist capability for java 
serialization


    This is a squashed and rebased commit of the work from the now-defunct
    whitelist-wip branch.  The work allows you to whitelist and blacklist
    use of Java deserialization for classes matching a pattern you provide.

    All distributedTests are now run with this enabled and with the default
    blacklist pattern of "!*", which blacklists everything not explicitely
    allowed by Geode in the sanctionedSerializables.txt files or by 
your test.


    Each layer needing one now has its own sanctionedSerializables.txt file
    that is a resource in the product tree.  These are installed as 
whitelists

    by a new DistributedSystemService in that layer.

    There are numerous examples of whitelisting classes in the tests.

    If you do not whitelist your class Geode will throw an 
IncompatibleClassException.
    If you happen to run into this look in your logs for "rejecting" 
and you

    can easily find the name of the offending class and add it to your
    whitelist.

    Two new cache properties have been added:

  validate-serializable-objects

    If true checks incoming java serializable objects against a filter 
(allows

    internal Geode classes and any others provided in the
    serializable-object-filter property).

    If you enable this property you must be using Java 8 build 121 or 
later.

    If you are not Geode will throw an exception and refuse to start.

    Default: "false"

  serializable-object-filter

    A user provided whitelist of objects that the system will allow to 
serialize.
    See java.io.ObjectInputFilter.Config for details on the syntax for 
creating filters.


https://docs.oracle.com/javase/9/docs/api/java/io/ObjectInputFilter.Config.html

    Default: "!*"

Re: [DISCUSS] changes to registerInterest API

2017-11-30 Thread John Blum 
For instance, this came up recently...

https://stackoverflow.com/questions/46551278/gemfire-composite-key-pojo-as-gemfire-key

I have seen other similar posts too!



On Thu, Nov 30, 2017 at 2:30 PM, John Blum  wrote:

> Does anyone actually do this in practice?  If so, yikes!
>
> Even if the List is immutable, the elements may not be, so using a List as
> a key starts to open 1 up to a lot of problems.
>
> As others have pointed out in SO and other channels, information should
> not be kept in the key.
>
> It is perfect fine to have a "Composite" Key, but then define a
> CompositeKey class type with properly implemented equals(:Object) and
> hashCode():int methods.
>
> For the most part, Keys should really only ever be simple Scalar values
> (e.g. Long, String, etc).
>
> -j
>
>
>
>
> On Thu, Nov 30, 2017 at 2:25 PM, Jason Huynh  wrote:
>
>> I started work on the following plan:
>> - deprecate current "ALL_KEYS" and List passing behavior in
>> registerInterest
>> ()
>> - add registerInterestForAllKeys();
>> - add registerInterest(T... keys)
>> - add registerInterest(Iterablekeys)
>>
>> I might be missing something here but:
>> With the addition of registerInterest(Iterable keys), I think we would
>> not be able to register interest a List as the key itself.  A list would
>> be
>> iterated over due to the addition of registerInterest(Iterable keys).
>> A
>> list in a list would be passed into registerInterest and again be iterated
>> over.  I could change the newly created registerInterest call and
>> explicitly name it something else or are we ok with Iterables not being
>> able to be registered as individual keys.
>>
>>
>>
>>
>>
>> On Mon, Nov 20, 2017 at 9:05 AM Kirk Lund  wrote:
>>
>> > John's approach looks best for when you need to specify keys.
>> >
>> > For ALL_KEYS, what about an API that doesn't require a token or all
>> keys:
>> >
>> > public void registerInterestForAllKeys();
>> >
>> > On Fri, Nov 17, 2017 at 1:24 PM, Jason Huynh  wrote:
>> >
>> > > Thanks John for the clarification!
>> > >
>> > > On Fri, Nov 17, 2017 at 1:12 PM John Blum  wrote:
>> > >
>> > > > This...
>> > > >
>> > > > > The Iterable version would handle any collection type by having
>> the
>> > > user
>> > > > pass
>> > > > in the iterator for the collection.
>> > > >
>> > > > Is not correct.
>> > > >
>> > > > The Collection interface itself "extends" the
>> java.lang.Iterable
>> > > > interface (see here...
>> > > > https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html
>> > > under
>> > > > "*All
>> > > > Superinterfaces*").
>> > > >
>> > > > Therefore a user can simply to this...
>> > > >
>> > > > *List* keys = ...
>> > > >
>> > > > region.registerInterest(keys); *// calls the
>> > > > Region.registerInterest(:Iterable) method.*
>> > > >
>> > > > Alternatively, this would also be allowed...
>> > > >
>> > > > *Set* keys = ...
>> > > >
>> > > > region.registerInterest(keys);
>> > > >
>> > > >
>> > > > On Fri, Nov 17, 2017 at 11:44 AM, Jason Huynh 
>> > wrote:
>> > > >
>> > > > > Current idea is to:
>> > > > > - deprecate current "ALL_KEYS" and List passing behavior in
>> > > > > registerInterest()
>> > > > > - add registerInterestAllKeys();
>> > > > > - add registerInterest(T... keys) and registerInterest(Iterable
>> > > keys)
>> > > > > and
>> > > > > not have one specifically for List or specific collections.
>> > > > >
>> > > > > The Iterable version would handle any collection type by having
>> the
>> > > user
>> > > > > pass in the iterator for the collection.
>> > > > >
>> > > > > On Fri, Nov 17, 2017 at 11:32 AM Jacob Barrett <
>> jbarr...@pivotal.io>
>> > > > > wrote:
>> > > > >
>> > > > > > I am failing to see where registerInterest(List keys) is an
>> > issue
>> > > > for
>> > > > > > the key type in the region. If our region is Region
>> then I
>> > > > would
>> > > > > > expect registerInterest(List). If the keys are unknown
>> or a
>> > > mix
>> > > > > > then you should have Region and thus
>> > > > > registerInterest(List> > > > > >
>> > > > > > I echo John's statements on VarArgs and type erasure as well as
>> his
>> > > > > > argument for Iterable.
>> > > > > >
>> > > > > > Also, List does not restrict you from List indexes. The
>> region
>> > > would
>> > > > > be
>> > > > > > Region> with registerInterest> ing>>().
>> > > > > >
>> > > > > > -Jake
>> > > > > >
>> > > > > >
>> > > > > > On Fri, Nov 17, 2017 at 10:04 AM John Blum 
>> > wrote:
>> > > > > >
>> > > > > > > Personally, I prefer the var args method
>> (registerInterest(T...
>> > > > keys))
>> > > > > > > myself.  It is way more convenient if I only have a few keys
>> when
>> > > > > calling
>> > > > > > > this method then to have to add the keys to a List, especially
>> > for
>> > > > > > testing
>> > > > > > > purposes.
>> > > > > > >
>> > > > > > > But, I typically like to pair that with a
>> > > > registerInterest(Iterable
>> > > > > > > keys) method
>> > > > > > > as well.  By having a overloaded Iterable variant, then I can
>> >

Re: [DISCUSS] changes to registerInterest API

2017-11-30 Thread John Blum 
Does anyone actually do this in practice?  If so, yikes!

Even if the List is immutable, the elements may not be, so using a List as
a key starts to open 1 up to a lot of problems.

As others have pointed out in SO and other channels, information should not
be kept in the key.

It is perfect fine to have a "Composite" Key, but then define a CompositeKey
class type with properly implemented equals(:Object) and hashCode():int
methods.

For the most part, Keys should really only ever be simple Scalar values
(e.g. Long, String, etc).

-j




On Thu, Nov 30, 2017 at 2:25 PM, Jason Huynh  wrote:

> I started work on the following plan:
> - deprecate current "ALL_KEYS" and List passing behavior in
> registerInterest
> ()
> - add registerInterestForAllKeys();
> - add registerInterest(T... keys)
> - add registerInterest(Iterablekeys)
>
> I might be missing something here but:
> With the addition of registerInterest(Iterable keys), I think we would
> not be able to register interest a List as the key itself.  A list would be
> iterated over due to the addition of registerInterest(Iterable keys).  A
> list in a list would be passed into registerInterest and again be iterated
> over.  I could change the newly created registerInterest call and
> explicitly name it something else or are we ok with Iterables not being
> able to be registered as individual keys.
>
>
>
>
>
> On Mon, Nov 20, 2017 at 9:05 AM Kirk Lund  wrote:
>
> > John's approach looks best for when you need to specify keys.
> >
> > For ALL_KEYS, what about an API that doesn't require a token or all keys:
> >
> > public void registerInterestForAllKeys();
> >
> > On Fri, Nov 17, 2017 at 1:24 PM, Jason Huynh  wrote:
> >
> > > Thanks John for the clarification!
> > >
> > > On Fri, Nov 17, 2017 at 1:12 PM John Blum  wrote:
> > >
> > > > This...
> > > >
> > > > > The Iterable version would handle any collection type by having the
> > > user
> > > > pass
> > > > in the iterator for the collection.
> > > >
> > > > Is not correct.
> > > >
> > > > The Collection interface itself "extends" the
> java.lang.Iterable
> > > > interface (see here...
> > > > https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html
> > > under
> > > > "*All
> > > > Superinterfaces*").
> > > >
> > > > Therefore a user can simply to this...
> > > >
> > > > *List* keys = ...
> > > >
> > > > region.registerInterest(keys); *// calls the
> > > > Region.registerInterest(:Iterable) method.*
> > > >
> > > > Alternatively, this would also be allowed...
> > > >
> > > > *Set* keys = ...
> > > >
> > > > region.registerInterest(keys);
> > > >
> > > >
> > > > On Fri, Nov 17, 2017 at 11:44 AM, Jason Huynh 
> > wrote:
> > > >
> > > > > Current idea is to:
> > > > > - deprecate current "ALL_KEYS" and List passing behavior in
> > > > > registerInterest()
> > > > > - add registerInterestAllKeys();
> > > > > - add registerInterest(T... keys) and registerInterest(Iterable
> > > keys)
> > > > > and
> > > > > not have one specifically for List or specific collections.
> > > > >
> > > > > The Iterable version would handle any collection type by having the
> > > user
> > > > > pass in the iterator for the collection.
> > > > >
> > > > > On Fri, Nov 17, 2017 at 11:32 AM Jacob Barrett <
> jbarr...@pivotal.io>
> > > > > wrote:
> > > > >
> > > > > > I am failing to see where registerInterest(List keys) is an
> > issue
> > > > for
> > > > > > the key type in the region. If our region is Region then
> I
> > > > would
> > > > > > expect registerInterest(List). If the keys are unknown
> or a
> > > mix
> > > > > > then you should have Region and thus
> > > > > registerInterest(List > > > > >
> > > > > > I echo John's statements on VarArgs and type erasure as well as
> his
> > > > > > argument for Iterable.
> > > > > >
> > > > > > Also, List does not restrict you from List indexes. The region
> > > would
> > > > > be
> > > > > > Region> with registerInterest>().
> > > > > >
> > > > > > -Jake
> > > > > >
> > > > > >
> > > > > > On Fri, Nov 17, 2017 at 10:04 AM John Blum 
> > wrote:
> > > > > >
> > > > > > > Personally, I prefer the var args method (registerInterest(T...
> > > > keys))
> > > > > > > myself.  It is way more convenient if I only have a few keys
> when
> > > > > calling
> > > > > > > this method then to have to add the keys to a List, especially
> > for
> > > > > > testing
> > > > > > > purposes.
> > > > > > >
> > > > > > > But, I typically like to pair that with a
> > > > registerInterest(Iterable
> > > > > > > keys) method
> > > > > > > as well.  By having a overloaded Iterable variant, then I can
> > pass
> > > in
> > > > > any
> > > > > > > Collection type I want (which shouldn't be restricted to just
> > > List).
> > > > > It
> > > > > > > also is a simple matter to convert any *Collection* (i.e.
> *List*,
> > > > > *Set*,
> > > > > > > etc) to an array, which can be passed to the var args method.
> By
> > > > using
> > > > > > > List,
> > > > > > > you are implying that "order matters" since a Lis

Re: [DISCUSS] changes to registerInterest API

2017-11-30 Thread Jason Huynh 
I started work on the following plan:
- deprecate current "ALL_KEYS" and List passing behavior in registerInterest
()
- add registerInterestForAllKeys();
- add registerInterest(T... keys)
- add registerInterest(Iterablekeys)

I might be missing something here but:
With the addition of registerInterest(Iterable keys), I think we would
not be able to register interest a List as the key itself.  A list would be
iterated over due to the addition of registerInterest(Iterable keys).  A
list in a list would be passed into registerInterest and again be iterated
over.  I could change the newly created registerInterest call and
explicitly name it something else or are we ok with Iterables not being
able to be registered as individual keys.





On Mon, Nov 20, 2017 at 9:05 AM Kirk Lund  wrote:

> John's approach looks best for when you need to specify keys.
>
> For ALL_KEYS, what about an API that doesn't require a token or all keys:
>
> public void registerInterestForAllKeys();
>
> On Fri, Nov 17, 2017 at 1:24 PM, Jason Huynh  wrote:
>
> > Thanks John for the clarification!
> >
> > On Fri, Nov 17, 2017 at 1:12 PM John Blum  wrote:
> >
> > > This...
> > >
> > > > The Iterable version would handle any collection type by having the
> > user
> > > pass
> > > in the iterator for the collection.
> > >
> > > Is not correct.
> > >
> > > The Collection interface itself "extends" the java.lang.Iterable
> > > interface (see here...
> > > https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html
> > under
> > > "*All
> > > Superinterfaces*").
> > >
> > > Therefore a user can simply to this...
> > >
> > > *List* keys = ...
> > >
> > > region.registerInterest(keys); *// calls the
> > > Region.registerInterest(:Iterable) method.*
> > >
> > > Alternatively, this would also be allowed...
> > >
> > > *Set* keys = ...
> > >
> > > region.registerInterest(keys);
> > >
> > >
> > > On Fri, Nov 17, 2017 at 11:44 AM, Jason Huynh 
> wrote:
> > >
> > > > Current idea is to:
> > > > - deprecate current "ALL_KEYS" and List passing behavior in
> > > > registerInterest()
> > > > - add registerInterestAllKeys();
> > > > - add registerInterest(T... keys) and registerInterest(Iterable
> > keys)
> > > > and
> > > > not have one specifically for List or specific collections.
> > > >
> > > > The Iterable version would handle any collection type by having the
> > user
> > > > pass in the iterator for the collection.
> > > >
> > > > On Fri, Nov 17, 2017 at 11:32 AM Jacob Barrett 
> > > > wrote:
> > > >
> > > > > I am failing to see where registerInterest(List keys) is an
> issue
> > > for
> > > > > the key type in the region. If our region is Region then I
> > > would
> > > > > expect registerInterest(List). If the keys are unknown or a
> > mix
> > > > > then you should have Region and thus
> > > > registerInterest(List > > > >
> > > > > I echo John's statements on VarArgs and type erasure as well as his
> > > > > argument for Iterable.
> > > > >
> > > > > Also, List does not restrict you from List indexes. The region
> > would
> > > > be
> > > > > Region> with registerInterest>().
> > > > >
> > > > > -Jake
> > > > >
> > > > >
> > > > > On Fri, Nov 17, 2017 at 10:04 AM John Blum 
> wrote:
> > > > >
> > > > > > Personally, I prefer the var args method (registerInterest(T...
> > > keys))
> > > > > > myself.  It is way more convenient if I only have a few keys when
> > > > calling
> > > > > > this method then to have to add the keys to a List, especially
> for
> > > > > testing
> > > > > > purposes.
> > > > > >
> > > > > > But, I typically like to pair that with a
> > > registerInterest(Iterable
> > > > > > keys) method
> > > > > > as well.  By having a overloaded Iterable variant, then I can
> pass
> > in
> > > > any
> > > > > > Collection type I want (which shouldn't be restricted to just
> > List).
> > > > It
> > > > > > also is a simple matter to convert any *Collection* (i.e. *List*,
> > > > *Set*,
> > > > > > etc) to an array, which can be passed to the var args method.  By
> > > using
> > > > > > List,
> > > > > > you are implying that "order matters" since a List is a order
> > > > collection
> > > > > of
> > > > > > elements.
> > > > > >
> > > > > > This ("*It might even cause problems of pushing in **multiple
> > > different
> > > > > > types.*"), regarding var args, does not even make sense.
> > Technically,
> > > > > > List is no different.  Java's type erasure essentially equates
> > var
> > > > > args
> > > > > > too "Object..." (or Object[]) and the List to List (or a List
> of
> > > > > > Objects,
> > > > > > essentially like if you just did this... List) So, while
> > the
> > > > > > compiler ensures compile-time type-safety of generics, there is
> no
> > > > > generics
> > > > > > type-safety guarantees at runtime.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, Nov 17, 2017 at 9:22 AM, Jason Huynh 
> > > > wrote:
> > > > > >
> > > > > > > Hi Mike,
> > > > > > >
> > > > > > > The current support for List leads to compila