Geode unit tests completed in 'release-1.4.0/DistributedTest' with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/release-1.4.0/jobs/DistributedTest/builds/12

Geode unit tests completed in 'develop/FlakyTest' with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/FlakyTest/builds/150

Geode unit tests completed in 'develop/FlakyTest' with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/FlakyTest/builds/148

Geode unit tests completed in 'develop/DistributedTest' with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/DistributedTest/builds/98

[Spring CI] Spring Data GemFire > Nightly-ApacheGeode > #807 was SUCCESSFUL (with 2324 tests). Change made by Mark Paluch.

[Spring CI]

---
Spring Data GemFire > Nightly-ApacheGeode > #807 was successful.
---
Scheduled with changes by Mark Paluch.
2326 tests in total.

https://build.spring.io/browse/SGF-NAG-807/




--
Code Changes
--
Mark Paluch (a9a06c662058c9c080bfeb5b012317c0923ceb89):

>DATAGEODE-67 - Updated changelog.



--
This message is automatically generated by Atlassian Bamboo

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Michael Stolz]

I should have said Geode 2.0 rather than GemFire 10...
Sorry. My GemFire roots are showing ;(

--
Mike Stolz
Principal Engineer, GemFire Product Lead
Mobile: +1-631-835-4771
Download the new GemFire book here.


On Wed, Jan 24, 2018 at 2:40 PM, Michael Stolz  wrote:

> I agree that we should have api level choice for whether to read
> serialized or not.
> We can deprecate read-serialized and get rid of it in GemFire 10.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Lead
> Mobile: +1-631-835-4771 <(631)%20835-4771>
> Download the new GemFire book here.
> 
>
> On Wed, Jan 24, 2018 at 12:21 PM, Dan Smith  wrote:
>
>> Is this really just a workaround for the fact that the read-serialized
>> flag
>> applies to the whole cache? I can see that if you have mix of regions with
>> and without domain classes on the server you might want this feature. Can
>> you provide some more background on your use case?
>>
>> IMO we should get rid of read-serialized in favor of APIs that let the
>> user
>> decide whether they get a domain class or a PdxInstance.
>>
>> -Dan
>>
>> On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan 
>> wrote:
>>
>> > Hi Addison,
>> >
>> > What kind of setup do you have that is causing you to have PDX
>> serialized
>> > objects that cannot be deserialized? Do you have classes that are
>> present
>> > on some servers but not others?
>> >
>> > This change would break the contract of region.get() , which is that it
>> > returns a value of a type that can be put into the region.
>> >
>> > Returning something that isn't what the user is expecting to be in the
>> > region would require users to check for PdxInstances every time they
>> get a
>> > value -- even though the type annotations say that you can't get a
>> > PdxInstance back (except for Region ).
>> >
>> > I think it would be better to have a second API that allows users to get
>> > and put PdxInstance objects in regions. That way, if they want to handle
>> > the exceptional case where they have a serialized object that can't be
>> > deserialized, they can catch the ClassNotFound exception and get the
>> > underlying PdxInstance.
>> >
>> > I do think that the possibility of a ClassNotFoundException should be
>> > documented in the Region API.
>> >
>> > Cheers,
>> > Galen
>> >
>> > On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy 
>> wrote:
>> >
>> > > Hi Geode Devs,
>> > >
>> > > I'm proposing the following change to how we handle deserialization
>> when
>> > > Domain Objects can't be found and pdx-serialize=false.
>> > >
>> > > https://issues.apache.org/jira/browse/GEODE-4367
>> > >
>> > > Looking forward to the discussion.
>> > >
>> > > \ah
>> > >
>> >
>>
>
>

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Michael Stolz]

I agree that we should have api level choice for whether to read serialized
or not.
We can deprecate read-serialized and get rid of it in GemFire 10.

--
Mike Stolz
Principal Engineer, GemFire Product Lead
Mobile: +1-631-835-4771
Download the new GemFire book here.


On Wed, Jan 24, 2018 at 12:21 PM, Dan Smith  wrote:

> Is this really just a workaround for the fact that the read-serialized flag
> applies to the whole cache? I can see that if you have mix of regions with
> and without domain classes on the server you might want this feature. Can
> you provide some more background on your use case?
>
> IMO we should get rid of read-serialized in favor of APIs that let the user
> decide whether they get a domain class or a PdxInstance.
>
> -Dan
>
> On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan 
> wrote:
>
> > Hi Addison,
> >
> > What kind of setup do you have that is causing you to have PDX serialized
> > objects that cannot be deserialized? Do you have classes that are present
> > on some servers but not others?
> >
> > This change would break the contract of region.get() , which is that it
> > returns a value of a type that can be put into the region.
> >
> > Returning something that isn't what the user is expecting to be in the
> > region would require users to check for PdxInstances every time they get
> a
> > value -- even though the type annotations say that you can't get a
> > PdxInstance back (except for Region ).
> >
> > I think it would be better to have a second API that allows users to get
> > and put PdxInstance objects in regions. That way, if they want to handle
> > the exceptional case where they have a serialized object that can't be
> > deserialized, they can catch the ClassNotFound exception and get the
> > underlying PdxInstance.
> >
> > I do think that the possibility of a ClassNotFoundException should be
> > documented in the Region API.
> >
> > Cheers,
> > Galen
> >
> > On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy 
> wrote:
> >
> > > Hi Geode Devs,
> > >
> > > I'm proposing the following change to how we handle deserialization
> when
> > > Domain Objects can't be found and pdx-serialize=false.
> > >
> > > https://issues.apache.org/jira/browse/GEODE-4367
> > >
> > > Looking forward to the discussion.
> > >
> > > \ah
> > >
> >
>

Re: Spring Data for Apache Geode 2.0.3.RELEASE available!

[Dennis Dai]

Right on the heels of Spring Framework 5.0.3 release, great job as always!

Dennis Dai
d...@epaysol.com

> On Jan 24, 2018, at 9:35 AM, John Blum  wrote:
> 
> Apache Geode Community-
> 
> I am very pleased to follow-up to the announcement by Mark Paluch on the 
> release of Spring Data Kay SR3.  This includes Spring Data for Apache Geode 
> 2.0.3.RELEASE.
> 
> The 3rd service release of SDG includes several import fixes and 
> enhancements, particular around the new Annotation config model.  You can see 
> a complete list of changes in the changelog 
> (https://docs.spring.io/spring-data/geode/docs/2.0.3.RELEASE/changelog.txt 
> ).
> 
> See the official announcement here 
> (https://spring.io/blog/2018/01/24/spring-data-ingalls-sr10-and-kay-sr3-released
>  
> ).
> 
> Feedback welcomed.
> 
> Happy coding!
> 
> --
> -John
> john.blum10101 (skype)



signature.asc
Description: Message signed with OpenPGP

Geode unit tests completed in 'develop/FlakyTest' with non-zero exit code

[apachegeodeci]

Pipeline results can be found at:

Concourse: 
https://concourse.apachegeode-ci.info/teams/main/pipelines/develop/jobs/FlakyTest/builds/147

Roadmap and Upcoming Features for Spring Data for Apache Geode 2.1 (SD Lovelace)...

[John Blum]

Hi Apache Geode Community-

Now that SDG 2.0 is fairly stable (entering its 3rd service release already
now) and I have resolved all known issues, particularly with the new SDG
Annotation configuration, I figured I would take this opportunity to let
you all know about some of the features I have planned for the next SD
Release Train, Lovelace [1], after Ada Lovelace [2].

First, let me clarify that I am not planning any new significant
enhancements or additions in the 2.0 line going forward; it will mostly
consist of bug fixes and minor enhancements, i.e. be in full maintenance
mode.

However, I will just add that I am particularly proud of the new SDG
Annotation config, largely introduced in 2.0, but technically started in
Spring Data GemFire 1.9.  It has paid dividends not only in developing new
(compound) features in SDG quickly, building on a solid, highly reusable
foundation, but it is greatly simplified my testing efforts.  Expect this
only to improve with time your feedback.



So, in SD Lovelace, I plan to...

1. Repository Query Projections - SGF-523. [3]
2. Repository Paging Support - SGF-524. [4] (NOTE: Sorting is already
supported)
3. Support for Function Executions on Repositories (i.e. *@OnRegion*) -
SGF-452. [5]
4. Reactive Repository Support, think Repository query invocations
returning a CQ wrapped with a Flux - SGF-377. [6]
5. POJO Annotated Cache Callbacks - SGF-453. [7]

In addition, some other notables include...

6. AEQ + Listener Annotation configuration support - SGF-551. [8]
7. Gateway Sender/Receiver Annotation configuration support - SGF-521 &
SGF-550. [9] & [10]
8. Integration with Spring Security - SGF-684. [11]
9. Integration with Micrometer (which Spring Boot 2.0's new Actuator is
based on) - SGF-671. [12]
10. (Possible) Integration with Pivotal Cloud Foundry's New Function
Service (Serverless with PFS) [13]

You can see more highlights and a summary of what came to be in Spring Data
Kay as well as the plan for Spring Data Lovelace in my ~30 min *SpringOne
Platform 2017* talk, "*Simplifying Apache Geode with Spring Data*". [14]



Along with SDG, I also have 2 new projects I am simultaneously launching...

1. *Spring Boot for Apache Geode* (or *Spring Boot Data Geode*) - building
on SDG's Annotation configuration model, but without need to explicitly
declare the Annotations, using auto-configuration and CLASSPATH dependency
detection/enablement.  [15]

This work is particularly important for PCF/PCC (Pivotal Cloud Cache)
integration making moving between PCF and your local dev environment
seamless, the goal being, 0 to very little configuration changes, i.e. it
should just work!  PCC on PCF really is the best way to manage Apache
Geode/Pivotal GemFire.

2. *Spring Data Tests 4 Pivotal GemFire/Apache Geode* - I have started
development/work on a test framework to simplify both Unit and Integration
Testing of Apache Geode applications in *Spring* context, using *Spring's*
TestContext framework [16] along with JUnit 5, Mockito and a few other
things.  Technically, this development began long ago (in a galaxy far far
away ;-) in SDG and I am just uplifting, refactoring and polishing this
code into a proper test framework that all Spring/Apache Geode users can
leverage.

Many people have asked me about this; I (eventually) plan to deliver, ;-).

You can follow this project here [17].

3. I am exploring enhancements to *Spring Session for Apache Geode* as
well, things such as fixed-length expiration along with TTL.  See here [18]
for more details.

4. Finally, I am also planning on an major overhaul of the examples [19]
along with making further improvements to the documentation [20] & [21],
adding more Guides on spring.io/guides [22], etc.

I do keep the *Contacts Application* Reference Implementation (RI) [23]
up-to-date and it is a very good reference for the latest developments in
SDG.


Clearly, there is no shortage of things to do.  All feedback and
contributions are welcomed.

Regards,
-- 
-John


[1]
https://github.com/spring-projects/spring-data-commons/wiki/Release-Train-Lovelace
[2] https://en.wikipedia.org/wiki/Ada_Lovelace
[3] https://jira.spring.io/browse/SGF-523
[4] https://jira.spring.io/browse/SGF-524
[5] https://jira.spring.io/browse/SGF-452
[6] https://jira.spring.io/browse/SGF-377
[7] https://jira.spring.io/browse/SGF-453
[8] https://jira.spring.io/browse/SGF-551
[9] https://jira.spring.io/browse/SGF-521
[10] https://jira.spring.io/browse/SGF-5510
[11] https://jira.spring.io/browse/SGF-684
[12] https://jira.spring.io/browse/SGF-671
[13] https://pivotal.io/platform/pivotal-function-service
[14] https://www.youtube.com/watch?v=OvY5wzCtOV0
[15]
https://github.com/spring-projects/spring-boot-data-geode#spring-boot-for-apache-geode
[16]
https://docs.spring.io/spring/docs/5.0.2.RELEASE/spring-framework-reference/testing.html#testing
[17] https://github.com/jxblum/spring-data-tests-4-gemfire
[18] https://github.com/spring-projects/spring-session-data-geode/issues
[19] 

Re: Addition of rmi-io library

[Anthony Baker]

Mike,

??  I think you meant 1.4.0 ??

Let’s keep the release discussion on the VOTE thread.  IMO, adding another port 
has implications for security, firewalls, containers, bind addresses, etc.  I 
haven’t cast my release vote yet since I’m still considering these things.

Anthony


> On Jan 24, 2018, at 11:24 AM, Michael Stolz  wrote:
> 
> There is going to be a 9.3.1 shortly after 9.3.0. Lets not hold 9.3.0 for
> this.
> 
> --
> Mike Stolz
> Principal Engineer - Gemfire Product Manager
> Mobile: 631-835-4771
> 
> On Jan 24, 2018 10:58 AM, "Jinmei Liao"  wrote:
> 
> yeah, Jens just found that out too. It's opening up a new port in either
> server/server and gfsh/jmManager cases. I think he has a solution to it and
> we will get it in soon.
> 
> On Wed, Jan 24, 2018 at 9:47 AM, Dan Smith  wrote:
> 
>>> 
>>> the content is going over the wire on whatever port that was port
> before.
>> 
>> 
>> From what I see, DownloadJarFunction is calling
>> SimpleRemoteInputStream.export() which will call
>> UnicastRemoteObject.exportObject. That's an RMI call to start a tcp server
>> socket listening for connections to interact with that object.
>> 
>> -Dan
>> 
>> On Tue, Jan 23, 2018 at 6:15 PM, Jinmei Liao  wrote:
>> 
>>> As far as I can see, we are utilizing the streaming capability provided
>> by
>>> the rmi-io, the content is going over the wire on whatever port that was
>>> port before. When streaming content from the gfsh to the jmxManager,
> it's
>>> using the jmx port; when getting jars between locator/servers, it's
> using
>>> the FunctionService, so it's whatever communication channel that
>>> FunctionService is using.
>>> 
>>> All the FileContent are saved in temp folder, and get cleaned up after
>> each
>>> deployment.
>>> 
>>> On Tue, Jan 23, 2018 at 3:17 PM, Dan Smith  wrote:
>>> 
 I don't have an issue with the dependency. But if we are opening up
> new
 ports for RMI connections, that seems like a potential security risk.
>> If
 someone has enabled cluster SSL we shouldn't be opening up an insecure
>>> port
 for RMI connections.
 
 We should also make sure this is not leaking open sockets/file
>>> decriptors.
 How does this SimpleRemoteInputStream we are creating get shutdown and
 cleaned up?
 
 -Dan
 
 On Tue, Jan 23, 2018 at 2:36 PM, Jens Deppe 
>>> wrote:
 
> Apologies that this was not raised earlier in discussion but I'm
>> happy
>>> to
> describe it now.
> 
> *Background:*
> 
> When deploying jars into Geode they are moved through the system as
 simple
> byte[] blobs. This obviously consumes memory. The various affected
>>> areas
> are:
> 
> - gfsh reads the jars into memory
> - the jars are pushed to the locator (via a jmx call) - again
>> creating
>>> a
> byte[] blob on the locator
> - from the locator, the jars are pushed to all servers via a
> function
 call
> (also sending the jars as byte[] blobs).
> 
> Obviously if the jar is small this would not be a problem, however
> in
> memory constrained systems or with large jars this is obviously
> going
>>> to
> put pressure on memory and possibly result in OOM situations. In
>> fact,
 the
> reason this came up was that some folks were unable to deploy a 40Mb
>>> jar
 to
> a 512Mb (heap) locator.
> 
> *rmi-io:*
> 
> After doing some research, it seemed that the ideal solution would
> be
> something that allows for serializing Input/OutputStreams. Java
>> doesn't
> provide anything natively.
> 
> One library that stood out as being robust and feature complete was
 rmi-io
> [1]. This allows for serializing a remote Input/OutputStream object
>>> which
> then lets us completely avoid having to pull deploying jars into
>> memory
> everywhere. Under the covers it uses RMI and allows for either
>>> 'pulling'
 or
> 'pushing' data. The reference page [2] has nice sequence diagrams.
> 
> If anyone sees any issues with this, please do raise them. The
>> current
> usage of this has not changed any user-facing interaction so
>> ultimately
> changing the actual implemented fix for this problem (if we needed
>> to)
> would not have any external effect.
> 
> Thanks
> --Jens
> 
> [1] http://openhms.sourceforge.net/rmiio
> [2] http://openhms.sourceforge.net/rmiio/class_reference.html
> 
 
>>> 
>>> 
>>> 
>>> --
>>> Cheers
>>> 
>>> Jinmei
>>> 
>> 
> 
> 
> 
> --
> Cheers
> 
> Jinmei

Re: Addition of rmi-io library

[Michael Stolz]

There is going to be a 9.3.1 shortly after 9.3.0. Lets not hold 9.3.0 for
this.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771

On Jan 24, 2018 10:58 AM, "Jinmei Liao"  wrote:

yeah, Jens just found that out too. It's opening up a new port in either
server/server and gfsh/jmManager cases. I think he has a solution to it and
we will get it in soon.

On Wed, Jan 24, 2018 at 9:47 AM, Dan Smith  wrote:

> >
> > the content is going over the wire on whatever port that was port
before.
>
>
> From what I see, DownloadJarFunction is calling
> SimpleRemoteInputStream.export() which will call
> UnicastRemoteObject.exportObject. That's an RMI call to start a tcp server
> socket listening for connections to interact with that object.
>
> -Dan
>
> On Tue, Jan 23, 2018 at 6:15 PM, Jinmei Liao  wrote:
>
> > As far as I can see, we are utilizing the streaming capability provided
> by
> > the rmi-io, the content is going over the wire on whatever port that was
> > port before. When streaming content from the gfsh to the jmxManager,
it's
> > using the jmx port; when getting jars between locator/servers, it's
using
> > the FunctionService, so it's whatever communication channel that
> > FunctionService is using.
> >
> > All the FileContent are saved in temp folder, and get cleaned up after
> each
> > deployment.
> >
> > On Tue, Jan 23, 2018 at 3:17 PM, Dan Smith  wrote:
> >
> > > I don't have an issue with the dependency. But if we are opening up
new
> > > ports for RMI connections, that seems like a potential security risk.
> If
> > > someone has enabled cluster SSL we shouldn't be opening up an insecure
> > port
> > > for RMI connections.
> > >
> > > We should also make sure this is not leaking open sockets/file
> > decriptors.
> > > How does this SimpleRemoteInputStream we are creating get shutdown and
> > > cleaned up?
> > >
> > > -Dan
> > >
> > > On Tue, Jan 23, 2018 at 2:36 PM, Jens Deppe 
> > wrote:
> > >
> > > > Apologies that this was not raised earlier in discussion but I'm
> happy
> > to
> > > > describe it now.
> > > >
> > > > *Background:*
> > > >
> > > > When deploying jars into Geode they are moved through the system as
> > > simple
> > > > byte[] blobs. This obviously consumes memory. The various affected
> > areas
> > > > are:
> > > >
> > > > - gfsh reads the jars into memory
> > > > - the jars are pushed to the locator (via a jmx call) - again
> creating
> > a
> > > > byte[] blob on the locator
> > > > - from the locator, the jars are pushed to all servers via a
function
> > > call
> > > > (also sending the jars as byte[] blobs).
> > > >
> > > > Obviously if the jar is small this would not be a problem, however
in
> > > > memory constrained systems or with large jars this is obviously
going
> > to
> > > > put pressure on memory and possibly result in OOM situations. In
> fact,
> > > the
> > > > reason this came up was that some folks were unable to deploy a 40Mb
> > jar
> > > to
> > > > a 512Mb (heap) locator.
> > > >
> > > > *rmi-io:*
> > > >
> > > > After doing some research, it seemed that the ideal solution would
be
> > > > something that allows for serializing Input/OutputStreams. Java
> doesn't
> > > > provide anything natively.
> > > >
> > > > One library that stood out as being robust and feature complete was
> > > rmi-io
> > > > [1]. This allows for serializing a remote Input/OutputStream object
> > which
> > > > then lets us completely avoid having to pull deploying jars into
> memory
> > > > everywhere. Under the covers it uses RMI and allows for either
> > 'pulling'
> > > or
> > > > 'pushing' data. The reference page [2] has nice sequence diagrams.
> > > >
> > > > If anyone sees any issues with this, please do raise them. The
> current
> > > > usage of this has not changed any user-facing interaction so
> ultimately
> > > > changing the actual implemented fix for this problem (if we needed
> to)
> > > > would not have any external effect.
> > > >
> > > > Thanks
> > > > --Jens
> > > >
> > > > [1] http://openhms.sourceforge.net/rmiio
> > > > [2] http://openhms.sourceforge.net/rmiio/class_reference.html
> > > >
> > >
> >
> >
> >
> > --
> > Cheers
> >
> > Jinmei
> >
>



--
Cheers

Jinmei

Re: [VOTE] Apache Geode release - 1.4.0 RC1

[Anthony Baker]

Reviewed:

- tags
- maven repo
- builds from source, also geode-examples
- checked for binaries

The pom added this dependency:

 
   org.slf4j
   slf4j-api
   1.7.24
   runtime
 


and then this exclusion for the the log4j-slf4j-impl dependency:


  org.apache.logging.log4j
  log4j-slf4j-impl
  2.8.2
  runtime
  

  slf4j-api
  *

  
  true



Is that correct?  See 
https://github.com/apache/geode/commit/61ba3b69b3ec3093ae0610f428de62c57499b123.


Anthony
 

> On Jan 23, 2018, at 2:32 PM, Swapnil Bawaskar  wrote:
> 
> This is the first release candidate for Apache Geode, version 1.4.0.
> Thanks to all the community members for their contributions to this
> release!
> 
> *** Please download, test and vote by Friday, January 26, 1200 hrs
> US Pacific. ***
> 
> It fixes 277 issues. release notes can be found at:
> https://issues.apache.org/jira/projects/GEODE/versions/12341842
> 
> Note that we are voting upon the source tags: rel/v1.4.0.RC1
> https://github.com/apache/geode/tree/rel/v1.4.0.RC1
> https://github.com/apache/geode-examples/tree/rel/v1.4.0.RC1
> 
> Commit ID:
> 7dbb280fbff0320efd427c5553a6e666328bc50c (geode)
> ced35f88173b04ac8e104b9cae10cc38289675fa (geode-examples)
> 
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/geode/1.4.0.RC1
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachegeode-1036
> 
> 
> Geode's KEYS file containing PGP keys we use to sign the release:
> https://github.com/apache/geode/blob/develop/KEYS
> 
> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

Re: Spring Data for Apache Geode 2.0.3.RELEASE available!

[Michael Stolz]

Congratulations. Great job as always.

--
Mike Stolz
Principal Engineer - Gemfire Product Manager
Mobile: 631-835-4771

On Jan 24, 2018 10:35 AM, "John Blum"  wrote:

> Apache Geode Community-
>
> I am very pleased to follow-up to the announcement by Mark Paluch on the
> release of Spring Data Kay SR3.  This includes Spring Data for Apache Geode
> 2.0.3.RELEASE.
>
> The 3rd service release of SDG includes several import fixes and
> enhancements, particular around the new Annotation config model.  You can
> see a complete list of changes in the changelog (https://docs.spring.io/
> spring-data/geode/docs/2.0.3.RELEASE/changelog.txt).
>
> See the official announcement here (https://spring.io/blog/2018/
> 01/24/spring-data-ingalls-sr10-and-kay-sr3-released).
>
> Feedback welcomed.
>
> Happy coding!
>
> --
> -John
> john.blum10101 (skype)
>

Re: Addition of rmi-io library

[Jens Deppe]

Right - it needs to piggy-back on the existing JMX/RMI port in which case
no new ports will be required.

We're testing this now.

--Jens

On Wed, Jan 24, 2018 at 9:58 AM, Jinmei Liao  wrote:

> yeah, Jens just found that out too. It's opening up a new port in either
> server/server and gfsh/jmManager cases. I think he has a solution to it and
> we will get it in soon.
>
> On Wed, Jan 24, 2018 at 9:47 AM, Dan Smith  wrote:
>
> > >
> > > the content is going over the wire on whatever port that was port
> before.
> >
> >
> > From what I see, DownloadJarFunction is calling
> > SimpleRemoteInputStream.export() which will call
> > UnicastRemoteObject.exportObject. That's an RMI call to start a tcp
> server
> > socket listening for connections to interact with that object.
> >
> > -Dan
> >
> > On Tue, Jan 23, 2018 at 6:15 PM, Jinmei Liao  wrote:
> >
> > > As far as I can see, we are utilizing the streaming capability provided
> > by
> > > the rmi-io, the content is going over the wire on whatever port that
> was
> > > port before. When streaming content from the gfsh to the jmxManager,
> it's
> > > using the jmx port; when getting jars between locator/servers, it's
> using
> > > the FunctionService, so it's whatever communication channel that
> > > FunctionService is using.
> > >
> > > All the FileContent are saved in temp folder, and get cleaned up after
> > each
> > > deployment.
> > >
> > > On Tue, Jan 23, 2018 at 3:17 PM, Dan Smith  wrote:
> > >
> > > > I don't have an issue with the dependency. But if we are opening up
> new
> > > > ports for RMI connections, that seems like a potential security risk.
> > If
> > > > someone has enabled cluster SSL we shouldn't be opening up an
> insecure
> > > port
> > > > for RMI connections.
> > > >
> > > > We should also make sure this is not leaking open sockets/file
> > > decriptors.
> > > > How does this SimpleRemoteInputStream we are creating get shutdown
> and
> > > > cleaned up?
> > > >
> > > > -Dan
> > > >
> > > > On Tue, Jan 23, 2018 at 2:36 PM, Jens Deppe 
> > > wrote:
> > > >
> > > > > Apologies that this was not raised earlier in discussion but I'm
> > happy
> > > to
> > > > > describe it now.
> > > > >
> > > > > *Background:*
> > > > >
> > > > > When deploying jars into Geode they are moved through the system as
> > > > simple
> > > > > byte[] blobs. This obviously consumes memory. The various affected
> > > areas
> > > > > are:
> > > > >
> > > > > - gfsh reads the jars into memory
> > > > > - the jars are pushed to the locator (via a jmx call) - again
> > creating
> > > a
> > > > > byte[] blob on the locator
> > > > > - from the locator, the jars are pushed to all servers via a
> function
> > > > call
> > > > > (also sending the jars as byte[] blobs).
> > > > >
> > > > > Obviously if the jar is small this would not be a problem, however
> in
> > > > > memory constrained systems or with large jars this is obviously
> going
> > > to
> > > > > put pressure on memory and possibly result in OOM situations. In
> > fact,
> > > > the
> > > > > reason this came up was that some folks were unable to deploy a
> 40Mb
> > > jar
> > > > to
> > > > > a 512Mb (heap) locator.
> > > > >
> > > > > *rmi-io:*
> > > > >
> > > > > After doing some research, it seemed that the ideal solution would
> be
> > > > > something that allows for serializing Input/OutputStreams. Java
> > doesn't
> > > > > provide anything natively.
> > > > >
> > > > > One library that stood out as being robust and feature complete was
> > > > rmi-io
> > > > > [1]. This allows for serializing a remote Input/OutputStream object
> > > which
> > > > > then lets us completely avoid having to pull deploying jars into
> > memory
> > > > > everywhere. Under the covers it uses RMI and allows for either
> > > 'pulling'
> > > > or
> > > > > 'pushing' data. The reference page [2] has nice sequence diagrams.
> > > > >
> > > > > If anyone sees any issues with this, please do raise them. The
> > current
> > > > > usage of this has not changed any user-facing interaction so
> > ultimately
> > > > > changing the actual implemented fix for this problem (if we needed
> > to)
> > > > > would not have any external effect.
> > > > >
> > > > > Thanks
> > > > > --Jens
> > > > >
> > > > > [1] http://openhms.sourceforge.net/rmiio
> > > > > [2] http://openhms.sourceforge.net/rmiio/class_reference.html
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Cheers
> > >
> > > Jinmei
> > >
> >
>
>
>
> --
> Cheers
>
> Jinmei
>

Re: [VOTE] Apache Geode release - 1.4.0 RC1

[Dan Smith]

-1 due to to the security risks introduced by GEODE-4131 (using rmi-io for
deploying jars).

-Dan

On Wed, Jan 24, 2018 at 8:08 AM, Anthony Baker  wrote:

> Here’s a better URL for release notes that doesn’t require logging in:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12318420=12341842  jira/secure/ReleaseNote.jspa?projectId=12318420=12341842>
>
> Let’s start crowd-sourcing a condensed version here:
> https://cwiki.apache.org/confluence/display/GEODE/
> Release+Notes#ReleaseNotes-1.4.0  confluence/display/GEODE/Release+Notes#ReleaseNotes-1.4.0>
>
>
> Thanks,
> Anthony
>
>
> > On Jan 23, 2018, at 2:32 PM, Swapnil Bawaskar 
> wrote:
> >
> > This is the first release candidate for Apache Geode, version 1.4.0.
> > Thanks to all the community members for their contributions to this
> > release!
> >
> > *** Please download, test and vote by Friday, January 26, 1200 hrs
> > US Pacific. ***
> >
> > It fixes 277 issues. release notes can be found at:
> > https://issues.apache.org/jira/projects/GEODE/versions/12341842
> >
> > Note that we are voting upon the source tags: rel/v1.4.0.RC1
> > https://github.com/apache/geode/tree/rel/v1.4.0.RC1
> > https://github.com/apache/geode-examples/tree/rel/v1.4.0.RC1
> >
> > Commit ID:
> > 7dbb280fbff0320efd427c5553a6e666328bc50c (geode)
> > ced35f88173b04ac8e104b9cae10cc38289675fa (geode-examples)
> >
> > Source and binary files:
> > https://dist.apache.org/repos/dist/dev/geode/1.4.0.RC1
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachegeode-1036
> >
> >
> > Geode's KEYS file containing PGP keys we use to sign the release:
> > https://github.com/apache/geode/blob/develop/KEYS
> >
> > Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> > Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
>
>

Re: Spring Data for Apache Geode 2.0.3.RELEASE available!

[Mark Secrist]

Awesome!

On Wed, Jan 24, 2018 at 10:35 AM, John Blum  wrote:

> Apache Geode Community-
>
> I am very pleased to follow-up to the announcement by Mark Paluch on the
> release of Spring Data Kay SR3.  This includes Spring Data for Apache Geode
> 2.0.3.RELEASE.
>
> The 3rd service release of SDG includes several import fixes and
> enhancements, particular around the new Annotation config model.  You can
> see a complete list of changes in the changelog (https://docs.spring.io/
> spring-data/geode/docs/2.0.3.RELEASE/changelog.txt).
>
> See the official announcement here (https://spring.io/blog/2018/
> 01/24/spring-data-ingalls-sr10-and-kay-sr3-released).
>
> Feedback welcomed.
>
> Happy coding!
>
> --
> -John
> john.blum10101 (skype)
>



-- 

*Mark Secrist | Director, **Global Education Delivery*

msecr...@pivotal.io

970.214.4567 Mobile

  *pivotal.io *

Follow Us: Twitter  | LinkedIn
 | Facebook
 | YouTube
 | Google+

Re: Addition of rmi-io library

[Jinmei Liao]

yeah, Jens just found that out too. It's opening up a new port in either
server/server and gfsh/jmManager cases. I think he has a solution to it and
we will get it in soon.

On Wed, Jan 24, 2018 at 9:47 AM, Dan Smith  wrote:

> >
> > the content is going over the wire on whatever port that was port before.
>
>
> From what I see, DownloadJarFunction is calling
> SimpleRemoteInputStream.export() which will call
> UnicastRemoteObject.exportObject. That's an RMI call to start a tcp server
> socket listening for connections to interact with that object.
>
> -Dan
>
> On Tue, Jan 23, 2018 at 6:15 PM, Jinmei Liao  wrote:
>
> > As far as I can see, we are utilizing the streaming capability provided
> by
> > the rmi-io, the content is going over the wire on whatever port that was
> > port before. When streaming content from the gfsh to the jmxManager, it's
> > using the jmx port; when getting jars between locator/servers, it's using
> > the FunctionService, so it's whatever communication channel that
> > FunctionService is using.
> >
> > All the FileContent are saved in temp folder, and get cleaned up after
> each
> > deployment.
> >
> > On Tue, Jan 23, 2018 at 3:17 PM, Dan Smith  wrote:
> >
> > > I don't have an issue with the dependency. But if we are opening up new
> > > ports for RMI connections, that seems like a potential security risk.
> If
> > > someone has enabled cluster SSL we shouldn't be opening up an insecure
> > port
> > > for RMI connections.
> > >
> > > We should also make sure this is not leaking open sockets/file
> > decriptors.
> > > How does this SimpleRemoteInputStream we are creating get shutdown and
> > > cleaned up?
> > >
> > > -Dan
> > >
> > > On Tue, Jan 23, 2018 at 2:36 PM, Jens Deppe 
> > wrote:
> > >
> > > > Apologies that this was not raised earlier in discussion but I'm
> happy
> > to
> > > > describe it now.
> > > >
> > > > *Background:*
> > > >
> > > > When deploying jars into Geode they are moved through the system as
> > > simple
> > > > byte[] blobs. This obviously consumes memory. The various affected
> > areas
> > > > are:
> > > >
> > > > - gfsh reads the jars into memory
> > > > - the jars are pushed to the locator (via a jmx call) - again
> creating
> > a
> > > > byte[] blob on the locator
> > > > - from the locator, the jars are pushed to all servers via a function
> > > call
> > > > (also sending the jars as byte[] blobs).
> > > >
> > > > Obviously if the jar is small this would not be a problem, however in
> > > > memory constrained systems or with large jars this is obviously going
> > to
> > > > put pressure on memory and possibly result in OOM situations. In
> fact,
> > > the
> > > > reason this came up was that some folks were unable to deploy a 40Mb
> > jar
> > > to
> > > > a 512Mb (heap) locator.
> > > >
> > > > *rmi-io:*
> > > >
> > > > After doing some research, it seemed that the ideal solution would be
> > > > something that allows for serializing Input/OutputStreams. Java
> doesn't
> > > > provide anything natively.
> > > >
> > > > One library that stood out as being robust and feature complete was
> > > rmi-io
> > > > [1]. This allows for serializing a remote Input/OutputStream object
> > which
> > > > then lets us completely avoid having to pull deploying jars into
> memory
> > > > everywhere. Under the covers it uses RMI and allows for either
> > 'pulling'
> > > or
> > > > 'pushing' data. The reference page [2] has nice sequence diagrams.
> > > >
> > > > If anyone sees any issues with this, please do raise them. The
> current
> > > > usage of this has not changed any user-facing interaction so
> ultimately
> > > > changing the actual implemented fix for this problem (if we needed
> to)
> > > > would not have any external effect.
> > > >
> > > > Thanks
> > > > --Jens
> > > >
> > > > [1] http://openhms.sourceforge.net/rmiio
> > > > [2] http://openhms.sourceforge.net/rmiio/class_reference.html
> > > >
> > >
> >
> >
> >
> > --
> > Cheers
> >
> > Jinmei
> >
>



-- 
Cheers

Jinmei

Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found

[Galen O'Sullivan]

Hi Addison,

What kind of setup do you have that is causing you to have PDX serialized
objects that cannot be deserialized? Do you have classes that are present
on some servers but not others?

This change would break the contract of region.get() , which is that it
returns a value of a type that can be put into the region.

Returning something that isn't what the user is expecting to be in the
region would require users to check for PdxInstances every time they get a
value -- even though the type annotations say that you can't get a
PdxInstance back (except for Region ).

I think it would be better to have a second API that allows users to get
and put PdxInstance objects in regions. That way, if they want to handle
the exceptional case where they have a serialized object that can't be
deserialized, they can catch the ClassNotFound exception and get the
underlying PdxInstance.

I do think that the possibility of a ClassNotFoundException should be
documented in the Region API.

Cheers,
Galen

On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy  wrote:

> Hi Geode Devs,
>
> I'm proposing the following change to how we handle deserialization when
> Domain Objects can't be found and pdx-serialize=false.
>
> https://issues.apache.org/jira/browse/GEODE-4367
>
> Looking forward to the discussion.
>
> \ah
>

Re: Addition of rmi-io library

[Dan Smith]

>
> the content is going over the wire on whatever port that was port before.


>From what I see, DownloadJarFunction is calling
SimpleRemoteInputStream.export() which will call
UnicastRemoteObject.exportObject. That's an RMI call to start a tcp server
socket listening for connections to interact with that object.

-Dan

On Tue, Jan 23, 2018 at 6:15 PM, Jinmei Liao  wrote:

> As far as I can see, we are utilizing the streaming capability provided by
> the rmi-io, the content is going over the wire on whatever port that was
> port before. When streaming content from the gfsh to the jmxManager, it's
> using the jmx port; when getting jars between locator/servers, it's using
> the FunctionService, so it's whatever communication channel that
> FunctionService is using.
>
> All the FileContent are saved in temp folder, and get cleaned up after each
> deployment.
>
> On Tue, Jan 23, 2018 at 3:17 PM, Dan Smith  wrote:
>
> > I don't have an issue with the dependency. But if we are opening up new
> > ports for RMI connections, that seems like a potential security risk. If
> > someone has enabled cluster SSL we shouldn't be opening up an insecure
> port
> > for RMI connections.
> >
> > We should also make sure this is not leaking open sockets/file
> decriptors.
> > How does this SimpleRemoteInputStream we are creating get shutdown and
> > cleaned up?
> >
> > -Dan
> >
> > On Tue, Jan 23, 2018 at 2:36 PM, Jens Deppe 
> wrote:
> >
> > > Apologies that this was not raised earlier in discussion but I'm happy
> to
> > > describe it now.
> > >
> > > *Background:*
> > >
> > > When deploying jars into Geode they are moved through the system as
> > simple
> > > byte[] blobs. This obviously consumes memory. The various affected
> areas
> > > are:
> > >
> > > - gfsh reads the jars into memory
> > > - the jars are pushed to the locator (via a jmx call) - again creating
> a
> > > byte[] blob on the locator
> > > - from the locator, the jars are pushed to all servers via a function
> > call
> > > (also sending the jars as byte[] blobs).
> > >
> > > Obviously if the jar is small this would not be a problem, however in
> > > memory constrained systems or with large jars this is obviously going
> to
> > > put pressure on memory and possibly result in OOM situations. In fact,
> > the
> > > reason this came up was that some folks were unable to deploy a 40Mb
> jar
> > to
> > > a 512Mb (heap) locator.
> > >
> > > *rmi-io:*
> > >
> > > After doing some research, it seemed that the ideal solution would be
> > > something that allows for serializing Input/OutputStreams. Java doesn't
> > > provide anything natively.
> > >
> > > One library that stood out as being robust and feature complete was
> > rmi-io
> > > [1]. This allows for serializing a remote Input/OutputStream object
> which
> > > then lets us completely avoid having to pull deploying jars into memory
> > > everywhere. Under the covers it uses RMI and allows for either
> 'pulling'
> > or
> > > 'pushing' data. The reference page [2] has nice sequence diagrams.
> > >
> > > If anyone sees any issues with this, please do raise them. The current
> > > usage of this has not changed any user-facing interaction so ultimately
> > > changing the actual implemented fix for this problem (if we needed to)
> > > would not have any external effect.
> > >
> > > Thanks
> > > --Jens
> > >
> > > [1] http://openhms.sourceforge.net/rmiio
> > > [2] http://openhms.sourceforge.net/rmiio/class_reference.html
> > >
> >
>
>
>
> --
> Cheers
>
> Jinmei
>

Re: Spring Data for Apache Geode 2.0.3.RELEASE available!

[Amit Pandey]

Congrats John and team.

Errored: apache/geode-examples#165 (rel/v1.4.0.RC1 - ced35f8)

[Travis CI]

Build Update for apache/geode-examples
-

Build: #165
Status: Errored

Duration: 1 minute and 46 seconds
Commit: ced35f8 (rel/v1.4.0.RC1)
Author: Swapnil Bawaskar
Message: updated version in preparation for the release

View the changeset: 
https://github.com/apache/geode-examples/compare/rel/v1.4.0.RC1

View the full build log and details: 
https://travis-ci.org/apache/geode-examples/builds/332529329?utm_source=email_medium=notification

--

You can configure recipients for build notifications in your .travis.yml file. 
See https://docs.travis-ci.com/user/notifications

Spring Data for Apache Geode 2.0.3.RELEASE available!

[John Blum]

Apache Geode Community-

I am very pleased to follow-up to the announcement by Mark Paluch on the
release of Spring Data Kay SR3.  This includes Spring Data for Apache Geode
2.0.3.RELEASE.

The 3rd service release of SDG includes several import fixes and
enhancements, particular around the new Annotation config model.  You can
see a complete list of changes in the changelog (
https://docs.spring.io/spring-data/geode/docs/2.0.3.RELEASE/changelog.txt).

See the official announcement here (
https://spring.io/blog/2018/01/24/spring-data-ingalls-sr10-and-kay-sr3-released
).

Feedback welcomed.

Happy coding!

-- 
-John
john.blum10101 (skype)

Re: [VOTE] Apache Geode release - 1.4.0 RC1

[Anthony Baker]

Here’s a better URL for release notes that doesn’t require logging in:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420=12341842
 


Let’s start crowd-sourcing a condensed version here:
https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.4.0
 



Thanks,
Anthony


> On Jan 23, 2018, at 2:32 PM, Swapnil Bawaskar  wrote:
> 
> This is the first release candidate for Apache Geode, version 1.4.0.
> Thanks to all the community members for their contributions to this
> release!
> 
> *** Please download, test and vote by Friday, January 26, 1200 hrs
> US Pacific. ***
> 
> It fixes 277 issues. release notes can be found at:
> https://issues.apache.org/jira/projects/GEODE/versions/12341842
> 
> Note that we are voting upon the source tags: rel/v1.4.0.RC1
> https://github.com/apache/geode/tree/rel/v1.4.0.RC1
> https://github.com/apache/geode-examples/tree/rel/v1.4.0.RC1
> 
> Commit ID:
> 7dbb280fbff0320efd427c5553a6e666328bc50c (geode)
> ced35f88173b04ac8e104b9cae10cc38289675fa (geode-examples)
> 
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/geode/1.4.0.RC1
> 
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachegeode-1036
> 
> 
> Geode's KEYS file containing PGP keys we use to sign the release:
> https://github.com/apache/geode/blob/develop/KEYS
> 
> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB

Re: build is broken

[Anthony Baker]

There was a PR for these changes:  https://github.com/apache/geode/pull/1212 


I didn’t comment on the PR but I did review the dependency for license 
compatibility (it’s ASL 2.0).

Anthony


> On Jan 23, 2018, at 2:29 PM, Mark Bretl  wrote:
> 
> I agree with Kirk. A simple notice to the dev list to say what dependency
> and why it is being added, this would also be an opportunity to make sure
> the dependency is not using a Cat-X license [1].
> 
> --Mark
> 
> [1] https://www.apache.org/legal/resolved.html#category-x
> 
> On Tue, Jan 23, 2018 at 2:09 PM, Kirk Lund  wrote:
> 
>> Discussing the addition of new dependencies is something the community
>> should be doing on the dev-list regardless of how many git repos we have.
>> It's part of the Apache Way that we are beholden to as an Apache project.
>> 
>> On Tue, Jan 23, 2018 at 11:25 AM, Udo Kohlmeyer 
>> wrote:
>> 
>>> This really just another reason that we should be pushing for the
>>> separation of Geode into different (independent) modules with potentially
>>> separate repos.
>>> This way modules can advance and iterate faster without having to be
>>> influenced by external factors causing unrelated failures.
>>> 
>>> I would push for this, rather than having to slow the whole development
>>> process down, whilst waiting for consensus on libraries used for external
>>> tooling.
>>> 
>>> --Udo
>>> 
>>> On Tue, Jan 23, 2018 at 11:15 AM, Kirk Lund  wrote:
>>> 
 I have it working now. My IntelliJ project was using JDK1.8.0_66 for
>>> Gradle
 which prevented updating of dependencies which then caused compilation
>> in
 IntelliJ to fail on that new dependency.
 
 This doesn't resolve my main issue here which is this: *all new
 dependencies should be discussed on the dev list before introducing
>> them
>>> in
 a commit*. Can you please point me at an email that proposes adding
>> this
 dependency?
 
 On Tue, Jan 23, 2018 at 10:10 AM, Anthony Baker 
>>> wrote:
 
> I just ran `gradle clean build` from HEAD and did not get any errors.
> 
>> On Jan 23, 2018, at 10:08 AM, Jens Deppe 
>> wrote:
>> 
>> The dependency is the rmi-io library. It provides functionality for
>> streaming data over RMI.
>> 
>> It was added a few weeks ago, so I'm not sure why your build is
 breaking
>> now.
>> 
>> --Jens
>> 
>> On Tue, Jan 23, 2018 at 9:41 AM, Kirk Lund 
>> wrote:
>> 
>>> Looks like this dependency is Jackcess (used for reading and
>> writing
 MS
>>> Access databases). I cannot find any discussion about adding this
>>> dependency on the dev-list. Why does Geode need to talk to MS
>> Access
>>> databases for Cluster Configuration?
>>> 
>>> It does have an Apache 2.0 license, but I think there are two
>>> problems
> with
>>> adding Jackcess: 1) all new dependencies should be discussed on
 dev-list
>>> BEFORE adding them to Geode, 2) we shouldn't need to talk to MS
>>> Access
> for
>>> Cluster Configuration.
>>> 
>>> On Tue, Jan 23, 2018 at 9:30 AM, Kirk Lund 
>>> wrote:
>>> 
 I just tried to compile Geode develop.
>> ClusterConfigurationService
 has
>>> two
 broken imports that don't exist:
 
 import com.healthmarketscience.rmiio.RemoteInputStream;
 import com.healthmarketscience.rmiio.RemoteInputStreamClient;
 
 Was there a new dependency added to Geode?
 
>>> 
> 
> 
 
>>> 
>>