Apologies that this was not raised earlier in discussion but I'm happy to
describe it now.

*Background:*

When deploying jars into Geode they are moved through the system as simple
byte[] blobs. This obviously consumes memory. The various affected areas
are:

- gfsh reads the jars into memory
- the jars are pushed to the locator (via a jmx call) - again creating a
byte[] blob on the locator
- from the locator, the jars are pushed to all servers via a function call
(also sending the jars as byte[] blobs).

Obviously if the jar is small this would not be a problem, however in
memory constrained systems or with large jars this is obviously going to
put pressure on memory and possibly result in OOM situations. In fact, the
reason this came up was that some folks were unable to deploy a 40Mb jar to
a 512Mb (heap) locator.

*rmi-io:*

After doing some research, it seemed that the ideal solution would be
something that allows for serializing Input/OutputStreams. Java doesn't
provide anything natively.

One library that stood out as being robust and feature complete was rmi-io
[1]. This allows for serializing a remote Input/OutputStream object which
then lets us completely avoid having to pull deploying jars into memory
everywhere. Under the covers it uses RMI and allows for either 'pulling' or
'pushing' data. The reference page [2] has nice sequence diagrams.

If anyone sees any issues with this, please do raise them. The current
usage of this has not changed any user-facing interaction so ultimately
changing the actual implemented fix for this problem (if we needed to)
would not have any external effect.

Thanks
--Jens

[1] http://openhms.sourceforge.net/rmiio
[2] http://openhms.sourceforge.net/rmiio/class_reference.html

Reply via email to