Darrel Schneider created GEODE-1281:
---------------------------------------
Summary: Client/Server implementation could do less copying when
sending messages
Key: GEODE-1281
URL: https://issues.apache.org/jira/browse/GEODE-1281
Project: Geode
Issue Type: Improvement
Components: offheap
Reporter: Darrel Schneider
Here are some rough ideas for improving the client/server messaging to do less
copying. In particular they involve not copying off-heap values to the heap
when trying to send the off-heap value back to a client.
Read them in the context of this method (which is used to send back a getAll
response):
com.gemstone.gemfire.internal.cache.tier.sockets.Message.serializeAndAddPartNoCopying(Object)
1. Change Part to look for an HDOS and just pass a reference to its
DirectByteBuffer. Then change HDOS sendTo(SocketChannel...) to use the
GatheringByteChannel to write a bunch of bbs.
2. This code optimizes one part which works pretty good for getAll since all
the values are returned in one part. But the following seems even better...
Change Message to consolidate all the part hdos bb lists into a single bb array
and have it do the GatheringByteChannel write. Message can use slice for the
small parts (msg header and part header) that are not in the parts data (its a
byte array, Chunk, or HDOS).
3. EVEN BETTER: the message can have a single HDOS which owns a direct comm
buffer. It can reserve space if it does not yet know the value to write (for
example the size of the message or part). If we write something to the HDOS
that is direct then it does not need to be copied. But large heap byte arrays
will need to be copied to the hdos (the socket write does this anyway). If the
direct buffer is full then we can allocate another one. If a part is already in
a heap byte array then we could defer copying it by slicing the current direct
bb and then adding the heap byte array as bb using ByteBuffer.wrap. Once we
have all the data in the HDOS we can finally generate the header and then start
working on sending the ByteBuffers to the channel. If we have room in a direct
bb then we can copy a heap bb to it. Otherwise we can write the bb ahead of it
which would free up room to copy the heap bb to the existing direct bb without
needing to allocate extra direct bbs. Delaying the flush uses more direct
memory but reduces the number of system calls.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
