On 8/19/11 2:26 PM, Alan D. Cabrera wrote:
On Aug 19, 2011, at 5:19 AM, Emmanuel Lecharny wrote:
On 8/19/11 2:16 PM, Alan D. Cabrera wrote:
I'm wondering. Do you guys think it's a good idea? It seems to make things
pretty complicated and adds another dimension to groking the behavior of your
service. I'm not sure that it's necessary.
Definitively a bad idea.
What we need is an abstraction on top of an Array of ByteBuffer (the Java NIO
class), which extends the size by adding new ByteBuffer on the fly.
The array must behave exactly as the ByteBuffer.
I wrote such a class 2 years ago, but I can't find the code. Will look again on
my USB keys this week-end.
So, what is the scenario that we're trying to support? I imagine appending
headers to binary data would be one. In this case is an Array of ByteBuffers
really needed? Why not just send down one ByteBuffer for the header and
another for the body that was sent to you?
Julien and Steve laready responded with clear examples where expandable
BB are necessary. I do think this is a common case, assuming you can
perfectly receive the data byte by byte, and you want to store those
bytes in a place which does not need to be reallocated every time.
The current IoBuffer, when full, is copied in a new loBuffer which size
is doubled. If you are expecting big PDUs, you might allocate huge
Iobuffer for nothing, wasting space. Plus you have to copy all the old
IoBuffer into the new one. Not really a good thing.
*if* we are using DirectBuffer, we even might want to use fixed size
IoBuffer (say 1k), and store them in a queue of avilable free buffers
for reuse, sparing the cost of allocating them (to be validated, I'm not
sure that the cost of managing concurrent access to this queue does not
overweight the cost of allocating a new direct buffer)
For HeapBuffers, i'm quite sure that we should allocate new ByteBuffer
with the exact size of the received data (if it's 10 bytes, then the new
ByteBuffer will be 10 byte slong. If it's bigger, then fine). This
additional ByteBuffer can be appended a the end of the ByteBuffer array,
and the filter will be able to process the list when it has everything
needed.
Julien mentionned the Cumulative decoder, which can be used in may cases :
- we are waiting for a \n (HTTP protocol)
- we are waiting for a closing tag (XML)
- we are dealing with LV (Length/value) PDU, and the received value is
not fully received (we are expecting Length bytes)
- we are dealing with fixed size PDU
One more thing to consider : at some point, we may want to flush the
ByteBuffer to disk, if we are waiting for a huge PDU, to avoid sucking
all the memory : this is typically the case in LDAP when receiving a
JpegPhot attribute, inside a PDU (we don't know that it's a JpegPhoto
attribute until we have read and processed teh full PDU). With a proxy
on top of the ByteBuffer, we can eventually hide the fact that the bytes
are on disk.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
