It's nice to see some cool discussion. :) David M. Lloyd wrote: > On 04/28/2008 08:57 AM, Emmanuel Lecharny wrote: >> "이희승 (Trustin Lee) <[EMAIL PROTECTED]>" wrote: >>> I thought about the current MINA API these days while I am idle, and got >>> some idea of improvements: >>> >>> 1) Split IoBuffer into two parts - array and buffer >>> >>> IoBuffer is basically an improvement to ByteBuffer. Improvement here >>> means it inherited some bad asset from ByteBuffer - all the stateful >>> properties such as position, limit and mark. I think they should be >>> provided as a separate class, and there should be classes dedicated for >>> storing bytes only (something like ByteArray?) >>> >> I was also thinking about this class. ByteBuffers are allocated once, >> with a specific size, and the JVM will optimize the way they are >> handled. When dealing with IO, I don't see a lot of reasons to have >> expandable BB. > > Indeed. This was my motivation for opening DIRMINA-489. > >> I think that using a byte[] (for instance in the encoder), transform >> it to a ByteBuffer, is another way to deal with the problem. >> >> One important point is that ByteBuffers are just mean to contains a >> fixed amount of data. It's a buffer, not a data structure. >> Transforming ByteBuffer to make them able to expand twist their >> intrinsic semantic. > > Yes, it makes far more sense to accumulate buffers until you can decode > your message from it. > >> So I would say that BB should be used on the very low level (reading >> data and sending data), but then, the other layers should use byte[] >> or a stream of bytes. > > I don't see the advantage of using byte[] honestly - using at the least > a wrapper object seems preferable. And if you're going to use a wrapper > object, why not just use ByteBuffer.
I'd prefer to introduce an interface type because ByteBuffer is
impossible to extend. I can do some quick and dirty coding to prove the
concept. How does it sound?
>> This will lead to very intersting performances questions :
>> - how to handle large stream of data ?
>
> One buffer at a time. :-)
>
>> - should we serialize the stream at some point ?
>
> What do you mean by "serialize"?
>
>> - how to write an efficient decoder, when you may receive fractions of
>> what you are waiting for ?
>
> An ideal decoder would be a state machine which can be entered and
> exited at any state. This way, even a partial buffer can be fully
> consumed before returning to wait for the next buffer.
We already have a state machine based codec but it's far from
documentation. Blame me. ;)
My long term idea is to write something similar to ANTLR (parser
generator) which works in a binary level; we can call it decoder generator.
>> - how to write an efficient encoder when you have no idea about the
>> size of the data you are going to send ?
>
> Use a buffer factory, such as IoBufferAllocator, or use an even simpler
> interface like this:
>
> public interface BufferFactory {
> ByteBuffer createBuffer();
> }
>
> which mass-produces pre-sized buffers. In the case of stream-oriented
> systems like TCP or serial, you could probably send buffers as you fill
> them. For message-oriented protocols like UDP, you can accumulate all
> the buffers to send, and then use a single gathering write to send them
> as a single message (yes, this stinks in the current NIO implementation,
> as Trustin pointed out in DIRMINA-518, but it's no worse than the
> repeated copying that auto-expanding buffers use; and APR and other
> possible backends [and, if I have any say at all in it, future OpenJDK
> implementations] would hopefully not suffer from this limitation).
Allocating a pre-sized buffer per connection can cause OOM when there
are massive number of idle connections. We could use composite buffer
in this case too because we can adjust the size of the buffer more
flexibly without causing reallocation.
Gathering write is also a piece of cake if composite buffer is realized.
Of course it should be friendly with your suggested allocation mechanism.
>> For all these reasons, the mail I sent a few days ago express my
>> personnal opinion that IoBuffer may be a little bit overkilling
>> (remember that this class -and the associated tests- represent around
>> 13% of all mina common code ! )
>
> Yes, that's very heavy. I looked at resolving DIRMINA-489 more than
> once, and was overwhelmed by the sheer number of methods that had to be
> implemented, and the overly complex class structure.
>
> One option could be to use ByteBuffer with some static support methods,
> and streams to act as the "user interface" into collections of buffers.
> For example, an InputStream that reads from a collection of buffers, and
> an OutputStream that is configurable to auto-allocate buffers,
> performing an action every time a buffer is filled:
>
> public interface BufferSink {
> void handleBuffer(ByteBuffer buffer);
> }
>
> Another option is to skip ByteBuffers and go with raw byte[] objects
> (though this closes the door completely to direct buffers).
>
> Yet another option is to have a simplified abstraction for byte arrays
> like Trustin proposes, and use the stream cleasses for the buffer state
> implementation.
>
> This is all in addition to Trustin's idea of providing a byte array
> abstraction and a buffer state abstraction class.
Thanks for mentioning all possibilities. I'm leaning toward to my
original idea of providing byte array abstraction and buffer state
abstraction types because it can cover all cases you've mentioned.
Also, we can provide the sheer number of getters and putters for users
who like them anyway - it should end up with much simpler one monolithic
class - a way simpler.
--
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/
signature.asc
Description: OpenPGP digital signature
