thanks for the excellent help Trustin, I'll try removing all the packetizing stuff from the encoders themselves, and putting it in an IO filter as you suggest - it sounds like a much better place for that stuff to be. It never seemed quite right putting it in the encoder, but I've only been looking at Mina for a couple of days, so I don't think I've quite got the hang of the architecture yet.
One thing I'm not sure I've understood - In your PacketisingFilter implementation, it looks as if the encoder will still be having to split the message data into buffer size chunks before sending it "down the pipe" to the Packetizing filter which seems to be just adding the header to each buffer. It would seem cleaner for the encoder to just create the encoded message as one big ByteBuffer and the PacketisingFilter to split this up into packets and add the header to each packet. If I did try to do all the packetizing in the filter, though, I'm still not sure how I would cause a "flush" downstream from the filter to actually write the header and data to the network as a single packet. Also, if I did write the IO filter like this, and there's a method to cause a "flush", does the method block until the data has actually been sent, or am I still going to have to deal with the problem of not being able to slice the buffer ?
Is there any movement on this bug yet - it sounds like this may be the next issue I have : I'd be willing to help with a solution if I can.
j
On 23 Jan 2006, at 14:40, Trustin Lee wrote: Hi Joshua,
2006/1/23, joshua portway <[EMAIL PROTECTED]>: Hi,
I have a protocol which communicates via UDP with a hardware device
which requires each message to bit split into separate network
packets, each of which is preceded by a header which contains
information about which message the packet is part of, what packet
number it is, the total length of data in the message, and the
position of the data in this packet in the message etc.
I'm not sure of the best way to implement this in Mina. I've
implemented a custom ProtocolCodecFactory, loosely based on the Mina
DemuxingProtocolCodecFactory, along with a MessageEncoder base class.
Specific MessageEncoder subclasses write a series of ByteBuffers
which make up th econtent of the message to a method in the baseclass
and then call a "flush" method which divides the data up into
packets, prepends the message header to the packet and sends it to
the ProtocolEncoderOutput.
It sounds like you did a great job!
The problem I'm getting is that if the message I'm sending has data
contained in a big ByteBuffer (which most of my messages do), then I
need to divide the buffer into specific packet sized chunks to send.
Since Mina ByteBuffers don't have a "split" method (presumably
because they get re-used) I have been setting the position and limit
of the input buffer to point to the chunk I want to send, and then
passing it off to the ProtocolEncoderOutput. The problem is that
because the ProtocolEncoderOutput runs on another thread, I can't
just change the buffer position and limit for the next chunk and
send it because I have to wait for the ProtocolEncoderOutput to
finish before I change the ByteBuffer positions. The only other
alternative would be to copy each packet into a new buffer which
seems quite wasteful. Latency and throughput are VERY important to my
app, because it's encoding video streams and pushing them to the
network in real time, so I'd rather not be doing a lot of copying of
buffers.
My current home-made network layer doesn't copy ByteBuffers, but
creates view buffers onto the source buffer using ByteBuffer.split,
and passes the header for a packet and the view buffer for the data
to the channels gathering write(ByteBuffer[]) method. Am I doing
something wrong in the way I'm using Mina ? Is there a way to do this
without copying buffers ? I understand your point completely. It's also an issue for other MINA users including ourselves. There's already the JIRA issue which is related with this problem: http://issues.apache.org/jira/browse/DIRMINA-165Please watch the issue, we'll keep updating our idea. Of course, you could post your idea, too. :) Ideally, I think there should be a "packetizing" API in Mina to
allow easy control of how a message is packetized, and to allow each
packet to be wrapped in a protocol-specific packet header. As far as
I can tell no such thing exists, and the closes I've found is the
"mergeAll" method of ProtocolEncoderOutput, but again- this seems to
be implemented by copying all the input ByteBuffers into a single
output ByteBuffer, which can't be the most optimal way of doing it,
surely ? Or am I being naiive ? does the GatheringByteChannel.write
(ByteBuffer[]) method just copy all it's input buffers into a single
output buffer anyway ?
This is a good idea. Actually, you can implement it as an IoFilter. You could implement IoFilter.filterWrite() method to write the header part:
public class PacketizingFilter extends IoFilterAdapter {
public void filterWrite( NextFilter next, WriteRequest req ) {
ByteBuffer buf = ByteBuffer.allocate( ... );
// fill header here
buf.put( ... ) ....
...
buf.flip ();
// Write header first
next.filterWrite( new WriteRequest( buf ) );
// and then body
next.filterWrite( req );
}
}
In your encoder, you don't need to call mergeAll in this case. Please just call write()s. The data will be flushed when encode() method returns. So it's basically similar with write(bb[]) unless you called flush() manually.
thanks for any help, sorry about the long length of the email :)
It was a lot of fun to know that MINA is going to be used to implement real-time media streamer! :)
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C
|
