Ok. I'll test the throughput on it with the tests I have. Thanks Chad
Sent from my iPad On Jan 17, 2012, at 11:33 AM, Emmanuel Lecharny <[email protected]> wrote: > IFAICT, on MINA 2, you should not have any issue. > > The loop where we write buffers into the channel is : > > ... > final int maxWrittenBytes = session.getConfig().getMaxReadBufferSize() > + (session.getConfig().getMaxReadBufferSize() >>> 1); > int writtenBytes = 0; > WriteRequest req = null; > > try { > // Clear OP_WRITE > setInterestedInWrite(session, false); > > do { > // Check for pending writes. > req = session.getCurrentWriteRequest(); > > if (req == null) { > req = writeRequestQueue.poll(session); > > if (req == null) { > break; > } > > session.setCurrentWriteRequest(req); > } > > int localWrittenBytes = 0; > Object message = req.getMessage(); > > if (message instanceof IoBuffer) { > localWrittenBytes = writeBuffer(session, req, > hasFragmentation, maxWrittenBytes - writtenBytes, > currentTime); > > if (( localWrittenBytes > 0 ) > && ((IoBuffer) message).hasRemaining()) { > // the buffer isn't empty, we re-interest it in writing > writtenBytes += localWrittenBytes; > setInterestedInWrite(session, true); > return false; > } > } else { // Blahhh } > > if (localWrittenBytes == 0) { > // Kernel buffer is full. > setInterestedInWrite(session, true); > return false; > } > > writtenBytes += localWrittenBytes; > > if (writtenBytes >= maxWrittenBytes) { > // Wrote too much > scheduleFlush(session); > return false; > } > } while (writtenBytes < maxWrittenBytes); > ... > > with : > > private int writeBuffer(S session, WriteRequest req, > boolean hasFragmentation, int maxLength, long currentTime) > throws Exception { > IoBuffer buf = (IoBuffer) req.getMessage(); > int localWrittenBytes = 0; > > if (buf.hasRemaining()) { > int length; > > if (hasFragmentation) { > length = Math.min(buf.remaining(), maxLength); > } else { > length = buf.remaining(); > } > > localWrittenBytes = write(session, buf, length); > } > > and : > > protected int write(NioSession session, IoBuffer buf, int length) > throws Exception { > if (buf.remaining() <= length) { > return session.getChannel().write(buf.buf()); > } > > int oldLimit = buf.limit(); > buf.limit(buf.position() + length); > try { > return session.getChannel().write(buf.buf()); > } finally { > buf.limit(oldLimit); > } > } > > So we try our best to stuff the channel with as many bytes as possible, > before giving up (either because we don't have anything to write, to because > the channel is full...) > > I don't see if we can do any better. > > On 1/17/12 3:57 PM, Chad Beaulac wrote: >> >> On Jan 17, 2012, at 9:32 AM, Emmanuel Lécharny<[email protected]> wrote: >> >>> On 1/17/12 3:02 PM, Chad Beaulac wrote: >>>> On Mon, Jan 16, 2012 at 1:10 PM, Emmanuel >>>> Lecharny<[email protected]>wrote: >>>> >>>>> On 1/16/12 2:56 PM, Chad Beaulac wrote: >>>>> >>>>>> Emmanuel, (all) >>>>>> >>>>>> I'm working on this Camel ticket: >>>>>> https://issues.apache.org/**jira/browse/CAMEL-2624<https://issues.apache.org/jira/browse/CAMEL-2624> >>>>>> >>>>>> I finished the initial cut of >>>>>> https://issues.apache.org/**jira/browse/CAMEL-3471<https://issues.apache.org/jira/browse/CAMEL-3471>to >>>>>> create a mina2 >>>>>> component in Camel. >>>>>> >>>>>> CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel. >>>>>> >>>>>> Would it be possible to backport the IoBuffer reading and writing >>>>>> discussed >>>>>> in this email thread from Mina3 to Mina2? >>>>>> Following the depth of the stack trace through >>>>>> AbstractIoSession.write(...), I'm a little concerned about the >>>>>> throughput. >>>>>> My current code (mina-less) is supporting single TCP channels with 320+ >>>>>> Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol >>>>>> Buffers codec I wrote. I know if this is a real problem soon when I >>>>>> finish >>>>>> CAMEL-2624 and setup some throughput tests. >>>>>> >>>>> Another option would be to port this to MINA3, and make it work with >>>>> Camel. Right now, MINA 3 is pretty rough, but we have made it works well >>>>> with Http and LDAP. I'd rather spend some time making it a bit more solid >>>>> and working well in your case instead of trying to inject the code in >>>>> MINA2. >>>>> >>>>> Now, it's your call. We can discuss the pros and cons of both approach if >>>>> you like. >>>>> >>>>> >>>> Hi Emmanuel, >>>> >>>> One of my pros/cons trade-offs is time-to-market. I can have a solution in >>>> Camel with Mina2 fairly quickly. Although I might have issues with high >>>> data rate streams. >>>> With that said, my approach would be the following: >>>> 1) Finish CAMEL-2624 with Mina2. This will give me the asynchronous >>>> endpoints I need and a quick time-to-market. I'll put off issues concerning >>>> high throughput. >>>> 2) Work on Mina3 to ensure it has low latency with small data rate streams >>>> and high throughput with large data pipes. >>>> 3) Upgrade my Google protocol buffers codec for Mina3. >>>> 4) When Mina3 is ready, open a new Camel ticket and create a new mina3 >>>> Camel Component. >>>> >>>> What do you think? >>> I'll try to squeeze 2 hours to backport the patch to MINA 2 today or >>> tomorrow. >>> >>> Feel free to ping me on mail or on #mina if I don't send any feedback in >>> the next 2 days (I'm pretty busy and may slip) >>> >>> >>> -- >>> Regards, >>> Cordialement, >>> Emmanuel Lécharny >>> www.iktek.com >>> >> Wow. That is nice! Look forward to checking it out. I'll move forward with >> my plan in the meantime. >> >> Chad >> Sent from my iPhone >> > > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com >
