Thanks Trustin. I'll discuss this issue with Emmanuel when he's back next week and work out exactly how to integrate this support into ApacheDS and the builds of MINA it uses etc (at this stage I think the upcoming AD 1.5.2 release is targetting MINA 1.1.6). I tried adding this class to my MINA 1.1 trunk build but it doesn't compile due to differences between 1.1 trunk and trunk, I'll investigate further if I find time in the next few days.
On Mon, Apr 14, 2008 at 11:00 PM, "이희승 (Trustin Lee) <[EMAIL PROTECTED]> wrote: > I checked in the solution for the (relatively) fast writer issue: > > http://tinyurl.com/3wawh2 > > WriteRequestFilter basically reuses IoEventQueueHandler interface and > its implementation (IoEventQueueThrottle) to make the IoSession.write() > call block and unblock. The code is quite straightforward. Please let > me know if you have any further questions. > > > > Norval Hope wrote: > > True enough about assuming the reader and writer are in different threads. > > > > My patch attached to the issue leaves the use of a blocking queue > > configurable, but because SessionBuffer is a private static class > > inside ExceutorFilter I used a system property which is pretty sucky, > > but if this choice is at least left configurable the "both is same > > thread" case can avoid a deadlock. But then again, if they are in the > > same thread there isn't really any point in a queue at all, is there? > > > > Thanks > > > > On Mon, Apr 14, 2008 at 3:48 PM, "이희승 (Trustin Lee) <[EMAIL PROTECTED]> > wrote: > >> Having a blocking queue will work if session.write() is called from a > >> different thread than an I/O processor thread. Otherwise, the whole > >> server will be dead-locked. However, I think it will be fine as long as > >> it's well-documented and provided as an IoFilter. > >> > >> Also, my idea about raising an exception doesn't look like a good idea > >> because a user will always handle the exception, which makes the > >> IoHandler implementation unnecessarily complicated. > >> > >> Let me try to roll out an IoFilter which prevents OOM on the writer side > >> as you requested. > >> > >> Thanks, > >> > >> > >> Norval Hope wrote: > >>> Hi Guys, > >>> > >>> Can I say from my point of view that I'll be happy with any solution > >>> matching roughly the same ApacheDS behaviour that my patch submitted > >>> to http://issues.apache.org/jira/browse/DIRSERVER-1161 achieves, which > >>> used a blocking queue as a simple way of throttling the server when > >>> talking to a slow client. I can see that checking scheduledWriteBytes > >>> could provide another alternative and don't have a problem with this > >>> approach as long as people more expert in MINA / ApacheDS do the > >>> coding :-) > >>> > >>> Emmanuel - in raising starting this thread on MINA-DEV I justed wanted > >>> to check if you think this is the whole answer to DIRSERVER-1161 or > >>> just one part of it? From what I've seen in my debugging sessions the > >>> rest of my patch is needed as otherwise the server keeps writing and > >>> flushing messages but they are never sent to the client, hence the > >>> need for the server to drain the search results in a separate thread. > >>> > >>> Thanks, > >>> Norval > >>> > >>> On Mon, Apr 14, 2008 at 11:38 AM, "이희승 (Trustin Lee) <[EMAIL PROTECTED]> > wrote: > >>>> Emmanuel Lecharny wrote: > >>>> > "이희승 (Trustin Lee) <[EMAIL PROTECTED]>" wrote: > >>>> >> Hi Emmanuel, > >>>> >> > >>>> > Hi again, > >>>> >> You might be interested in the following question from Gaston > Dombiak: > >>>> >> > >>>> >> http://markmail.org/message/uylaf2zauta6ppe7 > >>>> >> > >>>> >> And there's an answer there in my response made at 9 Apr 2008. > >>>> >> > >>>> > I don't think that your answer fits well in the scope of the > question I > >>>> > pushed. We don't want to throttle write on the server side, we just > want > >>>> > to send bytes as fast as possible, but be sure that those bytes are > sent > >>>> > and read by the client before sending some new (which will have to be > >>>> > accumulated somewhere on the server until you get an OOM). > >>>> > > >>>> > This is really a critical issue for every server using MINA as a > >>>> > malicious client could perfectly be use to kill the server just by > >>>> > sending a request and never reading more than a few bytes of data, > if I > >>>> > interpret correctly what I have seen. > >>>> > >>>> The client might be malicious or not because it can be simply slow > >>>> without its will. Whatever kind of client is connected, the server > >>>> should be responsible for slow connection. Again, there are two > mechanisms: > >>>> > >>>> 1) Check scheduledWriteBytes before you write. If you are going to > >>>> stream many messages, you might not want to write all of them > >>>> immediately but want to write them chunk by chunk. Then you can check > >>>> the scheduledWriteBytes in your WriteFuture listener or messageSent > >>>> event handler. > >>>> > >>>> 2) There's writeTimeout property if a client is not reading even one > >>>> message. > >>>> > >>>> > >>>> > Peter Royal suggested to use a Callback to handle with such a > problem, > >>>> > and it sounds like a good idea (we have listeners we can use for > that in > >>>> > WriteFuture), but this has to be done in the ProtocolCodecFilter > code, > >>>> > which is a part of the MINA API. > >>>> > >>>> It's a good idea to privde a solution for preventing OOM due to fast > >>>> write. Also, I think it's a must-have feature for all clients and > >>>> servers. Therefore, what about adding the following two properties: > >>>> > >>>> * IoSessionConfig.scheduledWriteBytesThreshold > >>>> * IoSessionConfig.scheduledWriteMessagesThreshold > >>>> > >>>> and making MINA to raise an exception when > >>>> scheduledWrite(Bytes|Messages) becomes too large. Then your IoHandler > >>>> will be notified with an exceptionCaught event. > >>>> > >>>> WDYT? > >>>> > >>>> > >>>> > >>>> -- > >>>> Trustin Lee - Principal Software Engineer, JBoss, Red Hat > >>>> -- > >>>> what we call human nature is actually human habit > >>>> -- > >>>> http://gleamynode.net/ > >>>> > >>>> > >> -- > >> > >> Trustin Lee - Principal Software Engineer, JBoss, Red Hat > >> -- > >> what we call human nature is actually human habit > >> -- > >> http://gleamynode.net/ > >> > >> > > -- > > > Trustin Lee - Principal Software Engineer, JBoss, Red Hat > -- > what we call human nature is actually human habit > -- > http://gleamynode.net/ > >
