But my server is broadcasting the message to all the clients.
2007/4/30, Paul Chen <[EMAIL PROTECTED]>:
After curbing the read rate (one quota per acceptor) at a conservative rate, the write problem is automatically gone. This is not an approach that leads to optimal performance, but just a way to protect our Mina-based server not run into OOM. Cheers On 4/28/07, mat <[EMAIL PROTECTED]> wrote: > > Does ReadThrottle filter only solve the "Read" problem? What about > "Write"? > BTW, what is the max concurrent users you can reach? > > > 2007/4/29, Paul Chen <[EMAIL PROTECTED]>: > > > > No matter how we tune JVM, there's still a certain threshold that when > it > > is > > reached, a server should stop taking new requests or even close the > > session > > to free up resources, depends on the logic in the server. > > > > I observed and categorized 2 OOM scenarios in our Mina-based server: > > > > [1] Incoming reqs from SocketAcceptors are too fast and too many. Input > > queue blows up. Particularly when the Mina-based server has persistent > or > > long-lived connections w/ the clients which just keep pumping in reqs > w/o > > even needing time to establish a connection. > > > > [2] Outgoing reqs from SocketConnectors to a service provider are too > > fast/too many, much faster > > than the service provider can handle. Then the output queue > > (writeRequestQueue) blows up. > > > > I reported this problem earlier but I can live with this now by tweaking > > the > > ReadThrottle filter, though > > it is very hard to determine the threshold, especially when the > > mina-based > > server has multiple > > Acceptors. I end up assigning a "quota" per Acceptor. > > > > Regards > > > > On 4/28/07, Mark Webb <[EMAIL PROTECTED]> wrote: > > > > > > have you looked into tweaking your JVM? > > > > > > http://java.sun.com/performance/reference/whitepapers/tuning.html > > > > > > On 4/28/07, mat <[EMAIL PROTECTED]> wrote: > > > > I also observed that the memory increaed way too fast when heavy > > > loading. > > > > > > > > 2007/4/26, Gaston Dombiak <[EMAIL PROTECTED]>: > > > > > > > > > > Hey Mark, > > > > > > > > > > Based on the statistics I've been collecting from MINA I can tell > > that > > > > > our OOM problems happen when: > > > > > > > > > > 1) the eventQueue instance variable inside of ExecutorFilter fills > > up > > > > > (that would be the case for incoming traffic) or > > > > > 2) the SocketSessionImpl#getScheduledWriteRequests() queue starts > to > > > > > fill up (that would be the case for outgoing traffic) > > > > > > > > > > Regards, > > > > > > > > > > -- Gato > > > > > > > > > > -----Original Message----- > > > > > From: Mark Webb [mailto:[EMAIL PROTECTED] > > > > > Sent: Wednesday, April 25, 2007 1:53 PM > > > > > To: [email protected] > > > > > Subject: Re: Permanent solution for OOM errors > > > > > > > > > > Maybe one approach would be to categorize the ways in which people > > > > > reach OOM conditions. Do these conditions happen only when the > > input > > > > > and/or output queues get really large, or are there other reasons. > > > > > Once we can start to categorize the OOM conditions that users are > > > > > experiencing, we can properly move forward with solutions. > > > > > > > > > > Basically what I mean is, if the OOM happens when queues fill up, > > then > > > > > throttle filters would be best. If the OOM happen for other > > reasons, > > > > > people could look at other areas of MINA. Just not sure there is > > one > > > > > solution that will fix all the problems. > > > > > > > > > > ...just my 2 cents. > > > > > > > > > > -- > > > > > ..Cheers > > > > > Mark > > > > > > > > > > > > > > > On 4/25/07, Gaston Dombiak <[EMAIL PROTECTED]> wrote: > > > > > > Hey Marcin, > > > > > > > > > > > > I see what you mean. I'm really fine with either option as long > as > > > > > MINA > > > > > > handles the OOM problem. As I posted a month or so ago the > > > throttling > > > > > > per connection is not enough. We (Openfire and probably others) > > also > > > > > > need a global throttling. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > -- Gato > > > > > > > > > > > > -----Original Message----- > > > > > > From: Marcin Waldowski [mailto:[EMAIL PROTECTED] > > > > > > Sent: Wednesday, April 25, 2007 1:36 PM > > > > > > To: [email protected] > > > > > > Subject: Re: Permanent solution for OOM errors > > > > > > > > > > > > Hej Gaston, > > > > > > > > > > > > Gaston Dombiak wrote: > > > > > > > Hey Marcin, > > > > > > > > > > > > > > I'm curious to hear from you why do you think so? I absolutely > > > agree > > > > > > > that MINA has to provide easy ways to handle OOM problems due > to > > > > > heavy > > > > > > > incoming or outgoing traffic. > > > > > > > > > > > > > > This is by far the more common problem people are reporting > with > > > > > > > Openfire now that we moved to MINA (when under heavy load). > > > > > > > > > > > > > > > > > > > Hmm, my first thought was that it introduce the complexity of > > > > > > configuration like ThreadModel. But probably I'm wrong. > > > > > > > > > > > > I agree definitely that MINA need solution for OOM, but it could > > be > > > > > also > > > > > > > > > > > > ReadThrottleFilterBuilder and WriteThrottleFilter. They are easy > > way > > > > > to > > > > > > handle OOM and (what is important) users are aware that they use > > it. > > > > > > > > > > > > But I'm not agains integrate it as default to > Acceptor/Connector, > > > > > really > > > > > > > > > > > > :) Meybe I even souldn't post my doubts to this subject because > I > > > > > > haven't contributed to MINA so far... > > > > > > > > > > > > Regards, Marcin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > ..Cheers > > > Mark > > > > > >
