Candid and I discussed this off list. The sum-up is this: Iceberg, you hit the nail on the head with my position.
However, it's clear that not everyone uses this feature and it may be more useful to others if it had a configurable thread-timeout. I told Candid that I'd investigate the effort it would take to add this feature. Stay tuned. Whether or not Massimo thinks this feature would be good for web2py is up to him, but even if added, it would not be the Rocket default. -tim On Jul 7, 9:13 am, Iceberg <iceb...@21cn.com> wrote: > I have different opinion, Candid. > > As Timbo and I have discussed in this thread, rocket differentiates "long > running requests", seeing it is caused by client side or server side. Rocket > only terminates long request comes from the client side, say, a very slow > http request from a client (possibly a DoS attack). On the contrary, on the > server side, it is the app (a.k.a. its developer) 's responsibility to ensure > most requests can be served quickly, and it is also developer's right to > occasionally do long time calculation whenever needed. Rocket gives developer > the flexibility to choose what they want. That is better than killing a > (long) running thread unconditionally. > > I love rocket. :-) > > Best regards, > Iceberg, 2010-Jul-07, 21:47(PM), Wed > > > > ----------------------- Original Message ----------------------- > From: Candid <roman.bat...@gmail.com> Sender: web2py@googlegroups.com > To: web2py-users <web2py@googlegroups.com> > Date: Wed, 7 Jul 2010 05:21:16 -0700 (PDT) > Subject: [web2py] Re: Will rocket close connections if it takes long time? > > ------------------- > > > Thank you for clarification, Timbo. > > > Is not #3 a problem though? If Rocket doesn't terminate the request if > > it's running for too long, is not Rocket's thread pool going to be > > exhausted if it runs too many long running requests? The problem here > > is that one application could potentially exhaust all working thread, > > preventing rocket from serving request for other applications. It > > looks like this is what's happening to one of my server - see this > > discussion:http://groups.google.com/group/web2py/browse_thread/thread/34f5411692... > > > Am I missing something or is this how it's supposed to be? > > > On Jul 5, 12:33 am, Timbo <tfarr...@owassobible.org> wrote: > > > 1) True > > > 2) Basically true. You seem to indicate that 1 & 2 should not co- > > > exist or are somehow at odds with each other. If that's what you're > > > indicating, I'm not following your logic as to why they should be > > > mutually exclusive. Can you explain a little more? > > > 3) In Rocket's case, no. While a client could disconnect at anytime, > > > Rocket will allow the application server to run a request as long as > > > it likes. If the client connection is still open when it's ready to > > > send it's response, it will send successfully. Rocket does not kill > > > threads that run for "too long". > > > 4) Rocket always applies the socket timeout (rocket.py line 854), this > > > should override Py2.5's default (whatever it may be). Rocket enables > > > Python's socket timeout protection on every Python platform that > > > supports it. > > > > Does this clear things up? > > > > On Jul 4, 4:30 am, Iceberg <iceb...@21cn.com> wrote: > > > > > Thanks for all these info, Tim. You said: > > > > "The timeout ... is a timeout based on inactivity between http > > > > requests. So the long requests that Iceberg is seeing are perfectly > > > > normal and acceptable." > > > > > According to my reading of the rocket-science code (and I could be > > > > wrong), here is my understanding and questions: > > > > > 1. The timeout is a timeout based on inactivity since one request > > > > socket's latest select-and-readable event. The time gap could be > > > > caused by two http requests which somehow reuse the socket (because > > > > keep-alive?), or caused by one very slow http request from a client > > > > (possibly a DoS attack). > > > > > 2. BUT the timeout check is only performed after the request's > > > > socket encountered socket.timeout exception. According to line 800 of > > > > rocket.py: > > > > if typ == SocketTimeout: > > > > self.err_log.debug('Socket timed out') > > > > self.wait_queue.put(self.conn) > > > > And then the request's control is transferred to the Monitor, the > > > > timeout check is at line 449 of rocket.py: > > > > if now - c.start_time >= self.timeout: > > > > stale.add(c) > > > > > 3. What if, a normal request is read from client side within > > > > seconds, then being calculated for quite some time on the server side, > > > > will this situation trigger the timeout protection? I guess not, > > > > because no socket.timeout will be raised during the calculation, > > > > although the final calculated result will probably not be sent due to > > > > the socket is already timeout or even closed from the client side. If > > > > this is true (is it?), then my maintenance scenario is still doable > > > > because no result is expected, I only need the maintenance job can be > > > > triggered and can run as long as it takes. > > > > > 4. On the other hand, at least on Windows XP, python 2.5.4, by > > > > default, socket has no socket.timeout, so the above check is not > > > > performed at all, is it? So the rocket timeout protection is not > > > > activated anyway. > > > > > -- > > > > Iceberg > > > > > On Jul4, 9:26am, Timbo <tfarr...@owassobible.org> wrote: > > > > > > Rocket has a longer default wait time before it closes connections in > > > > > comparison to CherryPy. You might check web2py code, it is not likely > > > > > using the same default timeout as Rocket proper. > > > > > See:http://packages.python.org/rocket/usage.html#timeout > > > > > > This is a DoS vulnerability on CherryPy but not Rocket because Rocket > > > > > pushes waiting threads to a monitor queue so that they don't occupy > > > > > worker threads. The monitor queue can easily handle the socket limit > > > > > of connections. In this manner, Rocket can handle many thousands of > > > > > connections without causing new connections to have to wait as > > > > > CherryPy does. There is a limit on number of active connections, but > > > > > that limit is the number of worker threads which is > > > > > configurable:http://packages.python.org/rocket/usage.html#max-threads > > > > > > The timeout mentioned in the link above is a timeout based on > > > > > inactivity between http requests. So the long requests that Iceberg > > > > > is seeing are perfectly normal and acceptable. > > > > > > Browsers such as Chrome and FF will leave a connection open for quite > > > > > some time or until the tab is closed (even if the user has browsed > > > > > away from the Rocket-running server domain). Rocket basically wants > > > > > to keep those connections open since they don't cost significantly to > > > > > wait in the monitor queue. > > > > > > -tim
