Re: [Zope] [BUG] Background processes interfere with Zope's HTTP responses
Hi Steve, Steve Spicklemire writes: Hmm.. I just tried this with my highly experimental product (LocalProc 0.0.1) and I didn't observe the same effect that you did behind my Proxy Server. Whether you see this effect or not depends on the HTTP processing in the chain between your browser and the server. When all chain members use multi-request mode, then the fact that the connection is kept open longer than necessary has no visible effect. However, applications that strictly follow old HTTP 1.0 versions will use single-request mode. Then you may observe the delay. An optimal application to view the effect is "ZPublisher.ZClient". ... The main difference I see between what you are doing and what I do.. is that I redirect stdout and stderr so that my subprocess has no open files shared with the parent process. That was also my first thought. But in fact, all currently open request sockets are shared with the process as well. Dieter ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope-dev] Re: [Zope] [BUG] Background processes interfere with Zope's HTTP responses
He reported, that this response page is build only half, then stops, to be completed only half a minute later, Depending on how long the page is I never see the bottom. When I use the products page I have to refresh 2-3 times to see the bottom part. This is thru a proxy only. I try to reproduce the behaviour in our test environment and fail. I do not have to wait, until the newsletter generation finished. Then, I use the official service URL and see, I observe the same behaviour. The difference: in the test environment, the browser connects directly to ZServer; with the official URL, it connects via a proxy. Puzzling! I'm having this same sort of problem w/Zope 2.3a1. I just started trying to use Zope through a proxy. If I work directly on the machine I have no problems. How to fix the problem: There should be a (file) control "CloseOnExec", that tells Unix to automatically close the file like object upon an exec. ZServer should probably use it on each of its sockets. I can not see a serious application that should have direct access to ZServer's HTTP request socket. I will soon file a bug report into the Collector. Dieter I'm on the win32 platform and thought that the problem was with my proxy software. Now I'm not so sure. -Jimbo ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [Zope] [BUG] Background processes interfere with Zope's HTTP responses
How to fix the problem: There should be a (file) control "CloseOnExec", that tells Unix to automatically close the file like object upon an exec. ZServer should probably use it on each of its sockets. I can not see a serious application that should have direct access to ZServer's HTTP request socket. I will soon file a bug report into the Collector. Dieter I'm on the win32 platform and thought that the problem was with my proxy software. Now I'm not so sure. Note that what Dieter is talking about would only affect you if you were spawning an external process that inherited the filehandles of the zope process. Brian Lloyd[EMAIL PROTECTED] Software Engineer 540.371.6909 Digital Creations http://www.digicool.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope] [BUG] Background processes interfere with Zope's HTTP responses
Hi Deiter,
Hmm.. I just tried this with my highly experimental product
(LocalProc 0.0.1) and I didn't observe the same effect that you did
behind my Proxy Server. (I normally run with Rewrite... but when I got
your note I tried ProxyPass and it worked on the first try.)
(Good howto Anser! http://www.zope.org/Members/anser/apache_zserver/)
Anyway... I ran 'sleep 20' as my local process (I checked the
'background' button on my product) and the web page returned
immediately, but the process ran on in the background as I wished. The
main difference I see between what you are doing and what I do.. is
that I redirect stdout and stderr so that my subprocess has no open
files shared with the parent process.. anyway.. you might try this to
see if it suits your needs. Lightly tested on FreeBSD only.. but any
unix should work the same...
http://www.zope.org/Members/sspickle/LocalProc
-steve
"Dieter" == Dieter Maurer [EMAIL PROTECTED] writes:
Dieter Today was again a hard Zope day.
Dieter Did you ever hear, that usually (i.e. if you do not do
Dieter special things) Zope renders a complete page before it
Dieter starts sending the result back to the client. This
Dieter implies that you should never see only half of a page.
Dieter I heard it and I saw the code that does it. Therefore, I
Dieter was convinced that this were really the case.
Dieter Then our client phoned:
Dieter He visits a page that allows him to generate a
Dieter newsletter. When he presses the "generate newsletter
Dieter button", a result page is build that tells him that the
Dieter newsletter is being generated and send is a few minutes.
Dieter He reported, that this response page is build only
Dieter half, then stops, to be completed only half a minute
Dieter later, at the same time when the newsletter arrives. The
Dieter effect is reproducible. If newsletter generation takes
Dieter longer, he must wait longer for the page to complete.
Dieter He is convinced that we generate the newsletter
Dieter synchronously and let him wait until the generation is
Dieter complete. And he is angry.
Dieter I know, that the newsletter is generated in a
Dieter background process, started in an external method with:
Dieteros.system("gen_newsletter ")
Dieter I try to reproduce the behaviour in our test
Dieter environment and fail. I do not have to wait, until the
Dieter newsletter generation finished. Then, I use the official
Dieter service URL and see, I observe the same behaviour. The
Dieter difference: in the test environment, the browser connects
Dieter directly to ZServer; with the official URL, it connects
Dieter via a proxy.
Dieter Puzzling!
Dieter What happened?
Dieter The background process inherits Zope's open file
Dieter descriptors. Among them are all currently open HTTP
Dieter request sockets. The process keeps these sockets open
Dieter until it finishes.
Dieter There are two HTTP modes:
Dieter1. single request mode a new TCP connection is
Dieter created for each HTTP request, the request is completed,
Dieter when the TCP connection is closed.
Dieter2. multi request mode several requests share a
Dieter single TCP connection. The "Content-Length" HTTP header
Dieter allows client and server to determine the request
Dieter boundaries.
Dieter That means: when a client uses the multi request
Dieter mode, everything is fine. Clients, however, that use the
Dieter single request mode wait until their connection is closed
Dieter and can observe serious delays.
Dieter These delays may be very difficult to explain, as the
Dieter background process not only delays its own request but may
Dieter delay arbitrary other requests that happen to be served at
Dieter the same time.
Dieter How to fix the problem:
Dieter There should be a (file) control "CloseOnExec", that
Dieter tells Unix to automatically close the file like object
Dieter upon an exec. ZServer should probably use it on each of
Dieter its sockets. I can not see a serious application that
Dieter should have direct access to ZServer's HTTP request
Dieter socket.
Dieter I will soon file a bug report into the Collector.
Dieter Dieter
Dieter ___ Zope
Dieter maillist - [EMAIL PROTECTED]
Dieter http://lists.zope.org/mailman/listinfo/zope ** No cross
Dieter posts or HTML encoding! ** (Related lists -
Dieter http://lists.zope.org/mailman/listinfo/zope-announce
Dieter http://lists.zope.org/mailman/listinfo/zope-dev )
___
Zope maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
**
Re: [Zope] [BUG] Background processes interfere with Zope's HTTP responses
He reported, that this response page is build only half, then stops, to be completed only half a minute later, Depending on how long the page is I never see the bottom. When I use the products page I have to refresh 2-3 times to see the bottom part. This is thru a proxy only. I try to reproduce the behaviour in our test environment and fail. I do not have to wait, until the newsletter generation finished. Then, I use the official service URL and see, I observe the same behaviour. The difference: in the test environment, the browser connects directly to ZServer; with the official URL, it connects via a proxy. Puzzling! I'm having this same sort of problem w/Zope 2.3a1. I just started trying to use Zope through a proxy. If I work directly on the machine I have no problems. How to fix the problem: There should be a (file) control "CloseOnExec", that tells Unix to automatically close the file like object upon an exec. ZServer should probably use it on each of its sockets. I can not see a serious application that should have direct access to ZServer's HTTP request socket. I will soon file a bug report into the Collector. Dieter I'm on the win32 platform and thought that the problem was with my proxy software. Now I'm not so sure. -Jimbo ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] [BUG] Background processes interfere with Zope's HTTP responses
How to fix the problem: There should be a (file) control "CloseOnExec", that tells Unix to automatically close the file like object upon an exec. ZServer should probably use it on each of its sockets. I can not see a serious application that should have direct access to ZServer's HTTP request socket. I will soon file a bug report into the Collector. Dieter I'm on the win32 platform and thought that the problem was with my proxy software. Now I'm not so sure. Note that what Dieter is talking about would only affect you if you were spawning an external process that inherited the filehandles of the zope process. Brian Lloyd[EMAIL PROTECTED] Software Engineer 540.371.6909 Digital Creations http://www.digicool.com ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] [BUG] Background processes interfere with Zope's HTTP responses
Today was again a hard Zope day.
Did you ever hear, that usually (i.e. if you do not do special
things) Zope renders a complete page before it starts
sending the result back to the client.
This implies that you should never see only half of a page.
I heard it and I saw the code that does it.
Therefore, I was convinced that this were really the case.
Then our client phoned:
He visits a page that allows him to generate a
newsletter. When he presses the "generate newsletter button",
a result page is build that tells him that the newsletter
is being generated and send is a few minutes.
He reported, that this response page is build only
half, then stops, to be completed only half a minute later,
at the same time when the newsletter arrives.
The effect is reproducible. If newsletter generation
takes longer, he must wait longer for the page to complete.
He is convinced that we generate the newsletter synchronously
and let him wait until the generation is complete.
And he is angry.
I know, that the newsletter is generated in a background
process, started in an external method with:
os.system("gen_newsletter ")
I try to reproduce the behaviour in our test environment
and fail. I do not have to wait, until the newsletter
generation finished. Then, I use the official service
URL and see, I observe the same behaviour.
The difference: in the test environment, the browser
connects directly to ZServer; with the official
URL, it connects via a proxy.
Puzzling!
What happened?
The background process inherits Zope's open file descriptors.
Among them are all currently open HTTP request sockets.
The process keeps these sockets open until it finishes.
There are two HTTP modes:
1. single request mode
a new TCP connection is created for each HTTP request,
the request is completed, when the TCP connection
is closed.
2. multi request mode
several requests share a single TCP connection.
The "Content-Length" HTTP header allows
client and server to determine the request boundaries.
That means: when a client uses the multi request mode,
everything is fine. Clients, however, that use
the single request mode wait until their connection
is closed and can observe serious delays.
These delays may be very difficult to explain, as
the background process not only delays its own
request but may delay arbitrary other requests
that happen to be served at the same time.
How to fix the problem:
There should be a (file) control "CloseOnExec", that tells
Unix to automatically close the file like object upon
an exec.
ZServer should probably use it on each of its sockets.
I can not see a serious application that should have
direct access to ZServer's HTTP request socket.
I will soon file a bug report into the Collector.
Dieter
___
Zope maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope-dev )
