Re: [Web-SIG] Closing long-running WSGI requests (possible?)
Chimezie Ogbuji ha scritto: > Hello. I have a problem with a WSGI-based SPARQL server that I have been > unable to resolve for some time. I was told this is the best place to ask > :). I'm building a SPARQL [1] server that is deployed as WSGI/Paste > server. SPARQL queries are handled by the server and evaluated against a > MySQL database using mysql-python/MySQLdb to manage the connection. > > My goal is to be able to allow clients to close the connection in order to > kill queries that have been dispatched (in order to 'abort' them). > Unfortunately, when the client kills the connection, the application is not > signaled in any way. So, the result is that (for long-running queries), the > MySQL query continues to run even after the connection is closed (by > clicking cancel in the browser for instance). > > [...] What you want to do is not possible. A more viable solution is to use JavaScript. Add a custom "abort button" on the web page so that a function is associate to the "click" event. Also, you should associate a function to the "unload" event (where you can check if there are active queries). In the JavaScript function you can issue an XMLHTTPRequest, using an unique identifier. Note that if you use PostgreSQL, you can use: http://www.postgresql.org/docs/8.3/interactive/protocol-flow.html#AEN73870 When you create a connection to PostgreSQL, the server will send you the backend process id an unique key. You can use this data to send a cancellation request. All you need to do is to pass the process id and the unique key to the client (with some encryption so that the client can use the data only once). Unfortunately, libpq does not offer a flexible interface to this feature. The PGCancel structure is opaque, so you need some hacking. Manlio Perillo ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Closing long-running WSGI requests (possible?)
No, cannot really be done. This has been discussed a couple of times on the mod_wsgi list. One such discussion is at: http://groups.google.com/group/modwsgi/browse_frm/thread/8ebd9aca9d317ac9 In general the same issues apply to all WSGI implementations. Graham 2009/4/14 Chimezie Ogbuji : > Hello. I have a problem with a WSGI-based SPARQL server that I have been > unable to resolve for some time. I was told this is the best place to ask > :). I'm building a SPARQL [1] server that is deployed as WSGI/Paste > server. SPARQL queries are handled by the server and evaluated against a > MySQL database using mysql-python/MySQLdb to manage the connection. > > My goal is to be able to allow clients to close the connection in order to > kill queries that have been dispatched (in order to 'abort' them). > Unfortunately, when the client kills the connection, the application is not > signaled in any way. So, the result is that (for long-running queries), the > MySQL query continues to run even after the connection is closed (by > clicking cancel in the browser for instance). > > I would expect that when the connection is closed at the client side, this > should trigger a chain reaction of garbage collection (deletion of the > application object, and all the objects attributed to it including the DB > connection, etc.) that bottoms out in the db connection closing and MySQLdb > killing the query as a side effect of calling __del__ on the cursor and > database connection. However, this is not what is happening and it appears > that the once the result is served back to the client, the server and the > client are completely 'disconnected' for that particular request. > > Am I going about his the wrong way? Does WSGI simply not have anything to > say about such a situation ? If the problem isn't > WSGI, is there another WSGI implementation that is known to behave as > expected (i.e., closing the connection dispatches the deletion of the > objects involved in the request handling)? > > I was told to look into keep-alive, but the specification doesn't seem to > suggest that this would help me as it has more to do with re-using > connections for subsequent requests rather than specifying that the server > maintains a connection between the request and the objects involved in > handling the request at the server. > > Any help would be greatly appreciated. > > Thanks > > [1] http://www.w3.org/TR/rdf-sparql-query/ > > > === > > P Please consider the environment before printing this e-mail > > Cleveland Clinic is ranked one of the top hospitals > in America by U.S. News & World Report (2008). > Visit us online at http://www.clevelandclinic.org for > a complete listing of our services, staff and > locations. > > > Confidentiality Note: This message is intended for use > only by the individual or entity to which it is addressed > and may contain information that is privileged, > confidential, and exempt from disclosure under applicable > law. If the reader of this message is not the intended > recipient or the employee or agent responsible for > delivering the message to the intended recipient, you are > hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If > you have received this communication in error, please > contact the sender immediately and destroy the material in > its entirety, whether electronic or hard copy. Thank you. > > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/graham.dumpleton%40gmail.com > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Closing long-running WSGI requests (possible?)
I agree with Ionel I personally wouldn't rely on "kill wsgi request". I'd run the update in a subprocess and kill the subprocess using a signal when the user requests (on unix, of course). I'd also check a log written by the subprocess to see whether it completed or not. If you "kill the wsgi request" you have the problem of not being quite sure whether the kill arrived in time, among other possible difficulties, some mentioned by Ionel. -- Aaron Watters http://aaron.oirt.rutgers.edu/myapp/docs/W0500.quickstart (apologies to Christian, who got this twice, I forgot to "reply all") --- On Mon, 4/13/09, Ionel Maries Cristian wrote: > From: Ionel Maries Cristian > Subject: Re: [Web-SIG] Closing long-running WSGI requests (possible?) > To: "Christian Wyglendowski" > Cc: "Chimezie Ogbuji" , web-sig@python.org > Date: Monday, April 13, 2009, 12:01 PM > That implies one would have extremely > reliable tcp connections, and clients > graciously shutdown the connection and the server is > notified of that. > > Most of the time that doesn't happen and the solution > is to continuously send > > keepalive packets (some small string or whatever) - I'm > assuming you run > a batch a set of queries and you can interleave yielding > some data while > you run that batch. > > For example if your client disconnects and the servers > tries to send some data > > it would fail - and trigger closing the app iterable. > > In contrast a server that just runs some backend processing > without moving > any data around doesn't have any way to know if the > connection is still valid. > > > Then again, even if the client properly shutdown the > connection the server > won't do anything about it if it doesn't try to do > anything with the socket due > to the synchronous nature (I'm assuming) of the whole > server/app. > > > -- ionel > > > > > On Mon, Apr 13, 2009 at 17:53, > Christian Wyglendowski > wrote: > > On Mon, Apr 13, 2009 at 10:40 AM, Chimezie > Ogbuji > wrote: > > > Hello. I have a problem with a WSGI-based SPARQL > server that I have been > > > unable to resolve for some time. I was told this is > the best place to ask > > > :). I'm building a SPARQL [1] server that is > deployed as WSGI/Paste > > > server. SPARQL queries are handled by the server and > evaluated against a > > > MySQL database using mysql-python/MySQLdb to manage > the connection. > > > > > > My goal is to be able to allow clients to close the > connection in order to > > > kill queries that have been dispatched (in order to > 'abort' them). > > > > This should be doable from what I understand. From > PEP 333: > > > > "If the iterable returned by the application has a > close() method, the > > server or gateway must call that method upon completion of > the current > > request, whether the request was completed normally, or > terminated > > early due to an error. (This is to support resource release > by the > > application. This protocol is intended to complement PEP > 325's > > generator support, and other common iterables with close() > methods." > > [1] > > > > So it sounds like you could add a close method on whatever > iterable > > that your application returns and have it do the required > resource > > release there. > > > > HTH, > > > > Christian > > http://www.dowski.com > > > > [1] http://www.python.org/dev/peps/pep-0333/#specification-details > > ___ > > Web-SIG mailing list > > Web-SIG@python.org > > Web SIG: http://www.python.org/sigs/web-sig > > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/ionel.mc%40gmail.com > > > > > -Inline Attachment Follows- > > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/arw1961%40yahoo.com > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Closing long-running WSGI requests (possible?)
That implies one would have extremely reliable tcp connections, and clients graciously shutdown the connection and the server is notified of that. Most of the time that doesn't happen and the solution is to continuously send keepalive packets (some small string or whatever) - I'm assuming you run a batch a set of queries and you can interleave yielding some data while you run that batch. For example if your client disconnects and the servers tries to send some data it would fail - and trigger closing the app iterable. In contrast a server that just runs some backend processing without moving any data around doesn't have any way to know if the connection is still valid. Then again, even if the client properly shutdown the connection the server won't do anything about it if it doesn't try to do anything with the socket due to the synchronous nature (I'm assuming) of the whole server/app. -- ionel On Mon, Apr 13, 2009 at 17:53, Christian Wyglendowski wrote: > On Mon, Apr 13, 2009 at 10:40 AM, Chimezie Ogbuji wrote: > > Hello. I have a problem with a WSGI-based SPARQL server that I have been > > unable to resolve for some time. I was told this is the best place to > ask > > :). I'm building a SPARQL [1] server that is deployed as WSGI/Paste > > server. SPARQL queries are handled by the server and evaluated against a > > MySQL database using mysql-python/MySQLdb to manage the connection. > > > > My goal is to be able to allow clients to close the connection in order > to > > kill queries that have been dispatched (in order to 'abort' them). > > This should be doable from what I understand. From PEP 333: > > "If the iterable returned by the application has a close() method, the > server or gateway must call that method upon completion of the current > request, whether the request was completed normally, or terminated > early due to an error. (This is to support resource release by the > application. This protocol is intended to complement PEP 325's > generator support, and other common iterables with close() methods." > [1] > > So it sounds like you could add a close method on whatever iterable > that your application returns and have it do the required resource > release there. > > HTH, > > Christian > http://www.dowski.com > > [1] http://www.python.org/dev/peps/pep-0333/#specification-details > ___ > Web-SIG mailing list > Web-SIG@python.org > Web SIG: http://www.python.org/sigs/web-sig > Unsubscribe: > http://mail.python.org/mailman/options/web-sig/ionel.mc%40gmail.com > ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Closing long-running WSGI requests (possible?)
On Mon, Apr 13, 2009 at 10:40 AM, Chimezie Ogbuji wrote: > Hello. I have a problem with a WSGI-based SPARQL server that I have been > unable to resolve for some time. I was told this is the best place to ask > :). I'm building a SPARQL [1] server that is deployed as WSGI/Paste > server. SPARQL queries are handled by the server and evaluated against a > MySQL database using mysql-python/MySQLdb to manage the connection. > > My goal is to be able to allow clients to close the connection in order to > kill queries that have been dispatched (in order to 'abort' them). This should be doable from what I understand. From PEP 333: "If the iterable returned by the application has a close() method, the server or gateway must call that method upon completion of the current request, whether the request was completed normally, or terminated early due to an error. (This is to support resource release by the application. This protocol is intended to complement PEP 325's generator support, and other common iterables with close() methods." [1] So it sounds like you could add a close method on whatever iterable that your application returns and have it do the required resource release there. HTH, Christian http://www.dowski.com [1] http://www.python.org/dev/peps/pep-0333/#specification-details ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
