Don't know why but gc.garbage is empty list. On Sunday, February 15, 2015 at 1:04:15 PM UTC+2, Graham Dumpleton wrote: > > If the objects cannot be reclaimed due to Python objects cycles where > objects have __del__() methods, they simply are not able to be reclaimed. > > You can try: > > import gc > gc.collect() > > but will likely have little effect. > > I would suggest you also read: > > http://christian.hofstaedtler.name/blog/2013/01/gc-garbage.html > > Graham > > On 15/02/2015, at 10:00 PM, Paul Royik <[email protected] <javascript:>> > wrote: > > Can I force cleaning memory after algorithm has finished? > > On Sunday, February 15, 2015 at 12:53:30 PM UTC+2, Graham Dumpleton wrote: >> >> >> On 15/02/2015, at 9:41 PM, Paul Royik <[email protected]> wrote: >> >> I reduced possibility by restricting very long and complex expressions. >> >> Code, that I use is simple. >> >> class TimeLimit(object): >> def __init__(self, timeout=60): >> self.timeout = timeout >> self.end = None >> >> def check_timeout(self): >> if self.end and time.time() > self.end: >> raise TimeoutException >> else: >> self.start() >> >> def start(self): >> if not self.end: >> self.end = time.time() + self.timeout >> >> class MainClass(object): >> def __init__(self): >> self.timer = TimeLimit() >> algo(timer) >> >> >> def algo(timer): >> do_something() >> external() >> timer.check_timeout() >> algo(timer) >> >> >> Which ultimately is no different to what was being done with the >> decorator. >> >> If external() can run a lot longer than the timeout, either way is not >> going to be much use. It is only going to be of much use if external() is >> always a short call and on subsequent calls you are simply calling it with >> different arguments. >> >> I debugged code thoroughly and it works as expected. And as shown memory >> is not drastically increased during execution, but later. Is that possible >> that reasong in apache conf? Or I need to look for problem in code? I have >> no clue where to search as everything works. Is it possible to see memory >> usage somehow on localhost (I'm under Windows)? >> >> >> It is unlikely to be the Apache configuration. It is more likely that >> after finishing the execution of the algorithm, the algorithm isn't >> clearing out any working data structures it creates when it is doing >> calculations and they are persisting in memory. Subsequent calls simply add >> more memory each time and so it grows in that way. >> >> There are various reasons that Python objects cannot be reclaimed and >> will keep using memory. >> >> There aren't really great tools for tracking that sort of stuff down. >> >> Graham >> >> On Sunday, February 15, 2015 at 12:31:36 PM UTC+2, Graham Dumpleton wrote: >>> >>> >>> On 15/02/2015, at 9:26 PM, Paul Royik <[email protected]> wrote: >>> >>> Yes. External library could run long (although I reduced this >>> possibility). >>> >>> >>> How have you reduced possibility? >>> >>> So, what should I do with locking threads? Is it resolvable? >>> >>> >>> No idea. It isn't my code. I can only highlight what looks suspicious in >>> the log output. >>> >>> I'm just now playing with code on stackoverflow. People suggested to >>> create class instead of decorator and pass its instance in all functions. I >>> did as they proposed and it worked. >>> >>> >>> Since you don't provide the code for that, I can't comment. >>> >>> However, with the flow of time I see increase in memory. >>> So, now nobody access site. >>> >>> I ran command ps -u simamura -o pid,rss,command | awk '{print >>> $0}{sum+=$2} END {print "Total", sum/1024, "MB"}' and see 200 MB. >>> Then I invoke integral calculator, while it works above command shows >>> 200 MB. >>> Calculator finished, I check memory - still 200 MB. In 10-15 minutes >>> (remember nobody access the site) I see 240 MB. >>> >>> In result, memory increased from 130 MB (memory immediately after server >>> restart) to 435 MB in 10 hours. >>> >>> How can I resolve this issue? I feel that in another 12 hours webfaction >>> will kill the process, because I memomy will be over limit. >>> >>> >>> Have you tried adding print() debug statements to your code to track >>> through what is being called and to determine whether your code is doing >>> what is expected? >>> >>> Graham >>> >>> On Sunday, February 15, 2015 at 12:12:39 PM UTC+2, Graham Dumpleton >>> wrote: >>>> >>>> >>>> On 15/02/2015, at 9:03 PM, Paul Royik <[email protected]> wrote: >>>> >>>> How do you know that it runs too long? >>>> >>>> >>>> In explained the log output previously and what was happening. >>>> >>>> If you go back far enough in the logs even before what you provided you >>>> will see some messages like: >>>> >>>> [Fri Feb 06 12:39:06.254056 2015] [wsgi:info] [pid 9731:tid >>>> 140178879313664] mod_wsgi (pid=9731): Daemon process request time limit >>>> exceeded, stopping process 'localhost:20241'. >>>> [Fri Feb 06 12:39:06.254108 2015] [wsgi:info] [pid 9731:tid >>>> 140179136616192] mod_wsgi (pid=9731): Shutdown requested 'localhost:20241'. >>>> [Fri Feb 06 12:39:06.286339 2015] [wsgi:info] [pid 9731:tid >>>> 140179136616192] mod_wsgi (pid=9731): Dumping stack trace for active >>>> Python >>>> threads. >>>> [Fri Feb 06 12:39:06.286359 2015] [wsgi:info] [pid 9731:tid >>>> 140179136616192] mod_wsgi (pid=9731): Thread 140178316318464 executing >>>> file >>>> "/usr/local/lib/python2.7/threading.py", line 147, in acquire >>>> [Fri Feb 06 12:39:06.286363 2015] [wsgi:info] [pid 9731:tid >>>> 140179136616192] mod_wsgi (pid=9731): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/compatibility.py", line 851, in >>>> wrapper, >>>> [Fri Feb 06 12:39:06.286366 2015] [wsgi:info] [pid 9731:tid >>>> 140179136616192] mod_wsgi (pid=9731): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/cache.py", line 89, in wrapper, >>>> [Fri Feb 06 12:39:06.286369 2015] [wsgi:info] [pid 9731:tid >>>> 140179136616192] mod_wsgi (pid=9731): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/operations.py", line 127, in >>>> _matches_commutative, >>>> [Fri Feb 06 12:39:06.286372 2015] [wsgi:info] [pid 9731:tid >>>> 140179136616192] mod_wsgi (pid=9731): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/mul.py", line 808, in matches, >>>> [Fri Feb 06 12:39:06.286375 2015] [wsgi:info] [pid 9731:tid >>>> 140179136616192] mod_wsgi (pid=9731): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/add.py", line 358, in >>>> _matches_simple, >>>> [Fri Feb 06 12:39:06.286378 2015] [wsgi:info] [pid 9731:tid >>>> 140179136616192] mod_wsgi (pid=9731): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/operations.py", line 127, in >>>> _matches_commutative, >>>> >>>> So it says that request time limit was exceeded. >>>> >>>> This is followed by stack dumps for what all the request threads were >>>> doing. >>>> >>>> In the part you did show, they both are stuck in: >>>> >>>> [Sat Feb 14 19:58:00.261118 2015] [wsgi:info] [pid 12528:tid >>>> 139862016030464] mod_wsgi (pid=12528): Thread 139861715523328 executing >>>> file "/usr/local/lib/python2.7/threading.py", line 147, in acquire >>>> [Sat Feb 14 19:58:00.261120 2015] [wsgi:info] [pid 12528:tid >>>> 139862016030464] mod_wsgi (pid=12528): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/compatibility.py", line 851, in >>>> wrapper, >>>> [Sat Feb 14 19:58:00.261122 2015] [wsgi:info] [pid 12528:tid >>>> 139862016030464] mod_wsgi (pid=12528): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/cache.py", line 89, in wrapper, >>>> [Sat Feb 14 19:58:00.261125 2015] [wsgi:info] [pid 12528:tid >>>> 139862016030464] mod_wsgi (pid=12528): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/mul.py", line 373, in _gather, >>>> [Sat Feb 14 19:58:00.261127 2015] [wsgi:info] [pid 12528:tid >>>> 139862016030464] mod_wsgi (pid=12528): called from file >>>> "/home/simamura/lib/python2.7/sympy/core/mul.py", line 98, in flatten, >>>> >>>> That is, they are both waiting on a thread lock. >>>> >>>> This suggests to me that the code isn't possibly dealing with locking >>>> properly, or a lock is held for a very long time while in some certain >>>> section and blocking other threads from running and so why they are >>>> delayed >>>> and take so long. >>>> >>>> No more than 300 seconds as I wrote in conf file (socket-timeout and >>>> request-timeout). >>>> >>>> I can not control it thoroughly. >>>> def algo(): >>>> do_something() >>>> invoke_external_library() # this could run slighlty longer, so >>>> timeout is not precise >>>> check_timeout() >>>> algo() >>>> >>>> >>>> So the thing that takes the unpredictable amount of time and you need >>>> to interrupt is not even your code? >>>> >>>> Graham >>>> >>>> On Sunday, February 15, 2015 at 11:36:05 AM UTC+2, Graham Dumpleton >>>> wrote: >>>>> >>>>> So we are back the fact that your requests are still running too long >>>>> and however you have set up the timeout check isn't working. >>>>> >>>>> Have you added print() debug statements in your code to validate that >>>>> the die on timeout check is even being run regularly? >>>>> >>>>> To be able to distinguish each request thread, you can include the >>>>> output of threading.currentThread() on the print statements. >>>>> >>>>> Graham >>>>> >>>>> On 15/02/2015, at 8:19 PM, Paul Royik <[email protected]> wrote: >>>>> >>>>> [Sat Feb 14 19:58:00.261309 2015] [wsgi:info] [pid 12528:tid >>>>> 139862016030464] mod_wsgi (pid=12528): called from file >>>>> "/home/simamura/lib/python2.7/mod_wsgi/server/__init__.py", line 1137, in >>>>> handle_request. >>>>> [Sat Feb 14 19:58:05.245850 2015] [wsgi:info] [pid 12528:tid >>>>> 139861541500672] mod_wsgi (pid=12528): Aborting process 'localhost:20241'. >>>>> [Sat Feb 14 19:58:05.245889 2015] [wsgi:info] [pid 12528:tid >>>>> 139861541500672] mod_wsgi (pid=12528): Exiting process 'localhost:20241'. >>>>> [Sun Feb 15 01:58:05.352861 2015] [wsgi:error] [pid 11272:tid >>>>> 139862014424832] [client 127.0.0.1:48193] Truncated or oversized >>>>> response headers received from daemon process 'localhost:20241': >>>>> /home/simamura/webapps/django_math/express/htdocs/calculators, referer: >>>>> http://www.emathhelp.net/calculators/calculus-2/integral-calculator/ >>>>> [Sun Feb 15 01:58:05.352890 2015] [wsgi:error] [pid 11272:tid >>>>> 139862015223552] [client 127.0.0.1:48563] Truncated or oversized >>>>> response headers received from daemon process 'localhost:20241': >>>>> /home/simamura/webapps/django_math/express/htdocs/calculators, referer: >>>>> http://www.emathhelp.net/calculators/calculus-2/integral-calculator/ >>>>> [Sun Feb 15 01:58:05.352958 2015] [wsgi:error] [pid 11272:tid >>>>> 139861948524288] [client 127.0.0.1:48240] Truncated or oversized >>>>> response headers received from daemon process 'localhost:20241': >>>>> /home/simamura/webapps/django_math/express/htdocs/calculators, referer: >>>>> http://www.emathhelp.net/calculators/calculus-2/integral-calculator/ >>>>> [Sun Feb 15 01:58:05.352992 2015] [wsgi:error] [pid 11272:tid >>>>> 139862015489792] [client 127.0.0.1:48440] Truncated or oversized >>>>> response headers received from daemon process 'localhost:20241': >>>>> /home/simamura/webapps/django_math/express/htdocs/calculators, referer: >>>>> http://www.emathhelp.net/calculators/calculus-2/integral-calculator/ >>>>> [Sun Feb 15 01:58:05.353008 2015] [wsgi:error] [pid 11666:tid >>>>> 139862014957312] [client 127.0.0.1:48329] Truncated or oversized >>>>> response headers received from daemon process 'localhost:20241': >>>>> /home/simamura/webapps/django_math/express/htdocs/calculators, referer: >>>>> http://www.emathhelp.net/calculators/calculus-2/integral-calculator/ >>>>> [Sun Feb 15 01:58:05.353015 2015] [wsgi:error] [pid 11272:tid >>>>> 139862014691072] [client 127.0.0.1:48620] Truncated or oversized >>>>> response headers received from daemon process 'localhost:20241': >>>>> /home/simamura/webapps/django_math/express/htdocs/calculators, referer: >>>>> http://www.emathhelp.net/calculators/calculus-2/integral-calculator/ >>>>> [Sun Feb 15 01:58:05.353081 2015] [wsgi:error] [pid 11272:tid >>>>> 139861949323008] [client 127.0.0.1:48778] Truncated or oversized >>>>> response headers received from daemon process 'localhost:20241': >>>>> /home/simamura/webapps/django_math/express/htdocs/calculators, referer: >>>>> http://www.emathhelp.net/calculators/calculus-2/integral-calculator/ >>>>> [Sun Feb 15 01:58:05.353081 2015] [wsgi:error] [pid 11666:tid >>>>> 139861948790528] [client 127.0.0.1:48735] Truncated or oversized >>>>> response headers received from daemon process 'localhost:20241': >>>>> /home/simamura/webapps/django_math/express/htdocs/calculators, referer: >>>>> http://www.emathhelp.net/calculators/calculus-2/integral-calculator/ >>>>> [Sun Feb 15 01:58:05.353096 2015] [wsgi:error] [pid 11666:tid >>>>> 139862015223552] [client 127.0.0.1:48792] Truncated or oversized >>>>> response headers received from daemon process 'localhost:20241': >>>>> /home/simamura/webapps/django_math/express/htdocs/calculators, referer: >>>>> http://www.emathhelp.net/calculators/calculus-2/integral-calculator/ >>>>> [Sun Feb 15 01:58:05.972993 2015] [wsgi:info] [pid 24792:tid >>>>> 139862016030464] mod_wsgi (pid=12528): Process 'localhost:20241' has >>>>> died, >>>>> deregister and restart it. >>>>> [Sun Feb 15 01:58:06.081051 2015] [wsgi:info] [pid 24792:tid >>>>> 139862016030464] mod_wsgi (pid=12528): Process 'localhost:20241' has been >>>>> deregistered and will no longer be monitored. >>>>> [Sun Feb 15 01:58:06.081333 2015] [wsgi:info] [pid 23028:tid >>>>> 139862016030464] mod_wsgi (pid=23028): Starting process 'localhost:20241' >>>>> with threads=10. >>>>> [Sun Feb 15 01:58:06.163910 2015] [wsgi:info] [pid 23028:tid >>>>> 139862016030464] mod_wsgi (pid=23028): Python home /usr/local. >>>>> [Sun Feb 15 01:58:06.163937 2015] [wsgi:info] [pid 23028:tid >>>>> 139862016030464] mod_wsgi (pid=23028): Initializing Python. >>>>> [Sun Feb 15 01:58:06.534740 2015] [wsgi:info] [pid 23028:tid >>>>> 139862016030464] mod_wsgi (pid=23028): Attach interpreter ''. >>>>> [Sun Feb 15 01:58:06.549849 2015] [wsgi:info] [pid 23028:tid >>>>> 139862016030464] mod_wsgi (pid=23028): Imported 'mod_wsgi'. >>>>> [Sun Feb 15 01:58:06.549942 2015] [wsgi:info] [pid 23028:tid >>>>> 139862016030464] mod_wsgi (pid=23028, process='localhost:20241', >>>>> application=''): Loading WSGI script >>>>> '/home/simamura/webapps/django_math/express/handler.wsgi'. >>>>> [Sun Feb 15 02:06:36.476939 2015] [core:info] [pid 11272:tid >>>>> 139861948524288] [client 127.0.0.1:56156] AH00128: File does not >>>>> exist: /home/simamura/webapps/math_root/browserconfig.xml >>>>> >>>>> >>>>> And that's all. >>>>> >>>>> On Sunday, February 15, 2015 at 10:11:25 AM UTC+2, Graham Dumpleton >>>>> wrote: >>>>>> >>>>>> Provide me with the logging from a period before the restart until a >>>>>> period after. >>>>>> >>>>>> The log can be very informative if you know what to look for. >>>>>> >>>>>> Graham >>>>>> >>>>>> On 15/02/2015, at 6:43 PM, Paul Royik <[email protected]> wrote: >>>>>> >>>>>> I actually have custom error 500 page. This means, that when Django >>>>>> throws error, custom page should be shown. >>>>>> But it is not the case. Standard white-background apache page is >>>>>> shown. >>>>>> So, I think that problem is not in Django exception. Otherwise Django >>>>>> would show custom page, which is not the case. >>>>>> In logs I saw only that server was restarted. >>>>>> Can I somehow tune logging, so that real error is written? >>>>>> >>>>>> On Sunday, February 15, 2015 at 2:44:50 AM UTC+2, Graham Dumpleton >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> On 15/02/2015, at 10:53 AM, Paul Royik <[email protected]> wrote: >>>>>>> >>>>>>> > I already tried this approach. >>>>>>> > >>>>>>> > It seems good. >>>>>>> > But when I tested it on server, by simultaneously executing page >>>>>>> on two different computers, it gave me 500 error. >>>>>>> > >>>>>>> > How this can be explained? Again something with apache? Logs >>>>>>> didn't show anything. But I noticed again serious memory usage. And >>>>>>> this >>>>>>> happens only when I use code for limiting time. >>>>>>> >>>>>>> >>>>>>> Some tips for you about ensuring you get the best help. >>>>>>> >>>>>>> Stop discarding the message content for the message you are replying >>>>>>> to. It can be frustrating to have to keep going back to old messages to >>>>>>> see >>>>>>> what was said originally and what you may be following up to, >>>>>>> especially >>>>>>> when on a phone. When asking questions on mailing list, let the person >>>>>>> you >>>>>>> are asking questions of decide what can be discarded from the message >>>>>>> chain >>>>>>> as they will know better what should be kept in the message to provide >>>>>>> easy >>>>>>> access to important information for context. >>>>>>> >>>>>>> When you reply to say you are have a new error, provide the actual >>>>>>> code you were using exactly at the time you had the error. Saves the >>>>>>> person >>>>>>> who is helping you having to go back and ask for it if it is unclear >>>>>>> what >>>>>>> code you were using. >>>>>>> >>>>>>> Your messages here and on StackOverflow show that you play around >>>>>>> with the example code I am giving you and I have reduced confidence you >>>>>>> were running with the code I suggested at the time of the problem you >>>>>>> are >>>>>>> now relating to. >>>>>>> >>>>>>> If you are getting 500 errors and nothing is being logged, it is >>>>>>> because Django is capturing the exception and converting it to a >>>>>>> generic >>>>>>> 500 error response page. Configure Django to send you emails with the >>>>>>> details of the exceptions. If you are on a local system, then set >>>>>>> DEBUG=True in the Django settings so the details of the error are shown >>>>>>> in >>>>>>> response that goes back to the browser. >>>>>>> >>>>>>> As to the code, I did test it this time, including under mod_wsgi >>>>>>> with 20 request threads and hitting it with concurrent and successive >>>>>>> requests with a benchmarking tool. I saw no issues and it appeared to >>>>>>> behave as I would expect. >>>>>>> >>>>>>> So in a test2.py file I had: >>>>>>> >>>>>>> import time >>>>>>> import functools >>>>>>> import threading >>>>>>> >>>>>>> def time_limit(seconds): >>>>>>> def decorator(func): >>>>>>> func.info = threading.local() >>>>>>> def check_timeout(): >>>>>>> if time.time() > func.info.end_time: >>>>>>> raise RuntimeError('timeout') >>>>>>> >>>>>>> func.check_timeout = check_timeout >>>>>>> >>>>>>> @functools.wraps(func) >>>>>>> def wrapper(*args, **kwargs): >>>>>>> print 'hasattr counter', hasattr(func.info, 'counter') >>>>>>> if not hasattr(func.info, 'counter'): >>>>>>> print 'init counter to 0' >>>>>>> func.info.counter = 0 >>>>>>> if func.info.counter == 0: >>>>>>> func.info.end_time = time.time() + seconds >>>>>>> print 'counter', func.info.counter >>>>>>> func.info.counter += 1 >>>>>>> try: >>>>>>> return func(*args, **kwargs) >>>>>>> finally: >>>>>>> func.info.counter -= 1 >>>>>>> >>>>>>> return wrapper >>>>>>> >>>>>>> return decorator >>>>>>> >>>>>>> @time_limit(5) >>>>>>> def algorithm(limit, nest=0): >>>>>>> algorithm.check_timeout() >>>>>>> print 'sleep' >>>>>>> time.sleep(1.0) >>>>>>> if nest == limit: >>>>>>> print 'return' >>>>>>> return >>>>>>> algorithm(limit, nest+1) >>>>>>> >>>>>>> In serial2.py I had a straight serialised test: >>>>>>> >>>>>>> from test2 import algorithm >>>>>>> >>>>>>> try: >>>>>>> algorithm(3) >>>>>>> except RuntimeError: >>>>>>> print 'timeout' >>>>>>> >>>>>>> try: >>>>>>> algorithm(10) >>>>>>> except RuntimeError: >>>>>>> print 'timeout' >>>>>>> >>>>>>> try: >>>>>>> algorithm(10) >>>>>>> except RuntimeError: >>>>>>> print 'timeout' >>>>>>> >>>>>>> And then in hello2.wsgi I had it used by a WSGI application. >>>>>>> >>>>>>> from test2 import algorithm >>>>>>> >>>>>>> def application(environ, start_response): >>>>>>> status = '200 OK' >>>>>>> output = b'Hello World!' >>>>>>> >>>>>>> algorithm(1) >>>>>>> >>>>>>> response_headers = [('Content-type', 'text/plain'), >>>>>>> ('Content-Length', str(len(output)))] >>>>>>> start_response(status, response_headers) >>>>>>> >>>>>>> return [output] >>>>>>> >>>>>>> In the latter case I ran mod_wsgi-express against it as: >>>>>>> >>>>>>> mod_wsgi-express start-server hello2.wsgi --port 8002 --threads=20 >>>>>>> >>>>>>> and then hit it with ab as: >>>>>>> >>>>>>> ab -n 100 -c 15 http://localhost:8002/ >>>>>>> >>>>>>> In this case it wasn't designed to timeout anything, but that should >>>>>>> not be a concern as the counter initialisation is still being tested. >>>>>>> >>>>>>> Do note that if you did cut and paste that last code, I did change >>>>>>> the exception type. >>>>>>> >>>>>>> Anyway, the best thing to do is setup Django so that it provides the >>>>>>> details of the exception it captured but then effectively discarded >>>>>>> because >>>>>>> it converted it to a 500 page. >>>>>>> >>>>>> >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "modwsgi" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at http://groups.google.com/group/modwsgi. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/modwsgi. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/modwsgi. >> For more options, visit https://groups.google.com/d/optout. >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] <javascript:> > . > Visit this group at http://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/d/optout. > > >
-- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
