Re: Deadlock bug in logging? Reproducible case

2020-04-24 Thread Brian Tiemann
Nah, I'm good — but thank you! On Friday, April 24, 2020 at 8:57:09 AM UTC-4, René Fleschenberg wrote: > > Hi, > > On 4/23/20 12:20 PM, Adam Johnson wrote: > > What version of Python René? > > I tested with 3.6.7. I can test with other versions, if that helps. > > Regards, > René > > > >

Re: Deadlock bug in logging? Reproducible case

2020-04-24 Thread René Fleschenberg
Hi, On 4/23/20 12:20 PM, Adam Johnson wrote: > What version of Python René? I tested with 3.6.7. I can test with other versions, if that helps. Regards, René -- René Fleschenberg -- You received this message because you are subscribed to the Google Groups "Django developers

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Brian Tiemann
True, I did notice it needed some more tweaking. I've got a long evening of rewriting years' worth of wsgi.py's and Apache configs ahead of me. I'll probably just jump straight to the envparse approach because (as you noted) my celery env var handling was making the WSGI approach moot anyway.

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Adam Johnson
Actually that might not work entirely, since your settings file will be imported before the environment variables are copied over. Instead you can lazily construct the django application on the first request: django_application = None def application(environ, start_response): global

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Brian Tiemann
Beautiful. That does the trick. Thank you! And I certainly can see there's plenty of other approaches such as envparse or django-environ that I could be using, that keeps the vars out of my Apache config. Quick fix and a slighly longer better fix. This'll change how I do all my new projects

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Adam Johnson
https://adamj.eu/tech/2019/05/27/the-simplest-wsgi-middleware/ django_application = get_wsgi_application() def application(environ, start_response): # pass the WSGI environment variables on through to os.environ for var in env_variables_to_pass: os.environ[var] = environ.get(var,

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Brian Tiemann
FYI, here's where that approach came from -- http://ericplumb.com/blog/passing-apache-environment-variables-to-django-via-mod_wsgi.html > > It's the top hit for "django apache environment variable" Which is literally an old friend of mine from grade school. Maybe that's a red flag ... -- You

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Brian Tiemann
Ah, interesting. Let me show you what my non-trimmed-down wsgi.py looked like -- import os, sys from django.core.wsgi import get_wsgi_application env_variables_to_pass = [ 'SECRET_KEY', ... other env vars ... ] def application(environ, start_response): # pass the WSGI environment

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Adam Johnson
I ran it on Python 3.7.6 and Python 3.8.2 and found the deadlock issue on both. I then adjusted the settings to *only* have the console logger, and added these lines at the end of the settings file: import faulthandler faulthandler.enable() print(os.getpid()) faulthandler allows getting a

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Brian Tiemann
It happens with Apache/mod_wsgi in prod. On Thursday, April 23, 2020 at 7:59:45 AM UTC-4, Tom Forbes wrote: > > Just to confirm - are you running into this issue while using the Django > development server in production, or does this occur with Gunicorn/uwsgi as > well? > > Tom > > On 23 Apr

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Tom Forbes
Just to confirm - are you running into this issue while using the Django development server in production, or does this occur with Gunicorn/uwsgi as well? Tom > On 23 Apr 2020, at 12:40, Brian Tiemann wrote: > > It also happens in 3.6.9, which is my prod environment. > > FWIW, by way of

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Brian Tiemann
It also happens in 3.6.9, which is my prod environment. FWIW, by way of background context, this is not a heavily used app, not enough so to warrant logging to a custom buffered consumer or remote agent. It's just a small office using it. But the main home view's UI has 5 AJAX calls that cause

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Brian Tiemann
Hi — Thanks for looking into this. Adam, I skipped over details like the Python version because I noted them in the readme on my gitlab demo. But I was using Python 3.7.2 for the record. I used "deadlock" because my digging into this had previously turned up https://bugs.python.org/issue6721 ,

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Adam Johnson
What version of Python René? On Thu, 23 Apr 2020 at 10:58, René Fleschenberg wrote: > Hi, > > On 4/23/20 9:40 AM, Adam Johnson wrote: > > Brian - You didn't describe the exact symptoms you're seeing. "Deadlock" > > has a particular technical meaning around multiple processes requesting > >

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread René Fleschenberg
Hi, On 4/23/20 9:40 AM, Adam Johnson wrote: > Brian - You didn't describe the exact symptoms you're seeing. "Deadlock" > has a particular technical meaning around multiple processes requesting > locks mutually off each other, and normally ends in one process being > terminated. But I presume

Re: Deadlock bug in logging? Reproducible case

2020-04-23 Thread Adam Johnson
Brian - You didn't describe the exact symptoms you're seeing. "Deadlock" has a particular technical meaning around multiple processes requesting locks mutually off each other, and normally ends in one process being terminated. But I presume you're experiencing more of a "standstill" and future

Re: Deadlock bug in logging? Reproducible case

2020-04-22 Thread Steven Mapes
This is more an issue at the file system level and the hardware not being able to keep up rather than anything Django is doing. It's the same fundamental principal for why you may turn off webserver logging to increase performance, writing to disk is expensive and so when dealing with high

Re: Deadlock bug in logging? Reproducible case

2020-04-22 Thread Dave Vernon
Sorry - that message was sent as a 'reply' in error - my colleague uses ajax with Django a lot and I was trying to forward this to him. Kind Regards, Dave On Wed, 22 Apr 2020 at 18:23, Brian Tiemann wrote: > Hi all, > > I was directed here after getting corroboration on #django and

Re: Deadlock bug in logging? Reproducible case

2020-04-22 Thread Dave Vernon
FYI Sent from my iPhone > On 22 Apr 2020, at 18:23, Brian Tiemann wrote: > >  > Hi all, > > I was directed here after getting corroboration on #django and elsewhere. > > I have what appears to be a bug in which a Django app can deadlock if you hit > it with a lot (>3) of AJAX requests

Deadlock bug in logging? Reproducible case

2020-04-22 Thread Brian Tiemann
Hi all, I was directed here after getting corroboration on #django and elsewhere. I have what appears to be a bug in which a Django app can deadlock if you hit it with a lot (>3) of AJAX requests within a short time (i.e. all triggered in parallel from a single HTML page). I have a