> On 30 Aug 2016, at 10:17 AM, kusimari share <[email protected]> wrote:
>
> Hi,
>
> I have two instances of a apache, mod_wsgi and flask based http rest service.
> The only difference between the two services are the files being loaded
> (python pickle files on flask start) and AWS regions, one is in US while the
> other is in EU. The flask service itself is a wrapper over scikit learn
> predictions which are the reason for the python pickle files mentioned
> earlier.
>
> Confusingly the instance in US works fine. No issues. However the EU instance
> keeps getting into a deadlock situation because of which mod_wsgi restarts.
> Apache is still up. I have had no luck using the different mod_wsgi debug
> mechanisms, apparently there is no deadlock in the flask, scikit learn side
> of the equation.
>
> Today, found that on deadlock mod_wsgi emitted:
> 21720 [Mon Aug 29 21:31:26.979825 2016] [wsgi:crit] [pid 4317:tid
> 140495687612160] (35)Resource deadlock avoided: │EOE
>
> mod_wsgi (pid=4317): Couldn't acquire accept mutex
> '/path-to-apache/var/state.14843.0.1.sock'. Shutting down daemon process
>
>
> Apache conf contains
> Mutex file:/path-to-pache/var/state mpm-accept
>
> My suspicion is that one of the instance is using fcntl while the other uses
> flock, and probably the one using fcntl has deadlocks based on apache
> documentation. But I have no way to verify it.
>
> Is there a way to find which of fcntl or flock is being used by Apache?
>
> httpd -V shows
> -D APR_HAS_SENDFILE
> -D APR_HAS_MMAP
> -D APR_USE_SYSVSEM_SERIALIZE
> -D APR_USE_PTHREAD_SERIALIZE
> -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
> -D APR_HAS_OTHER_CHILD
> -D AP_HAVE_RELIABLE_PIPED_LOGS
> -D DYNAMIC_MODULE_LIMIT=256
If you start up the Apache web server with LogLevel set to ‘debug’, you should
see a line like:
[mpm_prefork:debug] [pid 82122] prefork.c(1027): AH00165: Accept mutex:
none (default: flock)
I think this is what mod_wsgi will also use. There was an issue at some point
where Apache stopped allowing me to see the choice it made and so I had to
start calculating it myself.
That in itself shouldn’t cause a problem though as Apache’s own cross process
mutex locks are separate to those mod_wsgi creates, so there cannot be a clash
if types end up being different.
I would suggest the issue may be an ownership/permissions issue on the
directory rather than a mismatch of lock types. The locks on this are all
managed in mod_wsgi so can’t see how the type of locking used could differ at
different times.
So what are the ownership/permissions on the directory:
/path-to-apache/var/
Can you try setting the directive:
WSGISocketPrefix /tmp
as a test to see if that helps. The sockets and lock files should then be
placed in /tmp instead, which if this is a single user machine should be fine.
You can also override what the mutex type used by mod_wsgi for its own locks
is by using:
WSGIAcceptMutex flock
Graham
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
