Dos you have access to gdb on the system? The next step would be to attach gdb to the mod_wsgi daemon process and then send it traffic until it crashes. Then at gdb prompt, get the stack traces for all threads.
See: https://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Debugging_Crashes_With_GDB <https://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Debugging_Crashes_With_GDB> Graham > On 5 Jan 2016, at 5:15 PM, Mike Panoff <[email protected]> wrote: > > Thank you Joonas and Graham for the input! You were correct, I had > WSGIProcessGroup set to GLOBAL, but not WSGIApplicationGroup. However, when > I set both to GLOBAL, it is still crashing. > > I am running in embedded mode. I am happy to run in daemon mode, but I am > unable to find a deployment recipe for running Web2Py in daemon mode. Does > anyone know of any? > > Also, here are my config files followed by the error.log (with > WSGIApplicationGroup ${GLOBAL}) in case anyone see problems there. > > Once again, I really appreciate all the help! > > -Mike > > > wsgi.conf file: > > <IfModule mod_wsgi.c> > > > #This config file is provided to give an overview of the directives, > #which are only allowed in the 'server config' context. > #For a detailed description of all avaiable directives please read > #http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives > > > #WSGISocketPrefix: Configure directory to use for daemon sockets. > # > #Apache's DEFAULT_REL_RUNTIMEDIR should be the proper place for WSGI's > #Socket. In case you want to mess with the permissions of the directory, > #you need to define WSGISocketPrefix to an alternative directory. > #See http://code.google.com/p/modwsgi/wiki/ConfigurationIssues for more > #information > > #WSGISocketPrefix /var/run/apache2/wsgi > > > #WSGIPythonOptimize: Enables basic Python optimisation features. > # > #Sets the level of Python compiler optimisations. The default is '0' > #which means no optimisations are applied. > #Setting the optimisation level to '1' or above will have the effect > #of enabling basic Python optimisations and changes the filename > #extension for compiled (bytecode) files from .pyc to .pyo. > #When the optimisation level is set to '2', doc strings will not be > #generated and retained. This will result in a smaller memory footprint, > #but may cause some Python packages which interrogate doc strings in some > #way to fail. > > #WSGIPythonOptimize 0 > > > #WSGIPythonPath: Additional directories to search for Python modules, > # overriding the PYTHONPATH environment variable. > # > #Used to specify additional directories to search for Python modules. > #If multiple directories are specified they should be separated by a ':'. > > #WSGIPythonPath directory|directory-1:directory-2:... > > > #WSGIPythonEggs: Directory to use for Python eggs cache. > # > #Used to specify the directory to be used as the Python eggs cache > #directory for all sub interpreters created within embedded mode. > #This directive achieves the same affect as having set the > #PYTHON_EGG_CACHE environment variable. > #Note that the directory specified must exist and be writable by the user > #that the Apache child processes run as. The directive only applies to > #mod_wsgi embedded mode. To set the Python eggs cache directory for > #mod_wsgi daemon processes, use the 'python-eggs' option to the > #WSGIDaemonProcess directive instead. > > #WSGIPythonEggs directory > > > > #WSGIRestrictEmbedded: Enable restrictions on use of embedded mode. > # > #The WSGIRestrictEmbedded directive determines whether mod_wsgi embedded > #mode is enabled or not. If set to 'On' and the restriction on embedded > #mode is therefore enabled, any attempt to make a request against a > #WSGI application which hasn't been properly configured so as to be > #delegated to a daemon mode process will fail with a HTTP internal server > #error response. > > #WSGIRestrictEmbedded On|Off > > > > #WSGIRestrictStdin: Enable restrictions on use of STDIN. > #WSGIRestrictStdout: Enable restrictions on use of STDOUT. > #WSGIRestrictSignal: Enable restrictions on use of signal(). > # > #Well behaved WSGI applications neither should try to read/write from/to > #STDIN/STDOUT, nor should they try to register signal handlers. If your > #application needs an exception from this rule, you can disable the > #restrictions here. > > #WSGIRestrictStdin On > #WSGIRestrictStdout On > #WSGIRestrictSignal On > > > > #WSGIAcceptMutex: Specify type of accept mutex used by daemon processes. > # > #The WSGIAcceptMutex directive sets the method that mod_wsgi will use to > #serialize multiple daemon processes in a process group accepting requests > #on a socket connection from the Apache child processes. If this directive > #is not defined then the same type of mutex mechanism as used by Apache > for > #the main Apache child processes when accepting connections from a client > #will be used. If set the method types are the same as for the Apache > #AcceptMutex directive. > > #WSGIAcceptMutex default > > > > #WSGIImportScript: Specify a script file to be loaded on process start. > # > #The WSGIImportScript directive can be used to specify a script file to be > #loaded when a process starts. Options must be provided to indicate the > #name of the process group and the application group into which the script > #will be loaded. > > #WSGIImportScript process-group=name application-group=name > > > #WSGILazyInitialization: Enable/disable lazy initialisation of Python. > # > #The WSGILazyInitialization directives sets whether or not the Python > #interpreter is preinitialised within the Apache parent process or whether > #lazy initialisation is performed, and the Python interpreter only > #initialised in the Apache server processes or mod_wsgi daemon processes > #after they have forked from the Apache parent process. > > #WSGILazyInitialization On|Off > > </IfModule> > > > > > apache default.conf: > > WSGIDaemonProcess web2py user=www-data group=www-data processes=1 threads=1 > WSGIProcessGroup %{GLOBAL} > WSGIApplicationGroup %{GLOBAL} > > <VirtualHost xx.111.177.xx:80> > > WSGIApplicationGroup %{GLOBAL} > WSGIProcessGroup %{GLOBAL} > RewriteEngine On > RewriteCond %{HTTPS} !=on > RewriteRule ^/?(.*) https://app.myappapp.com/$1 [R,L] > > CustomLog /var/log/apache2/access.log common > ErrorLog /var/log/apache2/error.log > </VirtualHost> > > <VirtualHost xx.111.177.xx:443> > SSLEngine on > SSLCertificateFile /etc/apache2/ssl/fd5fd2b23620e633.crt > SSLCertificateKeyFile /etc/apache2/ssl/myappapp.key > SSLCACertificateFile /etc/apache2/ssl/sf_bundle-g2-g1.crt > > > WSGIApplicationGroup %{GLOBAL} > WSGIProcessGroup %{GLOBAL} > WSGIScriptAlias / /home/www-data/web2py/wsgihandler.py > WSGIPassAuthorization On > > <Directory /home/www-data/web2py> > AllowOverride None > Require all denied > <Files wsgihandler.py> > Require all granted > </Files> > </Directory> > > AliasMatch ^/([^/]+)/static/(?:_[\d]+.[\d]+.[\d]+/)?(.*) \ > /home/www-data/web2py/applications/$1/static/$2 > > <Directory /home/www-data/web2py/applications/*/static/> > Options -Indexes > ExpiresActive On > ExpiresDefault "access plus 1 hour" > Require all granted > </Directory> > > CustomLog /var/log/apache2/ssl-access.log common > ErrorLog /var/log/apache2/error.log > ServerName app.myappapp.com > </VirtualHost> > > > [Tue Jan 05 00:49:22.665890 2016] [ssl:info] [pid 1296] AH01887: Init: > Initializing (virtual) servers for SSL > [Tue Jan 05 00:49:22.665948 2016] [ssl:info] [pid 1296] AH01914: Configuring > server app.myapp.com:443 for SSL protocol > [Tue Jan 05 00:49:22.666577 2016] [ssl:debug] [pid 1296] > ssl_engine_init.c(1600): AH02209: CA certificate: CN=Starfield Secure > Certificate Authority - > G2,OU=http://certs.starfieldtech.com/repository/,O=Starfield Technologies\\, > Inc.,L=Scottsdale,ST=Arizona,C=US > [Tue Jan 05 00:49:22.666596 2016] [ssl:debug] [pid 1296] > ssl_engine_init.c(1600): AH02209: CA certificate: CN=Starfield Root > Certificate Authority - G2,O=Starfield Technologies\\, > Inc.,L=Scottsdale,ST=Arizona,C=US > [Tue Jan 05 00:49:22.666607 2016] [ssl:debug] [pid 1296] > ssl_engine_init.c(1600): AH02209: CA certificate: OU=Starfield Class 2 > Certification Authority,O=Starfield Technologies\\, Inc.,C=US > [Tue Jan 05 00:49:22.666659 2016] [ssl:debug] [pid 1296] > ssl_engine_init.c(390): AH01893: Configuring TLS extension handling > [Tue Jan 05 00:49:22.666859 2016] [ssl:debug] [pid 1296] ssl_util_ssl.c(343): > AH02412: [app.myapp.com:443] Cert matches for name 'app.myapp.com' [subject: > CN=app.myapp.com,OU=Domain Control Validated / issuer: CN=Starfield Secure > Certificate Authority - > G2,OU=http://certs.starfieldtech.com/repository/,O=Starfield Technologies\\, > Inc.,L=Scottsdale,ST=Arizona,C=US / serial: FD5FD2A73620E644 / notbefore: Sep > 7 05:25:43 2015 GMT / notafter: Sep 7 05:25:43 2016 GMT] > [Tue Jan 05 00:49:22.666869 2016] [ssl:info] [pid 1296] AH02568: Certificate > and private key app.myapp.com:443:0 configured from > /etc/apache2/ssl/FD5FD2A73620E644.crt and /etc/apache2/ssl/myapp.key > [Tue Jan 05 00:49:22.666962 2016] [ssl:info] [pid 1296] AH01876: > mod_ssl/2.4.10 compiled against Server: Apache/2.4.10, Library: OpenSSL/1.0.1f > [Tue Jan 05 00:49:22.697500 2016] [socache_shmcb:debug] [pid 1298] > mod_socache_shmcb.c(396): AH00821: shmcb_init allocated 512000 bytes of > shared memory > [Tue Jan 05 00:49:22.697525 2016] [socache_shmcb:debug] [pid 1298] > mod_socache_shmcb.c(412): AH00822: for 511912 bytes (512000 including > header), recommending 32 subcaches, 88 indexes each > [Tue Jan 05 00:49:22.697528 2016] [socache_shmcb:debug] [pid 1298] > mod_socache_shmcb.c(445): AH00824: shmcb_init_memory choices follow > [Tue Jan 05 00:49:22.697531 2016] [socache_shmcb:debug] [pid 1298] > mod_socache_shmcb.c(447): AH00825: subcache_num = 32 > [Tue Jan 05 00:49:22.697544 2016] [socache_shmcb:debug] [pid 1298] > mod_socache_shmcb.c(449): AH00826: subcache_size = 15992 > [Tue Jan 05 00:49:22.697548 2016] [socache_shmcb:debug] [pid 1298] > mod_socache_shmcb.c(451): AH00827: subcache_data_offset = 2128 > [Tue Jan 05 00:49:22.697550 2016] [socache_shmcb:debug] [pid 1298] > mod_socache_shmcb.c(453): AH00828: subcache_data_size = 13864 > [Tue Jan 05 00:49:22.697553 2016] [socache_shmcb:debug] [pid 1298] > mod_socache_shmcb.c(455): AH00829: index_num = 88 > [Tue Jan 05 00:49:22.697629 2016] [socache_shmcb:info] [pid 1298] AH00830: > Shared memory socache initialised > [Tue Jan 05 00:49:22.697633 2016] [ssl:info] [pid 1298] AH01887: Init: > Initializing (virtual) servers for SSL > [Tue Jan 05 00:49:22.697637 2016] [ssl:info] [pid 1298] AH01914: Configuring > server app.myapp.com:443 for SSL protocol > [Tue Jan 05 00:49:22.698297 2016] [ssl:debug] [pid 1298] > ssl_engine_init.c(1600): AH02209: CA certificate: CN=Starfield Secure > Certificate Authority - > G2,OU=http://certs.starfieldtech.com/repository/,O=Starfield Technologies\\, > Inc.,L=Scottsdale,ST=Arizona,C=US > [Tue Jan 05 00:49:22.698314 2016] [ssl:debug] [pid 1298] > ssl_engine_init.c(1600): AH02209: CA certificate: CN=Starfield Root > Certificate Authority - G2,O=Starfield Technologies\\, > Inc.,L=Scottsdale,ST=Arizona,C=US > [Tue Jan 05 00:49:22.698326 2016] [ssl:debug] [pid 1298] > ssl_engine_init.c(1600): AH02209: CA certificate: OU=Starfield Class 2 > Certification Authority,O=Starfield Technologies\\, Inc.,C=US > [Tue Jan 05 00:49:22.698378 2016] [ssl:debug] [pid 1298] > ssl_engine_init.c(390): AH01893: Configuring TLS extension handling > [Tue Jan 05 00:49:22.698579 2016] [ssl:debug] [pid 1298] ssl_util_ssl.c(343): > AH02412: [app.myapp.com:443] Cert matches for name 'app.myapp.com' [subject: > CN=app.myapp.com,OU=Domain Control Validated / issuer: CN=Starfield Secure > Certificate Authority - > G2,OU=http://certs.starfieldtech.com/repository/,O=Starfield Technologies\\, > Inc.,L=Scottsdale,ST=Arizona,C=US / serial: FD5FD2A73620E644 / notbefore: Sep > 7 05:25:43 2015 GMT / notafter: Sep 7 05:25:43 2016 GMT] > [Tue Jan 05 00:49:22.698589 2016] [ssl:info] [pid 1298] AH02568: Certificate > and private key app.myapp.com:443:0 configured from > /etc/apache2/ssl/FD5FD2A73620E644.crt and /etc/apache2/ssl/myapp.key > [Tue Jan 05 00:49:22.698684 2016] [ssl:info] [pid 1298] AH01876: > mod_ssl/2.4.10 compiled against Server: Apache/2.4.10, Library: OpenSSL/1.0.1f > [Tue Jan 05 00:49:22.698892 2016] [wsgi:debug] [pid 1298] > src/server/mod_wsgi.c(7804): mod_wsgi (pid=1298): Socket for 'web2py' is > '/var/run/apache2/wsgi.1298.0.1.sock'. > [Tue Jan 05 00:49:22.698918 2016] [wsgi:debug] [pid 1298] > src/server/mod_wsgi.c(7862): mod_wsgi (pid=1298): Listen backlog for socket > '/var/run/apache2/wsgi.1298.0.1.sock' is '100'. > [Tue Jan 05 00:49:22.699639 2016] [wsgi:info] [pid 1301] mod_wsgi (pid=1301): > Starting process 'web2py' with uid=33, gid=33 and threads=1. > [Tue Jan 05 00:49:22.699917 2016] [wsgi:info] [pid 1301] mod_wsgi (pid=1301): > Initializing Python. > [Tue Jan 05 00:49:22.700691 2016] [proxy:debug] [pid 1302] > proxy_util.c(1771): AH00925: initializing worker proxy:reverse shared > [Tue Jan 05 00:49:22.700728 2016] [proxy:debug] [pid 1302] > proxy_util.c(1813): AH00927: initializing worker proxy:reverse local > [Tue Jan 05 00:49:22.700760 2016] [proxy:debug] [pid 1302] > proxy_util.c(1864): AH00931: initialized single connection worker in child > 1302 for (*) > [Tue Jan 05 00:49:22.700836 2016] [wsgi:info] [pid 1302] mod_wsgi (pid=1302): > Initializing Python. > [Tue Jan 05 00:49:22.703923 2016] [mpm_prefork:notice] [pid 1298] AH00163: > Apache/2.4.10 (Ubuntu) OpenSSL/1.0.1f mod_wsgi/4.4.20 Python/2.7.9 configured > -- resuming normal operations > [Tue Jan 05 00:49:22.703966 2016] [mpm_prefork:info] [pid 1298] AH00164: > Server built: Jul 24 2015 17:25:18 > [Tue Jan 05 00:49:22.703979 2016] [core:notice] [pid 1298] AH00094: Command > line: '/usr/sbin/apache2' > [Tue Jan 05 00:49:22.703983 2016] [mpm_prefork:debug] [pid 1298] > prefork.c(995): AH00165: Accept mutex: fcntl (default: sysvsem) > [Tue Jan 05 00:49:22.704497 2016] [proxy:debug] [pid 1303] > proxy_util.c(1771): AH00925: initializing worker proxy:reverse shared > [Tue Jan 05 00:49:22.704523 2016] [proxy:debug] [pid 1303] > proxy_util.c(1813): AH00927: initializing worker proxy:reverse local > [Tue Jan 05 00:49:22.704558 2016] [proxy:debug] [pid 1303] > proxy_util.c(1864): AH00931: initialized single connection worker in child > 1303 for (*) > [Tue Jan 05 00:49:22.704634 2016] [wsgi:info] [pid 1303] mod_wsgi (pid=1303): > Initializing Python. > [Tue Jan 05 00:49:22.708721 2016] [proxy:debug] [pid 1304] > proxy_util.c(1771): AH00925: initializing worker proxy:reverse shared > [Tue Jan 05 00:49:22.708757 2016] [proxy:debug] [pid 1304] > proxy_util.c(1813): AH00927: initializing worker proxy:reverse local > [Tue Jan 05 00:49:22.708794 2016] [proxy:debug] [pid 1304] > proxy_util.c(1864): AH00931: initialized single connection worker in child > 1304 for (*) > [Tue Jan 05 00:49:22.708874 2016] [wsgi:info] [pid 1304] mod_wsgi (pid=1304): > Initializing Python. > [Tue Jan 05 00:49:22.712708 2016] [proxy:debug] [pid 1306] > proxy_util.c(1771): AH00925: initializing worker proxy:reverse shared > [Tue Jan 05 00:49:22.712723 2016] [proxy:debug] [pid 1305] > proxy_util.c(1771): AH00925: initializing worker proxy:reverse shared > [Tue Jan 05 00:49:22.712742 2016] [proxy:debug] [pid 1306] > proxy_util.c(1813): AH00927: initializing worker proxy:reverse local > [Tue Jan 05 00:49:22.712759 2016] [proxy:debug] [pid 1305] > proxy_util.c(1813): AH00927: initializing worker proxy:reverse local > [Tue Jan 05 00:49:22.712787 2016] [proxy:debug] [pid 1306] > proxy_util.c(1864): AH00931: initialized single connection worker in child > 1306 for (*) > [Tue Jan 05 00:49:22.712877 2016] [wsgi:info] [pid 1306] mod_wsgi (pid=1306): > Initializing Python. > [Tue Jan 05 00:49:22.714612 2016] [wsgi:info] [pid 1302] mod_wsgi (pid=1302): > Attach interpreter ''. > [Tue Jan 05 00:49:22.715366 2016] [wsgi:info] [pid 1302] mod_wsgi (pid=1302): > Imported 'mod_wsgi'. > [Tue Jan 05 00:49:22.716414 2016] [proxy:debug] [pid 1305] > proxy_util.c(1864): AH00931: initialized single connection worker in child > 1305 for (*) > [Tue Jan 05 00:49:22.716546 2016] [wsgi:info] [pid 1305] mod_wsgi (pid=1305): > Initializing Python. > [Tue Jan 05 00:49:22.726402 2016] [wsgi:info] [pid 1301] mod_wsgi (pid=1301): > Attach interpreter ''. > [Tue Jan 05 00:49:22.727207 2016] [wsgi:info] [pid 1301] mod_wsgi (pid=1301): > Imported 'mod_wsgi'. > [Tue Jan 05 00:49:22.729718 2016] [wsgi:info] [pid 1304] mod_wsgi (pid=1304): > Attach interpreter ''. > [Tue Jan 05 00:49:22.730080 2016] [wsgi:info] [pid 1305] mod_wsgi (pid=1305): > Attach interpreter ''. > [Tue Jan 05 00:49:22.730444 2016] [wsgi:info] [pid 1306] mod_wsgi (pid=1306): > Attach interpreter ''. > [Tue Jan 05 00:49:22.730458 2016] [wsgi:info] [pid 1304] mod_wsgi (pid=1304): > Imported 'mod_wsgi'. > [Tue Jan 05 00:49:22.730721 2016] [wsgi:debug] [pid 1301] > src/server/mod_wsgi.c(8417): mod_wsgi (pid=1301): Started thread 0 in daemon > process 'web2py'. > [Tue Jan 05 00:49:22.730824 2016] [wsgi:info] [pid 1305] mod_wsgi (pid=1305): > Imported 'mod_wsgi'. > [Tue Jan 05 00:49:22.731192 2016] [wsgi:info] [pid 1306] mod_wsgi (pid=1306): > Imported 'mod_wsgi'. > [Tue Jan 05 00:49:22.732563 2016] [wsgi:info] [pid 1303] mod_wsgi (pid=1303): > Attach interpreter ''. > [Tue Jan 05 00:49:22.733260 2016] [wsgi:info] [pid 1303] mod_wsgi (pid=1303): > Imported 'mod_wsgi'. > [Tue Jan 05 00:49:35.947693 2016] [ssl:info] [pid 1302] [client > 70.117.108.188:63443] AH01964: Connection to child 0 established (server > app.myapp.com:443) > [Tue Jan 05 00:49:35.948061 2016] [ssl:debug] [pid 1302] > ssl_engine_kernel.c(1908): [client 70.117.108.188:63443] AH02043: SSL virtual > host for servername app.myapp.com found > [Tue Jan 05 00:49:36.006521 2016] [ssl:debug] [pid 1302] > ssl_engine_kernel.c(1841): [client 70.117.108.188:63443] AH02041: Protocol: > TLSv1.2, Cipher: ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits) > [Tue Jan 05 00:49:36.055162 2016] [ssl:debug] [pid 1302] > ssl_engine_kernel.c(243): [client 70.117.108.188:63443] AH02034: Initial > (No.1) HTTPS request received for child 0 (server app.myapp.com:443) > [Tue Jan 05 00:49:36.055280 2016] [authz_core:debug] [pid 1302] > mod_authz_core.c(809): [client 70.117.108.188:63443] AH01626: authorization > result of Require all granted: granted > [Tue Jan 05 00:49:36.055286 2016] [authz_core:debug] [pid 1302] > mod_authz_core.c(809): [client 70.117.108.188:63443] AH01626: authorization > result of <RequireAny>: granted > [Tue Jan 05 00:49:36.055387 2016] [authz_core:debug] [pid 1302] > mod_authz_core.c(809): [client 70.117.108.188:63443] AH01626: authorization > result of Require all granted: granted > [Tue Jan 05 00:49:36.055392 2016] [authz_core:debug] [pid 1302] > mod_authz_core.c(809): [client 70.117.108.188:63443] AH01626: authorization > result of <RequireAny>: granted > [Tue Jan 05 00:49:36.055439 2016] [wsgi:info] [pid 1302] [client > 70.117.108.188:63443] mod_wsgi (pid=1302, process='', application=''): > Loading WSGI script '/home/www-data/web2py/wsgihandler.py'. > [Tue Jan 05 00:49:36.719326 2016] [core:notice] [pid 1298] AH00052: child pid > 1302 exit signal Segmentation fault (11) > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at https://groups.google.com/group/modwsgi > <https://groups.google.com/group/modwsgi>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
