For some context as to why it helps see: http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Python_Simplified_GIL_State_API
Graham On 8 November 2011 03:39, Hang-A <[email protected]> wrote: > That was it. > Thanks, Graham! > > Myung-Hwa > > On Nov 5, 5:42 pm, Graham Dumpleton <[email protected]> > wrote: >> Try setting: >> >> WSGIApplicationGroup %{GLOBAL} >> >> for that WSGI application in Apache configuration. >> >> Likely you are using a third party C extension module in there >> somewhere which isn't safe to use in sub interpreter. >> >> That directive will force your application to run in main Python >> interpreter within the process. >> >> Graham >> >> On 6 November 2011 06:56, Hang-A <[email protected]> wrote: >> >> >> >> >> >> >> >> > Hello, list! >> >> > First of all, this will be a long message, due to the complexity of >> > our environment. >> > So, please be patient with my question. >> >> > I am trying to run a simple django application in a cluster >> > environment. >> > And, my application hangs while it imports scipy.linalg, and both >> > scipy and apache do not write out error messages. >> > When I run my application in my local python shell, it imports >> > scipy.linalg. But, somehow it does not when it is run by apache. >> > So, after reading this message, please share any ideas about how to >> > debug this problem or new solutions to address this issue or deploy my >> > application. >> >> > Now, let me explain our current setup. >> > 1. OS >> > -- The server is a compute cluster where each node runs centos 6 that >> > was installed from a clean version of centos6 minimal. >> > 2. Apache >> > -- Apache 2.2 was also manually installed from one of default linux >> > repository. >> > To be specific, it was installed from its source code together with >> > httpd-dev. >> > 3. Python >> > -- Python 2.7.2 was also installed from its source code across all >> > nodes in the cluster. Its source code was downloaded from python.org's >> > ftp. >> > 4. Python packages: nose, numpy, scipy >> > -- Nose 1.1.2 was downloaded from pypi.python.org and installed from >> > its source code. >> > -- numpy 1.6.1 was downloaded and installed from a linux repository. >> > When building numpy, gnu95 fortran complier was used. >> > -- To install scipy, we installed atlas-3.8.4, lapack-3.3.1, and blas >> > from their source code. >> > ----- atlas was from sourceforge's 3.8.4 stable version. To compile >> > altas, gcc was used. >> > ----- lapack and blas was obtained from netlib.org's repository. To >> > compile the package of lapack and blas, gforan was used. >> > ----- Finally, after exporting paths to blas, lapack, and atlas, >> > scipy-0.9.0 was installed from its source code. >> > scipy was obtained from sourceforge.net's repository. >> >> > All of the above were installed in the same way across all nodes in >> > our cluster. >> > Since I am the only user of the cluster who needs to run python web >> > applications, >> > I installed python virtualenv package in my local directory. >> > Within my virtual environment, django-1.3 and pysal-1.2 (our own >> > package) were installed. >> > To deploy my web applications, we used mod_wsgi. >> > mod-wsgi was compiled with python-2.7.2 and loaded into apache-2.2. >> >> > My application is as follows: >> > import os, sys, site, time >> > print >> sys.stderr, str(time.time()) + ':' + str(time.ctime()) >> >> > root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), >> > '..')) >> > root_dir += '/myunghwa_python2' >> > print >> sys.stderr, root_dir >> > print >> sys.stderr, os.listdir(root_dir) >> > site.addsitedir(os.path.join(root_dir, 'lib/python2.7/site-packages')) >> > sys.path.append(root_dir) >> > print >> sys.stderr, sys.path >> > print >> sys.stderr, os.listdir(sys.path[-2]) >> >> > os.environ['PYTHON_EGG_CACHE'] = '/home/mhwang3/repos/django/python- >> > eggs' >> > os.environ['DJANGO_SETTINGS_MODULE'] = 'wsgi.settings' >> >> > def application(environ, start_response): >> > status = '200 OK' >> > output = 'Hello!' >> > try: >> > t1 = time.time() >> > print >> sys.stderr, "Going to import numpy: " + >> > str(time.time()) + ":" + str(time.time() - t1) >> > import numpy >> > print >> sys.stderr, "imported numpy" >> > print >> sys.stderr, "Going to import scipy" >> > import scipy >> > print >> sys.stderr, "imported scipy >> > from:"+str(scipy.__path__) >> > print >> sys.stderr, "Going to import scipy.linalg" + >> > str(time.time()) + ':' + str(time.time() - t1) >> > import scipy.stats >> > print >> sys.stderr, "imported scipy.linalg: " + >> > str(time.time()) + ':' + str(time.time() - t1) >> > print >> sys.stderr, "Going to import pysal" >> > import pysal >> > print >> sys.stderr, "imported pysal" >> > except: >> > print >> sys.stderr, "exception" >> > raise >> >> > response_headers = [('Content-type', 'text/plain'), >> > ('Content-Length', str(len(output)))] >> > start_response(status, response_headers) >> >> > return [output] >> >> > Basically, it is a 'hello world' application that tests if numpy, >> > scipy, and pysal can be imported. >> > In the attached file, lines 4-9 are just adding paths to django and >> > pysal so that apache knows where to find these packages. >> > Also, to let apache know where to find atlas-related packages, the >> > path to those packages was added to the LD_LIBRARY_PATH environment >> > variable in the /etc/sysconfig/httpd file. >> >> > When I first ran my application, it just hung and wrote no message. >> > So, across scipy.linalg modules, I added print out statements to >> > figure out at which point the import was broken. >> > Here is the messages I got when I imported scipy.linalg in my local >> > python shell. >> > ######################## >> > starting linalg.__init__ >> > pre __init__.__doc__ >> > pre __init__.__version__ >> > pre __init__.misc >> > pre __init__.basic >> > ####################### >> > Starting basic >> > pre basic.flinalg >> > pre basic.lapack >> > pre basic.misc >> > pre basic.scipy.linalg >> > pre basic.decomp_svd >> > pre __init__.decomp >> > ################ >> > starting decomp >> > pre decomp.array et al. >> > pre decomp.calc_lwork >> > pre decomp.LinAlgError >> > pre decomp.get_lapack_funcs >> > pre decomp.get_blas_funcs >> > #################### >> > Starting blas >> > pre blas.scipy.linalg.fblas >> > pre blas.scipy.linalg.cblas >> > pre __init__.decomp_lu >> > pre __init__.decomp_cholesky >> > pre __init__.decomp_qr >> > ################# >> > Starting special_matrices >> > pre special_matrices.math >> > pre special_matrices.np >> > pre __init__.decomp_svd >> > pre __init__.decomp_schur >> > ################## >> > starting schur... >> > pre decomp_schur.misc >> > pre decomp_schur.LinAlgError >> > pre decomp_schur.get_lapack_funcs >> > pre decomp_schur.eigvals:1320454147.23Fri Nov 4 17:49:07 2011 >> > schur testing >> > pre __init__.matfuncs >> > ##################### >> > Starting matfuncs >> > pre matfuncs. asarray et al >> > pre matfuncs.matrix >> > pre matfuncs.np >> > pre matfuncs.misc >> > pre matfuncs.basic >> > pre matfuncs.special_matrices >> > pre matfuncs.decomp >> > pre matfuncs.decomp_svd >> > pre matfuncs.decomp_schur >> > pre __init__.blas >> > pre __init__.special_matrices >> > When scipy.linalg is successfully imported, I should get these >> > messages. >> > But, when my web application tried to import scipy.linalg, the output >> > messages stop at line 41. >> > At line 41, decomp_schur.py tries to import decomp.py. Since decomp.py >> > was already imported at line 16, scipy ignores it and continues to >> > import other modules in my local shell. >> > But, somehow, in apache-mod_wsgi environment, scipy failed to ignore >> > or reload decomp.py and seems to kill my web application. >> > This is really odd, because python does not give any message about >> > this error and neither does apache. apache just hangs without sending >> > out any response. >> > Since lapack and blas functions were imported successfully, the >> > problem seems not related to path setup. >> >> > If anyone in the list has any insights into or experience into this >> > kind of symptom, >> > please share your insights and experience. In particular, debugging >> > techniques or less-known installation/compilation problems would be >> > helpful. >> > I feel like I am at a dead end. So, please help me. >> >> > Thanks for reading this post. >> > I will look forward to your responses. >> >> > -- Myunghwa-Hwang Hwang >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "modwsgi" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group >> > athttp://groups.google.com/group/modwsgi?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
