> On 11 Apr 2017, at 4:49 PM, Vishnu Prasad <[email protected]> wrote: > > Graham, > > I get the following when i run module-config. Using this only I copied the > wsgi module path. > > LoadFile "/Users/vishnuhari/anaconda/lib/libpython2.7.dylib" > LoadModule wsgi_module > "/Users/vishnuhari/anaconda/lib/python2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so" > WSGIPythonHome "/Users/vishnuhari/anaconda" >
Huh. You should be using all those lines. Are you using the LoadFile line or not. It is important in your case that it is included. Graham > Further if you look at vhost entry below..am giving all three paths in > python-home > <VirtualHost *:80> > ServerAdmin [email protected] <mailto:[email protected]> > ServerName www.groc-example.com <http://www.groc-example.com/> > ServerAlias groc-example.com <http://groc-example.com/> > WSGIScriptAlias /images/ /var/www/images/wsgiimagehandler.py > ErrorLog "/private/var/log/apache2/image_error_log" > CustomLog "/private/var/log/apache2/image_custom_log" common > WSGIDaemonProcess images python-home=/users/vishnuhari/anaconda > python-path=/var/www/images/ processes=1 threads=10 inactivity-timeout=30 > maximum-requests=20 > WSGIProcessGroup images > <Directory "/var/www/images/"> > Options +ExecCGI > Allow from all > Require all granted > </Directory> > </VirtualHost> > > Note that in my code when I do run the simple server, my make server is as > follows which is running on 8051. I have simplifed the entire code below for > brevity. The actual code can be seen @ > https://github.com/vishnuprasadh/imgprocessor/blob/master/wsgiimagehandler.py > <https://github.com/vishnuprasadh/imgprocessor/blob/master/wsgiimagehandler.py> > > #!usr/bin/env python > > from wsgiref.simple_server import make_server > from wsgiref.handlers import CGIHandler > import io > from mod_wsgi import server > from imagehandler import imagehandler > import cgi > > > def application(environment, start_response): > returnquerystring = "" > response_body = "Oops,we are sorry!" > status = '200 OK' > > try: > if environment["PATH_INFO"] == "/images/": > > img = imagehandler() > returnquerystring = img.generate() > > '''In the response header, we will also set imagekey for use in > varnish and other edgecache servers''' > response_headers = [ > ('Content-Type', 'image/jpeg'), > # ("Content-Type","text/html"), > ('Content-Length', str(len(response_body))), > ('Server', "Encrypt"), > ('imagekey', str(returnquerystring))] > > except Exception as genex: > status = '400 Bad Request' > response_body = str(genex) > response_headers = [ > # ('Content-Type', 'image/jpeg'), > ("Content-Type", "text/html"), > ('Content-Length', str(len(response_body))), > ('Server', "Encrypt"), > ('Response_QS', str(returnquerystring))] > finally: > start_response(status, response_headers) > return [response_body] > > > '''Make the server -note that the requests shoudl come to this port''' > httpd = make_server('', 8051, application) > '''Continue to listen on this server forever''' > httpd.serve_forever() > > The error I get is : > > [Tue Apr 11 13:07:04.618145 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] File > "/users/vishnuhari/anaconda/lib/python2.7/mimetools.py", line 6, in <module> > [Tue Apr 11 13:07:04.618475 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] import tempfile > [Tue Apr 11 13:07:04.618495 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] File > "/users/vishnuhari/anaconda/lib/python2.7/tempfile.py", line 32, in <module> > [Tue Apr 11 13:07:04.619091 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] import io as _io > [Tue Apr 11 13:07:04.619109 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] File "/users/vishnuhari/anaconda/lib/python2.7/io.py", > line 51, in <module> > [Tue Apr 11 13:07:04.619416 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] import _io > [Tue Apr 11 13:07:04.619444 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] ImportError: > dlopen(/users/vishnuhari/anaconda/lib/python2.7/lib-dynload/_io.so, 2): > Symbol not found: __PyCodecInfo_GetIncrementalDecoder > [Tue Apr 11 13:07:04.619449 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] Referenced from: > /users/vishnuhari/anaconda/lib/python2.7/lib-dynload/_io.so > [Tue Apr 11 13:07:04.619452 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] Expected in: flat namespace > [Tue Apr 11 13:07:04.619455 2017] [wsgi:error] [pid 3217] [remote > 127.0.0.1:55330] in > /users/vishnuhari/anaconda/lib/python2.7/lib-dynload/_io.so > Note that this is currently working if i run the script through python > standalone bound with nohup. It is also working through mod_wsgi-express.... > > First step is to resolve this to run out of core apache2 outside embedded. > > Again, my worry is in production setup..I dont have such luxury as in my > local. How do I simplify/streamline this entire thing because ultimately this > will run in a cluster of apache servers sitting on RHEL 6.x. > > > thanks in anticipation... > > > > On Saturday, April 8, 2017 at 9:29:16 AM UTC+5:30, Graham Dumpleton wrote: > On this issue, the problem is likely that mod_wsgi is picking up the wrong > Python library because Anaconda Python is in a non standard location and you > have a system Python of same major/minor version. > > When you configure/make/make install mod_wsgi from source code against > Anaconda Python, you need to set LD_RUN_PATH to the library directory for > Anaconda Python so it knows where to find the correct library at run time. > > Thus: > > make distclean > > ./configure --python=/users/vishnuhari/anaconda/bin/python > > LD_RUN_PATH=/users/vishnuhari/anaconda/lib make > LD_RUN_PATH=/users/vishnuhari/anaconda/lib make install > > Alternatively, you need to work out path for the Anaconda Python library and > add a line into Apache configuration before the LoadModule, which says: > > LoadFile /users/vishnuhari/anaconda/lib/libpython2.7.so > <http://libpython2.7.so/> > > The path make be slightly different depending on name given to the library by > Anaconda Python. Ie., could be: > > LoadFile /users/vishnuhari/anaconda/lib/libpython2.7m.so > <http://libpython2.7m.so/> > > If you ran: > > mod_wsgi-express module-config > > it should tell you the correct lines, including the LoadFile line, to put in > the Apache configuration. > > Send an email with what you get from output of the mod_wsgi-express > module-config command. > > Graham > >> On 8 Apr 2017, at 3:09 AM, Vishnu Prasad <vishnu...@ <>gmail.com >> <http://gmail.com/>> wrote: >> >> I am trying to move away from embedded and configuring the apache2 (not >> mod_wsgi embedded), virtual hosts but keep getting this issue of Symbol Not >> found , Expected in flat namespace. >> >> I added following virtualhost config. >> <VirtualHost *:80> >> ServerAdmin vis...@ <>bxl.com <http://bxl.com/> >> ServerName www.groc-example.com <http://www.groc-example.com/> >> ServerAlias groc-example.com <http://groc-example.com/> >> WSGIScriptAlias /images/ /var/www/images/wsgiimagehandler.py >> ErrorLog "/private/var/log/apache2/image_error_log" >> CustomLog "/private/var/log/apache2/image_custom_log" common >> WSGIDaemonProcess images python-home=/users/vishnu/anaconda >> python-path=/var/www/images/ processes=1 threads=10 inactivity-timeout=30 >> maximum-requests=20 >> WSGIProcessGroup images >> WSGIApplicationGroup %{GLOBAL} >> <Directory "/var/www/images/"> >> Options +ExecCGI >> Allow from all >> Require all granted >> </Directory> >> </VirtualHost> >> >> >> I added LoadModule wsgi_module >> "/Users/vishnuhari/anaconda/lib/python2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so" >> in the httpd.conf. >> >> Also uncommented the line which has following >> LoadModule cgi_module libexec/apache2/mod_cgi.so >> Include /private/etc/apache2/extra/httpd-vhosts.conf >> >> >> After i did tha above, have restarted and start apachectl, >> >> ERROR : >> >> [Fri Apr 07 22:26:25.838308 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] File >> "/users/vishnu/anaconda/lib/python2.7/mimetools.py", line 6, in <module> >> [Fri Apr 07 22:26:25.838389 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] import tempfile >> [Fri Apr 07 22:26:25.838398 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] File >> "/users/vishnu/anaconda/lib/python2.7/tempfile.py", line 32, in <module> >> [Fri Apr 07 22:26:25.838535 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] import io as _io >> [Fri Apr 07 22:26:25.838544 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] File >> "/users/vishnu/anaconda/lib/python2.7/io.py", line 51, in <module> >> [Fri Apr 07 22:26:25.838615 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] import _io >> [Fri Apr 07 22:26:25.838635 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] ImportError: >> dlopen(/users/vishnuhari/anaconda/lib/python2.7/lib-dynload/_io.so, 2): >> Symbol not found: __PyCodecInfo_GetIncrementalDecoder >> [Fri Apr 07 22:26:25.838640 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] Referenced from: >> /users/vishnuhari/anaconda/lib/python2.7/lib-dynload/_io.so >> [Fri Apr 07 22:26:25.838643 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] Expected in: flat namespace >> [Fri Apr 07 22:26:25.838646 2017] [wsgi:error] [pid 1634] [remote >> 127.0.0.1:54121 <http://127.0.0.1:54121/>] in >> /users/vishnuhari/anaconda/lib/python2.7/lib-dynload/_io.so >> >> >> >> >> >> On Friday, April 7, 2017 at 1:05:26 PM UTC+5:30, Vishnu Prasad wrote: >> Graham, Thanks a ton for the response. >> >> I have updated more inputs inline for your review and suggestions. >> >> thanks >> >> On Friday, April 7, 2017 at 8:57:22 AM UTC+5:30, Graham Dumpleton wrote: >> >>> On 7 Apr 2017, at 9:40 AM, Vishnu Prasad <[email protected] <>> wrote: >>> >>> All, >>> I am new to mod_wsgi and the entire configurations around related to this. >>> Detailing out the requirement, please do let me know your response. >>> >>> 1. I have written a simple python script to handle images which listens on >>> port 8041. There is some custom processing happening here and its not just >>> static rendering. The output of this is a content of type image and bytes >>> is the response. >>> 2. I am starting this python script file(application) using command >>> mod_wsgi-express start-server abc.py --threads 20 --process 2 >> >> Are you doing image processing? If you are and it is CPU intensive, then use >> of threads, rather than processes, isn't then best idea. >> [Vishnu:] Yes, am using Pillow-SIMD to dynamically generate or resize >> images for optimize image rendering. Once returned from here, caching in >> varnish. We have over 100,000 items to process and at any given point easily >> and i will need to generate about 20-30 images/sec. This webserver cluster >> is about 4 and sitting behind LB. I have done image domain randomization to >> leverage capability of browsers to download more parallel images. Created >> about 3 subdomains for this as well, hence processes of 2. Agree - Once >> performance tested we may tune this further. Further this webserver is >> dedicated to passthrough request to next level appserver outside of which >> this is the only CPU intense process working on these. >> >>> 3. I have varnish script that whenever gets any requests for images on 8080 >>> routes to this application on port 8041. >>> 4. This application is also running another default instance of apache2 >>> which is serving my all other application requests not related to images. >> >> And this is a Python application also hosted on mod_wsgi, but configured >> manually in normal Apache instance rather than using mod_wsgi-express? >> [Vishnu:] Yes, this is on mod_wsgi but not configured through normal apache >> instance. As you said in one of your blog, getting it working was first - so >> started in embedded mode and now reviewing if I should move out or keep it >> that way for one set of URL's. This means all images goes to >> mod_wsgi-express and others go to the normal apache default instance. >> Earlier I thought of having a virtualhost config to route this for all >> /image patterns i.e. Daemon config as you said in one of the blog, later >> realized about the modularity of this setup. I wanted to use this as >> stand-alone modular imageserver. I am reading through your link below on the >> issues with embedded mode of running. I will review both these options and >> run some performance tests. This is helpful. >>> The above setup is working. >>> >>> Problem/Help needed: >>> 1. I want to configure and run the above application as distinct >>> WSGIDaemonProcess. How do I achieve this ? >> >> If for (4) you are running a Python application hosted on mod_wsgi, you >> should already be using daemon mode as embedded node is generally not >> recommended as explained in: >> >> http://blog.dscpl.com.au/2012/10/why-are-you-using-embedded-mode-of.html >> <http://www.google.com/url?q=http%3A%2F%2Fblog.dscpl.com.au%2F2012%2F10%2Fwhy-are-you-using-embedded-mode-of.html&sa=D&sntz=1&usg=AFQjCNGiURKOSUfshe-g4r-68VMGRnLOYQ> >> [Vishnu:] I am running this app in embedded mode as i was planning to use >> this more to manage/handle only image requests. >> >>> 2. If the above ask is accomplished by a configuration in default apache >>> httpd.conf (not mod_wsgi httpd.conf) then i also want to know how will >>> apache identify my application vs other applications of mod_wsgi which may >>> be running ? Is it based on server url or ? >> >> How to vertically partition a URL namespace so as to directory subsets of >> URLs to a different daemon process group within the same Apache instance, is >> explained in: >> >> http://blog.dscpl.com.au/2014/02/vertically-partitioning-python-web.html >> <http://www.google.com/url?q=http%3A%2F%2Fblog.dscpl.com.au%2F2014%2F02%2Fvertically-partitioning-python-web.html&sa=D&sntz=1&usg=AFQjCNFSqPozGdQVlEtWMQUYSzWo1LEzOQ> >>> 3. For the above application, need to have better control over what is >>> loaded in httpd.conf. So, want to load my own httpd.conf including path of >>> log mounts etc. How do I achieve this ? >> [Vishnu:] My question was how do i specifically setup a httpd.conf for this >> embedded mode i.e. outside the core apache2 installation. Based on what i >> read from you its better to not run in embedded mode. However, if we need to >> then the configuration around logs for this specifically will need to happen >> in mod_wsgi hence the ask. >> Again, am not generating apache config instead am asking how do i tell >> mod_wsgi-express, instead of you reading from /tmp/....httpd.conf, read this >> from /usr/local/httpd.conf ? Is there some server-base path setting which i >> could use and set this or ? I completely understand your point and dont want >> to mess up over-engineering the solution. >> >> I don't understand what you mean by this. >> >> You should not be using mod_wsgi-express to generate Apache configuration to >> then use in an existing Apache installation. The mod_wsgi-express instance >> is self contained with a generated configuration. If customisations for that >> are required, most common things can be overridden using the command line >> options. For some special cases you can use --include-file to add additional >> configuration at the end of the generated configuration if necessary. >> >> >> >>> 4. Finally, at somepoint if I want to manage the httpd.conf common file >>> across WSGI and my default apache, how do I set up the application specific >>> configurations - through virtualhost section in httpd-vhost.conf or ? >> >> If for the other (4) above the answer is that you are also using a manually >> configure Apache/mod_wsgi, as well as a separate mod_wsgi-express for just >> one subset of URLs, then as per blog on vertically partition, you probably >> should just do it all manually in one Apache/mod_wsgi instance if want to >> have config all in one spot. Personally I would probably use multiple >> mod_wsgi-express instance sitting directly behind varnish or whatever front >> end you are using. The generated configuration from mod_wsgi-express is >> usually going to be better than what many would set up manually as it sets a >> whole lot of good stuff related to timeouts which makes things more robust. >> When Apache/mod_wsgi is manually configured there are no default timeouts >> set. >> [Vishnu:] Understand that you suggest against embdedded mode and against >> managing 2 apache conf files. Even if we need, then you suggest me to take >> just mod_wsgi-express. I will review this as said earlier as well. Also, i >> went through the httpd.conf of mod_wsgi process for WSGIDaemon node and its >> quite extensive. I will keep you posted on the outcome. thanks a ton for a >> detailed answer, the links are helpful and am reading through them for >> guidance. >> >> Anyway, suggest you read that blog on vertically partition a web application >> at least and clarify things by answering questions above. >> >> Graham >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to modwsgi+u...@ <>googlegroups.com <http://googlegroups.com/>. >> To post to this group, send email to mod...@ <>googlegroups.com >> <http://googlegroups.com/>. >> Visit this group at https://groups.google.com/group/modwsgi >> <https://groups.google.com/group/modwsgi>. >> For more options, visit https://groups.google.com/d/optout >> <https://groups.google.com/d/optout>. > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at https://groups.google.com/group/modwsgi > <https://groups.google.com/group/modwsgi>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
