On this issue, the problem is likely that mod_wsgi is picking up the wrong
Python library because Anaconda Python is in a non standard location and you
have a system Python of same major/minor version.
When you configure/make/make install mod_wsgi from source code against Anaconda
Python, you need to set LD_RUN_PATH to the library directory for Anaconda
Python so it knows where to find the correct library at run time.
Thus:
make distclean
./configure --python=/users/vishnuhari/anaconda/bin/python
LD_RUN_PATH=/users/vishnuhari/anaconda/lib make
LD_RUN_PATH=/users/vishnuhari/anaconda/lib make install
Alternatively, you need to work out path for the Anaconda Python library and
add a line into Apache configuration before the LoadModule, which says:
LoadFile /users/vishnuhari/anaconda/lib/libpython2.7.so
The path make be slightly different depending on name given to the library by
Anaconda Python. Ie., could be:
LoadFile /users/vishnuhari/anaconda/lib/libpython2.7m.so
If you ran:
mod_wsgi-express module-config
it should tell you the correct lines, including the LoadFile line, to put in
the Apache configuration.
Send an email with what you get from output of the mod_wsgi-express
module-config command.
Graham
> On 8 Apr 2017, at 3:09 AM, Vishnu Prasad <[email protected]> wrote:
>
> I am trying to move away from embedded and configuring the apache2 (not
> mod_wsgi embedded), virtual hosts but keep getting this issue of Symbol Not
> found , Expected in flat namespace.
>
> I added following virtualhost config.
> <VirtualHost *:80>
> ServerAdmin [email protected] <mailto:[email protected]>
> ServerName www.groc-example.com <http://www.groc-example.com/>
> ServerAlias groc-example.com <http://groc-example.com/>
> WSGIScriptAlias /images/ /var/www/images/wsgiimagehandler.py
> ErrorLog "/private/var/log/apache2/image_error_log"
> CustomLog "/private/var/log/apache2/image_custom_log" common
> WSGIDaemonProcess images python-home=/users/vishnu/anaconda
> python-path=/var/www/images/ processes=1 threads=10 inactivity-timeout=30
> maximum-requests=20
> WSGIProcessGroup images
> WSGIApplicationGroup %{GLOBAL}
> <Directory "/var/www/images/">
> Options +ExecCGI
> Allow from all
> Require all granted
> </Directory>
> </VirtualHost>
>
>
> I added LoadModule wsgi_module
> "/Users/vishnuhari/anaconda/lib/python2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so"
> in the httpd.conf.
>
> Also uncommented the line which has following
> LoadModule cgi_module libexec/apache2/mod_cgi.so
> Include /private/etc/apache2/extra/httpd-vhosts.conf
>
>
> After i did tha above, have restarted and start apachectl,
>
> ERROR :
>
> [Fri Apr 07 22:26:25.838308 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] File "/users/vishnu/anaconda/lib/python2.7/mimetools.py",
> line 6, in <module>
> [Fri Apr 07 22:26:25.838389 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] import tempfile
> [Fri Apr 07 22:26:25.838398 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] File "/users/vishnu/anaconda/lib/python2.7/tempfile.py",
> line 32, in <module>
> [Fri Apr 07 22:26:25.838535 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] import io as _io
> [Fri Apr 07 22:26:25.838544 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] File "/users/vishnu/anaconda/lib/python2.7/io.py", line
> 51, in <module>
> [Fri Apr 07 22:26:25.838615 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] import _io
> [Fri Apr 07 22:26:25.838635 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] ImportError:
> dlopen(/users/vishnuhari/anaconda/lib/python2.7/lib-dynload/_io.so, 2):
> Symbol not found: __PyCodecInfo_GetIncrementalDecoder
> [Fri Apr 07 22:26:25.838640 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] Referenced from:
> /users/vishnuhari/anaconda/lib/python2.7/lib-dynload/_io.so
> [Fri Apr 07 22:26:25.838643 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] Expected in: flat namespace
> [Fri Apr 07 22:26:25.838646 2017] [wsgi:error] [pid 1634] [remote
> 127.0.0.1:54121] in
> /users/vishnuhari/anaconda/lib/python2.7/lib-dynload/_io.so
>
>
>
>
>
> On Friday, April 7, 2017 at 1:05:26 PM UTC+5:30, Vishnu Prasad wrote:
> Graham, Thanks a ton for the response.
>
> I have updated more inputs inline for your review and suggestions.
>
> thanks
>
> On Friday, April 7, 2017 at 8:57:22 AM UTC+5:30, Graham Dumpleton wrote:
>
>> On 7 Apr 2017, at 9:40 AM, Vishnu Prasad <[email protected] <>> wrote:
>>
>> All,
>> I am new to mod_wsgi and the entire configurations around related to this.
>> Detailing out the requirement, please do let me know your response.
>>
>> 1. I have written a simple python script to handle images which listens on
>> port 8041. There is some custom processing happening here and its not just
>> static rendering. The output of this is a content of type image and bytes is
>> the response.
>> 2. I am starting this python script file(application) using command
>> mod_wsgi-express start-server abc.py --threads 20 --process 2
>
> Are you doing image processing? If you are and it is CPU intensive, then use
> of threads, rather than processes, isn't then best idea.
> [Vishnu:] Yes, am using Pillow-SIMD to dynamically generate or resize
> images for optimize image rendering. Once returned from here, caching in
> varnish. We have over 100,000 items to process and at any given point easily
> and i will need to generate about 20-30 images/sec. This webserver cluster is
> about 4 and sitting behind LB. I have done image domain randomization to
> leverage capability of browsers to download more parallel images. Created
> about 3 subdomains for this as well, hence processes of 2. Agree - Once
> performance tested we may tune this further. Further this webserver is
> dedicated to passthrough request to next level appserver outside of which
> this is the only CPU intense process working on these.
>
>> 3. I have varnish script that whenever gets any requests for images on 8080
>> routes to this application on port 8041.
>> 4. This application is also running another default instance of apache2
>> which is serving my all other application requests not related to images.
>
> And this is a Python application also hosted on mod_wsgi, but configured
> manually in normal Apache instance rather than using mod_wsgi-express?
> [Vishnu:] Yes, this is on mod_wsgi but not configured through normal apache
> instance. As you said in one of your blog, getting it working was first - so
> started in embedded mode and now reviewing if I should move out or keep it
> that way for one set of URL's. This means all images goes to mod_wsgi-express
> and others go to the normal apache default instance. Earlier I thought of
> having a virtualhost config to route this for all /image patterns i.e. Daemon
> config as you said in one of the blog, later realized about the modularity of
> this setup. I wanted to use this as stand-alone modular imageserver. I am
> reading through your link below on the issues with embedded mode of running.
> I will review both these options and run some performance tests. This is
> helpful.
>> The above setup is working.
>>
>> Problem/Help needed:
>> 1. I want to configure and run the above application as distinct
>> WSGIDaemonProcess. How do I achieve this ?
>
> If for (4) you are running a Python application hosted on mod_wsgi, you
> should already be using daemon mode as embedded node is generally not
> recommended as explained in:
>
> http://blog.dscpl.com.au/2012/10/why-are-you-using-embedded-mode-of.html
> <http://www.google.com/url?q=http%3A%2F%2Fblog.dscpl.com.au%2F2012%2F10%2Fwhy-are-you-using-embedded-mode-of.html&sa=D&sntz=1&usg=AFQjCNGiURKOSUfshe-g4r-68VMGRnLOYQ>
> [Vishnu:] I am running this app in embedded mode as i was planning to use
> this more to manage/handle only image requests.
>
>> 2. If the above ask is accomplished by a configuration in default apache
>> httpd.conf (not mod_wsgi httpd.conf) then i also want to know how will
>> apache identify my application vs other applications of mod_wsgi which may
>> be running ? Is it based on server url or ?
>
> How to vertically partition a URL namespace so as to directory subsets of
> URLs to a different daemon process group within the same Apache instance, is
> explained in:
>
> http://blog.dscpl.com.au/2014/02/vertically-partitioning-python-web.html
> <http://www.google.com/url?q=http%3A%2F%2Fblog.dscpl.com.au%2F2014%2F02%2Fvertically-partitioning-python-web.html&sa=D&sntz=1&usg=AFQjCNFSqPozGdQVlEtWMQUYSzWo1LEzOQ>
>> 3. For the above application, need to have better control over what is
>> loaded in httpd.conf. So, want to load my own httpd.conf including path of
>> log mounts etc. How do I achieve this ?
> [Vishnu:] My question was how do i specifically setup a httpd.conf for this
> embedded mode i.e. outside the core apache2 installation. Based on what i
> read from you its better to not run in embedded mode. However, if we need to
> then the configuration around logs for this specifically will need to happen
> in mod_wsgi hence the ask.
> Again, am not generating apache config instead am asking how do i tell
> mod_wsgi-express, instead of you reading from /tmp/....httpd.conf, read this
> from /usr/local/httpd.conf ? Is there some server-base path setting which i
> could use and set this or ? I completely understand your point and dont want
> to mess up over-engineering the solution.
>
> I don't understand what you mean by this.
>
> You should not be using mod_wsgi-express to generate Apache configuration to
> then use in an existing Apache installation. The mod_wsgi-express instance is
> self contained with a generated configuration. If customisations for that are
> required, most common things can be overridden using the command line
> options. For some special cases you can use --include-file to add additional
> configuration at the end of the generated configuration if necessary.
>
>
>
>> 4. Finally, at somepoint if I want to manage the httpd.conf common file
>> across WSGI and my default apache, how do I set up the application specific
>> configurations - through virtualhost section in httpd-vhost.conf or ?
>
> If for the other (4) above the answer is that you are also using a manually
> configure Apache/mod_wsgi, as well as a separate mod_wsgi-express for just
> one subset of URLs, then as per blog on vertically partition, you probably
> should just do it all manually in one Apache/mod_wsgi instance if want to
> have config all in one spot. Personally I would probably use multiple
> mod_wsgi-express instance sitting directly behind varnish or whatever front
> end you are using. The generated configuration from mod_wsgi-express is
> usually going to be better than what many would set up manually as it sets a
> whole lot of good stuff related to timeouts which makes things more robust.
> When Apache/mod_wsgi is manually configured there are no default timeouts set.
> [Vishnu:] Understand that you suggest against embdedded mode and against
> managing 2 apache conf files. Even if we need, then you suggest me to take
> just mod_wsgi-express. I will review this as said earlier as well. Also, i
> went through the httpd.conf of mod_wsgi process for WSGIDaemon node and its
> quite extensive. I will keep you posted on the outcome. thanks a ton for a
> detailed answer, the links are helpful and am reading through them for
> guidance.
>
> Anyway, suggest you read that blog on vertically partition a web application
> at least and clarify things by answering questions above.
>
> Graham
>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected]
> <mailto:[email protected]>.
> To post to this group, send email to [email protected]
> <mailto:[email protected]>.
> Visit this group at https://groups.google.com/group/modwsgi
> <https://groups.google.com/group/modwsgi>.
> For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/modwsgi.
For more options, visit https://groups.google.com/d/optout.
