Thank you very much, you put me on the right track! Actually it was a permission issue regarding the user/group under which the wsgi daemon process was running on the server.
Regis Le jeudi 26 septembre 2013 12:57:10 UTC+2, Graham Dumpleton a écrit : > > How is the file being accessed from the NFS file system being delivered > back to the client? > > If the code is ultimately using wsgi.file_wrapper, try setting: > > WSGIEnableSendfile Off > > The EnableSendfile directive only applies to static files Apache itself > serves. Those delivered by mod_wsgi using wsgi.file_wrapper are configured > by separate WSGIEnableSendfile. > > The WSGIEnabledSendfile defaults to On in mod_wsgi 3.X, but if mod_wsgi > 4.0 ever comes out, it will default to Off there. > > If it is any file you have issues accessing, not just ones being served > up, then check whether you have security features such as SELinux enabled > with restrictions on where the Apache user/process may be able access files > from. > > If you are sure that the Apache user can from command line access the > files on the NFS file system, then not likely to be a file system > permissions issue. > > Also try using os.listdir() on directories to actually attempt to get file > listings from directories and see what happens. > > Try that first and if no joy I explain another thing you might try to > allow you to more interactively work out what the process has access to. > > Graham > > On 26/09/2013, at 8:24 PM, regisrob > <[email protected]<javascript:>> > wrote: > > Dear all, > > I desperately come to the community because I'm facing a tricky issue with > mod_wsgi (even if I don't think it's a mod_wsgi bug in itself). I really > need some advice, at least to be sure that my problem is what I think it > is, and if possible have some help to find a solution or workaround. > > *## The issue* : it seems that mod_wsgi can't reach data stored on an > NFS-mounted volume > > *## The context* : I set up an image server called Loris ( > https://github.com/pulibrary/loris) written in Python, that runs behind > Apache using mod_wsgi. Images have to be stored on an NFS-mounted filesystem > > *## My config* : > - Debian Squeeze, Linux 2.6.32-5-amd64 > - Python 2.7.3 in virtualenv "/usr/local/loris/bin/python2.7" > - Apache/2.2.16 (Debian) > - mod_wsgi 3.4, configured with /usr/bin/apxs2 and > /usr/local/loris/bin/python2.7 > > The server is working as expected when it delivers images hosted on the > main partition (/dev/sda1). I checked with different locations as source > images root and everything was ok (home/user ; /usr/local/share/images …). > But once I change the source images root to a directory hosted on an NFS > mount point (/mnt/vdata), image requests fail with 404. > > The Loris error caught in Apache log : > *2013-09-25 14:20:49,356 (loris.resolver) [WARNING]: Source image not > found at /mnt/vdata/demos/images/ark_12148_btv1b84473026_T0000006.jp2 for > identifier ark_12148_btv1b84473026_T0000006.jp2.* > > sda1 and nfs in /etc/fstab : > *# / was on /dev/sda1 during installation > UUID=62dfe91f-8362-413c-ad76-07d348b92ba5 / ext4 > errors=remount-ro 0 1 > 10.10.8.12:/vol/vdata/wbiblissima /mnt/vdata nfs > hard,intr,nolock 0 0* > > > *## My debugging process :* > > *1/* check permissions on NFS /mnt/vdata/ => OK, Apache is able to reach > content of all kind from there... > > *2/* check if Python can reach NFS /mnt/vdata/ => OK, using different > methods in python command line : > * > >>> import os > >>> print os.path.ismount("/mnt/vdata") > True* > > ----------- > > ( > http://stackoverflow.com/questions/13685239/check-in-python-script-if-nfs-server-is-mounted-and-online > ) > > *from subprocess import Popen, check_output > import time > > def call_timeout(cmd, timeout): > start = time.time() > p = Popen(cmd) > while time.time() - start < timeout: > if p.poll() is not None: > return > time.sleep(0.1) > p.kill() > raise OSError('command timed out') > > call_timeout(["ls", "/mnt/vdata"], 5.0)* > > OK for "ls" command > > ----------- > > *>>> import os > >>> > os.path.isfile("/mnt/vdata/demos/images/ark_12148_btv1b84473026_T0000006.jp2") > True* > > > *3/* check if Python can reach NFS /mnt/vdata/ via Apache/mod_wsgi => NOT > OK !! > I ran the same tests as above within a sample wsgi app and the output was : > * > os.path.exists("/mnt/vdata/") = False > os.path.isfile("/mnt/vdata/demos/images/ark_12148_btv1b84473026_T0000006.jp2") > > = False* > > But, on the contrary : > > *os.path.exists("/usr/local/share/images") = True > os.path.isfile("/usr/local/share/images/ark_12148_btv1b84473026_T0000005.jp2") > > = True* > > => so it seems pretty sure that *Apache/mod_wsgi can't reach NFS-mounted > filesystem* (whereas there is no problem with other locations on the main > partition)… > > > *4/* then I read the "Debugging Techniques" page on modwsgi wiki, > searched Google, read some forum threads, but unfortunately nothing > concluding… : > > - I tried "EnableSendfile Off" in my virtualhost conf and in apache2.conf > (as explained here > http://pythonhaven.wordpress.com/2009/05/24/configuring-apache-mod_wsgi-django-on-virtual-box-serving-files-from-windows-7/; > > http://httpd.apache.org/docs/2.2/mod/core.html#enablesendfile) > > - I checked the "middleware wrapper application" given there : > http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Tracking_Request_and_Response > > Here is what i can see in the Apache error logs : > > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] ('REQUEST', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] {'DOCUMENT_ROOT': > '/var/www/vhosts/interne', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'GATEWAY_INTERFACE': 'CGI/1.1', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'HTTP_ACCEPT': > 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'HTTP_ACCEPT_ENCODING': 'gzip, deflate', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'HTTP_ACCEPT_LANGUAGE': 'fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'HTTP_CACHE_CONTROL': 'max-age=0', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'HTTP_CONNECTION': 'Keep-Alive', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'HTTP_COOKIE': > '__utma=10294765.500453923.1371827759.1377784603.1377793104.17; > __utmz=10294765.1371827759.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); > Drupal.toolbar.collapsed=0; > _pk_id.1.d439=6077972781a450a8.1373881136.2.1374154272.1373883517.; > Drupal.tableDrag.showWeight=0; > SESS129a620846951a2e9472e600d76a8e6d=0SFg8bdPQFKa0OBrEs4zHmVUs81BnvZfgXtFk2FKx1I', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'HTTP_HOST': ' > interne.exemple.fr', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:19.0) > Gecko/20100101 Firefox/19.0', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'HTTP_VIA': '1.1 > interne.exemple.fr', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'HTTP_X_FORWARDED_FOR': '191.234.187.220', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'HTTP_X_FORWARDED_HOST': 'interne.exemple.fr', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'HTTP_X_FORWARDED_SERVER': 'interne.exemple.fr', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'PATH_INFO': > '/ark_12148_btv1b84473026_T0000006.jp2/full/full/0/native.jpg', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'PATH_TRANSLATED': > '/var/www/vhosts/interne/ark_12148_btv1b84473026_T0000006.jp2/full/full/0/native.jpg', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'QUERY_STRING': > '', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'REMOTE_ADDR': > '10.14.1.38', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'REMOTE_PORT': > '35296', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'REQUEST_METHOD': > 'GET', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'REQUEST_URI': > '/iiif/ark_12148_btv1b84473026_T0000006.jp2/full/full/0/native.jpg', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'SCRIPT_FILENAME': '/var/www/iiif/loris/loris.wsgi', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'SCRIPT_NAME': > '/iiif', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'SERVER_ADDR': > '10.14.31.22', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'SERVER_ADMIN': ' > [email protected] <javascript:>', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'SERVER_NAME': ' > interne.exemple.fr', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'SERVER_PORT': > '80', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'SERVER_PROTOCOL': 'HTTP/1.1', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'SERVER_SIGNATURE': '', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'SERVER_SOFTWARE': 'Apache', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.application_group': 'interne.exemple.fr|/iiif', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.callable_object': 'application', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.enable_sendfile': '0', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.handler_script': '', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.input_chunked': '0', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.listener_host': '', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.listener_port': '80', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.process_group': 'loris', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.queue_start': '1380182347636428', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.request_handler': 'wsgi-script', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.script_reloading': '1', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'mod_wsgi.version': (3, 4), > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'wsgi.errors': > <mod_wsgi.Log object at 0x7f53f460cfb0>, > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'wsgi.file_wrapper': <built-in method file_wrapper of mod_wsgi.Adapter > object at 0x7f53f3f62030>, > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'wsgi.input': > <mod_wsgi.Input object at 0x7f53f3fc8370>, > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'wsgi.multiprocess': True, > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'wsgi.multithread': True, > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'wsgi.run_once': > False, > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] > 'wsgi.url_scheme': 'http', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] 'wsgi.version': > (1, 0)}) > 2013-09-26 09:59:08,734 (loris.resolver) [WARNING]: Source image not found > at /mnt/vdata/demos/images/ark_12148_btv1b84473026_T0000006.jp2 for > identifier: ark_12148_btv1b84473026_T0000006.jp2. > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] ('RESPONSE', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] '404 NOT FOUND', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] [('Content-Type', > 'text/plain; charset=utf-8'), > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] ('Link', > [Thu Sep 26 09:59:08 2013] [error] [client 10.14.1.38] '< > http://library.stanford.edu/iiif/image-api/1.1/compliance.html#level2 > >;rel="profile"')]) > > When I deploy Loris server with a source images root outside of NFS (e.g. > on /home or /usr/local or anywhere else…), the logs look exactly the same, > except the Response at the end (of course) : > > [Thu Sep 26 10:16:05 2013] [error] [client 10.14.1.38] ('RESPONSE', > [Thu Sep 26 10:16:05 2013] [error] [client 10.14.1.38] '200 OK', > [Thu Sep 26 10:16:05 2013] [error] [client 10.14.1.38] [('Content-Type', > 'image/jpeg'), > [Thu Sep 26 10:16:05 2013] [error] [client 10.14.1.38] ('Link', > [Thu Sep 26 10:16:05 2013] [error] [client 10.14.1.38] '< > http://library.stanford.edu/iiif/image-api/1.1/compliance.html#level2 > >;rel="profile"'), > [Thu Sep 26 10:16:05 2013] [error] [client 10.14.1.38] ('Last-Modified', > 'Thu, 26 Sep 2013 08:16:05 GMT'), > [Thu Sep 26 10:16:05 2013] [error] [client 10.14.1.38] > ('Content-Length', '2658052')]) > > > Has anyone ever faced that kind of problem before ? What can I do next ? > Do you have a clue for a workaround or another debugging technique ? > > Thank you very much for your help. > > Regis > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To post to this group, send email to [email protected] <javascript:> > . > Visit this group at http://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/groups/opt_out.
