If you don't use Anaconda Python and use standard Python distribution it will likely work fine. This is because the netCDF4 when installed would link with the system version of libcurl which would link with system libnghttp2 library. When you use Anaconda Python, it would link with libcurl/libnghttp2 shipped with Anaconda Python and it is that version which conflicts with mod_http2 in Apache httpd.
Note though that using "python -v venv" instead of "conda env" will not help unless the Python version being used at that point is system Python and not Anaconda distro based Python. To understand, if you look in the directory "/opt/miniforge3/envs/chap/lib" of your virtual environment created using Anaconda Python you should see lots of shared libraries (or symlinks for them to actual Python distro). Follow where those links are pointing at and you will see the target directory in the Anaconda Python distro has its own versions of lots of libraries. It is those libraries being used which is the problem since mod_http2 in Apache httpd will not be linked to those but the system versions. Hope this better explains it. Graham > On 19 Nov 2024, at 9:44 am, Molly Foley <molly.k.fo...@gmail.com> wrote: > > Thanks Graham. If I remade the two environments using venv instead, wouldn’t > it still break because an Apache module is still loading some other version > of a package that overrides the version required by a dependency in my Python > env? I guess I don’t get why venv could potentially work but conda breaks. Is > it just because the “base” environment of venv is your system install of > Python while the “base” environment of conda is, more or less, a virtual > environment itself that doesn’t interact with the system install of Python? > Is that where the conda copies of conflicting libraries occur? > > > On Fri, Nov 15, 2024 at 8:14 PM Graham Dumpleton <graham.dumple...@gmail.com > <mailto:graham.dumple...@gmail.com>> wrote: >> Two reasons why see debug output you do. >> >> 1. Apache httpd starts as root if starting it from systemd and then drops >> privileges. Even if you use mod_wsgi daemon mode and tell it to run daemon >> processes as a specific user, that users login environment is never sourced >> and so will not pick up PATH changes in users local >> .bashrc/.zshrc/.profile/etc. >> >> 2. When Python is run in an embedded system, the "python" executable is >> never run and instead in this case the mod_wsgi module links to the Python >> library. In this case sys.executable has no meaning and can't be relied on >> for anything. >> >> As to the missing nghttp2 symbol, it is likely due to Anaconda Python >> supplying its own copy of that library, but some other Apache module is >> linking to the system variant of that library and they are different >> versions. Any Python module installed in Anaconda Python will though have >> been compiled against the Anaconda version. Because though the system >> version gets loaded into the process first by way of the separate Apache >> module (or even Apache itself), the system one wins out and the Anaconda >> version of the library is never used, so if Python modules were built with >> expectation of a certain symbol existing, but that does not exist in system >> version of that library, then you will get that error. >> >> If you want to test this theory, then disable mod_http2 in Apache since it >> seems to link to nghttp2. https://httpd.apache.org/docs/2.4/howto/http2.html >> >> Make sure you do a full stop/start of Apache to ensure that system nghttp2 >> library is unloaded. >> >> Graham >> >>> On 16 Nov 2024, at 11:14 am, Molly Foley <molly.k.fo...@gmail.com >>> <mailto:molly.k.fo...@gmail.com>> wrote: >>> >>> I'm having issues with a python package we have installed and I'm trying to >>> determine if it's a problem with our mod_wsgi install or the python package >>> itself. I am suspecting our mod_wsgi setup because the application works >>> fine when debugging it locally with the exact same directory acting as the >>> python environment that mod_wsgi is supposed to be using as python-home in >>> our Apache config. >>> >>> Part of my debugging process is having the WSGI application (Flask app) log >>> a bunch of info: >>> User running the app >>> $PATH >>> sys.path >>> sys.prefix >>> sys.executable >>> sys.base_prefix >>> >>> Output: >>> DEBUG [11/15/2024 06:37:38 PM] Running as user: hydro >>> >>> DEBUG [11/15/2024 06:37:38 PM] $PATH: >>> /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin >>> >>> >>> DEBUG [11/15/2024 06:37:38 PM] sys.path: >>> ['/var/www/vhosts/climate-hydro-analytics-platform-v2/api', >>> '/var/www/vhosts/climate-hydro-analytics-platform-v2', >>> '/opt/miniforge3/envs/chap/lib/python312.zip', >>> '/opt/miniforge3/envs/chap/lib/python3.12', >>> '/opt/miniforge3/envs/chap/lib/python3.12/lib-dynload', >>> '/opt/miniforge3/envs/chap/lib/python3.12/site-packages', >>> '/var/www/vhosts/climate-hydro-analytics-platform-v2/api/models'] >>> >>> DEBUG [11/15/2024 06:37:45 PM] Prefix: /opt/miniforge3/envs/chap >>> >>> DEBUG [11/15/2024 06:37:45 PM] Exec: /usr/bin/python3 >>> >>> >>> DEBUG [11/15/2024 06:37:45 PM] Base Prefix: /opt/miniforge3/envs/chap >>> >>> >>> I am spotting two potential problems: >>> I see no /opt/miniforge3/bin or /opt/miniforge3/envs/chap/bin in the $PATH >>> The sys.executable seems to be pointing to the system installation of >>> python instead of the virtual environment set as python-home >>> What's confusing me is that it's getting the prefix right, and the >>> application does actually run and doesn't experience import errors, so I >>> think it is still somehow finding the the right python packages (except for >>> one which is responsible for the error). >>> >>> First question is is are these values expected for $PATH and >>> sys.executable? >>> >>> If I login as the user above (hydro) and echo $PATH, I get a very different >>> value where I do see our python env: >>> /opt/miniforge3/envs/chap/bin:/opt/miniforge3/condabin:/home/hydro/.local/bin:/home/hydro/bin:/opt/miniforge3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/var/cfengine/bin >>> >>> I have attempted to SetEnv in Apache virtual hosts .conf file as well as >>> export it as a new file in /etc/sysconfig, but the printed output above is >>> always the same. >>> >>> Here's our LoadModule directive: >>> >>> <IfModule !wsgi_module> >>> >>> LoadModule wsgi_module >>> /opt/miniforge3/lib/python3.12/site-packages/mod_wsgi/server/mod_wsgi-py312.cpython-312-x86_64-linux-gnu.so >>> <http://mod_wsgi-py312.cpython-312-x86_64-linux-gnu.so/> >>> WSGIPythonHome "/opt/miniforge3" >>> >>> WSGIRestrictEmbedded On >>> >>> >>> </IfModule> >>> >>> >>> And our WSGI config in vhosts (there are two hosts, each using their own >>> python env at python-home): >>> >>> <VirtualHost *:443> >>> >>> ServerName myServer >>> >>> ## Vhost docroot >>> >>> DocumentRoot "/var/www/vhosts/climate-hydro-analytics-platform-v2" >>> >>> ## Alias declarations for resources outside the DocumentRoot >>> >>> Alias /jobs "/output/jobs" >>> >>> ## Logging >>> >>> ErrorLog "/var/log/httpd/chap-error.log" >>> >>> LogLevel info >>> >>> ServerSignature Off >>> >>> CustomLog "/var/log/httpd/chap-access.log" combined >>> >>> ## SSL directives >>> >>> SSLEngine on >>> >>> SSLCertificateFile "/etc/pki/tls/certs/myServer.crt" >>> >>> SSLCertificateKeyFile "/etc/pki/tls/private/myKey.key" >>> >>> SSLVerifyClient none >>> >>> SSLCACertificatePath "/etc/pki/tls/certs" >>> >>> SSLCACertificateFile "/etc/pki/tls/certs/sectigo.ca.crt" >>> >>> ## WSGI configuration >>> >>> WSGIApplicationGroup %{GLOBAL} >>> >>> WSGIDaemonProcess chap display-name=%{GROUP} >>> home=/var/www/vhosts/climate-hydro-analytics-platform-v2 >>> python-home=/opt/miniforge3/envs/chap processes=10 threads=1 user=hydro >>> >>> WSGIProcessGroup chap >>> >>> WSGIScriptAlias / >>> "/var/www/vhosts/climate-hydro-analytics-platform-v2/wsgi.py" >>> >>> #...more stuff >>> >>> >>> </VirtualHost> >>> >>> <VirtualHost *:443> >>> >>> ServerName myServer2 >>> >>> ## Vhost docroot >>> >>> DocumentRoot "/var/www/vhosts/hydrosource-2" >>> >>> ## Logging >>> >>> ErrorLog "/var/log/httpd/dataexplorer-error.log" >>> >>> LogLevel warn >>> >>> ServerSignature Off >>> >>> CustomLog "/var/log/httpd/dataexplorer-access.log" combined >>> >>> ## SSL directives >>> >>> SSLEngine on >>> >>> SSLCertificateFile "/etc/pki/tls/certs/myServer.crt" >>> >>> SSLCertificateKeyFile "/etc/pki/tls/private/myKey.key" >>> >>> SSLVerifyClient none >>> >>> SSLCACertificatePath "/etc/pki/tls/certs" >>> >>> SSLCACertificateFile "/etc/pki/tls/certs/sectigo.ca.crt" >>> >>> ## WSGI configuration >>> >>> WSGIApplicationGroup %{GLOBAL} >>> >>> WSGIDaemonProcess dataexplorer display-name=%{GROUP} >>> home=/var/www/vhosts/hydrosource-2 >>> python-home=/opt/miniforge3/envs/dataexplorer processes=10 threads=1 >>> user=hydro >>> >>> WSGIProcessGroup dataexplorer >>> >>> WSGIScriptAlias / "/var/www/vhosts/hydrosource-2/wsgi.py" >>> >>> #...more stuff >>> >>> >>> >>> </VirtualHost> >>> >>> I realize that conda isn't recommend and pip is, but we're using conda due >>> to difficulties with managing scientific package dependencies (lots of C >>> libraries) with pip. Given the error we're receiving, it is likely it's >>> some C-library stuff conflicting somehow with whatever mod_wsgi is doing >>> under the hood, so we may have to abandon mod_wsgi due to that. But I am >>> very curious about the answer to my above question. >>> >>> If you are curious about the error, we have a python package called xarray >>> where we call the method xarray.open_dataset(...). This prompts the xarray >>> package to import a package called netCDF4. This causes an import error >>> like so: >>> >>> DEBUG [11/15/2024 06:37:45 PM] Traceback (most recent call last): >>> >>> File >>> "/var/www/vhosts/climate-hydro-analytics-platform-v2/api/blueprints/climate_viewer.py", >>> line 69, in get_chart_data >>> >>> ctrl_ds = xarr.open_dataset(control, decode_times=False) >>> >>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>> >>> File >>> "/opt/miniforge3/envs/chap/lib/python3.12/site-packages/xarray/backends/api.py", >>> line 671, in open_dataset >>> >>> backend_ds = backend.open_dataset( >>> >>> ^^^^^^^^^^^^^^^^^^^^^ >>> >>> File >>> "/opt/miniforge3/envs/chap/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", >>> line 666, in open_dataset >>> >>> store = NetCDF4DataStore.open( >>> >>> ^^^^^^^^^^^^^^^^^^^^^^ >>> >>> File >>> "/opt/miniforge3/envs/chap/lib/python3.12/site-packages/xarray/backends/netCDF4_.py", >>> line 415, in open >>> >>> import netCDF4 >>> >>> File >>> "/opt/miniforge3/envs/chap/lib/python3.12/site-packages/netCDF4/__init__.py", >>> line 3, in <module> >>> >>> from ._netCDF4 import * >>> >>> >>> ImportError: >>> /opt/miniforge3/envs/chap/lib/python3.12/site-packages/netCDF4/../../.././libcurl.so.4: >>> undefined symbol: >>> nghttp2_option_set_no_rfc9113_leading_and_trailing_ws_validation >>> >>> If I login as the hydro user, activate the /opt/miniforge3/envs/chap >>> environment, enter python and run the same piece of code >>> (xarray.open_dataset(...)), it works without error. So there's something >>> getting confused internally when going through mod_wsgi and Apache, but we >>> have yet to figure out what after days of trying different things. You told >>> me in my last question >>> <https://groups.google.com/g/modwsgi/c/-Dcai15uiZk>that PHP could be a >>> problem, but we do not have PHP installed (and we are not using Anaconda >>> official distribution either, but rather the open source distribution that >>> utilizes conda and mamba called Miniforge3). >>> >>> Thanks for your continued support! I appreciate it. >>> >>> >> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to modwsgi+unsubscr...@googlegroups.com >>> <mailto:modwsgi+unsubscr...@googlegroups.com>. >>> To view this discussion visit >>> https://groups.google.com/d/msgid/modwsgi/3fda8fd8-5847-4ee1-8849-80248a9ca006n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/modwsgi/3fda8fd8-5847-4ee1-8849-80248a9ca006n%40googlegroups.com?utm_medium=email&utm_source=footer>. >> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "modwsgi" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/modwsgi/gYt2YLmm0cY/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> modwsgi+unsubscr...@googlegroups.com >> <mailto:modwsgi+unsubscr...@googlegroups.com>. >> To view this discussion visit >> https://groups.google.com/d/msgid/modwsgi/53A671DA-388D-4EEA-A2F3-F2F268DE7601%40gmail.com >> >> <https://groups.google.com/d/msgid/modwsgi/53A671DA-388D-4EEA-A2F3-F2F268DE7601%40gmail.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to modwsgi+unsubscr...@googlegroups.com > <mailto:modwsgi+unsubscr...@googlegroups.com>. > To view this discussion visit > https://groups.google.com/d/msgid/modwsgi/CABLg_%3DOAdSHoAYV7z_A3GMt1k0bsvUxPckzcajJkRX0Zd7Wc2A%40mail.gmail.com > > <https://groups.google.com/d/msgid/modwsgi/CABLg_%3DOAdSHoAYV7z_A3GMt1k0bsvUxPckzcajJkRX0Zd7Wc2A%40mail.gmail.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to modwsgi+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/modwsgi/B271B617-D53B-4C41-9878-BD69607D935B%40gmail.com.
