Over at Astronomer we're just building out and testing an RHEL7 docker image - it will work outside of the Astronomer platform fine): https://github.com/astronomer/ap-airflow/tree/master/1.10.5/rhel7 (That is 1.10.5 with a few things backported, once we've tested that image we will update it to the latest version of Airflow.
If you want to test it out you can pull astronomerinc/ap-airflow:1.10.5-rhel7 and use that as a base image. (There are also more "fixed" tags such as 1.10.5-rhel7-2840 ) https://hub.docker.com/r/astronomerinc/ap-airflow/tags?page=1&name=rhel7 for the complete list. HTH, -Ash On Jan 31 2020, at 11:40 am, Michael Lutz <[email protected]> wrote: > I have, yes. We are running rhel 7.7. It appeared non-trivial to go down that > path so I hesitated. Would that direction be a general recommendation and > would I want to choose a centos or rhel build? Or maybe they are one in the > same. If you recommend docker then some pointing in to the best direction > would be greatly appreciated thanks! Mike > > On Fri, Jan 31, 2020, 6:33 AM Kamil Breguła <[email protected] > (mailto:[email protected])> wrote: > > Hello, > > > > Have you considered using Docker? > > Best regards, > > Kamil > > > > On Fri, Jan 31, 2020 at 12:07 PM Michael Lutz <[email protected] > > (mailto:[email protected])> wrote: > > > > > > Hi, I've been working on installing airflow for a3 days now on red hat > > > linux 7.7 and would greatly appreciate some help. My troubles boil down > > > to two dependencies not installing properly: psutil and setpproctitle, > > > both of which have c extensions that need to be built in the pip install > > > process for airflow. I have tried many combinations of installing in a > > > virtual python environment, installing as root, installing with both pip > > > and pip3. I keep running into these modules failing. > > > > > > I would greatly appreciate help at this point. The actual error depends > > > on how I run the install. Let's focus on psutil being installed in a > > > virtual environment I am using venv. > > > > > > Instead of looking at the airflow install, let's just focus on psutl. If > > > I run the following command we can troubleshoot as the temporary files > > > aren't cleaned up: > > > > > > pip3 install psutil --no-clean > > > > > > Then we receive an error message indicating that the following command > > > failed, which we can execute manually after the failure: > > > > > > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > > > tokenize; sys.argv[0] = > > > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > > > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > > > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > > > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > > > bdist_wheel -d /tmp/pip-wheel-9qo9x8zx > > > > > > I've attached the log below. If you read the end carefully you'll see > > > that the source code for psutil - specifically package init file > > > 'psutil/__init__.py' - is not being found. > > > > > > In various pip commands I've tried, the errors always seem to result in > > > some step not finding files for the package itself being installed. I am > > > very frustrated. Please help. > > > Mike > > > > > > Installing collected packages: psutil > > > Running setup.py install for psutil ... error > > > ERROR: Command errored out with exit status 1: > > > command: /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, > > > setuptools, tokenize; sys.argv[0] = > > > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > > > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > > > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > > > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > > > install --record /tmp/pip-record-kmdv6krv/install-record.txt > > > --single-version-externally-managed --compile --install-headers > > > /home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil > > > cwd: /tmp/pip-install-am5fhsq9/psutil/ > > > Complete output (47 lines): > > > running install > > > running build > > > running build_py > > > creating build > > > creating build/lib.linux-x86_64-3.6 > > > creating build/lib.linux-x86_64-3.6/psutil > > > copying psutil/__init__.py -> build/lib.linux-x86_64-3.6/psutil > > > copying psutil/_psposix.py -> build/lib.linux-x86_64-3.6/psutil > > > copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.6/psutil > > > copying psutil/_psaix.py -> build/lib.linux-x86_64-3.6/psutil > > > copying psutil/_common.py -> build/lib.linux-x86_64-3.6/psutil > > > copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.6/psutil > > > copying psutil/_compat.py -> build/lib.linux-x86_64-3.6/psutil > > > copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.6/psutil > > > copying psutil/_psosx.py -> build/lib.linux-x86_64-3.6/psutil > > > copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.6/psutil > > > creating build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/runner.py -> build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_system.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/__init__.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_posix.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_osx.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_process.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_bsd.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_unicode.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_memory_leaks.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_windows.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_contracts.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/__main__.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_misc.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_connections.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_aix.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_sunos.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > copying psutil/tests/test_linux.py -> > > > build/lib.linux-x86_64-3.6/psutil/tests > > > running build_ext > > > building 'psutil._psutil_linux' extension > > > creating build/temp.linux-x86_64-3.6 > > > creating build/temp.linux-x86_64-3.6/psutil > > > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > > > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > > > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > > > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > > > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > > > -I/usr/include/python3.6m -c psutil/_psutil_common.c -o > > > build/temp.linux-x86_64-3.6/psutil/_psutil_common.o > > > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > > > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > > > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > > > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > > > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > > > -I/usr/include/python3.6m -c psutil/_psutil_posix.c -o > > > build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o > > > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > > > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > > > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > > > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > > > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > > > -I/usr/include/python3.6m -c psutil/_psutil_linux.c -o > > > build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o > > > gcc -pthread -shared -Wl,-z,relro -g > > > build/temp.linux-x86_64-3.6/psutil/_psutil_common.o > > > build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o > > > build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o -L/usr/lib64 > > > -lpython3.6m -o > > > build/lib.linux-x86_64-3.6/psutil/_psutil_linux.cpython-36m-x86_64-linux-gnu.so > > > (http://psutil_linux.cpython-36m-x86_64-linux-gnu.so) > > > Unknown option: -- > > > usage: /usr/bin/ld [option] ... [-c cmd | -m mod | file | -] [arg] ... > > > Try `python -h' for more information. > > > collect2: error: ld returned 2 exit status > > > error: command 'gcc' failed with exit status 1 > > > ---------------------------------------- > > > ERROR: Command errored out with exit status 1: > > > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > > > tokenize; sys.argv[0] = > > > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > > > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > > > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > > > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > > > install --record /tmp/pip-record-kmdv6krv/install-record.txt > > > --single-version-externally-managed --compile --install-headers > > > /home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil Check the > > > logs for full command output. > > > (vairflow_dev) [bodi@fcetlappprd01 ~]$ > > > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > > > tokenize; sys.argv[0] = > > > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > > > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > > > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > > > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > > > bdist_wheel -d /tmp/pip-wheel-9qo9x8zx > > > running bdist_wheel > > > running build > > > running build_py > > > package init file 'psutil/__init__.py' not found (or not a regular file) > > > error: package directory 'psutil/tests' does not exist > > > (vairflow_dev) [bodi@fcetlappprd01 ~]$ > > > > > > > > >
