awesome I will give it a try. Thanks so much for your work. Mike On Fri, Jan 31, 2020 at 9:00 AM Ash Berlin-Taylor <[email protected]> wrote:
> Over at Astronomer we're just building out and testing an RHEL7 docker > image - it will work outside of the Astronomer platform fine): > https://github.com/astronomer/ap-airflow/tree/master/1.10.5/rhel7 (That > is 1.10.5 with a few things backported, once we've tested that image we > will update it to the latest version of Airflow. > > If you want to test it out you can pull astronomerinc/ap-airflow:1.10.5-rhel7 > and use that as a base image. (There are also more "fixed" tags such as > 1.10.5-rhel7-2840 ) > https://hub.docker.com/r/astronomerinc/ap-airflow/tags?page=1&name=rhel7 > for the complete list. > > HTH, > -Ash > On Jan 31 2020, at 11:40 am, Michael Lutz <[email protected]> > wrote: > > I have, yes. We are running rhel 7.7. It appeared non-trivial to go > down that path so I hesitated. Would that direction be a general > recommendation and would I want to choose a centos or rhel build? Or > maybe they are one in the same. If you recommend docker then some > pointing in to the best direction would be greatly appreciated thanks! > Mike > > On Fri, Jan 31, 2020, 6:33 AM Kamil Breguła <[email protected]> > wrote: > > Hello, > > Have you considered using Docker? > > Best regards, > Kamil > > On Fri, Jan 31, 2020 at 12:07 PM Michael Lutz <[email protected]> > wrote: > > > > Hi, I've been working on installing airflow for a3 days now on red hat > linux 7.7 and would greatly appreciate some help. My troubles boil down to > two dependencies not installing properly: psutil and setpproctitle, both of > which have c extensions that need to be built in the pip install process > for airflow. I have tried many combinations of installing in a virtual > python environment, installing as root, installing with both pip and pip3. > I keep running into these modules failing. > > > > I would greatly appreciate help at this point. The actual error depends > on how I run the install. Let's focus on psutil being installed in a > virtual environment I am using venv. > > > > Instead of looking at the airflow install, let's just focus on psutl. > If I run the following command we can troubleshoot as the temporary files > aren't cleaned up: > > > > pip3 install psutil --no-clean > > > > Then we receive an error message indicating that the following command > failed, which we can execute manually after the failure: > > > > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, > setuptools, tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > bdist_wheel -d /tmp/pip-wheel-9qo9x8zx > > > > I've attached the log below. If you read the end carefully you'll see > that the source code for psutil - specifically package init file > 'psutil/__init__.py' - is not being found. > > > > In various pip commands I've tried, the errors always seem to result in > some step not finding files for the package itself being installed. I am > very frustrated. Please help. > > Mike > > > > Installing collected packages: psutil > > Running setup.py install for psutil ... error > > ERROR: Command errored out with exit status 1: > > command: /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import > sys, setuptools, tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record /tmp/pip-record-kmdv6krv/install-record.txt > --single-version-externally-managed --compile --install-headers > /home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil > > cwd: /tmp/pip-install-am5fhsq9/psutil/ > > Complete output (47 lines): > > running install > > running build > > running build_py > > creating build > > creating build/lib.linux-x86_64-3.6 > > creating build/lib.linux-x86_64-3.6/psutil > > copying psutil/__init__.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_psposix.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_psaix.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_common.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_compat.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_psosx.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.6/psutil > > creating build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/runner.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_system.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/__init__.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_posix.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_osx.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_process.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_bsd.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_unicode.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_memory_leaks.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_windows.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_contracts.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/__main__.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_misc.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_connections.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_aix.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_sunos.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_linux.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > running build_ext > > building 'psutil._psutil_linux' extension > > creating build/temp.linux-x86_64-3.6 > > creating build/temp.linux-x86_64-3.6/psutil > > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > -I/usr/include/python3.6m -c psutil/_psutil_common.c -o > build/temp.linux-x86_64-3.6/psutil/_psutil_common.o > > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > -I/usr/include/python3.6m -c psutil/_psutil_posix.c -o > build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o > > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > -I/usr/include/python3.6m -c psutil/_psutil_linux.c -o > build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o > > gcc -pthread -shared -Wl,-z,relro -g > build/temp.linux-x86_64-3.6/psutil/_psutil_common.o > build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o > build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o -L/usr/lib64 > -lpython3.6m -o build/lib.linux-x86_64-3.6/psutil/_ > psutil_linux.cpython-36m-x86_64-linux-gnu.so > > Unknown option: -- > > usage: /usr/bin/ld [option] ... [-c cmd | -m mod | file | -] [arg] > ... > > Try `python -h' for more information. > > collect2: error: ld returned 2 exit status > > error: command 'gcc' failed with exit status 1 > > ---------------------------------------- > > ERROR: Command errored out with exit status 1: > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record /tmp/pip-record-kmdv6krv/install-record.txt > --single-version-externally-managed --compile --install-headers > /home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil Check the > logs for full command output. > > (vairflow_dev) [bodi@fcetlappprd01 ~]$ > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > bdist_wheel -d /tmp/pip-wheel-9qo9x8zx > > running bdist_wheel > > running build > > running build_py > > package init file 'psutil/__init__.py' not found (or not a regular file) > > error: package directory 'psutil/tests' does not exist > > (vairflow_dev) [bodi@fcetlappprd01 ~]$ > > > > > >
