I have, yes. We are running rhel 7.7. It appeared non-trivial to go down that path so I hesitated. Would that direction be a general recommendation and would I want to choose a centos or rhel build? Or maybe they are one in the same. If you recommend docker then some pointing in to the best direction would be greatly appreciated thanks! Mike
On Fri, Jan 31, 2020, 6:33 AM Kamil Breguła <[email protected]> wrote: > Hello, > > Have you considered using Docker? > > Best regards, > Kamil > > On Fri, Jan 31, 2020 at 12:07 PM Michael Lutz <[email protected]> > wrote: > > > > Hi, I've been working on installing airflow for a3 days now on red hat > linux 7.7 and would greatly appreciate some help. My troubles boil down to > two dependencies not installing properly: psutil and setpproctitle, both of > which have c extensions that need to be built in the pip install process > for airflow. I have tried many combinations of installing in a virtual > python environment, installing as root, installing with both pip and pip3. > I keep running into these modules failing. > > > > I would greatly appreciate help at this point. The actual error depends > on how I run the install. Let's focus on psutil being installed in a > virtual environment I am using venv. > > > > Instead of looking at the airflow install, let's just focus on psutl. > If I run the following command we can troubleshoot as the temporary files > aren't cleaned up: > > > > pip3 install psutil --no-clean > > > > Then we receive an error message indicating that the following command > failed, which we can execute manually after the failure: > > > > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, > setuptools, tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > bdist_wheel -d /tmp/pip-wheel-9qo9x8zx > > > > I've attached the log below. If you read the end carefully you'll see > that the source code for psutil - specifically package init file > 'psutil/__init__.py' - is not being found. > > > > In various pip commands I've tried, the errors always seem to result in > some step not finding files for the package itself being installed. I am > very frustrated. Please help. > > Mike > > > > Installing collected packages: psutil > > Running setup.py install for psutil ... error > > ERROR: Command errored out with exit status 1: > > command: /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import > sys, setuptools, tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record /tmp/pip-record-kmdv6krv/install-record.txt > --single-version-externally-managed --compile --install-headers > /home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil > > cwd: /tmp/pip-install-am5fhsq9/psutil/ > > Complete output (47 lines): > > running install > > running build > > running build_py > > creating build > > creating build/lib.linux-x86_64-3.6 > > creating build/lib.linux-x86_64-3.6/psutil > > copying psutil/__init__.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_psposix.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_psaix.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_common.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_compat.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_psosx.py -> build/lib.linux-x86_64-3.6/psutil > > copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.6/psutil > > creating build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/runner.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_system.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/__init__.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_posix.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_osx.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_process.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_bsd.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_unicode.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_memory_leaks.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_windows.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_contracts.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/__main__.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_misc.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_connections.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_aix.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_sunos.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > copying psutil/tests/test_linux.py -> > build/lib.linux-x86_64-3.6/psutil/tests > > running build_ext > > building 'psutil._psutil_linux' extension > > creating build/temp.linux-x86_64-3.6 > > creating build/temp.linux-x86_64-3.6/psutil > > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > -I/usr/include/python3.6m -c psutil/_psutil_common.c -o > build/temp.linux-x86_64-3.6/psutil/_psutil_common.o > > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > -I/usr/include/python3.6m -c psutil/_psutil_posix.c -o > build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o > > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > -I/usr/include/python3.6m -c psutil/_psutil_linux.c -o > build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o > > gcc -pthread -shared -Wl,-z,relro -g > build/temp.linux-x86_64-3.6/psutil/_psutil_common.o > build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o > build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o -L/usr/lib64 > -lpython3.6m -o build/lib.linux-x86_64-3.6/psutil/_ > psutil_linux.cpython-36m-x86_64-linux-gnu.so > > Unknown option: -- > > usage: /usr/bin/ld [option] ... [-c cmd | -m mod | file | -] [arg] > ... > > Try `python -h' for more information. > > collect2: error: ld returned 2 exit status > > error: command 'gcc' failed with exit status 1 > > ---------------------------------------- > > ERROR: Command errored out with exit status 1: > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record /tmp/pip-record-kmdv6krv/install-record.txt > --single-version-externally-managed --compile --install-headers > /home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil Check the > logs for full command output. > > (vairflow_dev) [bodi@fcetlappprd01 ~]$ > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > bdist_wheel -d /tmp/pip-wheel-9qo9x8zx > > running bdist_wheel > > running build > > running build_py > > package init file 'psutil/__init__.py' not found (or not a regular file) > > error: package directory 'psutil/tests' does not exist > > (vairflow_dev) [bodi@fcetlappprd01 ~]$ > > > > >
