Hello, Have you considered using Docker?
Best regards, Kamil On Fri, Jan 31, 2020 at 12:07 PM Michael Lutz <[email protected]> wrote: > > Hi, I've been working on installing airflow for a3 days now on red hat linux > 7.7 and would greatly appreciate some help. My troubles boil down to two > dependencies not installing properly: psutil and setpproctitle, both of which > have c extensions that need to be built in the pip install process for > airflow. I have tried many combinations of installing in a virtual python > environment, installing as root, installing with both pip and pip3. I keep > running into these modules failing. > > I would greatly appreciate help at this point. The actual error depends on > how I run the install. Let's focus on psutil being installed in a virtual > environment I am using venv. > > Instead of looking at the airflow install, let's just focus on psutl. If I > run the following command we can troubleshoot as the temporary files aren't > cleaned up: > > pip3 install psutil --no-clean > > Then we receive an error message indicating that the following command > failed, which we can execute manually after the failure: > > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > tokenize; sys.argv[0] = '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > bdist_wheel -d /tmp/pip-wheel-9qo9x8zx > > I've attached the log below. If you read the end carefully you'll see that > the source code for psutil - specifically package init file > 'psutil/__init__.py' - is not being found. > > In various pip commands I've tried, the errors always seem to result in some > step not finding files for the package itself being installed. I am very > frustrated. Please help. > Mike > > Installing collected packages: psutil > Running setup.py install for psutil ... error > ERROR: Command errored out with exit status 1: > command: /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, > setuptools, tokenize; sys.argv[0] = > '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record /tmp/pip-record-kmdv6krv/install-record.txt > --single-version-externally-managed --compile --install-headers > /home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil > cwd: /tmp/pip-install-am5fhsq9/psutil/ > Complete output (47 lines): > running install > running build > running build_py > creating build > creating build/lib.linux-x86_64-3.6 > creating build/lib.linux-x86_64-3.6/psutil > copying psutil/__init__.py -> build/lib.linux-x86_64-3.6/psutil > copying psutil/_psposix.py -> build/lib.linux-x86_64-3.6/psutil > copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.6/psutil > copying psutil/_psaix.py -> build/lib.linux-x86_64-3.6/psutil > copying psutil/_common.py -> build/lib.linux-x86_64-3.6/psutil > copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.6/psutil > copying psutil/_compat.py -> build/lib.linux-x86_64-3.6/psutil > copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.6/psutil > copying psutil/_psosx.py -> build/lib.linux-x86_64-3.6/psutil > copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.6/psutil > creating build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/runner.py -> build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_system.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/__init__.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_posix.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_osx.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_process.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_bsd.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_unicode.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_memory_leaks.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_windows.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_contracts.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/__main__.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_misc.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_connections.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_aix.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_sunos.py -> > build/lib.linux-x86_64-3.6/psutil/tests > copying psutil/tests/test_linux.py -> > build/lib.linux-x86_64-3.6/psutil/tests > running build_ext > building 'psutil._psutil_linux' extension > creating build/temp.linux-x86_64-3.6 > creating build/temp.linux-x86_64-3.6/psutil > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > -I/usr/include/python3.6m -c psutil/_psutil_common.c -o > build/temp.linux-x86_64-3.6/psutil/_psutil_common.o > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > -I/usr/include/python3.6m -c psutil/_psutil_posix.c -o > build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o > gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe > -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic > -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567 > -DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include > -I/usr/include/python3.6m -c psutil/_psutil_linux.c -o > build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o > gcc -pthread -shared -Wl,-z,relro -g > build/temp.linux-x86_64-3.6/psutil/_psutil_common.o > build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o > build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o -L/usr/lib64 -lpython3.6m > -o > build/lib.linux-x86_64-3.6/psutil/_psutil_linux.cpython-36m-x86_64-linux-gnu.so > Unknown option: -- > usage: /usr/bin/ld [option] ... [-c cmd | -m mod | file | -] [arg] ... > Try `python -h' for more information. > collect2: error: ld returned 2 exit status > error: command 'gcc' failed with exit status 1 > ---------------------------------------- > ERROR: Command errored out with exit status 1: > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > tokenize; sys.argv[0] = '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > install --record /tmp/pip-record-kmdv6krv/install-record.txt > --single-version-externally-managed --compile --install-headers > /home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil Check the logs > for full command output. > (vairflow_dev) [bodi@fcetlappprd01 ~]$ > /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools, > tokenize; sys.argv[0] = '"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"'; > __file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize, > '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', > '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' > bdist_wheel -d /tmp/pip-wheel-9qo9x8zx > running bdist_wheel > running build > running build_py > package init file 'psutil/__init__.py' not found (or not a regular file) > error: package directory 'psutil/tests' does not exist > (vairflow_dev) [bodi@fcetlappprd01 ~]$ > >
