Hi, I've been working on installing airflow for a3 days now on red hat
linux 7.7 and would greatly appreciate some help. My troubles boil down to
two dependencies not installing properly: psutil and setpproctitle, both of
which have c extensions that need to be built in the pip install process
for airflow. I have tried many combinations of installing in a virtual
python environment, installing as root, installing with both pip and pip3.
I keep running into these modules failing.
I would greatly appreciate help at this point. The actual error depends on
how I run the install. Let's focus on psutil being installed in a virtual
environment I am using venv.
Instead of looking at the airflow install, let's just focus on psutl. If I
run the following command we can troubleshoot as the temporary files aren't
cleaned up:
pip3 install psutil --no-clean
Then we receive an error message indicating that the following command
failed, which we can execute manually after the failure:
/home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools,
tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';
__file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
bdist_wheel -d /tmp/pip-wheel-9qo9x8zx
I've attached the log below. If you read the end carefully you'll see that
the source code for psutil - specifically package init file
'psutil/__init__.py' - is not being found.
In various pip commands I've tried, the errors always seem to result in
some step not finding files for the package itself being installed. I am
very frustrated. Please help.
Mike
Installing collected packages: psutil
Running setup.py install for psutil ... error
ERROR: Command errored out with exit status 1:
command: /home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys,
setuptools, tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';
__file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
install --record /tmp/pip-record-kmdv6krv/install-record.txt
--single-version-externally-managed --compile --install-headers
/home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil
cwd: /tmp/pip-install-am5fhsq9/psutil/
Complete output (47 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/psutil
copying psutil/__init__.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_psposix.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_psaix.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_common.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_compat.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_psosx.py -> build/lib.linux-x86_64-3.6/psutil
copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.6/psutil
creating build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/runner.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_system.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/__init__.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_posix.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_osx.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_process.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_bsd.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_unicode.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_memory_leaks.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_windows.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_contracts.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/__main__.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_misc.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_connections.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_aix.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_sunos.py ->
build/lib.linux-x86_64-3.6/psutil/tests
copying psutil/tests/test_linux.py ->
build/lib.linux-x86_64-3.6/psutil/tests
running build_ext
building 'psutil._psutil_linux' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/psutil
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic
-D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567
-DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include
-I/usr/include/python3.6m -c psutil/_psutil_common.c -o
build/temp.linux-x86_64-3.6/psutil/_psutil_common.o
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic
-D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567
-DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include
-I/usr/include/python3.6m -c psutil/_psutil_posix.c -o
build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic
-D_GNU_SOURCE -fPIC -fwrapv -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=567
-DPSUTIL_LINUX=1 -I/home/bodi/py_envs/vairflow_dev/include
-I/usr/include/python3.6m -c psutil/_psutil_linux.c -o
build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o
gcc -pthread -shared -Wl,-z,relro -g
build/temp.linux-x86_64-3.6/psutil/_psutil_common.o
build/temp.linux-x86_64-3.6/psutil/_psutil_posix.o
build/temp.linux-x86_64-3.6/psutil/_psutil_linux.o -L/usr/lib64
-lpython3.6m -o build/lib.linux-x86_64-3.6/psutil/_
psutil_linux.cpython-36m-x86_64-linux-gnu.so
Unknown option: --
usage: /usr/bin/ld [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.
collect2: error: ld returned 2 exit status
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1:
/home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools,
tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';
__file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
install --record /tmp/pip-record-kmdv6krv/install-record.txt
--single-version-externally-managed --compile --install-headers
/home/bodi/py_envs/vairflow_dev/include/site/python3.6/psutil Check the
logs for full command output.
(vairflow_dev) [bodi@fcetlappprd01 ~]$
/home/bodi/py_envs/vairflow_dev/bin/python -u -c 'import sys, setuptools,
tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';
__file__='"'"'/tmp/pip-install-am5fhsq9/psutil/setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))'
bdist_wheel -d /tmp/pip-wheel-9qo9x8zx
running bdist_wheel
running build
running build_py
package init file 'psutil/__init__.py' not found (or not a regular file)
error: package directory 'psutil/tests' does not exist
(vairflow_dev) [bodi@fcetlappprd01 ~]$
