Thank you Guillaume for your help, I am using : (running on AWS EMR-6.2) pip3 --version pip 9.0.3 from /usr/lib/python3.7/site-packages (python 3.7)
pip3 install scikit-learn Collecting scikit-learn Using cached https://files.pythonhosted.org/packages/f4/7b/d415b0c89babf23dcd8ee631015f043e2d76795edd9c7359d6e63257464b/scikit-learn-0.24.1.tar.gz Requirement already satisfied: numpy>=1.13.3 in /usr/local/lib64/python3.7/site-packages (from scikit-learn) Collecting scipy>=0.19.1 (from scikit-learn) Using cached https://files.pythonhosted.org/packages/58/9d/8296d8211318d690119eba6d293b7a149c1c51c945342dd4c3816f79e1ba/scipy-1.6.0-cp37-cp37m-manylinux1_x86_64.whl Requirement already satisfied: joblib>=0.11 in /usr/local/lib64/python3.7/site-packages (from scikit-learn) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/site-packages (from scikit-learn) Installing collected packages: scipy, scikit-learn Running setup.py install for scikit-learn ... error Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/mnt/tmp/pip-build-93pagltp/scikit-learn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-0ulalx36-record/install-record.txt --single-version-externally-managed --compile: Partial import of sklearn during the build process. Traceback (most recent call last): File "/mnt/tmp/pip-build-93pagltp/scikit-learn/sklearn/_build_utils/__init__.py", line 27, in _check_cython_version import Cython ModuleNotFoundError: No module named 'Cython' Upgrading pip to 20.3.3 : sudo pip3 install --upgrade pip sudo ln -s /usr/local/bin/pip3 /usr/bin/pip3 pip3 --version pip 20.3.3 from /usr/local/lib/python3.7/site-packages/pip (python 3.7) let me install from the whl file : pip3 install scikit-learn Collecting scikit-learn Downloading scikit_learn-0.24.1-cp37-cp37m-manylinux2010_x86_64.whl (22.3 MB) However, using the API sc.install_pypi_package("scikit-learn") still uses the tar file instead of the whl file (even after the pip upgrade). Collecting scikit-learn Using cached https://files.pythonhosted.org/packages/f4/7b/d415b0c89babf23dcd8ee631015f043e2d76795edd9c7359d6e63257464b/scikit-learn-0.24.1.tar.gz Thanks for your help, Cheers, Bertrand Le ven. 22 janv. 2021 à 04:13, Guillaume Lemaître <g.lemaitr...@gmail.com> a écrit : > @Bertrand Could you tell us which version of `pip` to you use (you need > pip >= 19.0 for manylinux2010 and pip >= 19.3 for manylinux2014) > > On Fri, 22 Jan 2021 at 09:49, Guillaume Lemaître <g.lemaitr...@gmail.com> > wrote: > >> We might experience an issue with PyPI not selecting the manylinux2010 >> wheel: https://github.com/scikit-learn/scikit-learn/issues/19233 >> We have to check but we will probably shortly upload manylinux1 wheels >> that should resolve the issue. >> >> I am curious if fetching the wheel by hand and installing via `pip` would >> be a workaround (not practical for automated usage thought). >> >> On Thu, 21 Jan 2021 at 00:34, The Helmbolds via scikit-learn < >> scikit-learn@python.org> wrote: >> >>> Use the Anaconda Python installation. >>> >>> "You won't find the right answers if you don't ask the right questions!" >>> (Robert Helmbold, 2013) >>> >>> >>> On Wednesday, January 20, 2021, 04:16:15 PM MST, Guillaume Lemaître < >>> g.lemaitr...@gmail.com> wrote: >>> >>> >>> Basically it get the tar with the source and recompile instead of using >>> the wheel. Could you force an install from PyPI without using the cached >>> file. >>> >>> We pushed wheels yesterday for 0.24.1 as well so it should not get the >>> 0.24.0 version. >>> >>> For 0.23.2, you can see that it used the wheel (.whl). >>> >>> Sent from my phone - sorry to be brief and potential misspell. >>> *From:* bertrand25...@gmail.com >>> *Sent:* 20 January 2021 23:21 >>> *To:* scikit-learn@python.org >>> *Reply to:* scikit-learn@python.org >>> *Subject:* [scikit-learn] scikit-learn 0.24 installation fails with >>> ModuleNotFoundError: No module named 'scipy' >>> >>> To whom it may concern, >>> >>> I am trying to install scikit-learn in a PySpark job using the >>> install_pypi_package PySpark API but the install fails with : >>> >>> sc.install_pypi_package("scikit-learn") >>> >>> Collecting scikit-learn >>> Using cached >>> https://files.pythonhosted.org/packages/db/e2/9c0bde5f81394b627f623557690536b12017b84988a4a1f98ec826edab9e/scikit-learn-0.24.0.tar.gz >>> Requirement already satisfied: numpy>=1.13.3 in >>> /usr/local/lib64/python3.7/site-packages (from scikit-learn) >>> Collecting scipy>=0.19.1 (from scikit-learn) >>> Using cached >>> https://files.pythonhosted.org/packages/58/9d/8296d8211318d690119eba6d293b7a149c1c51c945342dd4c3816f79e1ba/scipy-1.6.0-cp37-cp37m-manylinux1_x86_64.whl >>> Requirement already satisfied: joblib>=0.11 in >>> /usr/local/lib64/python3.7/site-packages (from scikit-learn) >>> Collecting threadpoolctl>=2.0.0 (from scikit-learn) >>> Using cached >>> https://files.pythonhosted.org/packages/f7/12/ec3f2e203afa394a149911729357aa48affc59c20e2c1c8297a60f33f133/threadpoolctl-2.1.0-py3-none-any.whl >>> Building wheels for collected packages: scikit-learn >>> Running setup.py bdist_wheelfor scikit-learn: started >>> Running setup.py bdist_wheelfor scikit-learn: finished with status 'error' >>> Complete output from command /tmp/1611000009300-0/bin/python -u -c >>> "import setuptools, >>> tokenize;__file__='/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py >>> ';f=getattr(tokenize, 'open', open)(__file__);code=f.read >>> ().replace('\r\n', '\n');f.close ();exec(compile(code, __file__, 'exec'))" >>> bdist_wheel -d /tmp/tmpry3gf9r0pip-wheel- --python-tag cp37: >>> Partial import of sklearn during the build process. >>> Traceback (most recent call last): >>> File "/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py ", line 201, in >>> check_package_status >>> module = importlib.import_module(package) >>> File "/tmp/1611000009300-0/lib64/python3.7/importlib/__init__.py", line >>> 127, in import_module >>> return _bootstrap._gcd_import(name[level:], package, level) >>> File "<frozen importlib._bootstrap>", line 1006, in _gcd_import >>> File "<frozen importlib._bootstrap>", line 983, in _find_and_load >>> File "<frozen importlib._bootstrap>", line 965, in >>> _find_and_load_unlocked >>> ModuleNotFoundError: No module named 'scipy' >>> Traceback (most recent call last): >>> File "<string>", line 1, in <module> >>> File "/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py ", line 306, in >>> <module> >>> setup_package() >>> File "/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py ", line 294, in >>> setup_package >>> check_package_status('scipy', min_deps.SCIPY_MIN_VERSION) >>> File "/mnt/tmp/pip-build-phc6p6gl/scikit-learn/setup.py ", line 227, in >>> check_package_status >>> .format(package, req_str, instructions)) >>> ImportError: scipy is not installed. >>> scikit-learn requires scipy >= 0.19.1. >>> >>> I do not encounter this error with scikit-learn 0.23.2 : >>> >>> sc.install_pypi_package("scikit-learn==0.23.2") >>> >>> Collecting scikit-learn==0.23.2 >>> Using cached >>> https://files.pythonhosted.org/packages/f4/cb/64623369f348e9bfb29ff898a57ac7c91ed4921f228e9726546614d63ccb/scikit_learn-0.23.2-cp37-cp37m-manylinux1_x86_64.whl >>> Requirement already satisfied: scipy>=0.19.1 in >>> /mnt/tmp/1611000009300-0/lib/python3.7/site-packages (from >>> scikit-learn==0.23.2) >>> Requirement already satisfied: numpy>=1.13.3 in >>> /usr/local/lib64/python3.7/site-packages (from scikit-learn==0.23.2) >>> Requirement already satisfied: joblib>=0.11 in >>> /usr/local/lib64/python3.7/site-packages (from scikit-learn==0.23.2) >>> Requirement already satisfied: threadpoolctl>=2.0.0 in >>> /mnt/tmp/1611000009300-0/lib/python3.7/site-packages (from >>> scikit-learn==0.23.2) >>> Installing collected packages: scikit-learn >>> Successfully installed scikit-learn-0.23.2 >>> >>> >>> Could you please help me understand why the scikit-learn 0.24 >>> installation fails ? >>> >>> Thank you for your help, >>> >>> Bertrand >>> _______________________________________________ >>> scikit-learn mailing list >>> scikit-learn@python.org >>> https://mail.python.org/mailman/listinfo/scikit-learn >>> _______________________________________________ >>> scikit-learn mailing list >>> scikit-learn@python.org >>> https://mail.python.org/mailman/listinfo/scikit-learn >>> >> >> >> -- >> Guillaume Lemaitre >> Scikit-learn @ Inria Foundation >> https://glemaitre.github.io/ >> > > > -- > Guillaume Lemaitre > Scikit-learn @ Inria Foundation > https://glemaitre.github.io/ > _______________________________________________ > scikit-learn mailing list > scikit-learn@python.org > https://mail.python.org/mailman/listinfo/scikit-learn >
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
