[ 
https://issues.apache.org/jira/browse/ARROW-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123479#comment-17123479
 ] 

Joris Van den Bossche commented on ARROW-8983:
----------------------------------------------

I think this is due to numpy starting to have a pyproject.toml file since 1.18 
(https://github.com/numpy/numpy/pull/14053) 
And apparently, when a package includes a pyproject.toml, pip will create a 
build environment, just to get the metadata (and in case of numpy, this means 
creating an environment with setuptools, wheel and cython packages installed). 
And this is what takes some more time.

> [Python] Downloading sources of pyarrow and its requirements from pypi takes 
> several minutes starting from 0.16.0
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: ARROW-8983
>                 URL: https://issues.apache.org/jira/browse/ARROW-8983
>             Project: Apache Arrow
>          Issue Type: Bug
>    Affects Versions: 0.16.0, 0.17.0, 0.17.1
>            Reporter: Valentyn Tymofieiev
>            Priority: Minor
>
> It appears that 
>           python -m pip download --dest /tmp pyarrow==0.17.1 --no-binary :all:
> takes several minutes to execute. 
> There seems to be an increase in runtime starting from 0.16.0: on Python 2 
>      python -m pip download --dest /tmp pyarrow==0.15.1 --no-binary :all:
> appears to be somewhat faster, but the same command is still slow on Py3.
> The command is stuck for a while with "Installing build dependencies ... ", 
> and increased CPU usage.
> The intent of this command is to download source tarball for a package and 
> its dependencies.
> Some investigation was started on the mailing list: 
> https://lists.apache.org/thread.html/r9baa48a9d1517834c285f0f238f29fcf54405cb7cf1e681314239d7f%40%3Cdev.arrow.apache.org%3E



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to