This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-2.4
in repository

The following commit(s) were added to refs/heads/branch-2.4 by this push:
     new 223b9fb  [SPARK-31231][BUILD] Explicitly setuptools version as 46.0.0 
in pip package test
223b9fb is described below

commit 223b9fb1eadeba0e05b1a300512c31c4f99f41e8
Author: HyukjinKwon <>
AuthorDate: Tue Mar 24 17:59:43 2020 +0900

    [SPARK-31231][BUILD] Explicitly setuptools version as 46.0.0 in pip package 
    ### What changes were proposed in this pull request?
    For a bit of background,
    PIP packaging test started to fail (see [this 
 as of  setuptools 46.1.0 release. In, they decided to don't keep the 
modes in `package_data`.
    In PySpark pip installation, we keep the executable scripts in 
 and expose their symbolic links as executable scripts.
    So, the symbolic links (or copied scripts) executes the scripts copied from 
`package_data`, which doesn't have the executable permission in its mode:
    /tmp/tmp.UmkEGNFdKF/3.6/bin/spark-submit: line 27: 
Permission denied
    /tmp/tmp.UmkEGNFdKF/3.6/bin/spark-submit: line 27: exec: 
cannot execute: Permission denied
    The current issue is being tracked at
    For what this PR proposes:
    It sets the upper bound in PR builder for now to unblock other PRs.  _This 
PR does not solve the issue yet. I will make a fix after monitoring
    ### Why are the changes needed?
    It currently affects users who uses the latest setuptools. So, _users seem 
unable to use PySpark with the latest setuptools._ See also
    ### Does this PR introduce any user-facing change?
    It makes CI pass for now. No user-facing change yet.
    ### How was this patch tested?
    Jenkins will test.
    Closes #27995 from HyukjinKwon/investigate-pip-packaging.
    Authored-by: HyukjinKwon <>
    Signed-off-by: HyukjinKwon <>
    (cherry picked from commit c181c45f863ba55e15ab9b41f635ffbddad9bac0)
    Signed-off-by: HyukjinKwon <>
 dev/run-pip-tests | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/dev/run-pip-tests b/dev/run-pip-tests
index 60cf4d8..f9cd94d 100755
--- a/dev/run-pip-tests
+++ b/dev/run-pip-tests
@@ -81,7 +81,7 @@ for python in "${PYTHON_EXECS[@]}"; do
     rm -rf "$VIRTUALENV_PATH"
     if [ -n "$USE_CONDA" ]; then
-      conda create -y -p "$VIRTUALENV_PATH" python=$python numpy pandas pip 
+      conda create -y -p "$VIRTUALENV_PATH" python=$python numpy pandas pip 
       source activate "$VIRTUALENV_PATH"
       mkdir -p "$VIRTUALENV_PATH"

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to