GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/19524
[SPARK-22302][INFRA] Remove manual backports for subprocess and print explicit message for < Python 2.7 ## What changes were proposed in this pull request? Seems there is a mistake from missing import for `subprocess.call` which should be used for Python 2.6 backports for some missing functions in `subprocess`. Reproduction is: ``` cd dev && python2.6 ``` ``` >>> from sparktestsupport import shellutils >>> shellutils.subprocess_check_call("ls") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "sparktestsupport/shellutils.py", line 46, in subprocess_check_call retcode = call(*popenargs, **kwargs) NameError: global name 'call' is not defined ``` For Jenkins logs, please see https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3950/console Since we dropped the Python 2.6.x support, looks better we remove those workarounds and print out explicit error messages in order to duplicate the efforts to find out the root causes for such cases. ## How was this patch tested? Manually tested: ``` cd dev && python2.6 ``` ``` >>> from sparktestsupport import shellutils [error] Python versions prior to 2.7 are not supported. ``` ``` cd dev && python2.7 ``` ``` >>> from sparktestsupport import shellutils >>> shellutils.subprocess_check_output("ls") 'README.md\nappveyor-guide.md\nappveyor-install-dependencies.ps1\nchange-scala-version.sh\ncheck-license\ncheckstyle-suppressions.xml\ncheckstyle.xml\ncreate-release\ndeps\ngithub_jira_sync.py\ngithub_jira_sync.pyc\nlint-java\nlint-python\nlint-r\nlint-r-report.log\nlint-r.R\nlint-scala\nmake-distribution.sh\nmerge_spark_pr.py\nmerge_spark_pr.pyc\nmerge_spark_pr_jira.py\nmerge_spark_pr_jira.pyc\nmima\npep8-1.7.0.py\npep8-1.7.0.pyc\npip-sanity-check.py\npip-sanity-check.pyc\nrequirements.txt\nrun-pip-tests\nrun-tests\nrun-tests-jenkins\nrun-tests-jenkins.py\nrun-tests-jenkins.pyc\nrun-tests.py\nrun-tests.pyc\nscalastyle\nsparktestsupport\ntest-dependencies.sh\ntests\ntox.ini\n' >>> shellutils.subprocess_check_call("ls") README.md checkstyle-suppressions.xml github_jira_sync.pyc lint-r.R merge_spark_pr_jira.py pip-sanity-check.py run-tests-jenkins scalastyle appveyor-guide.md checkstyle.xml lint-java lint-scala merge_spark_pr_jira.pyc pip-sanity-check.pyc run-tests-jenkins.py sparktestsupport appveyor-install-dependencies.ps1 create-release lint-python make-distribution.sh mima requirements.txt run-tests-jenkins.pyc test-dependencies.sh change-scala-version.sh deps lint-r merge_spark_pr.py pep8-1.7.0.py run-pip-tests run-tests.py tests check-license github_jira_sync.py lint-r-report.log merge_spark_pr.pyc pep8-1.7.0.pyc run-tests run-tests.pyc tox.ini 0 ``` ``` cd dev && python3.6 ``` ``` >>> from sparktestsupport import shellutils >>> shellutils.subprocess_check_output("ls") b'README.md\nappveyor-guide.md\nappveyor-install-dependencies.ps1\nchange-scala-version.sh\ncheck-license\ncheckstyle-suppressions.xml\ncheckstyle.xml\ncreate-release\ndeps\ngithub_jira_sync.py\ngithub_jira_sync.pyc\nlint-java\nlint-python\nlint-r\nlint-r-report.log\nlint-r.R\nlint-scala\nmake-distribution.sh\nmerge_spark_pr.py\nmerge_spark_pr.pyc\nmerge_spark_pr_jira.py\nmerge_spark_pr_jira.pyc\nmima\npep8-1.7.0.py\npep8-1.7.0.pyc\npip-sanity-check.py\npip-sanity-check.pyc\nrequirements.txt\nrun-pip-tests\nrun-tests\nrun-tests-jenkins\nrun-tests-jenkins.py\nrun-tests-jenkins.pyc\nrun-tests.py\nrun-tests.pyc\nscalastyle\nsparktestsupport\ntest-dependencies.sh\ntests\ntox.ini\n' >>> shellutils.subprocess_check_call("ls") README.md checkstyle-suppressions.xml github_jira_sync.pyc lint-r.R merge_spark_pr_jira.py pip-sanity-check.py run-tests-jenkins scalastyle appveyor-guide.md checkstyle.xml lint-java lint-scala merge_spark_pr_jira.pyc pip-sanity-check.pyc run-tests-jenkins.py sparktestsupport appveyor-install-dependencies.ps1 create-release lint-python make-distribution.sh mima requirements.txt run-tests-jenkins.pyc test-dependencies.sh change-scala-version.sh deps lint-r merge_spark_pr.py pep8-1.7.0.py run-pip-tests run-tests.py tests check-license github_jira_sync.py lint-r-report.log merge_spark_pr.pyc pep8-1.7.0.pyc run-tests run-tests.pyc tox.ini 0 You can merge this pull request into a Git repository by running: $ git pull https://github.com/HyukjinKwon/spark SPARK-22302 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/19524.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #19524 ---- ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org