[
https://issues.apache.org/jira/browse/SPARK-21573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109881#comment-16109881
]
shane knapp commented on SPARK-21573:
-------------------------------------
ok, we pushed that PR. it should fix things.
i also found the root cause: if a worker goes down, and the master doesn't,
when that worker reconnects to jenkins the environment variables set in the
worker config (in our case, PATH) disappear.
i tested this w/a quick build that just echoed $PATH (which showed the default
system path). then i disconnected and reconnected that worker. after running
the same build on that worker, PATH was as it should be in the config.
anyways, this PR should definitely fix things as well as protect us from this
happening again.
> Tests failing with run-tests.py SyntaxError occasionally in Jenkins
> -------------------------------------------------------------------
>
> Key: SPARK-21573
> URL: https://issues.apache.org/jira/browse/SPARK-21573
> Project: Spark
> Issue Type: Test
> Components: Tests
> Affects Versions: 2.3.0
> Reporter: Hyukjin Kwon
> Priority: Minor
>
> It looks default {{python}} in the path at few places such as
> {{./dev/run-tests}} use Python 2.6 in Jenkins and it fails to execute
> {{run-tests.py}}:
> {code}
> python2.6 run-tests.py
> File "run-tests.py", line 124
> {m: set(m.dependencies).intersection(modules_to_test) for m in
> modules_to_test}, sort=True)
> ^
> SyntaxError: invalid syntax
> {code}
> It looks there are quite some places to fix to support Python 2.6 in
> {{run-tests.py}} and related Python scripts.
> We might just try to set Python 2.7 in few other scripts running this if
> available.
> Please also see
> http://apache-spark-developers-list.1001551.n3.nabble.com/Tests-failing-with-run-tests-py-SyntaxError-td22030.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
