I saw that error in the latest branch-2.1 build failure, too.
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-2.1-test-sbt-hadoop-2.7/579/console
But, the code was written in Jan 2016. Didn’t we run it on Python 2.6 without
any problem?
ee74498de37 (Josh Rosen 2016-01-26 14:20:11 -0800 124) {m:
set(m.dependencies).intersection(modules_to_test) for m in modules_to_test},
sort=True)
Bests,
Dongjoon.
From: Hyukjin Kwon <gurwls...@gmail.com>
Date: Friday, July 28, 2017 at 7:06 AM
To: Sean Owen <so...@cloudera.com>
Cc: dev <dev@spark.apache.org>
Subject: Re: Tests failing with run-tests.py SyntaxError
Yes, that's my guess just given information here without a close look.
On 28 Jul 2017 11:03 pm, "Sean Owen"
<so...@cloudera.com<mailto:so...@cloudera.com>> wrote:
I see, does that suggest that a machine has 2.6, when it should use 2.7?
On Fri, Jul 28, 2017 at 2:58 PM Hyukjin Kwon
<gurwls...@gmail.com<mailto:gurwls...@gmail.com>> wrote:
That looks appearently due to dict comprehension which is, IIRC, not allowed in
Python 2.6.x. I checked the release note for sure before
-https://issues.apache.org/jira/browse/SPARK-20149
On 28 Jul 2017 9:56 pm, "Sean Owen"
<so...@cloudera.com<mailto:so...@cloudera.com>> wrote:
File "./dev/run-tests.py", line 124
{m: set(m.dependencies).intersection(modules_to_test) for m in
modules_to_test}, sort=True)
^
SyntaxError: invalid syntax
It seems like tests are failing intermittently with this type of error, which
leads me to guess there's some difference in the Python interpreter on one or
more machines but not all (?)
Does the error suggest anything to anyone who knows Python better than I?
The line has been around for a year so I don't think it's the script per se.
