(for the wider dev audience) there's an apache issue open for this, which i have commented on.
https://issues.apache.org/jira/browse/SPARK-21573 i have a workaround and will be getting this sorted asap. On Fri, Jul 28, 2017 at 7:27 PM, Hyukjin Kwon <gurwls...@gmail.com> wrote: > Or maybe in https://github.com/apache/spark/blob/master/dev/run-tests#L23 > > > On 29 Jul 2017 11:16 am, "Hyukjin Kwon" <gurwls...@gmail.com> wrote: > > I am sorry for saying just based on my wild guess because I have no way to > check and take a look into Jenkins but I think we might have to set the > explicit Python version in > https://github.com/apache/spark/blob/master/dev/run-tests-jenkins#L29 > > I guess we set the explicit Python version for running unit tests but it > looks the script itself is dependent python executable in the path. > > > > On 29 Jul 2017 7:39 am, "Dong Joon Hyun" <dh...@hortonworks.com> wrote: > > I saw that error in the latest branch-2.1 build failure, too. > > > > https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-2.1-test-sbt-hadoop-2.7/579/console > > > > But, the code was written in Jan 2016. Didn’t we run it on Python 2.6 > without any problem? > > > > ee74498de37 (Josh Rosen 2016-01-26 14:20:11 -0800 124) {m: > set(m.dependencies).intersection(modules_to_test) for m in modules_to_test}, > sort=True) > > > > Bests, > > Dongjoon. > > > > From: Hyukjin Kwon <gurwls...@gmail.com> > Date: Friday, July 28, 2017 at 7:06 AM > To: Sean Owen <so...@cloudera.com> > Cc: dev <dev@spark.apache.org> > Subject: Re: Tests failing with run-tests.py SyntaxError > > > > Yes, that's my guess just given information here without a close look. > > > > On 28 Jul 2017 11:03 pm, "Sean Owen" <so...@cloudera.com> wrote: > > I see, does that suggest that a machine has 2.6, when it should use 2.7? > > > > On Fri, Jul 28, 2017 at 2:58 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: > > That looks appearently due to dict comprehension which is, IIRC, not allowed > in Python 2.6.x. I checked the release note for sure before > -https://issues.apache.org/jira/browse/SPARK-20149 > > > > On 28 Jul 2017 9:56 pm, "Sean Owen" <so...@cloudera.com> wrote: > > File "./dev/run-tests.py", line 124 > > {m: set(m.dependencies).intersection(modules_to_test) for m in > modules_to_test}, sort=True) > > ^ > > SyntaxError: invalid syntax > > > It seems like tests are failing intermittently with this type of error, > which leads me to guess there's some difference in the Python interpreter on > one or more machines but not all (?) > > > Does the error suggest anything to anyone who knows Python better than I? > > > > The line has been around for a year so I don't think it's the script per se. > > > > > --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
