Re: python/run-tests fails at spark master branch

2015-04-21 Thread Saisai Shao
Hi Hrishikesh, Seems the behavior of Kafka-assembly is a little different when using Maven to sbt. The assembly jar name and location is different while using `mvn package`. This is a actually bug, I'm fixing this now. Thanks Jerry 2015-04-22 13:37 GMT+08:00 Hrishikesh Subramonian < hrishikesh.

Re: python/run-tests fails at spark master branch

2015-04-21 Thread Hrishikesh Subramonian
Hi, The /python/run-tests/ executes successfully after I ran /'build/sbt assembly/' command. But the tests fail if I run it after /'mvn -Dskiptests clean package'/ command. Why does it run in /sbt assembly/ and not in/mvn package/? -- Hrishikesh On Wednesday 22 April 2015 07:38 AM, Saisai S

Re: python/run-tests fails at spark master branch

2015-04-21 Thread Saisai Shao
Hi Hrishikesh, Now we add Kafka unit test for python which relies on Kafka assembly jar, so you need to run `sbt assembly` or mvn `package` at first to get an assemble jar. 2015-04-22 1:15 GMT+08:00 Marcelo Vanzin : > On Tue, Apr 21, 2015 at 1:30 AM, Hrishikesh Subramonian > wrote: > > > Run

Re: python/run-tests fails at spark master branch

2015-04-21 Thread Marcelo Vanzin
On Tue, Apr 21, 2015 at 1:30 AM, Hrishikesh Subramonian wrote: > Run streaming tests ... > Failed to find Spark Streaming Kafka assembly jar in > /home/xyz/spark/external/kafka-assembly > You need to build Spark with 'build/sbt assembly/assembly > streaming-kafka-assembly/assembly' or 'build/mvn