Hi Team,
When I tried to build master using "mvn clean test -Drat.skip=true
-DskipITs", I got unit test failures in rsc module. SPARK_HOME is being set
to "dev/spark" which only contains spark-submit but none of the spark
related jar file. So I am not sure how spark-submit is supposed to work. Is
it expected to load spark jars from the classpath since spark is a
dependency to compile livy? In any case the below 3 tests failed.
Tests in error:
TestSparkClient.testConnectToRunningContext:338->runTest:575->runTest:588
» Execution
TestSparkClient.testImpersonation:321->runTest:575->runTest:588 »
Execution ja...
TestSparkClient.testRemoteClient:213->runTest:575->runTest:588 »
Execution jav...
I tried to point SPARK_HOME to a downloaded spark tar ball (spark-2.3) and
the below tests passed.....but another test failed in the same rsc module
Failed tests:
TestSparkClient.testJobSubmission:88->runTest:575->runTest:590
Wanted but not invoked:
listener.onJobStarted(
org.apache.livy.rsc.JobHandleImpl@6e22d6bf
);
-> at org.apache.livy.rsc.TestSparkClient$1.call(TestSparkClient.java:101)
However, there were other interactions with this mock:
-> at
org.apache.livy.client.common.AbstractJobHandle.fireStateChange(AbstractJobHandle.java:97)
Any idea what I could be missing?
-Rahul
