Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/4961#issuecomment-78202972
Hi @davies @tdas , I added the Python Kafka API unit test, it works very
well in my local test, but seems always failed in Jenkins test with error like:
```
Traceback (most recent call last):
File "pyspark/streaming/tests.py", line 568, in setUp
.loadClass("org.apache.spark.streaming.kafka.KafkaTestUtils")
File
"/home/jenkins/workspace/SparkPullRequestBuilder/python/lib/py4j-0.8.2.1-src.zip/py4j/java_gateway.py",
line 538, in __call__
self.target_id, self.name)
File
"/home/jenkins/workspace/SparkPullRequestBuilder/python/lib/py4j-0.8.2.1-src.zip/py4j/protocol.py",
line 300, in get_return_value
format(target_id, '.', name), value)
Py4JJavaError: An error occurred while calling o4409.loadClass.
: java.lang.ClassNotFoundException:
org.apache.spark.streaming.kafka.KafkaTestUtils
```
I'm sure I added the kafka assembly jar with `--jars`, as you can see I
dump the log into console:
```
Additional argument: --jars
/home/jenkins/workspace/SparkPullRequestBuilder/external/kafka-assembly/target/scala-2.10/spark-streaming-kafka-assembly_2.10-1.3.0-SNAPSHOT.jar
```
I'm not sure is there any environment difference or something should be
taken care when testing in Jenkins, would you please give me some hints? Thanks
a lot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
