damccorm opened a new issue, #21092: URL: https://github.com/apache/beam/issues/21092
This was reported on the mailing list. \--\-- Using spark downloaded from below link, [https://www.apache.org/dyn/closer.lua/spark/spark-3.1.2/spark-3.1.2-bin-hadoop3.2.tgz](https://www.apache.org/dyn/closer.lua/spark/spark-3.1.2/spark-3.1.2-bin-hadoop3.2.tgz) I get below error when submitting a pipeline. Full error is on [https://gist.github.com/yuwtennis/7b0c1dc0dcf98297af1e3179852ca693](https://gist.github.com/yuwtennis/7b0c1dc0dcf98297af1e3179852ca693). \--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\-- 21/08/16 01:10:26 WARN TransportChannelHandler: Exception in connection from /[192.168.11.2:35601](http://192.168.11.2:35601/) java.io.InvalidClassException: scala.collection.mutable.WrappedArray$ofRef; local class incompatible: stream classdesc serialVersionUID = 3456489343829468865, local class serialVersionUID = 1028182004549731694 at java.base/java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:689) ... \--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\--\-- SDK Harness and Job service are deployed as below. 1. SDK Harness sudo docker run \--net=host apache/beam_spark3_job_server:2.31.0 \--spark-master-url=spark://localhost:7077 \--clean-artifacts-per-job true 2. Job service sudo docker run \--net=host apache/beam_python3.8_sdk:2.31.0 \--worker_pool * apache/beam_spark_job_server:2.31.0 for spark 2.4.8 3. SDK client code [https://gist.github.com/yuwtennis/2e4c13c79f71e8f713e947955115b3e2](https://gist.github.com/yuwtennis/2e4c13c79f71e8f713e947955115b3e2) Spark 2.4.8 succeeded without any errors using above components. [https://archive.apache.org/dist/spark/spark-2.4.8/spark-2.4.8-bin-hadoop2.7.tgz](https://archive.apache.org/dist/spark/spark-2.4.8/spark-2.4.8-bin-hadoop2.7.tgz) Imported from Jira [BEAM-12762](https://issues.apache.org/jira/browse/BEAM-12762). Original Jira may contain additional context. Reported by: ibzib. This issue has child subcomponents which were not migrated over. See the original Jira for more information. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
