Ngone51 commented on pull request #35492: URL: https://github.com/apache/spark/pull/35492#issuecomment-1054910730
cc @weixiuli @srowen This change would cause test failure with Hadoop2: ``` ./build/sbt -mem 4096 -Phadoop-2 "sql/testOnly org.apache.spark.sql.sources.PartitionedWriteSuite -- -z SPARK-27194" ``` ``` ... [info] Cause: org.apache.spark.SparkException: Task not serializable ... [info] Cause: java.io.NotSerializableException: org.apache.hadoop.fs.Path ... ``` It looks like it's because `org.apache.hadoop.fs.Path` is serializable in Hadoop3 but not in Hadoop2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
