liujinhui1994 opened a new issue #1273:
URL: https://github.com/apache/incubator-seatunnel/issues/1273


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22)
 and found no similar issues.
   
   
   ### What happened
   
   Use Hudi source  Serializable Exception!
   
   ### SeaTunnel Version
   
   2.0.5-SNAPSHOT
   
   ### SeaTunnel Config
   
   ```conf
   env {
     spark.app.name = "SeaTunnel"
     spark.master = local
   }
   
   source {
     hudi {
         hoodie.datasource.read.paths = "path"
         result_table_name="view_20220215"
     }
   }
   
   transform {
   }
   
   sink {
     Console {}
   }
   ```
   
   
   ### Running Command
   
   ```shell
   org.apache.seatunnel.example.spark.LocalSparkExample
   Use this class to run locally
   ```
   
   
   ### Error Exception
   
   ```log
   Caused by: org.apache.spark.SparkException: Job aborted due to stage 
failure: Failed to serialize task 1, not attempting to retry it. Exception 
during serialization: java.io.NotSerializableException: 
org.apache.hadoop.fs.Path
   Serialization stack:
        - object not serializable (class: org.apache.hadoop.fs.Path, value: 
file:/G:/tmp/2022)
        - element of array (index: 0)
        - array (class [Ljava.lang.Object;, size 1)
        - field (class: scala.collection.mutable.WrappedArray$ofRef, name: 
array, type: class [Ljava.lang.Object;)
        - object (class scala.collection.mutable.WrappedArray$ofRef, 
WrappedArray(file:/G:/tmp/2022))
        - writeObject data (class: 
org.apache.spark.rdd.ParallelCollectionPartition)
        - object (class org.apache.spark.rdd.ParallelCollectionPartition, 
org.apache.spark.rdd.ParallelCollectionPartition@6e3)
        - field (class: org.apache.spark.scheduler.ResultTask, name: partition, 
type: interface org.apache.spark.Partition)
        - object (class org.apache.spark.scheduler.ResultTask, ResultTask(1, 0))
        at 
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1887)
        at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1875)
        at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1874)
        at 
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
        at 
org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1874)
        at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at 
org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
        at scala.Option.foreach(Option.scala:257)
        at 
org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
        at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2108)
        at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2057)
        at 
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2046)
        at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
        at 
org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:737)
        at org.apache.spark.SparkContext.runJob(SparkContext.scala:2061)
   ```
   
   
   ### Flink or Spark Version
   
   2.0.5-SNAPSHOT run,unmodified version
   
   ### Java or Scala Version
   
   2.0.5-SNAPSHOT run,unmodified version
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to