CheneyYin opened a new issue, #5159: URL: https://github.com/apache/seatunnel/issues/5159
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened Fail to execute transform when schema of source table contains any row type fields. ### SeaTunnel Version v2.3.2 ### SeaTunnel Config ```conf env { execution.parallelism = 2 job.mode = "BATCH" checkpoint.interval = 10000 } source { FakeSource { parallelism = 2 result_table_name = "fake" row.num = 16 schema = { fields { c_row = { c_int = int } } } } } #transform { # Sql { # source_table_name = "fake" # result_table_name = "fake1" # query = "select c_row from fake" # } #} transform { Copy = { source_table_name = "fake" result_table_name = "fake1" fields { c_row_1 = c_row } } } sink { Console { source_table_name = "fake1" } } ``` ### Running Command ```shell ./bin/start-seatunnel-spark-3-connector-v2.sh --master 'local[*]' --deploy-mode client --config ./config/v2.batch.config.template ``` ### Error Exception ```log 23/07/26 14:22:29 INFO FakeSourceReader: 16 rows of data have been generated in split(0). Generation time: 1690352549377 23/07/26 14:22:29 INFO FakeSourceReader: 16 rows of data have been generated in split(1). Generation time: 1690352549377 23/07/26 14:22:29 INFO FakeSourceReader: Closed the bounded fake source 23/07/26 14:22:29 INFO FakeSourceReader: Closed the bounded fake source 23/07/26 14:22:29 ERROR Utils: Aborting task org.apache.seatunnel.core.starter.exception.TaskExecuteException: Row convert failed, caused: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:191) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:151) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$1(WriteToDataSourceV2Exec.scala:435) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1538) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:480) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:381) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.ClassCastException: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:173) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:142) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:181) ... 18 more 23/07/26 14:22:29 ERROR Utils: Aborting task org.apache.seatunnel.core.starter.exception.TaskExecuteException: Row convert failed, caused: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:191) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:151) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$1(WriteToDataSourceV2Exec.scala:435) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1538) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:480) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:381) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.ClassCastException: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:173) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:142) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:181) ... 18 more 23/07/26 14:22:29 ERROR DataWritingSparkTask: Aborting commit for partition 1 (task 1, attempt 0, stage 0.0) 23/07/26 14:22:29 ERROR DataWritingSparkTask: Aborting commit for partition 0 (task 0, attempt 0, stage 0.0) 23/07/26 14:22:29 ERROR DataWritingSparkTask: Aborted commit for partition 1 (task 1, attempt 0, stage 0.0) 23/07/26 14:22:29 ERROR DataWritingSparkTask: Aborted commit for partition 0 (task 0, attempt 0, stage 0.0) 23/07/26 14:22:29 ERROR Executor: Exception in task 1.0 in stage 0.0 (TID 1) org.apache.seatunnel.core.starter.exception.TaskExecuteException: Row convert failed, caused: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:191) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:151) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$1(WriteToDataSourceV2Exec.scala:435) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1538) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:480) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:381) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.ClassCastException: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:173) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:142) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:181) ... 18 more 23/07/26 14:22:29 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0) org.apache.seatunnel.core.starter.exception.TaskExecuteException: Row convert failed, caused: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:191) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:151) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$1(WriteToDataSourceV2Exec.scala:435) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1538) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:480) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:381) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.ClassCastException: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:173) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:142) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:181) ... 18 more 23/07/26 14:22:29 WARN TaskSetManager: Lost task 1.0 in stage 0.0 (TID 1) (192.168.2.191 executor driver): org.apache.seatunnel.core.starter.exception.TaskExecuteException: Row convert failed, caused: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:191) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:151) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$1(WriteToDataSourceV2Exec.scala:435) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1538) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:480) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:381) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.ClassCastException: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:173) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:142) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:181) ... 18 more 23/07/26 14:22:29 ERROR TaskSetManager: Task 1 in stage 0.0 failed 1 times; aborting job 23/07/26 14:22:29 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 23/07/26 14:22:29 INFO TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0) on 192.168.2.191, executor driver: org.apache.seatunnel.core.starter.exception.TaskExecuteException (Row convert failed, caused: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739)) [duplicate 1] 23/07/26 14:22:29 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 23/07/26 14:22:29 INFO TaskSchedulerImpl: Cancelling stage 0 23/07/26 14:22:29 INFO TaskSchedulerImpl: Killing all running tasks in stage 0: Stage cancelled 23/07/26 14:22:29 INFO DAGScheduler: ResultStage 0 (save at SinkExecuteProcessor.java:125) failed in 1.791 s due to Job aborted due to stage failure: Task 1 in stage 0.0 failed 1 times, most recent failure: Lost task 1.0 in stage 0.0 (TID 1) (192.168.2.191 executor driver): org.apache.seatunnel.core.starter.exception.TaskExecuteException: Row convert failed, caused: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:191) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:151) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$1(WriteToDataSourceV2Exec.scala:435) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1538) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:480) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:381) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.ClassCastException: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:173) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:142) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:181) ... 18 more Driver stacktrace: 23/07/26 14:22:29 INFO DAGScheduler: Job 0 failed: save at SinkExecuteProcessor.java:125, took 1.847413 s 23/07/26 14:22:29 ERROR AppendDataExec: Data source write support org.apache.seatunnel.translation.spark.sink.SeaTunnelBatchWrite@46c92d27 is aborting. 23/07/26 14:22:29 ERROR AppendDataExec: Data source write support org.apache.seatunnel.translation.spark.sink.SeaTunnelBatchWrite@46c92d27 aborted. 23/07/26 14:22:29 ERROR SparkTaskExecuteCommand: Run SeaTunnel on spark failed. org.apache.spark.SparkException: Writing job aborted at org.apache.spark.sql.errors.QueryExecutionErrors$.writingJobAbortedError(QueryExecutionErrors.scala:767) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.writeWithV2(WriteToDataSourceV2Exec.scala:409) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.writeWithV2$(WriteToDataSourceV2Exec.scala:353) at org.apache.spark.sql.execution.datasources.v2.AppendDataExec.writeWithV2(WriteToDataSourceV2Exec.scala:244) at org.apache.spark.sql.execution.datasources.v2.V2ExistingTableWriteExec.run(WriteToDataSourceV2Exec.scala:332) at org.apache.spark.sql.execution.datasources.v2.V2ExistingTableWriteExec.run$(WriteToDataSourceV2Exec.scala:331) at org.apache.spark.sql.execution.datasources.v2.AppendDataExec.run(WriteToDataSourceV2Exec.scala:244) at org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:43) at org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result(V2CommandExec.scala:43) at org.apache.spark.sql.execution.datasources.v2.V2CommandExec.executeCollect(V2CommandExec.scala:49) at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109) at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169) at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779) at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64) at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:98) at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:94) at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:584) at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:176) at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:584) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:30) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:267) at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:263) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30) at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:30) at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:560) at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:94) at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:81) at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:79) at org.apache.spark.sql.execution.QueryExecution.assertCommandExecuted(QueryExecution.scala:116) at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:860) at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:311) at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:247) at org.apache.seatunnel.core.starter.spark.execution.SinkExecuteProcessor.execute(SinkExecuteProcessor.java:125) at org.apache.seatunnel.core.starter.spark.execution.SparkExecution.execute(SparkExecution.java:74) at org.apache.seatunnel.core.starter.spark.command.SparkTaskExecuteCommand.execute(SparkTaskExecuteCommand.java:60) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.spark.SeaTunnelSpark.main(SeaTunnelSpark.java:35) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:958) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1046) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1055) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 0.0 failed 1 times, most recent failure: Lost task 1.0 in stage 0.0 (TID 1) (192.168.2.191 executor driver): org.apache.seatunnel.core.starter.exception.TaskExecuteException: Row convert failed, caused: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:191) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:151) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$1(WriteToDataSourceV2Exec.scala:435) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1538) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:480) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:381) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.ClassCastException: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:173) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:142) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:181) ... 18 more Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2672) at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2608) at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2607) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2607) at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1182) at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1182) at scala.Option.foreach(Option.scala:407) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1182) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2860) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2802) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2791) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49) at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:952) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2238) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.writeWithV2(WriteToDataSourceV2Exec.scala:377) ... 49 more Caused by: org.apache.seatunnel.core.starter.exception.TaskExecuteException: Row convert failed, caused: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:191) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:151) at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46) at scala.collection.Iterator$$anon$10.next(Iterator.scala:461) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.$anonfun$run$1(WriteToDataSourceV2Exec.scala:435) at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1538) at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:480) at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:381) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.ClassCastException: class org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to class org.apache.seatunnel.api.table.type.SeaTunnelRow (org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema is in unnamed module of loader 'app'; org.apache.seatunnel.api.table.type.SeaTunnelRow is in unnamed module of loader org.apache.spark.util.MutableURLClassLoader @1fcf9739) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:173) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:151) at org.apache.seatunnel.translation.spark.serialization.SeaTunnelRowConverter.reconvert(SeaTunnelRowConverter.java:142) at org.apache.seatunnel.core.starter.spark.execution.TransformExecuteProcessor$TransformIterator.next(TransformExecuteProcessor.java:181) ... 18 more 23/07/26 14:22:29 ERROR SeaTunnel: =============================================================================== ``` ### Flink or Spark Version `spark v3.3.2` ### Java or Scala Version ```shell openjdk version "11.0.18" 2023-01-17 OpenJDK Runtime Environment (build 11.0.18+10) OpenJDK 64-Bit Server VM (build 11.0.18+10, mixed mode) ``` ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
