liunaijie opened a new issue, #11499: URL: https://github.com/apache/seatunnel/issues/11499
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened got NPE exception ### SeaTunnel Version 2.3.13 ### SeaTunnel Config ```conf env { parallelism = 1 job.mode = "BATCH" spark.app.name = "SMkHXmRTFRxR" spark.driver.cores = 1 spark.driver.memory = "1024M" spark.executor.cores = 2 spark.executor.memory = "1024M" spark.executor.instances = 4 spark.yarn.queue = "default" } source { Hive { table_name = "a parquet format table" metastore_uri = "thrift://xxxx:9083" } } transform { } sink { Console { } } ``` ### Running Command ```shell ./bin/start-seatunnel-spark-3-connector-v2.sh --master local --config <the_above_config> ``` ### Error Exception ```log 26/07/20 12:33:57 INFO execution.MultiTableManager: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: MultiTableManager columnWithIndex:ColumnWithIndex{table=xxxxx, index=[0, 1], schema=ROW<id STRING>} 26/07/20 12:33:57 INFO reader.MultipleTableHiveSourceReader: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: Opened the MultipleTableHiveSourceReader 26/07/20 12:33:57 INFO event.LoggingEventHandler: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: log event: ReaderOpenEvent(createdTime=1784514837209, jobId=local-1784514832043, eventType=LIFECYCLE_READER_OPEN) 26/07/20 12:33:57 INFO event.LoggingEventHandler: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: log event: EnumeratorOpenEvent(createdTime=1784514837210, jobId=local-1784514832043, eventType=LIFECYCLE_ENUMERATOR_OPEN) 26/07/20 12:33:57 INFO split.MultipleTableHiveSourceSplitEnumerator: [parallel-split-enumerator-executor-0-1]: Assigned splits to reader [0] 26/07/20 12:33:57 INFO split.MultipleTableHiveSourceSplitEnumerator: [parallel-split-enumerator-executor-0-1]: SubTask 0 is assigned to [jmsau_ods.ods_task_operation_log_full_d_test_20260714_hdfs://yl-bg-jmsau/dw/hive/jmsau_ods.db/external/ods_task_operation_log_full_d_test_20260714/T_29dd1a37cb834ac2aea8eb38852c343b_18c1240ae3_3_1_0.parquet], size 1 26/07/20 12:33:57 INFO hadoop.InternalParquetRecordReader: [parallel-split-enumerator-executor-0-1]: RecordReader initialized will read a total of 53 records. 26/07/20 12:33:57 INFO hadoop.InternalParquetRecordReader: [parallel-split-enumerator-executor-0-1]: at row 0. reading next block 26/07/20 12:33:57 INFO hadoop.InternalParquetRecordReader: [parallel-split-enumerator-executor-0-1]: block read in memory in 23 ms. row count = 53 26/07/20 12:33:57 INFO reader.MultipleTableHiveSourceReader: [parallel-split-enumerator-executor-0-1]: There is no more element for the bounded MultipleTableLocalFileSourceReader 26/07/20 12:33:57 ERROR util.Utils: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: Aborting task java.lang.NullPointerException: Cannot invoke "org.apache.spark.unsafe.types.UTF8String.getBaseObject()" because "input" is null at org.apache.spark.sql.catalyst.expressions.codegen.UnsafeWriter.write(UnsafeWriter.java:110) ~[spark-catalyst_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source) ~[?:?] at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) ~[scala-library-2.12.19.jar:?] at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) ~[scala-library-2.12.19.jar:?] at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491) ~[scala-library-2.12.19.jar:?] at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) ~[scala-library-2.12.19.jar:?] at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) ~[?:?] at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$5(WriteToDataSourceV2Exec.scala:456) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1398) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:501) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:440) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:506) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:403) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.scheduler.Task.run(Task.scala:141) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:638) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64) [spark-common-utils_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61) [spark-common-utils_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:95) [spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:641) [spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:840) [?:?] 26/07/20 12:33:57 ERROR v2.DataWritingSparkTask: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: Aborting commit for partition 0 (task 0, attempt 0, stage 0.0) 26/07/20 12:33:57 ERROR v2.DataWritingSparkTask: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: Aborted commit for partition 0 (task 0, attempt 0, stage 0.0) 26/07/20 12:33:57 INFO reader.MultipleTableHiveSourceReader: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: Closed the MultipleTableHiveSourceReader 26/07/20 12:33:57 INFO event.LoggingEventHandler: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: log event: ReaderCloseEvent(createdTime=1784514837622, jobId=local-1784514832043, eventType=LIFECYCLE_READER_CLOSE) 26/07/20 12:33:57 INFO event.LoggingEventHandler: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: log event: EnumeratorCloseEvent(createdTime=1784514837622, jobId=local-1784514832043, eventType=LIFECYCLE_ENUMERATOR_CLOSE) 26/07/20 12:33:57 ERROR executor.Executor: [Executor task launch worker for task 0.0 in stage 0.0 (TID 0)]: Exception in task 0.0 in stage 0.0 (TID 0) java.lang.NullPointerException: Cannot invoke "org.apache.spark.unsafe.types.UTF8String.getBaseObject()" because "input" is null at org.apache.spark.sql.catalyst.expressions.codegen.UnsafeWriter.write(UnsafeWriter.java:110) ~[spark-catalyst_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source) ~[?:?] at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) ~[scala-library-2.12.19.jar:?] at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) ~[scala-library-2.12.19.jar:?] at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:491) ~[scala-library-2.12.19.jar:?] at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460) ~[scala-library-2.12.19.jar:?] at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source) ~[?:?] at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.$anonfun$run$5(WriteToDataSourceV2Exec.scala:456) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1398) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run(WriteToDataSourceV2Exec.scala:501) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.WritingSparkTask.run$(WriteToDataSourceV2Exec.scala:440) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.DataWritingSparkTask$.run(WriteToDataSourceV2Exec.scala:506) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.sql.execution.datasources.v2.V2TableWriteExec.$anonfun$writeWithV2$2(WriteToDataSourceV2Exec.scala:403) ~[spark-sql_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.scheduler.Task.run(Task.scala:141) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:638) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64) ~[spark-common-utils_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61) ~[spark-common-utils_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:95) ~[spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:641) [spark-core_2.12-3.5.4.7.3.2.0-957.jar:3.5.4.7.3.2.0-957] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:840) [?:?] ``` ### Zeta or Flink or Spark Version spark 3.3.0 ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
