gaotong521 opened a new issue, #6692: URL: https://github.com/apache/seatunnel/issues/6692
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened When the hive table storage type is orc, data sinks to the hive. Some fields in hive need to be passed NULL values, so you need to configure an SQL transform and set the null field to null. After the SQL transform is configured, task execution errors are reported ### SeaTunnel Version 2.3.4 ### SeaTunnel Config ```conf { "env": { "parallelism": 3, "job.mode": "BATCH", "checkpoint.interval": 30000, "job.name": "seatunnel_1712823979630" }, "source": [ { "plugin_name": "Jdbc", "result_table_name": "table_source", "user": "postgres", "password": "C3kk4v5_b4f2Jr", "driver": "org.postgresql.Driver", "url": "jdbc:postgresql://10.188.15.91:5434/gis", "query": "select event_id,event_type,event_radius,event_source,start_time,end_time,priority,latitude,longitude,elevation,node_ids,create_time,update_time from ghcloud.gh_traffic_event_info" } ], "transform": [ { "plugin_name": "FieldMapper", "source_table_name": "table_source", "result_table_name": "table_source_FieldMapper", "field_mapper": { "event_id": "event_id", "event_type": "event_type", "event_radius": "event_radius", "event_source": "event_source", "start_time": "start_time", "end_time": "end_time", "priority": "priority", "latitude": "latitude", "longitude": "longitude", "elevation": "elevation", "node_ids": "node_ids", "create_time": "create_time", "update_time": "update_time" } }, { "plugin_name": "Sql", "source_table_name": "table_source_FieldMapper", "result_table_name": "table_source_FieldMapper_Sql", "query": "select event_id,event_type,event_radius,event_source,start_time,end_time,priority,latitude,longitude,elevation,node_ids,create_time,update_time,null as region_code,null as pub_mode,null as end_latitude,null as end_longitude,null as end_elevation,null as custom_event_name,null as comb_mode,null as ref_event_id,null as lane_id from table_source_FieldMapper" } ], "sink": [ { "plugin_name": "Hive", "source_table_name": "table_source_FieldMapper_Sql", "table_name": "gh_cloud_data_model.dwd_pub_traffic_event", "metastore_uri": "thrift://cloudera-hadoop-61:9083" } ] } ``` ### Running Command ```shell Executed by dolphin scheduler ``` ### Error Exception ```log SHUTDOWN 2024-04-12 11:24:16,507 INFO [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client...... 2024-04-12 11:24:16,507 INFO [s.c.s.s.c.ClientExecuteCommand] [main] - Closed metrics executor service ...... 2024-04-12 11:24:16,507 ERROR [o.a.s.c.s.SeaTunnel ] [main] - =============================================================================== 2024-04-12 11:24:16,507 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Fatal Error, 2024-04-12 11:24:16,507 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues 2024-04-12 11:24:16,507 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Reason:SeaTunnel job executed failed 2024-04-12 11:24:16,508 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: org.apache.seatunnel.common.exception.SeaTunnelRuntimeException: ErrorCode:[COMMON-23], ErrorDescription:[FileConnector write SeaTunnelRow failed, the SeaTunnelRow value is 'SeaTunnelRow{tableId=default.default.default, kind=+I, fields=[121169, 20409913, null, 1, 2021-06-05T05:47:13.379, 2021-06-05T05:47:23.379, 7, 29.777297, 107.437978, 0.0, 337,342, 2021-06-05T05:46:40.505368, 2021-06-05T05:46:40.505368, null, null, null, null, null, null, null, null, null]}'.] at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:257) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:66) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:75) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:50) at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:51) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73) at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78) at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:648) at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:949) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.seatunnel.common.exception.SeaTunnelRuntimeException: ErrorCode:[COMMON-23], ErrorDescription:[FileConnector write SeaTunnelRow failed, the SeaTunnelRow value is 'SeaTunnelRow{tableId=default.default.default, kind=+I, fields=[121169, 20409913, null, 1, 2021-06-05T05:47:13.379, 2021-06-05T05:47:23.379, 7, 29.777297, 107.437978, 0.0, 337,342, 2021-06-05T05:46:40.505368, 2021-06-05T05:46:40.505368, null, null, null, null, null, null, null, null, null]}'.] at org.apache.seatunnel.common.exception.CommonError.writeSeaTunnelRowFailed(CommonError.java:86) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:136) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:46) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:247) ... 16 more Caused by: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-07], ErrorDescription:[Unsupported data type] - Orc file not support this type [NULL] at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.buildFieldWithRowType(OrcWriteStrategy.java:188) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.buildSchemaWithRowType(OrcWriteStrategy.java:196) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.getOrCreateWriter(OrcWriteStrategy.java:116) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.write(OrcWriteStrategy.java:75) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:134) ... 18 more at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:194) ... 2 more 2024-04-12 11:24:16,509 ERROR [o.a.s.c.s.SeaTunnel ] [main] - =============================================================================== Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:202) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: org.apache.seatunnel.common.exception.SeaTunnelRuntimeException: ErrorCode:[COMMON-23], ErrorDescription:[FileConnector write SeaTunnelRow failed, the SeaTunnelRow value is 'SeaTunnelRow{tableId=default.default.default, kind=+I, fields=[121169, 20409913, null, 1, 2021-06-05T05:47:13.379, 2021-06-05T05:47:23.379, 7, 29.777297, 107.437978, 0.0, 337,342, 2021-06-05T05:46:40.505368, 2021-06-05T05:46:40.505368, null, null, null, null, null, null, null, null, null]}'.] at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:257) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:66) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:75) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:50) at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:51) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73) at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78) at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:648) at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:949) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.seatunnel.common.exception.SeaTunnelRuntimeException: ErrorCode:[COMMON-23], ErrorDescription:[FileConnector write SeaTunnelRow failed, the SeaTunnelRow value is 'SeaTunnelRow{tableId=default.default.default, kind=+I, fields=[121169, 20409913, null, 1, 2021-06-05T05:47:13.379, 2021-06-05T05:47:23.379, 7, 29.777297, 107.437978, 0.0, 337,342, 2021-06-05T05:46:40.505368, 2021-06-05T05:46:40.505368, null, null, null, null, null, null, null, null, null]}'.] at org.apache.seatunnel.common.exception.CommonError.writeSeaTunnelRowFailed(CommonError.java:86) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:136) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:46) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:247) ... 16 more Caused by: org.apache.seatunnel.connectors.seatunnel.file.exception.FileConnectorException: ErrorCode:[COMMON-07], ErrorDescription:[Unsupported data type] - Orc file not support this type [NULL] at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.buildFieldWithRowType(OrcWriteStrategy.java:188) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.buildSchemaWithRowType(OrcWriteStrategy.java:196) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.getOrCreateWriter(OrcWriteStrategy.java:116) at org.apache.seatunnel.connectors.seatunnel.file.sink.writer.OrcWriteStrategy.write(OrcWriteStrategy.java:75) at org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSinkWriter.write(BaseFileSinkWriter.java:134) ... 18 more at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:194) ... 2 more 2024-04-12 11:24:16,510 INFO [s.c.s.s.c.ClientExecuteCommand] [ForkJoinPool.commonPool-worker-2] - run shutdown hook because get close signal [INFO] 2024-04-12 11:24:16.913 +0800 - FINALIZE_SESSION ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
