gitfortian opened a new issue, #4505: URL: https://github.com/apache/incubator-seatunnel/issues/4505
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened read mysql data and sink data to paimon casued nullpoint exception ### SeaTunnel Version 2.3.1 ### SeaTunnel Config ```conf env { execution.parallelism = 4 job.mode = "BATCH" checkpoint.interval = 360000 } source { Jdbc { url = "jdbc:mysql://xxx/dinky?useSSL=false" driver = "com.mysql.cj.jdbc.Driver" connection_check_timeout_sec = 100 user = "root" password = "xxx" query = "select * from dlink_job_history" } } sink { Paimon { warehouse = "hdfs://hdp01:8020/paimon/warehouse", database = "ods", table = "dlink_job_history" } } ``` ### Running Command ```shell bin/seatunnel.sh --config config/mysql-paimon.conf -m local ``` ### Error Exception ```log 2023-04-06 14:07:10,922 ERROR org.apache.seatunnel.engine.server.dag.physical.SubPlan - Task TaskGroupLocation{jobId=696239445162065921, pipelineId=1, taskGroupId=50000} Failed in Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)], Begin to cancel other tasks in this pipeline. 2023-04-06 14:07:10,923 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)] turn from state RUNNING to CANCELING. 2023-04-06 14:07:10,924 INFO org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start clean pending checkpoint cause Pipeline turn to end state. 2023-04-06 14:07:10,924 INFO org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - skip this checkpoint cause by completedCheckpoint is null 2023-04-06 14:07:10,926 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Jdbc-default-identifier]-SplitEnumerator (1/1)] turn from state RUNNING to CANCELING. 2023-04-06 14:07:10,927 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)], task: [pipeline-1 [Sink[0]-Paimon-default-identifier]-AggregatedCommitterTask (1/1)] turn from state RUNNING to CANCELING. 2023-04-06 14:07:10,928 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)], task: [pipeline-1 [Sink[0]-Paimon-default-identifier]-AggregatedCommitterTask (1/1)] operator to member [localhost]:5801 2023-04-06 14:07:10,928 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Jdbc-default-identifier]-SplitEnumerator (1/1)] operator to member [localhost]:5801 2023-04-06 14:07:10,928 INFO org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-863812] [5.1] Task (TaskGroupLocation{jobId=696239445162065921, pipelineId=1, taskGroupId=3}) need cancel. 2023-04-06 14:07:10,929 WARN org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-863812] [5.1] Interrupted task 40000 - org.apache.seatunnel.engine.server.task.SinkAggregatedCommitterTask@5bc4cebe 2023-04-06 14:07:10,929 INFO org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-863812] [5.1] Task (TaskGroupLocation{jobId=696239445162065921, pipelineId=1, taskGroupId=1}) need cancel. 2023-04-06 14:07:10,929 INFO org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-863812] [5.1] taskDone, taskId = 40000, taskGroup = TaskGroupLocation{jobId=696239445162065921, pipelineId=1, taskGroupId=3} 2023-04-06 14:07:10,929 INFO org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-863812] [5.1] Task TaskGroupLocation{jobId=696239445162065921, pipelineId=1, taskGroupId=3} complete with state CANCELED 2023-04-06 14:07:10,929 WARN org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-863812] [5.1] Interrupted task 20000 - org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask@4dc5c01a 2023-04-06 14:07:10,929 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)], task: [pipeline-1 [Sink[0]-Paimon-default-identifier]-AggregatedCommitterTask (1/1)] turn to end state CANCELED. 2023-04-06 14:07:10,929 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)], task: [pipeline-1 [Sink[0]-Paimon-default-identifier]-AggregatedCommitterTask (1/1)] end with state CANCELED 2023-04-06 14:07:10,929 INFO org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-863812] [5.1] taskDone, taskId = 20000, taskGroup = TaskGroupLocation{jobId=696239445162065921, pipelineId=1, taskGroupId=1} 2023-04-06 14:07:10,930 INFO org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-863812] [5.1] Task TaskGroupLocation{jobId=696239445162065921, pipelineId=1, taskGroupId=1} complete with state CANCELED 2023-04-06 14:07:10,932 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Jdbc-default-identifier]-SplitEnumerator (1/1)] turn to end state CANCELED. 2023-04-06 14:07:10,932 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-Jdbc-default-identifier]-SplitEnumerator (1/1)] end with state CANCELED 2023-04-06 14:07:11,005 INFO org.apache.seatunnel.engine.server.master.JobMaster - release the pipeline Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)] resource 2023-04-06 14:07:11,005 INFO org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot release request, jobID: 696239445162065921, slot: SlotProfile{worker=[localhost]:5801, slotID=3, ownerJobID=696239445162065921, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='98c2bb73-d06f-4f02-9672-feb96f9f9a6c'} 2023-04-06 14:07:11,006 INFO org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot release request, jobID: 696239445162065921, slot: SlotProfile{worker=[localhost]:5801, slotID=1, ownerJobID=696239445162065921, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='98c2bb73-d06f-4f02-9672-feb96f9f9a6c'} 2023-04-06 14:07:11,006 INFO org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot release request, jobID: 696239445162065921, slot: SlotProfile{worker=[localhost]:5801, slotID=2, ownerJobID=696239445162065921, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='98c2bb73-d06f-4f02-9672-feb96f9f9a6c'} 2023-04-06 14:07:11,010 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)] end with state FAILED 2023-04-06 14:07:11,010 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - cancel job Job SeaTunnel_Job (696239445162065921) because makeJobEndWhenPipelineEnded is true 2023-04-06 14:07:11,010 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (696239445162065921) turn from state RUNNING to CANCELLING. 2023-04-06 14:07:11,012 WARN org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (696239445162065921), Pipeline: [(1/1)] is in end state FAILED, can not be cancel 2023-04-06 14:07:11,012 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (696239445162065921) turn from state CANCELLING to FAILING. 2023-04-06 14:07:11,013 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (696239445162065921) turn from state FAILING to FAILED. 2023-04-06 14:07:11,030 INFO org.apache.seatunnel.engine.client.job.ClientJobProxy - Job (696239445162065921) end with state FAILED 2023-04-06 14:07:11,030 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-863812] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN 2023-04-06 14:07:11,037 INFO com.hazelcast.internal.server.tcp.TcpServerConnection - [localhost]:5801 [seatunnel-863812] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:55703, qualifier=null, endpoint=[127.0.0.1]:55703, remoteUuid=bb88be4a-263c-4b15-abf6-099277a17afe, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side 2023-04-06 14:07:11,038 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-863812] [5.1] Removed connection to endpoint: [localhost]:5801:9f6c09a4-b3aa-4450-be80-6260ed6a0937, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:55703->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2023-04-06 14:07:11.025, lastWriteTime=2023-04-06 14:07:09.206, closedTime=2023-04-06 14:07:11.033, connected server version=5.1} 2023-04-06 14:07:11,038 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-863812] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED 2023-04-06 14:07:11,041 INFO com.hazelcast.client.impl.ClientEndpointManager - [localhost]:5801 [seatunnel-863812] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:55703, qualifier=null, endpoint=[127.0.0.1]:55703, remoteUuid=bb88be4a-263c-4b15-abf6-099277a17afe, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=bb88be4a-263c-4b15-abf6-099277a17afe, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1680761219157, latest clientAttributes=lastStatisticsCollectionTime=1680761229193,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1680761219144,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=11219894272,os.freePhysicalMemorySize=2503786496,os.freeSwapSpaceSize=0,os.maxFileDescriptorCount=1024000,os.openFileDescriptorCount=140,os.processCpuTime=42300000000,os.systemLoadAverage=0.78,os.totalPhysicalMemorySize=33566371840,os.total SwapSpaceSize=0,runtime.availableProcessors=16,runtime.freeMemory=154326024,runtime.maxMemory=477626368,runtime.totalMemory=298319872,runtime.uptime=12292,runtime.usedMemory=143996168, labels=[]} 2023-04-06 14:07:11,041 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-863812] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN 2023-04-06 14:07:11,042 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client...... 2023-04-06 14:07:11,042 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-863812] [5.1] [localhost]:5801 is SHUTTING_DOWN 2023-04-06 14:07:11,049 INFO com.hazelcast.internal.partition.impl.MigrationManager - [localhost]:5801 [seatunnel-863812] [5.1] Shutdown request of Member [localhost]:5801 - 9f6c09a4-b3aa-4450-be80-6260ed6a0937 this is handled 2023-04-06 14:07:11,061 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-863812] [5.1] Shutting down connection manager... 2023-04-06 14:07:11,063 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-863812] [5.1] Shutting down node engine... 2023-04-06 14:07:11,087 INFO com.hazelcast.instance.impl.NodeExtension - [localhost]:5801 [seatunnel-863812] [5.1] Destroying node NodeExtension. 2023-04-06 14:07:11,087 INFO com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-863812] [5.1] Hazelcast Shutdown is completed in 40 ms. 2023-04-06 14:07:11,087 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-863812] [5.1] [localhost]:5801 is SHUTDOWN 2023-04-06 14:07:11,087 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed HazelcastInstance ...... 2023-04-06 14:07:11,087 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed metrics executor service ...... 2023-04-06 14:07:11,087 ERROR org.apache.seatunnel.core.starter.SeaTunnel - =============================================================================== 2023-04-06 14:07:11,087 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error, 2023-04-06 14:07:11,087 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/incubator-seatunnel/issues 2023-04-06 14:07:11,087 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed 2023-04-06 14:07:11,089 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: java.lang.NullPointerException at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:206) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:59) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:76) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:51) at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:52) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73) at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:161) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78) at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:526) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.NullPointerException at org.apache.paimon.data.AbstractBinaryWriter.writeString(AbstractBinaryWriter.java:60) at org.apache.paimon.data.BinaryRowWriter.writeString(BinaryRowWriter.java:25) at org.apache.seatunnel.connectors.seatunnel.paimon.utils.RowConverter.convert(RowConverter.java:361) at org.apache.seatunnel.connectors.seatunnel.paimon.sink.PaimonSinkWriter.write(PaimonSinkWriter.java:107) at org.apache.seatunnel.connectors.seatunnel.paimon.sink.PaimonSinkWriter.write(PaimonSinkWriter.java:49) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:201) ... 15 more at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:119) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:174) ... 2 more 2023-04-06 14:07:11,089 ERROR org.apache.seatunnel.core.starter.SeaTunnel - =============================================================================== Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: java.lang.RuntimeException: java.lang.NullPointerException at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:206) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:59) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:39) at org.apache.seatunnel.engine.server.task.SeaTunnelTransformCollector.collect(SeaTunnelTransformCollector.java:27) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.handleRecord(IntermediateBlockingQueue.java:76) at org.apache.seatunnel.engine.server.task.group.queue.IntermediateBlockingQueue.collect(IntermediateBlockingQueue.java:51) at org.apache.seatunnel.engine.server.task.flow.IntermediateQueueFlowLifeCycle.collect(IntermediateQueueFlowLifeCycle.java:52) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.collect(TransformSeaTunnelTask.java:73) at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:161) at org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask.call(TransformSeaTunnelTask.java:78) at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:526) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.NullPointerException at org.apache.paimon.data.AbstractBinaryWriter.writeString(AbstractBinaryWriter.java:60) at org.apache.paimon.data.BinaryRowWriter.writeString(BinaryRowWriter.java:25) at org.apache.seatunnel.connectors.seatunnel.paimon.utils.RowConverter.convert(RowConverter.java:361) at org.apache.seatunnel.connectors.seatunnel.paimon.sink.PaimonSinkWriter.write(PaimonSinkWriter.java:107) at org.apache.seatunnel.connectors.seatunnel.paimon.sink.PaimonSinkWriter.write(PaimonSinkWriter.java:49) at org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.received(SinkFlowLifeCycle.java:201) ... 15 more at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:119) at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:174) ... 2 more 2023-04-06 14:07:11,091 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - run shutdown hook because get close signal ``` ### Flink or Spark Version _No response_ ### Java or Scala Version scala2.12 ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
