DalongZhenZ commented on issue #5396:
URL: https://github.com/apache/seatunnel/issues/5396#issuecomment-1784848465
> i set ( execution.checkpoint.interval = 1 minutes),but it also have error
such as:
>
> 2023-10-30 17:34:15,455 ERROR
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - report
error from task org.apache.seatunnel.common.utils.SeaTunnelException:
java.lang.NullPointerException at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.lambda$buildSchemaWithRowType$0(AbstractWriteStrategy.java:135)
at java.util.ArrayList.forEach(ArrayList.java:1259) at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.buildSchemaWithRowType(AbstractWriteStrategy.java:133)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.setSeaTunnelRowTypeInfo(TextWriteStrategy.java:74)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriteStrategy(BaseFileSink.java:123)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriter(BaseFileSink.java:90)
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java
:286) at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$15(SeaTunnelTask.java:419)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:416)
at org.apache.
seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$null$0(NotifyTaskRestoreOperation.java:106)
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
>
> ```
> at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.reportCheckpointErrorFromTask(CheckpointCoordinator.java:346)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.reportCheckpointErrorFromTask(CheckpointManager.java:188)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointErrorReportOperation.run(CheckpointErrorReportOperation.java:48)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:213)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:175)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:139)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> ```
>
> 2023-10-30 17:34:15,458 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - HMSC::open(): Could not
find delegation token. Creating KERBEROS-based thrift connection. 2023-10-30
17:34:15,458 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start
clean pending checkpoint cause CheckpointCoordinator inside have error.
2023-10-30 17:34:15,459 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - Turn
checkpoint_state_771305889444397057_1 state from null to FAILED 2023-10-30
17:34:15,462 WARN org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] checkpoint have error,
cancel the pipeline 2023-10-30 17:34:15,465 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] turned from state RUNNING to CANCELING.
2023-10-30 17:34:15,468 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job Sea
Tunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state RUNNING to CANCELING. 2023-10-30 17:34:15,469 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] operator to
member [localhost]:5801 2023-10-30 17:34:15,469 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=1}) need cancel. 2023-10-30 17:34:15,469 WARN
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 20000 -
org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask@1e8fc099
2023-10-30 17:34:15,470 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [l
ocalhost]:5801 [seatunnel-280304] [5.1] taskDone, taskId = 20000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}
2023-10-30 17:34:15,470 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=1} complete with state CANCELED 2023-10-30
17:34:15,470 INFO org.apache.seatunnel.engine.server.CoordinatorService -
[localhost]:5801 [seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}, state
CANCELED 2023-10-30 17:34:15,471 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state RUNNING to CANCELING. 2023-10-30 17:34:15,471 INFO
org.apache.seatunnel.engine.server.dag.physical.Physical
Vertex - Send cancel Job SeaTunnel_Job (771305889444397057), Pipeline:
[(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] operator to
member [localhost]:5801 2023-10-30 17:34:15,471 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=3}) need cancel. 2023-10-30 17:34:15,472 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state CANCELING to CANCELED. 2023-10-30 17:34:15,472 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] state process
is stopped 2023-10-30 17:34:15,472 INFO org.apache.
seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] future complete
with state CANCELED 2023-10-30 17:34:15,473 WARN
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 40000 -
org.apache.seatunnel.engine.server.task.SinkAggregatedCommitterTask@5e4896a8
2023-10-30 17:34:15,473 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 40000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3}
2023-10-30 17:34:15,473 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=3} complete with state CANCELED 2023-10-30
17:34:15,473 INFO org.apache.seatunnel.engine.server.Co
ordinatorService - [localhost]:5801 [seatunnel-280304] [5.1] Received task end
from execution TaskGroupLocation{jobId=771305889444397057, pipelineId=1,
taskGroupId=3}, state CANCELED 2023-10-30 17:34:15,474 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
RUNNING to CANCELING. 2023-10-30 17:34:15,475 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] operator to member
[localhost]:5801 2023-10-30 17:34:15,475 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=50000}) need cancel. 2023-10-30 17:34:15,475 WARN
org.apache.
seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 60000 -
org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask@297cae0e 2023-10-30
17:34:15,475 WARN org.apache.seatunnel.engine.server.TaskExecutionService -
[localhost]:5801 [seatunnel-280304] [5.1] Interrupted task 70000 -
org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask@1b11648d
2023-10-30 17:34:15,475 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 60000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
2023-10-30 17:34:15,475 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 70000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
2023-10-30 17:34:15,476 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job Se
aTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state CANCELING to CANCELED. 2023-10-30 17:34:15,476 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] state process
is stopped 2023-10-30 17:34:15,476 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] future
complete with state CANCELED 2023-10-30 17:34:15,477 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=50000} complete with state CANCELED 2023-10-30
17:34:15,477 INFO org.apache.seatunnel.engi
ne.server.CoordinatorService - [localhost]:5801 [seatunnel-280304] [5.1]
Received task end from execution TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=50000}, state CANCELED 2023-10-30 17:34:15,478 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
CANCELING to CANCELED. 2023-10-30 17:34:15,478 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] state process is
stopped 2023-10-30 17:34:15,478 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] future complete with
state CANCELED 2023-10-30 17
:34:15,478 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] will end with state
FAILED 2023-10-30 17:34:15,479 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] turned from state CANCELING to FAILED.
2023-10-30 17:34:15,479 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Restore time 1,
pipeline Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] 2023-10-30
17:34:15,480 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan -
Reset pipeline Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] state
to CREATED 2023-10-30 17:34:15,480 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Reset pipeline Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] state to CREATED complete
2023-10-30 17:34:15,481 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (77130588
9444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turn to state
CREATED. 2023-10-30 17:34:15,482 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turn to state
CREATED. 2023-10-30 17:34:15,483 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turn to state
CREATED. 2023-10-30 17:34:15,483 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - reported
pipeline running stack: [java.lang.Thread.getStackTrace(Thread.java:1559),
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.reportedPipelineRunning(CheckpointManager.java:175),
org.apache.seatunnel.engine.server.dag.physical.SubP
lan.prepareRestorePipeline(SubPlan.java:429),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.stateProcess(SubPlan.java:627),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.updatePipelineState(SubPlan.java:341),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.lambda$addPhysicalVertexCallBack$2(SubPlan.java:203),
java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670),
java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646),
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456),
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149),
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624),
java.lang.Thread.run(Thread.java:748)] 2023-10-30 17:34:15,483 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - received
restore CheckpointCoordinator with alreadyStarted= false 2023-10-30
17:34:15,483 INFO org.apache.seatu
nnel.engine.server.checkpoint.CheckpointCoordinator - Turn
checkpoint_state_771305889444397057_1 state from FAILED to RUNNING 2023-10-30
17:34:15,483 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start
clean pending checkpoint cause CheckpointCoordinator reset. 2023-10-30
17:34:15,484 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] is in state CREATED
when init state future 2023-10-30 17:34:15,484 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] is in state
CREATED when init state future 2023-10-30 17:34:15,484 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
SeaTunnel_Job (771305889444397
057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] is in state
CREATED when init state future 2023-10-30 17:34:15,485 INFO
org.apache.seatunnel.engine.server.master.JobMaster - release the pipeline Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] resource 2023-10-30
17:34:15,485 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=1, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,486 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=3, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CP
U{core=0}, heapMemory=Memory{bytes=0}},
sequence='dca25825-c421-4dc2-92e2-43dffb60f386'} 2023-10-30 17:34:15,486 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=2, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,487 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] state process is start 2023-10-30
17:34:15,488 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] turned from state CREATED
to SCHEDULED. 2023-10-30 17:34:15,489 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] turned from state SCHEDULED
to DEPLOYING. 2023-10-30 17:34:15,489 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] state process
is start 2023-10-30 17:34:15,490 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state CREATED to DEPLOYING. 2023-10-30 17:34:15,491 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] received deploying task executionId
[771305893852807169] 2023-10-30 17:34:15,499 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Opened a connection to
metastore, current connections: 1 2023-10-30 17:34:15,499 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Connected to metastore.
2023-10-30 17:3
4:15,500 INFO org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Closed a
connection to metastore, current connections: 0 2023-10-30 17:34:15,501 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying task
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1},
executionId [771305893852807169] 2023-10-30 17:34:15,501 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1} init
success 2023-10-30 17:34:15,501 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}
success 2023-10-30 17:34:15,501 INFO
org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - starting
seatunnel source split enumerator task, source na
me: pipeline-1 [Source[0]-LocalFile-default-identifier] 2023-10-30
17:34:15,502 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state DEPLOYING to RUNNING. 2023-10-30 17:34:15,503 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] current state
equals target state: RUNNING, skip 2023-10-30 17:34:15,503 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] state process
is start 2023-10-30 17:34:15,504 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (7713058894
44397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state CREATED to DEPLOYING. 2023-10-30 17:34:15,506 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] received deploying task executionId
[771305893852872705] 2023-10-30 17:34:15,519 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying task
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3},
executionId [771305893852872705] 2023-10-30 17:34:15,520 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3} init
success 2023-10-30 17:34:15,520 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup TaskGroupLocation{jobId
=771305889444397057, pipelineId=1, taskGroupId=3} success 2023-10-30
17:34:15,521 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state DEPLOYING to RUNNING. 2023-10-30 17:34:15,521 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] current state
equals target state: RUNNING, skip 2023-10-30 17:34:15,521 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] state process is
start 2023-10-30 17:34:15,529 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444
397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
CREATED to DEPLOYING. 2023-10-30 17:34:15,531 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] received deploying task executionId
[771305893852938241] 2023-10-30 17:34:15,550 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying task
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000},
executionId [771305893852938241] 2023-10-30 17:34:15,550 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
init success 2023-10-30 17:34:15,551 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup TaskGroupLocation{jobId
=771305889444397057, pipelineId=1, taskGroupId=50000} success 2023-10-30
17:34:15,551 INFO
org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask - starting
seatunnel transform task, index 0 2023-10-30 17:34:15,551 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
DEPLOYING to RUNNING. 2023-10-30 17:34:15,551 INFO
org.apache.seatunnel.connectors.seatunnel.file.sink.util.FileSystemUtils -
Start Kerberos authentication using principal
hive/[[email protected]](mailto:[email protected]) and keytab
/opt/kibb_conf/hive.keytab 2023-10-30 17:34:15,551 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] current state equals
target state: RUNNIN
G, skip 2023-10-30 17:34:15,552 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] turned from state DEPLOYING to RUNNING.
2023-10-30 17:34:15,556 INFO
org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask - starting
seatunnel source task, index 0 2023-10-30 17:34:15,557 ERROR
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - report
error from task org.apache.seatunnel.common.utils.SeaTunnelException:
java.lang.NullPointerException at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.lambda$buildSchemaWithRowType$0(AbstractWriteStrategy.java:135)
at java.util.ArrayList.forEach(ArrayList.java:1259) at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.buildSchemaWithRowType(AbstractWriteStrategy.java:133)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.setSeaTunnelRowTypeInfo(TextWriteStrategy.java
:74) at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriteStrategy(BaseFileSink.java:123)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriter(BaseFileSink.java:90)
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:286)
at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$15(SeaTunnelTask.java:419)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.Fo
rEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:416)
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$null$0(NotifyTaskRestoreOperation.java:106)
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
>
> ```
> at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.reportCheckpointErrorFromTask(CheckpointCoordinator.java:346)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.reportCheckpointErrorFromTask(CheckpointManager.java:188)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointErrorReportOperation.run(CheckpointErrorReportOperation.java:48)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:213)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:175)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:139)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> ```
>
> 2023-10-30 17:34:15,557 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start
clean pending checkpoint cause CheckpointCoordinator inside have error.
2023-10-30 17:34:15,557 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - Turn
checkpoint_state_771305889444397057_1 state from RUNNING to FAILED 2023-10-30
17:34:15,558 WARN org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] checkpoint have error,
cancel the pipeline 2023-10-30 17:34:15,558 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] turned from state RUNNING to CANCELING.
2023-10-30 17:34:15,559 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state RUNNING to CANCELIN
G. 2023-10-30 17:34:15,560 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] operator to
member [localhost]:5801 2023-10-30 17:34:15,560 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=1}) need cancel. 2023-10-30 17:34:15,560 WARN
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 20000 -
org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask@1d303141
2023-10-30 17:34:15,560 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 20000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}
2023-10-30 17:34:15,
560 INFO org.apache.seatunnel.engine.server.TaskExecutionService -
[localhost]:5801 [seatunnel-280304] [5.1] Task
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}
complete with state CANCELED 2023-10-30 17:34:15,560 INFO
org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801
[seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}, state
CANCELED 2023-10-30 17:34:15,561 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state CANCELING to CANCELED. 2023-10-30 17:34:15,561 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state RU
NNING to CANCELING. 2023-10-30 17:34:15,561 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] state process
is stopped 2023-10-30 17:34:15,561 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] future complete
with state CANCELED 2023-10-30 17:34:15,562 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] operator to
member [localhost]:5801 2023-10-30 17:34:15,562 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=7713058894
44397057, pipelineId=1, taskGroupId=3}) need cancel. 2023-10-30 17:34:15,562
WARN org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 40000 -
org.apache.seatunnel.engine.server.task.SinkAggregatedCommitterTask@14095abb
2023-10-30 17:34:15,562 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 40000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3}
2023-10-30 17:34:15,562 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=3} complete with state CANCELED 2023-10-30
17:34:15,562 INFO org.apache.seatunnel.engine.server.CoordinatorService -
[localhost]:5801 [seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3}, sta
te CANCELED 2023-10-30 17:34:15,563 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
RUNNING to CANCELING. 2023-10-30 17:34:15,563 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state CANCELING to CANCELED. 2023-10-30 17:34:15,563 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] state process
is stopped 2023-10-30 17:34:15,563 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipel
ine-1 [Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] operator to
member [localhost]:5801 2023-10-30 17:34:15,564 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=50000}) need cancel. 2023-10-30 17:34:15,564 WARN
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 70000 -
org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask@9136c7c
2023-10-30 17:34:15,564 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 70000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
2023-10-30 17:34:15,564 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-Agg
regatedCommitterTask (1/1)] future complete with state CANCELED 2023-10-30
17:34:15,564 WARN org.apache.seatunnel.engine.server.TaskExecutionService -
[localhost]:5801 [seatunnel-280304] [5.1] Interrupted task 60000 -
org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask@5e00db2a 2023-10-30
17:34:15,564 INFO org.apache.seatunnel.engine.server.TaskExecutionService -
[localhost]:5801 [seatunnel-280304] [5.1] taskDone, taskId = 60000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
2023-10-30 17:34:15,564 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=50000} complete with state CANCELED 2023-10-30
17:34:15,564 INFO org.apache.seatunnel.engine.server.CoordinatorService -
[localhost]:5801 [seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGrou
pId=50000}, state CANCELED 2023-10-30 17:34:15,565 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
CANCELING to CANCELED. 2023-10-30 17:34:15,565 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] state process is
stopped 2023-10-30 17:34:15,565 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] future complete with
state CANCELED 2023-10-30 17:34:15,565 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] will end with state FAILED 2023-10-30 17
:34:15,566 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] turned from state
CANCELING to FAILED. 2023-10-30 17:34:15,566 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Restore time 2,
pipeline Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] 2023-10-30
17:34:15,566 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan -
Reset pipeline Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] state
to CREATED 2023-10-30 17:34:15,566 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Reset pipeline Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] state to CREATED complete
2023-10-30 17:34:15,567 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turn to state
CREATED. 2023-10-30 17:34:15,567 INFO org.ap
ache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turn to state
CREATED. 2023-10-30 17:34:15,568 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turn to state
CREATED. 2023-10-30 17:34:15,568 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - reported
pipeline running stack: [java.lang.Thread.getStackTrace(Thread.java:1559),
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.reportedPipelineRunning(CheckpointManager.java:175),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.prepareRestorePipeline(SubPlan.java:429),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.stateProcess(SubPlan.java:627),
org.apache.seatunnel.engine.server.dag
.physical.SubPlan.updatePipelineState(SubPlan.java:341),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.lambda$addPhysicalVertexCallBack$2(SubPlan.java:203),
java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670),
java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646),
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456),
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149),
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624),
java.lang.Thread.run(Thread.java:748)] 2023-10-30 17:34:15,568 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - received
restore CheckpointCoordinator with alreadyStarted= false 2023-10-30
17:34:15,568 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - Turn
checkpoint_state_771305889444397057_1 state from FAILED to RUNNING 2023-10-30
17:34:15,568 INFO org.apache.seatun
nel.engine.server.checkpoint.CheckpointCoordinator - start clean pending
checkpoint cause CheckpointCoordinator reset. 2023-10-30 17:34:15,568 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] is in state CREATED
when init state future 2023-10-30 17:34:15,569 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] is in state
CREATED when init state future 2023-10-30 17:34:15,569 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] is in state
CREATED when init state future 2023-10-30 17:34:15,5
69 INFO org.apache.seatunnel.engine.server.master.JobMaster - release the
pipeline Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] resource
2023-10-30 17:34:15,569 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=4, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,569 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=6, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,569 INFO
org.apache.seatunnel.engine.server.service.slot.Defaul
tSlotService - received slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=5, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,570 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] state process is start 2023-10-30
17:34:15,571 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] turned from state CREATED
to SCHEDULED. 2023-10-30 17:34:15,572 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] turned from state SCHEDULED to
DEPLOYING. 2023-10-30 17:34:15,572 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task:
[pipeline-1 [Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)]
state process is start 2023-10-30 17:34:15,573 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state CREATED to DEPLOYING. 2023-10-30 17:34:15,574 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] received deploying task executionId
[771305893853003777] 2023-10-30 17:34:15,574 INFO
org.apache.hadoop.security.UserGroupInformation - Login successful for user
hive/[[email protected]](mailto:[email protected]) using
keytab file /opt/kibb_conf/hive.keytab 2023-10-30 17:34:15,574 INFO
org.apache.seatunnel.connectors.seatunnel.file.sink.util.FileSystemUtils -
Kerberos authentication successful 2023-10-30 17:34:15,581 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreCli
ent - Trying to connect to metastore with URI thrift://devcdp03.test.com:9083
2023-10-30 17:34:15,582 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - HMSC::open(): Could not
find delegation token. Creating KERBEROS-based thrift connection. 2023-10-30
17:34:15,585 INFO org.apache.seatunnel.engine.server.TaskExecutionService -
[localhost]:5801 [seatunnel-280304] [5.1] deploying task
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1},
executionId [771305893853003777] 2023-10-30 17:34:15,585 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1} init
success 2023-10-30 17:34:15,586 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}
success 2023-10-30 17:34:15,586 INFO
org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - starting
seatunnel source split enumerator task, source name: pipeline-1
[Source[0]-LocalFile-default-identifier] 2023-10-30 17:34:15,587 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state DEPLOYING to RUNNING. 2023-10-30 17:34:15,588 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] current state
equals target state: RUNNING, skip 2023-10-30 17:34:15,588 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] state process
is sta
rt 2023-10-30 17:34:15,592 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state CREATED to DEPLOYING. 2023-10-30 17:34:15,594 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] received deploying task executionId
[771305893853069313] 2023-10-30 17:34:15,614 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying task
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3},
executionId [771305893853069313] 2023-10-30 17:34:15,614 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3} init
success 2023-10-30 17:34:15,614 INFO org.apache.seatun
nel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-280304]
[5.1] deploying TaskGroup TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=3} success 2023-10-30 17:34:15,616 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state DEPLOYING to RUNNING. 2023-10-30 17:34:15,616 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] current state
equals target state: RUNNING, skip 2023-10-30 17:34:15,616 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] state process is
start
2023-10-30 17:34:15,618 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
CREATED to DEPLOYING. 2023-10-30 17:34:15,620 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] received deploying task executionId
[771305893853134849] 2023-10-30 17:34:15,625 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Opened a connection to
metastore, current connections: 1 2023-10-30 17:34:15,625 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Connected to metastore.
2023-10-30 17:34:15,626 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Closed a connection to
metastore, current connections: 0 2023-10-30 17:34:15,650 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying task TaskGr
oupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000},
executionId [771305893853134849] 2023-10-30 17:34:15,650 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
init success 2023-10-30 17:34:15,652 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
success 2023-10-30 17:34:15,652 INFO
org.apache.seatunnel.connectors.seatunnel.file.sink.util.FileSystemUtils -
Start Kerberos authentication using principal
hive/[[email protected]](mailto:[email protected]) and keytab
/opt/kibb_conf/hive.keytab 2023-10-30 17:34:15,659 INFO
org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask - starting
seatunnel transform task, index 0 2023-10-30 17:34:15,660 I
NFO org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask - starting
seatunnel source task, index 0 2023-10-30 17:34:15,660 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
DEPLOYING to RUNNING. 2023-10-30 17:34:15,662 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] current state equals
target state: RUNNING, skip 2023-10-30 17:34:15,662 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] turned from state DEPLOYING to RUNNING.
2023-10-30 17:34:15,670 ERROR
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - report
error from task org.apache.seatunnel.common.utils.
SeaTunnelException: java.lang.NullPointerException at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.lambda$buildSchemaWithRowType$0(AbstractWriteStrategy.java:135)
at java.util.ArrayList.forEach(ArrayList.java:1259) at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.buildSchemaWithRowType(AbstractWriteStrategy.java:133)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.setSeaTunnelRowTypeInfo(TextWriteStrategy.java:74)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriteStrategy(BaseFileSink.java:123)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriter(BaseFileSink.java:90)
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:286)
at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$15(SeaTunnelTask.java:419)
at java.util.stream.ForEachOps$ForEachOp$OfRef.acce
pt(ForEachOps.java:184) at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:416)
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$null$0(NotifyTaskRestoreOperation.java:106)
at java.util.concurrent.CompletableFut
ure$AsyncRun.run(CompletableFuture.java:1640) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
>
> ```
> at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.reportCheckpointErrorFromTask(CheckpointCoordinator.java:346)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.reportCheckpointErrorFromTask(CheckpointManager.java:188)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointErrorReportOperation.run(CheckpointErrorReportOperation.java:48)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:213)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:175)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:139)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> ```
>
> 2023-10-30 17:34:15,671 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start
clean pending checkpoint cause CheckpointCoordinator inside have error.
2023-10-30 17:34:15,671 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - Turn
checkpoint_state_771305889444397057_1 state from RUNNING to FAILED 2023-10-30
17:34:15,671 WARN org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] checkpoint have error,
cancel the pipeline 2023-10-30 17:34:15,672 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] turned from state RUNNING to CANCELING.
2023-10-30 17:34:15,673 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state RUNNING to CANCELIN
G. 2023-10-30 17:34:15,674 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] operator to
member [localhost]:5801 2023-10-30 17:34:15,674 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=1}) need cancel. 2023-10-30 17:34:15,674 WARN
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 20000 -
org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask@754e9e8
2023-10-30 17:34:15,674 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 20000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}
2023-10-30 17:34:15,6
75 INFO org.apache.seatunnel.engine.server.TaskExecutionService -
[localhost]:5801 [seatunnel-280304] [5.1] Task
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}
complete with state CANCELED 2023-10-30 17:34:15,675 INFO
org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801
[seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}, state
CANCELED 2023-10-30 17:34:15,675 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state RUNNING to CANCELING. 2023-10-30 17:34:15,675 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state CANC
ELING to CANCELED. 2023-10-30 17:34:15,675 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] state process
is stopped 2023-10-30 17:34:15,676 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] future complete
with state CANCELED 2023-10-30 17:34:15,678 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] operator to
member [localhost]:5801 2023-10-30 17:34:15,678 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=77130588944
4397057, pipelineId=1, taskGroupId=3}) need cancel. 2023-10-30 17:34:15,678
WARN org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 40000 -
org.apache.seatunnel.engine.server.task.SinkAggregatedCommitterTask@26ac1977
2023-10-30 17:34:15,678 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 40000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3}
2023-10-30 17:34:15,678 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=3} complete with state CANCELED 2023-10-30
17:34:15,678 INFO org.apache.seatunnel.engine.server.CoordinatorService -
[localhost]:5801 [seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3}, stat
e CANCELED 2023-10-30 17:34:15,679 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state CANCELING to CANCELED. 2023-10-30 17:34:15,679 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] state process
is stopped 2023-10-30 17:34:15,679 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] future
complete with state CANCELED 2023-10-30 17:34:15,679 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-L
ocalFile-default-identifier]-SourceTask (1/1)] turned from state RUNNING to
CANCELING. 2023-10-30 17:34:15,680 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] operator to member
[localhost]:5801 2023-10-30 17:34:15,680 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=50000}) need cancel. 2023-10-30 17:34:15,680 WARN
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 60000 -
org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask@2f97ae14 2023-10-30
17:34:15,680 WARN org.apache.seatunnel.engine.server.TaskExecutionService -
[localhost]:5801 [seatunnel-280304] [5.1] Interrupted task 70000 -
org.apache.seatunnel.engine.s
erver.task.TransformSeaTunnelTask@4ceca4a 2023-10-30 17:34:15,680 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 60000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
2023-10-30 17:34:15,680 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 70000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
2023-10-30 17:34:15,680 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=50000} complete with state CANCELED 2023-10-30
17:34:15,681 INFO org.apache.seatunnel.engine.server.CoordinatorService -
[localhost]:5801 [seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroup
Id=50000}, state CANCELED 2023-10-30 17:34:15,681 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
CANCELING to CANCELED. 2023-10-30 17:34:15,681 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] state process is
stopped 2023-10-30 17:34:15,681 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] future complete with
state CANCELED 2023-10-30 17:34:15,681 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] will end with state FAILED 2023-10-30
17:
34:15,682 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] turned from state
CANCELING to FAILED. 2023-10-30 17:34:15,682 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Restore time 3,
pipeline Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] 2023-10-30
17:34:15,682 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan -
Reset pipeline Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] state
to CREATED 2023-10-30 17:34:15,683 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Reset pipeline Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] state to CREATED complete
2023-10-30 17:34:15,683 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turn to state
CREATED. 2023-10-30 17:34:15,684 INFO org.apa
che.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turn to state
CREATED. 2023-10-30 17:34:15,684 INFO
org.apache.hadoop.security.UserGroupInformation - Login successful for user
hive/[[email protected]](mailto:[email protected]) using
keytab file /opt/kibb_conf/hive.keytab 2023-10-30 17:34:15,684 INFO
org.apache.seatunnel.connectors.seatunnel.file.sink.util.FileSystemUtils -
Kerberos authentication successful 2023-10-30 17:34:15,684 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turn to state
CREATED. 2023-10-30 17:34:15,684 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager - reported
pipeline running stack: [java.lang.Thread.getStackTrace(Thr
ead.java:1559),
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.reportedPipelineRunning(CheckpointManager.java:175),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.prepareRestorePipeline(SubPlan.java:429),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.stateProcess(SubPlan.java:627),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.updatePipelineState(SubPlan.java:341),
org.apache.seatunnel.engine.server.dag.physical.SubPlan.lambda$addPhysicalVertexCallBack$2(SubPlan.java:203),
java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670),
java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646),
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456),
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149),
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624),
java.lang.Thread.run(Thread.java:748)] 2023-10-30 17:34:15,684 I
NFO org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator -
received restore CheckpointCoordinator with alreadyStarted= false 2023-10-30
17:34:15,684 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - Turn
checkpoint_state_771305889444397057_1 state from FAILED to RUNNING 2023-10-30
17:34:15,684 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start
clean pending checkpoint cause CheckpointCoordinator reset. 2023-10-30
17:34:15,685 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] is in state CREATED
when init state future 2023-10-30 17:34:15,685 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1
/1)] is in state CREATED when init state future 2023-10-30 17:34:15,685 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] is in state
CREATED when init state future 2023-10-30 17:34:15,685 INFO
org.apache.seatunnel.engine.server.master.JobMaster - release the pipeline Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] resource 2023-10-30
17:34:15,686 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=7, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,686 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received s
lot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=8, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,686 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=9, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,687 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] state process is start 2023-10-30
17:34:15,690 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] turned from state CREATED
to
SCHEDULED. 2023-10-30 17:34:15,691 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Trying to connect to
metastore with URI thrift://devcdp03.test.com:9083 2023-10-30 17:34:15,691 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - HMSC::open(): Could not
find delegation token. Creating KERBEROS-based thrift connection. 2023-10-30
17:34:15,691 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] turned from state
SCHEDULED to DEPLOYING. 2023-10-30 17:34:15,691 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] state process
is start 2023-10-30 17:34:15,692 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier
]-SplitEnumerator (1/1)] turned from state CREATED to DEPLOYING. 2023-10-30
17:34:15,692 INFO org.apache.seatunnel.engine.server.TaskExecutionService -
[localhost]:5801 [seatunnel-280304] [5.1] received deploying task executionId
[771305893853200385] 2023-10-30 17:34:15,703 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying task
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1},
executionId [771305893853200385] 2023-10-30 17:34:15,703 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1} init
success 2023-10-30 17:34:15,703 INFO
org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - starting
seatunnel source split enumerator task, source name: pipeline-1
[Source[0]-LocalFile-default-identifier] 2023-10-30 17:34:15,704 INFO
org.apache.seatunnel
.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-280304]
[5.1] deploying TaskGroup TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=1} success 2023-10-30 17:34:15,705 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state DEPLOYING to RUNNING. 2023-10-30 17:34:15,705 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] current state
equals target state: RUNNING, skip 2023-10-30 17:34:15,705 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] state process
is star
t 2023-10-30 17:34:15,707 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state CREATED to DEPLOYING. 2023-10-30 17:34:15,709 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] received deploying task executionId
[771305893853265921] 2023-10-30 17:34:15,723 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying task
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3},
executionId [771305893853265921] 2023-10-30 17:34:15,723 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3} init
success 2023-10-30 17:34:15,724 INFO org.apache.seatunn
el.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-280304]
[5.1] deploying TaskGroup TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=3} success 2023-10-30 17:34:15,725 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state DEPLOYING to RUNNING. 2023-10-30 17:34:15,727 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] current state
equals target state: RUNNING, skip 2023-10-30 17:34:15,727 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] state process is
start
2023-10-30 17:34:15,728 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
CREATED to DEPLOYING. 2023-10-30 17:34:15,729 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] received deploying task executionId
[771305893853331457] 2023-10-30 17:34:15,735 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Opened a connection to
metastore, current connections: 1 2023-10-30 17:34:15,736 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Connected to metastore.
2023-10-30 17:34:15,736 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Closed a connection to
metastore, current connections: 0 2023-10-30 17:34:15,766 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying task TaskGro
upLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000},
executionId [771305893853331457] 2023-10-30 17:34:15,766 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
init success 2023-10-30 17:34:15,767 INFO
org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask - starting
seatunnel transform task, index 0 2023-10-30 17:34:15,770 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] deploying TaskGroup
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
success 2023-10-30 17:34:15,771 INFO
org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask - starting
seatunnel source task, index 0 2023-10-30 17:34:15,775 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)],
task: [pipeline-1 [Source[0]-LocalFile-default-identifier]-SourceTask (1/1)]
turned from state DEPLOYING to RUNNING. 2023-10-30 17:34:15,776 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] current state equals
target state: RUNNING, skip 2023-10-30 17:34:15,781 ERROR
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - report
error from task org.apache.seatunnel.common.utils.SeaTunnelException:
java.lang.NullPointerException at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.lambda$buildSchemaWithRowType$0(AbstractWriteStrategy.java:135)
at java.util.ArrayList.forEach(ArrayList.java:1259) at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.buildSchemaWithRowType(AbstractWriteStrategy.java:133)
at org.apache.seatunnel.connectors.seatunnel.file.sin
k.writer.TextWriteStrategy.setSeaTunnelRowTypeInfo(TextWriteStrategy.java:74)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriteStrategy(BaseFileSink.java:123)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriter(BaseFileSink.java:90)
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:286)
at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$15(SeaTunnelTask.java:419)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at
java.util.stream.ForEachOps
$ForEachOp.evaluateSequential(ForEachOps.java:151) at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:416)
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$null$0(NotifyTaskRestoreOperation.java:106)
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
>
> ```
> at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.reportCheckpointErrorFromTask(CheckpointCoordinator.java:346)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.reportCheckpointErrorFromTask(CheckpointManager.java:188)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointErrorReportOperation.run(CheckpointErrorReportOperation.java:48)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:213)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:175)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:139)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
[seatunnel-starter.jar:2.3.4-SNAPSHOT]
> ```
>
> 2023-10-30 17:34:15,781 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start
clean pending checkpoint cause CheckpointCoordinator inside have error.
2023-10-30 17:34:15,782 INFO
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - Turn
checkpoint_state_771305889444397057_1 state from RUNNING to FAILED 2023-10-30
17:34:15,782 WARN org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] checkpoint have error,
cancel the pipeline 2023-10-30 17:34:15,787 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] turned from state DEPLOYING to RUNNING.
2023-10-30 17:34:15,789 INFO
org.apache.seatunnel.connectors.seatunnel.file.sink.util.FileSystemUtils -
Start Kerberos authentication using principal
hive/[[email protected]](mailto:[email protected]) and keytab
/opt/kibb_conf/hive.keytab 2023-10-30 17:3
4:15,791 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] turned from state RUNNING
to CANCELING. 2023-10-30 17:34:15,791 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] turned from
state RUNNING to CANCELING. 2023-10-30 17:34:15,792 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] operator to
member [localhost]:5801 2023-10-30 17:34:15,792 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=1}) need cancel. 2023-10-30 17:34:15,792 WARN
org.apache.seatunn
el.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-280304]
[5.1] Interrupted task 20000 -
org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask@5d1a9700
2023-10-30 17:34:15,792 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 20000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}
2023-10-30 17:34:15,792 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=1} complete with state CANCELED 2023-10-30
17:34:15,792 INFO org.apache.seatunnel.engine.server.CoordinatorService -
[localhost]:5801 [seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=1}, state
CANCELED 2023-10-30 17:34:15,792 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVerte
x - Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task:
[pipeline-1 [Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)]
turned from state RUNNING to CANCELING. 2023-10-30 17:34:15,794 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] operator to
member [localhost]:5801 2023-10-30 17:34:15,794 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=3}) need cancel. 2023-10-30 17:34:15,796 WARN
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 40000 -
org.apache.seatunnel.engine.server.task.SinkAggregatedCommitterTask@30f8f33a
2023-10-30 17:34:15,798 INFO org.apache.seatunnel.engine.server.dag.physic
al.PhysicalVertex - Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)],
task: [pipeline-1 [Source[0]-LocalFile-default-identifier]-SplitEnumerator
(1/1)] turned from state CANCELING to CANCELED. 2023-10-30 17:34:15,798 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] taskDone, taskId = 40000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3}
2023-10-30 17:34:15,798 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] state process
is stopped 2023-10-30 17:34:15,799 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SplitEnumerator (1/1)] future complete
with state CANCELED 2023-10-30 17:34:15,799 INFO or
g.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=3} complete with state CANCELED 2023-10-30
17:34:15,799 INFO org.apache.seatunnel.engine.server.CoordinatorService -
[localhost]:5801 [seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=3}, state
CANCELED 2023-10-30 17:34:15,799 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
RUNNING to CANCELING. 2023-10-30 17:34:15,800 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Send cancel
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] operator to member
[localhost]:
5801 2023-10-30 17:34:15,800 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task (TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=50000}) need cancel. 2023-10-30 17:34:15,800 WARN
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 70000 -
org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask@7e2d5f03
2023-10-30 17:34:15,800 WARN
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Interrupted task 60000 -
org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask@27c2ceb 2023-10-30
17:34:15,800 INFO org.apache.seatunnel.engine.server.TaskExecutionService -
[localhost]:5801 [seatunnel-280304] [5.1] taskDone, taskId = 70000, taskGroup =
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000}
2023-10-30 17:34:15,800 INFO org.apache.seatunnel.engine.server.TaskExecut
ionService - [localhost]:5801 [seatunnel-280304] [5.1] taskDone, taskId =
60000, taskGroup = TaskGroupLocation{jobId=771305889444397057, pipelineId=1,
taskGroupId=50000} 2023-10-30 17:34:15,801 INFO
org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801
[seatunnel-280304] [5.1] Task TaskGroupLocation{jobId=771305889444397057,
pipelineId=1, taskGroupId=50000} complete with state CANCELED 2023-10-30
17:34:15,801 INFO org.apache.seatunnel.engine.server.CoordinatorService -
[localhost]:5801 [seatunnel-280304] [5.1] Received task end from execution
TaskGroupLocation{jobId=771305889444397057, pipelineId=1, taskGroupId=50000},
state CANCELED 2023-10-30 17:34:15,801 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] turned from state
CANCELING to CANCELED. 2023-10-30 17:34:15,801 INFO
org.apache.seatunnel.engine.server
.dag.physical.PhysicalVertex - Job SeaTunnel_Job (771305889444397057),
Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] state process is
stopped 2023-10-30 17:34:15,801 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Source[0]-LocalFile-default-identifier]-SourceTask (1/1)] future complete with
state CANCELED 2023-10-30 17:34:15,804 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] turned from
state CANCELING to CANCELED. 2023-10-30 17:34:15,805 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] state process
is stopped 2023-10-
30 17:34:15,805 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan -
Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)], task: [pipeline-1
[Sink[0]-Hive-default-identifier]-AggregatedCommitterTask (1/1)] future
complete with state CANCELED 2023-10-30 17:34:15,805 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] will end with state FAILED 2023-10-30
17:34:15,806 INFO org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job
SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] turned from state
CANCELING to FAILED. 2023-10-30 17:34:15,814 INFO
org.apache.hadoop.security.UserGroupInformation - Login successful for user
hive/[[email protected]](mailto:[email protected]) using
keytab file /opt/kibb_conf/hive.keytab 2023-10-30 17:34:15,814 INFO
org.apache.seatunnel.connectors.seatunnel.file.sink.util.FileSystemUtils -
Kerberos authentication successful 2023-10-30 17:34:15,822 IN
FO org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Trying to connect to
metastore with URI thrift://devcdp03.test.com:9083 2023-10-30 17:34:15,822 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - HMSC::open(): Could not
find delegation token. Creating KERBEROS-based thrift connection. 2023-10-30
17:34:15,841 INFO org.apache.seatunnel.engine.server.master.JobMaster - release
the pipeline Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] resource
2023-10-30 17:34:15,842 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=10, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,842 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771
305889444397057, slot: SlotProfile{worker=[localhost]:5801, slotID=12,
ownerJobID=771305889444397057, assigned=true,
resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}},
sequence='dca25825-c421-4dc2-92e2-43dffb60f386'} 2023-10-30 17:34:15,842 INFO
org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received
slot release request, jobID: 771305889444397057, slot:
SlotProfile{worker=[localhost]:5801, slotID=11, ownerJobID=771305889444397057,
assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0},
heapMemory=Memory{bytes=0}}, sequence='dca25825-c421-4dc2-92e2-43dffb60f386'}
2023-10-30 17:34:15,845 INFO
org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job
(771305889444397057), Pipeline: [(1/1)] state process is stop 2023-10-30
17:34:15,846 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan
- Job SeaTunnel_Job (771305889444397057), Pipeline: [(1/1)] future complete
with state FAILED 2023-10-30 17:34:15
,846 INFO org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan -
cancel job Job SeaTunnel_Job (771305889444397057) because
makeJobEndWhenPipelineEnded is true 2023-10-30 17:34:15,847 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job
SeaTunnel_Job (771305889444397057) turned from state RUNNING to FAILING.
2023-10-30 17:34:15,848 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job
SeaTunnel_Job (771305889444397057) turned from state FAILING to FAILED.
2023-10-30 17:34:15,848 INFO
org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job
SeaTunnel_Job (771305889444397057) state process is stop 2023-10-30
17:34:15,870 INFO org.apache.seatunnel.engine.client.job.ClientJobProxy - Job
(771305889444397057) end with state FAILED 2023-10-30 17:34:15,870 INFO
com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-280304] [5.1]
HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN 2023-10-30
17:34:15,872 INFO com.hazelcast.int
ernal.server.tcp.TcpServerConnection - [localhost]:5801 [seatunnel-280304]
[5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:39690, qualifier=null,
endpoint=[127.0.0.1]:39690, remoteUuid=0c824c31-5a66-4c69-b8f6-9277bc78485b,
alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection
closed by the other side 2023-10-30 17:34:15,873 INFO
com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1
[seatunnel-280304] [5.1] Removed connection to endpoint:
[localhost]:5801:9d5798ff-4f69-484b-9489-fbd734eb370a, connection:
ClientConnection{alive=false, connectionId=1,
channel=NioChannel{/127.0.0.1:39690->localhost/127.0.0.1:5801},
remoteAddress=[localhost]:5801, lastReadTime=2023-10-30 17:34:15.868,
lastWriteTime=2023-10-30 17:34:15.241, closedTime=2023-10-30 17:34:15.871,
connected server version=5.1} 2023-10-30 17:34:15,873 INFO
com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-280304] [5.1]
HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DIS
CONNECTED 2023-10-30 17:34:15,875 INFO
com.hazelcast.client.impl.ClientEndpointManager - [localhost]:5801
[seatunnel-280304] [5.1] Destroying ClientEndpoint{connection=Connection[id=1,
/127.0.0.1:5801->/127.0.0.1:39690, qualifier=null, endpoint=[127.0.0.1]:39690,
remoteUuid=0c824c31-5a66-4c69-b8f6-9277bc78485b, alive=false,
connectionType=JVM, planeIndex=-1],
clientUuid=0c824c31-5a66-4c69-b8f6-9277bc78485b, clientName=hz.client_1,
authenticated=true, clientVersion=5.1, creationTime=1698658454088, latest
clientAttributes=lastStatisticsCollectionTime=1698658454115,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1698658454042,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=4806692864,os.freePhysicalMemorySize=521990144,os.freeSwapSpaceSize=3996905472,os.maxFileDescriptorCount=4096,os.openFileDescriptorCount=73,os.processCpuTime=4310000000,os.systemLoadAverage=0.27,os.totalPhysicalMemorySize=3973349376,
os.totalSwapSpaceSize=4160745472,runtime.availableProcessors=4,runtime.freeMemory=145390648,runtime.maxMemory=883949568,runtime.totalMemory=184025088,runtime.uptime=2634,runtime.usedMemory=38634440,
labels=[]} 2023-10-30 17:34:15,875 INFO com.hazelcast.core.LifecycleService -
hz.client_1 [seatunnel-280304] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7)
is SHUTDOWN 2023-10-30 17:34:15,875 INFO
org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand -
Closed SeaTunnel client...... 2023-10-30 17:34:15,875 INFO
com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-280304] [5.1]
[localhost]:5801 is SHUTTING_DOWN 2023-10-30 17:34:15,879 INFO
com.hazelcast.internal.partition.impl.MigrationManager - [localhost]:5801
[seatunnel-280304] [5.1] Shutdown request of Member [localhost]:5801 -
9d5798ff-4f69-484b-9489-fbd734eb370a this is handled 2023-10-30 17:34:15,879
INFO org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Opened a connection
to metastore, current
connections: 1 2023-10-30 17:34:15,879 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Connected to metastore.
2023-10-30 17:34:15,880 INFO
org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Closed a connection to
metastore, current connections: 0 2023-10-30 17:34:15,885 INFO
com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-280304] [5.1]
Shutting down connection manager... 2023-10-30 17:34:15,889 INFO
com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-280304] [5.1]
Shutting down node engine... 2023-10-30 17:34:18,078 INFO
com.hazelcast.instance.impl.NodeExtension - [localhost]:5801 [seatunnel-280304]
[5.1] Destroying node NodeExtension. 2023-10-30 17:34:18,078 INFO
com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-280304] [5.1]
Hazelcast Shutdown is completed in 2200 ms. 2023-10-30 17:34:18,079 INFO
com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-280304] [5.1]
[localhost]:5801 is SHUTDOWN 2023-10-30 17:34:18,07
9 INFO
org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand -
Closed HazelcastInstance ...... 2023-10-30 17:34:18,079 INFO
org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand -
Closed metrics executor service ...... 2023-10-30 17:34:18,079 ERROR
org.apache.seatunnel.core.starter.SeaTunnel -
>
>
===============================================================================
>
> 2023-10-30 17:34:18,079 ERROR org.apache.seatunnel.core.starter.SeaTunnel
- Fatal Error,
>
> 2023-10-30 17:34:18,079 ERROR org.apache.seatunnel.core.starter.SeaTunnel
- Please submit bug report in https://github.com/apache/seatunnel/issues
>
> 2023-10-30 17:34:18,079 ERROR org.apache.seatunnel.core.starter.SeaTunnel
- Reason:SeaTunnel job executed failed
>
> 2023-10-30 17:34:18,080 ERROR org.apache.seatunnel.core.starter.SeaTunnel
- Exception
StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException:
SeaTunnel job executed failed at
org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:190)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at
org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by:
org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException:
org.apache.seatunnel.engine.server.checkpoint.CheckpointException:
CheckpointCoordinator inside have error. at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:257)
at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:253)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.report
CheckpointErrorFromTask(CheckpointCoordinator.java:346) at
org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.reportCheckpointErrorFromTask(CheckpointManager.java:188)
at
org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointErrorReportOperation.run(CheckpointErrorReportOperation.java:48)
at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:213)
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:175)
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:139)
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:12
3) at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
Caused by: org.apache.seatunnel.common.utils.SeaTunnelException:
java.lang.NullPointerException at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.lambda$buildSchemaWithRowType$0(AbstractWriteStrategy.java:135)
at java.util.ArrayList.forEach(ArrayList.java:1259) at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.buildSchemaWithRowType(AbstractWriteStrategy.java:133)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.setSeaTunnelRowTypeInfo(TextWriteStrategy.java:74)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriteStrategy(BaseFileSink.java:123)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriter(BaseFileSink.java:90)
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:286)
at org.
apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$15(SeaTunnelTask.java:419)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:416)
at org.apache.seatunnel.eng
ine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$null$0(NotifyTaskRestoreOperation.java:106)
at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
>
> ```
> ... 11 more
>
> at
org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122)
> at
org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:183)
> ... 2 more
> ```
>
> # 2023-10-30 17:34:18,080 ERROR
org.apache.seatunnel.core.starter.SeaTunnel -
> Exception in thread "main"
org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel
job executed failed at
org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:190)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at
org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by:
org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException:
org.apache.seatunnel.engine.server.checkpoint.CheckpointException:
CheckpointCoordinator inside have error. at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:257)
at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:253)
at
org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.reportCheckpointErrorFromTask(CheckpointCoordinator.java:346)
at org.apache.
seatunnel.engine.server.checkpoint.CheckpointManager.reportCheckpointErrorFromTask(CheckpointManager.java:188)
at
org.apache.seatunnel.engine.server.checkpoint.operation.CheckpointErrorReportOperation.run(CheckpointErrorReportOperation.java:48)
at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:213)
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:175)
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:139)
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(
HazelcastManagedThread.java:102) Caused by:
org.apache.seatunnel.common.utils.SeaTunnelException:
java.lang.NullPointerException at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.lambda$buildSchemaWithRowType$0(AbstractWriteStrategy.java:135)
at java.util.ArrayList.forEach(ArrayList.java:1259) at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.AbstractWriteStrategy.buildSchemaWithRowType(AbstractWriteStrategy.java:133)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.writer.TextWriteStrategy.setSeaTunnelRowTypeInfo(TextWriteStrategy.java:74)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriteStrategy(BaseFileSink.java:123)
at
org.apache.seatunnel.connectors.seatunnel.file.sink.BaseFileSink.createWriter(BaseFileSink.java:90)
at
org.apache.seatunnel.engine.server.task.flow.SinkFlowLifeCycle.restoreState(SinkFlowLifeCycle.java:286)
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.lambda$restoreState$
15(SeaTunnelTask.java:419) at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
org.apache.seatunnel.engine.server.task.SeaTunnelTask.restoreState(SeaTunnelTask.java:416)
at
org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$null
$0(NotifyTaskRestoreOperation.java:106) at
java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1640)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
>
> ```
> ... 11 more
>
> at
org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:122)
> at
org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:183)
> ```
This is my config:
env {
execution.parallelism = 1
job.mode = "BATCH"
}
source {
Hive {
table_name = "raw.guava_test_source"
metastore_uri = "thrift://devcdp03.test.com:9083",
kerberos_principal = "hive/[email protected]",
kerberos_keytab_path = "/opt/kibb_conf/hive.keytab",
hdfs_site_path = "/opt/kibb_conf/hdfs-site.xml",
hive_site_path = "/opt/kibb_conf/hive-site.xml"
}
}
sink {
Hive {
table_name = "raw.test_et"
metastore_uri = "thrift://devcdp03.test.com:9083",
kerberos_principal = "hive/[email protected]",
kerberos_keytab_path = "/opt/kibb_conf/hive.keytab",
hdfs_site_path = "/opt/kibb_conf/hdfs-site.xml",
hive_site_path = "/opt/kibb_conf/hive-site.xml"
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
