chenyz1984 opened a new issue, #8688: URL: https://github.com/apache/seatunnel/issues/8688
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened 为 checkpoint 配置 HDFS 持久化存储（启用Namenode HA）后，执行任务时报错“org.apache.seatunnel.engine.common.exception.JobException: java.lang.IllegalArgumentException: java.net.UnknownHostException: sybdata” ### SeaTunnel Version 2.3.9 ### SeaTunnel Config ```conf seatunnel.yaml的配置如下所示（其中sybdata为namenode ha的名称）： seatunnel: engine: http: enable-http: true port: 8090 context-path: /seatunnel history-job-expire-minutes: 1440 # 历史作业过期时长（分钟） backup-count: 1 # Imap 数据的副本数，2 表示每个数据会同时存储在2个不同的节点中。因 Master 宕机而不足时，自动补齐 queue-type: blockingqueue classloader-cache-mode: true print-execution-info-interval: 60 print-job-metrics-info-interval: 60 slot-service: # Slot 数量决定了 Worker 节点可以并行运行的任务组数量。此配置对 Master 节点无效，只对 Worker 节点有效。 dynamic-slot: false # true 表示不限制 slot 数量（自动）。建议将 dynamic-slot 设置为 false，同时增加 slot-num 并设置为 'CPU核心数 x2'，以防止 OOM slot-num: 128 # 将 dynamic-slot 设置为 false，同时增加 slot-num 并设置为 'CPU 核心数 x2'，以防止 OOM job-schedule-strategy: WAIT # 当资源不足时，作业的调度策略：WAIT 等待资源可用；REJECT 拒绝作业（默认配置）。当 dynamic-slot 为 true 时，此配置强制为 REJECT。 checkpoint: interval: 300000 timeout: 120000 storage: type: hdfs max-retained: 3 plugin-config: namespace: /seatunnel/checkpoint/ # 持久化存储 checkpoint 的路径，必须以 / 结尾 storage.type: hdfs fs.defaultFS: hdfs://sybdata # Ensure that the directory has written permission seatunnel.hadoop.dfs.nameservices: sybdata # sybdata seatunnel.hadoop.dfs.ha.namenodes.sybdata: nn1,nn2 seatunnel.hadoop.dfs.namenode.rpc-address.sybdatann1: h77005:8020 seatunnel.hadoop.dfs.namenode.rpc-address.sybdatann2: h77006:8020 seatunnel.hadoop.dfs.client.failover.proxy.provider.sybdata: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider map: # IMap 数据持久化设置 engine*: map-store: enabled: true initial-mode: EAGER factory-class-name: org.apache.seatunnel.engine.server.persistence.FileMapStoreFactory properties: type: hdfs # 持久化存储类型，仅支持 hdfs namespace: /seatunnel/imap # 持久化存储 imap 的路径 clusterName: # 用于区分不同的 seatunnel 集群 storage.type: hdfs fs.defaultFS: hdfs://sybdata # 若单点运行 Master 可设置为 file:/// 以使用本地存储 ``` ### Running Command ```shell ${SEATUNNEL_HOME}/bin/seatunnel.sh --config /tmp/dolphinscheduler/exec/process/seatunnel/132587039644032/132927532342660_63/65/646/seatunnel_65_646.conf --deploy-mode cluster -n BAT_JIUYE_AC01_PK -i BY_DAY_BEGIN=${BY_DAY_BEGIN} -i BY_DAY_END=${BY_DAY_END} ``` ### Error Exception ```log 2025-02-13 15:20:50,492 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:228) at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40) at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34) Caused by: java.util.concurrent.CompletionException: org.apache.seatunnel.engine.common.exception.JobException: java.lang.IllegalArgumentException: java.net.UnknownHostException: sybdata at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:445) at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:140) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:356) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:290) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:172) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:227) at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.initStorage(HdfsStorage.java:68) at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.<init>(HdfsStorage.java:57) at org.apache.seatunnel.engine.checkpoint.storage.hdfs.common.HdfsFileStorageInstance.getOrCreateStorage(HdfsFileStorageInstance.java:53) at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorageFactory.create(HdfsStorageFactory.java:74) at org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.<init>(CheckpointManager.java:107) at org.apache.seatunnel.engine.server.master.JobMaster.initCheckPointManager(JobMaster.java:301) at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:284) at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$6(CoordinatorService.java:649) at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.net.UnknownHostException: sybdata ... 25 more at com.hazelcast.spi.impl.AbstractInvocationFuture.wrapInCompletionException(AbstractInvocationFuture.java:1347) at com.hazelcast.spi.impl.AbstractInvocationFuture.cascadeException(AbstractInvocationFuture.java:1340) at com.hazelcast.spi.impl.AbstractInvocationFuture.access$200(AbstractInvocationFuture.java:65) at com.hazelcast.spi.impl.AbstractInvocationFuture$ApplyNode.execute(AbstractInvocationFuture.java:1478) at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockOtherNode(AbstractInvocationFuture.java:797) at com.hazelcast.spi.impl.AbstractInvocationFuture.unblockAll(AbstractInvocationFuture.java:759) at com.hazelcast.spi.impl.AbstractInvocationFuture.complete0(AbstractInvocationFuture.java:1235) at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionallyInternal(AbstractInvocationFuture.java:1223) at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionally(AbstractInvocationFuture.java:709) at com.hazelcast.client.impl.spi.impl.ClientInvocation.completeExceptionally(ClientInvocation.java:294) at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyExceptionWithOwnedPermission(ClientInvocation.java:321) at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyException(ClientInvocation.java:304) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.handleResponse(ClientResponseHandlerSupplier.java:164) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.process(ClientResponseHandlerSupplier.java:141) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier.access$300(ClientResponseHandlerSupplier.java:60) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:251) at com.hazelcast.client.impl.spi.impl.ClientResponseHandlerSupplier$DynamicResponseHandler.accept(ClientResponseHandlerSupplier.java:243) at com.hazelcast.client.impl.connection.tcp.TcpClientConnection.handleClientMessage(TcpClientConnection.java:245) at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.handleMessage(ClientMessageDecoder.java:135) at com.hazelcast.client.impl.protocol.util.ClientMessageDecoder.onRead(ClientMessageDecoder.java:89) at com.hazelcast.internal.networking.nio.NioInboundPipeline.process(NioInboundPipeline.java:136) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKey(NioThread.java:383) at com.hazelcast.internal.networking.nio.NioThread.processSelectionKeys(NioThread.java:368) at com.hazelcast.internal.networking.nio.NioThread.selectLoop(NioThread.java:294) at com.hazelcast.internal.networking.nio.NioThread.executeRun(NioThread.java:249) at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) Caused by: org.apache.seatunnel.engine.common.exception.JobException: java.lang.IllegalArgumentException: java.net.UnknownHostException: sybdata at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:445) at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:140) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:356) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:290) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:172) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:227) at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.initStorage(HdfsStorage.java:68) at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage.<init>(HdfsStorage.java:57) at org.apache.seatunnel.engine.checkpoint.storage.hdfs.common.HdfsFileStorageInstance.getOrCreateStorage(HdfsFileStorageInstance.java:53) at org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorageFactory.create(HdfsStorageFactory.java:74) at org.apache.seatunnel.engine.server.checkpoint.CheckpointManager.<init>(CheckpointManager.java:107) at org.apache.seatunnel.engine.server.master.JobMaster.initCheckPointManager(JobMaster.java:301) at org.apache.seatunnel.engine.server.master.JobMaster.init(JobMaster.java:284) at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$6(CoordinatorService.java:649) at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.net.UnknownHostException: sybdata ... 25 more at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$6(CoordinatorService.java:656) at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2025-02-13 15:20:50,492 ERROR org.apache.seatunnel.core.starter.SeaTunnel - ``` ### Zeta or Flink or Spark Version Zeta 2.3.9 ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
