HSLife1991 opened a new issue, #6937: URL: https://github.com/apache/seatunnel/issues/6937
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened when i use deploy seatunnel 2.3.5 with cluster mode, configured hdfs ha for seatunnel.yaml and hazelcast.yaml. i have put the jar file "seatunnel-hadoop3-3.1.4-uber.jar" in lib directory. But it reports below error: Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.conf.Configuration.getTimeDuration(Ljava/lang/String;JLjava/util/concurrent/TimeUnit;Ljava/util/concurrent/TimeUnit;)J at org.apache.hadoop.hdfs.client.impl.DfsClientConf.<init>(DfsClientConf.java:248) 。 ### SeaTunnel Version 2.3.5 ### SeaTunnel Config ```conf #seatunnel.yaml seatunnel: engine: history-job-expire-minutes: 1440 backup-count: 1 queue-type: blockingqueue print-execution-info-interval: 60 print-job-metrics-info-interval: 60 slot-service: dynamic-slot: true checkpoint: interval: 10000 timeout: 60000 storage: type: hdfs max-retained: 3 plugin-config: namespace: /tmp/seatunnel/checkpoint_snapshot storage.type: hdfs fs.defaultFS: hdfs://fmxprod # Ensure that the directory has written permission hdfs_site_path: /usr/hdp/3.1.4.0-315/hadoop/etc/hadoop/hdfs-site.xml seatunnel.hadoop.dfs.nameservices: fmxprod seatunnel.hadoop.dfs.ha.namenodes.fmxprod: nn1,nn2 seatunnel.hadoop.dfs.namenode.rpc-address.fmxprod.nn1: bc-gcp-sg-ol-node01.c.proj-bc.internal:8020 seatunnel.hadoop.dfs.namenode.rpc-address.fmxprod.nn2: bc-gcp-sg-ol-node02.c.proj-bc.internal:8020 seatunnel.hadoop.dfs.client.failover.proxy.provider.fmxprod: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider # hazelcast.yaml hazelcast: cluster-name: seatunnel network: rest-api: enabled: true endpoint-groups: CLUSTER_WRITE: enabled: true DATA: enabled: true join: tcp-ip: enabled: true member-list: - bc-gcp-sg-ol-node03.c.proj-bc.internal - bc-gcp-sg-ol-node04.c.proj-bc.internal - bc-gcp-sg-ol-node05.c.proj-bc.internal port: auto-increment: false port: 5801 properties: hazelcast.invocation.max.retry.count: 20 hazelcast.tcp.join.port.try.count: 30 hazelcast.logging.type: log4j2 hazelcast.operation.generic.thread.count: 50 map: engine*: map-store: enabled: true initial-mode: EAGER factory-class-name: org.apache.seatunnel.engine.server.persistence.FileMapStoreFactory properties: type: hdfs namespace: /tmp/seatunnel/imap clusterName: seatunnel storage.type: hdfs fs.defaultFS: hdfs://fmxprod # Ensure that the directory has written permission hdfs_site_path: /usr/hdp/3.1.4.0-315/hadoop/etc/hadoop/hdfs-site.xml seatunnel.hadoop.dfs.nameservices: fmxprod seatunnel.hadoop.dfs.ha.namenodes.fmxprod: nn1,nn2 seatunnel.hadoop.dfs.namenode.rpc-address.fmxprod.nn1: bc-gcp-sg-ol-node01.c.proj-bc.internal:8020 seatunnel.hadoop.dfs.namenode.rpc-address.fmxprod.nn2: bc-gcp-sg-ol-node02.c.proj-bc.internal:8020 seatunnel.hadoop.dfs.client.failover.proxy.provider.fmxprod: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider # setunnelEngine client配置 hazelcast-client: cluster-name: seatunnel properties: hazelcast.logging.type: log4j2 network: cluster-members: - bc-gcp-sg-ol-node03.c.proj-bc.internal:5801 - bc-gcp-sg-ol-node04.c.proj-bc.internal:5801 - bc-gcp-sg-ol-node05.c.proj-bc.internal:5801 ``` ### Running Command ```shell $SEATUNNEL_HOME/bin/seatunnel-cluster.sh -d ``` ### Error Exception ```log 2024-06-03 19:32:19,061 WARN [c.h.s.i.p.ProxyService ] [pool-5-thread-1] - [bc-gcp-sg-ol-node03.c.proj-bc.internal]:5801 [seatunnel] [5.1] Error while initializing proxy: IMap{name='engine_runningJobInfo'} com.hazelcast.core.HazelcastException: java.lang.NoSuchMethodError: org.apache.hadoop.conf.Configuration.getTimeDuration(Ljava/lang/String;JLjava/util/concurrent/TimeUnit;Ljava/util/concurrent/TimeUnit;)J at com.hazelcast.internal.util.ExceptionUtil.lambda$static$0(ExceptionUtil.java:47) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.ExceptionUtil.peel(ExceptionUtil.java:125) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.ExceptionUtil.peel(ExceptionUtil.java:115) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.ExceptionUtil.peel(ExceptionUtil.java:68) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.internal.util.ExceptionUtil.rethrow(ExceptionUtil.java:130) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.proxy.MapProxySupport.waitUntilLoaded(MapProxySupport.java:775) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.proxy.MapProxyImpl.waitUntilLoaded(MapProxyImpl.java:112) ~[seatunnel-starter.jar:2.3.5] at com.hazelcast.map.impl.proxy.MapProxySupport.initializeMapStoreLoad(MapProxySupport.java:330) ~[seatunnel-starter.jar:2.3.5] ``` ### Zeta or Flink or Spark Version use default zeta ### Java or Scala Version java1.8 ### Screenshots _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
