lichaohao opened a new issue, #3586:
URL: https://github.com/apache/amoro/issues/3586
### What happened?
#my external sparkOptimizer command
export HADOOP_USER_NAME=hive && export HADOOP_CONF_DIR=/etc/hadoop/conf/ &&
export SPARK_CONF_DIR=/etc/spark3/conf/ &&
/data/hadoop22/spark-3.3.4/bin/spark-submit
--class org.apache.amoro.optimizer.spark.SparkOptimizer
--master yarn --deploy-mode=cluster --driver-memory 10g --executor-memory
30g --executor-cores 2 \
--num-executors 20 --name amoro-spark-optimizer \
--conf spark.dynamicAllocation.shuffleTracking.enabled=true \
--conf spark.executor.userClassPathFirst=true \
--conf spark.dynamicAllocation.maxExecutors=20 \
--conf spark.driver.userClassPathFirst=true \
--conf spark.dynamicAllocation.enabled=true \
--conf spark.yarn.queue=flink-amoro --proxy-user hadoop \
/data/hadoop22/master_amoro/amoro-0.8-SNAPSHOT/plugin/optimizer/spark/optimizer-job.jar
\
-a zookeeper://zyxfcdp01:2181,zyxfcdp02:2181,zyxfcdp03:2181/tancha-amoro -p
20 -g tancha-amoro
amoro-config.yaml
#High availability configuration.
ha:
enabled: true
cluster-name: tancha-amoro
zookeeper-address: zyxfcdp01:2181,zyxfcdp02:2181,zyxfcdp03:2181
ams can not connect the external sparkOptimizer
### Affects Versions
0,8.0
### What table formats are you seeing the problem on?
_No response_
### What engines are you seeing the problem on?
_No response_
### How to reproduce
_No response_
### Relevant log output
```shell
application（spark-log）
25/05/30 11:34:33 WARN client.ThriftClientPool: Maybe server is restarting,
wait a while
25/05/30 11:34:39 ERROR common.AbstractOptimizerOperator: Call ams got an
error and will try again later
org.apache.amoro.client.ThriftException: Client can not connect.
at
org.apache.amoro.client.ThriftClientPool.iface(ThriftClientPool.java:209)
at
org.apache.amoro.client.OptimizingClientPools.getClient(OptimizingClientPools.java:38)
at
org.apache.amoro.optimizer.common.AbstractOptimizerOperator.callAms(AbstractOptimizerOperator.java:53)
at
org.apache.amoro.optimizer.common.OptimizerToucher.checkToken(OptimizerToucher.java:74)
at
org.apache.amoro.optimizer.common.OptimizerToucher.start(OptimizerToucher.java:59)
at
org.apache.amoro.optimizer.common.Optimizer.startOptimizing(Optimizer.java:65)
at
org.apache.amoro.optimizer.spark.SparkOptimizer.main(SparkOptimizer.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:737)
25/05/30 11:34:44 WARN client.ThriftClientPool: Maybe server is restarting,
wait a while
25/05/30 11:34:46 WARN client.ThriftClientPool: Maybe server is restarting,
wait a while
25/05/30 11:34:46 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint:
Registered executor NettyRpcEndpointRef(spark-
```
### Anything else
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
