Hi everyone, I'm running a YARN session on a cluster with one master and one core and would like to use the Monitoring API programmatically to submit jobs. I have found that the configuration variables are read but ignored when starting the session - it seems to choose a random port each run.
Here's a snippet from the startup logs: 2018-09-06 21:44:38,763 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: env.yarn.conf.dir, /etc/hadoop/conf 2018-09-06 21:44:38,764 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: env.hadoop.conf.dir, /etc/hadoop/conf 2018-09-06 21:44:38,765 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: rest.port, 44477 2018-09-06 21:44:38,765 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: jobmanager.web.port, 44477 2018-09-06 21:44:38,765 INFO org.apache.flink.configuration.GlobalConfiguration - Loading configuration property: high-availability.jobmanager.port, 44477 2018-09-06 21:44:38,775 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-hadoop. 2018-09-06 21:44:39,615 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2018-09-06 21:44:39,799 INFO org.apache.flink.runtime.security.modules.HadoopModule - Hadoop user set to hadoop (auth:SIMPLE) 2018-09-06 21:44:40,045 INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at ip-10-2-3-71.ec2.internal/10.2.3.71:8032 2018-09-06 21:44:40,312 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Cluster specification: ClusterSpecification{masterMemoryMB=1024, taskManagerMemoryMB=4096, numberTaskManagers=1, slotsPerTaskManager=1} 2018-09-06 21:44:43,564 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Submitting application master application_1536250520330_0007 2018-09-06 21:44:43,802 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1536250520330_0007 2018-09-06 21:44:43,802 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Waiting for the cluster to be allocated 2018-09-06 21:44:43,804 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying cluster, current state ACCEPTED 2018-09-06 21:44:48,326 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN application has been deployed successfully. 2018-09-06 21:44:48,326 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - The Flink YARN client has been started in detached mode. In order to stop Flink on YARN, use the following command or a YARN web interface to stop it: yarn application -kill application_1536250520330_0007 Please also note that the temporary files of the YARN session in the home directory will not be removed. 2018-09-06 21:44:48,821 INFO org.apache.flink.runtime.rest.RestClient - Rest client endpoint started. Flink JobManager is now running on ip-10-2-3-25.ec2.internal:38683 with leader id 00000000-0000-0000-0000-000000000000. JobManager Web Interface: http://ip-10-2-3-25.ec2.internal:38683 I'm setting both the rest.port and jobmanager.web.port, but both are ignored. Has anyone seen this before? Thanks!