rmetzger commented on pull request #10964:
URL: https://github.com/apache/flink/pull/10964#issuecomment-734166399
Are you guys sure this works as expected?
```
[root@sandbox-hdp flink-1.12.0]# export HADOOP_CLASSPATH=`hadoop classpath`
[root@sandbox-hdp flink-1.12.0]# yarn application -list
WARNING: YARN_LOGFILE has been replaced by HADOOP_LOGFILE. Using value of
YARN_LOGFILE.
WARNING: YARN_PID_DIR has been replaced by HADOOP_PID_DIR. Using value of
YARN_PID_DIR.
20/11/26 08:56:08 INFO client.RMProxy: Connecting to ResourceManager at
sandbox-hdp.hortonworks.com/172.18.0.2:8050
20/11/26 08:56:10 INFO client.AHSProxy: Connecting to Application History
server at sandbox-hdp.hortonworks.com/172.18.0.2:10200
Total number of applications (application-types: [], states: [SUBMITTED,
ACCEPTED, RUNNING] and tags: []):1
Application-Id Application-Name
Application-Type User Queue State
Final-State Progress Tracking-URL
application_1606316139480_0003 Flink session cluster Apache
Flink root default RUNNING
UNDEFINED 100%http://sandbox-hdp.hortonworks.com:33009
[root@sandbox-hdp flink-1.12.0]# echo "stop" | ./bin/yarn-session.sh -id
application_1606316139480_0003
Setting HBASE_CONF_DIR=/etc/hbase/conf because no HBASE_CONF_DIR was set.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/root/flink-1.12.0/lib/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/hdp/3.0.1.0-187/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type
[org.apache.logging.slf4j.Log4jLoggerFactory]
2020-11-26 08:56:36,089 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.address, localhost
2020-11-26 08:56:36,134 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.port, 6123
2020-11-26 08:56:36,134 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.memory.process.size, 1600m
2020-11-26 08:56:36,134 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.memory.process.size, 1728m
2020-11-26 08:56:36,134 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.numberOfTaskSlots, 1
2020-11-26 08:56:36,134 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: parallelism.default, 1
2020-11-26 08:56:36,134 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.execution.failover-strategy, region
2020-11-26 08:56:36,694 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli
[] - Found Yarn properties file under /tmp/.yarn-properties-root.
2020-11-26 08:56:37,191 WARN org.apache.flink.runtime.util.HadoopUtils
[] - Could not find Hadoop configuration via any of the supported
methods (Flink configuration, environment variables).
2020-11-26 08:56:38,042 WARN org.apache.hadoop.util.NativeCodeLoader
[] - Unable to load native-hadoop library for your platform...
using builtin-java classes where applicable
2020-11-26 08:56:38,791 INFO
org.apache.flink.runtime.security.modules.HadoopModule [] - Hadoop user
set to root (auth:SIMPLE)
2020-11-26 08:56:38,847 INFO
org.apache.flink.runtime.security.modules.JaasModule [] - Jaas file
will be created as /tmp/jaas-9102704015477031274.conf.
2020-11-26 08:56:39,085 WARN
org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The
configuration directory ('/root/flink-1.12.0/conf') already contains a LOG4J
config file.If you want to use logback, then please delete or rename the log
configuration file.
2020-11-26 08:56:40,396 INFO org.apache.hadoop.yarn.client.RMProxy
[] - Connecting to ResourceManager at
sandbox-hdp.hortonworks.com/172.18.0.2:8050
2020-11-26 08:56:41,643 INFO org.apache.hadoop.yarn.client.AHSProxy
[] - Connecting to Application History server at
sandbox-hdp.hortonworks.com/172.18.0.2:10200
2020-11-26 08:56:41,943 INFO
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils [] - The derived
from fraction jvm overhead memory (160.000mb (167772162 bytes)) is less than
its min value 192.000mb (201326592 bytes), min value will be used instead
2020-11-26 08:56:42,189 INFO
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils [] - The derived
from fraction jvm overhead memory (172.800mb (181193935 bytes)) is less than
its min value 192.000mb (201326592 bytes), min value will be used instead
2020-11-26 08:56:43,186 INFO org.apache.hadoop.conf.Configuration
[] - found resource resource-types.xml at
file:/etc/hadoop/3.0.1.0-187/0/resource-types.xml
2020-11-26 08:56:43,336 WARN org.apache.flink.yarn.YarnClusterDescriptor
[] - Neither the HADOOP_CONF_DIR nor the YARN_CONF_DIR
environment variable is set. The Flink YARN Client needs one of these to be set
to properly load the Hadoop configuration for accessing YARN.
2020-11-26 08:56:43,736 INFO org.apache.flink.yarn.YarnClusterDescriptor
[] - The configured JobManager memory is 1600 MB. YARN will
allocate 1792 MB to make up an integer multiple of its minimum allocation
memory (256 MB, configured via 'yarn.scheduler.minimum-allocation-mb'). The
extra 192 MB may not be used by Flink.
2020-11-26 08:56:43,736 INFO org.apache.flink.yarn.YarnClusterDescriptor
[] - The configured TaskManager memory is 1728 MB. YARN will
allocate 1792 MB to make up an integer multiple of its minimum allocation
memory (256 MB, configured via 'yarn.scheduler.minimum-allocation-mb'). The
extra 64 MB may not be used by Flink.
2020-11-26 08:56:43,737 INFO org.apache.flink.yarn.YarnClusterDescriptor
[] - Cluster specification:
ClusterSpecification{masterMemoryMB=1600, taskManagerMemoryMB=1728,
slotsPerTaskManager=1}
2020-11-26 08:56:48,697 WARN
org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2020-11-26 08:57:06,492 INFO
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils [] - The derived
from fraction jvm overhead memory (160.000mb (167772162 bytes)) is less than
its min value 192.000mb (201326592 bytes), min value will be used instead
2020-11-26 08:57:06,687 INFO org.apache.flink.yarn.YarnClusterDescriptor
[] - Submitting application master application_1606316139480_0005
2020-11-26 08:57:07,240 INFO
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl [] - Submitted
application application_1606316139480_0005
2020-11-26 08:57:07,240 INFO org.apache.flink.yarn.YarnClusterDescriptor
[] - Waiting for the cluster to be allocated
2020-11-26 08:57:07,289 INFO org.apache.flink.yarn.YarnClusterDescriptor
[] - Deploying cluster, current state ACCEPTED
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
