[
https://issues.apache.org/jira/browse/FLINK-21264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278540#comment-17278540
]
tonychan edited comment on FLINK-21264 at 2/4/21, 5:19 AM:
-----------------------------------------------------------
this is the summit code
./bin/flink run -m yarn-cluster -yjm 2048 -ytm 8086
./examples/streaming/SocketWindowWordCount.jar --hostname ip --port 9998
the yarn log :
2021-02-03 11:03:11,334 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - ----------------
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - ip:2181
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - /flink
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - 9250-9290
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - cxf.identify1
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - ----------------
2021-02-03 11:03:11,341 INFO
org.apache.flink.metrics.prometheus.PrometheusReporter [] - Started
PrometheusReporter HTTP server on port 9251.
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf,
built on 03/06/2019 16:18 GMT
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:host.name=host.rdc.com
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.version=1.8.0_241
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.vendor=Oracle Corporation
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.home=/usr/java/jdk1.8.0_241/jre
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.class.path=:SocketWindowWordCount.jar:lib/flink-csv-1.11.2.jar:lib/flink-json-1.11.2.jar:lib/flink-shaded-hadoop-2-uber-2.6.5-10.0.jar:lib/flink-shaded-zookeeper-3.4.14.jar:lib/flink-table-blink_2.11-1.11.2.jar:lib/flink-table_2.11-1.11.2.jar:lib/log4j-1.2-api-2.12.1.jar:lib/log4j-api-2.12.1.jar:lib/log4j-core-2.12.1.jar:lib/log4j-slf4j-impl-2.12.1.jar:flink-dist_2.11-1.11.2.jar:job.graph:flink-conf.yaml::/etc/hadoop/conf.cloudera.yarn:/run/cloudera-scm-agent/process/1071-yarn-NODEMANAGER:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-annotations.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-auth.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-aws.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-azure-datalake.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-nfs.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-nfs-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common-2.6.0-cdh5.16.2-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-azure-datalake-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-aws-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-auth-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-annotations-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-format.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-format-sources.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-format-javadoc.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-tools.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-thrift.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-test-hadoop2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-scrooge_2.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-scala_2.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-protobuf.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-pig.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-pig-bundle.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-jackson.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-hadoop.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-generator.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-encoding.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-common.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-column.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-cascading.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-avro.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/hue-plugins-3.9.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/curator-client-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/gson-2.2.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/curator-recipes-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/httpclient-4.2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/curator-framework-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/hamcrest-core-1.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jetty-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/httpcore-4.2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jersey-json-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jackson-xc-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/zookeeper.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/avro.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/slf4j-api-1.7.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/netty-3.10.5.Final.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/logredactor-1.0.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jets3t-0.9.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-digester-1.8.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-beanutils-1.9.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/azure-data-lake-store-sdk-2.2.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/aws-java-sdk-bundle-1.11.134.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/api-util-1.0.0-M20.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/mockito-all-1.8.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/junit-4.11.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jsch-0.1.42.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jackson-jaxrs-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-2.6.0-cdh5.16.2-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/netty-3.10.5.Final.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.10-cloudera.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-api.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-client.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-common.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-registry.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-tests-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-common-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-registry-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-common-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-client-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-api-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/spark-yarn-shuffle.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/spark-1.6.0-cdh5.16.2-yarn-shuffle.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jline-2.11.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jetty-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/javax.inject-1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-xc-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.10-cloudera.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-jaxrs-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-core-asl-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/guice-3.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/zookeeper.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/xz-1.0.jar
2021-02-03 11:03:11,347 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.library.path=:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.io.tmpdir=/tmp
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.compiler=<NA>
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:os.name=Linux
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:os.arch=amd64
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:os.version=3.10.0-862.11.6.el7.x86_64
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:user.name=yarn
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:user.home=/var/lib/hadoop-yarn
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:user.dir=/mnt/disk2/yarn/nm/usercache/root/appcache/application_1608618104524_0555/container_e04_1608618104524_0555_01_000001
2021-02-03 11:03:11,349 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] -
Initiating client connection, connectString=10.20.30.113:2181
sessionTimeout=10000
watcher=org.apache.flink.metrics.prometheus.PrometheusReporter$1@2b0b4d53
2021-02-03 11:03:11,367 WARN
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - SASL
configuration failed: javax.security.auth.login.LoginException: No JAAS
configuration section named 'Client' was found in specified JAAS configuration
file:
'/mnt/disk1/yarn/nm/usercache/root/appcache/application_1608618104524_0555/jaas-697756762480862695.conf'.
Will continue connection to Zookeeper server without SASL authentication, if
Zookeeper server allows it.
2021-02-03 11:03:11,370 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Opening
socket connection to server bdp-4.rdc.com/ip:2181
2021-02-03 11:03:11,370 INFO
org.apache.flink.metrics.prometheus.PrometheusReporter [] -
zookeeperip:2181触发了None事件
2021-02-03 11:03:11,374 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Socket
connection established to host/ip:2181, initiating session
2021-02-03 11:03:11,380 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Session
establishment complete on server host/ip:2181, sessionid = 0x100e4bb743b0193,
negotiated timeout = 10000
2021-02-03 11:03:11,380 INFO
org.apache.flink.metrics.prometheus.PrometheusReporter [] -
zookeeperip:2181触发了None事件
2021-02-03 11:03:11,390 INFO
org.apache.flink.runtime.metrics.MetricRegistryImpl [] - Reporting metrics for
reporter prom of type org.apache.flink.metrics.prometheus.PrometheusReporter.
2021-02-03 11:03:11,397 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Trying to start
actor system, external address host.rdc.com:0, bind address 0.0.0.0:0.
2021-02-03 11:03:11,416 INFO akka.event.slf4j.Slf4jLogger [] - Slf4jLogger
started
2021-02-03 11:03:11,420 INFO akka.remote.Remoting [] - Starting remoting
2021-02-03 11:03:11,431 INFO akka.remote.Remoting [] - Remoting started;
listening on addresses :[akka.tcp://flink-metr...@host.rdc.com:45458]
2021-02-03 11:03:11,468 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Actor system started
at akka.tcp://flink-metr...@host.rdc.com:45458
2021-02-03 11:03:11,484 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for
org.apache.flink.runtime.metrics.dump.MetricQueryService at
akka://flink-metrics/user/rpc/MetricQueryService .
2021-02-03 11:03:11,530 WARN org.apache.flink.configuration.Configuration [] -
Config uses deprecated configuration key 'web.port' instead of proper key
'rest.bind-port'
2021-02-03 11:03:11,532 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Upload
directory /tmp/flink-web-abe884bc-3165-4799-99b9-870d5641194a/flink-web-upload
does not exist.
2021-02-03 11:03:11,532 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Created
directory /tmp/flink-web-abe884bc-3165-4799-99b9-870d5641194a/flink-web-upload
for file uploads.
2021-02-03 11:03:11,547 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Starting
rest endpoint.
2021-02-03 11:03:11,824 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils [] - Determined location of
main cluster component log file:
/mnt/disk1/yarn/container-logs/application_1608618104524_0555/container_e04_1608618104524_0555_01_000001/jobmanager.log
2021-02-03 11:03:11,824 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils [] - Determined location of
main cluster component stdout file:
/mnt/disk1/yarn/container-logs/application_1608618104524_0555/container_e04_1608618104524_0555_01_000001/jobmanager.out
2021-02-03 11:03:12,015 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Rest
endpoint listening at host.rdc.com:40148
2021-02-03 11:03:12,016 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
http://host.rdc.com:40148 was granted leadership with
leaderSessionID=00000000-0000-0000-0000-000000000000
2021-02-03 11:03:12,018 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Web frontend
listening at http://host.rdc.com:40148.
2021-02-03 11:03:12,082 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at
akka://flink/user/rpc/resourcemanager_0 .
2021-02-03 11:03:12,091 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: taskmanager.memory.process.size, 8086 mb
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: metrics.reporter.prom.port, 9250-9290
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: metrics.reporter.prom.identify,
cxf.identify1
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: internal.jobgraph-path, job.graph
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: metrics.reporter.prom.zookeeperPath, /flink
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: high-availability.cluster-id,
application_1608618104524_0555
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: jobmanager.rpc.address, localhost
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: execution.target, yarn-per-job
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: jobmanager.memory.process.size, 2 gb
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: jobmanager.rpc.port, 6123
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property:
execution.savepoint.ignore-unclaimed-state, false
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: execution.attached, true
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: internal.cluster.execution-mode, NORMAL
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: execution.shutdown-on-attached-exit, false
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: pipeline.jars,
file:/opt/cxf/flink/./examples/streaming/SocketWindowWordCount.jar
2021-02-03 11:03:12,092 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: parallelism.default, 1
2021-02-03 11:03:12,093 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: taskmanager.numberOfTaskSlots, 1
2021-02-03 11:03:12,093 WARN org.apache.flink.configuration.GlobalConfiguration
[] - Error while trying to split key and value in configuration file
/mnt/disk2/yarn/nm/usercache/root/appcache/application_1608618104524_0555/container_e04_1608618104524_0555_01_000001/flink-conf.yaml:18:
"pipeline.classpaths: "
2021-02-03 11:03:12,093 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: metrics.reporter.prom.class,
org.apache.flink.metrics.prometheus.PrometheusReporter
2021-02-03 11:03:12,093 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: metrics.reporter.prom.zookeeper, ip:2181
2021-02-03 11:03:12,093 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: $internal.deployment.config-dir,
/opt/cxf/flink/conf
2021-02-03 11:03:12,093 INFO org.apache.flink.configuration.GlobalConfiguration
[] - Loading configuration property: $internal.yarn.log-config-file,
/opt/cxf/flink/conf/log4j.properties
2021-02-03 11:03:12,127 INFO
org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled
external resources: []
2021-02-03 11:03:12,176 INFO
org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcess [] -
Start JobDispatcherLeaderProcess.
2021-02-03 11:03:12,182 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for
org.apache.flink.runtime.dispatcher.MiniDispatcher at
akka://flink/user/rpc/dispatcher_1 .
2021-02-03 11:03:12,226 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at
akka://flink/user/rpc/jobmanager_2 .
2021-02-03 11:03:12,234 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Initializing job Socket Window WordCount (78c332655ca97966303a64b07cfdf433).
2021-02-03 11:03:12,250 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Using restart back off time strategy NoRestartBackoffTimeStrategy for Socket
Window WordCount (78c332655ca97966303a64b07cfdf433).
2021-02-03 11:03:12,285 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Running initialization on master for job Socket Window WordCount
(78c332655ca97966303a64b07cfdf433).
2021-02-03 11:03:12,285 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Successfully ran initialization on master in 0 ms.
2021-02-03 11:03:12,302 INFO
org.apache.flink.runtime.scheduler.adapter.DefaultExecutionTopology [] - Built
1 pipelined regions in 1 ms
2021-02-03 11:03:12,315 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
No state backend has been configured, using default (Memory / JobManager)
MemoryStateBackend (data in heap memory / checkpoints to JobManager)
(checkpoints: 'null', savepoints: 'null', asynchronous: TRUE, maxStateSize:
5242880)
2021-02-03 11:03:12,335 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Using failover strategy
org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy@36d087c1
for Socket Window WordCount (78c332655ca97966303a64b07cfdf433).
2021-02-03 11:03:12,339 INFO
org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl [] - JobManager runner
for job Socket Window WordCount (78c332655ca97966303a64b07cfdf433) was granted
leadership with session id 00000000-0000-0000-0000-000000000000 at
akka.tcp://fl...@host-3.rdc.com:36810/user/rpc/jobmanager_2.
2021-02-03 11:03:12,350 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Starting execution of job Socket Window WordCount
(78c332655ca97966303a64b07cfdf433) under job master id
00000000000000000000000000000000.
2021-02-03 11:03:12,354 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Starting scheduling with scheduling strategy
[org.apache.flink.runtime.scheduler.strategy.EagerSchedulingStrategy]
2021-02-03 11:03:12,355 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Socket Window
WordCount (78c332655ca97966303a64b07cfdf433) switched from state CREATED to
RUNNING.
2021-02-03 11:03:12,365 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Socket
Stream -> Flat Map (1/1) (6d75dc1fadd5d7190c4471a573ad93f0) switched from
CREATED to SCHEDULED.
2021-02-03 11:03:12,365 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] -
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(426d3d4c23b0ba189ead74465076cf15) switched from CREATED to SCHEDULED.
2021-02-03 11:03:12,376 INFO org.apache.flink.yarn.YarnResourceManager [] -
Recovered 0 containers from previous attempts ([]).
2021-02-03 11:03:12,378 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Cannot serve slot
request, no ResourceManager connected. Adding as pending request [SlotRequestId
{5c3e8039f959fcf109bac0a8833592c7}]
2021-02-03 11:03:12,378 INFO org.apache.flink.yarn.YarnResourceManager [] -
Register application master response contains scheduler resource types:
[MEMORY, CPU].
2021-02-03 11:03:12,379 INFO org.apache.flink.yarn.YarnResourceManager [] -
Container matching strategy: MATCH_VCORE.
2021-02-03 11:03:12,385 INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Upper bound
of the thread pool size is 500
2021-02-03 11:03:12,387 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Connecting to ResourceManager
akka.tcp://fl...@host3.rdc.com:36810/user/rpc/resourcemanager_*(00000000000000000000000000000000)
2021-02-03 11:03:12,387 INFO
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] -
yarn.client.max-cached-nodemanagers-proxies : 0
2021-02-03 11:03:12,392 INFO org.apache.flink.yarn.YarnResourceManager [] -
ResourceManager akka.tcp://fl...@host3.rdc.com:36810/user/rpc/resourcemanager_0
was granted leadership with fencing token 00000000000000000000000000000000
2021-02-03 11:03:12,395 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] -
Starting the SlotManager.
2021-02-03 11:03:12,402 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Resolved ResourceManager address, beginning registration
2021-02-03 11:03:12,407 INFO org.apache.flink.yarn.YarnResourceManager [] -
Registering job manager
00000000000000000000000000000...@akka.tcp://fl...@host3.rdc.com:36810/user/rpc/jobmanager_2
for job 78c332655ca97966303a64b07cfdf433.
2021-02-03 11:03:12,416 INFO org.apache.flink.yarn.YarnResourceManager [] -
Registered job manager
00000000000000000000000000000...@akka.tcp://fl...@host3.rdc.com:36810/user/rpc/jobmanager_2
for job 78c332655ca97966303a64b07cfdf433.
2021-02-03 11:03:12,421 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
JobManager successfully registered at ResourceManager, leader id:
00000000000000000000000000000000.
2021-02-03 11:03:12,421 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Requesting new
slot [SlotRequestId
{5c3e8039f959fcf109bac0a8833592c7}
] and profile ResourceProfile\{UNKNOWN} from resource manager.
2021-02-03 11:03:12,422 INFO org.apache.flink.yarn.YarnResourceManager [] -
Request slot with profile ResourceProfile\{UNKNOWN} for job
78c332655ca97966303a64b07cfdf433 with allocation id
ec899e694d4ac7e2d0e27afc780b688c.
2021-02-03 11:03:12,442 INFO org.apache.flink.yarn.YarnResourceManager [] -
Requesting new TaskExecutor container with resource WorkerResourceSpec
\{cpuCores=1.0, taskHeapSize=3.178gb (3412800247 bytes), taskOffHeapSize=0
bytes, networkMemSize=702.140mb (736247162 bytes), managedMemSize=2.743gb
(2944988649 bytes)}. Number pending workers of this resource is 1.
2021-02-03 11:03:17,932 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl [] - Received new token
for : host2.rdc.com:8041
2021-02-03 11:03:17,937 INFO org.apache.flink.yarn.YarnResourceManager [] -
Received 1 containers.
2021-02-03 11:03:17,946 INFO org.apache.flink.yarn.YarnResourceManager [] -
Received 1 containers with resource <memory:8192, vCores:1>, 1 pending
container requests.
2021-02-03 11:03:17,962 INFO org.apache.flink.yarn.YarnResourceManager [] -
TaskExecutor container_e04_1608618104524_0555_01_000002 will be started on
host2.rdc.com with TaskExecutorProcessSpec \{cpuCores=1.0,
frameworkHeapSize=128.000mb (134217728 bytes), frameworkOffHeapSize=128.000mb
(134217728 bytes), taskHeapSize=3.178gb (3412800247 bytes), taskOffHeapSize=0
bytes, networkMemSize=702.140mb (736247162 bytes), managedMemorySize=2.743gb
(2944988649 bytes), jvmMetaspaceSize=256.000mb (268435456 bytes),
jvmOverheadSize=808.600mb (847878566 bytes)}.
2021-02-03 11:03:17,979 INFO org.apache.flink.yarn.YarnResourceManager [] -
Creating container launch context for TaskManagers
2021-02-03 11:03:17,980 INFO org.apache.flink.yarn.YarnResourceManager [] -
Starting TaskManagers
2021-02-03 11:03:17,992 INFO org.apache.flink.yarn.YarnResourceManager [] -
Removing container request Capability[<memory:8192, vCores:1>]Priority[1].
2021-02-03 11:03:17,992 INFO org.apache.flink.yarn.YarnResourceManager [] -
Accepted 1 requested containers, returned 0 excess containers, 0 pending
container requests of resource <memory:8192, vCores:1>.
2021-02-03 11:03:17,995 INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Processing
Event EventType: START_CONTAINER for Container
container_e04_1608618104524_0555_01_000002
2021-02-03 11:03:17,997 INFO
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] -
Opening proxy : host2.rdc.com:8041
2021-02-03 11:03:23,126 INFO org.apache.flink.yarn.YarnResourceManager [] -
Registering TaskManager with ResourceID
container_e04_1608618104524_0555_01_000002
(akka.tcp://fl...@host2.rdc.com:32861/user/rpc/taskmanager_0) at ResourceManager
2021-02-03 11:03:23,287 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Socket
Stream -> Flat Map (1/1) (6d75dc1fadd5d7190c4471a573ad93f0) switched from
SCHEDULED to DEPLOYING.
2021-02-03 11:03:23,287 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source:
Socket Stream -> Flat Map (1/1) (attempt #0) to
container_e04_1608618104524_0555_01_000002 @ host2.rdc.com (dataPort=35265)
2021-02-03 11:03:23,293 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] -
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(426d3d4c23b0ba189ead74465076cf15) switched from SCHEDULED to DEPLOYING.
2021-02-03 11:03:23,294 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(attempt #0) to container_e04_1608618104524_0555_01_000002 @ host2.rdc.com
(dataPort=35265)
2021-02-03 11:03:23,461 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Socket
Stream -> Flat Map (1/1) (6d75dc1fadd5d7190c4471a573ad93f0) switched from
DEPLOYING to RUNNING.
2021-02-03 11:03:23,462 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] -
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(426d3d4c23b0ba189ead74465076cf15) switched from DEPLOYING to RUNNING.
2021-02-04 09:19:32,165 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Socket
Stream -> Flat Map (1/1) (6d75dc1fadd5d7190c4471a573ad93f0) switched from
RUNNING to FINISHED.
2021-02-04 09:19:32,175 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] -
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(426d3d4c23b0ba189ead74465076cf15) switched from RUNNING to FINISHED.
2021-02-04 09:19:32,179 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Socket Window
WordCount (78c332655ca97966303a64b07cfdf433) switched from state RUNNING to
FINISHED.
2021-02-04 09:19:32,179 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Stopping
checkpoint coordinator for job 78c332655ca97966303a64b07cfdf433.
2021-02-04 09:19:32,185 INFO
org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore [] -
Shutting down
2021-02-04 09:19:32,200 INFO org.apache.flink.runtime.dispatcher.MiniDispatcher
[] - Job 78c332655ca97966303a64b07cfdf433 reached globally terminal state
FINISHED.
2021-02-04 09:19:32,205 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Stopping the JobMaster for job Socket Window
WordCount(78c332655ca97966303a64b07cfdf433).
2021-02-04 09:19:32,216 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Suspending
SlotPool.
2021-02-04 09:19:32,216 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Close ResourceManager connection 33d0c682153f6a737ebea3e419a62d2f: JobManager
is shutting down..
2021-02-04 09:19:32,217 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Stopping SlotPool.
2021-02-04 09:19:32,217 INFO org.apache.flink.yarn.YarnResourceManager [] -
Disconnect job manager
00000000000000000000000000000...@akka.tcp://fl...@host3.rdc.com:36810/user/rpc/jobmanager_2
for job 78c332655ca97966303a64b07cfdf433 from the resource manager.
2021-02-04 09:20:05,839 INFO org.apache.flink.yarn.YarnResourceManager [] -
Stopping container container_e04_1608618104524_0555_01_000002.
2021-02-04 09:20:05,840 INFO org.apache.flink.yarn.YarnResourceManager [] -
Closing TaskExecutor connection container_e04_1608618104524_0555_01_000002
because: TaskExecutor exceeded the idle timeout.
2021-02-04 09:20:05,842 INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Processing
Event EventType: STOP_CONTAINER for Container
container_e04_1608618104524_0555_01_000002
2021-02-04 09:20:05,845 INFO
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] -
Opening proxy : host2.rdc.com:8041
2021-02-04 09:20:05,882 WARN org.apache.flink.yarn.YarnResourceManager [] -
Discard registration from TaskExecutor
container_e04_1608618104524_0555_01_000002 at
(akka.tcp://fl...@host2.rdc.com:32861/user/rpc/taskmanager_0) because the
framework did not recognize it
2021-02-04 09:20:06,389 WARN akka.remote.ReliableDeliverySupervisor [] -
Association with remote system [akka.tcp://flink-metr...@host2.rdc.com:46488]
has failed, address is now gated for [50] ms. Reason: [Disassociated]
2021-02-04 09:20:06,389 WARN akka.remote.ReliableDeliverySupervisor [] -
Association with remote system [akka.tcp://fl...@host2.rdc.com:32861] has
failed, address is now gated for [50] ms. Reason: [Disassociated]
2021-02-04 09:46:58,767 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - RECEIVED SIGNAL 15:
SIGTERM. Shutting down as requested.
2021-02-04 09:46:58,772 INFO org.apache.flink.runtime.blob.BlobServer [] -
Stopped BLOB server at 0.0.0.0:42928
was (Author: tonychan):
this is the summit code
./bin/flink run -m yarn-cluster -yjm 2048 -ytm 8086
./examples/streaming/SocketWindowWordCount.jar --hostname ip --port 9998
the yarn log :
2021-02-03 11:03:11,334 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - ----------------
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - ip:2181
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - /flink
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - 9250-9290
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - cxf.identify1
2021-02-03 11:03:11,335 ERROR
org.apache.flink.metrics.prometheus.PrometheusReporter [] - ----------------
2021-02-03 11:03:11,341 INFO
org.apache.flink.metrics.prometheus.PrometheusReporter [] - Started
PrometheusReporter HTTP server on port 9251.
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf,
built on 03/06/2019 16:18 GMT
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:host.name=bdp-3.rdc.com
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.version=1.8.0_241
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.vendor=Oracle Corporation
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.home=/usr/java/jdk1.8.0_241/jre
2021-02-03 11:03:11,346 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.class.path=:SocketWindowWordCount.jar:lib/flink-csv-1.11.2.jar:lib/flink-json-1.11.2.jar:lib/flink-shaded-hadoop-2-uber-2.6.5-10.0.jar:lib/flink-shaded-zookeeper-3.4.14.jar:lib/flink-table-blink_2.11-1.11.2.jar:lib/flink-table_2.11-1.11.2.jar:lib/log4j-1.2-api-2.12.1.jar:lib/log4j-api-2.12.1.jar:lib/log4j-core-2.12.1.jar:lib/log4j-slf4j-impl-2.12.1.jar:flink-dist_2.11-1.11.2.jar:job.graph:flink-conf.yaml::/etc/hadoop/conf.cloudera.yarn:/run/cloudera-scm-agent/process/1071-yarn-NODEMANAGER:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-annotations.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-auth.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-aws.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-azure-datalake.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-nfs.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-nfs-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-common-2.6.0-cdh5.16.2-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-azure-datalake-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-aws-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-auth-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/hadoop-annotations-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-format.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-format-sources.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-format-javadoc.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-tools.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-thrift.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-test-hadoop2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-scrooge_2.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-scala_2.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-protobuf.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-pig.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-pig-bundle.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-jackson.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-hadoop.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-generator.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-encoding.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-common.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-column.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-cascading.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/parquet-avro.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/hue-plugins-3.9.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/curator-client-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/gson-2.2.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/curator-recipes-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/httpclient-4.2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/curator-framework-2.7.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/hamcrest-core-1.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jetty-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/httpcore-4.2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jersey-json-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jackson-xc-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/api-asn1-api-1.0.0-M20.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/apacheds-i18n-2.0.0-M15.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/zookeeper.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/avro.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/slf4j-api-1.7.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/netty-3.10.5.Final.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/logredactor-1.0.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jets3t-0.9.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/htrace-core4-4.0.1-incubating.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-digester-1.8.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-beanutils-1.9.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/azure-data-lake-store-sdk-2.2.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/aws-java-sdk-bundle-1.11.134.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/api-util-1.0.0-M20.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/mockito-all-1.8.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/junit-4.11.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jsch-0.1.42.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/jackson-jaxrs-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-nfs.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-nfs-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/hadoop-hdfs-2.6.0-cdh5.16.2-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/xml-apis-1.3.04.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/xercesImpl-2.9.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/netty-3.10.5.Final.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.10-cloudera.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/htrace-core4-4.0.1-incubating.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-hdfs/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-api.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-client.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-common.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-registry.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-common.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-nodemanager.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-tests.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-web-proxy.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-web-proxy-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-tests-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-resourcemanager-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-nodemanager-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-common-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-registry-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-common-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-client-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-applications-distributedshell-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/hadoop-yarn-api-2.6.0-cdh5.16.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/spark-yarn-shuffle.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/spark-1.6.0-cdh5.16.2-yarn-shuffle.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/leveldbjni-all-1.8.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jsr305-3.0.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jline-2.11.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jetty-util-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jetty-6.1.26.cloudera.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-json-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-guice-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jersey-client-1.9.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/javax.inject-1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-xc-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.10-cloudera.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-jaxrs-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/jackson-core-asl-1.8.10.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/guice-3.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-collections-3.2.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/zookeeper.jar:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop-yarn/lib/xz-1.0.jar
2021-02-03 11:03:11,347 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.library.path=:/opt/cloudera/parcels/CDH-5.16.2-1.cdh5.16.2.p0.8/lib/hadoop/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.io.tmpdir=/tmp
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:java.compiler=<NA>
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:os.name=Linux
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:os.arch=amd64
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:os.version=3.10.0-862.11.6.el7.x86_64
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:user.name=yarn
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:user.home=/var/lib/hadoop-yarn
2021-02-03 11:03:11,348 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client
environment:user.dir=/mnt/disk2/yarn/nm/usercache/root/appcache/application_1608618104524_0555/container_e04_1608618104524_0555_01_000001
2021-02-03 11:03:11,349 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] -
Initiating client connection, connectString=10.20.30.113:2181
sessionTimeout=10000
watcher=org.apache.flink.metrics.prometheus.PrometheusReporter$1@2b0b4d53
2021-02-03 11:03:11,367 WARN
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - SASL
configuration failed: javax.security.auth.login.LoginException: No JAAS
configuration section named 'Client' was found in specified JAAS configuration
file:
'/mnt/disk1/yarn/nm/usercache/root/appcache/application_1608618104524_0555/jaas-697756762480862695.conf'.
Will continue connection to Zookeeper server without SASL authentication, if
Zookeeper server allows it.
2021-02-03 11:03:11,370 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Opening
socket connection to server bdp-4.rdc.com/ip:2181
2021-02-03 11:03:11,370 INFO
org.apache.flink.metrics.prometheus.PrometheusReporter [] -
zookeeperip:2181触发了None事件
2021-02-03 11:03:11,374 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Socket
connection established to host/ip:2181, initiating session
2021-02-03 11:03:11,380 INFO
org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Session
establishment complete on server host/ip:2181, sessionid = 0x100e4bb743b0193,
negotiated timeout = 10000
2021-02-03 11:03:11,380 INFO
org.apache.flink.metrics.prometheus.PrometheusReporter [] -
zookeeperip:2181触发了None事件
2021-02-03 11:03:11,390 INFO
org.apache.flink.runtime.metrics.MetricRegistryImpl [] - Reporting metrics for
reporter prom of type org.apache.flink.metrics.prometheus.PrometheusReporter.
2021-02-03 11:03:11,397 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Trying to start
actor system, external address bdp-3.rdc.com:0, bind address 0.0.0.0:0.
2021-02-03 11:03:11,416 INFO akka.event.slf4j.Slf4jLogger [] - Slf4jLogger
started
2021-02-03 11:03:11,420 INFO akka.remote.Remoting [] - Starting remoting
2021-02-03 11:03:11,431 INFO akka.remote.Remoting [] - Remoting started;
listening on addresses :[akka.tcp://flink-metr...@bdp-3.rdc.com:45458]
2021-02-03 11:03:11,468 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Actor system started
at akka.tcp://flink-metr...@bdp-3.rdc.com:45458
2021-02-03 11:03:11,484 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for
org.apache.flink.runtime.metrics.dump.MetricQueryService at
akka://flink-metrics/user/rpc/MetricQueryService .
2021-02-03 11:03:11,530 WARN org.apache.flink.configuration.Configuration [] -
Config uses deprecated configuration key 'web.port' instead of proper key
'rest.bind-port'
2021-02-03 11:03:11,532 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Upload
directory /tmp/flink-web-abe884bc-3165-4799-99b9-870d5641194a/flink-web-upload
does not exist.
2021-02-03 11:03:11,532 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Created
directory /tmp/flink-web-abe884bc-3165-4799-99b9-870d5641194a/flink-web-upload
for file uploads.
2021-02-03 11:03:11,547 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Starting
rest endpoint.
2021-02-03 11:03:11,824 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils [] - Determined location of
main cluster component log file:
/mnt/disk1/yarn/container-logs/application_1608618104524_0555/container_e04_1608618104524_0555_01_000001/jobmanager.log
2021-02-03 11:03:11,824 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils [] - Determined location of
main cluster component stdout file:
/mnt/disk1/yarn/container-logs/application_1608618104524_0555/container_e04_1608618104524_0555_01_000001/jobmanager.out
2021-02-03 11:03:12,015 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Rest
endpoint listening at bdp-3.rdc.com:40148
2021-02-03 11:03:12,016 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
[http://bdp-3.rdc.com:40148|http://bdp-3.rdc.com:40148/] was granted leadership
with leaderSessionID=00000000-0000-0000-0000-000000000000
2021-02-03 11:03:12,018 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Web frontend
listening at [http://bdp-3.rdc.com:40148|http://bdp-3.rdc.com:40148/].
2021-02-03 11:03:12,082 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at
akka://flink/user/rpc/resourcemanager_0 .
2021-02-03 11:03:12,091 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: taskmanager.memory.process.size, 8086 mb
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: metrics.reporter.prom.port, 9250-9290
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: metrics.reporter.prom.identify, cxf.identify1
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: internal.jobgraph-path, job.graph
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: metrics.reporter.prom.zookeeperPath, /flink
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: high-availability.cluster-id, application_1608618104524_0555
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: jobmanager.rpc.address, localhost
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: execution.target, yarn-per-job
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: jobmanager.memory.process.size, 2 gb
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: jobmanager.rpc.port, 6123
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: execution.savepoint.ignore-unclaimed-state, false
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: execution.attached, true
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: internal.cluster.execution-mode, NORMAL
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: execution.shutdown-on-attached-exit, false
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: pipeline.jars,
[file:/opt/cxf/flink/./examples/streaming/SocketWindowWordCount.jar|file://opt/cxf/flink/examples/streaming/SocketWindowWordCount.jar]
2021-02-03 11:03:12,092 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: parallelism.default, 1
2021-02-03 11:03:12,093 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: taskmanager.numberOfTaskSlots, 1
2021-02-03 11:03:12,093 WARN
org.apache.flink.configuration.GlobalConfiguration [] - Error while trying to
split key and value in configuration file
/mnt/disk2/yarn/nm/usercache/root/appcache/application_1608618104524_0555/container_e04_1608618104524_0555_01_000001/flink-conf.yaml:18:
"pipeline.classpaths: "
2021-02-03 11:03:12,093 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: metrics.reporter.prom.class,
org.apache.flink.metrics.prometheus.PrometheusReporter
2021-02-03 11:03:12,093 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: metrics.reporter.prom.zookeeper, ip:2181
2021-02-03 11:03:12,093 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: $internal.deployment.config-dir, /opt/cxf/flink/conf
2021-02-03 11:03:12,093 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration
property: $internal.yarn.log-config-file, /opt/cxf/flink/conf/log4j.properties
2021-02-03 11:03:12,127 INFO
org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled
external resources: []
2021-02-03 11:03:12,176 INFO
org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcess [] -
Start JobDispatcherLeaderProcess.
2021-02-03 11:03:12,182 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for
org.apache.flink.runtime.dispatcher.MiniDispatcher at
akka://flink/user/rpc/dispatcher_1 .
2021-02-03 11:03:12,226 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at
akka://flink/user/rpc/jobmanager_2 .
2021-02-03 11:03:12,234 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Initializing job Socket Window WordCount (78c332655ca97966303a64b07cfdf433).
2021-02-03 11:03:12,250 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Using restart back off time strategy NoRestartBackoffTimeStrategy for Socket
Window WordCount (78c332655ca97966303a64b07cfdf433).
2021-02-03 11:03:12,285 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Running initialization on master for job Socket Window WordCount
(78c332655ca97966303a64b07cfdf433).
2021-02-03 11:03:12,285 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Successfully ran initialization on master in 0 ms.
2021-02-03 11:03:12,302 INFO
org.apache.flink.runtime.scheduler.adapter.DefaultExecutionTopology [] - Built
1 pipelined regions in 1 ms
2021-02-03 11:03:12,315 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
No state backend has been configured, using default (Memory / JobManager)
MemoryStateBackend (data in heap memory / checkpoints to JobManager)
(checkpoints: 'null', savepoints: 'null', asynchronous: TRUE, maxStateSize:
5242880)
2021-02-03 11:03:12,335 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Using failover strategy
org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy@36d087c1
for Socket Window WordCount (78c332655ca97966303a64b07cfdf433).
2021-02-03 11:03:12,339 INFO
org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl [] - JobManager runner
for job Socket Window WordCount (78c332655ca97966303a64b07cfdf433) was granted
leadership with session id 00000000-0000-0000-0000-000000000000 at
akka.tcp://fl...@bdp-3.rdc.com:36810/user/rpc/jobmanager_2.
2021-02-03 11:03:12,350 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Starting execution of job Socket Window WordCount
(78c332655ca97966303a64b07cfdf433) under job master id
00000000000000000000000000000000.
2021-02-03 11:03:12,354 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Starting scheduling with scheduling strategy
[org.apache.flink.runtime.scheduler.strategy.EagerSchedulingStrategy]
2021-02-03 11:03:12,355 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Socket Window
WordCount (78c332655ca97966303a64b07cfdf433) switched from state CREATED to
RUNNING.
2021-02-03 11:03:12,365 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Socket
Stream -> Flat Map (1/1) (6d75dc1fadd5d7190c4471a573ad93f0) switched from
CREATED to SCHEDULED.
2021-02-03 11:03:12,365 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] -
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(426d3d4c23b0ba189ead74465076cf15) switched from CREATED to SCHEDULED.
2021-02-03 11:03:12,376 INFO org.apache.flink.yarn.YarnResourceManager [] -
Recovered 0 containers from previous attempts ([]).
2021-02-03 11:03:12,378 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Cannot serve slot
request, no ResourceManager connected. Adding as pending request
[SlotRequestId\\{5c3e8039f959fcf109bac0a8833592c7}]
2021-02-03 11:03:12,378 INFO org.apache.flink.yarn.YarnResourceManager [] -
Register application master response contains scheduler resource types:
[MEMORY, CPU].
2021-02-03 11:03:12,379 INFO org.apache.flink.yarn.YarnResourceManager [] -
Container matching strategy: MATCH_VCORE.
2021-02-03 11:03:12,385 INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Upper bound
of the thread pool size is 500
2021-02-03 11:03:12,387 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Connecting to ResourceManager
akka.tcp://fl...@host3.rdc.com:36810/user/rpc/resourcemanager_*(00000000000000000000000000000000)
2021-02-03 11:03:12,387 INFO
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] -
yarn.client.max-cached-nodemanagers-proxies : 0
2021-02-03 11:03:12,392 INFO org.apache.flink.yarn.YarnResourceManager [] -
ResourceManager akka.tcp://fl...@host3.rdc.com:36810/user/rpc/resourcemanager_0
was granted leadership with fencing token 00000000000000000000000000000000
2021-02-03 11:03:12,395 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] -
Starting the SlotManager.
2021-02-03 11:03:12,402 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Resolved ResourceManager address, beginning registration
2021-02-03 11:03:12,407 INFO org.apache.flink.yarn.YarnResourceManager [] -
Registering job manager
00000000000000000000000000000...@akka.tcp://fl...@host3.rdc.com:36810/user/rpc/jobmanager_2
for job 78c332655ca97966303a64b07cfdf433.
2021-02-03 11:03:12,416 INFO org.apache.flink.yarn.YarnResourceManager [] -
Registered job manager
00000000000000000000000000000...@akka.tcp://fl...@host3.rdc.com:36810/user/rpc/jobmanager_2
for job 78c332655ca97966303a64b07cfdf433.
2021-02-03 11:03:12,421 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
JobManager successfully registered at ResourceManager, leader id:
00000000000000000000000000000000.
2021-02-03 11:03:12,421 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Requesting new
slot [SlotRequestId\\{5c3e8039f959fcf109bac0a8833592c7}] and profile
ResourceProfile\{UNKNOWN} from resource manager.
2021-02-03 11:03:12,422 INFO org.apache.flink.yarn.YarnResourceManager [] -
Request slot with profile ResourceProfile\{UNKNOWN} for job
78c332655ca97966303a64b07cfdf433 with allocation id
ec899e694d4ac7e2d0e27afc780b688c.
2021-02-03 11:03:12,442 INFO org.apache.flink.yarn.YarnResourceManager [] -
Requesting new TaskExecutor container with resource WorkerResourceSpec
\{cpuCores=1.0, taskHeapSize=3.178gb (3412800247 bytes), taskOffHeapSize=0
bytes, networkMemSize=702.140mb (736247162 bytes), managedMemSize=2.743gb
(2944988649 bytes)}. Number pending workers of this resource is 1.
2021-02-03 11:03:17,932 INFO
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl [] - Received new token
for : host2.rdc.com:8041
2021-02-03 11:03:17,937 INFO org.apache.flink.yarn.YarnResourceManager [] -
Received 1 containers.
2021-02-03 11:03:17,946 INFO org.apache.flink.yarn.YarnResourceManager [] -
Received 1 containers with resource <memory:8192, vCores:1>, 1 pending
container requests.
2021-02-03 11:03:17,962 INFO org.apache.flink.yarn.YarnResourceManager [] -
TaskExecutor container_e04_1608618104524_0555_01_000002 will be started on
host2.rdc.com with TaskExecutorProcessSpec \{cpuCores=1.0,
frameworkHeapSize=128.000mb (134217728 bytes), frameworkOffHeapSize=128.000mb
(134217728 bytes), taskHeapSize=3.178gb (3412800247 bytes), taskOffHeapSize=0
bytes, networkMemSize=702.140mb (736247162 bytes), managedMemorySize=2.743gb
(2944988649 bytes), jvmMetaspaceSize=256.000mb (268435456 bytes),
jvmOverheadSize=808.600mb (847878566 bytes)}.
2021-02-03 11:03:17,979 INFO org.apache.flink.yarn.YarnResourceManager [] -
Creating container launch context for TaskManagers
2021-02-03 11:03:17,980 INFO org.apache.flink.yarn.YarnResourceManager [] -
Starting TaskManagers
2021-02-03 11:03:17,992 INFO org.apache.flink.yarn.YarnResourceManager [] -
Removing container request Capability[<memory:8192, vCores:1>]Priority[1].
2021-02-03 11:03:17,992 INFO org.apache.flink.yarn.YarnResourceManager [] -
Accepted 1 requested containers, returned 0 excess containers, 0 pending
container requests of resource <memory:8192, vCores:1>.
2021-02-03 11:03:17,995 INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Processing
Event EventType: START_CONTAINER for Container
container_e04_1608618104524_0555_01_000002
2021-02-03 11:03:17,997 INFO
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] -
Opening proxy : host2.rdc.com:8041
2021-02-03 11:03:23,126 INFO org.apache.flink.yarn.YarnResourceManager [] -
Registering TaskManager with ResourceID
container_e04_1608618104524_0555_01_000002
(akka.tcp://fl...@host2.rdc.com:32861/user/rpc/taskmanager_0) at ResourceManager
2021-02-03 11:03:23,287 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Socket
Stream -> Flat Map (1/1) (6d75dc1fadd5d7190c4471a573ad93f0) switched from
SCHEDULED to DEPLOYING.
2021-02-03 11:03:23,287 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source:
Socket Stream -> Flat Map (1/1) (attempt #0) to
container_e04_1608618104524_0555_01_000002 @ host2.rdc.com (dataPort=35265)
2021-02-03 11:03:23,293 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] -
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(426d3d4c23b0ba189ead74465076cf15) switched from SCHEDULED to DEPLOYING.
2021-02-03 11:03:23,294 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(attempt #0) to container_e04_1608618104524_0555_01_000002 @ host2.rdc.com
(dataPort=35265)
2021-02-03 11:03:23,461 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Socket
Stream -> Flat Map (1/1) (6d75dc1fadd5d7190c4471a573ad93f0) switched from
DEPLOYING to RUNNING.
2021-02-03 11:03:23,462 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] -
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(426d3d4c23b0ba189ead74465076cf15) switched from DEPLOYING to RUNNING.
2021-02-04 09:19:32,165 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: Socket
Stream -> Flat Map (1/1) (6d75dc1fadd5d7190c4471a573ad93f0) switched from
RUNNING to FINISHED.
2021-02-04 09:19:32,175 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] -
Window(TumblingProcessingTimeWindows(5000), ProcessingTimeTrigger,
ReduceFunction$1, PassThroughWindowFunction) -> Sink: Print to Std. Out (1/1)
(426d3d4c23b0ba189ead74465076cf15) switched from RUNNING to FINISHED.
2021-02-04 09:19:32,179 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job Socket Window
WordCount (78c332655ca97966303a64b07cfdf433) switched from state RUNNING to
FINISHED.
2021-02-04 09:19:32,179 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Stopping
checkpoint coordinator for job 78c332655ca97966303a64b07cfdf433.
2021-02-04 09:19:32,185 INFO
org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore [] -
Shutting down
2021-02-04 09:19:32,200 INFO
org.apache.flink.runtime.dispatcher.MiniDispatcher [] - Job
78c332655ca97966303a64b07cfdf433 reached globally terminal state FINISHED.
2021-02-04 09:19:32,205 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Stopping the JobMaster for job Socket Window
WordCount(78c332655ca97966303a64b07cfdf433).
2021-02-04 09:19:32,216 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Suspending
SlotPool.
2021-02-04 09:19:32,216 INFO org.apache.flink.runtime.jobmaster.JobMaster [] -
Close ResourceManager connection 33d0c682153f6a737ebea3e419a62d2f: JobManager
is shutting down..
2021-02-04 09:19:32,217 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Stopping SlotPool.
2021-02-04 09:19:32,217 INFO org.apache.flink.yarn.YarnResourceManager [] -
Disconnect job manager
00000000000000000000000000000...@akka.tcp://fl...@host3.rdc.com:36810/user/rpc/jobmanager_2
for job 78c332655ca97966303a64b07cfdf433 from the resource manager.
2021-02-04 09:20:05,839 INFO org.apache.flink.yarn.YarnResourceManager [] -
Stopping container container_e04_1608618104524_0555_01_000002.
2021-02-04 09:20:05,840 INFO org.apache.flink.yarn.YarnResourceManager [] -
Closing TaskExecutor connection container_e04_1608618104524_0555_01_000002
because: TaskExecutor exceeded the idle timeout.
2021-02-04 09:20:05,842 INFO
org.apache.hadoop.yarn.client.api.async.impl.NMClientAsyncImpl [] - Processing
Event EventType: STOP_CONTAINER for Container
container_e04_1608618104524_0555_01_000002
2021-02-04 09:20:05,845 INFO
org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy [] -
Opening proxy : host2.rdc.com:8041
2021-02-04 09:20:05,882 WARN org.apache.flink.yarn.YarnResourceManager [] -
Discard registration from TaskExecutor
container_e04_1608618104524_0555_01_000002 at
(akka.tcp://fl...@host2.rdc.com:32861/user/rpc/taskmanager_0) because the
framework did not recognize it
2021-02-04 09:20:06,389 WARN akka.remote.ReliableDeliverySupervisor [] -
Association with remote system [akka.tcp://flink-metr...@host2.rdc.com:46488]
has failed, address is now gated for [50] ms. Reason: [Disassociated]
2021-02-04 09:20:06,389 WARN akka.remote.ReliableDeliverySupervisor [] -
Association with remote system [akka.tcp://fl...@host2.rdc.com:32861] has
failed, address is now gated for [50] ms. Reason: [Disassociated]
2021-02-04 09:46:58,767 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - RECEIVED SIGNAL 15:
SIGTERM. Shutting down as requested.
2021-02-04 09:46:58,772 INFO org.apache.flink.runtime.blob.BlobServer [] -
Stopped BLOB server at 0.0.0.0:42928
> at yarn-per-job mode, tm stopped ,the jm dont relase resource,its also running
> ------------------------------------------------------------------------------
>
> Key: FLINK-21264
> URL: https://issues.apache.org/jira/browse/FLINK-21264
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.12.0
> Reporter: tonychan
> Priority: Blocker
> Attachments: image-2021-02-03-17-42-53-269.png,
> image-2021-02-03-17-43-28-017.png
>
>
> i make a socket (nc -lk 9999) for flink test, and at first the flink running
> is ok,when the source server stopped, the flink tm is stopped and the status
> is finish ,but the jm is also running, !image-2021-02-03-17-42-53-269.png!
>
> !image-2021-02-03-17-43-28-017.png!
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
