[
https://issues.apache.org/jira/browse/FLINK-26248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17494543#comment-17494543
]
zhisheng commented on FLINK-26248:
----------------------------------
start job shell:
{code:java}
./bin/flink run-application -p 1 -t kubernetes-application \
-Dkubernetes.cluster-id=state-machine-cluster \
-Dtaskmanager.memory.process.size=1024m \ -Dkubernetes.taskmanager.cpu=0.5 \
-Dtaskmanager.numberOfTaskSlots=1 \
-Dkubernetes.container.image=harbor.cn/flink/statemachine:v0.0.6 \
-Dkubernetes.namespace=hke-flink \
-Dkubernetes.jobmanager.service-account=flink \
-Dkubernetes.container.image.pull-secrets=docker-registry-test \
-Dkubernetes.jobmanager.node-selector=kubernetes.io/role:flink-node \
-Dkubernetes.taskmanager.node-selector=kubernetes.io/role:flink-node \
-Dkubernetes.rest-service.exposed.type=NodePort \
-Dhigh-availability.storageDir=hdfs:///flink/ha/k8s \
local:///opt/flink/usrlib/StateMachineExample.jar {code}
the jm log:
{code:java}
2022-02-18 10:41:44,930 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------
2022-02-18 10:41:44,987 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Starting
KubernetesApplicationClusterEntrypoint (Version: 1.12.0, Scala: 2.11,
Rev:a41d55f, Date:2021-12-09T10:38:36+01:00)
2022-02-18 10:41:44,987 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS current
user: flink
2022-02-18 10:41:45,590 WARN org.apache.hadoop.util.NativeCodeLoader
[] - Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
2022-02-18 10:41:45,714 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Current
Hadoop/Kerberos user: flink
2022-02-18 10:41:45,714 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM: OpenJDK
64-Bit Server VM - Oracle Corporation - 1.8/25.322-b06
2022-02-18 10:41:45,715 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Maximum heap
size: 1024 MiBytes
2022-02-18 10:41:45,715 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JAVA_HOME:
/usr/local/openjdk-8
2022-02-18 10:41:45,717 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Hadoop
version: 2.7.3
2022-02-18 10:41:45,717 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM Options:
2022-02-18 10:41:45,717 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Xmx1073741824
2022-02-18 10:41:45,717 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Xms1073741824
2022-02-18 10:41:45,717 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:MaxMetaspaceSize=268435456
2022-02-18 10:41:45,717 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -Denv=dev
2022-02-18 10:41:45,717 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog4j2.formatMsgNoLookups=true
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Xloggc:./gc-%t.log
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+PrintGCDetails
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:-OmitStackTraceInFastThrow
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+PrintGCTimeStamps
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+PrintHeapAtGC
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+PrintGCDateStamps
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+UseGCLogFileRotation
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+PrintReferenceGC
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+PrintTenuringDistribution
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:NumberOfGCLogFiles=5
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:GCLogFileSize=20M
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+PrintPromotionFailure
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+PrintGCCause
2022-02-18 10:41:45,718 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+UseG1GC
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+UnlockExperimentalVMOptions
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+UnlockDiagnosticVMOptions
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:MaxGCPauseMillis=200
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:-ResizePLAB
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:+ParallelRefProcEnabled
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:ParallelGCThreads=8
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:ConcGCThreads=4
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:G1HeapWastePercent=3
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:InitiatingHeapOccupancyPercent=35
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-XX:G1MixedGCLiveThresholdPercent=85
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog.file=/opt/flink/log/jobmanager.log
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
-Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties
2022-02-18 10:41:45,719 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Program
Arguments:
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.off-heap.size=134217728b
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.jvm-overhead.min=201326592b
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.jvm-metaspace.size=268435456b
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.heap.size=1073741824b
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
jobmanager.memory.jvm-overhead.max=201326592b
2022-02-18 10:41:45,721 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Classpath:
/opt/flink/lib/Log4j2KafkaAppender-1.0.jar:/opt/flink/lib/flink-connector-jdbc_2.11-1.12.0.jar:/opt/flink/lib/flink-csv-1.12.0.jar:/opt/flink/lib/flink-json-1.12.0.jar:/opt/flink/lib/flink-metrics-kafka_2.11-1.12.0.jar:/opt/flink/lib/flink-metrics-prometheus-1.12.0.jar:/opt/flink/lib/flink-sequence-file-1.12.0.jar:/opt/flink/lib/flink-shaded-zookeeper-3.4.14.jar:/opt/flink/lib/flink-sql-avro-1.12.0.jar:/opt/flink/lib/flink-sql-connector-elasticsearch_2.11-1.12.0.jar:/opt/flink/lib/flink-sql-connector-hbase-2.2_2.11-1.12.0.jar:/opt/flink/lib/flink-sql-connector-hive-2.2.0_2.11-1.12.0.jar:/opt/flink/lib/flink-sql-connector-kafka_2.11-1.12.0.jar:/opt/flink/lib/flink-sql-orc_2.11-1.12.0.jar:/opt/flink/lib/flink-sql-parquet_2.11-1.12.0.jar:/opt/flink/lib/flink-table-blink_2.11-1.12.0.jar:/opt/flink/lib/flink-table_2.11-1.12.0.jar:/opt/flink/lib/log4j-1.2-api-2.12.1.jar:/opt/flink/lib/log4j-api-2.12.1.jar:/opt/flink/lib/log4j-core-2.12.1.jar:/opt/flink/lib/log4j-slf4j-impl-2.12.1.jar:/opt/flink/lib/mysql-connector-java-5.1.47.jar:/opt/flink/lib/postgresql-42.2.8.jar:/opt/flink/lib/flink-dist_2.11-1.12.0.jar:/opt/hadoop/conf:/opt/hadoop/share/hadoop/common/lib/activation-1.1.jar:/opt/hadoop/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/opt/hadoop/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hadoop/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/opt/hadoop/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/opt/hadoop/share/hadoop/common/lib/asm-3.2.jar:/opt/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/opt/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/opt/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/opt/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/opt/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/opt/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/opt/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/opt/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/opt/hadoop/share/hadoop/common/lib/curator-client-2.7.1.jar:/opt/hadoop/share/hadoop/common/lib/curator-framework-2.7.1.jar:/opt/hadoop/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/opt/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/opt/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/opt/hadoop/share/hadoop/common/lib/hadoop-annotations-2.7.3.jar:/opt/hadoop/share/hadoop/common/lib/hadoop-auth-2.7.3.jar:/opt/hadoop/share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/hadoop/share/hadoop/common/lib/-rack-awareness-policy-1.0.jar:/opt/hadoop/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar:/opt/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/opt/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/opt/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/opt/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/opt/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/opt/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/opt/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/opt/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/opt/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/opt/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/opt/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/opt/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/opt/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/opt/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/opt/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/opt/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/hadoop/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/hadoop/share/hadoop/common/lib/junit-4.11.jar:/opt/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/opt/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/opt/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/hadoop/share/hadoop/common/lib/slf4j-api-1.7.10.jar:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar:/opt/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/opt/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/hadoop/share/hadoop/common/lib/xz-1.0.jar:/opt/hadoop/share/hadoop/common/lib/zookeeper-3.4.6.jar:/opt/hadoop/share/hadoop/common/hadoop-common-2.7.3-tests.jar:/opt/hadoop/share/hadoop/common/hadoop-common-2.7.3.jar:/opt/hadoop/share/hadoop/common/hadoop-nfs-2.7.3.jar:/opt/hadoop/share/hadoop/hdfs:/opt/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/opt/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/opt/hadoop/share/hadoop/hdfs/lib/-block-placement-policy-1.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar:/opt/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/opt/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/opt/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/opt/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/opt/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/ranger-hdfs-plugin-shim-1.1.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/ranger-plugin-classloader-1.1.0.jar:/opt/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/opt/hadoop/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/opt/hadoop/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/opt/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.7.3-tests.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.7.3.jar:/opt/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/lib/activation-1.1.jar:/opt/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/hadoop/share/hadoop/yarn/lib/asm-3.2.jar:/opt/hadoop/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/hadoop/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/hadoop/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/opt/hadoop/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/hadoop/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/hadoop/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/hadoop/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/hadoop/share/hadoop/yarn/lib/guava-11.0.2.jar:/opt/hadoop/share/hadoop/yarn/lib/guice-3.0.jar:/opt/hadoop/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/hadoop/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/opt/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/opt/hadoop/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/opt/hadoop/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/opt/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/hadoop/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/hadoop/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/hadoop/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/hadoop/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/hadoop/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/hadoop/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/hadoop/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/hadoop/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/hadoop/share/hadoop/yarn/lib/jetty-6.1.26.jar:/opt/hadoop/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/opt/hadoop/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/opt/hadoop/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/hadoop/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/hadoop/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/opt/hadoop/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/hadoop/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/hadoop/share/hadoop/yarn/lib/spark-2.3.1-yarn-shuffle.jar:/opt/hadoop/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/hadoop/share/hadoop/yarn/lib/xz-1.0.jar:/opt/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar:/opt/hadoop/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-api-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-client-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-registry-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.3.jar:/opt/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.3.jar:/opt/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/opt/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/opt/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/opt/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/opt/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/opt/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/opt/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/opt/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.3.jar:/opt/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/opt/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/opt/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/opt/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/opt/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/opt/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/opt/hadoop/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/hadoop/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/opt/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/opt/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/opt/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/opt/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.3.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.3.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.3.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.3.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.3.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.3-tests.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.3.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.3.jar:/opt/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar:/opt/hadoop/share/hadoop/tools/lib/hadoop-aliyun-2.9.2-jar-with-dependencies.jar:/opt/hadoop/conf:
2022-02-18 10:41:45,722 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
--------------------------------------------------------------------------------
2022-02-18 10:41:45,723 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Registered
UNIX signal handlers for [TERM, HUP, INT]
2022-02-18 10:41:45,736 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.lingerMs, 5
2022-02-18 10:41:45,736 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: env.java.opts, -Denv=dev
-Dlog4j2.formatMsgNoLookups=true -Xloggc:./gc-%t.log -XX:+PrintGCDetails
-XX:-OmitStackTraceInFastThrow -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC
-XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:+PrintReferenceGC
-XX:+PrintTenuringDistribution -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=20M
-XX:+PrintPromotionFailure -XX:+PrintGCCause -XX:+UseG1GC
-XX:+UnlockExperimentalVMOptions -XX:+UnlockDiagnosticVMOptions
-XX:MaxGCPauseMillis=200 -XX:-ResizePLAB -XX:+ParallelRefProcEnabled
-XX:ParallelGCThreads=8 -XX:ConcGCThreads=4 -XX:G1HeapWastePercent=3
-XX:InitiatingHeapOccupancyPercent=35 -XX:G1MixedGCLiveThresholdPercent=85
2022-02-18 10:41:45,736 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.address, state-machine-cluster.hke-flink
2022-02-18 10:41:45,736 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.bootstrapServers,
http://fat-kafka1..com.cn:9092,http://fat-kafka2..com.cn:9092,http://fat-kafka3..com.cn:9092
2022-02-18 10:41:45,736 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.cluster-id, state-machine-cluster
2022-02-18 10:41:45,736 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: high-availability.storageDir, hdfs:///flink/ha/k8s
2022-02-18 10:41:45,736 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.maxRequestSize, 1048576
2022-02-18 10:41:45,737 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.retries, 0
2022-02-18 10:41:45,737 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: parallelism.default, 1
2022-02-18 10:41:45,737 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.namespace, hke-flink
2022-02-18 10:41:45,737 WARN
org.apache.flink.configuration.GlobalConfiguration [] - Error while
trying to split key and value in configuration file
/opt/flink/conf/flink-conf.yaml:11: "pipeline.classpaths: "
2022-02-18 10:41:45,737 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: historyserver.archive.fs.dir,
hdfs:///flink/completed-jobs/
2022-02-18 10:41:45,737 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.class,
org.apache.flink.metrics.kafka.KafkaReporter
2022-02-18 10:41:45,737 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.memory.process.size, 1024m
2022-02-18 10:41:45,737 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.internal.jobmanager.entrypoint.class,
org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint
2022-02-18 10:41:45,737 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.backend.incremental, true
2022-02-18 10:41:45,737 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: historyserver.archive.fs.refresh-interval, 10000
2022-02-18 10:41:45,738 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.port, 6123
2022-02-18 10:41:45,738 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: high-availability.zookeeper.quorum,
10.69.1.15:2181,10.69.1.16:2181,10.69.1.17:2181
2022-02-18 10:41:45,738 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.backend, rocksdb
2022-02-18 10:41:45,738 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.requestTimeoutMs, 30
2022-02-18 10:41:45,738 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.jobmanager.service-account, flink
2022-02-18 10:41:45,738 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: blob.server.port, 6124
2022-02-18 10:41:45,738 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.checkpoints.num-retained, 2
2022-02-18 10:41:45,738 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: historyserver.web.port, 8082
2022-02-18 10:41:45,738 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.acks, 0
2022-02-18 10:41:45,787 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.batchSize, 16384
2022-02-18 10:41:45,787 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.topic, metrics-yarn-flink-jobs
2022-02-18 10:41:45,787 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.execution.failover-strategy, region
2022-02-18 10:41:45,787 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.savepoints.dir, hdfs:///flink/savepoints
2022-02-18 10:41:45,787 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.taskmanager.cpu, 0.5
2022-02-18 10:41:45,787 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.compressionType, lz4
2022-02-18 10:41:45,787 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.savepoint.ignore-unclaimed-state, false
2022-02-18 10:41:45,787 WARN
org.apache.flink.configuration.GlobalConfiguration [] - Error while
trying to split key and value in configuration file
/opt/flink/conf/flink-conf.yaml:34: "$internal.application.program-args: "
2022-02-18 10:41:45,787 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.container.image,
harbor.xxx.cn/flink/statemachine:v0.0.6
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.numberOfTaskSlots, 1
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.rest-service.exposed.type, NodePort
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.bufferMemory, 33554432
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.jobmanager.node-selector,
kubernetes.io/role:flink-node
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.archive.fs.dir, hdfs:///flink/completed-jobs/
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.target, kubernetes-application
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.memory.process.size, 1600m
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.container.image.pull-secrets, ******
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.rpc.port, 6122
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.attached, true
2022-02-18 10:41:45,788 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: internal.cluster.execution-mode, NORMAL
2022-02-18 10:41:45,789 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.shutdown-on-attached-exit, false
2022-02-18 10:41:45,789 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: pipeline.jars,
local:///opt/flink/usrlib/StateMachineExample.jar
2022-02-18 10:41:45,789 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.taskmanager.node-selector,
kubernetes.io/role:flink-node
2022-02-18 10:41:45,789 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.checkpoints.dir, hdfs:///flink/checkpoints
2022-02-18 10:41:45,944 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Starting
KubernetesApplicationClusterEntrypoint.
2022-02-18 10:41:45,952 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Install
default filesystem.
2022-02-18 10:41:46,017 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Install
security context.
2022-02-18 10:41:46,119 INFO
org.apache.flink.runtime.security.modules.HadoopModule [] - Hadoop user
set to flink (auth:SIMPLE)
2022-02-18 10:41:46,124 INFO
org.apache.flink.runtime.security.modules.JaasModule [] - Jaas file
will be created as /tmp/jaas-2474453062061446432.conf.
2022-02-18 10:41:46,189 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Initializing
cluster services.
2022-02-18 10:41:46,290 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Trying to
start actor system, external address state-machine-cluster.hke-flink:6123, bind
address 0.0.0.0:6123.
2022-02-18 10:41:47,398 INFO akka.event.slf4j.Slf4jLogger
[] - Slf4jLogger started
2022-02-18 10:41:47,488 INFO akka.remote.Remoting
[] - Starting remoting
2022-02-18 10:41:47,788 INFO akka.remote.Remoting
[] - Remoting started; listening on addresses
:[akka.tcp://[email protected]:6123]
2022-02-18 10:41:48,026 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Actor system
started at akka.tcp://[email protected]:6123
2022-02-18 10:41:48,188 INFO org.apache.flink.configuration.Configuration
[] - Config uses fallback configuration key 'jobmanager.rpc.address'
instead of key 'rest.address'
2022-02-18 10:41:48,201 INFO org.apache.flink.runtime.blob.BlobServer
[] - Created BLOB server storage directory
/tmp/blobStore-2c4cb5c6-c2f0-4558-aaa0-c27afbcd97f6
2022-02-18 10:41:48,206 INFO org.apache.flink.runtime.blob.BlobServer
[] - Started BLOB server at 0.0.0.0:6124 - max concurrent requests:
50 - max backlog: 1000
2022-02-18 10:41:48,225 WARN org.apache.flink.runtime.metrics.ReporterSetup
[] - Multiple implementations of the same reporter were found in
'lib' and/or 'plugins' directories for
org.apache.flink.metrics.prometheus.PrometheusReporterFactory. It is
recommended to remove redundant reporter JARs to resolve used versions'
ambiguity.
2022-02-18 10:41:48,225 WARN org.apache.flink.runtime.metrics.ReporterSetup
[] - Multiple implementations of the same reporter were found in
'lib' and/or 'plugins' directories for
org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporterFactory. It is
recommended to remove redundant reporter JARs to resolve used versions'
ambiguity.
2022-02-18 10:41:48,316 INFO org.apache.kafka.clients.producer.ProducerConfig
[] - ProducerConfig values:
acks = 0
batch.size = 16384
bootstrap.servers = [http://fat-kafka1.com.cn:9092,
http://fat-kafka2.com.cn:9092, http://fat-kafka3.com.cn:9092]
buffer.memory = 33554432
client.dns.lookup = default
client.id = flink_opt_metrics
compression.type = lz4
connections.max.idle.ms = 540000
delivery.timeout.ms = 120000
enable.idempotence = false
interceptor.classes = []
key.serializer = class
org.apache.kafka.common.serialization.StringSerializer
linger.ms = 5
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class
org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30
retries = 0
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
security.providers = null
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = https
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = null
value.serializer = class
org.apache.kafka.common.serialization.StringSerializer2022-02-18 10:41:48,432
INFO org.apache.kafka.common.utils.AppInfoParser [] - Kafka
version: 2.4.1
2022-02-18 10:41:48,433 INFO org.apache.kafka.common.utils.AppInfoParser
[] - Kafka commitId: c57222ae8cd7866b
2022-02-18 10:41:48,433 INFO org.apache.kafka.common.utils.AppInfoParser
[] - Kafka startTimeMs: 1645180908430
2022-02-18 10:41:48,491 INFO
org.apache.flink.runtime.metrics.MetricRegistryImpl [] - Periodically
reporting metrics in intervals of 10 s for reporter kafka of type
org.apache.flink.metrics.kafka.KafkaReporter.
2022-02-18 10:41:48,499 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Trying to
start actor system, external address state-machine-cluster.hke-flink:0, bind
address 0.0.0.0:0.
2022-02-18 10:41:48,589 INFO akka.event.slf4j.Slf4jLogger
[] - Slf4jLogger started
2022-02-18 10:41:48,594 INFO akka.remote.Remoting
[] - Starting remoting
2022-02-18 10:41:48,687 INFO akka.remote.Remoting
[] - Remoting started; listening on addresses
:[akka.tcp://[email protected]:38189]
2022-02-18 10:41:48,794 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Actor system
started at akka.tcp://[email protected]:38189
2022-02-18 10:41:48,811 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for
org.apache.flink.runtime.metrics.dump.MetricQueryService at
akka://flink-metrics/user/rpc/MetricQueryService .
2022-02-18 10:41:49,301 INFO org.apache.flink.configuration.Configuration
[] - Config uses fallback configuration key 'jobmanager.rpc.address'
instead of key 'rest.address'
2022-02-18 10:41:49,303 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Upload
directory /tmp/flink-web-9f65223b-94cd-46b4-aff8-e79861f9b00a/flink-web-upload
does not exist.
2022-02-18 10:41:49,304 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Created
directory /tmp/flink-web-9f65223b-94cd-46b4-aff8-e79861f9b00a/flink-web-upload
for file uploads.
2022-02-18 10:41:49,322 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Starting
rest endpoint.
2022-02-18 10:41:49,513 WARN org.apache.kafka.clients.NetworkClient
[] - [Producer clientId=flink_opt_metrics] Bootstrap broker
fat-kafka2..com.cn:9092 (id: -2 rack: null) disconnected
2022-02-18 10:41:49,812 INFO org.apache.kafka.clients.Metadata
[] - [Producer clientId=flink_opt_metrics] Cluster ID:
HUGQ34tqTTWfjqP4fwawuw
2022-02-18 10:41:49,992 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils [] - Determined
location of main cluster component log file: /opt/flink/log/jobmanager.log
2022-02-18 10:41:49,992 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils [] - Determined
location of main cluster component stdout file: /opt/flink/log/jobmanager.out
2022-02-18 10:41:50,224 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Rest
endpoint listening at state-machine-cluster.hke-flink:8081
2022-02-18 10:41:50,226 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] -
http://state-machine-cluster.hke-flink:8081 was granted leadership with
leaderSessionID=00000000-0000-0000-0000-000000000000
2022-02-18 10:41:50,227 INFO
org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Web frontend
listening at http://state-machine-cluster.hke-flink:8081.
2022-02-18 10:41:50,245 INFO
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils [] - The derived
from fraction jvm overhead memory (102.400mb (107374184 bytes)) is less than
its min value 192.000mb (201326592 bytes), min value will be used instead
2022-02-18 10:41:50,246 INFO
org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils [] - The derived
from fraction network memory (57.600mb (60397978 bytes)) is less than its min
value 64.000mb (67108864 bytes), min value will be used instead
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.lingerMs, 5
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: env.java.opts, -Denv=dev
-Dlog4j2.formatMsgNoLookups=true -Xloggc:./gc-%t.log -XX:+PrintGCDetails
-XX:-OmitStackTraceInFastThrow -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC
-XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:+PrintReferenceGC
-XX:+PrintTenuringDistribution -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=20M
-XX:+PrintPromotionFailure -XX:+PrintGCCause -XX:+UseG1GC
-XX:+UnlockExperimentalVMOptions -XX:+UnlockDiagnosticVMOptions
-XX:MaxGCPauseMillis=200 -XX:-ResizePLAB -XX:+ParallelRefProcEnabled
-XX:ParallelGCThreads=8 -XX:ConcGCThreads=4 -XX:G1HeapWastePercent=3
-XX:InitiatingHeapOccupancyPercent=35 -XX:G1MixedGCLiveThresholdPercent=85
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.address, state-machine-cluster.hke-flink
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.bootstrapServers,
http://fat-kafka1..com.cn:9092,http://fat-kafka2..com.cn:9092,http://fat-kafka3..com.cn:9092
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.cluster-id, state-machine-cluster
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: high-availability.storageDir, hdfs:///flink/ha/k8s
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.maxRequestSize, 1048576
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.retries, 0
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: parallelism.default, 1
2022-02-18 10:41:50,308 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.namespace, hke-flink
2022-02-18 10:41:50,309 WARN
org.apache.flink.configuration.GlobalConfiguration [] - Error while
trying to split key and value in configuration file
/opt/flink/conf/flink-conf.yaml:11: "pipeline.classpaths: "
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: historyserver.archive.fs.dir,
hdfs:///flink/completed-jobs/
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.class,
org.apache.flink.metrics.kafka.KafkaReporter
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.memory.process.size, 1024m
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.internal.jobmanager.entrypoint.class,
org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.backend.incremental, true
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: historyserver.archive.fs.refresh-interval, 10000
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.rpc.port, 6123
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: high-availability.zookeeper.quorum,
10.69.1.15:2181,10.69.1.16:2181,10.69.1.17:2181
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.backend, rocksdb
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.requestTimeoutMs, 30
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.jobmanager.service-account, flink
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: blob.server.port, 6124
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.checkpoints.num-retained, 2
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: historyserver.web.port, 8082
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.acks, 0
2022-02-18 10:41:50,309 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.batchSize, 16384
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.topic, metrics-yarn-flink-jobs
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.execution.failover-strategy, region
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.savepoints.dir, hdfs:///flink/savepoints
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.taskmanager.cpu, 0.5
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.compressionType, lz4
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.savepoint.ignore-unclaimed-state, false
2022-02-18 10:41:50,310 WARN
org.apache.flink.configuration.GlobalConfiguration [] - Error while
trying to split key and value in configuration file
/opt/flink/conf/flink-conf.yaml:34: "$internal.application.program-args: "
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.container.image,
harbor..cn/flink/statemachine:v0.0.6
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.numberOfTaskSlots, 1
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.rest-service.exposed.type, NodePort
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: metrics.reporter.kafka.bufferMemory, 33554432
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.jobmanager.node-selector,
kubernetes.io/role:flink-node
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.archive.fs.dir, hdfs:///flink/completed-jobs/
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.target, kubernetes-application
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: jobmanager.memory.process.size, 1600m
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.container.image.pull-secrets, ******
2022-02-18 10:41:50,310 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: taskmanager.rpc.port, 6122
2022-02-18 10:41:50,311 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.attached, true
2022-02-18 10:41:50,311 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: internal.cluster.execution-mode, NORMAL
2022-02-18 10:41:50,311 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: execution.shutdown-on-attached-exit, false
2022-02-18 10:41:50,311 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: pipeline.jars,
local:///opt/flink/usrlib/StateMachineExample.jar
2022-02-18 10:41:50,311 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: kubernetes.taskmanager.node-selector,
kubernetes.io/role:flink-node
2022-02-18 10:41:50,311 INFO
org.apache.flink.configuration.GlobalConfiguration [] - Loading
configuration property: state.checkpoints.dir, hdfs:///flink/checkpoints
2022-02-18 10:41:50,320 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager at
akka://flink/user/rpc/resourcemanager_0 .
2022-02-18 10:41:50,411 INFO
org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess [] -
Start SessionDispatcherLeaderProcess.
2022-02-18 10:41:50,414 INFO
org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess [] -
Recover all persisted job graphs.
2022-02-18 10:41:50,415 INFO
org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess [] -
Successfully recovered 0 persisted job graphs.
2022-02-18 10:41:50,499 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for
org.apache.flink.runtime.dispatcher.StandaloneDispatcher at
akka://flink/user/rpc/dispatcher_1 .
2022-02-18 10:41:50,604 INFO org.apache.flink.client.ClientUtils
[] - Starting program (detached: false)
2022-02-18 10:41:50,698 INFO stdout
[] - Usage with built-in data generator: StateMachineExample
[--error-rate <probability-of-invalid-transition>] [--sleep
<sleep-per-record-in-ms>]
2022-02-18 10:41:50,698 INFO stdout
[] - Usage with Kafka: StateMachineExample --kafka-topic <topic>
[--brokers <brokers>]
2022-02-18 10:41:50,698 INFO stdout
[] - Options for both the above setups:
2022-02-18 10:41:50,698 INFO stdout
[] - [--backend <file|rocks>]
2022-02-18 10:41:50,698 INFO stdout
[] - [--checkpoint-dir <filepath>]
2022-02-18 10:41:50,698 INFO stdout
[] - [--async-checkpoints <true|false>]
2022-02-18 10:41:50,698 INFO stdout
[] - [--incremental-checkpoints <true|false>]
2022-02-18 10:41:50,698 INFO stdout
[] - [--output <filepath> OR null for stdout]
2022-02-18 10:41:50,698 INFO stdout
[] -
2022-02-18 10:41:50,700 INFO stdout
[] - Using standalone source with error rate 0.000000 and sleep
delay 1 millis
2022-02-18 10:41:50,700 INFO stdout
[] -
2022-02-18 10:41:50,806 INFO
org.apache.flink.contrib.streaming.state.RocksDBStateBackend [] - Using
predefined options: DEFAULT.
2022-02-18 10:41:50,806 INFO
org.apache.flink.contrib.streaming.state.RocksDBStateBackend [] - Using default
options factory: DefaultConfigurableOptionsFactory{configuredOptions={}}.
2022-02-18 10:41:50,999 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class
org.apache.flink.streaming.examples.statemachine.event.Event does not contain a
setter for field type
2022-02-18 10:41:51,000 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - Class class
org.apache.flink.streaming.examples.statemachine.event.Event cannot be used as
a POJO type because not all fields are valid POJO fields, and must be processed
as GenericType. Please read the Flink documentation on "Data Types &
Serialization" for details of the effect on performance.
2022-02-18 10:41:51,195 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - class
org.apache.flink.streaming.examples.statemachine.event.Alert does not contain a
setter for field address
2022-02-18 10:41:51,195 INFO org.apache.flink.api.java.typeutils.TypeExtractor
[] - Class class
org.apache.flink.streaming.examples.statemachine.event.Alert cannot be used as
a POJO type because not all fields are valid POJO fields, and must be processed
as GenericType. Please read the Flink documentation on "Data Types &
Serialization" for details of the effect on performance.
2022-02-18 10:41:51,707 INFO
org.apache.flink.client.deployment.application.executors.EmbeddedExecutor [] -
Job 7422577421a74b52bb14d76b076728f0 is submitted.
2022-02-18 10:41:51,707 INFO
org.apache.flink.client.deployment.application.executors.EmbeddedExecutor [] -
Submitting Job with JobId=7422577421a74b52bb14d76b076728f0.
2022-02-18 10:41:52,614 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Received
JobGraph submission 7422577421a74b52bb14d76b076728f0 (State machine job).
2022-02-18 10:41:52,615 INFO
org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Submitting
job 7422577421a74b52bb14d76b076728f0 (State machine job).
2022-02-18 10:41:52,794 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService
[] - Starting RPC endpoint for
org.apache.flink.runtime.jobmaster.JobMaster at
akka://flink/user/rpc/jobmanager_2 .
2022-02-18 10:41:52,803 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Initializing job State machine job
(7422577421a74b52bb14d76b076728f0).
2022-02-18 10:41:52,905 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Using restart back off time strategy
FixedDelayRestartBackoffTimeStrategy(maxNumberRestartAttempts=2147483647,
backoffTimeMS=1000) for State machine job (7422577421a74b52bb14d76b076728f0).
2022-02-18 10:41:53,028 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Running initialization on master for job State machine job
(7422577421a74b52bb14d76b076728f0).
2022-02-18 10:41:53,028 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Successfully ran initialization on master in 0 ms.
2022-02-18 10:41:53,100 INFO
org.apache.flink.runtime.scheduler.adapter.DefaultExecutionTopology [] - Built
1 pipelined regions in 0 ms
2022-02-18 10:41:53,114 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Using job/cluster config to configure application-defined state
backend: RocksDBStateBackend{checkpointStreamBackend=File State Backend
(checkpoints: 'hdfs:/flink/checkpoints', savepoints: 'hdfs:/flink/savepoints',
asynchronous: TRUE, fileStateThreshold: 20480), localRocksDbDirectories=null,
enableIncrementalCheckpointing=TRUE, numberOfTransferThreads=1,
writeBatchSize=2097152}
2022-02-18 10:41:53,114 INFO
org.apache.flink.contrib.streaming.state.RocksDBStateBackend [] - Using
predefined options: DEFAULT.
2022-02-18 10:41:53,114 INFO
org.apache.flink.contrib.streaming.state.RocksDBStateBackend [] - Using
application-defined options factory:
DefaultConfigurableOptionsFactory{configuredOptions={}}.
2022-02-18 10:41:53,114 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Using application-defined state backend:
RocksDBStateBackend{checkpointStreamBackend=File State Backend (checkpoints:
'hdfs:/flink/checkpoints', savepoints: 'hdfs:/flink/savepoints', asynchronous:
TRUE, fileStateThreshold: 20480), localRocksDbDirectories=null,
enableIncrementalCheckpointing=TRUE, numberOfTransferThreads=1,
writeBatchSize=2097152}
2022-02-18 10:41:54,507 INFO
org.apache.flink.kubernetes.KubernetesResourceManagerDriver [] - Recovered 0
pods from previous attempts, current attempt id is 1.
2022-02-18 10:41:54,507 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Recovered 0 workers from previous attempt.
2022-02-18 10:41:54,511 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
ResourceManager
akka.tcp://[email protected]:6123/user/rpc/resourcemanager_0
was granted leadership with fencing token 00000000000000000000000000000000
2022-02-18 10:41:54,589 INFO
org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl [] -
Starting the SlotManager.
2022-02-18 10:41:54,615 WARN org.apache.hadoop.hdfs.DFSUtil
[] - Namenode for flashHadoop remains unresolved for ID nn1. Check
your hdfs-site.xml file to ensure namenodes are configured properly.
2022-02-18 10:41:54,618 WARN org.apache.hadoop.hdfs.DFSUtil
[] - Namenode for flashHadoop remains unresolved for ID nn2. Check
your hdfs-site.xml file to ensure namenodes are configured properly.
2022-02-18 10:41:55,401 WARN
org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2022-02-18 10:41:55,594 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - No checkpoint
found during restore.
2022-02-18 10:41:55,597 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Using failover strategy
org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy@58528d1a
for State machine job (7422577421a74b52bb14d76b076728f0).
2022-02-18 10:41:55,610 INFO
org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl [] - JobManager
runner for job State machine job (7422577421a74b52bb14d76b076728f0) was granted
leadership with session id 00000000-0000-0000-0000-000000000000 at
akka.tcp://[email protected]:6123/user/rpc/jobmanager_2.
2022-02-18 10:41:55,616 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Starting execution of job State machine job
(7422577421a74b52bb14d76b076728f0) under job master id
00000000000000000000000000000000.
2022-02-18 10:41:55,620 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Starting scheduling with scheduling strategy
[org.apache.flink.runtime.scheduler.strategy.PipelinedRegionSchedulingStrategy]
2022-02-18 10:41:55,620 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job State
machine job (7422577421a74b52bb14d76b076728f0) switched from state CREATED to
RUNNING.
2022-02-18 10:41:55,696 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (1/1) (044fe8cb5885eb314fae6f509e0bab3b) switched from CREATED to
SCHEDULED.
2022-02-18 10:41:55,696 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Flat Map ->
Sink: Print to Std. Out (1/1) (6c1dddddc37ce9aa065bc64e56fe7e94) switched from
CREATED to SCHEDULED.
2022-02-18 10:41:55,713 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Cannot serve
slot request, no ResourceManager connected. Adding as pending request
[SlotRequestId{2a31f675258b9e57ed442593af159570}]
2022-02-18 10:41:55,727 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Connecting to ResourceManager
akka.tcp://[email protected]:6123/user/rpc/resourcemanager_*(00000000000000000000000000000000)
2022-02-18 10:41:55,732 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - Resolved ResourceManager address, beginning registration
2022-02-18 10:41:55,734 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Registering job manager
[email protected]://[email protected]:6123/user/rpc/jobmanager_2
for job 7422577421a74b52bb14d76b076728f0.
2022-02-18 10:41:55,789 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Registered job manager
[email protected]://[email protected]:6123/user/rpc/jobmanager_2
for job 7422577421a74b52bb14d76b076728f0.
2022-02-18 10:41:55,793 INFO org.apache.flink.runtime.jobmaster.JobMaster
[] - JobManager successfully registered at ResourceManager, leader
id: 00000000000000000000000000000000.
2022-02-18 10:41:55,795 INFO
org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl [] - Requesting
new slot [SlotRequestId{2a31f675258b9e57ed442593af159570}] and profile
ResourceProfile{UNKNOWN} with allocation id c81ba6f12840d3a70e72a0367e3c0921
from resource manager.
2022-02-18 10:41:55,796 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Request slot with profile ResourceProfile{UNKNOWN} for job
7422577421a74b52bb14d76b076728f0 with allocation id
c81ba6f12840d3a70e72a0367e3c0921.
2022-02-18 10:41:55,801 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Requesting new worker with resource spec WorkerResourceSpec {cpuCores=0.5,
taskHeapSize=25.600mb (26843542 bytes), taskOffHeapSize=0 bytes,
networkMemSize=64.000mb (67108864 bytes), managedMemSize=230.400mb (241591914
bytes)}, current pending count: 1.
2022-02-18 10:41:55,809 INFO
org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled
external resources: []
2022-02-18 10:41:56,005 INFO
org.apache.flink.kubernetes.KubernetesResourceManagerDriver [] - Creating new
TaskManager pod with name state-machine-cluster-taskmanager-1-1 and resource
<1024,0.5>.
2022-02-18 10:41:56,319 INFO
org.apache.flink.kubernetes.KubernetesResourceManagerDriver [] - Pod
state-machine-cluster-taskmanager-1-1 is created.
2022-02-18 10:41:56,389 INFO
org.apache.flink.kubernetes.KubernetesResourceManagerDriver [] - Received new
TaskManager pod: state-machine-cluster-taskmanager-1-1
2022-02-18 10:41:56,391 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Requested worker state-machine-cluster-taskmanager-1-1 with resource spec
WorkerResourceSpec {cpuCores=0.5, taskHeapSize=25.600mb (26843542 bytes),
taskOffHeapSize=0 bytes, networkMemSize=64.000mb (67108864 bytes),
managedMemSize=230.400mb (241591914 bytes)}.
2022-02-18 10:41:57,318 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but SCHEDULED instead.
Aborting checkpoint.
2022-02-18 10:41:59,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but SCHEDULED instead.
Aborting checkpoint.
2022-02-18 10:42:01,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but SCHEDULED instead.
Aborting checkpoint.
2022-02-18 10:42:03,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but SCHEDULED instead.
Aborting checkpoint.
2022-02-18 10:42:05,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but SCHEDULED instead.
Aborting checkpoint.
2022-02-18 10:42:07,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but SCHEDULED instead.
Aborting checkpoint.
2022-02-18 10:42:09,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but SCHEDULED instead.
Aborting checkpoint.
2022-02-18 10:42:11,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but SCHEDULED instead.
Aborting checkpoint.
2022-02-18 10:42:13,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but SCHEDULED instead.
Aborting checkpoint.
2022-02-18 10:42:14,292 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Registering TaskManager with ResourceID state-machine-cluster-taskmanager-1-1
(akka.tcp://[email protected]:6122/user/rpc/taskmanager_0) at ResourceManager
2022-02-18 10:42:14,380 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Registering TaskManager with ResourceID state-machine-cluster-taskmanager-1-1
(akka.tcp://[email protected]:6122/user/rpc/taskmanager_0) at ResourceManager
2022-02-18 10:42:14,480 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Worker state-machine-cluster-taskmanager-1-1 is registered.
2022-02-18 10:42:14,480 INFO
org.apache.flink.runtime.resourcemanager.active.ActiveResourceManager [] -
Worker state-machine-cluster-taskmanager-1-1 with resource spec
WorkerResourceSpec {cpuCores=0.5, taskHeapSize=25.600mb (26843542 bytes),
taskOffHeapSize=0 bytes, networkMemSize=64.000mb (67108864 bytes),
managedMemSize=230.400mb (241591914 bytes)} was requested in current attempt.
Current pending count after registering: 0.
2022-02-18 10:42:14,885 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (1/1) (044fe8cb5885eb314fae6f509e0bab3b) switched from SCHEDULED
to DEPLOYING.
2022-02-18 10:42:14,886 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Source: Custom Source (1/1) (attempt #0) with attempt id
044fe8cb5885eb314fae6f509e0bab3b to state-machine-cluster-taskmanager-1-1 @
10.74.2.229 (dataPort=42891) with allocation id c81ba6f12840d3a70e72a0367e3c0921
2022-02-18 10:42:14,893 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Flat Map ->
Sink: Print to Std. Out (1/1) (6c1dddddc37ce9aa065bc64e56fe7e94) switched from
SCHEDULED to DEPLOYING.
2022-02-18 10:42:14,893 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying
Flat Map -> Sink: Print to Std. Out (1/1) (attempt #0) with attempt id
6c1dddddc37ce9aa065bc64e56fe7e94 to state-machine-cluster-taskmanager-1-1 @
10.74.2.229 (dataPort=42891) with allocation id c81ba6f12840d3a70e72a0367e3c0921
2022-02-18 10:42:15,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but DEPLOYING instead.
Aborting checkpoint.
2022-02-18 10:42:17,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but DEPLOYING instead.
Aborting checkpoint.
2022-02-18 10:42:19,316 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Checkpoint
triggering task Source: Custom Source (1/1) of job
7422577421a74b52bb14d76b076728f0 is not in state RUNNING but DEPLOYING instead.
Aborting checkpoint.
2022-02-18 10:42:20,382 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Flat Map ->
Sink: Print to Std. Out (1/1) (6c1dddddc37ce9aa065bc64e56fe7e94) switched from
DEPLOYING to RUNNING.
2022-02-18 10:42:20,383 INFO
org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source:
Custom Source (1/1) (044fe8cb5885eb314fae6f509e0bab3b) switched from DEPLOYING
to RUNNING.
2022-02-18 10:42:21,333 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering
checkpoint 1 (type=CHECKPOINT) @ 1645180941316 for job
7422577421a74b52bb14d76b076728f0.
2022-02-18 10:42:24,329 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Completed
checkpoint 1 for job 7422577421a74b52bb14d76b076728f0 (8916 bytes in 3009 ms).
2022-02-18 10:42:24,333 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - {code}
> flink job not recover from latest checkpoint on native k8s
> ----------------------------------------------------------
>
> Key: FLINK-26248
> URL: https://issues.apache.org/jira/browse/FLINK-26248
> Project: Flink
> Issue Type: Bug
> Components: Deployment / Kubernetes
> Affects Versions: 1.12.0
> Reporter: zhisheng
> Priority: Major
>
> flink version: 1.12.0
> ha: zk
> mode: application mode
> native k8s
>
> flink job start with savepoint, running for a while，then i delete the JM pod,
> the job will restart with the old savepoint not the latest checkpoint. this
> is not what i want
> if i delete the TM pod, the job will restart with the latest the
> checkpoint，this is what i want
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
