[
https://issues.apache.org/jira/browse/FLINK-8154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16266782#comment-16266782
]
Patrick Lucas commented on FLINK-8154:
--------------------------------------
Is `act-monitor-flink-jobmanager` a Kubernetes Service? If so, it's looking
like the Service's IP is `10.3.0.81`, but the Pod's IP is `10.2.43.51`.
Depending on your network configuration, these may not be routable to each
other.
Can you confirm connectivity by, for example, running `nc -v 10.2.43.51 6123`
from within the Pod? (You may have to install `netcat`)
> JobSubmissionClientActor submited job,but there is no connection to a
> JobManager
> ---------------------------------------------------------------------------------
>
> Key: FLINK-8154
> URL: https://issues.apache.org/jira/browse/FLINK-8154
> Project: Flink
> Issue Type: Bug
> Components: JobManager
> Affects Versions: 1.3.2
> Environment: Kubernetes 1.8.3, Platform "Linux/amd64"
> Reporter: Gregory Melekh
> Priority: Blocker
>
> There is JobManager log file bellow.
> 2017-11-26 08:17:13,435 INFO org.apache.flink.client.CliFrontend
> -
> --------------------------------------------------------------------------------
> 2017-11-26 08:17:13,437 INFO org.apache.flink.client.CliFrontend
> - Starting Command Line Client (Version: 1.3.2, Rev:0399bee,
> Date:03.08.2017 @ 10:23:11 UTC)
> 2017-11-26 08:17:13,437 INFO org.apache.flink.client.CliFrontend
> - Current user: root
> 2017-11-26 08:17:13,437 INFO org.apache.flink.client.CliFrontend
> - JVM: OpenJDK 64-Bit Server VM - Oracle Corporation -
> 1.8/25.131-b11
> 2017-11-26 08:17:13,437 INFO org.apache.flink.client.CliFrontend
> - Maximum heap size: 6252 MiBytes
> 2017-11-26 08:17:13,437 INFO org.apache.flink.client.CliFrontend
> - JAVA_HOME: /usr/lib/jvm/java-1.8-openjdk/jre
> 2017-11-26 08:17:13,439 INFO org.apache.flink.client.CliFrontend
> - Hadoop version: 2.7.2
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - JVM Options:
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> -
> -Dlog.file=/opt/flink/log/flink--client-act-monitor-flink-jobmanager-66cd4bdb5c-8kxbh.log
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - -Dlog4j.configuration=file:/etc/flink/log4j-cli.properties
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - -Dlogback.configurationFile=file:/etc/flink/logback.xml
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - Program Arguments:
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - run
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - -c
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - monitoring.flow.AccumulateAll
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - /tmp/monitoring-0.0.1-SNAPSHOT.jar
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - Classpath:
> /opt/flink/lib/flink-python_2.11-1.3.2.jar:/opt/flink/lib/flink-shaded-hadoop2-uber-1.3.2.jar:/opt/flink/lib/log4j-1.2.17.jar:/opt/flink/lib/slf4j-log4j12-1.7.7.jar:/opt/flink/lib/flink-dist_2.11-1.3.2.jar:::
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> -
> --------------------------------------------------------------------------------
> 2017-11-26 08:17:13,440 INFO org.apache.flink.client.CliFrontend
> - Using configuration directory /etc/flink
> 2017-11-26 08:17:13,441 INFO org.apache.flink.client.CliFrontend
> - Trying to load configuration file
> 2017-11-26 08:17:13,443 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: blob.server.port, 6124
> 2017-11-26 08:17:13,443 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.rpc.address, act-monitor-flink-jobmanager
> 2017-11-26 08:17:13,443 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.rpc.port, 6123
> 2017-11-26 08:17:13,443 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.heap.mb, 1024
> 2017-11-26 08:17:13,443 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.web.log.path,
> /etc/flink/log/act-monitor-flink-jobmanager.log
> 2017-11-26 08:17:13,444 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: taskmanager.heap.mb, 1024
> 2017-11-26 08:17:13,444 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: taskmanager.numberOfTaskSlots, 2
> 2017-11-26 08:17:13,612 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: blob.server.port, 6124
> 2017-11-26 08:17:13,612 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.rpc.address, act-monitor-flink-jobmanager
> 2017-11-26 08:17:13,612 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.rpc.port, 6123
> 2017-11-26 08:17:13,612 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.heap.mb, 1024
> 2017-11-26 08:17:13,612 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.web.log.path,
> /etc/flink/log/act-monitor-flink-jobmanager.log
> 2017-11-26 08:17:13,612 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: taskmanager.heap.mb, 1024
> 2017-11-26 08:17:13,613 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: taskmanager.numberOfTaskSlots, 2
> 2017-11-26 08:17:13,639 INFO
> org.apache.flink.runtime.security.modules.HadoopModule - Hadoop user
> set to root (auth:SIMPLE)
> 2017-11-26 08:17:13,669 INFO org.apache.flink.client.CliFrontend
> - Running 'run' command.
> 2017-11-26 08:17:13,675 INFO org.apache.flink.client.CliFrontend
> - Building program from JAR file
> 2017-11-26 08:17:13,764 INFO org.apache.flink.client.CliFrontend
> - Cluster configuration: Standalone cluster with JobManager at
> act-monitor-flink-jobmanager/10.3.0.81:6123
> 2017-11-26 08:17:13,765 INFO org.apache.flink.client.CliFrontend
> - Using address act-monitor-flink-jobmanager:6123 to connect to
> JobManager.
> 2017-11-26 08:17:13,765 INFO org.apache.flink.client.CliFrontend
> - JobManager web interface address
> http://act-monitor-flink-jobmanager:8081
> 2017-11-26 08:17:13,765 INFO org.apache.flink.client.CliFrontend
> - Starting execution of program
> 2017-11-26 08:17:13,765 INFO
> org.apache.flink.client.program.StandaloneClusterClient - Starting
> program in interactive mode
> 2017-11-26 08:17:13,903 INFO org.I0Itec.zkclient.ZkClient
> - JAAS File name: /tmp/jaas-4454069391186324149.conf
> 2017-11-26 08:17:13,906 INFO org.I0Itec.zkclient.ZkEventThread
> - Starting ZkClient event thread.
> 2017-11-26 08:17:13,912 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:zookeeper.version=3.4.6-1569965, built on
> 02/20/2014 09:09 GMT
> 2017-11-26 08:17:13,912 INFO org.apache.zookeeper.ZooKeeper
> - Client
> environment:host.name=act-monitor-flink-jobmanager-66cd4bdb5c-8kxbh
> 2017-11-26 08:17:13,912 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:java.version=1.8.0_131
> 2017-11-26 08:17:13,913 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:java.vendor=Oracle Corporation
> 2017-11-26 08:17:13,913 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:java.home=/usr/lib/jvm/java-1.8-openjdk/jre
> 2017-11-26 08:17:13,913 INFO org.apache.zookeeper.ZooKeeper
> - Client
> environment:java.class.path=/opt/flink/lib/flink-python_2.11-1.3.2.jar:/opt/flink/lib/flink-shaded-hadoop2-uber-1.3.2.jar:/opt/flink/lib/log4j-1.2.17.jar:/opt/flink/lib/slf4j-log4j12-1.7.7.jar:/opt/flink/lib/flink-dist_2.11-1.3.2.jar:::
> 2017-11-26 08:17:13,913 INFO org.apache.zookeeper.ZooKeeper
> - Client
> environment:java.library.path=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 2017-11-26 08:17:13,913 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:java.io.tmpdir=/tmp
> 2017-11-26 08:17:13,915 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:java.compiler=<NA>
> 2017-11-26 08:17:13,915 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:os.name=Linux
> 2017-11-26 08:17:13,915 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:os.arch=amd64
> 2017-11-26 08:17:13,915 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:os.version=4.13.9-coreos
> 2017-11-26 08:17:13,920 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:user.name=root
> 2017-11-26 08:17:13,921 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:user.home=/root
> 2017-11-26 08:17:13,921 INFO org.apache.zookeeper.ZooKeeper
> - Client environment:user.dir=/opt/flink
> 2017-11-26 08:17:13,922 INFO org.apache.zookeeper.ZooKeeper
> - Initiating client connection,
> connectString=act-monitor-zookeeper:2181 sessionTimeout=15000
> watcher=org.I0Itec.zkclient.ZkClient@1807f5a7
> 2017-11-26 08:17:13,937 INFO org.I0Itec.zkclient.ZkClient
> - Waiting for keeper state SyncConnected
> 2017-11-26 08:17:13,946 WARN org.apache.zookeeper.ClientCnxn
> - SASL configuration failed:
> javax.security.auth.login.LoginException: No JAAS configuration section named
> 'Client' was found in specified JAAS configuration file:
> '/tmp/jaas-4454069391186324149.conf'. Will continue connection to Zookeeper
> server without SASL authentication, if Zookeeper server allows it.
> 2017-11-26 08:17:13,948 INFO org.apache.zookeeper.ClientCnxn
> - Opening socket connection to server
> act-monitor-zookeeper.act-monitor.svc.cluster.local/10.3.0.18:2181
> 2017-11-26 08:17:13,949 INFO org.I0Itec.zkclient.ZkClient
> - zookeeper state changed (AuthFailed)
> 2017-11-26 08:17:13,955 INFO org.apache.zookeeper.ClientCnxn
> - Socket connection established to
> act-monitor-zookeeper.act-monitor.svc.cluster.local/10.3.0.18:2181,
> initiating session
> 2017-11-26 08:17:13,967 INFO org.apache.zookeeper.ClientCnxn
> - Session establishment complete on server
> act-monitor-zookeeper.act-monitor.svc.cluster.local/10.3.0.18:2181, sessionid
> = 0x25fde5492b0346c, negotiated timeout = 15000
> 2017-11-26 08:17:13,967 INFO org.I0Itec.zkclient.ZkClient
> - zookeeper state changed (SyncConnected)
> 2017-11-26 08:17:14,048 INFO monitoring.flow.KafkaTopicCreator
> - Topic activity already exists
> 2017-11-26 08:17:14,048 INFO org.I0Itec.zkclient.ZkEventThread
> - Terminate ZkClient event thread.
> 2017-11-26 08:17:14,053 INFO org.apache.zookeeper.ZooKeeper
> - Session: 0x25fde5492b0346c closed
> 2017-11-26 08:17:14,053 INFO org.apache.zookeeper.ClientCnxn
> - EventThread shut down
> 2017-11-26 08:17:14,064 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: blob.server.port, 6124
> 2017-11-26 08:17:14,065 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.rpc.address, act-monitor-flink-jobmanager
> 2017-11-26 08:17:14,065 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.rpc.port, 6123
> 2017-11-26 08:17:14,065 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.heap.mb, 1024
> 2017-11-26 08:17:14,065 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: jobmanager.web.log.path,
> /etc/flink/log/act-monitor-flink-jobmanager.log
> 2017-11-26 08:17:14,065 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: taskmanager.heap.mb, 1024
> 2017-11-26 08:17:14,065 INFO
> org.apache.flink.configuration.GlobalConfiguration - Loading
> configuration property: taskmanager.numberOfTaskSlots, 2
> 2017-11-26 08:17:14,195 INFO
> org.apache.flink.api.java.typeutils.TypeExtractor - class
> java.util.HashMap does not contain a getter for field threshold
> 2017-11-26 08:17:14,195 INFO
> org.apache.flink.api.java.typeutils.TypeExtractor - class
> java.util.HashMap does not contain a setter for field threshold
> 2017-11-26 08:17:14,196 INFO
> org.apache.flink.api.java.typeutils.TypeExtractor - class
> java.util.HashMap is not a valid POJO type because not all fields are valid
> POJO fields.
> 2017-11-26 08:17:14,305 INFO
> org.apache.flink.client.program.StandaloneClusterClient - Starting
> client actor system.
> 2017-11-26 08:17:14,306 INFO
> org.apache.flink.runtime.util.LeaderRetrievalUtils - Trying to
> select the network interface and address to use by connecting to the leading
> JobManager.
> 2017-11-26 08:17:14,306 INFO
> org.apache.flink.runtime.util.LeaderRetrievalUtils - TaskManager
> will try to connect for 10000 milliseconds before falling back to heuristics
> 2017-11-26 08:17:14,307 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Retrieved new target address
> act-monitor-flink-jobmanager/10.3.0.81:6123.
> 2017-11-26 08:17:16,666 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Trying to connect to address
> act-monitor-flink-jobmanager/10.3.0.81:6123
> 2017-11-26 08:17:16,867 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> 'act-monitor-flink-jobmanager-66cd4bdb5c-8kxbh/10.2.43.51': connect timed out
> 2017-11-26 08:17:16,868 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> '/fe80:0:0:0:98e8:4eff:fe17:e7d3%eth0': Network unreachable (connect failed)
> 2017-11-26 08:17:16,919 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/10.2.43.51': connect timed
> out
> 2017-11-26 08:17:16,920 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network
> unreachable (connect failed)
> 2017-11-26 08:17:16,970 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/127.0.0.1': connect timed out
> 2017-11-26 08:17:16,971 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> '/fe80:0:0:0:98e8:4eff:fe17:e7d3%eth0': Network unreachable (connect failed)
> 2017-11-26 08:17:17,972 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/10.2.43.51': connect timed
> out
> 2017-11-26 08:17:17,973 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network
> unreachable (connect failed)
> 2017-11-26 08:17:18,974 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/127.0.0.1': connect timed out
> 2017-11-26 08:17:19,075 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Trying to connect to address
> act-monitor-flink-jobmanager/10.3.0.81:6123
> 2017-11-26 08:17:19,276 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> 'act-monitor-flink-jobmanager-66cd4bdb5c-8kxbh/10.2.43.51': connect timed out
> 2017-11-26 08:17:19,276 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> '/fe80:0:0:0:98e8:4eff:fe17:e7d3%eth0': Network unreachable (connect failed)
> 2017-11-26 08:17:19,327 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/10.2.43.51': connect timed
> out
> 2017-11-26 08:17:19,327 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network
> unreachable (connect failed)
> 2017-11-26 08:17:19,377 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/127.0.0.1': connect timed out
> 2017-11-26 08:17:19,378 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> '/fe80:0:0:0:98e8:4eff:fe17:e7d3%eth0': Network unreachable (connect failed)
> 2017-11-26 08:17:20,379 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/10.2.43.51': connect timed
> out
> 2017-11-26 08:17:20,380 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network
> unreachable (connect failed)
> 2017-11-26 08:17:21,381 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/127.0.0.1': connect timed out
> 2017-11-26 08:17:21,582 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Trying to connect to address
> act-monitor-flink-jobmanager/10.3.0.81:6123
> 2017-11-26 08:17:21,783 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> 'act-monitor-flink-jobmanager-66cd4bdb5c-8kxbh/10.2.43.51': connect timed out
> 2017-11-26 08:17:21,783 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> '/fe80:0:0:0:98e8:4eff:fe17:e7d3%eth0': Network unreachable (connect failed)
> 2017-11-26 08:17:21,834 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/10.2.43.51': connect timed
> out
> 2017-11-26 08:17:21,835 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network
> unreachable (connect failed)
> 2017-11-26 08:17:21,885 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/127.0.0.1': connect timed out
> 2017-11-26 08:17:21,885 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> '/fe80:0:0:0:98e8:4eff:fe17:e7d3%eth0': Network unreachable (connect failed)
> 2017-11-26 08:17:22,887 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/10.2.43.51': connect timed
> out
> 2017-11-26 08:17:22,887 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network
> unreachable (connect failed)
> 2017-11-26 08:17:23,888 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/127.0.0.1': connect timed out
> 2017-11-26 08:17:24,288 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Trying to connect to address
> act-monitor-flink-jobmanager/10.3.0.81:6123
> 2017-11-26 08:17:24,489 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> 'act-monitor-flink-jobmanager-66cd4bdb5c-8kxbh/10.2.43.51': connect timed out
> 2017-11-26 08:17:24,490 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> '/fe80:0:0:0:98e8:4eff:fe17:e7d3%eth0': Network unreachable (connect failed)
> 2017-11-26 08:17:24,540 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/10.2.43.51': connect timed
> out
> 2017-11-26 08:17:24,540 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network
> unreachable (connect failed)
> 2017-11-26 08:17:24,591 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/127.0.0.1': connect timed out
> 2017-11-26 08:17:24,591 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address
> '/fe80:0:0:0:98e8:4eff:fe17:e7d3%eth0': Network unreachable (connect failed)
> 2017-11-26 08:17:25,593 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/10.2.43.51': connect timed
> out
> 2017-11-26 08:17:25,593 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/0:0:0:0:0:0:0:1%lo': Network
> unreachable (connect failed)
> 2017-11-26 08:17:26,595 INFO org.apache.flink.runtime.net.ConnectionUtils
> - Failed to connect from address '/127.0.0.1': connect timed out
> 2017-11-26 08:17:26,595 WARN org.apache.flink.runtime.net.ConnectionUtils
> - Could not connect to
> act-monitor-flink-jobmanager/10.3.0.81:6123. Selecting a local address using
> heuristics.
> 2017-11-26 08:17:26,965 INFO akka.event.slf4j.Slf4jLogger
> - Slf4jLogger started
> 2017-11-26 08:17:27,003 INFO Remoting
> - Starting remoting
> 2017-11-26 08:17:27,142 INFO Remoting
> - Remoting started; listening on addresses
> :[akka.tcp://flink@act-monitor-flink-jobmanager-66cd4bdb5c-8kxbh:43889]
> 2017-11-26 08:17:27,148 INFO
> org.apache.flink.client.program.StandaloneClusterClient - Submitting
> job with JobID: be5c6113768afaeb06ff238e90e65dd8. Waiting for job completion.
> 2017-11-26 08:17:27,159 INFO
> org.apache.flink.runtime.client.JobSubmissionClientActor - Disconnect
> from JobManager null.
> 2017-11-26 08:17:27,167 INFO
> org.apache.flink.runtime.client.JobSubmissionClientActor - Received
> SubmitJobAndWait(JobGraph(jobId: be5c6113768afaeb06ff238e90e65dd8)) but there
> is no connection to a JobManager yet.
> 2017-11-26 08:17:27,167 INFO
> org.apache.flink.runtime.client.JobSubmissionClientActor - Received job
> Collect data into Cassandra (be5c6113768afaeb06ff238e90e65dd8).
> 2017-11-26 08:17:47,219 INFO akka.remote.transport.ProtocolStateActor
> - No response from remote for outbound association. Associate
> timed out after [20000 ms].
> 2017-11-26 08:17:47,227 WARN akka.remote.ReliableDeliverySupervisor
> - Association with remote system
> [akka.tcp://flink@act-monitor-flink-jobmanager:6123] has failed, address is
> now gated for [5000] ms. Reason: [Association failed with
> [akka.tcp://flink@act-monitor-flink-jobmanager:6123]] Caused by: [No response
> from remote for outbound association. Associate timed out after [20000 ms].]
> 2017-11-26 08:18:27,189 ERROR org.apache.flink.client.CliFrontend
> - Error while running the command.
> org.apache.flink.client.program.ProgramInvocationException: The program
> execution failed: Couldn't retrieve the JobExecutionResult from the
> JobManager.
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:478)
> at
> org.apache.flink.client.program.StandaloneClusterClient.submitJob(StandaloneClusterClient.java:105)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:442)
> at
> org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:73)
> at monitoring.flow.AccumulateAll.main(AccumulateAll.java:113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:419)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:381)
> at
> org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:838)
> at
> org.apache.flink.client.CliFrontend.run(CliFrontend.java:259)
> at
> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1086)
> at
> org.apache.flink.client.CliFrontend$2.call(CliFrontend.java:1133)
> at
> org.apache.flink.client.CliFrontend$2.call(CliFrontend.java:1130)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
> at
> org.apache.flink.client.CliFrontend.main(CliFrontend.java:1130)
> Caused by: org.apache.flink.runtime.client.JobExecutionException: Couldn't
> retrieve the JobExecutionResult from the JobManager.
> at
> org.apache.flink.runtime.client.JobClient.awaitJobResult(JobClient.java:309)
> at
> org.apache.flink.runtime.client.JobClient.submitJobAndWait(JobClient.java:396)
> at
> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:467)
> ... 22 more
> Caused by:
> org.apache.flink.runtime.client.JobClientActorConnectionTimeoutException:
> Lost connection to the JobManager.
> at
> org.apache.flink.runtime.client.JobClientActor.handleMessage(JobClientActor.java:219)
> at
> org.apache.flink.runtime.akka.FlinkUntypedActor.handleLeaderSessionID(FlinkUntypedActor.java:101)
> at
> org.apache.flink.runtime.akka.FlinkUntypedActor.onReceive(FlinkUntypedActor.java:68)
> at
> akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
> at
> akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
> at akka.actor.ActorCell.invoke(ActorCell.scala:487)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
> at akka.dispatch.Mailbox.run(Mailbox.scala:220)
> at
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
> at
> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> at
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> at
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> at
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> 2017-11-26 08:18:27,188 INFO
> org.apache.flink.runtime.client.JobSubmissionClientActor - Terminate
> JobClientActor.
> 2017-11-26 08:18:27,194 INFO
> org.apache.flink.runtime.client.JobSubmissionClientActor - Disconnect
> from JobManager null.
> 2017-11-26 08:18:27,207 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator - Shutting down
> remote daemon.
> 2017-11-26 08:18:27,207 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remote daemon
> shut down; proceeding with flushing remote transports.
> 2017-11-26 08:18:27,234 INFO
> akka.remote.RemoteActorRefProvider$RemotingTerminator - Remoting shut
> down.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
