Hi Gordon,I am moving forward using FlinkKafkaConsumer08.Any best practices
recomm. integrating Kafka+Flink?Thanks+regards
From: Tzu-Li (Gordon) Tai <tzuli...@apache.org>
To: dev@flink.apache.org
Sent: Sunday, April 16, 2017 10:25 PM
Subject: Re: Jobmanager drops upon submitting a jar
Hi Amir,
What do you mean by “modernizing” the FlinkKafkaConsumer implementation? Could
you explain a bit more?
Thanks :) Chiming in just to check if there’s any issue we need to be aware of
..
Cheers,
Gordon
On 14 April 2017 at 5:35:43 AM, amir bahmanyari (amirto...@yahoo.com.invalid)
wrote:
It ends up to be a release gap between the build env libs and the
runtime.nothing else.Am updating everything to the latest+greatest.With the
latest Flink, and the current (old) code the Maven reports:[ERROR] symbol:
class FlinkKafkaConsumer08
Meaning it needs to be replaced with the latest consumer object.
Any suggestions on modernizing the FlinkKafkaConsumer
implementation?Thanks+regards
From: wenlong.lwl <wenlong88....@gmail.com>
To: dev@flink.apache.org; amir bahmanyari <amirto...@yahoo.com>
Sent: Wednesday, April 12, 2017 8:01 PM
Subject: Re: Jobmanager drops upon submitting a jar
Hi, amir, I think you could check the log of job manager to make sure that
job manager [192.168.56.101:6123 <http://flink@192.168.56.101:6123/>] is
running well firstly, you may get what is wrong in the log.
On 13 April 2017 at 08:54, amir bahmanyari <amirto...@yahoo.com.invalid>
wrote:
>
>
>
> Hi Colleagues,I have a simple test job when I submit it to the Flink
> cluster the JM seems to disconnect.Its a one node cluster implemented in a
> VirtualBox Centos 7 VM.Flink starts fine and everything else look fine.
> Following is stack trace.I appreciate a feedback.Cheers
>
> 17/04/12 15:53:04 INFO node.Node: Connected to Node 192.168.56.101
> 17/04/12 15:53:04 INFO config.ConfigurationProvider: Opened bucket default
> 17/04/12 15:53:04 INFO config.ConfigurationProvider: Closed bucket default
> 17/04/12 15:53:04 INFO node.Node: Disconnected from Node 192.168.56.101
> 17/04/12 15:53:07 INFO kafka.FlinkKafkaConsumerBase: Trying to get topic
> metadata from broker localhost:9092 in try 0/3
> 17/04/12 15:53:07 INFO kafka.FlinkKafkaConsumerBase: Consumer is going to
> read the following topics (with number of partitions): abc_pharma_qa (2),
> 17/04/12 15:53:07 INFO environment.RemoteStreamEnvironment: Running
> remotely at 192.168.56.101:6123
> 17/04/12 15:53:07 INFO program.StandaloneClusterClient: Submitting job
> with JobID: c9c717d6a6d0d5ce9a8758b0fb7dae7c. Waiting for job completion.
> Submitting job with JobID: c9c717d6a6d0d5ce9a8758b0fb7dae7c. Waiting for
> job completion.
> 17/04/12 15:53:07 INFO program.StandaloneClusterClient: Starting client
> actor system.
> 17/04/12 15:53:08 INFO slf4j.Slf4jLogger: Slf4jLogger started
> 17/04/12 15:53:08 INFO Remoting: Starting remoting
> 17/04/12 15:53:08 INFO Remoting: Remoting started; listening on addresses
> :[akka.tcp://flink@127.0.0.1:32776]
> 17/04/12 15:53:08 INFO client.JobClientActor: Received job test (
> c9c717d6a6d0d5ce9a8758b0fb7dae7c).
> 17/04/12 15:53:08 INFO client.JobClientActor: Could not submit job test (
> c9c717d6a6d0d5ce9a8758b0fb7dae7c), because there is no connection to a
> JobManager.
> 17/04/12 15:53:08 INFO client.JobClientActor: Disconnect from JobManager
> null.
>
> 17/04/12 15:53:08 WARN remote.ReliableDeliverySupervisor: Association
> with remote system [akka.tcp://flink@192.168.56.101:6123] has failed,
> address is now gated for [5000] ms. Reason is: [Disassociated].
> 17/04/12 15:54:08 INFO client.JobClientActor: Terminate JobClientActor.
> 17/04/12 15:54:08 INFO client.JobClientActor: Disconnect from JobManager
> null.
> 17/04/12 15:54:08 INFO remote.RemoteActorRefProvider$RemotingTerminator:
> Shutting down remote daemon.
> 17/04/12 15:54:08 INFO remote.RemoteActorRefProvider$RemotingTerminator:
> Remote daemon shut down; proceeding with flushing remote transports.
> 17/04/12 15:54:08 INFO remote.RemoteActorRefProvider$RemotingTerminator:
> Remoting shut down.
> Exception in thread "main"
> org.apache.flink.client.program.ProgramInvocationException:
> The program execution failed: Communication with JobManager failed: Lost
> connection to the JobManager.
>
> at org.apache.flink.client.program.ClusterClient.run(
> ClusterClient.java:413)
> at org.apache.flink.client.program.StandaloneClusterClient.
> submitJob(StandaloneClusterClient.java:92)
> at org.apache.flink.client.program.ClusterClient.run(
> ClusterClient.java:389)
> at org.apache.flink.client.program.ClusterClient.run(
> ClusterClient.java:381)
> at org.apache.flink.streaming.api.environment.
> RemoteStreamEnvironment.executeRemotely(RemoteStreamEnvironment.java:209)
> at org.apache.flink.streaming.api.environment.
> RemoteStreamEnvironment.execute(RemoteStreamEnvironment.java:173)
> at com.rfxcel.rts.operations.EventProcessorDriver.start(
> EventProcessorDriver.java:103)
> at com.rfxcel.rts.operations.EventProcessorDriver.main(
> EventProcessorDriver.java:109)
> Caused by: org.apache.flink.runtime.client.JobExecutionException:
> Communication with JobManager failed: Lost connection to the JobManager.
> at org.apache.flink.runtime.client.JobClient.
> submitJobAndWait(JobClient.java:137)
> at org.apache.flink.client.program.ClusterClient.run(
> ClusterClient.java:409)
> ... 7 more
> Caused by:
> org.apache.flink.runtime.client.JobClientActorConnectionTimeoutException:
> Lost connection to the JobManager.
> at org.apache.flink.runtime.client.JobClientActor.
> handleMessage(JobClientActor.java:245)
> at org.apache.flink.runtime.akka.FlinkUntypedActor.
> handleLeaderSessionID(FlinkUntypedActor.java:90)
> at org.apache.flink.runtime.akka.FlinkUntypedActor.onReceive(
> FlinkUntypedActor.java:70)
> at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(
> UntypedActor.scala:167)
> at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
> at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97)
> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
> at akka.actor.ActorCell.invoke(ActorCell.scala:487)
> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
> at akka.dispatch.Mailbox.run(Mailbox.scala:221)
> at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(
> ForkJoinTask.java:260)
> at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.
> pollAndExecAll(ForkJoinPool.java:1253)
> at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.
> runTask(ForkJoinPool.java:1346)
> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(
> ForkJoinPool.java:1979)
> at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(
> ForkJoinWorkerThread.java:107)
>
