Re: Cassandra POJO sink flink 1.4.0 in scala

Hi, shashank agarwal Not sure if I can answer fully your question, but after digging some code, I am not sure if C* connector totally supports Scala case class + CQL data mapping at the moment. I may be totally wrong, and you need to

Re: Cassandra POJO sink flink 1.4.0 in scala

Hi, shashank agarwal AFAIK, in java side, for a pojo data type, you don't need to set query since the CQL data mapping would take care of that whereas dealing with java tuples, you do need to provide a upsert query so that cassandra knows what to insert into the table. Scala tuple case is clear,

Re: org.apache.zookeeper.ClientCnxn, Client session timed out

Thanks! Great to know I do not have to worry duplicates inside Flink. One more question, why this happens? Because TM and JM both check leadership in different interval? > The TM noticed the loss of leadership before the JM did. On Wed, Dec 27, 2017, 13:52 Ufuk Celebi wrote: >

Re: org.apache.zookeeper.ClientCnxn, Client session timed out

On Wed, Dec 27, 2017 at 4:41 PM, Hao Sun wrote: > Somehow TM detected JM leadership loss from ZK and self disconnected? > And couple of seconds later, JM failed to connect to ZK? > Yes, exactly as you describe. The TM noticed the loss of leadership before the JM did. >

org.apache.zookeeper.ClientCnxn, Client session timed out

Hi I need some help to figure out the root cause of this error. I am running flink 1.3.2 on K8S. My cluster has been up and running for almost two weeks and all of a sudden I see this familiar error again, my task manager is killed/lost. There are many ways cause this error, I need help to figure

Re: MergingWindow

Please check your email before sending it the next time as three emails for the same message is a little spammy ;-) This is internal code that is used to implement session windows as far as I can tell. The idea is to not merge the new window as it never had any state associated with it. The

Re: Apache Flink - broadcasting DataStream

Hey Mans! This refers to how sub tasks are connected to each other in your program. If you have a single sub task A1 and three sub tasks B1, B2, B3, broadcast will emit each incoming record at A1 to all B1, B2, B3: A1 --+-> B1 +-> B2 +-> B3 Does this help? On Mon, Dec 25, 2017 at

Re: flink yarn-cluster run job --files

The file URL needs to be accessible from all nodes, e.g. something like S3://... or hdfs://... >From the CLI: ``` Adds a URL to each user code classloader on all nodes in the cluster. The paths must specify a protocol (e.g. file://) and be accessible on all nodes (e.g. by means of a NFS share).

Re: Fetching TaskManager log failed

Thanks for reporting this issue. A few questions: - Which version of Flink are you using? - Does it work up to the point that the Exception is thrown? e.g. for smaller files it's OK? Let me pull in Chesnay (cc'd) who has worked on log fetching for the web runtime. – Ufuk On Tue, Dec 26, 2017

Re: How to stop FlinkKafkaConsumer and make job finished?

Hey Jaxon, I don't think it's possible to control this via the life-cycle methods of your functions. Note that Flink currently does not support graceful stop in a meaningful manner and you can only cancel running jobs. What comes to my mind to cancel on EOF: 1) Extend Kafka consumer to stop

Re: Flink network access control documentation

Hey Elias, thanks for opening a ticket (for reference: https://issues.apache.org/jira/browse/FLINK-8311). I fully agree with adding docs for this. I will try to write something down this week. Your point about JobManagers only coordinating via ZK is correct though. I had a look into the