AsyncFunction used twice with Asyncdatastream.unorderedWait - 2nd function's asyncInvoke not getting called

2018-07-26 Thread Vijay Balakrishnan
Hi, I have 2 AsyncFunctions SampleCopyAsyncFunction and SampleSinkAsyncFunction called with AsyncDataStream.unorderedWait. The 1st AsyncDataStream.unorderedWait’s SampleCopyAsyncFunction .asyncInvoke gets called properly but the 2nd SampleSinkAsyncFunction.asyncInvoke never gets called(though open

Re: Flink resource manager unable to connect to mesos after restart

2018-07-26 Thread Renjie Liu
OK, I'll close 7470 On Thu, Jul 26, 2018 at 11:25 PM Gary Yao wrote: > Hi, > > Sorry for the late reply. I have seen that you debugged this already and > created FLINK-9936. Thank you for looking into the issue. I think your > conclusions are correct. I just wanted to note that there is an even

Committing Kafka Transactions during Savepoint

2018-07-26 Thread Scott Kidder
I recently began using the exactly-once processing semantic with the Kafka 0.11 producer in Flink 1.4.2. It's been working great! Are Kafka transactions committed when creating a Flink savepoint? How does this affect the recovery behavior in Flink if, before the completion of the next checkpoint,

Flink Cluster and Pipeline Version Compatibility?

2018-07-26 Thread jlist9
I was trying to find some comparability tables between various versions of Flink clusters and pipeline jars but haven't run into any so far. Is it so that the pipeline jars must be build with the same version of the cluster they'll be running on? Or there are some backward comparability? If it's al

Re: Flink resource manager unable to connect to mesos after restart

2018-07-26 Thread Gary Yao
Hi, Sorry for the late reply. I have seen that you debugged this already and created FLINK-9936. Thank you for looking into the issue. I think your conclusions are correct. I just wanted to note that there is an even older ticket describing the same problem: https://issues.apache.org/jira/bro

Re: Questions on Unbounded number of keys

2018-07-26 Thread Andrey Zagrebin
Hi Chang Liu, The unbounded nature of the stream keyed or not should not lead to out of memory. Flink parallel keyed operator instances have fixed number (parallelism) and just process some range of keyed elements, in your example it is a subrange of session ids. The keyed processed element