Re: Multi-stream question

2018-04-07 Thread Puneet Kinra
Seeking for the same answer, you want connect multiple streams? On Sat, Apr 7, 2018 at 5:34 AM, Michael Latta wrote: > I would like to “join” several streams (>3) in a custom operator. Is this > feasible in Flink? > > > Michael > -- *Cheers * *Puneet Kinra* *Mobile:+918800167808 | Skype :

Re: logging question

2018-04-07 Thread JP de Vooght
Nico, all, I am still stuck with this. Upgraded the docker image to 1.4.2 and the AMIDST library to 0.7.0 Just noticed this issue which signals logging issues outside transforms: https://issues.apache.org/jira/browse/FLINK-7990 Could this be related? Although I don't see the relation to logback.

Re: logging question

2018-04-07 Thread JP de Vooght
Just to recap I use Flink 1.4.2 with Docker compose which launches a jobmanager and a taskmanager. My hope is to learn another library which can be used with Flink, so logging is important to me. I start the cluster and deploy the following task (I dropped all calls to that library so I can focus

Volume question

2018-04-07 Thread TechnoMage
I have a use case that I wonder if Flink handles well: 1) 5M+ keys in a KeyedStream 2) Using RichFlatMap to track data on each key Will Flink spread one operator’s partitions over multiple machines/taskmanager/jobmanager? Michael

Two questions on Flink externalized checkpoints

2018-04-07 Thread James Yu
I left a question on stackoverflow ( https://stackoverflow.com/questions/49712817/two-questions-on-flink-externalized-checkpoints) and ask again in mail-list in case you check mail more often. I have two questions on Flink externalized checkpoints (Q1) I can set "state.checkpoints.dir" in flink-c

回复:Checkpoints very slow with high backpressure

2018-04-07 Thread Zhijiang(wangzhijiang999)
   Backpressure is indeed delayed the checkpoints because of gradually accumulated inflighting network buffers before barrier alignment.   As Piotr explained, 1.5 can improve to some extent.    After 1.5 we plan to further speed the checkpoint by controlling the channel reader to imp