Re: How can I handle backpressure with event time.

2017-11-05 Thread yunfan123
It seems can't support consume multi topics with different deserialization schema. I use protobuf, different topic mapping to different schema. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

ExecutionGraph not serializable

2017-11-05 Thread XiangWei Huang
Hi Flink users, Flink Jobmanager throw a NotSerializableException when i used JobMasterGateway to get ExecutionGraph of a specific job with message RequestJob(jobID). Blow is the detail of Exception: [ERROR] [akka.remote.EndpointWriter] - Transient association error (association

Kafka Consumer fetch-size/rate and Producer queue timeout

2017-11-05 Thread Ashish Pokharel
All, I am starting to notice a strange behavior in a particular streaming app. I initially thought it was a Producer issue as I was seeing timeout exceptions (records expiring in queue. I did try to modify request.timeout.ms, linger.ms etc to help with the issue if it were caused by a sudden

Re: Flink memory usage

2017-11-05 Thread Jürgen Thomann
Can you use wget (curl will work as well)? You can find the taskmanagers with wget -O - http://localhost:8081/taskmanagers and wget -O - http://localhost:8081/taskmanagers/request> to see detailed jvm memory stats. localhost:8081 is in my example the jobmanager. On 04.11.2017 16:19, AndreaKinn

RE: Negative values using latency marker

2017-11-05 Thread Sofer, Tovi
Hi Nico, Actually the run below is on my local machine, and both Kafka and flink run on it. Thanks and regards, Tovi -Original Message- From: Nico Kruber [mailto:n...@data-artisans.com] Sent: יום ו 03 נובמבר 2017 15:22 To: user@flink.apache.org Cc: Sofer, Tovi [ICG-IT]

Re: JobManager web interface redirect strategy when running in HA

2017-11-05 Thread Jürgen Thomann
I think you can solve this already with a health check (health monitor in OpenStack?). I'm currently using GET requests to / and if they don't reply with a  200 code the LB will not use them. Only the Leader answers with a 200 code whereas the others send a redirect with 30x code which should