Great! Thanks for reporting back :-)
2017-08-09 22:52 GMT+02:00 Chao Wang :
> It seems that the observed long latencies were due to certain one-time
> internal mechanism that only occurred after Flink has received the first
> message. Based on my measurement that mechanism took around 100 ms.
>
>
It seems that the observed long latencies were due to certain one-time
internal mechanism that only occurred after Flink has received the first
message. Based on my measurement that mechanism took around 100 ms.
Now I setup my application the following way, and I observed that the
end-to-end l
Thank you, Fabian.
Maybe there's also some buffers sit between data source and the first
operator? I observed that in my implementation of SourceFunction (using
a socket server, as listed in the previous email), for receiving two
messages, in terms of event time, it took 0.2 ms before the
Sou
One pointer is the StreamExecutionEnvironment.setBufferTimeout() parameter.
Flink's network stack collects records in buffers to send them over the
network. A buffer is sent when it is completely filled or after a
configurable timeout.
So if your program does not process many records, these records
Following the original post, I've tried stripping down my Flink app to
only the following, and then it still exhibits long latencies: after the
second source socket write, it took 90+ milliseconds from data source to
the socket-front in Flink. I would like to ask for pointers about how to
inves
