[
https://issues.apache.org/jira/browse/FLINK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311962#comment-14311962
]
Márton Balassi commented on FLINK-1493:
---------------------------------------
Hey Matthias,
Thanks for looking into this.
The basic model you described seems appealing to me as it avoids deadlocks, but
it might result in blowing up the buffers. This morning I've overheard a
discussion between [~gyfora] and [~StephanEwen] on this issue in means of fault
tolerance, so I'd hand this over to them.
As for your questions I can answer the second one: StreamRecord is serialized
through the StreamRecordSerializer, motivated by the TupleSerializer and the
TypeSerializer in general. Compared to the old Record/Value types this
separates the data type from its serialization.
> Support for streaming jobs preserving global ordering of records
> ----------------------------------------------------------------
>
> Key: FLINK-1493
> URL: https://issues.apache.org/jira/browse/FLINK-1493
> Project: Flink
> Issue Type: New Feature
> Components: Streaming
> Reporter: Márton Balassi
>
> Distributed streaming jobs do not give total, global ordering guarantees for
> records only partial ordering is provided by the system: records travelling
> on the same exact route of the physical plan are ordered, but they aren't
> between routes.
> It turns out that although this feature can only be implemented via "merge
> sorting" in the input buffers on a timestamp field thus creating substantial
> latency is still desired for a number of applications.
> Just a heads up for the implementation: the sorting introduces back pressure
> in the buffers and might cause deadlocks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
