Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/879#issuecomment-117974230
Concerning watermarks and performance: Have we considered sending
watermarks as records, rather than events?
- Events in data channels are inevitably more heavyweight than records.
They have less efficient serialization and flush buffers.
- In cases where we can guarantee that records have ascending time stamps
in a stream (sub) partition, every record is a watermark by itself, and there
is no need to have extra watermarks.
The downside of having records rather than events is that we sometimes
really want to flush the buffers with a watermark. On the other hand, buffers
are flushed periodically anyways, so no big deal, probably.
If our stream records are always 2-tuples (timestamp, record) , then we
could do something like shipping (timestamp, null) records as watermarks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
