[
https://issues.apache.org/jira/browse/FLINK-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116153#comment-17116153
]
Stephan Ewen commented on FLINK-12675:
--------------------------------------
The existing Kafka Connector is already super complex (partially due to some
bad architecture) and very hard to keep stable.
We have been burned a bit in the past with adding more non-trivial features.
Given that this is probably the most important connector, I would favor to not
add more features to it at this point.
Can you fork the Kafka Connector and add the event-time alignment to that fork?
That way we don't affect the connector in the core. Putting the connector onto
https://flink-packages.org would be a way to share it with more users.
For the new Source Interface, I would be up to start a discussion about how a
generic event-time alignment mechanism would look like, implemented in the
{{SourceReaderBase}} and {{SplitReader}} classes. Something like an extended
{{SplitReader}} interface that can handle callbacks in the form of
{{suspendSplitReadind(splitId)}} and {{resumeSplitReading(splitId)}} or so.
> Event time synchronization in Kafka consumer
> --------------------------------------------
>
> Key: FLINK-12675
> URL: https://issues.apache.org/jira/browse/FLINK-12675
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Kafka
> Reporter: Thomas Weise
> Assignee: Akshay Aggarwal
> Priority: Major
> Labels: pull-request-available
> Attachments: 0001-Kafka-event-time-alignment.patch
>
>
> Integrate the source watermark tracking into the Kafka consumer and implement
> the sync mechanism (different consumer model, compared to Kinesis).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
