Tan-JiaLiang commented on PR #52: URL: https://github.com/apache/flink-connector-kafka/pull/52#issuecomment-1765621807
@mas-chen Hi Mason, thanks for your feedback. I think empty splits may not be the point in this problem. The point is we should not store the marker (i.e. -1) into state. Currently we handle the empty splits on TMs. If we handle on the enumerator: 1. We need to store the UNASIGNED partition's offset into state in JM when checkpoint/savepoint. 2. We need to consider compatibility with the history state. 3. We need to reconsider the bounded mode and unbounded mode cases. I don't want to make too many changes because current connector seems still healthy, although it has some problem. I'd like to fix it instead of changing it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
