Hello the current version of Kinesis source (5.0.0) lacks support for KLP de-aggregation [1] There is a pending PR [2] but it breaks the API and will require a new major version of the connector.
I implemented an alternative solution using a DeserializationSchema wrapper which adds de-aggregation to any other DeserializationSchema [3]. Would it be a reasonable approach including this DeserializationSchema wrapper (with proper tests) as part of the connector distribution? A user can decide to use the wrapper if they need de-aggregation. The wrapper works fine even if the stream is not aggregated, with some minimal overhead. The wrapper will be redundant when/if the main fix [1] is released. However, requiring a new major version, this may take a while and some users are blocked from upgrading to the new source by lack of de-aggregation support. Any opinion on this approach? Lorenzo [1] https://issues.apache.org/jira/browse/FLINK-32097 [2] https://github.com/apache/flink-connector-aws/pull/188 [3] https://github.com/aws-samples/amazon-managed-service-for-apache-flink-examples/blob/main/java/KinesisSourceDeaggregation/flink-app/src/main/java/com/amazonaws/services/msf/deaggregation/KinesisDeaggregatingDeserializationSchemaWrapper.java
