Hello Mani, For Write part it should be already supported since KinesisIO uses KPL to write records under the hood. So, it’s just a question of proper configuration [1][2][3] For Read part, since it’s based on AWS API, it’s more complicated and we need to add a support for this explicitly.
[1] https://github.com/apache/beam/blob/afa59c463ca9c5d42a9ab13402a72c8b95240fa5/sdks/java/io/kinesis/src/main/java/org/apache/beam/sdk/io/kinesis/KinesisIO.java#L657 [2] https://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-config.html <https://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-config.html> [3] https://github.com/awslabs/amazon-kinesis-producer/blob/master/java/amazon-kinesis-producer-sample/default_config.properties <https://github.com/awslabs/amazon-kinesis-producer/blob/master/java/amazon-kinesis-producer-sample/default_config.properties> > On 31 Aug 2020, at 16:12, Sunny, Mani Kolbe <[email protected]> wrote: > > Hello, > > Does Beam have plans to support Kinesis records with aggregation[1]? I see > some code [2] in the repo related to that. Is it planned for any near future > releases? > > Regards, > Mani > > > [1] > https://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-concepts.html#kinesis-kpl-concepts-aggretation > > <https://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-concepts.html#kinesis-kpl-concepts-aggretation> > [2] > https://github.com/apache/beam/blob/22822b8c611f4d8888d3b2039e4f7e7d98fcab39/sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/SimplifiedKinesisClient.java#L181 > > <https://github.com/apache/beam/blob/22822b8c611f4d8888d3b2039e4f7e7d98fcab39/sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/SimplifiedKinesisClient.java#L181>
