Hi all, Just to give a quick update: there will be contributors from the AWS Kinesis team working on contributing enhanced fan out support to the connector. You can follow the progress here: https://issues.apache.org/jira/browse/FLINK-17688
Cheers, Gordon On Fri, May 15, 2020 at 5:55 PM orionemail <orionem...@protonmail.com> wrote: > Hi, > > We also recently needed this functionality, unfortunately we were unable > to implement it ourselves so changed our plan accordingly. > > However we very much see the benefit for this feature and would be > interested in following the JIRA ticket. > > Thanks > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Thursday, 14 May 2020 11:34, Xiaolong Wang <xiaolong.w...@smartnews.com> > wrote: > > Thanks, I'll check it out. > > On Thu, May 14, 2020 at 6:26 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org> > wrote: > >> Hi Xiaolong, >> >> You are right, the way the Kinesis connector is implemented / the way the >> AWS APIs are used, does not allow it to consume Kinesis streams with >> enhanced fan-out enabled consumers [1]. >> Could you open a JIRA ticket for this? >> As far as I can tell, this could be a valuable contribution to the >> connector for Kinesis users who require dedicated throughput isolated from >> other running consumers. >> >> Cheers, >> Gordon >> >> [1] >> https://docs.aws.amazon.com/streams/latest/dev/building-enhanced-consumers-api.html >> >> On Wed, May 13, 2020 at 1:44 PM Xiaolong Wang < >> xiaolong.w...@smartnews.com> wrote: >> >>> Hello Flink Community! >>> >>> I'm currently coding on a project relying on AWS Kinesis. With the >>> provided connector (flink-connector-kinesis_2.11;1.10.0), I can consume the >>> message. >>> >>> But as the main stream is used among several other teams, I was >>> required to use the enhance fanout of Kinesis. I checked the connector code >>> and found no implementations. >>> >>> Has this issue occurred to anyone before? >>> >>> Thanks for your help. >>> >> >