[
https://issues.apache.org/jira/browse/BEAM-13443?focusedWorklogId=698833&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-698833
]
ASF GitHub Bot logged work on BEAM-13443:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Dec/21 17:12
Start Date: 20/Dec/21 17:12
Worklog Time Spent: 10m
Work Description: aromanenko-dev merged pull request #16286:
URL: https://github.com/apache/beam/pull/16286
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 698833)
Time Spent: 1h (was: 50m)
> Poor handling of aggregated records in KinesisIO.read
> -----------------------------------------------------
>
> Key: BEAM-13443
> URL: https://issues.apache.org/jira/browse/BEAM-13443
> Project: Beam
> Issue Type: Bug
> Components: io-java-aws
> Reporter: Moritz Mack
> Assignee: Moritz Mack
> Priority: P2
> Labels: aws, aws-sdk-v1, aws-sdk-v2, performance
> Fix For: 2.36.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The way the Kinesis source is implemented it doesn't play well with
> aggregated records.
> Even using configuration options it's fairly hard to configure it in a way
> that becomes sufficiently performant.
> One of the key issues is around bundle size & record queue size vs the number
> of aggregated records per message. These might, in certain situations, exceed
> the internal queue size by far unnecessarily blocking threads and requiring
> thread pools to be forcefully taken down.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
