[
https://issues.apache.org/jira/browse/BEAM-13824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Moritz Mack updated BEAM-13824:
-------------------------------
Status: Open (was: Triage Needed)
> Migrate SnsIO.write to async implementation
> -------------------------------------------
>
> Key: BEAM-13824
> URL: https://issues.apache.org/jira/browse/BEAM-13824
> Project: Beam
> Issue Type: Improvement
> Components: io-java-aws
> Reporter: Moritz Mack
> Priority: P2
> Labels: aws-sdk-v2
>
> In BEAM-13203 {{SnsIO.writeAsync}} was deprecated to address the risk of
> data loss in that transform. Unfortunately, it has considerable issues in
> design and implementation with respect to the beam programming model as well
> as general async programming:
> * Output for a bundle may be emitted even after the bundle is finished (lack
> of join / waiting for pending futures), additionally output is emitted from
> multiple threads risking concurrent modification issues on underlying data
> structures.
> * Exceptions disappear into the void (respective tests are implemented
> incorrectly as they don't throw asynchronously and could make one think this
> works).
> * Parallelism is unlimited causing the writer to instantly accumulate
> everything into memory.
> With BEAM-13563 it will be possible to change {{SnsIO.write}} into a more
> performant async implementation without any breaking changes to the public
> user facing API making {{SnsIO.writeAsync}} obsolete.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
