[
https://issues.apache.org/jira/browse/BEAM-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niel Markwick resolved BEAM-10047.
----------------------------------
Resolution: Fixed
> SpannerIO: Combine sorting and batching
> ---------------------------------------
>
> Key: BEAM-10047
> URL: https://issues.apache.org/jira/browse/BEAM-10047
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0, 2.20.0, 2.21.0, 2.22.0
> Reporter: Brian Hulette
> Assignee: Niel Markwick
> Priority: P2
> Fix For: 2.23.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> having Spanner IO's grouping, sorting and batching as separate stages makes
> no sense as these stages will be fused in any runner that supports fusion.
> In addition during these stages, even with fusion, at least 3 full copies of
> each mutation are made (serialized then deserialized objects), which leads to
> an extremely large use of memory.
>
> Combining these stages would significantly reduce memory usage, and improve
> performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
