[ https://issues.apache.org/jira/browse/BEAM-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Bradshaw resolved BEAM-4030. ----------------------------------- Resolution: Fixed Fix Version/s: 2.11.0 > Add CombineFn.compact, similar to Java > -------------------------------------- > > Key: BEAM-4030 > URL: https://issues.apache.org/jira/browse/BEAM-4030 > Project: Beam > Issue Type: Bug > Components: sdk-py-core > Reporter: Robert Bradshaw > Assignee: Robert Bradshaw > Priority: Major > Fix For: 2.11.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Some CombineFns buffer elements in their add_inputs because a combining > operation cost can be effectively amortized across many elements. However, > this introduces the extra (possibly higher) cost of potentially serializing > more expensive buffers through shuffle. We should add a > CombineFn.compact(self, accumulator) method (defaulting to the identity) > similar to what the Java SDK provides which is called when flushing an > element from the PGBKCV table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)