[
https://issues.apache.org/jira/browse/BEAM-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Burke closed BEAM-8920.
------------------------------
Fix Version/s: Not applicable
Resolution: Fixed
[~stephydx] resolved this in the linked PR. Thanks!
> Go SDK: faster transforms/filter.Distinct with CombinePerKey
> ------------------------------------------------------------
>
> Key: BEAM-8920
> URL: https://issues.apache.org/jira/browse/BEAM-8920
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: D. Yang
> Priority: Minor
> Fix For: Not applicable
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The current implementation:
> 1. add fixed value 1: P<T> --> P<<T, 1>>
> 2. group by key: P<<T, 1>> --> GBK<T, 1>
> 3. drop the value: P<distinct T>
> The new proposed implementation:
> 1. ditto
> 2. combine by key: P<<T, 1>> --> P<<distinct T, 1>>
> 3. ditto
> CombinePerKey performs a pre-GBK ParDo, which is useful to reduce the shuffle
> size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
