lostluck opened a new issue, #22704: URL: https://github.com/apache/beam/issues/22704
### What would you like to happen? The Go SDK's combiner lifting currently encodes keys twice for caching. First it encodes values into a hash function, and then it repeats it to get the []byte for byte by byte comparisons in the cache. https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/combine.go#L469 https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/combine.go#L483 For expensive keys, this can become very expensive without additional caching schemes. Ideally this encoding happens once, instead of twice for each key being looked up. Note: Windowed byte equality is important to use for final lookups because that's the only way to ensure equivalence to runners. Beam Model GBKs use byte equality for keys as their basis. ### Issue Priority Priority: 2 ### Issue Component Component: sdk-go -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
