Robert Burke created BEAM-5977:
----------------------------------
Summary: Benchmark buffer use in the Go SDK Harness
Key: BEAM-5977
URL: https://issues.apache.org/jira/browse/BEAM-5977
Project: Beam
Issue Type: Improvement
Components: sdk-go
Reporter: Robert Burke
There's opportunity to reduce CPU & RAM usage with better buffer re-use in
[datamgr.go|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/harness/datamgr.go]
WRT both the small element batch buffer and with large elements.
There should be some benchmarking around both large elements, and smaller
elements of varying sizes, (and ideally mixing), and using that we can measure
subsequent improvements.
As a performance improvement, offhand, maintaining a pair of a couple of
`chunkSize` buffers could be handy and avoid associated GRPC costs, to handle
the smaller elements, as well as flushing large elements immediately and
without copying.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
