Robert Burke created BEAM-13082:
-----------------------------------
Summary: [Go SDK] Reduce churn in dataWriter by retaining byte
slice.
Key: BEAM-13082
URL: https://issues.apache.org/jira/browse/BEAM-13082
Project: Beam
Issue Type: Improvement
Components: sdk-go
Reporter: Robert Burke
It's been noted that we can reduce allocations and GC overhead produced by the
dataWriter if we change the `w.buf = nil` to `w.buf = w.buf[:0]`. We should
still nil out the buffer after the final flush in Close() however, to avoid
retaining larger byte buffers after bundle termination.
A dataWriter is created per bundle, and is only used and is safe to use by that
bundle 's processing thread. Further, GRPC's Send call doesn't maintain
ownership of the Proto message data after Send returns, allowing this re-use.
A later optimization could use a sync.Pool to maintain a "freelist" of buffers
to further reduce per bundle allocations but this would likely only be
noticeable in streaming contexts. Such a free list should have a cap of keeping
buffers under some threshold (say slices under 64MB in cap) to avoid retaining
overly large buffers that aren't in active use. This idea though is out of
scope for a first pass.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
