[
https://issues.apache.org/jira/browse/BEAM-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Burke updated BEAM-13082:
--------------------------------
Status: Open (was: Triage Needed)
> [Go SDK] Reduce churn in dataWriter by retaining byte slice.
> ------------------------------------------------------------
>
> Key: BEAM-13082
> URL: https://issues.apache.org/jira/browse/BEAM-13082
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Priority: P2
>
> It's been noted that we can reduce allocations and GC overhead produced by
> the dataWriter if we change the `w.buf = nil` to `w.buf = w.buf[:0]`. We
> should still nil out the buffer after the final flush in Close() however, to
> avoid retaining larger byte buffers after bundle termination.
> A dataWriter is created per bundle, and is only used and is safe to use by
> that bundle 's processing thread. Further, GRPC's Send call doesn't maintain
> ownership of the Proto message data after Send returns, allowing this re-use.
> A later optimization could use a sync.Pool to maintain a "freelist" of
> buffers to further reduce per bundle allocations but this would likely only
> be noticeable in streaming contexts. Such a free list should have a cap of
> keeping buffers under some threshold (say slices under 64MB in cap) to avoid
> retaining overly large buffers that aren't in active use. This idea though is
> out of scope for a first pass.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
