[
https://issues.apache.org/jira/browse/BEAM-13082?focusedWorklogId=667945&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-667945
]
ASF GitHub Bot logged work on BEAM-13082:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Oct/21 20:01
Start Date: 20/Oct/21 20:01
Worklog Time Spent: 10m
Work Description: lostluck commented on a change in pull request #15762:
URL: https://github.com/apache/beam/pull/15762#discussion_r733104462
##########
File path: sdks/go/pkg/beam/core/runtime/harness/datamgr.go
##########
@@ -529,7 +531,6 @@ func (w *dataWriter) Flush() error {
if w.buf == nil {
Review comment:
Good call! It's a microoptimization (it's very unlikely for the buffer
to be empty when flushing, but no harm in it. Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 667945)
Time Spent: 50m (was: 40m)
> [Go SDK] Reduce churn in dataWriter by retaining byte slice.
> ------------------------------------------------------------
>
> Key: BEAM-13082
> URL: https://issues.apache.org/jira/browse/BEAM-13082
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: P2
> Time Spent: 50m
> Remaining Estimate: 0h
>
> It's been noted that we can reduce allocations and GC overhead produced by
> the dataWriter if we change the `w.buf = nil` to `w.buf = w.buf[:0]`. We
> should still nil out the buffer after the final flush in Close() however, to
> avoid retaining larger byte buffers after bundle termination.
> A dataWriter is created per bundle, and is only used and is safe to use by
> that bundle 's processing thread. Further, GRPC's Send call doesn't maintain
> ownership of the Proto message data after Send returns, allowing this re-use.
> A later optimization could use a sync.Pool to maintain a "freelist" of
> buffers to further reduce per bundle allocations but this would likely only
> be noticeable in streaming contexts. Such a free list should have a cap of
> keeping buffers under some threshold (say slices under 64MB in cap) to avoid
> retaining overly large buffers that aren't in active use. This idea though is
> out of scope for a first pass.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
