[
https://issues.apache.org/jira/browse/BEAM-4725?focusedWorklogId=178831&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-178831
]
ASF GitHub Bot logged work on BEAM-4725:
----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Dec/18 00:58
Start Date: 27/Dec/18 00:58
Worklog Time Spent: 10m
Work Description: lostluck commented on pull request #7357: [BEAM-4725]
Use unsafe to avoid small allocations to the heap.
URL: https://github.com/apache/beam/pull/7357
This avoids tiny allocations on encode and decode of all the ints,varints,
and window headers necessary for Beam Go to function over the FnAPI.
Go's escape analysis is pretty good, except when it comes to indirect
function calls and interfaces. If an argument is passed through to one of
those, the compiler has no choice but to allocate it to the heap, as the
lifetime can't be known without additional analysis.
Using the unsafe package, it's possible to wrap invocations of interfaces
like io.Reader and io.Writer, and encourage the compiler to allocate the small
buffers to the stack instead. This reduces garbage generation, which saves CPU
time in the garbage collector.
Go's tooling permits reading the escape analysis results and justification
for why something was moved to the heap or not. For example, before I changed
the coder/int.go this was the analysis output for the file:
```
~/code/src/github.com/apache/beam/sdks/go/pkg/beam/core/graph/coder
% go build -gcflags '-m -m -l' 2>&1 | grep /int.go
./int.go:27:13: make([]byte, 8) escapes to heap
./int.go:27:13: from ret (assigned) at ./int.go:27:6
./int.go:27:13: from w.Write(ret) (parameter to indirect call) at
./int.go:29:19
./int.go:26:33: leaking param: w
./int.go:26:33: from w.Write(ret) (receiver in indirect call) at
./int.go:29:19
./int.go:34:19: leaking param: r
./int.go:34:19: from r (passed to call[argument escapes]) at
./int.go:35:28
./int.go:44:13: make([]byte, 4) escapes to heap
./int.go:44:13: from ret (assigned) at ./int.go:44:6
./int.go:44:13: from w.Write(ret) (parameter to indirect call) at
./int.go:46:19
./int.go:43:33: leaking param: w
./int.go:43:33: from w.Write(ret) (receiver in indirect call) at
./int.go:46:19
./int.go:51:19: leaking param: r
./int.go:51:19: from r (passed to call[argument escapes]) at
./int.go:52:28
./int.go:60:31: leaking param: w
./int.go:60:31: from w (passed to call[argument escapes]) at
./int.go:61:21
./int.go:65:18: leaking param: r
./int.go:65:18: from r (passed to call[argument escapes]) at
./int.go:66:26
```
Note that all the make []bytes are escaping.
```
~/code/src/github.com/apache/beam/sdks/go/pkg/beam/core/graph/coder
% go build -gcflags '-m -m -l' 2>&1 | grep /int.go
./int.go:26:33: leaking param: w
./int.go:26:33: from w (passed to call[argument escapes]) at
./int.go:29:31
./int.go:28:33: EncodeUint64 data does not escape
./int.go:29:39: EncodeUint64 data does not escape
./int.go:34:19: leaking param: r
./int.go:34:19: from r (passed to call[argument escapes]) at
./int.go:36:34
./int.go:36:42: DecodeUint64 data does not escape
./int.go:39:37: DecodeUint64 data does not escape
./int.go:44:13: make([]byte, 4) escapes to heap
./int.go:44:13: from ret (assigned) at ./int.go:44:6
./int.go:44:13: from w.Write(ret) (parameter to indirect call) at
./int.go:46:19
./int.go:43:33: leaking param: w
./int.go:43:33: from w.Write(ret) (receiver in indirect call) at
./int.go:46:19
./int.go:51:19: leaking param: r
./int.go:51:19: from r (passed to call[argument escapes]) at
./int.go:53:34
./int.go:53:42: DecodeUint32 data does not escape
./int.go:56:37: DecodeUint32 data does not escape
./int.go:60:31: leaking param: w
./int.go:60:31: from w (passed to call[argument escapes]) at
./int.go:61:21
./int.go:65:18: leaking param: r
./int.go:65:18: from r (passed to call[argument escapes]) at
./int.go:66:26
```
Afterwards, they are not escaping.
An alternative of this would be to pass around concrete reader and writers,
so escape analysis would function properly, however, this would tie the code
too in-depth to the implementation.
------------------------
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone
(e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)
[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
</br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
| --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 178831)
Time Spent: 10m
Remaining Estimate: 0h
> Reduce framework overhead in Go SDK
> -----------------------------------
>
> Key: BEAM-4725
> URL: https://issues.apache.org/jira/browse/BEAM-4725
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The Go SDK produces correct results, but it also does redundant
> recomputations, or allocations. This Jira is to track performance
> improvements along any per-element path, where that overhead could be
> significant.
> Contributions should include at least one benchmark with before and after
> values to demonstrate the value of the change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
