[
https://issues.apache.org/jira/browse/BEAM-3612?focusedWorklogId=160798&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-160798
]
ASF GitHub Bot logged work on BEAM-3612:
----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Oct/18 19:32
Start Date: 30/Oct/18 19:32
Worklog Time Spent: 10m
Work Description: lostluck opened a new pull request #6893: [BEAM-3612]
Add a benchmark for method invocation methods.
URL: https://github.com/apache/beam/pull/6893
With the Go SDK's necessary default use of reflection to achieve more
idiomatic user code, it's up to the framework to reduce this overhead as much
as possible. This benchmark illustrates the various methods available to the Go
SDK and informs work I've been doing on [BEAM-3612].
Having properly generated type assertion shims at present represents a
reduction of around ~400ns per function invocation. As mentioned in the JIRA,
I've got a tool I've been working on to make generating these shims easier.
Further, this reveals there's a way to change how we handle *method*
invocation which is relevant for Structural DoFns. The way we use a
reflect.Value.Interface() call to get to back to an indirect interface{}
version of the method, we typically go through the "Implicit Receiver" route,
so that the receiver we're operating on (the Structural DoFn with the
associated state) is implicitely cached and instated by the reflection code
when we call it. This adds ~180ns overhead (on my machine) compared to the the
Explicit Receiver version.
The tricky part here is that moving to Explicit Receivers requires a minor
overhaul through the beam function handling code, to properly mark the receiver
parameter for the type checking phase. This is better accomplished after we
have *tools* to re-generate type assertion shims for the SDK, if we're doing it
at all.
Finally, by having arity Explicit invocation of the shims reduces overhead
by another 30ns. This is probably something that can be generated one off (with
the above concession to the Explicit Receiver approach) to get a base set of
combinatorial values. However, the extra overhead of a arity specialized
invoker might be more than the 30ns cost of allocating & extracting the values
from the slice, so this should be more specifically benchmarked in context of
invoking for ParDos.
------------------------
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone
(e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
</br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
| --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 160798)
Time Spent: 2h 10m (was: 2h)
> Make it easy to generate type-specialized Go SDK reflectx.Funcs
> ---------------------------------------------------------------
>
> Key: BEAM-3612
> URL: https://issues.apache.org/jira/browse/BEAM-3612
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Henning Rohde
> Assignee: Robert Burke
> Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
