[
https://issues.apache.org/jira/browse/BEAM-8018?focusedWorklogId=298260&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-298260
]
ASF GitHub Bot logged work on BEAM-8018:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Aug/19 22:23
Start Date: 20/Aug/19 22:23
Worklog Time Spent: 10m
Work Description: lostluck commented on pull request #9387: [BEAM-8018]
Detect unexported fields earlier
URL: https://github.com/apache/beam/pull/9387
Values of unregistered types in the Beam Go SDK are synthetically
constructed with the reflect package. However, it's not possible to
synthetically construct types with unexported fields with reflect.StructOf.
This can be worked around by registering the given type, which avoids the
expensive synthetic construction and this failure mode.
The failure manifests at pipeline execution time which is worse.
This PR moves that failure to pipeline construction time, where it presently
manifests as a panic (until BEAM-8017 is resolved) with instructions how to
register the type.
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] [**Choose
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and
mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)
| --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/)
| --- | --- | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/)
| --- | ---
See
[.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md)
for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 298260)
Remaining Estimate: 0h
Time Spent: 10m
> Detect unexported fields in unregistered types for better error messages
> ------------------------------------------------------------------------
>
> Key: BEAM-8018
> URL: https://issues.apache.org/jira/browse/BEAM-8018
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Robert Burke
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Unregistered types with unexported fields cause pipeline execution time
> panics rather than construction time panics/errors, which would be preferable.
> Improved error messages encouraging users to register their types already
> exists, but isn't currently triggered in this instance, so it's a matter of
> returning an error in this case.
> eg. for using time.Time as a value in a DoFn...
> panic:
> <Coroner crash reporting disabled due to unit tests.>
> reflect.StructOf: StructOf does not allow unexported fields [recovered]
> panic: reflect.StructOf: StructOf does not allow unexported fields
> goroutine 195 [running]:
> panic(0x7e0a060, 0x84beb70)
> third_party/go/gc/src/runtime/panic.go:567 +0x2da fp=0xc000dc1178
> sp=0xc000dc10c0 pc=0xee24daa
> testing.tRunner.func1(0xc000d6ec00)
> third_party/go/gc/src/testing/testing.go:830 +0x388 fp=0xc000dc11f8
> sp=0xc000dc1178 pc=0xfafda58
> runtime.call32(0x0, 0x833e1e0, 0xc000caaab0, 0x800000008)
> third_party/go/gc/src/runtime/asm_amd64.s:519 +0x3b fp=0xc000dc1228
> sp=0xc000dc11f8 pc=0xee53acb
> panic(0x7e0a060, 0x84beb70)
> third_party/go/gc/src/runtime/panic.go:522 +0x1b5 fp=0xc000dc12e0
> sp=0xc000dc1228 pc=0xee24c85
> reflect.runtimeStructField(0xc000e267e0, 0x4, 0xc000e267e4, 0x4, 0x8541880,
> 0x7e0a060, 0x0, 0x0, 0x0, 0xc000e26990, ...)
> third_party/go/gc/src/reflect/type.go:2765 +0x1c2 fp=0xc000dc1348
> sp=0xc000dc12e0 pc=0xee80ea2
> reflect.StructOf(0xc0006dd040, 0x3, 0x4, 0x0, 0x0)
> third_party/go/gc/src/reflect/type.go:2371 +0x21f6 fp=0xc000dc1b28
> sp=0xc000dc1348 pc=0xee7f816
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48800|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48800],
> 0xc000dc1e68, 0xee3c4f2, 0xc000e26920, 0x0)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/serialize.go:558
> +0xa6c fp=0xc000dc1df8 sp=0xc000dc1b28 pc=0xf53a7ac
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48780|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48780],
> 0x0, 0x0, 0x0, 0x1)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/serialize.go:533
> +0x471 fp=0xc000dc20c8 sp=0xc000dc1df8 pc=0xf53a1b1
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48680|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48680],
> 0xc000369bf0, 0xc000b24a40, 0x3e, 0xc000961728)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/serialize.go:542
> +0x6c5 fp=0xc000dc2398 sp=0xc000dc20c8 pc=0xf53a405
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48600|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48600],
> 0xc0001af340, 0x1, 0x1, 0x2)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/serialize.go:583
> +0x12fc fp=0xc000dc2668 sp=0xc000dc2398 pc=0xf53b03c
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48480|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48480],
> 0xc0001af2d0, 0x1, 0x1, 0x2)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/serialize.go:542
> +0x6c5 fp=0xc000dc2938 sp=0xc000dc2668 pc=0xf53a405
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48280|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48280],
> 0x20, 0xc000e265c0, 0x0, 0x1)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/serialize.go:542
> +0x6c5 fp=0xc000dc2c08 sp=0xc000dc2938 pc=0xf53a405
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48200|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48200],
> 0x0, 0x0, 0x0, 0x1)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/serialize.go:533
> +0x471 fp=0xc000dc2ed8 sp=0xc000dc2c08 pc=0xf53a1b1
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48080|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeType(0xc000c48080],
> 0x0, 0xc000dc3230, 0xef9eb65, 0xc000dc3208)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/serialize.go:542
> +0x6c5 fp=0xc000dc31a8 sp=0xc000dc2ed8 pc=0xf53a405
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeCustomCoder(0xc000cc2750|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.decodeCustomCoder(0xc000cc2750],
> 0x698, 0x8512f20, 0xc000cc2750)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/serialize.go:146
> +0x44 fp=0xc000dc32b0 sp=0xc000dc31a8 pc=0xf535e14
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.(*CoderUnmarshaller).makeCoder(0xc0003250e0|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.(*CoderUnmarshaller).makeCoder(0xc0003250e0],
> 0xc000985500, 0xc000ec7c18, 0x2, 0xc000dcdb10)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/coder.go:227
> +0x1bab fp=0xc000dc35a0 sp=0xc000dc32b0 pc=0xf52eefb
> [google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.(*CoderUnmarshaller).Coder(0xc0003250e0|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/graphx.(*CoderUnmarshaller).Coder(0xc0003250e0],
> 0xc000ec7c18, 0x2, 0xc0000511c8, 0xc000dc3748, 0xc000051da0)
>
> third_party/golang/apache_beam/pkg/beam/core/runtime/graphx/coder.go:111
> +0xe0 fp=0xc000dc3650 sp=0xc000dc35a0 pc=0xf52ccc0
> [google3/third_party/golang/apache_beam/pkg/beam/runners/dataflow/dataflowlib/dataflowlib.(*translator).translateCoder(0xc000325100|https://cs.corp.google.com/piper///depot/google3/third_party/golang/apache_beam/pkg/beam/runners/dataflow/dataflowlib/dataflowlib.(*translator).translateCoder(0xc000325100],
> 0xc000eec660, 0xc000ec7c18, 0x2, 0xc000051e48)
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
