[
https://issues.apache.org/jira/browse/BEAM-4034?focusedWorklogId=89181&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89181
]
ASF GitHub Bot logged work on BEAM-4034:
----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Apr/18 23:15
Start Date: 09/Apr/18 23:15
Worklog Time Spent: 10m
Work Description: tweise closed pull request #5065: [BEAM-4034] Fix hooks
panic with Go on Flink
URL: https://github.com/apache/beam/pull/5065
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/go/pkg/beam/core/runtime/harness/harness.go
b/sdks/go/pkg/beam/core/runtime/harness/harness.go
index f8efffece71..5cdf5053cc8 100644
--- a/sdks/go/pkg/beam/core/runtime/harness/harness.go
+++ b/sdks/go/pkg/beam/core/runtime/harness/harness.go
@@ -38,7 +38,7 @@ import (
// "pipeline-construction time" -- on each worker. It is a FnAPI client and
// ultimately responsible for correctly executing user code.
func Main(ctx context.Context, loggingEndpoint, controlEndpoint string) error {
- hooks.DeserializeHooksFromOptions()
+ hooks.DeserializeHooksFromOptions(ctx)
hooks.RunInitHooks(ctx)
setupRemoteLogging(ctx, loggingEndpoint)
diff --git a/sdks/go/pkg/beam/core/util/hooks/hooks.go
b/sdks/go/pkg/beam/core/util/hooks/hooks.go
index ea30cc6e50f..cacb9ee5e02 100644
--- a/sdks/go/pkg/beam/core/util/hooks/hooks.go
+++ b/sdks/go/pkg/beam/core/util/hooks/hooks.go
@@ -128,8 +128,12 @@ func SerializeHooksToOptions() {
// DeserializeHooksFromOptions extracts the hook configuration information
from the options and configures
// the hooks with the supplied options.
-func DeserializeHooksFromOptions() {
+func DeserializeHooksFromOptions(ctx context.Context) {
cfg := runtime.GlobalOptions.Get("hooks")
+ if cfg == "" {
+ log.Warn(ctx, "SerializeHooksToOptions was never called. No
hooks enabled")
+ return
+ }
if err := json.Unmarshal([]byte(cfg), &enabledHooks); err != nil {
// Shouldn't happen, since all the data is strings.
panic(fmt.Sprintf("DeserializeHooks failed on input %q: %v",
cfg, err))
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 89181)
Time Spent: 50m (was: 40m)
> Hooks panic on flink
> --------------------
>
> Key: BEAM-4034
> URL: https://issues.apache.org/jira/browse/BEAM-4034
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Reporter: Henning Rohde
> Assignee: Henning Rohde
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> It seems the serialized hooks are set only for Dataflow. It should probably
> not panic if nothing is set. I see the below on Flink.
> $ docker logs 0f5
> 2018/04/09 21:03:48 Initializing Go harness: /opt/apache/beam/boot --id=-123
> --logging_endpoint=docker.for.mac.host.internal:61723
> --artifact_endpoint=docker.for.mac.host.internal:61724
> --provision_endpoint=docker.for.mac.host.internal:61725
> --control_endpoint=docker.for.mac.host.internal:61721
> --semi_persist_dir=/tmp/semi_persistent_dir2198305397089608348
> Worker panic: DeserializeHooks failed on input "": unexpected end of JSON
> inputgoroutine 1 [running]:
> runtime/debug.Stack(0x4f, 0x0, 0x0)
> /usr/local/go/src/runtime/debug/stack.go:24 +0xa7
> runtime/debug.PrintStack()
> /usr/local/go/src/runtime/debug/stack.go:16 +0x22
> github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/init.hook.func1()
>
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/init/init.go:76
> +0xb6
> panic(0xc65060, 0xc4200402a0)
> /usr/local/go/src/runtime/panic.go:491 +0x283
> github.com/apache/beam/sdks/go/pkg/beam/core/util/hooks.DeserializeHooksFromOptions()
>
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/pkg/beam/core/util/hooks/hooks.go:135
> +0x402
> github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness.Main(0x136dd00,
> 0xc420060010, 0x7ffdcfb0dda8, 0x22, 0x7ffdcfb0ddde, 0x22, 0x0, 0x0)
>
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/harness.go:41
> +0x4f
> github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/init.hook()
>
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/harness/init/init.go:84
> +0xc1
> github.com/apache/beam/sdks/go/pkg/beam/core/runtime.Init()
>
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/pkg/beam/core/runtime/init.go:42
> +0x4c
> github.com/apache/beam/sdks/go/pkg/beam.Init()
>
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/pkg/beam/forward.go:63
> +0x20
> main.main()
>
> /Users/herohde/go/src/github.com/apache/beam/sdks/go/examples/wordcount/wordcount.go:156
> +0x39
> 2018/04/09 21:03:49 User program exited: exit status 2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
