[
https://issues.apache.org/jira/browse/BEAM-4424?focusedWorklogId=664457&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-664457
]
ASF GitHub Bot logged work on BEAM-4424:
----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Oct/21 03:53
Start Date: 13/Oct/21 03:53
Worklog Time Spent: 10m
Work Description: youngoli commented on a change in pull request #15713:
URL: https://github.com/apache/beam/pull/15713#discussion_r727678118
##########
File path: sdks/go/pkg/beam/core/runtime/harness/harness.go
##########
@@ -47,8 +47,10 @@ const cacheSize = 20
func Main(ctx context.Context, loggingEndpoint, controlEndpoint string) error {
hooks.DeserializeHooksFromOptions(ctx)
+ // Pass in the logging endpoint for use w/the default remote logging
hook.
+ ctx = context.WithValue(ctx, loggingEndpointCtxKey, loggingEndpoint)
Review comment:
Just a question: If a runner disables the default logging endpoint, then
this does nothing, right? Just want to make sure I'm not missing anything.
##########
File path: sdks/go/pkg/beam/core/runtime/harness/logging.go
##########
@@ -95,6 +96,28 @@ func convertSeverity(sev log.Severity)
fnpb.LogEntry_Severity_Enum {
}
}
+type remoteLoggingKey string
+
+// DefaultRemoteLoggingHook is the key used for the default remote logging
hook.
+// If a runner wants to use an alternative logging solution, it can be
+// disabled with hooks.DisableHook(harness.DefaultRemoteLoggingHook).
+const DefaultRemoteLoggingHook = "default_remote_logging"
+
+var loggingEndpointCtxKey = remoteLoggingKey(DefaultRemoteLoggingHook)
+
+func init() {
Review comment:
So I'm trying to understand how hooks work correctly, and how this
init() function interacts with hooks.
So this remote logging hook is meant to send SDK harness logs back to the
runner (in this comment called FnHarness), right? So looking at its lifecycle:
1. It gets registered and enabled in this init function in the user's
pipeline code (i.e. before beam.Init)
2. It gets serialized in options and deserialized in the SDK harness.
3. Init gets called on the hook, which connects the SDK harness to the
remote logging endpoint (i.e the runner).
Another thing I want to double check: Doesn't this init() function, and by
extension it's registering and enabling the hook, also get called in the SDK
harness? I'm guessing DeserializeHooksFromOptions replaces any registered hooks
when it gets called in harness.go?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 664457)
Time Spent: 40m (was: 0.5h)
> Improvements to hooks module
> ----------------------------
>
> Key: BEAM-4424
> URL: https://issues.apache.org/jira/browse/BEAM-4424
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Affects Versions: Not applicable
> Reporter: Bill Neubauer
> Assignee: Robert Burke
> Priority: P3
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Proposed improvements to the Go hooks API:
> Execution order of the hooks should be based on the order in which the hook
> is enabled. This gives the runner precise control over ordering so
> dependencies on hooked behavior can be well-managed.
> Provide an API to disable a hook. Disabling a hook removes it from the
> ordered list. If the same hook is later re-Enabled, it has lost its previous
> ordering and would be placed at the end of the ordered list.
> The invocation of setupRemoteLogging() in harness.Main() will be replaced by
> a hook. This new hook will be called by the default translate code. The net
> effect is the default behavior for runners remains unchanged. If a runner
> wants a different logging behavior, it can disable the default logging hook
> and enable its own hook.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
