[ 
https://issues.apache.org/jira/browse/GOBBLIN-1992?focusedWorklogId=902006&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-902006
 ]

ASF GitHub Bot logged work on GOBBLIN-1992:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 26/Jan/24 18:49
            Start Date: 26/Jan/24 18:49
    Worklog Time Spent: 10m 
      Work Description: phet commented on code in PR #3868:
URL: https://github.com/apache/gobblin/pull/3868#discussion_r1468017947


##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/launcher/ProcessWorkUnitsJobLauncher.java:
##########
@@ -93,4 +97,11 @@ public void submitJob(List<WorkUnit> workunits) {
       throw new RuntimeException(e);
     }
   }
+
+  private void setupMDCFromJobState(JobState jobState) {

Review Comment:
   I expect this will be reused for other job launchers, even if we're merely 
creating them during dev (like the GenWUs I'm about to release).  to support 
that, maybe situate in `Help` or `JobStateUtils`?
   
   and since I believe this MDC stuff all happens transparently behind the 
scenes, maybe less impl jargon in the name.  e.g. 
`propagateGaaSFlowExecutionContext` or `initFlowExecutionContext`?



##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/launcher/ProcessWorkUnitsJobLauncher.java:
##########
@@ -93,4 +97,11 @@ public void submitJob(List<WorkUnit> workunits) {
       throw new RuntimeException(e);
     }
   }
+
+  private void setupMDCFromJobState(JobState jobState) {
+    // TODO: log4j2 has better syntax around conditional logging such that the 
key does not need to be included in the value
+    MDC.put(ConfigurationKeys.FLOW_NAME_KEY, 
String.format("%s:%s",ConfigurationKeys.FLOW_NAME_KEY, 
jobState.getProp(ConfigurationKeys.FLOW_NAME_KEY, "NA")));
+    MDC.put(ConfigurationKeys.FLOW_GROUP_KEY, 
String.format("%s:%s",ConfigurationKeys.FLOW_GROUP_KEY, 
jobState.getProp(ConfigurationKeys.FLOW_GROUP_KEY, "NA")));

Review Comment:
   nit: I don't like `NA`, since it's technically possible that could be a 
valid gaas flow group.  would this token work - `<<NOT SET>>`?



##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/work/assistance/MDCContextPropagator.java:
##########
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.gobblin.temporal.ddm.work.assistance;
+
+import com.google.protobuf.ByteString;
+import io.temporal.api.common.v1.Payload;
+import io.temporal.common.context.ContextPropagator;
+import java.nio.charset.StandardCharsets;
+import java.util.HashMap;
+import java.util.Map;
+import lombok.extern.slf4j.Slf4j;
+import org.slf4j.MDC;
+
+
+/**
+ * Context propagator for MDC in Temporal GaaS flows, which is used to share 
context between workflows and activities.
+ * Keys added to MDC will show up in Temporal headers serialized as protobufs
+ */
+@Slf4j
+public class MDCContextPropagator implements ContextPropagator {
+
+  public String getName() {
+    return this.getClass().getName();
+  }
+
+  public Object getCurrentContext() {

Review Comment:
   nit: I presume most, if not all of these methods implement declarations from 
`ContextPropagator`.  let's make clear which do by adding `@Override` to them



##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/launcher/ProcessWorkUnitsJobLauncher.java:
##########
@@ -93,4 +97,11 @@ public void submitJob(List<WorkUnit> workunits) {
       throw new RuntimeException(e);
     }
   }
+
+  private void setupMDCFromJobState(JobState jobState) {
+    // TODO: log4j2 has better syntax around conditional logging such that the 
key does not need to be included in the value

Review Comment:
   the key may actually assist machine processing.  I'd suggest just changing 
the key names, so they're actually ready to be fed directly back into the GaaS 
REST endpoints, such as:
   ```
   GET 
/flowexecutions/(flowGroup:<groupName>,flowName:<flowName>,flowExecutionId:<flowExecutionId>)
   ```
   hence `flowName`, rather than `flow.name`.  bonus points if these can come 
out comma-separated, and so ready to use w/ that REST API, rather than 
space-separated as you showed in the example output.



##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/work/assistance/MDCContextPropagator.java:
##########
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.gobblin.temporal.ddm.work.assistance;
+
+import com.google.protobuf.ByteString;
+import io.temporal.api.common.v1.Payload;
+import io.temporal.common.context.ContextPropagator;
+import java.nio.charset.StandardCharsets;
+import java.util.HashMap;
+import java.util.Map;
+import lombok.extern.slf4j.Slf4j;
+import org.slf4j.MDC;
+
+
+/**
+ * Context propagator for MDC in Temporal GaaS flows, which is used to share 
context between workflows and activities.
+ * Keys added to MDC will show up in Temporal headers serialized as protobufs

Review Comment:
   not clear what "Temporal headers" are... is this about HTTP requests?  
please explain





Issue Time Tracking
-------------------

    Worklog Id:     (was: 902006)
    Time Spent: 0.5h  (was: 20m)

> Allow per job level logging in Gobblin Temporal workflows
> ---------------------------------------------------------
>
>                 Key: GOBBLIN-1992
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1992
>             Project: Apache Gobblin
>          Issue Type: Improvement
>            Reporter: William Lo
>            Priority: Major
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Currently the Temporal job context does not include the job metadata e.g. 
> flow name, flow group, flow execution ID, which is critical for debugging 
> workflows in the temporal cluster.
> We want to have per job logging so that it is easier to debug multiple 
> Gobblin workflows on Temporal sent through GaaS.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to