[
https://issues.apache.org/jira/browse/GOBBLIN-2136?focusedWorklogId=933455&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-933455
]
ASF GitHub Bot logged work on GOBBLIN-2136:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 05/Sep/24 18:19
Start Date: 05/Sep/24 18:19
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #4031:
URL: https://github.com/apache/gobblin/pull/4031#discussion_r1745965920
##########
gobblin-api/src/main/java/org/apache/gobblin/configuration/ConfigurationKeys.java:
##########
@@ -1071,13 +1071,17 @@ public class ConfigurationKeys {
* Configuration properties related to Flows
*/
public static final String FLOW_RUN_IMMEDIATELY = "flow.runImmediately";
- public static final String GOBBLIN_FLOW_SLA_TIME = "gobblin.flow.sla.time";
- public static final String GOBBLIN_FLOW_SLA_TIME_UNIT =
"gobblin.flow.sla.timeunit";
- public static final String DEFAULT_GOBBLIN_FLOW_SLA_TIME_UNIT =
TimeUnit.MINUTES.name();
- public static final String GOBBLIN_JOB_START_SLA_TIME =
"gobblin.job.start.sla.time";
- public static final String GOBBLIN_JOB_START_SLA_TIME_UNIT =
"gobblin.job.start.sla.timeunit";
- public static final long FALLBACK_GOBBLIN_JOB_START_SLA_TIME = 10L;
- public static final String FALLBACK_GOBBLIN_JOB_START_SLA_TIME_UNIT =
TimeUnit.MINUTES.name();
+ /*
+ The following config names are different from variable name to maintain
backward compatibility.
Review Comment:
nit: can you name this as a "TODO", so anyone searching for string finds it?
##########
gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/DagActionStoreChangeMonitor.java:
##########
@@ -407,20 +288,12 @@ protected static class LaunchSubmissionMetricProxy {
public LaunchSubmissionMetricProxy() {}
- public void markSuccess() {
- getSuccessMeter().mark();
- }
-
Review Comment:
we need to keep both of these `markSuccess` methods. It appears to be an
oversight that `DagManagementDagActionStoreChangeMonitor` is not marking the
success metric upon success, the way L370 (above in
`submitFlowToDagManagerHelper` is).
I believe it belongs here, just before the `break`):
```
case REEVALUATE :
case RESUME:
dagManagement.addDagAction(new
DagActionStore.LeaseParams(dagAction));
break;
default:
```
##########
gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/DagActionStoreChangeMonitor.java:
##########
@@ -252,123 +234,14 @@ protected void
processMessage(DecodeableKafkaRecord<String, DagActionStoreChange
dagActionsSeenCache.put(changeIdentifier, changeIdentifier);
}
- protected void handleDagAction(String operation, DagActionStore.DagAction
dagAction, String flowGroup,
- String flowName, long flowExecutionId, DagActionStore.DagActionType
dagActionType) {
- // We only expect INSERT and DELETE operations done to this table. INSERTs
correspond to any type of
- // {@link DagActionStore.FlowActionType} flow requests that have to be
processed. DELETEs require no action.
- try {
- switch (operation) {
- case "INSERT":
- handleDagAction(dagAction, false);
- this.dagProcEngineMetrics.markDagActionsObserved(dagActionType);
- break;
- case "UPDATE":
- // TODO: change this warning message and process updates if for
launch or reevaluate type
- log.warn("Received an UPDATE action to the DagActionStore when
values in this store are never supposed to be "
- + "updated. Flow group: {} name {} executionId {} were
updated to action {}", flowGroup, flowName,
- flowExecutionId, dagActionType);
- this.unexpectedErrors.mark();
- break;
- case "DELETE":
- log.debug("Deleted dagAction from DagActionStore: {}", dagAction);
- break;
- default:
- log.warn(
- "Received unsupported change type of operation {}. Expected
values to be in [INSERT, UPDATE, DELETE]",
- operation);
- this.unexpectedErrors.mark();
- break;
- }
- } catch (Exception e) {
- log.warn("Ran into unexpected error processing DagActionStore changes:
", e);
- this.unexpectedErrors.mark();
- }
- }
Review Comment:
shouldn't we leave all of this impl as-is for the derived class to continue
using?
that said, I do agree w/ making **the other form** of `handleDagAction`
`abstract`
Issue Time Tracking
-------------------
Worklog Id: (was: 933455)
Time Spent: 5.5h (was: 5h 20m)
> remove obsolete code related to DagManager
> ------------------------------------------
>
> Key: GOBBLIN-2136
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2136
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Arjun Singh Bora
> Priority: Major
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
