ZihanLi58 commented on a change in pull request #3439:
URL: https://github.com/apache/gobblin/pull/3439#discussion_r766931170
##########
File path:
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/ClusterEventMetadataGenerator.java
##########
@@ -43,17 +43,19 @@
List<TaskState> taskStates = jobContext.getJobState().getTaskStates();
String taskException =
EventMetadataUtils.getTaskFailureExceptions(taskStates);
String jobException =
EventMetadataUtils.getJobFailureExceptions(jobContext.getJobState());
-
+ ImmutableMap.Builder<String, String> metadataBuilder =
ImmutableMap.builder();
+ metadataBuilder.put(TimingEvent.FlowEventConstants.HIGH_WATERMARK_FIELD,
jobContext.getJobState().getProp(TimingEvent.FlowEventConstants.HIGH_WATERMARK_FIELD,
""));
Review comment:
Yeah, I realize during this ticket that jobStates event will not be
consumed by status monitor, so we need to add those information in other events
emitted by gobblin cluster as well. Will create a separate PR for this
##########
File path:
gobblin-service/src/main/java/org/apache/gobblin/service/modules/spec/JobExecutionPlan.java
##########
@@ -65,6 +65,7 @@
private final SpecExecutor specExecutor;
private ExecutionStatus executionStatus = ExecutionStatus.PENDING;
private final int maxAttempts;
+ private int currentGeneration = 1;
private int currentAttempts = 0;
Review comment:
1. we only increase the number of current generation when we send
pending resume, and for currentAttempts, we increase it every time we call
orchestrated. So for the first attempt, once we orchestrate it, the current
attempt and current generation will all be 1. Before first job orchestrated,
when the job status is pending, it will be 1 generation and 0 attempts.
2. No, both of the default value is 1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
