phet commented on code in PR #3843:
URL: https://github.com/apache/gobblin/pull/3843#discussion_r1470613326
##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/workflows/metrics/EventSubmitterContext.java:
##########
@@ -28,13 +45,13 @@
* like original Azkaban flow name and gobblin job id
*/
@Getter
-public class TrackingEventMetadata {
+public class EventSubmitterContext {
Review Comment:
javadoc says it's "package private"... although I agree w/ `public`.
also, please resolve `TODO: Fix this description`
##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/ddm/workflow/impl/ProcessWorkUnitsWorkflowImpl.java:
##########
@@ -56,45 +55,42 @@
import org.apache.gobblin.temporal.util.nesting.work.Workload;
import org.apache.gobblin.temporal.util.nesting.workflow.NestingExecWorkflow;
import org.apache.gobblin.temporal.workflows.metrics.EventTimer;
-import org.apache.gobblin.temporal.workflows.metrics.SubmitGTEActivity;
import org.apache.gobblin.temporal.workflows.metrics.TemporalEventTimer;
-import org.apache.gobblin.temporal.workflows.metrics.TrackingEventMetadata;
+import org.apache.gobblin.temporal.workflows.metrics.EventSubmitterContext;
@Slf4j
public class ProcessWorkUnitsWorkflowImpl implements ProcessWorkUnitsWorkflow {
public static final String CHILD_WORKFLOW_ID_BASE = "NestingExecWorkUnits";
public static final String COMMIT_STEP_WORKFLOW_ID_BASE =
"CommitStepWorkflow";
- private final ActivityOptions options = ActivityOptions.newBuilder()
- .setStartToCloseTimeout(Duration.ofSeconds(60))
- .build();
- private final SubmitGTEActivity submitGTEActivity = Workflow.newActivityStub(
- SubmitGTEActivity.class, options);
-
- private static final int MAX_DESERIALIZATION_FS_LOAD_ATTEMPTS = 5;
@Override
public int process(WUProcessingSpec workSpec) {
- // NOTE: We are using the metrics tags from Job Props to create the metric
context for the timer and NOT
- // the deserialized jobState from HDFS that is created by the real distcp
job. This is because the AZ runtime
- // settings we want are for the job launcher that launched this Yarn job.
-
try {
- FileSystem fs = Help.loadFileSystemForce(workSpec);
- JobState jobState = Help.loadJobStateUncached(workSpec, fs);
- List<Tag<?>> tagsFromCurrentJob = workSpec.getTags();
- String metricsSuffix = workSpec.getMetricsSuffix();
- List<Tag<?>> tags = getTags(tagsFromCurrentJob, metricsSuffix, jobState);
-
- TemporalEventTimer.Factory timerFactory = new
TemporalEventTimer.Factory(submitGTEActivity, new TrackingEventMetadata(tags,
JobMetrics.NAMESPACE));
- try (EventTimer timer = timerFactory.getJobTimer()) {
+ EventSubmitterContext eventSubmitterContext =
getEventSubmitterContext(workSpec);
+ TemporalEventTimer.Factory timerFactory = new
TemporalEventTimer.Factory(eventSubmitterContext);
+ try (EventTimer timer = timerFactory.createJobTimer()) {
Review Comment:
the type could be more specific as `TemporalEventTimer`, which is probably
what we want if we're to `addMetadata`
##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/workflows/metrics/TemporalEventTimer.java:
##########
@@ -40,61 +41,73 @@
public class TemporalEventTimer implements EventTimer {
private final SubmitGTEActivity trackingEventActivity;
private final GobblinEventBuilder eventBuilder;
- private final TrackingEventMetadata trackingEventMetadata;
+ private final EventSubmitterContext _eventSubmitterContext;
private final Instant startTime;
@Override
public void stop() {
stop(getCurrentTime());
}
+ /**
+ * Add additional metadata that will be submitted via the {@link
EventSubmitter} when the timer is stopped
+ * via {@link #stop()}
+ * @param key
+ * @param metadata
+ */
+ public void addMetadata(String key, String metadata) {
+ this.eventBuilder.addMetadata(key, metadata);
+ }
+
+
private void stop(Instant endTime) {
this.eventBuilder.addMetadata(EventSubmitter.EVENT_TYPE,
TimingEvent.METADATA_TIMING_EVENT);
this.eventBuilder.addMetadata(TimingEvent.METADATA_START_TIME,
Long.toString(this.startTime.toEpochMilli()));
this.eventBuilder.addMetadata(TimingEvent.METADATA_END_TIME,
Long.toString(endTime.toEpochMilli()));
Duration duration = Duration.between(this.startTime, endTime);
this.eventBuilder.addMetadata(TimingEvent.METADATA_DURATION,
Long.toString(duration.toMillis()));
- trackingEventActivity.submitGTE(this.eventBuilder, trackingEventMetadata);
+ trackingEventActivity.submitGTE(this.eventBuilder, _eventSubmitterContext);
}
private static Instant getCurrentTime() {
return Instant.ofEpochMilli(Workflow.currentTimeMillis());
}
public static class Factory {
+ private static final ActivityOptions DEFAULT_OPTS =
ActivityOptions.newBuilder().build();
private final SubmitGTEActivity submitGTEActivity;
- private final TrackingEventMetadata trackingEventMetadata;
+ private final EventSubmitterContext _eventSubmitterContext;
Review Comment:
same here with the `_`
##########
gobblin-temporal/src/main/java/org/apache/gobblin/temporal/workflows/metrics/TemporalEventTimer.java:
##########
@@ -40,61 +41,73 @@
public class TemporalEventTimer implements EventTimer {
private final SubmitGTEActivity trackingEventActivity;
private final GobblinEventBuilder eventBuilder;
- private final TrackingEventMetadata trackingEventMetadata;
+ private final EventSubmitterContext _eventSubmitterContext;
Review Comment:
nit: none of the other members have `_` prefix... why this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]