[
https://issues.apache.org/jira/browse/GOBBLIN-1875?focusedWorklogId=876233&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-876233
]
ASF GitHub Bot logged work on GOBBLIN-1875:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 15/Aug/23 00:20
Start Date: 15/Aug/23 00:20
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3737:
URL: https://github.com/apache/gobblin/pull/3737#discussion_r1294078711
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/FlowTriggerHandler.java:
##########
@@ -179,18 +179,31 @@ private void scheduleReminderForEvent(Properties
jobProps, MultiActiveLeaseArbit
String.valueOf(originalEventTimeMillis));
JobKey key = new JobKey(flowAction.getFlowName(),
flowAction.getFlowGroup());
// Create a new trigger for the flow in job scheduler that is set to fire
at the minimum reminder wait time calculated
- Trigger trigger = JobScheduler.createTriggerForJob(key, jobProps);
+ Trigger trigger = JobScheduler.createTriggerForJob(key, jobProps,
+ Optional.of(createSuffixForJobTrigger(status.getEventTimeMillis())));
try {
log.info("Flow Trigger Handler - [{}, eventTimestamp: {}] - attempting
to schedule reminder for event {} in {} millis",
flowAction, originalEventTimeMillis, status.getEventTimeMillis(),
trigger.getNextFireTime());
this.schedulerService.getScheduler().scheduleJob(trigger);
} catch (SchedulerException e) {
- log.warn("Failed to add job reminder due to SchedulerException for job
{} trigger event {} ", key, status.getEventTimeMillis(), e);
+ log.warn("Failed to add job reminder due to SchedulerException for job
{} trigger event {} ", key,
+ status.getEventTimeMillis(), e);
+ // TODO: emit a metric for failed job reminders
}
log.info(String.format("Flow Trigger Handler - [%s, eventTimestamp: %s] -
SCHEDULED REMINDER for event %s in %s millis",
flowAction, originalEventTimeMillis, status.getEventTimeMillis(),
trigger.getNextFireTime()));
}
+ /**
+ * Create suffix to add to end of flow name to differentiate reminder
triggers from the original job schedule trigger
+ * and ensure they are added to the scheduler.
+ * @param eventToRevisitMillis
+ * @return
+ */
+ public static String createSuffixForJobTrigger(long eventToRevisitMillis) {
+ return "reminder_for_" + (eventToRevisitMillis);
Review Comment:
nit: why the parens?
##########
gobblin-runtime/src/main/java/org/apache/gobblin/scheduler/JobScheduler.java:
##########
@@ -583,12 +582,17 @@ public void close() throws IOException {
}
/**
- * Get a {@link org.quartz.Trigger} from the given job configuration
properties.
+ * Get a {@link org.quartz.Trigger} from the given job configuration
properties. If triggerSuffix is provided, appends
+ * it to the end of the flow name.
*/
- public static Trigger createTriggerForJob(JobKey jobKey, Properties
jobProps) {
- // Build a trigger for the job with the given cron-style schedule
+ public static Trigger createTriggerForJob(JobKey jobKey, Properties
jobProps, Optional<String> triggerSuffix) {
+ /*
+ Build a trigger for the job with the given cron-style schedule
+ Append a random integer to job name to be able to add multiple triggers
associated with the same job.
Review Comment:
out-of-date comment
##########
gobblin-runtime/src/main/java/org/apache/gobblin/scheduler/JobScheduler.java:
##########
@@ -583,12 +582,17 @@ public void close() throws IOException {
}
/**
- * Get a {@link org.quartz.Trigger} from the given job configuration
properties.
+ * Get a {@link org.quartz.Trigger} from the given job configuration
properties. If triggerSuffix is provided, appends
+ * it to the end of the flow name.
*/
- public static Trigger createTriggerForJob(JobKey jobKey, Properties
jobProps) {
- // Build a trigger for the job with the given cron-style schedule
+ public static Trigger createTriggerForJob(JobKey jobKey, Properties
jobProps, Optional<String> triggerSuffix) {
+ /*
+ Build a trigger for the job with the given cron-style schedule
+ Append a random integer to job name to be able to add multiple triggers
associated with the same job.
+ */
return TriggerBuilder.newTrigger()
- .withIdentity(jobProps.getProperty(ConfigurationKeys.JOB_NAME_KEY),
+ .withIdentity(jobProps.getProperty(ConfigurationKeys.JOB_NAME_KEY)
+ + (triggerSuffix.isPresent() ? "_" + triggerSuffix.get() : ""),
Review Comment:
```
triggerSuffix.transform(s -> "_" + s).or("")
```
?
Issue Time Tracking
-------------------
Worklog Id: (was: 876233)
Time Spent: 0.5h (was: 20m)
> Create Unique Trigger identifiers
> ----------------------------------
>
> Key: GOBBLIN-1875
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1875
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> We are seeing the following errors when trying to schedule reminders for
> Multi-active scheduler after losing lease acquisition:
> {{Unable to store Trigger with name: '...' and group: '...' because one
> already exists with this identification}}
> The Quartz scheduler allows multiple unique triggers mapping to one job in a
> N:1 relation, but this requires the trigger having a unique identifier
> associated with it. When we create triggers, we were previously using only
> the job name and group to identify the trigger so we're unable to add
> additional triggers for the same job. The trigger key is only used internally
> by the Quartz scheduler, so it is safe to manipulate by adding a random
> number to identify the trigger. We also don't expect more than a handful of
> triggers associated with a job, so an int 0 to 1000 will be sufficient to
> uniquely identify the trigger.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
