[
https://issues.apache.org/jira/browse/GOBBLIN-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Urmi Mustafi updated GOBBLIN-1881:
----------------------------------
Description:
The following error is thrown when attempting to schedule reminder events.
{{The job ... referenced by the trigger does not exist.","stackTrace":[
{"index":0,"call":"storeTrigger","columnNumber":null,"fileName":"RAMJobStore.java","lineNumber":422,"nativeMethod":"0","source":"org.quartz.simpl.RAMJobStore"}
,
{"index":1,"call":"scheduleJob","columnNumber":null,"fileName":"QuartzScheduler.java","lineNumber":932,"nativeMethod":"0","source":"org.quartz.core.QuartzScheduler"}
,
{"index":2,"call":"scheduleJob","columnNumber":null,"fileName":"StdScheduler.java","lineNumber":258,"nativeMethod":"0","source":"org.quartz.impl.StdScheduler"}
,
{"index":3,"call":"scheduleReminderForEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":187,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"}
,
{"index":4,"call":"handleTriggerEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":124,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"}
,
{"index":5,"call":"orchestrate","columnNumber":null,"fileName":"Orchestrator.java","lineNumber":267,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.Orchestrator"}
,\{"index":6,"call":"runJob","columnNumber":null,"fileName":"GobblinServiceJobScheduler.java","lineNumber":47}}
The issue above from FlowTriggerHandler reveals two problems
* The job referenced is not found by scheduler -> The job key is constructed
with flow name/group instead of job name/group, which is probably causing this
issue. We add a check for if this happens by searching for the jobKey within
the scheduler and emit a metric in case we encounter it in the future.
* Upon looking closely at the code, we also need to update the job props of
the job in the scheduler to keep track of the event to set reminder for. The
job props we set
[here|https://github.com/apache/gobblin/blob/69d7e0fa4df2499934462854514ddc9ddbfe7dc7/gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/FlowTriggerHandler.java#L176]
are never used in
[createTriggerForJob|https://github.com/apache/gobblin/blob/69d7e0fa4df2499934462854514ddc9ddbfe7dc7/gobblin-runtime/src/main/java/org/apache/gobblin/scheduler/JobScheduler.java#L588]
We want to use {{StdScheduler.scheduleJob(JobDetail jobDetail, Set<? extends
Trigger> triggersForJob, boolean replace)}} to replace the {{jobDetail}} we
construct when making a reminder event.
*
was:
The following error is thrown when attempting to schedule reminder events.
{{The job ... referenced by the trigger does not
exist.","stackTrace":[\\{"index":0,"call":"storeTrigger","columnNumber":null,"fileName":"RAMJobStore.java","lineNumber":422,"nativeMethod":"0","source":"org.quartz.simpl.RAMJobStore"},\\{"index":1,"call":"scheduleJob","columnNumber":null,"fileName":"QuartzScheduler.java","lineNumber":932,"nativeMethod":"0","source":"org.quartz.core.QuartzScheduler"},\\{"index":2,"call":"scheduleJob","columnNumber":null,"fileName":"StdScheduler.java","lineNumber":258,"nativeMethod":"0","source":"org.quartz.impl.StdScheduler"},\\{"index":3,"call":"scheduleReminderForEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":187,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"},\\{"index":4,"call":"handleTriggerEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":124,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"},\\{"index":5,"call":"orchestrate","columnNumber":null,"fileName":"Orchestrator.java","lineNumber":267,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.Orchestrator"},\{"index":6,"call":"runJob","columnNumber":null,"fileName":"GobblinServiceJobScheduler.java","lineNumber":47}}
The issue above from FlowTriggerHandler reveals two problems
* The job referenced is not found by scheduler
** We add a check for if this happens by searching for the jobKey within the
scheduler and emit a metric
** The job key is constructed with flow name/group instead of job name/group,
this could be an issue for multi-hop jobs but in general should not be.
* Upon looking closely at the code, we also need to update the job props of
the job in the scheduler to keep track of the event to set reminder for. The
job props we set
[here|https://jarvis.corp.linkedin.com/codesearch/result/?name=FlowTriggerHandler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Forchestration&reponame=linkedin%2Fgobblin-elr#176]
are never used in
[createTriggerForJob|https://jarvis.corp.linkedin.com/codesearch/result/?name=JobScheduler.java&path=gobblin-elr%2Fgobblin-runtime%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fscheduler&reponame=linkedin%2Fgobblin-elr#588]
We want to use {{StdScheduler.scheduleJob(JobDetail jobDetail, Set<? extends
Trigger> triggersForJob, boolean replace)}} to replace the jobDetail we
construct when making a reminder event.
> Fix Setting Reminder Scheduler Event
> ------------------------------------
>
> Key: GOBBLIN-1881
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1881
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: Urmi Mustafi
> Priority: Major
>
> The following error is thrown when attempting to schedule reminder events.
> {{The job ... referenced by the trigger does not exist.","stackTrace":[
> {"index":0,"call":"storeTrigger","columnNumber":null,"fileName":"RAMJobStore.java","lineNumber":422,"nativeMethod":"0","source":"org.quartz.simpl.RAMJobStore"}
> ,
> {"index":1,"call":"scheduleJob","columnNumber":null,"fileName":"QuartzScheduler.java","lineNumber":932,"nativeMethod":"0","source":"org.quartz.core.QuartzScheduler"}
> ,
> {"index":2,"call":"scheduleJob","columnNumber":null,"fileName":"StdScheduler.java","lineNumber":258,"nativeMethod":"0","source":"org.quartz.impl.StdScheduler"}
> ,
> {"index":3,"call":"scheduleReminderForEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":187,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"}
> ,
> {"index":4,"call":"handleTriggerEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":124,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"}
> ,
> {"index":5,"call":"orchestrate","columnNumber":null,"fileName":"Orchestrator.java","lineNumber":267,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.Orchestrator"}
> ,\{"index":6,"call":"runJob","columnNumber":null,"fileName":"GobblinServiceJobScheduler.java","lineNumber":47}}
>
> The issue above from FlowTriggerHandler reveals two problems
> * The job referenced is not found by scheduler -> The job key is constructed
> with flow name/group instead of job name/group, which is probably causing
> this issue. We add a check for if this happens by searching for the jobKey
> within the scheduler and emit a metric in case we encounter it in the future.
> * Upon looking closely at the code, we also need to update the job props of
> the job in the scheduler to keep track of the event to set reminder for. The
> job props we set
> [here|https://github.com/apache/gobblin/blob/69d7e0fa4df2499934462854514ddc9ddbfe7dc7/gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/FlowTriggerHandler.java#L176]
> are never used in
> [createTriggerForJob|https://github.com/apache/gobblin/blob/69d7e0fa4df2499934462854514ddc9ddbfe7dc7/gobblin-runtime/src/main/java/org/apache/gobblin/scheduler/JobScheduler.java#L588]
> We want to use {{StdScheduler.scheduleJob(JobDetail jobDetail, Set<? extends
> Trigger> triggersForJob, boolean replace)}} to replace the {{jobDetail}} we
> construct when making a reminder event.
> *
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
