[
https://issues.apache.org/jira/browse/GOBBLIN-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Urmi Mustafi updated GOBBLIN-1881:
----------------------------------
Description:
The following error is thrown when attempting to schedule reminder events.
{{The job ... referenced by the trigger does not
exist.","stackTrace":[\\{"index":0,"call":"storeTrigger","columnNumber":null,"fileName":"RAMJobStore.java","lineNumber":422,"nativeMethod":"0","source":"org.quartz.simpl.RAMJobStore"},\\{"index":1,"call":"scheduleJob","columnNumber":null,"fileName":"QuartzScheduler.java","lineNumber":932,"nativeMethod":"0","source":"org.quartz.core.QuartzScheduler"},\\{"index":2,"call":"scheduleJob","columnNumber":null,"fileName":"StdScheduler.java","lineNumber":258,"nativeMethod":"0","source":"org.quartz.impl.StdScheduler"},\\{"index":3,"call":"scheduleReminderForEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":187,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"},\\{"index":4,"call":"handleTriggerEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":124,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"},\\{"index":5,"call":"orchestrate","columnNumber":null,"fileName":"Orchestrator.java","lineNumber":267,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.Orchestrator"},\{"index":6,"call":"runJob","columnNumber":null,"fileName":"GobblinServiceJobScheduler.java","lineNumber":47}}
The issue above from FlowTriggerHandler reveals two problems
* The job referenced is not found by scheduler
** We add a check for if this happens by searching for the jobKey within the
scheduler and emit a metric
** The job key is constructed with flow name/group instead of job name/group,
this could be an issue for multi-hop jobs but in general should not be.
* Upon looking closely at the code, we also need to update the job props of
the job in the scheduler to keep track of the event to set reminder for. The
job props we set
[here|https://jarvis.corp.linkedin.com/codesearch/result/?name=FlowTriggerHandler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Forchestration&reponame=linkedin%2Fgobblin-elr#176]
are never used in
[createTriggerForJob|https://jarvis.corp.linkedin.com/codesearch/result/?name=JobScheduler.java&path=gobblin-elr%2Fgobblin-runtime%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fscheduler&reponame=linkedin%2Fgobblin-elr#588]
We want to use {{StdScheduler.scheduleJob(JobDetail jobDetail, Set<? extends
Trigger> triggersForJob, boolean replace)}} to replace the jobDetail we
construct when making a reminder event.
was:
{{The job (StressTest.gobblin-test-1638398777124) referenced by the trigger
does not
exist.","stackTrace":[\{"index":0,"call":"storeTrigger","columnNumber":null,"fileName":"RAMJobStore.java","lineNumber":422,"nativeMethod":"0","source":"org.quartz.simpl.RAMJobStore"},\{"index":1,"call":"scheduleJob","columnNumber":null,"fileName":"QuartzScheduler.java","lineNumber":932,"nativeMethod":"0","source":"org.quartz.core.QuartzScheduler"},\{"index":2,"call":"scheduleJob","columnNumber":null,"fileName":"StdScheduler.java","lineNumber":258,"nativeMethod":"0","source":"org.quartz.impl.StdScheduler"},\{"index":3,"call":"scheduleReminderForEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":187,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"},\{"index":4,"call":"handleTriggerEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":124,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"},\{"index":5,"call":"orchestrate","columnNumber":null,"fileName":"Orchestrator.java","lineNumber":267,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.Orchestrator"},\{"index":6,"call":"runJob","columnNumber":null,"fileName":"GobblinServiceJobScheduler.java","lineNumber":476,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.scheduler.GobblinServiceJobScheduler"},\{"index":7,"call":"executeImpl","columnNumber":null,"fileName":"GobblinServiceJobScheduler.java","lineNumber":725,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.scheduler.GobblinServiceJobScheduler$GobblinServiceJob"},\{"index":8,"call":"execute","columnNumber":null,"fileName":"BaseGobblinJob.java","lineNumber":58,"nativeMethod":"0","source":"org.apache.gobblin.scheduler.BaseGobblinJob"},\{"index":9,"call":"run","columnNumber":null,"fileName":"JobRunShell.java","lineNumber":202,"nativeMethod":"0","source":"org.quartz.core.JobRunShell"},\{"index":10,"call":"run","columnNumber":null,"fileName":"SimpleThreadPool.java","lineNumber":573,"nativeMethod":"0","source":"org.quartz.simpl.SimpleThreadPool$WorkerThread"}],"type":"org.quartz.JobPersistenceException}}
The issue above from FlowTriggerHandler reveals two problems
* the job referenced is not found by scheduler
** let's check why/when this is the case by searching for the jobKey within
the scheduler
** the job key is constructed with flow name/group instead of job name/group,
is that the issue?
* We also need to update the job props of the job in the scheduler to keep
track of the event to set reminder for. The job props we set
[here|https://jarvis.corp.linkedin.com/codesearch/result/?name=FlowTriggerHandler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Forchestration&reponame=linkedin%2Fgobblin-elr#176]
are never used in
[createTriggerForJob|https://jarvis.corp.linkedin.com/codesearch/result/?name=JobScheduler.java&path=gobblin-elr%2Fgobblin-runtime%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fscheduler&reponame=linkedin%2Fgobblin-elr#588]
We want to use {{StdScheduler.scheduleJob(JobDetail jobDetail, Set<? extends
Trigger> triggersForJob, boolean replace)}} to replace the jobDetail we
construct when making a reminder event.
*
> Fix Setting Reminder Scheduler Event
> ------------------------------------
>
> Key: GOBBLIN-1881
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1881
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: Urmi Mustafi
> Priority: Major
>
> The following error is thrown when attempting to schedule reminder events.
> {{The job ... referenced by the trigger does not
> exist.","stackTrace":[\\{"index":0,"call":"storeTrigger","columnNumber":null,"fileName":"RAMJobStore.java","lineNumber":422,"nativeMethod":"0","source":"org.quartz.simpl.RAMJobStore"},\\{"index":1,"call":"scheduleJob","columnNumber":null,"fileName":"QuartzScheduler.java","lineNumber":932,"nativeMethod":"0","source":"org.quartz.core.QuartzScheduler"},\\{"index":2,"call":"scheduleJob","columnNumber":null,"fileName":"StdScheduler.java","lineNumber":258,"nativeMethod":"0","source":"org.quartz.impl.StdScheduler"},\\{"index":3,"call":"scheduleReminderForEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":187,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"},\\{"index":4,"call":"handleTriggerEvent","columnNumber":null,"fileName":"FlowTriggerHandler.java","lineNumber":124,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.FlowTriggerHandler"},\\{"index":5,"call":"orchestrate","columnNumber":null,"fileName":"Orchestrator.java","lineNumber":267,"nativeMethod":"0","source":"org.apache.gobblin.service.modules.orchestration.Orchestrator"},\{"index":6,"call":"runJob","columnNumber":null,"fileName":"GobblinServiceJobScheduler.java","lineNumber":47}}
>
> The issue above from FlowTriggerHandler reveals two problems
> * The job referenced is not found by scheduler
> ** We add a check for if this happens by searching for the jobKey within the
> scheduler and emit a metric
> ** The job key is constructed with flow name/group instead of job
> name/group, this could be an issue for multi-hop jobs but in general should
> not be.
> * Upon looking closely at the code, we also need to update the job props of
> the job in the scheduler to keep track of the event to set reminder for. The
> job props we set
> [here|https://jarvis.corp.linkedin.com/codesearch/result/?name=FlowTriggerHandler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Forchestration&reponame=linkedin%2Fgobblin-elr#176]
> are never used in
> [createTriggerForJob|https://jarvis.corp.linkedin.com/codesearch/result/?name=JobScheduler.java&path=gobblin-elr%2Fgobblin-runtime%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fscheduler&reponame=linkedin%2Fgobblin-elr#588]
> We want to use {{StdScheduler.scheduleJob(JobDetail jobDetail, Set<? extends
> Trigger> triggersForJob, boolean replace)}} to replace the jobDetail we
> construct when making a reminder event.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
