Arnold Galovics created FINERACT-2923:
-----------------------------------------
Summary: Execute missed scheduler job runs after restart or resume
Key: FINERACT-2923
URL: https://issues.apache.org/jira/browse/FINERACT-2923
Project: Apache Fineract
Issue Type: Improvement
Reporter: Arnold Galovics
Assignee: Arnold Galovics
✅ Acceptance Criteria
• New property fineract.job.execute-missed-runs (env
FINERACT_JOB_EXECUTE_MISSED_RUNS), default false. With it off, behaviour is
unchanged.
• Boot: an active job whose stored next_run_time is in the past gets its
trigger started at that time, so Quartz applies the misfire instruction. The
default policy fires once.
• INCREASE_BUSINESS_DATE_BY_1_DAY uses
MISFIRE_INSTRUCTION_IGNORE_MISFIRE_POLICY and fires once per missed run, one
after another. INCREASE_COB_DATE_BY_1_DAY stays fire-once.
• Boot un-suspends a tenant with reset_scheduler_on_bootup before registering
its jobs, so replayed fires are not vetoed.
• Resume (POST /scheduler?command=start) re-registers each active job with a
past next_run_time the same way boot does and clears is_misfired. With the flag
off, the existing is_misfired replay is unchanged.
• A job never runs concurrently with itself. Fires while suspended stay vetoed
and do not advance next_run_time.
• SchedulerTriggerListener#triggerMisfired logs at INFO instead of ERROR.
• Unit tests with a real in-memory Quartz scheduler cover restart,
suspend/resume, restart while suspended, and a multi-day outage (business date
advances once per missed run).
• The property and the known limits are documented in batch-jobs.adoc.
📘 Background
• Quartz runs in memory per tenant (no QRTZ tables). Boot builds every trigger
to start now, so nothing ever looks missed. A fire while suspended is vetoed,
which Quartz counts as executed, not missed.
• job.next_run_time already records the first run that never happened: a vetoed
fire never advances it. That is enough to drive Quartz's own misfire handling,
with no persistent job store.
• The existing resume replay (execute_misfired_jobs) races with itself, see
FINERACT-2918.
ℹ️ Notes
• Known limits: periodic Standing Instructions match an exact date, so skipped
dates are lost. Retained Earning is wrong if the business date jumps over a
year end in one step. Jobs outside COB date their postings on the catch-up day
(due savings charges, transfer fee, dormancy, delinquency history). Report
mailing sends one duplicate per missed run after a long gap.
• Checked per job (40 jobs, 1.15.0): fire-once is correct for nearly all of
them. Exceptions: Increase Business Date (once per missed run), Loan COB and
Working Capital COB (catch-up, see the COB ticket), Increase COB Date (must not
be replayed per missed run while enable_automatic_cob_date_adjustment is on,
which is the default).
• Depends on the reset-on-bootup property ticket for the cleanest boot ordering
(same lines).
⚠️ Out of Scope
• COB catch-up on a replayed COB run (separate ticket).
• Keeping the scheduler suspended across restarts (separate ticket).
• A persistent Quartz job store, and a cross-job dependency or ordering concept.
🔧 Technical Notes
• JobRegisterServiceImpl#createTrigger: if the job is active and next_run_time
is before now, CronTriggerFactoryBean#setStartTime(nextRunTime). With the flag
on, INCREASE_BUSINESS_DATE_BY_1_DAY gets
setMisfireInstruction(IGNORE_MISFIRE_POLICY), resolved via jobNameService like
createJobDetail.
• JobRegisterServiceImpl#startScheduler: with the flag on, re-register active
jobs with a past next_run_time (deleteJob, scheduleJob, saveOrUpdate) and clear
is_misfired.
• JobSchedulerServiceImpl#onApplicationEvent: move the
reset_scheduler_on_bootup block above the job loop.
• Quartz's RAMJobStore misfire threshold is the 5 second default, so a gap
shorter than that simply fires late.
• Code is identical on 1.15.0 and develop apart from Spring Batch 6 imports.
• Tests cannot go in integration-tests: it runs one long-lived server and
cannot restart it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
