waterWang opened a new pull request, #13927:
URL: https://github.com/apache/cloudstack/pull/13927

   Fixes #13906
   
   ### Problem
   `UsageManagerImpl.parse()` rewinds the aggregation start date to the oldest 
unprocessed event, but has no bound on how far back it can rewind. If an event 
cannot be successfully processed, the rewind pins the window to that event's 
date permanently. Each subsequent run re-aggregates from that date to the 
present, growing by one aggregation period per run while the job keeps 
reporting `success = 1`.
   
   Reported symptoms (4.22.1.0):
   - 1,678 consecutive successful jobs whose `start_millis` never advanced past 
a single event 71 days earlier
   - `exec_time` up to 42.6 min per hourly run, growing by 24 aggregation 
periods/day
   - `cloud_usage` at 54.4M rows / 14 GB, mostly duplicate replayed rows
   - Not specific to one event type (also #13112 on 4.21.0.0)
   
   ### Fix
   Bound the rewind to a 24-hour window. The rewind exists to absorb clock skew 
between the cloud and usage databases (events created during the previous run 
window), not to replay the full history. Events older than 24h from the current 
window start will still be retried, but the aggregation window will not be 
rewound to them.
   
   ```java
   startDateMillis = Math.max(oldestEventDate.getTime(), startDateMillis - 
MAX_EVENT_REWIND_MILLIS);
   ```
   
   ### Testing
   - Unit test added? No (existing test class only covers event handlers; 
parse() has no prior tests)
   - Verified the constant is referenced exactly twice (declaration + rewind 
bound)
   - No behavior change for events within the 24h boundary window — they still 
rewind the window as before


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to