jnioche opened a new pull request, #2290: URL: https://github.com/apache/stormcrawler/pull/2290
`AbstractStatusUpdaterBolt.execute()` rounds the next fetch date with `DateUtils.round`, which builds a `GregorianCalendar` for every URL. When profiling the status updater with `StatusUpdaterBenchmark` going through `execute()`, this was the largest share of the CPU used by the bolt threads, about a third of it, more than the cache or the filtering of the metadata. Rounding to the second or the minute is now done on the epoch time, half up, which gives the same dates as `DateUtils`. A quick timing gives about 230 ns per call with `DateUtils` and a few ns without it. Rounding to the hour depends on the time zone, e.g. with an offset of 5:30, so it still goes through `DateUtils`. The tests compare the new rounding with `DateUtils` on edge values and on 100,000 random dates. They pass with UTC, Asia/Kolkata, America/St_Johns, Asia/Kathmandu, Europe/Paris, America/New_York and Australia/Lord_Howe as default time zone. While testing, I found that `DateUtils` can move a date in the hour repeated when the clocks go back to the other occurrence of that hour. In Paris, 02:22:47 CEST is rounded to the minute as 02:23 CET, an hour later. The new rounding keeps it at 02:23 CEST, a test covers this case. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
