Claus Ibsen created CAMEL-25041:
-----------------------------------
Summary: Dev mode reload: consider a polling directory scan
instead of the JDK WatchService
Key: CAMEL-25041
URL: https://issues.apache.org/jira/browse/CAMEL-25041
Project: Camel
Issue Type: Improvement
Components: camel-core
Reporter: Claus Ibsen
Dev-mode reload watches files with {{java.nio.file.WatchService}}. A directory
scan that compares modification times against a remembered snapshot, the way
the file component's consumer works, would be a better fit for what the reload
strategy actually needs.
*A scan is a batch by construction.* CAMEL-25032 had to reconstruct "the files
of one save" from the watch service, which reports per-file events and gives no
batch boundary: a key's events, plus whatever arrives in the next polls,
bounded by a timeout. A scan returns exactly the set of files that changed
since the last scan, with nothing to guess.
*It is already a scan on macOS, and a slow one.* Java has no native file
notification on macOS, so {{WatchService}} falls back to a polling
implementation. {{FileWatcherResourceReloadStrategy}} knows this and reaches
for {{com.sun.nio.file.SensitivityWatchEventModifier.HIGH}} to get its interval
down to 2 s, warning "it may take up till 10 seconds to notice file changes"
when the class cannot be resolved. On JDK 25 the class *does* resolve and HIGH
*is* applied -- and the interval is still about 10 s. Measured from a dev-mode
run, the gaps between distinct reloads:
{noformat}
01:13:44.885
01:13:52.881 (+8.0s)
01:14:28.891 (+34.0s) <- no change in between
01:14:38.880 (+10.0s)
01:14:48.882 (+10.0s)
01:14:56.886 (+8.0s)
{noformat}
So the workaround has no effect on a current JDK, and everyone developing Camel
routes on a Mac waits up to ten seconds for a save to take effect. A scan with
a 1 s interval would fix that outright, and would drop the dependency on a
{{com.sun.nio.file}} class that was never standard API.
*Other things it would settle:*
* the same behaviour and the same latency on every OS, instead of inotify on
Linux and a hidden poll on macOS;
* a new directory is found for free. The watch service only reports what was
registered when the event happened, which needed the workaround of CAMEL-24862
(a class added under {{src/main/java}} while running was never seen) --
{{registerNewDirectory}} could go;
* no {{OVERFLOW}}: the watch service may drop events under load, a scan cannot;
* a deleted file is a missing entry in the next scan, which is more reliable
than depending on {{ENTRY_DELETE}}.
*Costs to weigh:*
* on Linux, detection goes from effectively immediate to one scan interval. For
dev mode a 1 s scan seems a fair trade, but it is a change for everyone;
* walking the watched folders every second. A source directory is small, and
the existing file filter applies, but {{camel run --source-dir}} pointed at a
large tree would do more work than an inotify watch;
* {{FileWatcherResourceReloadStrategy}} is public, with {{pollTimeout}},
{{isRecursive}}, {{getFileFilter()}} and a protected {{WatchFileChangesTask}}.
Replacing the mechanism inside it changes what those mean, so the class name
and its options want a decision of their own.
Not folded into CAMEL-25032: that one batches the *dispatch* and is
deliberately mechanism-agnostic -- it takes a set of changed files, wherever
the set came from. Once it is in, swapping the watch service for a scan
replaces only how that set is produced, with no rework of the reload logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
