Hi Claus, Thanks for the +1.
Unless someone has concerns, I will leave the thread open a few more days and then open a JIRA for it as stage 2 of CAMEL-25738: the TimeSource in camel-util, the ~30 call sites in core that change behaviour, and camel-test-dst installing it, so the rewriting is only needed for the component under test. On more fake worlds: I have worlds for IMAP, NATS JetStream, Azure Service Bus, Azure Storage Queue and JMS that found or confirmed bugs this week, and I plan to add them to their components together with the fixes. Looking forward to hearing your other ideas for controlling time once DST and chaos are in place. Thanks, Shashank On Mon, 19 Oct 2026 at 22:53, Claus Ibsen <[email protected]> wrote: > Hi Shashank > > Thanks for doing these benchmarks so we can see what the affect would be. > > Surely the feature of allowing to "control the time" to make deterministic > testing in all the complex camel components possible, > to ensure that we can have a really really well robust component that > handles all kind of edge cases is worth this feature in the core. > > So its a +1 from me. > > PS: I have other ideas where this "control the time" can be useful, but > lets get the DST and the Chaos feature built first. And more fake worlds in > various components so we can make Camel very roboust. > > > > On Mon, Oct 19, 2026 at 1:43 PM Shashank Jain <[email protected]> > wrote: > > > Hi all, > > > > Here are the JMH numbers I promised. JMH 1.37, JDK 21.0.11, Apple M4 Pro > > laptop (8P+4E cores), camel 4.24.0-SNAPSHOT; > > 3 forks, 5 x 1 s warmup, 10 x 1 s measurement, average time in ns/op. A > > laptop result, so indicative only. > > > > 1 thread 8 threads > > System.nanoTime() 8.92 +- 0.01 356.1 +- 3.3 > > TimeSource.nanoTime(), volatile 9.10 +- 0.06 354.9 +- 3.2 > > TimeSource.nanoTime(), plain 9.15 +- 0.08 358.0 +- 3.3 > > TimeSource.nanoTime(), bimorphic 9.22 +- 0.13 355.7 +- 2.8 > > System.currentTimeMillis() 13.59 +- 0.09 186.5 +- 7.8 > > TimeSource.currentTimeMillis() 13.62 +- 0.02 185.2 +- 9.5 > > new StopWatch().taken(), today 19.96 +- 0.04 709.6 +- 7.6 > > the same with TimeSource 20.52 +- 0.12 710.8 +- 7.8 > > > > direct:a -> log:x?level=OFF, route statistics on (today's core, for > > scale): > > 2.43M msg/s 10.6M msg/s (8 > > threads) > > > > "bimorphic" means a second provider class was installed and used once in > > setup, then SYSTEM was put back, so the > > call site has seen two receiver types. With 8 threads the clock calls are > > much slower on this machine for every > > variant, System.nanoTime() included; I have not dug into why, but the > > variants are the same there. > > > > What it means: on one thread the indirection costs about 0.2 ns per > > nanoTime() call and about 0.5 ns per StopWatch, > > nothing measurable for currentTimeMillis(), and nothing measurable with 8 > > threads. The volatile field is not > > measurably slower than a plain static field here (9.10 vs 9.15 ns), so I > > would keep volatile, which also keeps > > install() safe without extra rules. Against about 410 ns per message for > > the simplest direct -> log route, a few > > tenths of a nanosecond per clock read is well below noise. On x86 a > > volatile read is also a plain load, but > > I have only measured on AArch64; numbers from an x86 box would be > welcome. > > > > I can put the benchmark in camel-performance-tests if useful. > > > > Thanks, > > Shashank > > > > On Mon, 19 Oct 2026 at 12:44, Shashank Jain <[email protected]> > > wrote: > > > > > > > > Hi all, > > > > > > camel-test-dst (CAMEL-25738, PR > > https://github.com/apache/camel/pull/28067) > > > runs the real consumer code of a > > > component under one seeded scheduler with virtual time, so a test can > try > > > thousands of interleavings and faults in > > > seconds and replay a failing seed exactly. Worlds written with it found > > or > > > confirmed several timing bugs, for > > > example CAMEL-25683, CAMEL-25737, CAMEL-25741, CAMEL-25756 and > > CAMEL-25758. > > > > > > The harness controls Camel's thread pools through the existing > > > ThreadPoolFactory SPI. Time is the gap: core reads > > > System directly, so today a test either rewrites the byte code of core > > > classes or lives with real time and real > > > sleeps in core. Claus suggested a small time source in camel-util, and > I > > > would like to discuss that here before > > > anything goes into core. > > > > > > Where core reads the clock or sleeps today (src/main/java on main, > > > 2026-10-17): > > > > > > camel-util 4 currentTimeMillis, 5 nanoTime, 3 Thread.sleep > > > camel-support 24 currentTimeMillis, 6 nanoTime, 6 > Thread.sleep, > > 5 > > > Instant.now > > > camel-base 43 currentTimeMillis (nearly all event > timestamps) > > > camel-base-engine 9 currentTimeMillis, 2 Thread.sleep, 1 > > > Instant.now, 1 new Date() > > > camel-core-processor 6 currentTimeMillis, 7 nanoTime, 5 Thread.sleep > > > camel-management 9 currentTimeMillis, 1 Thread.sleep, 2 new > Date() > > > others (api, languages, console, health, main): a handful > > > > > > About 30 of these change behaviour, and only those need to be > controlled > > > in a test: > > > - waits: RedeliveryPolicy.sleep and the sleep loop of > > > RedeliveryErrorHandler, ForegroundTask, the wait loop of > > > DefaultShutdownStrategy, DelayProcessorSupport, MulticastProcessor, > > > OptimisticLockRetryPolicy, > > > DefaultStartupConditionStrategy; > > > - task timing: StopWatch, MonotonicClock, AbstractTask, BackgroundTask, > > > TimeBoundedBudget, BackOffTimerTask, > > > DefaultTimeoutMap; > > > - throttling: TotalRequestsThrottler, ConcurrentRequestsThrottler, > > > SamplingThrottler, > > > ThrottlingExceptionRoutePolicy. > > > > > > Event timestamps, the debugger, the tracer, management and the consoles > > > can stay on System. > > > > > > Options I see: > > > > > > 1. Byte code rewriting only (what stage 1 does). No core change, but > > > runtime agent attach is being phased out in the > > > JDK, it only covers the calls it is told about, and rewritten core > > > classes stay rewritten for the whole test JVM. > > > > > > 2. A time source for all of core: about 120 call sites, many on hot > > paths, > > > for no gain in tests. > > > > > > 3. In between, which is what I would propose: > > > - a small static, internal TimeSource in camel-util: > > > currentTimeMillis(), nanoTime(), instant(), sleep(long). > > > The default delegates to System and Thread.sleep; a test harness > can > > > install another provider. Static rather > > > than per CamelContext, because StopWatch and MonotonicClock have > no > > > context; > > > - move only the ~30 call sites above; > > > - keep byte code rewriting for the component under test, pointed at > > > TimeSource, with the agent loaded by > > > -javaagent instead of attached at runtime; > > > - later, components use it when they are touched anyway, and a > > > forbidden-apis rule keeps new > > > Instant.now() / Thread.sleep() out of consumers. > > > > > > Rough shape: > > > > > > public final class TimeSource { > > > public interface Provider { > > > long currentTimeMillis(); > > > long nanoTime(); > > > void sleep(long millis) throws InterruptedException; > > > } > > > private static volatile Provider provider = SYSTEM; > > > public static long nanoTime() { return provider.nanoTime(); } > > > ... > > > public static Provider install(Provider p) { ... } // test > > > harnesses only > > > } > > > > > > Open points: package and name; a volatile field or a plain static field > > > set once before the context starts; and > > > whether sleep belongs there or in a separate wait helper. > > > > > > On cost: I will run a JMH benchmark (direct System.nanoTime vs. > > TimeSource > > > with a volatile and a plain field, > > > StopWatch, and a direct -> log route with statistics on) and post the > > > numbers in this thread. My expectation is > > > that the default path inlines to the same call plus one ordinary load, > so > > > it should be within noise. > > > > > > Thoughts? > > > > > > Thanks, > > > Shashank > > > > > > > > -- > Claus Ibsen >
