allthingssecurity opened a new pull request, #30138: URL: https://github.com/apache/camel/pull/30138
# Description [CAMEL-26958](https://issues.apache.org/jira/browse/CAMEL-26958), part of CAMEL-26610. What the world found on main, up front: two cases fail on main and are looked at separately (see Tests). With Paho 1.2.5, a connect that comes while the callback thread is still in `messageArrived` after the connection closed (the reconnect of the consumer after a failed message on a kept session, or the automatic reconnect, while another message is in the route) never completes, as read in the byte code of Paho 1.2.5 (not reproduced against a broker yet): `CommsCallback.start` restarts the thread only when it is stopped and then waits until it runs, so the client stays connecting, delivers nothing more and cannot be closed (32110). By default, when the client of the consumer hangs that way at the end of the run, the world does not check that every message is routed, and does not check that a hung client is closed. And a route stop while the client connects makes the close of the client fail (Paho throws 32110), so the client goes on connecting next to the client of the next start , and the two take the session from each other; by default the operator does not stop the route then. Two differences from the JIRA text: an acknowledgement that blocks is not driven (the fake answers a manual acknowledgement at once, as Paho only queues the PUBACK), and CAMEL-24523 is a fix of camel-paho (MQTT 3), so the self-check uses CAMEL-25793 and CAMEL-25944; the restart of the route after a failed resubscribe, which CAMEL-24523 brought to camel-paho and camel-paho-mqtt5 has too, is driven. A deterministic simulation test (camel-test-dst) of the camel-paho-mqtt5 consumer, in the `dst` package of its tests, in the same form as the SQS world: `PahoMqtt5ConsumerDstTest` (1000 seeds), `PahoMqtt5World` and `FakeMqtt`. Test only; no main code changes. What the world models: the real consumer (`PahoMqtt5Consumer`, its client callback, its reconnect after a failed message and its route restart after a failed resubscribe) and Camel's routing run on the virtual-time loop; only the broker and the network calls of the Paho client are faked. The consumer creates its own client in the package-private `createClient()`, so a small test component in the component's own test package, `PahoMqtt5DstComponent`, returns a consumer whose `createClient()` gives the fake client, a subclass of the Paho `MqttClient` whose network calls go to the fake broker; the endpoint and the consumer are otherwise the real ones, and the options come from the configuration of the component (the generated configurer of the endpoint does not apply to a subclass). The fake is written from the MQTT 5.0 specification and Paho 1.2.5 (read with javap where the behaviour is in its code), and cites them: the session of the client id (subscriptions, QoS 1 and 2 messages i n flight, messages queued while the client is offline), Clean Start, the Session Expiry Interval, Session Present, the resend of unacknowledged messages on a reconnect to a kept session, Receive Maximum, manual acknowledgements, deliveries on one callback thread (a message that arrives while no callback is set is dropped, the callback thread that a disconnect stops and that the next connect starts again only when it has stopped, and the connect that hangs when the thread is still busy), a connection closed by the broker and the automatic reconnect of Paho, a connect or a close while a connect is in progress (32110), a broker that cannot be reached for a while, and a connection lost during a subscribe (Paho waits for a SUBACK without a time limit). The seed picks the QoS, `manualAcksEnabled`, `cleanStart`, `sessionExpiryInterval`, `automaticReconnect`, `receiveMaximum`, `maxReconnectDelay` and the fault rates, and injects connections closed by the broker (also during a subscribe), unreachable periods, failed and slow processing, and route stops and starts. Invariants: with manual acknowledgements a QoS 1 message is acknowledged only after its exchange completed, or after it failed (or was refused by a route that stops) when the broker would not send it again; the client drops no message; once the faults cleared every message that the broker queued for the session (and did not lose with a session that ended, or with the queue of a client) is routed, a QoS 1 message with manual acknowledgements on a kept session completes, and nothing stays in flight or queued; after the final stop the client is disconnected and closed; the consumer uses the client as Paho allows (no disconnect from the callback thread, no call on a closed client). Limits, written in the world's Javadoc: the operator stops the route only while no message is in the route, and a message that arrives while a stop runs is processed at once (on the one stack the shutdown could not see an exchange below it finish); for the same reason a subscribe fails only while no message is in the route. Without `automaticReconnect` Paho does not reconnect after the broker closed the connection, so the operator restarts the route then. A message delivered while the route stops is refused by the route; without manual acknowledgements, or when the broker does not keep the session, its acknowledgement leaves it lost (as the Javadoc of the consumer's rollback says), and the world does not count it. Not explored: a client given with the `client` option (shared by several consumers), the persistence of the client, retained messages and wildcards. Other changes: - `camel-test-dst` as a test dependency of camel-paho-mqtt5, and its byte code agent on the surefire command line, as for the other worlds. - The test log of camel-paho-mqtt5 goes to a size-capped rolling file (50 MB, one backup), as a failing world can log a lot. - The `@DstWorld` annotation lists the world in the component JSON (and its catalog copy) and in the table of the camel-test-dst page (and its catalog copy, byte-identical). Tests: - On main: 5000 of 5000 seeds pass (about 7 ms a seed). Seeds 8 and 77 replayed in two JVMs give the same trace hashes. - Self-check against fixed bugs, each on the code from before its fix (the runs stop at 100 failing seeds; these runs and the 5000 seeds on main used the same world code before the rebase onto current main, which only brought the opt-in wall clock jumps to camel-test-dst): - CAMEL-25793, with its failure handling of manual acknowledgements taken out by hand (a failed message is only logged and stays unacknowledged, as before b7594f06e0b7): 100 of the first 1415 seeds fail, all on "every message is routed once the faults cleared" (the failed message is never sent again on a kept session). - before CAMEL-25944 (6044f38166f9 reverted, so the client connects before its callback is set): 100 of the first 1118 seeds fail, all on "the client drops no message" (the messages the broker sends right after the connect to a kept session arrive while no callback is set). - The cases left out of the default set, each with its property (these are being looked at separately): - `-Ddst.paho.strictCallbackThread=true`: 100 of the first 1157 seeds fail, all on "every message is routed once the faults cleared": the client hangs in a connect that came while the callback thread was still in `messageArrived`. These counts are of the fake, which leaves out the quiesce of a disconnect (Paho waits up to 30 seconds for the callback queues to empty), so the busy window of the real client differs; a reproduction outside DST comes before any JIRA. - `-Ddst.paho.stopWhileConnecting=true`: 7 of 2000 seeds fail, 6 on "the stopped consumer disconnects and closes its client" and 1 on "every message is routed once the faults cleared": the close fails with 32110 while the client connects, and that client goes on next to the client of the next start. - camel-paho-mqtt5 unit tests: 1075 run (75, plus 1000 seeds), 0 failures, 0 errors, 0 skipped (the ITs need Docker and were not run). What was not run locally: the ITs (they need Docker) and the full root build. I built and tested offline with the versions cached on my machine: Caffeine 3.3.0 for the tests (main has 3.3.1), commons-lang3 3.21.0 and Guava 33.7.2 (main has 3.22.0 and 33.8.0-jre). Paho 1.2.5 is the version of main. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-26958](https://issues.apache.org/jira/browse/CAMEL-26958). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built and tested camel-test-dst and camel-paho-mqtt5, including the formatter and import-sort plugins. The generated files touched are the component JSON with its catalog copy, and the table of the camel-test-dst page with its catalog copy (byte-identical). I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
