allthingssecurity opened a new pull request, #30138:
URL: https://github.com/apache/camel/pull/30138

   # Description
   
   [CAMEL-26958](https://issues.apache.org/jira/browse/CAMEL-26958), part of 
CAMEL-26610.
   
   What the world found on main, up front: two cases fail on main and are 
looked at separately (see Tests). With Paho 1.2.5, a connect that comes while 
the callback thread is still in `messageArrived` after the connection closed 
(the reconnect of the consumer after a failed message on a kept session, or the 
automatic reconnect, while another message is in the route) never completes, as 
read in the byte code of Paho 1.2.5 (not reproduced against a broker yet): 
`CommsCallback.start` restarts the thread only when it is stopped and then 
waits until it runs, so the client stays connecting, delivers nothing more and 
cannot be closed (32110). By default, when the client of the consumer hangs 
that way at the end of the run, the world does not check that every message is 
routed, and does not check that a hung client is closed. And a route stop while 
the client connects makes the close of the client fail (Paho throws 32110), so 
the client goes on connecting next to the client of the next start
 , and the two take the session from each other; by default the operator does 
not stop the route then.
   
   Two differences from the JIRA text: an acknowledgement that blocks is not 
driven (the fake answers a manual acknowledgement at once, as Paho only queues 
the PUBACK), and CAMEL-24523 is a fix of camel-paho (MQTT 3), so the self-check 
uses CAMEL-25793 and CAMEL-25944; the restart of the route after a failed 
resubscribe, which CAMEL-24523 brought to camel-paho and camel-paho-mqtt5 has 
too, is driven.
   
   A deterministic simulation test (camel-test-dst) of the camel-paho-mqtt5 
consumer, in the `dst` package of its tests, in the same form as the SQS world: 
`PahoMqtt5ConsumerDstTest` (1000 seeds), `PahoMqtt5World` and `FakeMqtt`. Test 
only; no main code changes.
   
   What the world models: the real consumer (`PahoMqtt5Consumer`, its client 
callback, its reconnect after a failed message and its route restart after a 
failed resubscribe) and Camel's routing run on the virtual-time loop; only the 
broker and the network calls of the Paho client are faked. The consumer creates 
its own client in the package-private `createClient()`, so a small test 
component in the component's own test package, `PahoMqtt5DstComponent`, returns 
a consumer whose `createClient()` gives the fake client, a subclass of the Paho 
`MqttClient` whose network calls go to the fake broker; the endpoint and the 
consumer are otherwise the real ones, and the options come from the 
configuration of the component (the generated configurer of the endpoint does 
not apply to a subclass). The fake is written from the MQTT 5.0 specification 
and Paho 1.2.5 (read with javap where the behaviour is in its code), and cites 
them: the session of the client id (subscriptions, QoS 1 and 2 messages i
 n flight, messages queued while the client is offline), Clean Start, the 
Session Expiry Interval, Session Present, the resend of unacknowledged messages 
on a reconnect to a kept session, Receive Maximum, manual acknowledgements, 
deliveries on one callback thread (a message that arrives while no callback is 
set is dropped, the callback thread that a disconnect stops and that the next 
connect starts again only when it has stopped, and the connect that hangs when 
the thread is still busy), a connection closed by the broker and the automatic 
reconnect of Paho, a connect or a close while a connect is in progress (32110), 
a broker that cannot be reached for a while, and a connection lost during a 
subscribe (Paho waits for a SUBACK without a time limit).
   
   The seed picks the QoS, `manualAcksEnabled`, `cleanStart`, 
`sessionExpiryInterval`, `automaticReconnect`, `receiveMaximum`, 
`maxReconnectDelay` and the fault rates, and injects connections closed by the 
broker (also during a subscribe), unreachable periods, failed and slow 
processing, and route stops and starts.
   
   Invariants: with manual acknowledgements a QoS 1 message is acknowledged 
only after its exchange completed, or after it failed (or was refused by a 
route that stops) when the broker would not send it again; the client drops no 
message; once the faults cleared every message that the broker queued for the 
session (and did not lose with a session that ended, or with the queue of a 
client) is routed, a QoS 1 message with manual acknowledgements on a kept 
session completes, and nothing stays in flight or queued; after the final stop 
the client is disconnected and closed; the consumer uses the client as Paho 
allows (no disconnect from the callback thread, no call on a closed client).
   
   Limits, written in the world's Javadoc: the operator stops the route only 
while no message is in the route, and a message that arrives while a stop runs 
is processed at once (on the one stack the shutdown could not see an exchange 
below it finish); for the same reason a subscribe fails only while no message 
is in the route. Without `automaticReconnect` Paho does not reconnect after the 
broker closed the connection, so the operator restarts the route then. A 
message delivered while the route stops is refused by the route; without manual 
acknowledgements, or when the broker does not keep the session, its 
acknowledgement leaves it lost (as the Javadoc of the consumer's rollback 
says), and the world does not count it. Not explored: a client given with the 
`client` option (shared by several consumers), the persistence of the client, 
retained messages and wildcards.
   
   Other changes:
   - `camel-test-dst` as a test dependency of camel-paho-mqtt5, and its byte 
code agent on the surefire command line, as for the other worlds.
   - The test log of camel-paho-mqtt5 goes to a size-capped rolling file (50 
MB, one backup), as a failing world can log a lot.
   - The `@DstWorld` annotation lists the world in the component JSON (and its 
catalog copy) and in the table of the camel-test-dst page (and its catalog 
copy, byte-identical).
   
   Tests:
   - On main: 5000 of 5000 seeds pass (about 7 ms a seed). Seeds 8 and 77 
replayed in two JVMs give the same trace hashes.
   - Self-check against fixed bugs, each on the code from before its fix (the 
runs stop at 100 failing seeds; these runs and the 5000 seeds on main used the 
same world code before the rebase onto current main, which only brought the 
opt-in wall clock jumps to camel-test-dst):
     - CAMEL-25793, with its failure handling of manual acknowledgements taken 
out by hand (a failed message is only logged and stays unacknowledged, as 
before b7594f06e0b7): 100 of the first 1415 seeds fail, all on "every message 
is routed once the faults cleared" (the failed message is never sent again on a 
kept session).
     - before CAMEL-25944 (6044f38166f9 reverted, so the client connects before 
its callback is set): 100 of the first 1118 seeds fail, all on "the client 
drops no message" (the messages the broker sends right after the connect to a 
kept session arrive while no callback is set).
   - The cases left out of the default set, each with its property (these are 
being looked at separately):
     - `-Ddst.paho.strictCallbackThread=true`: 100 of the first 1157 seeds 
fail, all on "every message is routed once the faults cleared": the client 
hangs in a connect that came while the callback thread was still in 
`messageArrived`. These counts are of the fake, which leaves out the quiesce of 
a disconnect (Paho waits up to 30 seconds for the callback queues to empty), so 
the busy window of the real client differs; a reproduction outside DST comes 
before any JIRA.
     - `-Ddst.paho.stopWhileConnecting=true`: 7 of 2000 seeds fail, 6 on "the 
stopped consumer disconnects and closes its client" and 1 on "every message is 
routed once the faults cleared": the close fails with 32110 while the client 
connects, and that client goes on next to the client of the next start.
   - camel-paho-mqtt5 unit tests: 1075 run (75, plus 1000 seeds), 0 failures, 0 
errors, 0 skipped (the ITs need Docker and were not run).
   
   What was not run locally: the ITs (they need Docker) and the full root 
build. I built and tested offline with the versions cached on my machine: 
Caffeine 3.3.0 for the tests (main has 3.3.1), commons-lang3 3.21.0 and Guava 
33.7.2 (main has 3.22.0 and 33.8.0-jre). Paho 1.2.5 is the version of main.
   
   # Target
   
   - [x] I checked that the commit is targeting the correct branch (Camel 4 
uses the `main` branch)
   
   # Tracking
   - [x] If this is a large change, bug fix, or code improvement, I checked 
there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for 
the change (usually before you start working on it): 
[CAMEL-26958](https://issues.apache.org/jira/browse/CAMEL-26958).
   
   # Apache Camel coding standards and style
   
   - [x] I checked that each commit in the pull request has a meaningful 
subject line and body.
   - [ ] I have run `mvn clean install -DskipTests` locally from root folder 
and I have committed all auto-generated changes.
     (I built and tested camel-test-dst and camel-paho-mqtt5, including the 
formatter and import-sort plugins. The generated files touched are the 
component JSON with its catalog copy, and the table of the camel-test-dst page 
with its catalog copy (byte-identical). I did not run the full root build.)
   
   # AI-assisted contributions
   
   - [x] If this PR includes AI-generated code, commits have proper 
co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR 
description identifies the AI tool used.
     This PR was prepared with Claude Code (Claude Opus 5.5). The commit 
carries a `Co-Authored-By` trailer.
   
   _Claude Code on behalf of allthingssecurity_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to