davsclaus opened a new pull request, #3207:
URL: https://github.com/apache/camel-kamelets/pull/3207

   Part of #3170: `azure-servicebus-source` can read a queue or subscription 
with sessions enabled, and its dead-letter queue. `azure-servicebus-sink` can 
send to a queue with sessions enabled. Both can log in with an Azure identity.
   
   ### The problem
   
   - `azure-servicebus-source` had no `sessionEnabled`. On a queue with 
sessions, it started but received nothing, and every 5 seconds it logged `It is 
not possible for an entity that requires sessions to create a non-sessionful 
message receiver`.
   - It had no `subQueue` either, so it could not read the dead-letter queue.
   - `azure-servicebus-sink` had no `sessionId`. Every send to a queue with 
sessions failed with `The SessionId was not set on a message, and it cannot be 
sent to the entity`.
   - `credentialType` offered `AZURE_IDENTITY` and `TOKEN_CREDENTIAL`, but both 
need `fullyQualifiedNamespace`, which the Kamelets did not set. They also 
required `connectionString`. So an Azure identity login worked only through a 
component-level setting plus a dummy connection string.
   
   ### The options
   
   | Kamelet | Option | Template mapping | What it does |
   |---|---|---|---|
   | source | `sessionEnabled` | `sessionEnabled: "{{?sessionEnabled}}"` | 
`true` reads a queue or subscription with sessions, through the session 
processor. |
   | source | `maxConcurrentSessions` | `maxConcurrentSessions: 
"{{?maxConcurrentSessions}}"` | How many sessions are read at the same time. 
When unset, the component default of 1 applies. |
   | source | `subQueue` | `subQueue: "{{?subQueue}}"` | `DEAD_LETTER_QUEUE` or 
`TRANSFER_DEAD_LETTER_QUEUE`. |
   | sink | `sessionId` | `sessionId: "RAW({{?sessionId}})"` | Every message is 
sent with this session ID. |
   | both | `fullyQualifiedNamespace` | `fullyQualifiedNamespace: 
"{{?fullyQualifiedNamespace}}"` | The namespace host name, needed by 
`AZURE_IDENTITY` and `TOKEN_CREDENTIAL`. |
   | both | `connectionString` (changed) | `RAW({{connectionString}})` becomes 
`RAW({{?connectionString}})`, no longer `required` | Can be left out with an 
Azure identity. |
   
   - **Defaults:** all the new options are optional and deliberately have no 
`default`, as in #3174. When they are unset, the optional placeholders drop 
them, and the endpoint URIs are identical to `main`'s.
     - With the default `CONNECTION_STRING` and no `connectionString`, the 
integration does not start: `Azure ServiceBus ConnectionString or FQNS must be 
specified`.
   - **`sessionId` is a fixed option, not a header.** camel-azure-servicebus 
4.24 reads the session ID only from the endpoint configuration. Its 
`CamelAzureServiceBusSessionId` header is a consumer header: the producer does 
not read it.
     - The Kamelet does not build the URI from a header with `toD`. That would 
put message data into the endpoint URI.
     - The descriptions say so, and say to use one sink per session ID. A 
per-message session ID needs the component to read the header, which is an 
`apache/camel` change.
     - `RAW()` keeps the value as written: `a+b %20c&d=1` arrived as `a+b 
%20c&d=1`.
   - **`pattern:`:**
     - `fullyQualifiedNamespace`: `^[A-Za-z0-9][A-Za-z0-9.-]*$`, a host name, 
so `sb://…` or `https://…` is rejected.
     - `sessionId`: `^.{1,128}$`, the Service Bus limit.
   - **Headers:** the source's `dataTypes` now list 
`CamelAzureServiceBusSessionId`, `CamelAzureServiceBusDeadLetterReason` and 
`CamelAzureServiceBusDeadLetterErrorDescription`. The consumer already set 
them; this lists them as metadata only, with no transformer.
   - **Descriptions:**
     - When sessions are needed, and what the error looks like without them.
     - Ordering: the messages of one session are received in the order they 
were sent. The next one comes only after the previous one has been processed, 
unless a step works asynchronously. A failing message is received again before 
the next one of its session, and different sessions have no order between them.
     - `maxConcurrentSessions` 1 holds a session until it has been idle for a 
minute.
     - The dead-letter queue has no sessions.
     - A failing dead letter is received again at once, over and over.
     - For a topic, a message without a session ID is accepted, but a 
subscription with sessions dead-letters it.
     - How to log in with `AZURE_IDENTITY`.
   - **Partial pages:** the authentication section no longer says that the 
Kamelet cannot set the namespace. New sections: "Example with Sessions" and 
"Example Reading the Dead-Letter Queue" for the source, and "Example with a 
Session ID" for the sink.
   
   ### Verification
   
   Tested with Camel JBang 4.24.0-SNAPSHOT against the Azure Service Bus 
emulator (`mcr.microsoft.com/azure-messaging/servicebus-emulator`, with 
`mcr.microsoft.com/mssql/server:2022-latest`). The emulator was configured 
through `Config.json` with:
   - `sessq`: a queue with `RequiresSession: true`;
   - `dlq`: a queue with `MaxDeliveryCount: 2`;
   - `plainq`: a plain queue;
   - `sesstopic`: a topic, with the subscription `sesssub` that has 
`RequiresSession: true`.
   
   Messages were dead-lettered by failing the processing on purpose 
(`throwException`).
   
   | Case | Result |
   |---|---|
   | Options unset, both Kamelets, `main` and this branch, `plainq` | `camel 
get endpoint --json`: identical URIs 
(`azure-servicebus://plainq?connectionString=…&credentialType=CONNECTION_STRING&serviceBusReceiveMode=PEEK_LOCK&serviceBusType=queue`,
 and `…&producerOperation=sendMessages&serviceBusType=queue`); message sent and 
received |
   | Sink to `sessq`, no `sessionId` | Fails: `The SessionId was not set on a 
message, and it cannot be sent to the entity` |
   | Sink to `sessq`, `sessionId: A` | Sent; the reader sees 
`CamelAzureServiceBusSessionId=A` |
   | Sink, `sessionId: "a+b %20c&d=1"` | Endpoint has `sessionId=RAW(…)`; 
received with the session ID `a+b %20c&d=1`, as written |
   | Sink to `plainq` (no sessions), `sessionId: X` | Sent and received; the 
session ID is kept on the message |
   | Sink to `sesstopic`, no `sessionId` | Sent. In `sesssub` it lands in the 
dead-letter queue: reason `Session id is null.`, `Session enabled entity 
doesn't allow a message whose session identifier is null.` |
   | Source on `sessq`, `sessionEnabled` unset | Starts, receives nothing; WARN 
every ~5 s: `It is not possible for an entity that requires sessions to create 
a non-sessionful message receiver` |
   | Source on `sessq`, `sessionEnabled: true` | Receives everything, with the 
session ID header |
   | Same, sessions A and B (5 each), `maxConcurrentSessions` unset | All of A 
at 15:24:36, then all of B at 15:25:37: one session at a time, the next after 
~60 s idle |
   | Same, `maxConcurrentSessions: 2`, synchronous 200 ms step | A and B 
interleaved; within each, 1→5 in order, ~230 ms apart (one at a time) |
   | Same with an asynchronous delay of 200 ms | Within a session still 1→5 in 
order, but 3 ms apart: the next message starts before the previous one is 
finished |
   | `sessionEnabled: true`, A-2 fails on every try | A-1, then A-2 ten times 
(delivery count 0…9), then A-3, A-4: order kept. A-2 dead-lettered |
   | `subQueue: DEAD_LETTER_QUEUE` on `sessq`, `sessionEnabled: true` | WARN: 
`Cannot create a MessageSession for a sub-queue` |
   | `subQueue: DEAD_LETTER_QUEUE` on `sessq`, `sessionEnabled` unset | 
Receives A-2: reason `MaxDeliveryCountExceeded`, `Message could not be consumed 
after 10 delivery attempts.` |
   | `sessionEnabled: true` on `plainq` | WARN every ~5 s: `A sessionful 
message receiver cannot be created on an entity that does not require sessions` 
|
   | `dlq`, 2 messages, processing always fails | Each delivered twice, then 
dead-lettered |
   | `subQueue: DEAD_LETTER_QUEUE` on `dlq` | Both received, with 
`CamelAzureServiceBusDeadLetterReason=MaxDeliveryCountExceeded`; a second run 
receives nothing (completed) |
   | The same, processing fails | The first dead letter is received 871 times 
in 15 s; the second never arrives |
   | `subQueue: TRANSFER_DEAD_LETTER_QUEUE` | Endpoint has 
`subQueue=TRANSFER_DEAD_LETTER_QUEUE`; the consumer starts without error |
   | Topic `sesstopic` / `sesssub`, `sessionEnabled` unset, then `true` | 
Unset: the same WARN as the queue. `true`: T-1, T-2, T-3 received in order |
   | `credentialType: AZURE_IDENTITY`, `fullyQualifiedNamespace: localhost`, no 
`connectionString` | Endpoints have 
`credentialType=AZURE_IDENTITY&fullyQualifiedNamespace=localhost`, with no 
`connectionString`; the client connects to `localhost:5671` (`NAMESPACE: 
localhost`) |
   | `AZURE_IDENTITY`, no `fullyQualifiedNamespace` | Does not start: `Azure 
ServiceBus ConnectionString or FQNS must be specified` |
   | `CONNECTION_STRING`, no `connectionString` | Does not start: the same 
message |
   | `AZURE_IDENTITY` without `connectionString`, `main`'s Kamelets | Does not 
start: `Failure creating route from template` (missing required 
`connectionString`) |
   
   **Not verified:**
   - The Azure identity login itself. The emulator accepts only its SAS 
connection string and speaks plain AMQP on 5672, so the identity case stops at 
the connection to `localhost:5671`. It does show that the namespace reaches the 
client.
   - Dead-lettering of expired messages. The sink cannot set a time to live.
   - On the emulator, a dead letter's delivery count stayed at 2 while it was 
received again and again.
   
   `./mvnw clean verify -DskipTests` and then `./mvnw clean verify` passed from 
the root:
   - Before building, the library copies of the two Kamelets were refreshed 
with `./mvnw -pl library/camel-kamelets 
resources:copy-resources@copy-resource-one`.
   - `KameletsCatalogTest` now expects 10 headers for 
`azure-servicebus-source`, up from 7.
   - The build regenerated `kamelets-index.json`.
   
   _Claude Code on behalf of Claus Ibsen_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to