oscerd commented on PR #2978:
URL: https://github.com/apache/camel-kamelets/pull/2978#issuecomment-5407107616
@davsclaus thanks — adopted, and chasing it down changed what this PR does,
so flagging that up front.
## Adopted: `exec-sink` and `aws-ec2-sink`
Both now use a single pattern strip placed **before** the Kamelet sets its
own headers, as you suggested:
```yaml
- removeHeaders:
pattern: "CamelExec*"
- choice: # args / ce-args -> CamelExecCommandArgs
```
```yaml
- removeHeaders:
pattern: "CamelAwsEC2*"
excludePattern: "CamelAwsEC2(NextToken|MaxResults)"
- choice: # instanceIds / ce-instanceids ->
CamelAwsEC2InstancesIds
```
Moving the strip ahead of the `choice` removes the need to enumerate
exceptions for the headers the Kamelet itself sets, and it caught keys the
enumerated version had missed — `CamelExecCommandTimeout`,
`CamelExecExitValues`, `CamelExecCommandLogLevel`.
The one `excludePattern`: `aws-ec2-sink` pins `operation:
describeInstancesStatus`, and `CamelAwsEC2NextToken` / `CamelAwsEC2MaxResults`
are the pagination headers that operation actually reads. They are functional
rather than dispatch-controlling, so stripping them would have been a silent
regression with no security benefit.
Verified with Camel JBang against the real templates (terminal `to:` swapped
for a `log:` so the headers at dispatch time are visible):
```
PROBEEXEC : Exchange[Headers: {args=hello-args,
CamelExecCommandArgs=hello-args, keepMe=yes}]
PROBEEC2 : Exchange[Headers: {CamelAwsEC2InstancesIds=i-123,
CamelAwsEC2MaxResults=10,
CamelAwsEC2NextToken=PAGE2,
instanceIds=i-123, keepMe=yes}]
```
`CamelExecCommandExecutable` / `WorkingDir` / `OutFile` / `Timeout` and
`CamelAwsEC2Operation` / `ImageId` / `InstanceType` are gone; each Kamelet’s
own contract header survives; unrelated headers are untouched.
## Not adopted: `kafka-sink` is reverted
Your question — *should this Kamelet allow any existing headers?* — is the
one that broke this half. For `kafka-sink` the answer turns out to be **yes**,
and the original enumerated strip was already wrong.
Three shipped Kamelets exist for the sole purpose of setting
`CamelKafkaOverrideTopic` for a downstream `kafka-sink`:
| Kamelet | bean |
|---|---|
| `regex-router-action` | `kafka.transform.RegexRouter` |
| `timestamp-router-action` | `kafka.transform.TimestampRouter` |
| `message-timestamp-router-action` |
`kafka.transform.MessageTimestampRouter` |
There is a Pipe test for exactly that composition —
`tests/.../kafka/kafka-router-pipe.yaml` is `webhook-source ->
timestamp-router-action -> ... -> kafka-sink (topic: dummy)`, and the Citrus
assertion consumes from the **overridden** topic, with `dummy` deliberately a
throwaway.
Reproduced locally against the PR’s own template:
```
BEFORE SINK -> CamelKafkaOverrideTopic=prefix-my-topic
PROBE : Exchange[Headers: {CamelKafkaTopic=my-topic}, BodyType: String]
```
So the premise in that file’s comment — *“the topic is pinned to
`{{topic}}`”* — was simply false. Header-driven topic override is a designed
catalog feature, and the strip silently disabled three Kamelets. `kafka-sink`
is now byte-identical to `main`.
This also rules out a blanket `CamelKafka*` there: it would additionally
strip `CamelKafkaKey` (set by `value-to-key-action`) and the consumer-side
`CamelKafkaManualCommit` / `Offset` / `LastRecordBeforeCommit` that a
`kafka-source` -> `kafka-sink` Pipe carries.
## One thing worth a maintainer’s eye
`kafka-router-pipe-test` **passed** on the previous head despite the above,
so the itests are not running against the catalog built in the same run. Filed
separately as #2984 — no change here.
Since the changeset moved (`kafka-sink` dropped, `exec-sink` /
`aws-ec2-sink` reshaped), happy to have another look when you have a moment.
---
_Claude Code on behalf of Andrea Cosentino_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
