This is an automated email from the ASF dual-hosted git repository.
oscerd pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
The following commit(s) were added to refs/heads/main by this push:
new 4dbc3b3c4 chore: aws-sqs-source can read the attributes of a message
(#3170) (#3192)
4dbc3b3c4 is described below
commit 4dbc3b3c4192c03c53186f9053f56297d3135eb5
Author: Claus Ibsen <[email protected]>
AuthorDate: Wed Nov 18 15:10:23 2026 +0100
chore: aws-sqs-source can read the attributes of a message (#3170) (#3192)
aws-sqs-source asked SQS for no attributes, so the message attributes
the sender set and the system attributes of SQS (SentTimestamp,
ApproximateReceiveCount, ...) were never read.
Two optional options, mapped with optional placeholders so that the
endpoint is exactly as before when they are not set:
- messageAttributeNames -> messageAttributeNames: All, a comma-separated
list, or a prefix such as trace.*
- attributeNames -> attributeNames: All or a comma-separated list of
system attribute names
Each attribute that is read becomes a header of the same name; a message
attribute whose name starts with Camel stays only in the
CamelAwsSqsMessageAttributes Map (default header filter of aws2-sqs).
The descriptions, the dataTypes headers and the partial page say what
the user gets, checked with Camel JBang against Floci.
Co-authored-by: Claude Opus 5.5 <[email protected]>
---
.../ROOT/partials/aws-sqs-source-description.adoc | 30 ++++++++++++++++++++++
kamelets/aws-sqs-source.kamelet.yaml | 18 ++++++++++---
.../kamelets/catalog/KameletsCatalogTest.java | 4 ++-
.../main/resources/META-INF/kamelets-index.json | 2 +-
.../resources/kamelets/aws-sqs-source.kamelet.yaml | 18 ++++++++++---
5 files changed, 64 insertions(+), 8 deletions(-)
diff --git a/docs/modules/ROOT/partials/aws-sqs-source-description.adoc
b/docs/modules/ROOT/partials/aws-sqs-source-description.adoc
index 29b5ef89c..dd3d71c39 100644
--- a/docs/modules/ROOT/partials/aws-sqs-source-description.adoc
+++ b/docs/modules/ROOT/partials/aws-sqs-source-description.adoc
@@ -18,3 +18,33 @@ You can also use the Profile Credentials Provider, by
setting the `useProfileCre
Only one of access key/secret key or default credentials provider could be used
For more information, see the
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html[AWS
credentials documentation]
+
+=== Reading the attributes of a message
+
+By default the Kamelet reads only the body of each message. Two options ask
SQS for the attributes too:
+
+- **Message Attribute Names** (`messageAttributeNames`): the message
attributes, which the sender sets on a message. Use `All`, a comma-separated
list of names, or a name that ends in `.*` for all the names with that start,
such as `trace.*`.
+- **System Attribute Names** (`attributeNames`): the attributes that SQS keeps
for each message, such as `SentTimestamp` (in milliseconds since 1970) and
`ApproximateReceiveCount`. Use `All` or a comma-separated list of names.
+
+Each attribute that is read becomes a header of the same name, so the next
steps can use it as `customer` or `SentTimestamp`. The values are:
+
+[width="100%",cols="1,3",options="header"]
+|===
+| Attribute | Header value
+| String | Text.
+| Number | Text, such as `5`. With a type label such as `Number.int`, a number.
+| Binary | Bytes, as an AWS SDK `SdkBytes`.
+| System attribute | Text.
+|===
+
+A message attribute whose name starts with `Camel` does not become a header.
It is still in the `CamelAwsSqsMessageAttributes` header, a Map from each name
to an AWS SDK `MessageAttributeValue`.
+
+For example, to read all the message attributes and the time each message was
sent:
+
+[source,yaml]
+----
+queueNameOrArn: orders
+region: eu-west-1
+messageAttributeNames: All
+attributeNames: SentTimestamp
+----
diff --git a/kamelets/aws-sqs-source.kamelet.yaml
b/kamelets/aws-sqs-source.kamelet.yaml
index b60483842..a2d4f3bc5 100644
--- a/kamelets/aws-sqs-source.kamelet.yaml
+++ b/kamelets/aws-sqs-source.kamelet.yaml
@@ -37,7 +37,7 @@ spec:
Use it to process work that other systems put in a queue, such as
orders, jobs or notifications, or the events that SNS, S3 or EventBridge send
to a queue. For the events of an S3 bucket, `aws-s3-event-based-source` can
also fetch the files.
- Output: the body is the body of the SQS message, as text. The headers
give the message ID (`CamelAwsSqsMessageId`), the receipt handle and the MD5
checksum of the body. The message attributes of the SQS message are not read.
+ Output: the body is the body of the SQS message, as text. The headers
give the message ID (`CamelAwsSqsMessageId`), the receipt handle and the MD5
checksum of the body. The message attributes, which the sender sets on a
message, are read only when `messageAttributeNames` names them, and the system
attributes of SQS, such as `SentTimestamp`, only when `attributeNames` names
them. Each attribute that is read becomes a header of the same name, except a
message attribute whose name sta [...]
Good to know: a message is deleted only after it has been processed
without error; a message that fails stays in the queue and is received again
when its visibility timeout ends. With `deleteAfterRead` set to false, the
messages stay in the queue and are received again and again. Without
`queueURL`, the queue is looked up by listing the queues, which needs the
`sqs:ListQueues` permission.
required:
@@ -162,6 +162,16 @@ spec:
example: 30
type: integer
minimum: "0"
+ messageAttributeNames:
+ title: Message Attribute Names
+ description: "The message attributes to read: `All`, or a
comma-separated list of names, such as `customer,priority`. A name that ends in
`.*`, such as `trace.*`, reads the attributes whose names start with `trace.`.
Each attribute that is read becomes a header of the same name: a String
attribute as text, a Number attribute as text, or as a number when its type has
a label such as `Number.int`, and a Binary attribute as bytes (an AWS SDK
`SdkBytes`). A message attribute whose na [...]
+ example: All
+ type: string
+ attributeNames:
+ title: System Attribute Names
+ description: "Which of the system attributes that SQS keeps for each
message to read: `All`, or a comma-separated list of names from `SenderId`,
`SentTimestamp`, `ApproximateReceiveCount`, `ApproximateFirstReceiveTimestamp`,
`SequenceNumber`, `MessageDeduplicationId`, `MessageGroupId`, `AWSTraceHeader`
and `DeadLetterQueueSourceArn`. A name that is not one of these is ignored,
with a warning in the log. Each attribute that is read becomes a header of the
same name, as text; the t [...]
+ example: SentTimestamp,ApproximateReceiveCount
+ type: string
dataTypes:
out:
default: text
@@ -181,11 +191,11 @@ spec:
type: string
CamelAwsSqsAttributes:
title: System Attributes
- description: A Map of the system attributes of the message, such as
`SentTimestamp`. The Kamelet asks for none, so it is empty.
+ description: A Map of the system attributes that `attributeNames`
asks for, such as `SentTimestamp`, with their values as text. Its keys are AWS
SDK `MessageSystemAttributeName` values, not text, so read one attribute from
its own header instead. Empty when `attributeNames` is not set.
type: object
CamelAwsSqsMessageAttributes:
title: Message Attributes
- description: A Map of the message attributes of the message. The
Kamelet asks for none, so it is empty.
+ description: A Map of the message attributes that
`messageAttributeNames` asks for, from the name of each attribute to an AWS SDK
`MessageAttributeValue`, which holds its data type and its value. It also holds
the attributes whose names start with `Camel`. Empty when
`messageAttributeNames` is not set.
type: object
types:
text:
@@ -227,11 +237,13 @@ spec:
queueNameOrArn: "{{queueNameOrArn}}"
accessKey: "RAW({{?accessKey}})"
amazonAWSHost: "{{?amazonAWSHost}}"
+ attributeNames: "{{?attributeNames}}"
autoCreateQueue: "{{autoCreateQueue}}"
delay: "{{delay}}"
deleteAfterRead: "{{deleteAfterRead}}"
greedy: "{{greedy}}"
maxMessagesPerPoll: "{{maxMessagesPerPoll}}"
+ messageAttributeNames: "{{?messageAttributeNames}}"
overrideEndpoint: "{{overrideEndpoint}}"
profileCredentialsName: "{{?profileCredentialsName}}"
protocol: "{{?protocol}}"
diff --git
a/library/camel-kamelets-catalog/src/test/java/org/apache/camel/kamelets/catalog/KameletsCatalogTest.java
b/library/camel-kamelets-catalog/src/test/java/org/apache/camel/kamelets/catalog/KameletsCatalogTest.java
index 050cd42ee..db292526a 100644
---
a/library/camel-kamelets-catalog/src/test/java/org/apache/camel/kamelets/catalog/KameletsCatalogTest.java
+++
b/library/camel-kamelets-catalog/src/test/java/org/apache/camel/kamelets/catalog/KameletsCatalogTest.java
@@ -56,7 +56,7 @@ public class KameletsCatalogTest {
@Test
void testGetKameletsDefinition() throws Exception {
Definition props = catalog.getKameletDefinition("aws-sqs-source");
- assertEquals(21, props.getProperties().keySet().size());
+ assertEquals(23, props.getProperties().keySet().size());
assertTrue(props.getProperties().containsKey("queueNameOrArn"));
assertTrue(props.getProperties().containsKey("deleteAfterRead"));
assertTrue(props.getProperties().containsKey("accessKey"));
@@ -73,6 +73,8 @@ public class KameletsCatalogTest {
assertTrue(props.getProperties().containsKey("greedy"));
assertTrue(props.getProperties().containsKey("maxMessagesPerPoll"));
assertTrue(props.getProperties().containsKey("waitTimeSeconds"));
+ assertTrue(props.getProperties().containsKey("messageAttributeNames"));
+ assertTrue(props.getProperties().containsKey("attributeNames"));
assertTrue(props.getProperties().containsKey("visibilityTimeout"));
}
diff --git
a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
index 10e988857..274acd112 100644
--- a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
+++ b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
@@ -305,7 +305,7 @@
"supportLevel" : "Stable",
"title" : "AWS SQS Source",
"summary" : "Receives the messages of an Amazon SQS queue, one message
each, and by default deletes each message from the queue once it has been
processed.",
- "description" : "Receives the messages of an Amazon SQS queue, one message
each, and by default deletes each message from the queue once it has been
processed.\n\nUse it to process work that other systems put in a queue, such as
orders, jobs or notifications, or the events that SNS, S3 or EventBridge send
to a queue. For the events of an S3 bucket, `aws-s3-event-based-source` can
also fetch the files.\n\nOutput: the body is the body of the SQS message, as
text. The headers give the m [...]
+ "description" : "Receives the messages of an Amazon SQS queue, one message
each, and by default deletes each message from the queue once it has been
processed.\n\nUse it to process work that other systems put in a queue, such as
orders, jobs or notifications, or the events that SNS, S3 or EventBridge send
to a queue. For the events of an S3 bucket, `aws-s3-event-based-source` can
also fetch the files.\n\nOutput: the body is the body of the SQS message, as
text. The headers give the m [...]
"group" : "AWS SQS",
"namespace" : "AWS",
"provider" : "Apache Software Foundation"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/aws-sqs-source.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/aws-sqs-source.kamelet.yaml
index b60483842..a2d4f3bc5 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/aws-sqs-source.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/aws-sqs-source.kamelet.yaml
@@ -37,7 +37,7 @@ spec:
Use it to process work that other systems put in a queue, such as
orders, jobs or notifications, or the events that SNS, S3 or EventBridge send
to a queue. For the events of an S3 bucket, `aws-s3-event-based-source` can
also fetch the files.
- Output: the body is the body of the SQS message, as text. The headers
give the message ID (`CamelAwsSqsMessageId`), the receipt handle and the MD5
checksum of the body. The message attributes of the SQS message are not read.
+ Output: the body is the body of the SQS message, as text. The headers
give the message ID (`CamelAwsSqsMessageId`), the receipt handle and the MD5
checksum of the body. The message attributes, which the sender sets on a
message, are read only when `messageAttributeNames` names them, and the system
attributes of SQS, such as `SentTimestamp`, only when `attributeNames` names
them. Each attribute that is read becomes a header of the same name, except a
message attribute whose name sta [...]
Good to know: a message is deleted only after it has been processed
without error; a message that fails stays in the queue and is received again
when its visibility timeout ends. With `deleteAfterRead` set to false, the
messages stay in the queue and are received again and again. Without
`queueURL`, the queue is looked up by listing the queues, which needs the
`sqs:ListQueues` permission.
required:
@@ -162,6 +162,16 @@ spec:
example: 30
type: integer
minimum: "0"
+ messageAttributeNames:
+ title: Message Attribute Names
+ description: "The message attributes to read: `All`, or a
comma-separated list of names, such as `customer,priority`. A name that ends in
`.*`, such as `trace.*`, reads the attributes whose names start with `trace.`.
Each attribute that is read becomes a header of the same name: a String
attribute as text, a Number attribute as text, or as a number when its type has
a label such as `Number.int`, and a Binary attribute as bytes (an AWS SDK
`SdkBytes`). A message attribute whose na [...]
+ example: All
+ type: string
+ attributeNames:
+ title: System Attribute Names
+ description: "Which of the system attributes that SQS keeps for each
message to read: `All`, or a comma-separated list of names from `SenderId`,
`SentTimestamp`, `ApproximateReceiveCount`, `ApproximateFirstReceiveTimestamp`,
`SequenceNumber`, `MessageDeduplicationId`, `MessageGroupId`, `AWSTraceHeader`
and `DeadLetterQueueSourceArn`. A name that is not one of these is ignored,
with a warning in the log. Each attribute that is read becomes a header of the
same name, as text; the t [...]
+ example: SentTimestamp,ApproximateReceiveCount
+ type: string
dataTypes:
out:
default: text
@@ -181,11 +191,11 @@ spec:
type: string
CamelAwsSqsAttributes:
title: System Attributes
- description: A Map of the system attributes of the message, such as
`SentTimestamp`. The Kamelet asks for none, so it is empty.
+ description: A Map of the system attributes that `attributeNames`
asks for, such as `SentTimestamp`, with their values as text. Its keys are AWS
SDK `MessageSystemAttributeName` values, not text, so read one attribute from
its own header instead. Empty when `attributeNames` is not set.
type: object
CamelAwsSqsMessageAttributes:
title: Message Attributes
- description: A Map of the message attributes of the message. The
Kamelet asks for none, so it is empty.
+ description: A Map of the message attributes that
`messageAttributeNames` asks for, from the name of each attribute to an AWS SDK
`MessageAttributeValue`, which holds its data type and its value. It also holds
the attributes whose names start with `Camel`. Empty when
`messageAttributeNames` is not set.
type: object
types:
text:
@@ -227,11 +237,13 @@ spec:
queueNameOrArn: "{{queueNameOrArn}}"
accessKey: "RAW({{?accessKey}})"
amazonAWSHost: "{{?amazonAWSHost}}"
+ attributeNames: "{{?attributeNames}}"
autoCreateQueue: "{{autoCreateQueue}}"
delay: "{{delay}}"
deleteAfterRead: "{{deleteAfterRead}}"
greedy: "{{greedy}}"
maxMessagesPerPoll: "{{maxMessagesPerPoll}}"
+ messageAttributeNames: "{{?messageAttributeNames}}"
overrideEndpoint: "{{overrideEndpoint}}"
profileCredentialsName: "{{?profileCredentialsName}}"
protocol: "{{?protocol}}"