This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
The following commit(s) were added to refs/heads/main by this push:
new 94299aaf5 chore: the Azure Blob Kamelets log in with a SAS token, the
source filters and moves blobs, and the event-based source reads from a topic
(#3170) (#3213)
94299aaf5 is described below
commit 94299aaf5412f1dde460a7f939fa3da75973a7e2
Author: Claus Ibsen <[email protected]>
AuthorDate: Wed Nov 18 22:30:37 2026 +0100
chore: the Azure Blob Kamelets log in with a SAS token, the source filters
and moves blobs, and the event-based source reads from a topic (#3170) (#3213)
azure-storage-blob-source, azure-storage-blob-sink and
azure-storage-blob-append-sink take a sasToken, used with the new
credentialType AZURE_SAS. The token is a secret (format: password,
credentials group), goes into the URI inside RAW(), and has a pattern
that accepts a token with or without the leading '?' and rejects a
whole SAS URL or an access key. The source also passes it to its
deleteBlob step.
azure-storage-blob-source takes prefix and regex, to read only some of
the blobs, and moveAfterRead with destinationContainer, which moves each
blob to another container of the account once it has been processed.
When moveAfterRead and deleteAfterRead are both true, the template no
longer runs its own delete step, so the blob is moved: before, the step
deleted the blob first and the move then failed.
azure-storage-blob-event-based-source takes serviceBusType, to read the
events from a subscription of a topic (subscriptionName was there but
unused), and maxConcurrentCalls.
All the new options are optional placeholders with no default: with
them unset, the endpoint URIs are the same as before.
Left out: maxResultsPerPage, as the component lists every page on each
poll, so it does not limit what a poll reads; blobType and
createPageBlob on the sinks, as the sink uploads a block blob and the
append sink an append blob whatever blobType says, and a page blob
upload needs a PageRange object in a header.
Co-authored-by: Claude Opus 5.5 <[email protected]>
---
...azure-storage-blob-append-sink-description.adoc | 9 +++++
...torage-blob-event-based-source-description.adoc | 30 ++++++++++++++-
.../azure-storage-blob-sink-description.adoc | 9 +++++
.../azure-storage-blob-source-description.adoc | 39 +++++++++++++++++++
.../azure-storage-blob-append-sink.kamelet.yaml | 14 +++++--
...re-storage-blob-event-based-source.kamelet.yaml | 25 +++++++++---
kamelets/azure-storage-blob-sink.kamelet.yaml | 14 +++++--
kamelets/azure-storage-blob-source.kamelet.yaml | 44 +++++++++++++++++++---
.../main/resources/META-INF/kamelets-index.json | 8 ++--
.../azure-storage-blob-append-sink.kamelet.yaml | 14 +++++--
...re-storage-blob-event-based-source.kamelet.yaml | 25 +++++++++---
.../kamelets/azure-storage-blob-sink.kamelet.yaml | 14 +++++--
.../azure-storage-blob-source.kamelet.yaml | 44 +++++++++++++++++++---
13 files changed, 250 insertions(+), 39 deletions(-)
diff --git
a/docs/modules/ROOT/partials/azure-storage-blob-append-sink-description.adoc
b/docs/modules/ROOT/partials/azure-storage-blob-append-sink-description.adoc
index f08d4121d..5031407b6 100644
--- a/docs/modules/ROOT/partials/azure-storage-blob-append-sink-description.adoc
+++ b/docs/modules/ROOT/partials/azure-storage-blob-append-sink-description.adoc
@@ -6,6 +6,7 @@ In this Kamelet, you can use these Azure authentication methods:
- Azure Identity mechanism: `AZURE_IDENTITY`
- Plain Shared Account Key: `SHARED_ACCOUNT_KEY`
+- Shared access signature (SAS) token: `AZURE_SAS`, with the token in
`sasToken`
The order of evaluation for `AZURE_IDENTITY` is the following:
@@ -21,6 +22,14 @@ To log in as an app registration (a service principal), set
`credentialType` to
For more information, see the
https://learn.microsoft.com/en-us/java/api/overview/azure/identity-readme[Azure
Identity documentation]
+=== Log in with a SAS Token
+
+A shared access signature (SAS) token gives access to the storage account, or
to one container, with the permissions
+and the expiry chosen when it was made, without handing out an access key.
Create one in the Azure portal: the storage
+account, then *Security + networking*, then *Shared access signature*, or the
container, then *Shared access tokens*.
+Set **credentialType** to `AZURE_SAS` and **sasToken** to the token, without
the URL in front of it; a leading `?` is
+allowed. The append sink needs the Read and Write permissions. A token that
lacks a permission, or has expired, fails with status 403.
+
=== Optional Headers
In the headers, you can set the `file` / `ce-file` property to specify the
filename to upload. If you do not set it, the blob gets a generated unique name.
diff --git
a/docs/modules/ROOT/partials/azure-storage-blob-event-based-source-description.adoc
b/docs/modules/ROOT/partials/azure-storage-blob-event-based-source-description.adoc
index a00b3a5de..e9acbd665 100644
---
a/docs/modules/ROOT/partials/azure-storage-blob-event-based-source-description.adoc
+++
b/docs/modules/ROOT/partials/azure-storage-blob-event-based-source-description.adoc
@@ -4,7 +4,7 @@
For this Kamelet the Connection string is the basic method for authenticating
to the Azure Servicebus Queue.
-To use this Kamelet you'll need to set up Events on your Azure Storage Blob
account and select as an endpoint an Azure Servicebus Queue, with the Event
Grid event schema. The Kamelet reads from a queue, not from a topic.
+To use this Kamelet you'll need to set up Events on your Azure Storage Blob
account and select as an endpoint an Azure Servicebus Queue or Topic, with the
Event Grid event schema.
For the Azure Storage Blob access, you can use these Azure authentication
methods:
@@ -15,3 +15,31 @@ For the Azure Storage Blob access, you can use these Azure
authentication method
To log in as an app registration (a service principal), set `credentialType`
to `AZURE_IDENTITY` and provide `clientId`, `clientSecret`, and `tenantId`:
when all three are set, they are used instead of the identity of the
environment. `CLIENT_SECRET` does not use these three properties.
For more information, see the
https://learn.microsoft.com/en-us/java/api/overview/azure/identity-readme[Azure
Identity documentation]
+
+=== Example: Read the Events from a Topic
+
+When the events go to a Service Bus topic, so that other applications can
receive them from subscriptions of their
+own, set **serviceBusType** to `topic`, **topicOrQueueName** to the name of
the topic, and **subscriptionName** to the
+subscription this Kamelet reads from:
+
+[source,yaml,subs='+attributes,macros']
+----
+- route:
+ from:
+ uri: "kamelet:azure-storage-blob-event-based-source"
+ parameters:
+ connectionString:
"Endpoint=sb://my-namespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=..."
+ serviceBusType: "topic"
+ topicOrQueueName: "blob-events"
+ subscriptionName: "invoice-processor"
+ maxConcurrentCalls: 4
+ accountName: "mystorageaccount"
+ containerName: "invoices"
+ accessKey: "..."
+ getBlob: true
+ steps:
+ - to: "log:blob"
+----
+
+Without **subscriptionName** the Kamelet does not start.
**maxConcurrentCalls** processes
+up to four events at the same time; when it is not set, events are processed
one at a time, in order.
diff --git
a/docs/modules/ROOT/partials/azure-storage-blob-sink-description.adoc
b/docs/modules/ROOT/partials/azure-storage-blob-sink-description.adoc
index 8fd79a8ad..2e25e855b 100644
--- a/docs/modules/ROOT/partials/azure-storage-blob-sink-description.adoc
+++ b/docs/modules/ROOT/partials/azure-storage-blob-sink-description.adoc
@@ -6,6 +6,7 @@ In this Kamelet, you can use these Azure authentication methods:
- Azure Identity mechanism: `AZURE_IDENTITY`
- Plain Shared Account Key: `SHARED_ACCOUNT_KEY`
+- Shared access signature (SAS) token: `AZURE_SAS`, with the token in
`sasToken`
The order of evaluation for `AZURE_IDENTITY` is the following:
@@ -21,6 +22,14 @@ To log in as an app registration (a service principal), set
`credentialType` to
For more information, see the
https://learn.microsoft.com/en-us/java/api/overview/azure/identity-readme[Azure
Identity documentation]
+=== Log in with a SAS Token
+
+A shared access signature (SAS) token gives access to the storage account, or
to one container, with the permissions
+and the expiry chosen when it was made, without handing out an access key.
Create one in the Azure portal: the storage
+account, then *Security + networking*, then *Shared access signature*, or the
container, then *Shared access tokens*.
+Set **credentialType** to `AZURE_SAS` and **sasToken** to the token, without
the URL in front of it; a leading `?` is
+allowed. The sink needs the Write permission. A token that lacks a permission,
or has expired, fails with status 403.
+
=== Optional Headers
In the headers, you can set the `file` / `ce-file` property to specify the
filename to upload. If you do not set it, the blob gets a generated unique name.
diff --git
a/docs/modules/ROOT/partials/azure-storage-blob-source-description.adoc
b/docs/modules/ROOT/partials/azure-storage-blob-source-description.adoc
index 25f6b86f0..f63545507 100644
--- a/docs/modules/ROOT/partials/azure-storage-blob-source-description.adoc
+++ b/docs/modules/ROOT/partials/azure-storage-blob-source-description.adoc
@@ -6,6 +6,7 @@ In this Kamelet, you can use these Azure authentication methods:
- Azure Identity mechanism: `AZURE_IDENTITY`
- Plain Shared Account Key: `SHARED_ACCOUNT_KEY`
+- Shared access signature (SAS) token: `AZURE_SAS`, with the token in
`sasToken`
The order of evaluation for `AZURE_IDENTITY` is the following:
@@ -20,3 +21,41 @@ The order of evaluation for `AZURE_IDENTITY` is the
following:
To log in as an app registration (a service principal), set `credentialType`
to `AZURE_IDENTITY` and provide `clientId`, `clientSecret`, and `tenantId`:
when all three are set, they are used instead of the order above.
`CLIENT_SECRET` does not use these three properties.
For more information, see the
https://learn.microsoft.com/en-us/java/api/overview/azure/identity-readme[Azure
Identity documentation]
+
+=== Log in with a SAS Token
+
+A shared access signature (SAS) token gives access to the storage account, or
to one container, with the permissions
+and the expiry chosen when it was made, without handing out an access key.
Create one in the Azure portal: the storage
+account, then *Security + networking*, then *Shared access signature*, or the
container, then *Shared access tokens*.
+Set **credentialType** to `AZURE_SAS` and **sasToken** to the token, without
the URL in front of it; a leading `?` is
+allowed. The source needs the List and Read permissions, and also Delete with
**deleteAfterRead**, or Write and Delete with **moveAfterRead**. A token that
lacks a permission, or has expired, fails with status 403.
+
+=== Example: Read Some of the Blobs and Move Them Once Read
+
+By default the source reads every blob of the container on every poll. Use
**prefix** or **regex** to read only some of
+them, and **moveAfterRead** to move each blob to another container once it has
been processed:
+
+[source,yaml,subs='+attributes,macros']
+----
+- route:
+ from:
+ uri: "kamelet:azure-storage-blob-source"
+ parameters:
+ accountName: "mystorageaccount"
+ containerName: "uploads"
+ credentialType: "AZURE_SAS"
+ sasToken:
"sv=2022-11-02&ss=b&srt=sco&sp=rwdl&se=2026-12-31T00:00:00Z&spr=https&sig=..."
+ prefix: "in/"
+ moveAfterRead: true
+ destinationContainer: "processed"
+ steps:
+ - to: "log:blob?showHeaders=true"
+----
+
+The source reads only the blobs whose name starts with `in/`, such as
`in/orders.csv` and `in/2026/report.csv`. Once a
+blob has been processed, it is copied to the container `processed` under the
same name, `in/orders.csv`, and deleted
+from `uploads`. The container `processed` must exist: when the copy fails, the
blob stays in `uploads` and is read again
+on the next poll.
+
+To select blobs by a pattern instead, set **regex** to a Java regular
expression that matches the whole name, such as
+`.*\.csv` for the CSV files in any folder. When **regex** is set, **prefix**
is ignored.
diff --git a/kamelets/azure-storage-blob-append-sink.kamelet.yaml
b/kamelets/azure-storage-blob-append-sink.kamelet.yaml
index 6fec07d6d..a5495ed79 100644
--- a/kamelets/azure-storage-blob-append-sink.kamelet.yaml
+++ b/kamelets/azure-storage-blob-append-sink.kamelet.yaml
@@ -86,12 +86,20 @@ spec:
format: password
x-descriptors:
- urn:camel:group:credentials
+ sasToken:
+ title: SAS Token
+ description: "A shared access signature (SAS) token for the storage
account or for the container, from the Azure portal: the storage account, then
Security + networking, then Shared access signature, or the container, then
Shared access tokens. Used when `credentialType` is `AZURE_SAS`. Give the token
only, such as
`sv=2022-11-02&ss=b&srt=sco&sp=rw&se=2026-12-31T00:00:00Z&spr=https&sig=...`,
with or without the leading `?`, not the whole SAS URL. It needs the Read and
Write permi [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^\\??[A-Za-z]+=[^&]*(&[A-Za-z]+=[^&]*)*$"
credentialType:
title: Credential Type
- description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_IDENTITY` uses
`clientId`, `clientSecret` and `tenantId` when all three are set, and otherwise
the Azure identity of the environment, such as the `AZURE_CLIENT_ID`,
`AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET` environment variables, a managed
identity or the Azure CLI login. Do not use `CLIENT_SECRET`: it does not use
the three client options."
+ description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_SAS` uses
`sasToken`. `AZURE_IDENTITY` uses `clientId`, `clientSecret` and `tenantId`
when all three are set, and otherwise the Azure identity of the environment,
such as the `AZURE_CLIENT_ID`, `AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET`
environment variables, a managed identity or the Azure CLI login. Do not use
`CLIENT_SECRET`: it does not use the three client options."
type: string
default: "SHARED_ACCOUNT_KEY"
- enum: ["SHARED_ACCOUNT_KEY", "AZURE_IDENTITY", "CLIENT_SECRET"]
+ enum: ["SHARED_ACCOUNT_KEY", "AZURE_SAS", "AZURE_IDENTITY",
"CLIENT_SECRET"]
dataTypes:
in:
headers:
@@ -138,4 +146,4 @@ spec:
simple:
expression: "${exchangeId}"
- to:
- uri:
"azure-storage-blob://{{accountName}}/{{containerName}}?accessKey=RAW({{?accessKey}})&azureClientId=RAW({{?clientId}})&azureClientSecret=RAW({{?clientSecret}})&azureTenantId={{?tenantId}}&blobType=appendBlob&credentialType={{credentialType}}&operation=commitAppendBlob"
+ uri:
"azure-storage-blob://{{accountName}}/{{containerName}}?accessKey=RAW({{?accessKey}})&azureClientId=RAW({{?clientId}})&azureClientSecret=RAW({{?clientSecret}})&azureTenantId={{?tenantId}}&blobType=appendBlob&credentialType={{credentialType}}&operation=commitAppendBlob&sasToken=RAW({{?sasToken}})"
diff --git a/kamelets/azure-storage-blob-event-based-source.kamelet.yaml
b/kamelets/azure-storage-blob-event-based-source.kamelet.yaml
index fa3c9444a..235337b64 100644
--- a/kamelets/azure-storage-blob-event-based-source.kamelet.yaml
+++ b/kamelets/azure-storage-blob-event-based-source.kamelet.yaml
@@ -32,13 +32,13 @@ spec:
definition:
title: "Azure Storage Blob Event-based Source"
description: |-
- Receives the Event Grid events of an Azure Storage account from a
Service Bus queue and can read the content of each new blob in a container.
+ Receives the Event Grid events of an Azure Storage account from a
Service Bus queue or topic, and can read the content of each new blob in a
container.
- Use it to react quickly to new blobs, such as uploaded files, without
reading the container over and over. It needs an Event Grid subscription on the
storage account that sends the events, in the Event Grid schema, to a Service
Bus queue. To read the container on a timer instead, use
`azure-storage-blob-source`.
+ Use it to react quickly to new blobs, such as uploaded files, without
reading the container over and over. It needs an Event Grid subscription on the
storage account that sends the events, in the Event Grid schema, to a Service
Bus queue, or to a topic so that other applications can receive the same events
from subscriptions of their own. To read the container on a timer instead, use
`azure-storage-blob-source`.
Output: when `getBlob` is false, the default, the body is the Event Grid
event as JSON (an Azure `BinaryData`, which converts to text or bytes). When
`getBlob` is true, the body of a `Microsoft.Storage.BlobCreated` event for a
blob of `containerName` is the content of that blob, as a stream of bytes; the
other events are passed on as parsed JSON (a Jackson JSON node).
- Good to know: with the default `serviceBusReceiveMode` of
`RECEIVE_AND_DELETE`, an event is removed from the queue as soon as it is
received, so an event whose processing fails is lost. With `getBlob`, only
blobs at the top of the container can be read: an event for a blob in a virtual
folder, such as `in/a.txt`, fails. It reads from a queue, not from a topic.
+ Good to know: with the default `serviceBusReceiveMode` of
`RECEIVE_AND_DELETE`, an event is removed from the queue, or from the
subscription of the topic, as soon as it is received, so an event whose
processing fails is lost. With `getBlob`, only blobs at the top of the
container can be read: an event for a blob in a virtual folder, such as
`in/a.txt`, fails. Events are processed one at a time, in order, unless
`maxConcurrentCalls` is set.
required:
- topicOrQueueName
- connectionString
@@ -47,8 +47,8 @@ spec:
type: object
properties:
topicOrQueueName:
- title: Queue Name
- description: The name of the Service Bus queue that the Event Grid
subscription of the storage account sends its events to.
+ title: Queue or Topic Name
+ description: The name of the Service Bus queue, or of the topic when
`serviceBusType` is `topic`, that the Event Grid subscription of the storage
account sends its events to.
type: string
example: blob-events
connectionString:
@@ -66,8 +66,19 @@ spec:
enum: ["RECEIVE_AND_DELETE", "PEEK_LOCK"]
subscriptionName:
title: Subscription Name
- description: "The name of a subscription of a Service Bus topic. Not
used: this Kamelet reads from a queue."
+ description: The name of the subscription of the topic to read the
events from. Required when `serviceBusType` is `topic`, without it the Kamelet
does not start. Not used with a queue.
type: string
+ serviceBusType:
+ title: Service Bus Type
+ description: "`queue` reads the events from the queue
`topicOrQueueName`. `topic` reads them from the subscription `subscriptionName`
of the topic `topicOrQueueName`. When it is not set, it reads from a queue."
+ type: string
+ enum: ["queue", "topic"]
+ maxConcurrentCalls:
+ title: Max Concurrent Events
+ description: How many events are processed at the same time. When it
is not set, one at a time, in order. With a higher number, events are processed
in parallel and can finish out of order.
+ type: integer
+ minimum: 1
+ example: 4
accountName:
title: Account Name
description: The name of the storage account, as shown in the Azure
portal under Storage accounts. It is the first part of the address of the
account, `https://<accountName>.blob.core.windows.net`.
@@ -136,6 +147,8 @@ spec:
connectionString: "RAW({{connectionString}})"
serviceBusReceiveMode: "{{serviceBusReceiveMode}}"
subscriptionName: "{{?subscriptionName}}"
+ serviceBusType: "{{?serviceBusType}}"
+ maxConcurrentCalls: "{{?maxConcurrentCalls}}"
steps:
- choice:
precondition: true
diff --git a/kamelets/azure-storage-blob-sink.kamelet.yaml
b/kamelets/azure-storage-blob-sink.kamelet.yaml
index ae370a09c..af6572759 100644
--- a/kamelets/azure-storage-blob-sink.kamelet.yaml
+++ b/kamelets/azure-storage-blob-sink.kamelet.yaml
@@ -86,12 +86,20 @@ spec:
format: password
x-descriptors:
- urn:camel:group:credentials
+ sasToken:
+ title: SAS Token
+ description: "A shared access signature (SAS) token for the storage
account or for the container, from the Azure portal: the storage account, then
Security + networking, then Shared access signature, or the container, then
Shared access tokens. Used when `credentialType` is `AZURE_SAS`. Give the token
only, such as
`sv=2022-11-02&ss=b&srt=sco&sp=w&se=2026-12-31T00:00:00Z&spr=https&sig=...`,
with or without the leading `?`, not the whole SAS URL. It needs the Write
permission."
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^\\??[A-Za-z]+=[^&]*(&[A-Za-z]+=[^&]*)*$"
credentialType:
title: Credential Type
- description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_IDENTITY` uses
`clientId`, `clientSecret` and `tenantId` when all three are set, and otherwise
the Azure identity of the environment, such as the `AZURE_CLIENT_ID`,
`AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET` environment variables, a managed
identity or the Azure CLI login. Do not use `CLIENT_SECRET`: it does not use
the three client options."
+ description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_SAS` uses
`sasToken`. `AZURE_IDENTITY` uses `clientId`, `clientSecret` and `tenantId`
when all three are set, and otherwise the Azure identity of the environment,
such as the `AZURE_CLIENT_ID`, `AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET`
environment variables, a managed identity or the Azure CLI login. Do not use
`CLIENT_SECRET`: it does not use the three client options."
type: string
default: "SHARED_ACCOUNT_KEY"
- enum: ["SHARED_ACCOUNT_KEY", "AZURE_IDENTITY", "CLIENT_SECRET"]
+ enum: ["SHARED_ACCOUNT_KEY", "AZURE_SAS", "AZURE_IDENTITY",
"CLIENT_SECRET"]
dataTypes:
in:
headers:
@@ -138,4 +146,4 @@ spec:
simple:
expression: "${exchangeId}"
- to:
- uri:
"azure-storage-blob://{{accountName}}/{{containerName}}?accessKey=RAW({{?accessKey}})&azureClientId=RAW({{?clientId}})&azureClientSecret=RAW({{?clientSecret}})&azureTenantId={{?tenantId}}&credentialType={{credentialType}}&operation=uploadBlockBlob"
+ uri:
"azure-storage-blob://{{accountName}}/{{containerName}}?accessKey=RAW({{?accessKey}})&azureClientId=RAW({{?clientId}})&azureClientSecret=RAW({{?clientSecret}})&azureTenantId={{?tenantId}}&credentialType={{credentialType}}&operation=uploadBlockBlob&sasToken=RAW({{?sasToken}})"
diff --git a/kamelets/azure-storage-blob-source.kamelet.yaml
b/kamelets/azure-storage-blob-source.kamelet.yaml
index 4698d4414..93bb1aca5 100644
--- a/kamelets/azure-storage-blob-source.kamelet.yaml
+++ b/kamelets/azure-storage-blob-source.kamelet.yaml
@@ -32,13 +32,13 @@ spec:
definition:
title: "Azure Storage Blob Source"
description: |-
- Reads the files (blobs) in a container of Azure Blob Storage, one
message per blob, and can delete each blob once it has been read.
+ Reads the files (blobs) in a container of Azure Blob Storage, one
message per blob, and can delete each blob, or move it to another container,
once it has been read.
Use it to process files that another system drops into a container, such
as uploaded documents, exported reports or log files. To react to new blobs
through Event Grid events instead of reading the container over and over, use
`azure-storage-blob-event-based-source`.
Output: the body is the content of the blob, as a stream of bytes. The
headers give the name of the blob (`CamelAzureStorageBlobBlobName`, with its
folder path), its type, content type, size, ETag and last-modified time.
- Good to know: blobs are kept after they are read, so the same blobs are
read again on every poll, every 500 milliseconds by default; set
`deleteAfterRead` to true to delete each blob once it has been processed. It
reads all the blobs of the container, those in virtual folders too. With
`deleteAfterRead`, the blob deleted is the one named in the
`CamelAzureStorageBlobBlobName` header when processing ends, so do not change
that header.
+ Good to know: blobs are kept after they are read, so the same blobs are
read again on every poll, every 500 milliseconds by default; set
`deleteAfterRead` to true to delete each blob once it has been processed, or
`moveAfterRead` to move it to another container. It reads all the blobs of the
container, those in virtual folders too, unless `prefix` or `regex` narrows
them down. With `deleteAfterRead`, the blob deleted is the one named in the
`CamelAzureStorageBlobBlobName` header wh [...]
required:
- accountName
- containerName
@@ -84,6 +84,14 @@ spec:
format: password
x-descriptors:
- urn:camel:group:credentials
+ sasToken:
+ title: SAS Token
+ description: "A shared access signature (SAS) token for the storage
account or for the container, from the Azure portal: the storage account, then
Security + networking, then Shared access signature, or the container, then
Shared access tokens. Used when `credentialType` is `AZURE_SAS`. Give the token
only, such as
`sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-12-31T00:00:00Z&spr=https&sig=...`,
with or without the leading `?`, not the whole SAS URL. It needs the List and
Read permis [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^\\??[A-Za-z]+=[^&]*(&[A-Za-z]+=[^&]*)*$"
delay:
title: Delay
description: How long to wait between two polls of the container, in
milliseconds.
@@ -94,12 +102,32 @@ spec:
description: If true, each blob is deleted from the container once it
has been processed. When false, the blobs stay and are read again on every poll.
type: boolean
default: false
+ prefix:
+ title: Blob Name Prefix
+ description: Reads only the blobs whose name starts with this text,
such as `in/` for the blobs in the virtual folder `in` and its subfolders, or
`invoice-`. Ignored when `regex` is set. When neither is set, all the blobs of
the container are read.
+ type: string
+ example: "in/"
+ regex:
+ title: Blob Name Pattern
+ description: "Reads only the blobs whose whole name, with its virtual
folder path, matches this Java regular expression, such as `.*\\.csv` for the
CSV files in any folder, or `in/[^/]*\\.csv` for those directly in the folder
`in`. It must match the whole name: `csv` alone matches no blob. When it is
set, `prefix` is ignored, and the names of all the blobs of the container are
listed on each poll and then filtered."
+ type: string
+ example: ".*\\.csv"
+ moveAfterRead:
+ title: Move Blob After Read
+ description: If true, each blob is moved to `destinationContainer`
once it has been processed. It is copied there under the same name, with its
virtual folder path, and then deleted from `containerName`. If the copy fails,
for example because `destinationContainer` does not exist, the blob is kept and
read again on the next poll, and a warning is logged. When both `moveAfterRead`
and `deleteAfterRead` are true, the blob is moved.
+ type: boolean
+ destinationContainer:
+ title: Destination Container
+ description: The name of the container, in the same storage account,
to move the blobs to when `moveAfterRead` is true, such as `processed`. It must
exist. A container name has 3 to 63 lowercase letters, digits and hyphens.
Required with `moveAfterRead`, without it the Kamelet does not start.
+ type: string
+ pattern: "^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$"
+ example: processed
credentialType:
title: Credential Type
- description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_IDENTITY` uses
`clientId`, `clientSecret` and `tenantId` when all three are set, and otherwise
the Azure identity of the environment, such as the `AZURE_CLIENT_ID`,
`AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET` environment variables, a managed
identity or the Azure CLI login. Do not use `CLIENT_SECRET`: it does not use
the three client options."
+ description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_SAS` uses
`sasToken`. `AZURE_IDENTITY` uses `clientId`, `clientSecret` and `tenantId`
when all three are set, and otherwise the Azure identity of the environment,
such as the `AZURE_CLIENT_ID`, `AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET`
environment variables, a managed identity or the Azure CLI login. Do not use
`CLIENT_SECRET`: it does not use the three client options."
type: string
default: "SHARED_ACCOUNT_KEY"
- enum: ["SHARED_ACCOUNT_KEY", "AZURE_IDENTITY", "CLIENT_SECRET"]
+ enum: ["SHARED_ACCOUNT_KEY", "AZURE_SAS", "AZURE_IDENTITY",
"CLIENT_SECRET"]
dataTypes:
out:
default: binary
@@ -182,7 +210,12 @@ spec:
azureClientSecret: "RAW({{?clientSecret}})"
azureTenantId: "{{?tenantId}}"
credentialType: "{{credentialType}}"
+ sasToken: "RAW({{?sasToken}})"
delay: "{{delay}}"
+ prefix: "RAW({{?prefix}})"
+ regex: "RAW({{?regex}})"
+ moveAfterRead: "{{?moveAfterRead}}"
+ destinationContainer: "{{?destinationContainer}}"
steps:
- to:
uri: kamelet:sink
@@ -191,7 +224,7 @@ spec:
when:
- expression:
simple:
- expression: "${properties:deleteAfterRead:false}"
+ expression: "${properties:deleteAfterRead:false} == true
&& ${properties:moveAfterRead:false} != true"
steps:
- to:
uri: azure-storage-blob
@@ -203,5 +236,6 @@ spec:
azureClientSecret: "RAW({{?clientSecret}})"
azureTenantId: "{{?tenantId}}"
credentialType: "{{credentialType}}"
+ sasToken: "RAW({{?sasToken}})"
operation: deleteBlob
diff --git
a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
index 10cfd9b03..bcbe83874 100644
--- a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
+++ b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
@@ -414,8 +414,8 @@
"type" : "source",
"supportLevel" : "Preview",
"title" : "Azure Storage Blob Event-based Source",
- "summary" : "Receives the Event Grid events of an Azure Storage account
from a Service Bus queue and can read the content of each new blob in a
container.",
- "description" : "Receives the Event Grid events of an Azure Storage
account from a Service Bus queue and can read the content of each new blob in a
container.\n\nUse it to react quickly to new blobs, such as uploaded files,
without reading the container over and over. It needs an Event Grid
subscription on the storage account that sends the events, in the Event Grid
schema, to a Service Bus queue. To read the container on a timer instead, use
`azure-storage-blob-source`.\n\nOutput: w [...]
+ "summary" : "Receives the Event Grid events of an Azure Storage account
from a Service Bus queue or topic, and can read the content of each new blob in
a container.",
+ "description" : "Receives the Event Grid events of an Azure Storage
account from a Service Bus queue or topic, and can read the content of each new
blob in a container.\n\nUse it to react quickly to new blobs, such as uploaded
files, without reading the container over and over. It needs an Event Grid
subscription on the storage account that sends the events, in the Event Grid
schema, to a Service Bus queue, or to a topic so that other applications can
receive the same events from sub [...]
"group" : "Azure Storage Blob Event Based",
"namespace" : "Azure",
"provider" : "Apache Software Foundation"
@@ -434,8 +434,8 @@
"type" : "source",
"supportLevel" : "Stable",
"title" : "Azure Storage Blob Source",
- "summary" : "Reads the files (blobs) in a container of Azure Blob Storage,
one message per blob, and can delete each blob once it has been read.",
- "description" : "Reads the files (blobs) in a container of Azure Blob
Storage, one message per blob, and can delete each blob once it has been
read.\n\nUse it to process files that another system drops into a container,
such as uploaded documents, exported reports or log files. To react to new
blobs through Event Grid events instead of reading the container over and over,
use `azure-storage-blob-event-based-source`.\n\nOutput: the body is the content
of the blob, as a stream of bytes [...]
+ "summary" : "Reads the files (blobs) in a container of Azure Blob Storage,
one message per blob, and can delete each blob, or move it to another
container, once it has been read.",
+ "description" : "Reads the files (blobs) in a container of Azure Blob
Storage, one message per blob, and can delete each blob, or move it to another
container, once it has been read.\n\nUse it to process files that another
system drops into a container, such as uploaded documents, exported reports or
log files. To react to new blobs through Event Grid events instead of reading
the container over and over, use
`azure-storage-blob-event-based-source`.\n\nOutput: the body is the content
[...]
"group" : "Azure Storage Blob",
"namespace" : "Azure",
"provider" : "Apache Software Foundation"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-append-sink.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-append-sink.kamelet.yaml
index 6fec07d6d..a5495ed79 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-append-sink.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-append-sink.kamelet.yaml
@@ -86,12 +86,20 @@ spec:
format: password
x-descriptors:
- urn:camel:group:credentials
+ sasToken:
+ title: SAS Token
+ description: "A shared access signature (SAS) token for the storage
account or for the container, from the Azure portal: the storage account, then
Security + networking, then Shared access signature, or the container, then
Shared access tokens. Used when `credentialType` is `AZURE_SAS`. Give the token
only, such as
`sv=2022-11-02&ss=b&srt=sco&sp=rw&se=2026-12-31T00:00:00Z&spr=https&sig=...`,
with or without the leading `?`, not the whole SAS URL. It needs the Read and
Write permi [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^\\??[A-Za-z]+=[^&]*(&[A-Za-z]+=[^&]*)*$"
credentialType:
title: Credential Type
- description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_IDENTITY` uses
`clientId`, `clientSecret` and `tenantId` when all three are set, and otherwise
the Azure identity of the environment, such as the `AZURE_CLIENT_ID`,
`AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET` environment variables, a managed
identity or the Azure CLI login. Do not use `CLIENT_SECRET`: it does not use
the three client options."
+ description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_SAS` uses
`sasToken`. `AZURE_IDENTITY` uses `clientId`, `clientSecret` and `tenantId`
when all three are set, and otherwise the Azure identity of the environment,
such as the `AZURE_CLIENT_ID`, `AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET`
environment variables, a managed identity or the Azure CLI login. Do not use
`CLIENT_SECRET`: it does not use the three client options."
type: string
default: "SHARED_ACCOUNT_KEY"
- enum: ["SHARED_ACCOUNT_KEY", "AZURE_IDENTITY", "CLIENT_SECRET"]
+ enum: ["SHARED_ACCOUNT_KEY", "AZURE_SAS", "AZURE_IDENTITY",
"CLIENT_SECRET"]
dataTypes:
in:
headers:
@@ -138,4 +146,4 @@ spec:
simple:
expression: "${exchangeId}"
- to:
- uri:
"azure-storage-blob://{{accountName}}/{{containerName}}?accessKey=RAW({{?accessKey}})&azureClientId=RAW({{?clientId}})&azureClientSecret=RAW({{?clientSecret}})&azureTenantId={{?tenantId}}&blobType=appendBlob&credentialType={{credentialType}}&operation=commitAppendBlob"
+ uri:
"azure-storage-blob://{{accountName}}/{{containerName}}?accessKey=RAW({{?accessKey}})&azureClientId=RAW({{?clientId}})&azureClientSecret=RAW({{?clientSecret}})&azureTenantId={{?tenantId}}&blobType=appendBlob&credentialType={{credentialType}}&operation=commitAppendBlob&sasToken=RAW({{?sasToken}})"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-event-based-source.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-event-based-source.kamelet.yaml
index fa3c9444a..235337b64 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-event-based-source.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-event-based-source.kamelet.yaml
@@ -32,13 +32,13 @@ spec:
definition:
title: "Azure Storage Blob Event-based Source"
description: |-
- Receives the Event Grid events of an Azure Storage account from a
Service Bus queue and can read the content of each new blob in a container.
+ Receives the Event Grid events of an Azure Storage account from a
Service Bus queue or topic, and can read the content of each new blob in a
container.
- Use it to react quickly to new blobs, such as uploaded files, without
reading the container over and over. It needs an Event Grid subscription on the
storage account that sends the events, in the Event Grid schema, to a Service
Bus queue. To read the container on a timer instead, use
`azure-storage-blob-source`.
+ Use it to react quickly to new blobs, such as uploaded files, without
reading the container over and over. It needs an Event Grid subscription on the
storage account that sends the events, in the Event Grid schema, to a Service
Bus queue, or to a topic so that other applications can receive the same events
from subscriptions of their own. To read the container on a timer instead, use
`azure-storage-blob-source`.
Output: when `getBlob` is false, the default, the body is the Event Grid
event as JSON (an Azure `BinaryData`, which converts to text or bytes). When
`getBlob` is true, the body of a `Microsoft.Storage.BlobCreated` event for a
blob of `containerName` is the content of that blob, as a stream of bytes; the
other events are passed on as parsed JSON (a Jackson JSON node).
- Good to know: with the default `serviceBusReceiveMode` of
`RECEIVE_AND_DELETE`, an event is removed from the queue as soon as it is
received, so an event whose processing fails is lost. With `getBlob`, only
blobs at the top of the container can be read: an event for a blob in a virtual
folder, such as `in/a.txt`, fails. It reads from a queue, not from a topic.
+ Good to know: with the default `serviceBusReceiveMode` of
`RECEIVE_AND_DELETE`, an event is removed from the queue, or from the
subscription of the topic, as soon as it is received, so an event whose
processing fails is lost. With `getBlob`, only blobs at the top of the
container can be read: an event for a blob in a virtual folder, such as
`in/a.txt`, fails. Events are processed one at a time, in order, unless
`maxConcurrentCalls` is set.
required:
- topicOrQueueName
- connectionString
@@ -47,8 +47,8 @@ spec:
type: object
properties:
topicOrQueueName:
- title: Queue Name
- description: The name of the Service Bus queue that the Event Grid
subscription of the storage account sends its events to.
+ title: Queue or Topic Name
+ description: The name of the Service Bus queue, or of the topic when
`serviceBusType` is `topic`, that the Event Grid subscription of the storage
account sends its events to.
type: string
example: blob-events
connectionString:
@@ -66,8 +66,19 @@ spec:
enum: ["RECEIVE_AND_DELETE", "PEEK_LOCK"]
subscriptionName:
title: Subscription Name
- description: "The name of a subscription of a Service Bus topic. Not
used: this Kamelet reads from a queue."
+ description: The name of the subscription of the topic to read the
events from. Required when `serviceBusType` is `topic`, without it the Kamelet
does not start. Not used with a queue.
type: string
+ serviceBusType:
+ title: Service Bus Type
+ description: "`queue` reads the events from the queue
`topicOrQueueName`. `topic` reads them from the subscription `subscriptionName`
of the topic `topicOrQueueName`. When it is not set, it reads from a queue."
+ type: string
+ enum: ["queue", "topic"]
+ maxConcurrentCalls:
+ title: Max Concurrent Events
+ description: How many events are processed at the same time. When it
is not set, one at a time, in order. With a higher number, events are processed
in parallel and can finish out of order.
+ type: integer
+ minimum: 1
+ example: 4
accountName:
title: Account Name
description: The name of the storage account, as shown in the Azure
portal under Storage accounts. It is the first part of the address of the
account, `https://<accountName>.blob.core.windows.net`.
@@ -136,6 +147,8 @@ spec:
connectionString: "RAW({{connectionString}})"
serviceBusReceiveMode: "{{serviceBusReceiveMode}}"
subscriptionName: "{{?subscriptionName}}"
+ serviceBusType: "{{?serviceBusType}}"
+ maxConcurrentCalls: "{{?maxConcurrentCalls}}"
steps:
- choice:
precondition: true
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-sink.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-sink.kamelet.yaml
index ae370a09c..af6572759 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-sink.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-sink.kamelet.yaml
@@ -86,12 +86,20 @@ spec:
format: password
x-descriptors:
- urn:camel:group:credentials
+ sasToken:
+ title: SAS Token
+ description: "A shared access signature (SAS) token for the storage
account or for the container, from the Azure portal: the storage account, then
Security + networking, then Shared access signature, or the container, then
Shared access tokens. Used when `credentialType` is `AZURE_SAS`. Give the token
only, such as
`sv=2022-11-02&ss=b&srt=sco&sp=w&se=2026-12-31T00:00:00Z&spr=https&sig=...`,
with or without the leading `?`, not the whole SAS URL. It needs the Write
permission."
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^\\??[A-Za-z]+=[^&]*(&[A-Za-z]+=[^&]*)*$"
credentialType:
title: Credential Type
- description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_IDENTITY` uses
`clientId`, `clientSecret` and `tenantId` when all three are set, and otherwise
the Azure identity of the environment, such as the `AZURE_CLIENT_ID`,
`AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET` environment variables, a managed
identity or the Azure CLI login. Do not use `CLIENT_SECRET`: it does not use
the three client options."
+ description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_SAS` uses
`sasToken`. `AZURE_IDENTITY` uses `clientId`, `clientSecret` and `tenantId`
when all three are set, and otherwise the Azure identity of the environment,
such as the `AZURE_CLIENT_ID`, `AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET`
environment variables, a managed identity or the Azure CLI login. Do not use
`CLIENT_SECRET`: it does not use the three client options."
type: string
default: "SHARED_ACCOUNT_KEY"
- enum: ["SHARED_ACCOUNT_KEY", "AZURE_IDENTITY", "CLIENT_SECRET"]
+ enum: ["SHARED_ACCOUNT_KEY", "AZURE_SAS", "AZURE_IDENTITY",
"CLIENT_SECRET"]
dataTypes:
in:
headers:
@@ -138,4 +146,4 @@ spec:
simple:
expression: "${exchangeId}"
- to:
- uri:
"azure-storage-blob://{{accountName}}/{{containerName}}?accessKey=RAW({{?accessKey}})&azureClientId=RAW({{?clientId}})&azureClientSecret=RAW({{?clientSecret}})&azureTenantId={{?tenantId}}&credentialType={{credentialType}}&operation=uploadBlockBlob"
+ uri:
"azure-storage-blob://{{accountName}}/{{containerName}}?accessKey=RAW({{?accessKey}})&azureClientId=RAW({{?clientId}})&azureClientSecret=RAW({{?clientSecret}})&azureTenantId={{?tenantId}}&credentialType={{credentialType}}&operation=uploadBlockBlob&sasToken=RAW({{?sasToken}})"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-source.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-source.kamelet.yaml
index 4698d4414..93bb1aca5 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-source.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/azure-storage-blob-source.kamelet.yaml
@@ -32,13 +32,13 @@ spec:
definition:
title: "Azure Storage Blob Source"
description: |-
- Reads the files (blobs) in a container of Azure Blob Storage, one
message per blob, and can delete each blob once it has been read.
+ Reads the files (blobs) in a container of Azure Blob Storage, one
message per blob, and can delete each blob, or move it to another container,
once it has been read.
Use it to process files that another system drops into a container, such
as uploaded documents, exported reports or log files. To react to new blobs
through Event Grid events instead of reading the container over and over, use
`azure-storage-blob-event-based-source`.
Output: the body is the content of the blob, as a stream of bytes. The
headers give the name of the blob (`CamelAzureStorageBlobBlobName`, with its
folder path), its type, content type, size, ETag and last-modified time.
- Good to know: blobs are kept after they are read, so the same blobs are
read again on every poll, every 500 milliseconds by default; set
`deleteAfterRead` to true to delete each blob once it has been processed. It
reads all the blobs of the container, those in virtual folders too. With
`deleteAfterRead`, the blob deleted is the one named in the
`CamelAzureStorageBlobBlobName` header when processing ends, so do not change
that header.
+ Good to know: blobs are kept after they are read, so the same blobs are
read again on every poll, every 500 milliseconds by default; set
`deleteAfterRead` to true to delete each blob once it has been processed, or
`moveAfterRead` to move it to another container. It reads all the blobs of the
container, those in virtual folders too, unless `prefix` or `regex` narrows
them down. With `deleteAfterRead`, the blob deleted is the one named in the
`CamelAzureStorageBlobBlobName` header wh [...]
required:
- accountName
- containerName
@@ -84,6 +84,14 @@ spec:
format: password
x-descriptors:
- urn:camel:group:credentials
+ sasToken:
+ title: SAS Token
+ description: "A shared access signature (SAS) token for the storage
account or for the container, from the Azure portal: the storage account, then
Security + networking, then Shared access signature, or the container, then
Shared access tokens. Used when `credentialType` is `AZURE_SAS`. Give the token
only, such as
`sv=2022-11-02&ss=b&srt=sco&sp=rl&se=2026-12-31T00:00:00Z&spr=https&sig=...`,
with or without the leading `?`, not the whole SAS URL. It needs the List and
Read permis [...]
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ pattern: "^\\??[A-Za-z]+=[^&]*(&[A-Za-z]+=[^&]*)*$"
delay:
title: Delay
description: How long to wait between two polls of the container, in
milliseconds.
@@ -94,12 +102,32 @@ spec:
description: If true, each blob is deleted from the container once it
has been processed. When false, the blobs stay and are read again on every poll.
type: boolean
default: false
+ prefix:
+ title: Blob Name Prefix
+ description: Reads only the blobs whose name starts with this text,
such as `in/` for the blobs in the virtual folder `in` and its subfolders, or
`invoice-`. Ignored when `regex` is set. When neither is set, all the blobs of
the container are read.
+ type: string
+ example: "in/"
+ regex:
+ title: Blob Name Pattern
+ description: "Reads only the blobs whose whole name, with its virtual
folder path, matches this Java regular expression, such as `.*\\.csv` for the
CSV files in any folder, or `in/[^/]*\\.csv` for those directly in the folder
`in`. It must match the whole name: `csv` alone matches no blob. When it is
set, `prefix` is ignored, and the names of all the blobs of the container are
listed on each poll and then filtered."
+ type: string
+ example: ".*\\.csv"
+ moveAfterRead:
+ title: Move Blob After Read
+ description: If true, each blob is moved to `destinationContainer`
once it has been processed. It is copied there under the same name, with its
virtual folder path, and then deleted from `containerName`. If the copy fails,
for example because `destinationContainer` does not exist, the blob is kept and
read again on the next poll, and a warning is logged. When both `moveAfterRead`
and `deleteAfterRead` are true, the blob is moved.
+ type: boolean
+ destinationContainer:
+ title: Destination Container
+ description: The name of the container, in the same storage account,
to move the blobs to when `moveAfterRead` is true, such as `processed`. It must
exist. A container name has 3 to 63 lowercase letters, digits and hyphens.
Required with `moveAfterRead`, without it the Kamelet does not start.
+ type: string
+ pattern: "^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$"
+ example: processed
credentialType:
title: Credential Type
- description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_IDENTITY` uses
`clientId`, `clientSecret` and `tenantId` when all three are set, and otherwise
the Azure identity of the environment, such as the `AZURE_CLIENT_ID`,
`AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET` environment variables, a managed
identity or the Azure CLI login. Do not use `CLIENT_SECRET`: it does not use
the three client options."
+ description: "How to log in to the storage account.
`SHARED_ACCOUNT_KEY`, the default, uses `accessKey`. `AZURE_SAS` uses
`sasToken`. `AZURE_IDENTITY` uses `clientId`, `clientSecret` and `tenantId`
when all three are set, and otherwise the Azure identity of the environment,
such as the `AZURE_CLIENT_ID`, `AZURE_TENANT_ID` and `AZURE_CLIENT_SECRET`
environment variables, a managed identity or the Azure CLI login. Do not use
`CLIENT_SECRET`: it does not use the three client options."
type: string
default: "SHARED_ACCOUNT_KEY"
- enum: ["SHARED_ACCOUNT_KEY", "AZURE_IDENTITY", "CLIENT_SECRET"]
+ enum: ["SHARED_ACCOUNT_KEY", "AZURE_SAS", "AZURE_IDENTITY",
"CLIENT_SECRET"]
dataTypes:
out:
default: binary
@@ -182,7 +210,12 @@ spec:
azureClientSecret: "RAW({{?clientSecret}})"
azureTenantId: "{{?tenantId}}"
credentialType: "{{credentialType}}"
+ sasToken: "RAW({{?sasToken}})"
delay: "{{delay}}"
+ prefix: "RAW({{?prefix}})"
+ regex: "RAW({{?regex}})"
+ moveAfterRead: "{{?moveAfterRead}}"
+ destinationContainer: "{{?destinationContainer}}"
steps:
- to:
uri: kamelet:sink
@@ -191,7 +224,7 @@ spec:
when:
- expression:
simple:
- expression: "${properties:deleteAfterRead:false}"
+ expression: "${properties:deleteAfterRead:false} == true
&& ${properties:moveAfterRead:false} != true"
steps:
- to:
uri: azure-storage-blob
@@ -203,5 +236,6 @@ spec:
azureClientSecret: "RAW({{?clientSecret}})"
azureTenantId: "{{?tenantId}}"
credentialType: "{{credentialType}}"
+ sasToken: "RAW({{?sasToken}})"
operation: deleteBlob