This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
The following commit(s) were added to refs/heads/main by this push:
new c949024e0 chore: ceph-source and ceph-sink take forcePathStyle (#3170)
(#3171)
c949024e0 is described below
commit c949024e0a7f5b05042732e89d9146eb8ac70e7d
Author: Claus Ibsen <[email protected]>
AuthorDate: Tue Nov 17 16:08:31 2026 +0100
chore: ceph-source and ceph-sink take forcePathStyle (#3170) (#3171)
The Ceph Kamelets always put the bucket in the host name, which works only
when the DNS of the gateway has a wildcard name for its buckets. The new
forcePathStyle option (default false, today's behaviour) puts the bucket in
the path instead, which the Ceph Object Gateway always accepts.
Co-authored-by: Claude Opus 5.5 <[email protected]>
---
kamelets/ceph-sink.kamelet.yaml | 9 +++++++--
kamelets/ceph-source.kamelet.yaml | 9 +++++++--
.../src/main/resources/META-INF/kamelets-index.json | 4 ++--
.../src/main/resources/kamelets/ceph-sink.kamelet.yaml | 9 +++++++--
.../src/main/resources/kamelets/ceph-source.kamelet.yaml | 9 +++++++--
5 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/kamelets/ceph-sink.kamelet.yaml b/kamelets/ceph-sink.kamelet.yaml
index d35597fcd..878348ca7 100644
--- a/kamelets/ceph-sink.kamelet.yaml
+++ b/kamelets/ceph-sink.kamelet.yaml
@@ -40,7 +40,7 @@ spec:
Output: the body is unchanged.
- Good to know: a file with the same key is overwritten, so with `keyName`
set and no `file` header each message replaces the previous one. The bucket is
addressed as part of the host name (such as `my-bucket.ceph.example.com`)
unless `cephUrl` uses an IP address, so the gateway must accept that style of
address.
+ Good to know: a file with the same key is overwritten, so with `keyName`
set and no `file` header each message replaces the previous one. The bucket is
addressed as part of the host name (such as `my-bucket.ceph.example.com`)
unless `cephUrl` uses an IP address; set `forcePathStyle` to true when the DNS
of the gateway has no wildcard name for its buckets.
required:
- cephUrl
- accessKey
@@ -83,6 +83,11 @@ spec:
description: The address of the Ceph Object Gateway (RGW), as a URL
with the scheme and the port.
type: string
example: "http://ceph-rgw.example.com:7480"
+ forcePathStyle:
+ title: Force Path Style
+ description: If true, the bucket name is put in the path of the
address (`http://ceph-rgw.example.com:7480/my-bucket/key`) instead of in the
host name (`http://my-bucket.ceph-rgw.example.com:7480/key`). The Ceph Object
Gateway always accepts the path style; the host name style works only when the
gateway has a wildcard DNS name for its buckets.
+ type: boolean
+ default: false
keyName:
title: Key Name
description: The key to store the files under, with its folder if any,
used when a message has no `file` or `ce-file` header. When it is not set
either, each file gets a generated unique name.
@@ -136,4 +141,4 @@ spec:
simple:
expression: "${exchangeId}"
- to:
- uri:
"aws2-s3:{{bucketName}}?accessKey=RAW({{?accessKey}})&autoCreateBucket={{autoCreateBucket}}&keyName={{?keyName}}&overrideEndpoint=true®ion={{zoneGroup}}&secretKey=RAW({{?secretKey}})&uriEndpointOverride={{?cephUrl}}"
+ uri:
"aws2-s3:{{bucketName}}?accessKey=RAW({{?accessKey}})&autoCreateBucket={{autoCreateBucket}}&keyName={{?keyName}}&forcePathStyle={{forcePathStyle}}&overrideEndpoint=true®ion={{zoneGroup}}&secretKey=RAW({{?secretKey}})&uriEndpointOverride={{?cephUrl}}"
diff --git a/kamelets/ceph-source.kamelet.yaml
b/kamelets/ceph-source.kamelet.yaml
index bfec49ff1..09984a7d1 100644
--- a/kamelets/ceph-source.kamelet.yaml
+++ b/kamelets/ceph-source.kamelet.yaml
@@ -38,7 +38,7 @@ spec:
Output: the body is the content of the file, as bytes. The headers give
the bucket name, the key (the path of the file in the bucket, `CamelAwsS3Key`),
the content type and the ETag of the file.
- Good to know: files are deleted from the bucket after they are read,
unless `deleteAfterRead` is set to false; then the same files are read again on
every poll. Files in folders are read too. The bucket is addressed as part of
the host name (such as `my-bucket.ceph.example.com`) unless `cephUrl` uses an
IP address, so the gateway must accept that style of address.
+ Good to know: files are deleted from the bucket after they are read,
unless `deleteAfterRead` is set to false; then the same files are read again on
every poll. Files in folders are read too. The bucket is addressed as part of
the host name (such as `my-bucket.ceph.example.com`) unless `cephUrl` uses an
IP address; set `forcePathStyle` to true when the DNS of the gateway has no
wildcard name for its buckets.
required:
- cephUrl
- accessKey
@@ -101,6 +101,11 @@ spec:
description: The address of the Ceph Object Gateway (RGW), as a URL
with the scheme and the port.
type: string
example: "http://ceph-rgw.example.com:7480"
+ forcePathStyle:
+ title: Force Path Style
+ description: If true, the bucket name is put in the path of the
address (`http://ceph-rgw.example.com:7480/my-bucket/key`) instead of in the
host name (`http://my-bucket.ceph-rgw.example.com:7480/key`). The Ceph Object
Gateway always accepts the path style; the host name style works only when the
gateway has a wildcard DNS name for its buckets.
+ type: boolean
+ default: false
delay:
title: Delay
description: How long to wait between two polls of the bucket, in
milliseconds.
@@ -134,7 +139,7 @@ spec:
- "camel:kamelet"
template:
from:
- uri:
"aws2-s3:{{bucketName}}?accessKey=RAW({{?accessKey}})&autoCreateBucket={{autoCreateBucket}}&delay={{delay}}&deleteAfterRead={{deleteAfterRead}}&ignoreBody={{ignoreBody}}&includeBody={{includeBody}}&overrideEndpoint=true&prefix={{?prefix}}®ion={{zoneGroup}}&secretKey=RAW({{?secretKey}})&uriEndpointOverride={{cephUrl}}"
+ uri:
"aws2-s3:{{bucketName}}?accessKey=RAW({{?accessKey}})&autoCreateBucket={{autoCreateBucket}}&delay={{delay}}&deleteAfterRead={{deleteAfterRead}}&ignoreBody={{ignoreBody}}&includeBody={{includeBody}}&forcePathStyle={{forcePathStyle}}&overrideEndpoint=true&prefix={{?prefix}}®ion={{zoneGroup}}&secretKey=RAW({{?secretKey}})&uriEndpointOverride={{cephUrl}}"
steps:
- to:
uri: kamelet:sink
diff --git
a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
index dd0f7af7e..20b3569fc 100644
--- a/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
+++ b/library/camel-kamelets/src/main/resources/META-INF/kamelets-index.json
@@ -525,7 +525,7 @@
"supportLevel" : "Stable",
"title" : "Ceph Sink",
"summary" : "Uploads each message as a file (object) to a bucket of a Ceph
Object Gateway, the S3-compatible storage of Ceph.",
- "description" : "Uploads each message as a file (object) to a bucket of a
Ceph Object Gateway, the S3-compatible storage of Ceph.\n\nUse it to store data
in Ceph, such as reports, exports, images or received messages to keep as an
archive. For MinIO use `minio-sink`, and for Amazon S3 use
`aws-s3-sink`.\n\nInput: the body is the content of the file, as text, bytes or
a stream. The key (the name of the file in the bucket) comes from the optional
`file` header, or the `ce-file` header; [...]
+ "description" : "Uploads each message as a file (object) to a bucket of a
Ceph Object Gateway, the S3-compatible storage of Ceph.\n\nUse it to store data
in Ceph, such as reports, exports, images or received messages to keep as an
archive. For MinIO use `minio-sink`, and for Amazon S3 use
`aws-s3-sink`.\n\nInput: the body is the content of the file, as text, bytes or
a stream. The key (the name of the file in the bucket) comes from the optional
`file` header, or the `ce-file` header; [...]
"group" : "Ceph",
"namespace" : "Cloud",
"provider" : "Apache Software Foundation"
@@ -535,7 +535,7 @@
"supportLevel" : "Stable",
"title" : "Ceph Source",
"summary" : "Reads the files (objects) in a bucket of a Ceph Object
Gateway, one message per file, and by default deletes each file once it has
been read.",
- "description" : "Reads the files (objects) in a bucket of a Ceph Object
Gateway, one message per file, and by default deletes each file once it has
been read.\n\nUse it to process files that another system drops into a Ceph
bucket, such as uploaded documents, exported reports or log files. To react to
the bucket notifications of Ceph instead of reading the bucket over and over,
use `ceph-event-based-source`.\n\nOutput: the body is the content of the file,
as bytes. The headers give t [...]
+ "description" : "Reads the files (objects) in a bucket of a Ceph Object
Gateway, one message per file, and by default deletes each file once it has
been read.\n\nUse it to process files that another system drops into a Ceph
bucket, such as uploaded documents, exported reports or log files. To react to
the bucket notifications of Ceph instead of reading the bucket over and over,
use `ceph-event-based-source`.\n\nOutput: the body is the content of the file,
as bytes. The headers give t [...]
"group" : "Ceph",
"namespace" : "Cloud",
"provider" : "Apache Software Foundation"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/ceph-sink.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/ceph-sink.kamelet.yaml
index d35597fcd..878348ca7 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/ceph-sink.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/ceph-sink.kamelet.yaml
@@ -40,7 +40,7 @@ spec:
Output: the body is unchanged.
- Good to know: a file with the same key is overwritten, so with `keyName`
set and no `file` header each message replaces the previous one. The bucket is
addressed as part of the host name (such as `my-bucket.ceph.example.com`)
unless `cephUrl` uses an IP address, so the gateway must accept that style of
address.
+ Good to know: a file with the same key is overwritten, so with `keyName`
set and no `file` header each message replaces the previous one. The bucket is
addressed as part of the host name (such as `my-bucket.ceph.example.com`)
unless `cephUrl` uses an IP address; set `forcePathStyle` to true when the DNS
of the gateway has no wildcard name for its buckets.
required:
- cephUrl
- accessKey
@@ -83,6 +83,11 @@ spec:
description: The address of the Ceph Object Gateway (RGW), as a URL
with the scheme and the port.
type: string
example: "http://ceph-rgw.example.com:7480"
+ forcePathStyle:
+ title: Force Path Style
+ description: If true, the bucket name is put in the path of the
address (`http://ceph-rgw.example.com:7480/my-bucket/key`) instead of in the
host name (`http://my-bucket.ceph-rgw.example.com:7480/key`). The Ceph Object
Gateway always accepts the path style; the host name style works only when the
gateway has a wildcard DNS name for its buckets.
+ type: boolean
+ default: false
keyName:
title: Key Name
description: The key to store the files under, with its folder if any,
used when a message has no `file` or `ce-file` header. When it is not set
either, each file gets a generated unique name.
@@ -136,4 +141,4 @@ spec:
simple:
expression: "${exchangeId}"
- to:
- uri:
"aws2-s3:{{bucketName}}?accessKey=RAW({{?accessKey}})&autoCreateBucket={{autoCreateBucket}}&keyName={{?keyName}}&overrideEndpoint=true®ion={{zoneGroup}}&secretKey=RAW({{?secretKey}})&uriEndpointOverride={{?cephUrl}}"
+ uri:
"aws2-s3:{{bucketName}}?accessKey=RAW({{?accessKey}})&autoCreateBucket={{autoCreateBucket}}&keyName={{?keyName}}&forcePathStyle={{forcePathStyle}}&overrideEndpoint=true®ion={{zoneGroup}}&secretKey=RAW({{?secretKey}})&uriEndpointOverride={{?cephUrl}}"
diff --git
a/library/camel-kamelets/src/main/resources/kamelets/ceph-source.kamelet.yaml
b/library/camel-kamelets/src/main/resources/kamelets/ceph-source.kamelet.yaml
index bfec49ff1..09984a7d1 100644
---
a/library/camel-kamelets/src/main/resources/kamelets/ceph-source.kamelet.yaml
+++
b/library/camel-kamelets/src/main/resources/kamelets/ceph-source.kamelet.yaml
@@ -38,7 +38,7 @@ spec:
Output: the body is the content of the file, as bytes. The headers give
the bucket name, the key (the path of the file in the bucket, `CamelAwsS3Key`),
the content type and the ETag of the file.
- Good to know: files are deleted from the bucket after they are read,
unless `deleteAfterRead` is set to false; then the same files are read again on
every poll. Files in folders are read too. The bucket is addressed as part of
the host name (such as `my-bucket.ceph.example.com`) unless `cephUrl` uses an
IP address, so the gateway must accept that style of address.
+ Good to know: files are deleted from the bucket after they are read,
unless `deleteAfterRead` is set to false; then the same files are read again on
every poll. Files in folders are read too. The bucket is addressed as part of
the host name (such as `my-bucket.ceph.example.com`) unless `cephUrl` uses an
IP address; set `forcePathStyle` to true when the DNS of the gateway has no
wildcard name for its buckets.
required:
- cephUrl
- accessKey
@@ -101,6 +101,11 @@ spec:
description: The address of the Ceph Object Gateway (RGW), as a URL
with the scheme and the port.
type: string
example: "http://ceph-rgw.example.com:7480"
+ forcePathStyle:
+ title: Force Path Style
+ description: If true, the bucket name is put in the path of the
address (`http://ceph-rgw.example.com:7480/my-bucket/key`) instead of in the
host name (`http://my-bucket.ceph-rgw.example.com:7480/key`). The Ceph Object
Gateway always accepts the path style; the host name style works only when the
gateway has a wildcard DNS name for its buckets.
+ type: boolean
+ default: false
delay:
title: Delay
description: How long to wait between two polls of the bucket, in
milliseconds.
@@ -134,7 +139,7 @@ spec:
- "camel:kamelet"
template:
from:
- uri:
"aws2-s3:{{bucketName}}?accessKey=RAW({{?accessKey}})&autoCreateBucket={{autoCreateBucket}}&delay={{delay}}&deleteAfterRead={{deleteAfterRead}}&ignoreBody={{ignoreBody}}&includeBody={{includeBody}}&overrideEndpoint=true&prefix={{?prefix}}®ion={{zoneGroup}}&secretKey=RAW({{?secretKey}})&uriEndpointOverride={{cephUrl}}"
+ uri:
"aws2-s3:{{bucketName}}?accessKey=RAW({{?accessKey}})&autoCreateBucket={{autoCreateBucket}}&delay={{delay}}&deleteAfterRead={{deleteAfterRead}}&ignoreBody={{ignoreBody}}&includeBody={{includeBody}}&forcePathStyle={{forcePathStyle}}&overrideEndpoint=true&prefix={{?prefix}}®ion={{zoneGroup}}&secretKey=RAW({{?secretKey}})&uriEndpointOverride={{cephUrl}}"
steps:
- to:
uri: kamelet:sink