This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-jbang-examples.git
The following commit(s) were added to refs/heads/main by this push:
new d020ff4 Added IBM COS Examples (#47)
d020ff4 is described below
commit d020ff4cd52326a00e5eb8a0e60d242c886777aa
Author: Andrea Cosentino <[email protected]>
AuthorDate: Fri Oct 31 11:13:34 2025 +0100
Added IBM COS Examples (#47)
Signed-off-by: Andrea Cosentino <[email protected]>
---
ibm/ibm-cos/README.adoc | 270 ++++++++++++++++++++++++++++++++
ibm/ibm-cos/application.properties | 39 +++++
ibm/ibm-cos/ibm-cos-sink.kamelet.yaml | 134 ++++++++++++++++
ibm/ibm-cos/ibm-cos-source.kamelet.yaml | 177 +++++++++++++++++++++
ibm/ibm-cos/ibm-cos-to-log.camel.yaml | 39 +++++
ibm/ibm-cos/kafka-to-ibm-cos.camel.yaml | 42 +++++
6 files changed, 701 insertions(+)
diff --git a/ibm/ibm-cos/README.adoc b/ibm/ibm-cos/README.adoc
new file mode 100644
index 0000000..eb03f12
--- /dev/null
+++ b/ibm/ibm-cos/README.adoc
@@ -0,0 +1,270 @@
+= IBM Cloud Object Storage Examples
+
+This directory contains examples demonstrating the IBM COS Kamelets for Apache
Camel.
+
+== Available Examples
+
+=== 1. Kafka to IBM COS Sink (kafka-to-ibm-cos.camel.yaml)
+
+This example uses the IBM COS Sink Kamelet to upload data to IBM Cloud Object
Storage.
+
+Messages consumed from a Kafka topic are uploaded to an IBM COS bucket.
+
+=== 2. IBM COS Source to Log (ibm-cos-to-log.camel.yaml)
+
+This example uses the IBM COS Source Kamelet to consume data from IBM Cloud
Object Storage.
+
+The example polls an IBM COS bucket for new objects and logs their content and
metadata.
+
+== Prerequisites
+
+You must have:
+- A valid IBM Cloud account and an IBM Cloud Object Storage service instance
+- A running Kafka broker (for the Kafka to IBM COS sink example)
+
+== Setup Kafka (for Kafka to IBM COS example)
+
+You need a running Kafka broker. The easiest way is to use Camel JBang's
built-in infrastructure support.
+
+=== Start Kafka with JBang
+
+Start a local Kafka broker using Camel JBang:
+
+[source,shell]
+----
+$ camel infra run kafka
+----
+
+This command will start a local Kafka broker on port 9092 using Docker Compose
in the background.
+
+The Kafka broker will automatically create topics on first use, so you don't
need to manually create the `ibm-cos-topic` topic.
+
+== Setup IBM Cloud Object Storage
+
+=== Create an IBM Cloud Object Storage Instance
+
+1. Log in to the https://cloud.ibm.com/[IBM Cloud Console]
+2. Navigate to the
https://cloud.ibm.com/catalog/services/cloud-object-storage[IBM Cloud Object
Storage] service in the catalog
+3. Create a new instance with a name of your choice
+4. Once created, note the **Service Instance ID (CRN)**
+
+=== Create a Bucket
+
+1. In your COS instance, go to "Buckets" and click "Create bucket"
+2. Choose a bucket name (e.g., `my-camel-bucket`)
+3. Select a resiliency (Regional, Cross-region, or Single Data Center)
+4. Select a location (e.g., `us-south`)
+5. Click "Create bucket"
+
+=== Upload Test Files (for IBM COS Source example)
+
+If you want to test the IBM COS Source to Log example, upload some test files
to your bucket:
+
+1. Navigate to your bucket in the IBM Cloud console
+2. Click "Upload" and select one or more files
+3. Alternatively, use the IBM Cloud CLI:
+
+[source,shell]
+----
+ibmcloud cos upload --bucket my-camel-bucket --key test-file.txt --file
/path/to/local/file.txt
+----
+
+=== Create API Credentials
+
+1. In your COS instance, navigate to "Service credentials"
+2. Click "New credential"
+3. Provide a name for the credential
+4. Ensure the role includes at least "Writer" permissions
+5. Click "Add"
+6. View the created credentials and note the **apikey** value
+
+=== Get the Endpoint URL
+
+IBM COS provides different endpoint URLs based on the bucket's location and
access type (public, private, or direct).
+
+For example:
+- Public endpoint (US South):
`https://s3.us-south.cloud-object-storage.appdomain.cloud`
+- Public endpoint (EU GB):
`https://s3.eu-gb.cloud-object-storage.appdomain.cloud`
+
+For a complete list of endpoints, see the
https://cloud.ibm.com/docs/cloud-object-storage?topic=cloud-object-storage-endpoints[IBM
COS Endpoints documentation].
+
+== Configuration
+
+Update the `application.properties` file with your Kafka and IBM Cloud Object
Storage credentials:
+
+[source,properties]
+----
+# Kafka Configuration
+kafka.topic=ibm-cos-topic
+kafka.bootstrapServers=localhost:9092
+kafka.consumerGroup=kafka-ibm-cos-group
+kafka.autoOffsetReset=earliest
+
+# IBM Cloud Object Storage Configuration
+ibm.bucketName=my-camel-bucket
+ibm.apiKey=<your-ibm-cloud-api-key>
+ibm.serviceInstanceId=<your-service-instance-id>
+ibm.endpointUrl=https://s3.us-south.cloud-object-storage.appdomain.cloud
+ibm.location=us-south
+ibm.autoCreateBucket=false
+----
+
+Replace:
+- `<your-ibm-cloud-api-key>` with your IBM Cloud API key
+- `<your-service-instance-id>` with your COS Service Instance ID (CRN)
+- Update the endpoint URL and location to match your bucket's region
+
+Kafka Configuration:
+- `kafka.topic`: The Kafka topic to consume messages from
+- `kafka.bootstrapServers`: Comma-separated list of Kafka broker URLs
+- `kafka.consumerGroup`: Consumer group ID for the Kafka consumer
+- `kafka.autoOffsetReset`: What to do when there is no initial offset
(earliest or latest)
+
+IBM COS Source Configuration (for ibm-cos-to-log.camel.yaml):
+- `ibm.deleteAfterRead`: Delete objects after consuming them (default: true)
+- `ibm.moveAfterRead`: Move objects to another bucket instead of deleting
(default: false)
+- `ibm.prefix`: Only consume objects with keys starting with this prefix
(optional)
+- `ibm.delay`: Delay in milliseconds between polling (default: 5000)
+- `ibm.maxMessagesPerPoll`: Maximum number of objects to consume per poll
(default: 10)
+
+== Quick Start
+
+Here's a quick walkthrough to run the Kafka to IBM COS example:
+
+1. Start Kafka infrastructure:
++
+[source,shell]
+----
+$ camel infra run kafka
+----
+
+2. Run the Camel integration:
++
+[source,shell]
+----
+$ jbang -Dcamel.jbang.version=4.16.0-SNAPSHOT camel@apache/camel run
--properties=application.properties kafka-to-ibm-cos.camel.yaml
ibm-cos-sink.kamelet.yaml
+----
+
+3. In another terminal, send test messages:
++
+[source,shell]
+----
+$ docker exec -it camel-infra-kafka-kafka-1
/opt/kafka/bin/kafka-console-producer.sh \
+ --topic ibm-cos-topic \
+ --bootstrap-server localhost:9092
+----
++
+Then type your messages and press Enter after each one.
+
+4. When done, stop Kafka:
++
+[source,shell]
+----
+$ camel infra stop kafka
+----
+
+=== Running the IBM COS Source to Log Example
+
+You can run the source example using:
+
+[source,shell]
+----
+$ jbang -Dcamel.jbang.version=4.16.0-SNAPSHOT camel@apache/camel run
--properties=application.properties ibm-cos-to-log.camel.yaml
ibm-cos-source.kamelet.yaml
+----
+
+=== Running Both Examples
+
+Note: Running both examples simultaneously (using `camel run *`) is not
recommended as they will interfere with each other (source consuming what sink
produces). Run them separately or configure them to use different buckets.
+
+== Developer Web Console
+
+You can enable the developer console via `--console` flag as shown:
+
+[source,shell]
+----
+$ camel run * --console --deps=camel:kamelet
+----
+
+Then you can browse: http://localhost:8080/q/dev to introspect the running
Camel application.
+
+== What happens
+
+=== Kafka to IBM COS Sink Example
+
+The Kafka source consumes messages from the configured Kafka topic.
+
+Each message is uploaded to your IBM COS bucket with a filename like
`kafka-20241029-143025-123.txt` (timestamp-based with milliseconds).
+
+You should see output similar to:
+
+[source,shell]
+----
+2024-10-29 14:30:25.123 INFO 12345 --- [kafka-consumer] route1 : Uploading
to IBM COS: kafka-20241029-143025-123.txt with body: Message from Kafka
+2024-10-29 14:30:26.456 INFO 12345 --- [kafka-consumer] route1 :
Successfully uploaded file to IBM COS bucket
+----
+
+You can verify the uploaded files in the IBM Cloud console by navigating to
your bucket.
+
+==== Send Test Messages to Kafka
+
+To test the example, send some messages to the Kafka topic:
+
+[source,shell]
+----
+$ docker exec -it camel-infra-kafka-kafka-1
/opt/kafka/bin/kafka-console-producer.sh \
+ --topic ibm-cos-topic \
+ --bootstrap-server localhost:9092
+----
+
+This will open an interactive console where you can type messages. Each line
you type and press Enter will be sent as a message to Kafka:
+
+[source,text]
+----
+> Hello from Kafka!
+> First message
+> Second message
+> Third message
+----
+
+Press Ctrl+C to exit the producer console.
+
+Each message will be consumed by the Camel route and uploaded to IBM COS.
+
+=== IBM COS Source to Log Example
+
+The IBM COS source polls the bucket every 5 seconds (configurable via `delay`
parameter) and retrieves up to 10 objects per poll (configurable via
`maxMessagesPerPoll`).
+
+For each object found:
+1. The object is downloaded from IBM COS
+2. Metadata headers are set (bucket name, key, content type, ETag)
+3. The object content and headers are logged
+4. By default, the object is deleted from IBM COS (configurable via
`deleteAfterRead`)
+
+You should see output similar to:
+
+[source,shell]
+----
+2024-10-29 14:30:25.123 INFO 12345 --- [ibm-cos-source] route1 : Received
file from IBM COS - Key: test-file.txt, Bucket: my-camel-bucket, ContentType:
text/plain
+2024-10-29 14:30:25.456 INFO 12345 --- [ibm-cos-source] info :
Exchange[ExchangePattern: InOnly, BodyType:
org.apache.camel.converter.stream.InputStreamCache, Body: Hello from IBM Cloud
Object Storage!]
+----
+
+== Cleanup
+
+When you're done with the example, you can stop the Kafka infrastructure:
+
+[source,shell]
+----
+$ camel infra stop kafka
+----
+
+This will stop and remove the Kafka broker.
+
+== Help and contributions
+
+If you hit any problem using Camel or have some feedback, then please
+https://camel.apache.org/community/support/[let us know].
+
+We also love contributors, so
+https://camel.apache.org/community/contributing/[get involved] :-)
+
+The Camel riders!
diff --git a/ibm/ibm-cos/application.properties
b/ibm/ibm-cos/application.properties
new file mode 100644
index 0000000..8d13fd9
--- /dev/null
+++ b/ibm/ibm-cos/application.properties
@@ -0,0 +1,39 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Kafka Configuration
+kafka.topic=ibm-cos-topic
+kafka.bootstrapServers=localhost:9092
+kafka.consumerGroup=kafka-ibm-cos-group
+kafka.autoOffsetReset=earliest
+
+# IBM Cloud Object Storage Configuration
+ibm.bucketName=my-camel-bucket
+ibm.apiKey=AjcgPGjQMXXgzjlNPpBtQs7O_Va1Ytl-pEaDjKwtth0b
+ibm.serviceInstanceId=crn:v1:bluemix:public:iam-identity::a/f062e6fa73d148d3b562f07b940b723f::serviceid:ServiceId-8c2e9e55-5bd6-4430-8811-4e290b69cebd
+ibm.endpointUrl=https://s3.eu-de.cloud-object-storage.appdomain.cloud
+ibm.location=eu-de
+
+# Sink Configuration
+ibm.autoCreateBucket=false
+
+# Source Configuration
+ibm.deleteAfterRead=true
+ibm.moveAfterRead=false
+ibm.prefix=
+ibm.delay=5000
+ibm.maxMessagesPerPoll=10
diff --git a/ibm/ibm-cos/ibm-cos-sink.kamelet.yaml
b/ibm/ibm-cos/ibm-cos-sink.kamelet.yaml
new file mode 100644
index 0000000..a5a30f2
--- /dev/null
+++ b/ibm/ibm-cos/ibm-cos-sink.kamelet.yaml
@@ -0,0 +1,134 @@
+# ---------------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ---------------------------------------------------------------------------
+
+apiVersion: camel.apache.org/v1
+kind: Kamelet
+metadata:
+ name: ibm-cos-sink
+ annotations:
+ camel.apache.org/kamelet.support.level: "Preview"
+ camel.apache.org/catalog.version: "4.16.0-SNAPSHOT"
+ camel.apache.org/kamelet.icon:
"data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjUwMCIgaGVpZ2h0PSIyNTAwIiB2aWV3Qm94PSIwIDAgMjU2IDI1NiIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiBwcmVzZXJ2ZUFzcGVjdFJhdGlvPSJ4TWlkWU1pZCI+PHBhdGggZD0iTTAgMGgyNTZ2MjU2SDB6IiBmaWxsPSIjMDUzMzYyIi8+PHBhdGggZD0iTTE4Ny43MzIgMTI1LjkxOGMtLjAzNi0xLjM1Ny0uMDktMi43MTQtLjE4LTQuMDcyLS4wMzYtLjUyMi0uMDkxLTEuMDQ1LS4xMjgtMS41NjctLjA1NC0uNjEzLS4xMDgtMS4yMjYtLjE4LTEuODM5LS4wNzItLjc3LS4xOC0xLjU0LS4yODgtMi4zMWEzOS43NjYgMzkuNz
[...]
+ camel.apache.org/provider: "Apache Software Foundation"
+ camel.apache.org/kamelet.group: "IBM Cloud Object Storage"
+ camel.apache.org/kamelet.namespace: "IBM"
+ labels:
+ camel.apache.org/kamelet.type: "sink"
+spec:
+ definition:
+ title: "IBM Cloud Object Storage Sink"
+ description: Upload data to an IBM Cloud Object Storage Bucket.
+ required:
+ - bucketName
+ - apiKey
+ - serviceInstanceId
+ - endpointUrl
+ type: object
+ properties:
+ bucketName:
+ title: Bucket Name
+ description: The IBM COS Bucket name.
+ type: string
+ apiKey:
+ title: API Key
+ description: IBM Cloud API Key for authentication.
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ serviceInstanceId:
+ title: Service Instance ID
+ description: IBM COS Service Instance ID (CRN).
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ endpointUrl:
+ title: Endpoint URL
+ description: IBM COS Endpoint URL (e.g.,
https://s3.us-south.cloud-object-storage.appdomain.cloud).
+ type: string
+ example: "https://s3.us-south.cloud-object-storage.appdomain.cloud";
+ location:
+ title: Location
+ description: IBM COS Location/Region (e.g., us-south, eu-gb).
+ type: string
+ example: "us-south"
+ autoCreateBucket:
+ title: Autocreate Bucket
+ description: Specifies to automatically create the IBM COS bucket.
+ type: boolean
+ default: false
+ keyName:
+ title: Key Name
+ description: The key name for saving an element in the bucket.
+ type: string
+ multiPartUpload:
+ title: Multi-Part Upload
+ description: Use multi-part upload for large files.
+ type: boolean
+ default: false
+ partSize:
+ title: Part Size
+ description: Part size for multi-part uploads (default 25MB).
+ type: integer
+ default: 26214400
+ storageClass:
+ title: Storage Class
+ description: The storage class to use when storing objects (e.g.,
STANDARD, VAULT, COLD, FLEX).
+ type: string
+ dependencies:
+ - "camel:core"
+ - "camel:ibm-cos"
+ - "camel:kamelet"
+ template:
+ from:
+ uri: "kamelet:source"
+ steps:
+ - choice:
+ precondition: true
+ when:
+ - simple: '${propertiesExist:!keyName}'
+ steps:
+ - choice:
+ when:
+ - simple: "${header[file]}"
+ steps:
+ - setHeader:
+ name: CamelIbmCosKey
+ simple: "${header[file]}"
+ - simple: "${header[ce-file]}"
+ steps:
+ - setHeader:
+ name: CamelIbmCosKey
+ simple: "${header[ce-file]}"
+ otherwise:
+ steps:
+ - setHeader:
+ name: CamelIbmCosKey
+ simple: "${exchangeId}"
+ - to:
+ uri: "ibm-cos:{{bucketName}}"
+ parameters:
+ apiKey: "{{apiKey}}"
+ serviceInstanceId: "{{serviceInstanceId}}"
+ endpointUrl: "{{endpointUrl}}"
+ location: "{{?location}}"
+ autoCreateBucket: "{{autoCreateBucket}}"
+ keyName: "{{?keyName}}"
+ multiPartUpload: "{{multiPartUpload}}"
+ partSize: "{{partSize}}"
+ storageClass: "{{?storageClass}}"
diff --git a/ibm/ibm-cos/ibm-cos-source.kamelet.yaml
b/ibm/ibm-cos/ibm-cos-source.kamelet.yaml
new file mode 100644
index 0000000..24e24c6
--- /dev/null
+++ b/ibm/ibm-cos/ibm-cos-source.kamelet.yaml
@@ -0,0 +1,177 @@
+# ---------------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ---------------------------------------------------------------------------
+
+apiVersion: camel.apache.org/v1
+kind: Kamelet
+metadata:
+ name: ibm-cos-source
+ annotations:
+ camel.apache.org/kamelet.support.level: "Preview"
+ camel.apache.org/catalog.version: "4.16.0-SNAPSHOT"
+ camel.apache.org/kamelet.icon:
"data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjUwMCIgaGVpZ2h0PSIyNTAwIiB2aWV3Qm94PSIwIDAgMjU2IDI1NiIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiBwcmVzZXJ2ZUFzcGVjdFJhdGlvPSJ4TWlkWU1pZCI+PHBhdGggZD0iTTAgMGgyNTZ2MjU2SDB6IiBmaWxsPSIjMDUzMzYyIi8+PHBhdGggZD0iTTE4Ny43MzIgMTI1LjkxOGMtLjAzNi0xLjM1Ny0uMDktMi43MTQtLjE4LTQuMDcyLS4wMzYtLjUyMi0uMDkxLTEuMDQ1LS4xMjgtMS41NjctLjA1NC0uNjEzLS4xMDgtMS4yMjYtLjE4LTEuODM5LS4wNzItLjc3LS4xOC0xLjU0LS4yODgtMi4zMWEzOS43NjYgMzkuNz
[...]
+ camel.apache.org/provider: "Apache Software Foundation"
+ camel.apache.org/kamelet.group: "IBM Cloud Object Storage"
+ camel.apache.org/kamelet.namespace: "IBM"
+ labels:
+ camel.apache.org/kamelet.type: "source"
+spec:
+ definition:
+ title: "IBM Cloud Object Storage Source"
+ description: Receive data from an IBM Cloud Object Storage Bucket.
+ required:
+ - bucketName
+ - apiKey
+ - serviceInstanceId
+ - endpointUrl
+ type: object
+ properties:
+ bucketName:
+ title: Bucket Name
+ description: The IBM COS Bucket name.
+ type: string
+ apiKey:
+ title: API Key
+ description: IBM Cloud API Key for authentication.
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ serviceInstanceId:
+ title: Service Instance ID
+ description: IBM COS Service Instance ID (CRN).
+ type: string
+ format: password
+ x-descriptors:
+ - urn:camel:group:credentials
+ endpointUrl:
+ title: Endpoint URL
+ description: IBM COS Endpoint URL (e.g.,
https://s3.us-south.cloud-object-storage.appdomain.cloud).
+ type: string
+ example: "https://s3.us-south.cloud-object-storage.appdomain.cloud";
+ location:
+ title: Location
+ description: IBM COS Location/Region (e.g., us-south, eu-gb).
+ type: string
+ example: "us-south"
+ deleteAfterRead:
+ title: Auto-delete Objects
+ description: Specifies to delete objects after consuming them.
+ type: boolean
+ default: true
+ moveAfterRead:
+ title: Move Objects After Read
+ description: Move objects from IBM COS bucket to a different bucket
after they have been retrieved.
+ type: boolean
+ default: false
+ destinationBucket:
+ title: Destination Bucket
+ description: Define the destination bucket where an object must be
moved when moveAfterRead is set to true.
+ type: string
+ destinationBucketPrefix:
+ title: Destination Bucket Prefix
+ description: Define the destination bucket prefix to use when an
object must be moved, and moveAfterRead is set to true.
+ type: string
+ destinationBucketSuffix:
+ title: Destination Bucket Suffix
+ description: Define the destination bucket suffix to use when an
object must be moved, and moveAfterRead is set to true.
+ type: string
+ autoCreateBucket:
+ title: Autocreate Bucket
+ description: Specifies to automatically create the IBM COS bucket.
+ type: boolean
+ default: false
+ prefix:
+ title: Prefix
+ description: The IBM COS bucket prefix to consider while searching.
+ type: string
+ example: 'folder/'
+ delimiter:
+ title: Delimiter
+ description: The delimiter to use for listing objects.
+ type: string
+ includeBody:
+ title: Include Body
+ description: If true, the object body is included in the exchange. If
false, only headers are set.
+ type: boolean
+ default: true
+ includeFolders:
+ title: Include Folders
+ description: Include folders/directories when listing objects.
+ type: boolean
+ default: true
+ delay:
+ title: Delay
+ description: The number of milliseconds before the next poll of the
selected bucket.
+ type: integer
+ default: 500
+ maxMessagesPerPoll:
+ title: Max Messages Per Poll
+ description: Gets the maximum number of messages as a limit to poll at
each polling. The default value is 10. Use 0 or a negative number to set it as
unlimited.
+ type: integer
+ default: 10
+ dataTypes:
+ out:
+ default: binary
+ headers:
+ CamelIbmCosBucketName:
+ title: IBM COS Bucket Name
+ description: The bucket name which has been used to retrieve objects
+ type: string
+ CamelIbmCosKey:
+ title: IBM COS Key
+ description: The key under which the retrieved object is stored.
+ type: string
+ CamelIbmCosContentType:
+ title: Content Type
+ description: The content type of the retrieved object.
+ default: application/octet-stream
+ type: string
+ CamelIbmCosETag:
+ title: ETag Value
+ description: The hex encoded 128-bit MD5 digest of the associated
object according to RFC 1864.
+ type: string
+ types:
+ binary:
+ format: "application-octet-stream"
+ description: Default binary representation of the IBM COS object
retrieved from the bucket.
+ mediaType: application/octet-stream
+ dependencies:
+ - "camel:core"
+ - "camel:ibm-cos"
+ - "camel:kamelet"
+ template:
+ from:
+ uri: "ibm-cos:{{bucketName}}"
+ parameters:
+ apiKey: "{{apiKey}}"
+ serviceInstanceId: "{{serviceInstanceId}}"
+ endpointUrl: "{{endpointUrl}}"
+ location: "{{?location}}"
+ deleteAfterRead: "{{deleteAfterRead}}"
+ moveAfterRead: "{{moveAfterRead}}"
+ destinationBucket: "{{?destinationBucket}}"
+ destinationBucketPrefix: "{{?destinationBucketPrefix}}"
+ destinationBucketSuffix: "{{?destinationBucketSuffix}}"
+ autoCreateBucket: "{{autoCreateBucket}}"
+ prefix: "{{?prefix}}"
+ delimiter: "{{?delimiter}}"
+ includeBody: "{{includeBody}}"
+ includeFolders: "{{includeFolders}}"
+ delay: "{{delay}}"
+ maxMessagesPerPoll: "{{maxMessagesPerPoll}}"
+ steps:
+ - to: "kamelet:sink"
diff --git a/ibm/ibm-cos/ibm-cos-to-log.camel.yaml
b/ibm/ibm-cos/ibm-cos-to-log.camel.yaml
new file mode 100644
index 0000000..0ebe2f6
--- /dev/null
+++ b/ibm/ibm-cos/ibm-cos-to-log.camel.yaml
@@ -0,0 +1,39 @@
+## ---------------------------------------------------------------------------
+## Licensed to the Apache Software Foundation (ASF) under one or more
+## contributor license agreements. See the NOTICE file distributed with
+## this work for additional information regarding copyright ownership.
+## The ASF licenses this file to You under the Apache License, Version 2.0
+## (the "License"); you may not use this file except in compliance with
+## the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing, software
+## distributed under the License is distributed on an "AS IS" BASIS,
+## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+## See the License for the specific language governing permissions and
+## limitations under the License.
+## ---------------------------------------------------------------------------
+
+- route:
+ from:
+ uri: "kamelet:ibm-cos-source"
+ parameters:
+ bucketName: "{{ibm.bucketName}}"
+ apiKey: "{{ibm.apiKey}}"
+ serviceInstanceId: "{{ibm.serviceInstanceId}}"
+ endpointUrl: "{{ibm.endpointUrl}}"
+ location: "{{ibm.location}}"
+ deleteAfterRead: "{{ibm.deleteAfterRead:true}}"
+ moveAfterRead: "{{ibm.moveAfterRead:false}}"
+ prefix: "{{ibm.prefix:}}"
+ delay: "{{ibm.delay:5000}}"
+ maxMessagesPerPoll: "{{ibm.maxMessagesPerPoll:10}}"
+ steps:
+ - log:
+ message: "Received file from IBM COS - Key:
${header.CamelIbmCosKey}, Bucket: ${header.CamelIbmCosBucketName}, ContentType:
${header.CamelIbmCosContentType}"
+ - to:
+ uri: "kamelet:log-sink"
+ parameters:
+ showStreams: true
+ showHeaders: true
diff --git a/ibm/ibm-cos/kafka-to-ibm-cos.camel.yaml
b/ibm/ibm-cos/kafka-to-ibm-cos.camel.yaml
new file mode 100644
index 0000000..46d0eca
--- /dev/null
+++ b/ibm/ibm-cos/kafka-to-ibm-cos.camel.yaml
@@ -0,0 +1,42 @@
+## ---------------------------------------------------------------------------
+## Licensed to the Apache Software Foundation (ASF) under one or more
+## contributor license agreements. See the NOTICE file distributed with
+## this work for additional information regarding copyright ownership.
+## The ASF licenses this file to You under the Apache License, Version 2.0
+## (the "License"); you may not use this file except in compliance with
+## the License. You may obtain a copy of the License at
+##
+## http://www.apache.org/licenses/LICENSE-2.0
+##
+## Unless required by applicable law or agreed to in writing, software
+## distributed under the License is distributed on an "AS IS" BASIS,
+## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+## See the License for the specific language governing permissions and
+## limitations under the License.
+## ---------------------------------------------------------------------------
+
+- route:
+ from:
+ uri: "kamelet:kafka-not-secured-source"
+ parameters:
+ topic: "{{kafka.topic}}"
+ bootstrapServers: "{{kafka.bootstrapServers}}"
+ consumerGroup: "{{kafka.consumerGroup:kafka-ibm-cos-group}}"
+ autoOffsetReset: "{{kafka.autoOffsetReset:earliest}}"
+ steps:
+ - setHeader:
+ name: CamelIbmCosKey
+ simple: "kafka-${date:now:yyyyMMdd-HHmmss-SSS}.txt"
+ - log:
+ message: "Uploading to IBM COS: ${header.CamelIbmCosKey} with body:
${body}"
+ - to:
+ uri: "kamelet:ibm-cos-sink"
+ parameters:
+ bucketName: "{{ibm.bucketName}}"
+ apiKey: "{{ibm.apiKey}}"
+ serviceInstanceId: "{{ibm.serviceInstanceId}}"
+ endpointUrl: "{{ibm.endpointUrl}}"
+ location: "{{ibm.location}}"
+ autoCreateBucket: "{{ibm.autoCreateBucket:false}}"
+ - log:
+ message: "Successfully uploaded file to IBM COS bucket"
