This is an automated email from the ASF dual-hosted git repository. acosentino pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector-examples.git
commit 616bd3d26b7b66c98f25a0c4f3091dab09820acd Author: Andrea Cosentino <[email protected]> AuthorDate: Thu Feb 25 13:25:13 2021 +0100 AWS Examples: Moved to tar.gz --- aws-s3-to-jms/README.adoc | 8 +++--- aws2-iam/aws2-iam-sink-create-group/README.adoc | 6 ++-- aws2-iam/aws2-iam-sink-create-user/README.adoc | 6 ++-- aws2-iam/aws2-iam-sink-delete-group/README.adoc | 6 ++-- aws2-iam/aws2-iam-sink-delete-user/README.adoc | 6 ++-- .../aws2-kinesis-firehose-sink/README.adoc | 8 +++--- aws2-kinesis/aws2-kinesis-sink/README.adoc | 4 +-- aws2-kinesis/aws2-kinesis-source/README.adoc | 2 +- aws2-kms/aws2-kms-sink-create-key/README.adoc | 6 ++-- aws2-kms/aws2-kms-sink-delete-key/README.adoc | 6 ++-- aws2-lambda/aws2-lambda-sink/README.adoc | 8 +++--- aws2-s3/aws2-s3-minio-source/README.adoc | 4 +-- aws2-s3/aws2-s3-move-after-read/README.adoc | 8 +++--- aws2-s3/aws2-s3-move-bucket-to-bucket/README.adoc | 4 +-- aws2-s3/aws2-s3-sink-with-aggregation/README.adoc | 8 +++--- .../aws2-s3-sink-with-zip-aggregation/README.adoc | 32 +++++++++++----------- .../config/CamelAWS2S3SinkConnector.properties | 4 +-- .../aws2-s3-sink-with-zip-aggregation.yaml | 4 +-- .../aws2-s3-sink-with-zip-dataformat/README.adoc | 32 +++++++++++----------- .../config/CamelAWS2S3SinkConnector.properties | 4 +-- .../aws2-s3-sink-with-zip-dataformat.yaml | 4 +-- aws2-s3/aws2-s3-source/README.adoc | 8 +++--- aws2-sns/aws2-sns-sink/README.adoc | 10 +++---- aws2-sqs/aws2-sqs-sink/README.adoc | 10 +++---- aws2-sqs/aws2-sqs-source/README.adoc | 10 +++---- 25 files changed, 104 insertions(+), 104 deletions(-) diff --git a/aws-s3-to-jms/README.adoc b/aws-s3-to-jms/README.adoc index 8669c66..aa7b045 100644 --- a/aws-s3-to-jms/README.adoc +++ b/aws-s3-to-jms/README.adoc @@ -36,10 +36,10 @@ In this example we'll use `/home/oscerd/connectors/` ``` > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-sjms2-kafka-connector/0.8.0/camel-sjms2-kafka-connector-0.8.0-package.zip -> unzip camel-sjms2-kafka-connector-0.8.0-package.zip -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws-s3-kafka-connector/0.8.0/camel-aws-s3-kafka-connector-0.8.0-package.zip -> unzip camel-aws-s3-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-sjms2-kafka-connector/0.8.0/camel-sjms2-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-sjms2-kafka-connector-0.8.0-package.tar.gz +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws-s3-kafka-connector/0.8.0/camel-aws-s3-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws-s3-kafka-connector-0.8.0-package.tar.gz ``` These are the basic connectors. For camel-sjms2 we have a bunch of provided dependencies we need to add in our path, so run the following commands: diff --git a/aws2-iam/aws2-iam-sink-create-group/README.adoc b/aws2-iam/aws2-iam-sink-create-group/README.adoc index 57e63e6..cf8fd95 100644 --- a/aws2-iam/aws2-iam-sink-create-group/README.adoc +++ b/aws2-iam/aws2-iam-sink-create-group/README.adoc @@ -19,13 +19,13 @@ $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --rep === Download the connector package -Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` +Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` [source] ---- > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-iam-kafka-connector/0.8.0/camel-aws2-iam-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-iam-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-iam-kafka-connector/0.8.0/camel-aws2-iam-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-iam-kafka-connector-0.8.0-package.tar.gz ---- === Configuring Kafka Connect diff --git a/aws2-iam/aws2-iam-sink-create-user/README.adoc b/aws2-iam/aws2-iam-sink-create-user/README.adoc index 2191c13..bc41772 100644 --- a/aws2-iam/aws2-iam-sink-create-user/README.adoc +++ b/aws2-iam/aws2-iam-sink-create-user/README.adoc @@ -19,13 +19,13 @@ $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --rep === Download the connector package -Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` +Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` [source] ---- > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-iam-kafka-connector/0.8.0/camel-aws2-iam-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-iam-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-iam-kafka-connector/0.8.0/camel-aws2-iam-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-iam-kafka-connector-0.8.0-package.tar.gz ---- === Configuring Kafka Connect diff --git a/aws2-iam/aws2-iam-sink-delete-group/README.adoc b/aws2-iam/aws2-iam-sink-delete-group/README.adoc index 7c9e540..66174e7 100644 --- a/aws2-iam/aws2-iam-sink-delete-group/README.adoc +++ b/aws2-iam/aws2-iam-sink-delete-group/README.adoc @@ -20,13 +20,13 @@ $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --rep === Download the connector package -Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` +Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` [source] ---- > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-iam-kafka-connector/0.8.0/camel-aws2-iam-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-iam-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-iam-kafka-connector/0.8.0/camel-aws2-iam-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-iam-kafka-connector-0.8.0-package.tar.gz ---- === Configuring Kafka Connect diff --git a/aws2-iam/aws2-iam-sink-delete-user/README.adoc b/aws2-iam/aws2-iam-sink-delete-user/README.adoc index 72735bc..ea9faf4 100644 --- a/aws2-iam/aws2-iam-sink-delete-user/README.adoc +++ b/aws2-iam/aws2-iam-sink-delete-user/README.adoc @@ -20,13 +20,13 @@ $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --rep === Download the connector package -Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` +Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` [source] ---- > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-iam-kafka-connector/0.8.0/camel-aws2-iam-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-iam-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-iam-kafka-connector/0.8.0/camel-aws2-iam-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-iam-kafka-connector-0.8.0-package.tar.gz ---- === Configuring Kafka Connect diff --git a/aws2-kinesis-firehose/aws2-kinesis-firehose-sink/README.adoc b/aws2-kinesis-firehose/aws2-kinesis-firehose-sink/README.adoc index 0e65173..b03f1e2 100644 --- a/aws2-kinesis-firehose/aws2-kinesis-firehose-sink/README.adoc +++ b/aws2-kinesis-firehose/aws2-kinesis-firehose-sink/README.adoc @@ -30,8 +30,8 @@ In this example we'll use `/home/oscerd/connectors/` ``` > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kinesis-firehose-kafka-connector/0.8.0/camel-aws2-kinesis-firehose-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-kinesis-firehose-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kinesis-firehose-kafka-connector/0.8.0/camel-aws2-kinesis-firehose-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-kinesis-firehose-kafka-connector-0.8.0-package.tar.gz ``` On AWS console create a Kinesis firehose delivery stream named firehose-stream and choose to send it to an S3 bucket from the available list (in the same region) @@ -146,8 +146,8 @@ So we need to do something like this: ``` > cd my-connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kinesis-firehose-kafka-connector/0.8.0/camel-aws2-kinesis-firehose-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-kinesis-firehose-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kinesis-firehose-kafka-connector/0.8.0/camel-aws2-kinesis-firehose-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-kinesis-firehose-kafka-connector-0.8.0-package.tar.gz ``` Now we can start the build diff --git a/aws2-kinesis/aws2-kinesis-sink/README.adoc b/aws2-kinesis/aws2-kinesis-sink/README.adoc index 31c9d7d..aadd04d 100644 --- a/aws2-kinesis/aws2-kinesis-sink/README.adoc +++ b/aws2-kinesis/aws2-kinesis-sink/README.adoc @@ -30,8 +30,8 @@ In this example we'll use `/home/oscerd/connectors/` ``` > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kinesis-kafka-connector/0.8.0/camel-aws2-kinesis-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-kinesis-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kinesis-kafka-connector/0.8.0/camel-aws2-kinesis-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-kinesis-kafka-connector-0.8.0-package.tar.gz ``` On AWS console create a Kinesis stream delivery stream named streamTest. diff --git a/aws2-kinesis/aws2-kinesis-source/README.adoc b/aws2-kinesis/aws2-kinesis-source/README.adoc index 76a703d..d221f18 100644 --- a/aws2-kinesis/aws2-kinesis-source/README.adoc +++ b/aws2-kinesis/aws2-kinesis-source/README.adoc @@ -374,7 +374,7 @@ Now we need to build the connector: > mvn clean package ``` -And move the zip package in target to my-connectors folder and unzipped it. +And move the tar.gz package in target to my-connectors folder and untar.gzped it. Now we can start the build diff --git a/aws2-kms/aws2-kms-sink-create-key/README.adoc b/aws2-kms/aws2-kms-sink-create-key/README.adoc index 4fad842..da43ea3 100644 --- a/aws2-kms/aws2-kms-sink-create-key/README.adoc +++ b/aws2-kms/aws2-kms-sink-create-key/README.adoc @@ -19,13 +19,13 @@ $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --rep === Download the connector package -Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` +Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` [source] ---- > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kms-kafka-connector/0.8.0/camel-aws2-kms-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-kms-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kms-kafka-connector/0.8.0/camel-aws2-kms-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-kms-kafka-connector-0.8.0-package.tar.gz ---- === Configuring Kafka Connect diff --git a/aws2-kms/aws2-kms-sink-delete-key/README.adoc b/aws2-kms/aws2-kms-sink-delete-key/README.adoc index d4a52d5..b976b30 100644 --- a/aws2-kms/aws2-kms-sink-delete-key/README.adoc +++ b/aws2-kms/aws2-kms-sink-delete-key/README.adoc @@ -19,13 +19,13 @@ $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --rep === Download the connector package -Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` +Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` [source] ---- > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kms-kafka-connector/0.8.0/camel-aws2-kms-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-kms-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-kms-kafka-connector/0.8.0/camel-aws2-kms-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-kms-kafka-connector-0.8.0-package.tar.gz ---- === Configuring Kafka Connect diff --git a/aws2-lambda/aws2-lambda-sink/README.adoc b/aws2-lambda/aws2-lambda-sink/README.adoc index 1d3af70..f5634e2 100644 --- a/aws2-lambda/aws2-lambda-sink/README.adoc +++ b/aws2-lambda/aws2-lambda-sink/README.adoc @@ -29,8 +29,8 @@ In this example we'll use `/home/oscerd/connectors/` ``` > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-lambda-kafka-connector/0.8.0/camel-aws2-lambda-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-sns-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-lambda-kafka-connector/0.8.0/camel-aws2-lambda-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-sns-kafka-connector-0.8.0-package.tar.gz ``` ### Deploying the AWS Lambda function @@ -161,8 +161,8 @@ So we need to do something like this: ``` > cd my-connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-lambda-kafka-connector/0.8.0/camel-aws2-lambda-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-lambda-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-lambda-kafka-connector/0.8.0/camel-aws2-lambda-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-lambda-kafka-connector-0.8.0-package.tar.gz ``` Now we can start the build diff --git a/aws2-s3/aws2-s3-minio-source/README.adoc b/aws2-s3/aws2-s3-minio-source/README.adoc index fdaa0a8..64add7e 100644 --- a/aws2-s3/aws2-s3-minio-source/README.adoc +++ b/aws2-s3/aws2-s3-minio-source/README.adoc @@ -29,8 +29,8 @@ In this example we'll use `/home/oscerd/connectors/` ``` > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-s3-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz ``` Now we need to setup a MinIO instance diff --git a/aws2-s3/aws2-s3-move-after-read/README.adoc b/aws2-s3/aws2-s3-move-after-read/README.adoc index c95a271..7b16e57 100644 --- a/aws2-s3/aws2-s3-move-after-read/README.adoc +++ b/aws2-s3/aws2-s3-move-after-read/README.adoc @@ -28,8 +28,8 @@ In this example we'll use `/home/oscerd/connectors/` ``` > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-s3-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz ``` Now it's time to setup the connectors @@ -145,8 +145,8 @@ So we need to do something like this: ``` > cd my-connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-s3-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz ``` Now we can start the build diff --git a/aws2-s3/aws2-s3-move-bucket-to-bucket/README.adoc b/aws2-s3/aws2-s3-move-bucket-to-bucket/README.adoc index a94c27a..b0482a6 100644 --- a/aws2-s3/aws2-s3-move-bucket-to-bucket/README.adoc +++ b/aws2-s3/aws2-s3-move-bucket-to-bucket/README.adoc @@ -31,9 +31,9 @@ Since this is example is related to a new feature you'll need to build the lates ``` > cd <ckc_project> > mvn clean package -> cp <ckc_project>/connectors/camel-aws2-s3-kafka-connector/target/camel-aws2-s3-kafka-connector-0.8.0-SNAPSHOT-package.zip /home/oscerd/connectors/ +> cp <ckc_project>/connectors/camel-aws2-s3-kafka-connector/target/camel-aws2-s3-kafka-connector-0.8.0-SNAPSHOT-package.tar.gz /home/oscerd/connectors/ > cd /home/oscerd/connectors/ -> unzip camel-aws2-s3-kafka-connector-0.8.0-SNAPSHOT-package.zip +> untar.gz camel-aws2-s3-kafka-connector-0.8.0-SNAPSHOT-package.tar.gz ``` Now it's time to setup the connectors diff --git a/aws2-s3/aws2-s3-sink-with-aggregation/README.adoc b/aws2-s3/aws2-s3-sink-with-aggregation/README.adoc index 0333854..6c8eb6c 100644 --- a/aws2-s3/aws2-s3-sink-with-aggregation/README.adoc +++ b/aws2-s3/aws2-s3-sink-with-aggregation/README.adoc @@ -28,8 +28,8 @@ In this example we'll use `/home/oscerd/connectors/` ``` > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-s3-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz ``` Now it's time to setup the connectors @@ -157,8 +157,8 @@ So we need to do something like this: ``` > cd my-connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-s3-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz ``` Now we can start the build diff --git a/aws2-s3/aws2-s3-sink-with-zip-aggregation/README.adoc b/aws2-s3/aws2-s3-sink-with-zip-aggregation/README.adoc index b7dc401..a25859d 100644 --- a/aws2-s3/aws2-s3-sink-with-zip-aggregation/README.adoc +++ b/aws2-s3/aws2-s3-sink-with-zip-aggregation/README.adoc @@ -102,7 +102,7 @@ Now we need to edit the POM </dependency> <dependency> <groupId>org.apache.camel</groupId> - <artifactId>camel-zipfile</artifactId> + <artifactId>camel-tar.gzfile</artifactId> <version>3.4.2</version> </dependency> </dependencies> @@ -111,7 +111,7 @@ Now we need to edit the POM . ``` -In the dependencies section you'll need to uncomment the aws2-s3 connector dependency and adding the camel-zipfile component +In the dependencies section you'll need to uncomment the aws2-s3 connector dependency and adding the camel-tar.gzfile component Now we need to build the connector: @@ -119,12 +119,12 @@ Now we need to build the connector: > mvn clean package ``` -In this example we'll use `/home/oscerd/connectors/` as plugin.path, but we'll need the generated zip from the previois build +In this example we'll use `/home/oscerd/connectors/` as plugin.path, but we'll need the generated tar.gz from the previois build ``` > cd /home/oscerd/connectors/ -> cp /home/workspace/miscellanea/aws2s3/target/aws2s3-1.0-SNAPSHOT-package.zip . -> unzip aws2s3-1.0-SNAPSHOT-package.zip +> cp /home/workspace/miscellanea/aws2s3/target/aws2s3-1.0-SNAPSHOT-package.tar.gz . +> untar.gz aws2s3-1.0-SNAPSHOT-package.tar.gz ``` Now it's time to setup the connectors @@ -145,9 +145,9 @@ camel.component.aws2-s3.accessKey=xxxx camel.component.aws2-s3.secretKey=yyyy camel.component.aws2-s3.region=eu-west-1 -camel.sink.endpoint.keyName=${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip +camel.sink.endpoint.keyName=${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz -camel.beans.aggregate=#class:org.apache.camel.processor.aggregate.zipfile.ZipAggregationStrategy +camel.beans.aggregate=#class:org.apache.camel.processor.aggregate.tar.gzfile.ZipAggregationStrategy camel.aggregation.size=10 camel.aggregation.timeout=5000 ``` @@ -173,7 +173,7 @@ Kafka to S3 message 4 Kafka to S3 message 5 ``` -You should see (after the timeout has been reached) a file with date-exchangeId.zip name containing the following multiple files. Those files will contain the messages. +You should see (after the timeout has been reached) a file with date-exchangeId.tar.gz name containing the following multiple files. Those files will contain the messages. ``` Kafka to S3 message 1 @@ -326,7 +326,7 @@ Now we need to edit the POM </dependency> <dependency> <groupId>org.apache.camel</groupId> - <artifactId>camel-zipfile</artifactId> + <artifactId>camel-tar.gzfile</artifactId> <version>3.5.0</version> </dependency> </dependencies> @@ -341,7 +341,7 @@ Now we need to build the connector: > mvn clean package ``` -And move the zip package in targe to my-connectors folder and unzipped it. +And move the tar.gz package in targe to my-connectors folder and untar.gzped it. Now we can start the build @@ -415,8 +415,8 @@ oc exec -i `oc get pods --field-selector status.phase=Running -l strimzi.io/name "value.converter": "org.apache.kafka.connect.storage.StringConverter", "topics": "s3-topic", "camel.sink.path.bucketNameOrArn": "camel-kafka-connector", - "camel.sink.endpoint.keyName": "${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip", - "camel.beans.aggregate": "#class:org.apache.camel.processor.aggregate.zipfile.ZipAggregationStrategy", + "camel.sink.endpoint.keyName": "${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz", + "camel.beans.aggregate": "#class:org.apache.camel.processor.aggregate.tar.gzfile.ZipAggregationStrategy", "camel.aggregation.size": "10", "camel.aggregation.timeout": "5000", "camel.component.aws2-s3.accessKey": "xxx", @@ -447,8 +447,8 @@ spec: value.converter: org.apache.kafka.connect.storage.StringConverter topics: s3-topic camel.sink.path.bucketNameOrArn: camel-kafka-connector - camel.sink.endpoint.keyName: ${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip - camel.beans.aggregate: #class:org.apache.camel.processor.aggregate.zipfile.ZipAggregationStrategy + camel.sink.endpoint.keyName: ${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz + camel.beans.aggregate: #class:org.apache.camel.processor.aggregate.tar.gzfile.ZipAggregationStrategy camel.aggregation.size: 10 camel.aggregation.timeout: 5000 camel.component.aws2-s3.accessKey: xxxx @@ -461,7 +461,7 @@ If you followed the optional step for secret credentials you can run the followi [source,bash,options="nowrap"] ---- -oc apply -f config/openshift/aws2-s3-sink-with-zip-aggregation.yaml +oc apply -f config/openshift/aws2-s3-sink-with-tar.gz-aggregation.yaml ---- You can check the status of the connector using @@ -484,7 +484,7 @@ Kafka to S3 message 4 Kafka to S3 message 5 ``` -You should see (after the timeout has been reached) a file with date-exchangeId.zip name containing the following multiple files. Those files will contain the messages. +You should see (after the timeout has been reached) a file with date-exchangeId.tar.gz name containing the following multiple files. Those files will contain the messages. ``` Kafka to S3 message 1 diff --git a/aws2-s3/aws2-s3-sink-with-zip-aggregation/config/CamelAWS2S3SinkConnector.properties b/aws2-s3/aws2-s3-sink-with-zip-aggregation/config/CamelAWS2S3SinkConnector.properties index af635c4..0e75f88 100644 --- a/aws2-s3/aws2-s3-sink-with-zip-aggregation/config/CamelAWS2S3SinkConnector.properties +++ b/aws2-s3/aws2-s3-sink-with-zip-aggregation/config/CamelAWS2S3SinkConnector.properties @@ -28,8 +28,8 @@ camel.component.aws2-s3.accessKey=xxxx camel.component.aws2-s3.secretKey=yyyy camel.component.aws2-s3.region=eu-west-1 -camel.sink.endpoint.keyName=${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip +camel.sink.endpoint.keyName=${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz -camel.beans.aggregate=#class:org.apache.camel.processor.aggregate.zipfile.ZipAggregationStrategy +camel.beans.aggregate=#class:org.apache.camel.processor.aggregate.tar.gzfile.ZipAggregationStrategy camel.aggregation.size=10 camel.aggregation.timeout=5000 diff --git a/aws2-s3/aws2-s3-sink-with-zip-aggregation/config/openshift/aws2-s3-sink-with-zip-aggregation.yaml b/aws2-s3/aws2-s3-sink-with-zip-aggregation/config/openshift/aws2-s3-sink-with-zip-aggregation.yaml index 2f46f08..8e59809 100644 --- a/aws2-s3/aws2-s3-sink-with-zip-aggregation/config/openshift/aws2-s3-sink-with-zip-aggregation.yaml +++ b/aws2-s3/aws2-s3-sink-with-zip-aggregation/config/openshift/aws2-s3-sink-with-zip-aggregation.yaml @@ -13,8 +13,8 @@ spec: value.converter: org.apache.kafka.connect.storage.StringConverter topics: s3-topic camel.sink.path.bucketNameOrArn: camel-kafka-connector - camel.sink.endpoint.keyName: ${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip - camel.beans.aggregate: '#class:org.apache.camel.processor.aggregate.zipfile.ZipAggregationStrategy' + camel.sink.endpoint.keyName: ${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz + camel.beans.aggregate: '#class:org.apache.camel.processor.aggregate.tar.gzfile.ZipAggregationStrategy' camel.aggregation.size: 10 camel.aggregation.timeout: 5000 camel.component.aws2-s3.accessKey: ${file:/opt/kafka/external-configuration/aws-credentials/aws2-s3-cred.properties:accessKey} diff --git a/aws2-s3/aws2-s3-sink-with-zip-dataformat/README.adoc b/aws2-s3/aws2-s3-sink-with-zip-dataformat/README.adoc index a88c1b5..b680f8b 100644 --- a/aws2-s3/aws2-s3-sink-with-zip-dataformat/README.adoc +++ b/aws2-s3/aws2-s3-sink-with-zip-dataformat/README.adoc @@ -102,7 +102,7 @@ Now we need to edit the POM </dependency> <dependency> <groupId>org.apache.camel</groupId> - <artifactId>camel-zipfile</artifactId> + <artifactId>camel-tar.gzfile</artifactId> </dependency> </dependencies> . @@ -110,7 +110,7 @@ Now we need to edit the POM . ``` -In the dependencies section you'll need to uncomment the aws2-s3 connector dependency and adding the camel-zipfile component +In the dependencies section you'll need to uncomment the aws2-s3 connector dependency and adding the camel-tar.gzfile component Now we need to build the connector: @@ -118,12 +118,12 @@ Now we need to build the connector: > mvn clean package ``` -In this example we'll use `/home/oscerd/connectors/` as plugin.path, but we'll need the generated zip from the previois build +In this example we'll use `/home/oscerd/connectors/` as plugin.path, but we'll need the generated tar.gz from the previois build ``` > cd /home/oscerd/connectors/ -> cp /home/workspace/miscellanea/aws2s3/target/aws2s3-1.0-SNAPSHOT-package.zip . -> unzip aws2s3-1.0-SNAPSHOT-package.zip +> cp /home/workspace/miscellanea/aws2s3/target/aws2s3-1.0-SNAPSHOT-package.tar.gz . +> untar.gz aws2s3-1.0-SNAPSHOT-package.tar.gz ``` Now it's time to setup the connectors @@ -137,7 +137,7 @@ key.converter=org.apache.kafka.connect.storage.StringConverter value.converter=org.apache.kafka.connect.storage.StringConverter topics=mytopic -camel.sink.marshal=zipfile +camel.sink.marshal=tar.gzfile camel.sink.path.bucketNameOrArn=camel-kafka-connector @@ -145,7 +145,7 @@ camel.component.aws2-s3.accessKey=xxxx camel.component.aws2-s3.secretKey=yyyy camel.component.aws2-s3.region=eu-west-1 -camel.sink.endpoint.keyName=${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip +camel.sink.endpoint.keyName=${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz ``` and add the correct credentials for AWS. @@ -165,7 +165,7 @@ bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic mytopic Kafka to S3 message 1 ``` -You should see a file with date-exchangeId.zip name containing the one file named date-exchangeId. This file will contain the message. +You should see a file with date-exchangeId.tar.gz name containing the one file named date-exchangeId. This file will contain the message. ``` Kafka to S3 message 1 @@ -314,7 +314,7 @@ Now we need to edit the POM </dependency> <dependency> <groupId>org.apache.camel</groupId> - <artifactId>camel-zipfile</artifactId> + <artifactId>camel-tar.gzfile</artifactId> </dependency> </dependencies> . @@ -328,7 +328,7 @@ Now we need to build the connector: > mvn clean package ``` -And move the zip package in targe to my-connectors folder and unzipped it. +And move the tar.gz package in targe to my-connectors folder and untar.gzped it. Now we can start the build @@ -402,8 +402,8 @@ oc exec -i `oc get pods --field-selector status.phase=Running -l strimzi.io/name "value.converter": "org.apache.kafka.connect.storage.StringConverter", "topics": "s3-topic", "camel.sink.path.bucketNameOrArn": "camel-kafka-connector", - "camel.sink.endpoint.keyName": "${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip", - "camel.sink.marshal": "zipfile", + "camel.sink.endpoint.keyName": "${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz", + "camel.sink.marshal": "tar.gzfile", "camel.component.aws2-s3.accessKey": "xxx", "camel.component.aws2-s3.secretKey": "xxx", "camel.component.aws2-s3.region": "xxx" @@ -432,8 +432,8 @@ spec: value.converter: org.apache.kafka.connect.storage.StringConverter topics: s3-topic camel.sink.path.bucketNameOrArn: camel-kafka-connector - camel.sink.endpoint.keyName: ${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip - camel.sink.marshal: zipfile + camel.sink.endpoint.keyName: ${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz + camel.sink.marshal: tar.gzfile camel.component.aws2-s3.accessKey: xxxx camel.component.aws2-s3.secretKey: yyyy camel.component.aws2-s3.region: region @@ -444,7 +444,7 @@ If you followed the optional step for secret credentials you can run the followi [source,bash,options="nowrap"] ---- -oc apply -f config/openshift/aws2-s3-sink-with-zip-dataformat.yaml +oc apply -f config/openshift/aws2-s3-sink-with-tar.gz-dataformat.yaml ---- You can check the status of the connector using @@ -463,7 +463,7 @@ oc exec -i -c kafka my-cluster-kafka-0 -- bin/kafka-console-producer.sh --bootst Kafka to S3 message 1 ``` -You should see a file with date-exchangeId.zip name containing the one file named date-exchangeId. This file will contain the message. +You should see a file with date-exchangeId.tar.gz name containing the one file named date-exchangeId. This file will contain the message. ``` Kafka to S3 message 1 diff --git a/aws2-s3/aws2-s3-sink-with-zip-dataformat/config/CamelAWS2S3SinkConnector.properties b/aws2-s3/aws2-s3-sink-with-zip-dataformat/config/CamelAWS2S3SinkConnector.properties index f905ed4..16d5d3e 100644 --- a/aws2-s3/aws2-s3-sink-with-zip-dataformat/config/CamelAWS2S3SinkConnector.properties +++ b/aws2-s3/aws2-s3-sink-with-zip-dataformat/config/CamelAWS2S3SinkConnector.properties @@ -21,7 +21,7 @@ key.converter=org.apache.kafka.connect.storage.StringConverter value.converter=org.apache.kafka.connect.storage.StringConverter topics=mytopic -camel.sink.marshal=zipfile +camel.sink.marshal=tar.gzfile camel.sink.path.bucketNameOrArn=camel-kafka-connector @@ -29,4 +29,4 @@ camel.component.aws2-s3.accessKey=xxxx camel.component.aws2-s3.secretKey=yyyy camel.component.aws2-s3.region=eu-west-1 -camel.sink.endpoint.keyName=${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip +camel.sink.endpoint.keyName=${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz diff --git a/aws2-s3/aws2-s3-sink-with-zip-dataformat/config/openshift/aws2-s3-sink-with-zip-dataformat.yaml b/aws2-s3/aws2-s3-sink-with-zip-dataformat/config/openshift/aws2-s3-sink-with-zip-dataformat.yaml index 3c185f5..520409b 100644 --- a/aws2-s3/aws2-s3-sink-with-zip-dataformat/config/openshift/aws2-s3-sink-with-zip-dataformat.yaml +++ b/aws2-s3/aws2-s3-sink-with-zip-dataformat/config/openshift/aws2-s3-sink-with-zip-dataformat.yaml @@ -13,8 +13,8 @@ spec: value.converter: org.apache.kafka.connect.storage.StringConverter topics: s3-topic camel.sink.path.bucketNameOrArn: camel-kafka-connector - camel.sink.endpoint.keyName: ${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.zip - camel.sink.marshal: zipfile + camel.sink.endpoint.keyName: ${date:now:yyyyMMdd-HHmmssSSS}-${exchangeId}.tar.gz + camel.sink.marshal: tar.gzfile camel.component.aws2-s3.accessKey: ${file:/opt/kafka/external-configuration/aws-credentials/aws2-s3-cred.properties:accessKey} camel.component.aws2-s3.secretKey: ${file:/opt/kafka/external-configuration/aws-credentials/aws2-s3-cred.properties:secretKey} camel.component.aws2-s3.region: ${file:/opt/kafka/external-configuration/aws-credentials/aws2-s3-cred.properties:region} diff --git a/aws2-s3/aws2-s3-source/README.adoc b/aws2-s3/aws2-s3-source/README.adoc index 74e5ff4..a9eac6a 100644 --- a/aws2-s3/aws2-s3-source/README.adoc +++ b/aws2-s3/aws2-s3-source/README.adoc @@ -28,8 +28,8 @@ In this example we'll use `/home/oscerd/connectors/` ``` > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-s3-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz ``` Now it's time to setup the connectors @@ -139,8 +139,8 @@ So we need to do something like this: ``` > cd my-connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-s3-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-s3-kafka-connector/0.8.0/camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-s3-kafka-connector-0.8.0-package.tar.gz ``` Now we can start the build diff --git a/aws2-sns/aws2-sns-sink/README.adoc b/aws2-sns/aws2-sns-sink/README.adoc index 4696f80..ab83593 100644 --- a/aws2-sns/aws2-sns-sink/README.adoc +++ b/aws2-sns/aws2-sns-sink/README.adoc @@ -19,13 +19,13 @@ $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --rep === Download the connector package -Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` +Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` [source] ---- > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sns-kafka-connector/0.8.0/camel-aws2-sns-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-sns-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sns-kafka-connector/0.8.0/camel-aws2-sns-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-sns-kafka-connector-0.8.0-package.tar.gz ---- === Configuring Kafka Connect @@ -153,8 +153,8 @@ So we need to do something like this: [source] ---- > cd my-connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sns-kafka-connector/0.8.0/camel-aws2-sns-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-sns-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sns-kafka-connector/0.8.0/camel-aws2-sns-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-sns-kafka-connector-0.8.0-package.tar.gz ---- Now we can start the build diff --git a/aws2-sqs/aws2-sqs-sink/README.adoc b/aws2-sqs/aws2-sqs-sink/README.adoc index afe9602..47fa856 100644 --- a/aws2-sqs/aws2-sqs-sink/README.adoc +++ b/aws2-sqs/aws2-sqs-sink/README.adoc @@ -19,13 +19,13 @@ $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --rep === Download the connector package -Download the connector package zip and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` +Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` [source] ---- > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sqs-kafka-connector/0.8.0/camel-aws2-sqs-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-sqs-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sqs-kafka-connector/0.8.0/camel-aws2-sqs-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-sqs-kafka-connector-0.8.0-package.tar.gz ---- === Configuring Kafka Connect @@ -156,8 +156,8 @@ So we need to do something like this: [source] ---- > cd my-connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sqs-kafka-connector/0.8.0/camel-aws2-sqs-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-sqs-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sqs-kafka-connector/0.8.0/camel-aws2-sqs-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-sqs-kafka-connector-0.8.0-package.tar.gz ---- Now we can start the build diff --git a/aws2-sqs/aws2-sqs-source/README.adoc b/aws2-sqs/aws2-sqs-source/README.adoc index 9833238..56740a3 100644 --- a/aws2-sqs/aws2-sqs-source/README.adoc +++ b/aws2-sqs/aws2-sqs-source/README.adoc @@ -19,13 +19,13 @@ $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --rep === Download the connector package -Download the connector package zip and extract the content to a directory.In this example we'll use `/home/oscerd/connectors/` +Download the connector package tar.gz and extract the content to a directory.In this example we'll use `/home/oscerd/connectors/` [source] ---- > cd /home/oscerd/connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sqs-kafka-connector/0.8.0/camel-aws2-sqs-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-sqs-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sqs-kafka-connector/0.8.0/camel-aws2-sqs-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-sqs-kafka-connector-0.8.0-package.tar.gz ---- === Configuring Kafka Connect @@ -157,8 +157,8 @@ so that each one is in its own subfolder [source] ---- > cd my-connectors/ -> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sqs-kafka-connector/0.8.0/camel-aws2-sqs-kafka-connector-0.8.0-package.zip -> unzip camel-aws2-sqs-kafka-connector-0.8.0-package.zip +> wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-aws2-sqs-kafka-connector/0.8.0/camel-aws2-sqs-kafka-connector-0.8.0-package.tar.gz +> untar.gz camel-aws2-sqs-kafka-connector-0.8.0-package.tar.gz ---- Now we can start the build
