Re: [ANNOUNCE] Apache Camel 3.10.0 Released

[Omar Al-Safi]

Thanks Andrea!

Cheers,

On Thu, May 20, 2021 at 12:18 PM Andrea Cosentino  wrote:

> Hello Omar,
>
> No 3.11 will be the next LTS. This release introduce a lot of new stuff.
> The next will stabilize a bit and we'll release the LTS 3.11.0
>
> Cheers
>
>
> Il gio 20 mag 2021, 12:14 Omar Al-Safi  ha scritto:
>
> > Thanks Gregor!
> >
> > By the way, shouldn't 3.10 be supposed to be an LTS version?
> >
> > Regards,
> > Omar
> >
> > On Thu, May 20, 2021 at 11:38 AM Gregor Zurowski <
> gre...@list.zurowski.org
> > >
> > wrote:
> >
> > > The Camel PMC is pleased to announce the release of Apache Camel
> 3.10.0.
> > >
> > > Apache Camel is an open source integration framework that empowers you
> > > to quickly and easily integrate various systems consuming or producing
> > > data.
> > >
> > > This release is a new minor release and contains 208 bug fixes and
> > > improvements. The release is available for immediate download at:
> > >
> > > https://camel.apache.org/download/
> > >
> > > For more details please take a look at the release notes at:
> > >
> > > https://camel.apache.org/releases/release-3.10.0/
> > >
> >
>

Re: [ANNOUNCE] Apache Camel 3.10.0 Released

[Omar Al-Safi]

Thanks Gregor!

By the way, shouldn't 3.10 be supposed to be an LTS version?

Regards,
Omar

On Thu, May 20, 2021 at 11:38 AM Gregor Zurowski 
wrote:

> The Camel PMC is pleased to announce the release of Apache Camel 3.10.0.
>
> Apache Camel is an open source integration framework that empowers you
> to quickly and easily integrate various systems consuming or producing
> data.
>
> This release is a new minor release and contains 208 bug fixes and
> improvements. The release is available for immediate download at:
>
> https://camel.apache.org/download/
>
> For more details please take a look at the release notes at:
>
> https://camel.apache.org/releases/release-3.10.0/
>

Re: Kafka - Issue with additionalProperties

[Omar Al-Safi]

Hi,

I guess you are using the Endpoint DSL? Which Camel version do you use
here?

In case you are running Camel 3.7.0, I recall the additionalProperties was
broken in the kafka component and it was fixed in 3.7.1 and 3.8.0.

Regards,
Omar

On Wed, Apr 28, 2021 at 3:19 PM anandakrishnan ramakrishnan <
anand140...@gmail.com> wrote:

> I am having issue with setting additionalProperties. I am setting the
> property as below
>
> Map additionalProperties = new HashMap<>();
> additionalProperties.put("sasl.login.callback.handler.class",
>
>
>  
> "io.confluent.kafka.clients.plugins.auth.token.TokenUserLoginCallbackHandler");
>
> Followed by the Kafka route builder is setup as below
>
> .additionalProperties(additionalProperties)
>
> However the property i see populated in the kafka config show up as
>
> "sasl"->"io.confluent.kafka.clients.plugins.auth.token.TokenUserLoginCallbackHandler"
>
> The reason being the PropertyBindingSupport.java is splitting by dot
>
> // we should only walk and create OGNL path for the middle graph
>
> String[] parts;
> if (isDotKey(name)) {
> parts = splitKey(name);
> } else {
> parts = new String[] { name };
> }
>
> // which only picks up "sasl" instead of
> "sasl.login.callback-handler.class"
>
> newName = parts[i + 1];
>
> Any help is appreciated.
>

Re: kafka consumer to kafka producer parition key issue

[Omar Al-Safi]

Weird, from experience, I have seen this timeout happen if the specified
partition doesn't exist.

However, since you are using Camel 3.9 and you didn't set a message key,
the partition key won't be set in the ProducerRecord, it will be only set
if you set the message key, however this behavior has changed in Camel 3.10
where the partition key won't require setting the message key. Thus, even
if you set the partition key but you didn't set a message key, I will
expect camel-kafka to ignore the partition (based on Camel 3.9) as you can
see in this code
<https://github.com/apache/camel/blob/camel-3.9.0/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/KafkaProducer.java#L275>
.

How about if you try using camel-vertx-kafka
<https://camel.apache.org/components/3.9.x/vertx-kafka-component.html>
component
and see if this solves the problem?

Regards,
Omar

On Mon, Apr 19, 2021 at 5:26 PM Mark Nuttall  wrote:

> Setting the message key does not help. :(
>
> On Mon, Apr 19, 2021 at 11:20 AM Mark Nuttall  wrote:
>
> > I've never set the message key.  I might give that a shot.
> >
> > The route is pretty simple. And the only headers being set are the
> > partition key and another string value i need to be passed. The body is
> > JSON.
> > I am running Kafka locally (so whatever partitions is the default) and
> AWS
> > MSK -  2 partitions.
> >
> > Mark
> >
> >
> >
> > On Mon, Apr 19, 2021 at 4:14 AM Omar Al-Safi  wrote:
> >
> >> Hi Mark,
> >>
> >> When you send your message to Kafka, do you as well set the message key
> or
> >> you just set the partition key? Perhaps it would be great if you can
> post
> >> an example for your route with the headers/body that you set, that could
> >> help us here to troubleshoot the issue. Also, how many partitions do you
> >> have in your topic?
> >>
> >> Regards,
> >> Omar
> >>
> >> On Fri, Apr 16, 2021 at 5:10 PM Mark Nuttall 
> wrote:
> >>
> >> > Camel version: 3.9.0
> >> >
> >> > Issue: when consuming from one kafka topic and producing to another,
> >> > setting the partition key causes the write to fail. After a few
> seconds
> >> it
> >> > will fail with this error:
> >> >
> >> > Error during processing. Exchange[2729AA824AE4E97-].
> >> Caused
> >> > by: [org.apache.kafka.common.errors.TimeoutException - Topic
> some-topic
> >> not
> >> > present in metadata after 6 ms.]
> >> >
> >> > The topic does exist. If i produce to the topic without the partition
> >> key
> >> > OR I produce to the topic with a partition key via the
> producerTemplate
> >> it
> >> > works just fine.
> >> > I am pretty sure in the past i have done this and it worked.
> >> >
> >> > Has anyone else seen this? Any ideas? I do have a work around (i.e.
> use
> >> > Spring's kafkaTemplate.).
> >> >
> >> > Mark
> >> >
> >>
> >
>

Re: kafka consumer to kafka producer parition key issue

[Omar Al-Safi]

Hi Mark,

When you send your message to Kafka, do you as well set the message key or
you just set the partition key? Perhaps it would be great if you can post
an example for your route with the headers/body that you set, that could
help us here to troubleshoot the issue. Also, how many partitions do you
have in your topic?

Regards,
Omar

On Fri, Apr 16, 2021 at 5:10 PM Mark Nuttall  wrote:

> Camel version: 3.9.0
>
> Issue: when consuming from one kafka topic and producing to another,
> setting the partition key causes the write to fail. After a few seconds it
> will fail with this error:
>
> Error during processing. Exchange[2729AA824AE4E97-]. Caused
> by: [org.apache.kafka.common.errors.TimeoutException - Topic some-topic not
> present in metadata after 6 ms.]
>
> The topic does exist. If i produce to the topic without the partition key
> OR I produce to the topic with a partition key via the producerTemplate it
> works just fine.
> I am pretty sure in the past i have done this and it worked.
>
> Has anyone else seen this? Any ideas? I do have a work around (i.e. use
> Spring's kafkaTemplate.).
>
> Mark
>

Re: Bug camel-azure-storage-blob 3.7.0

[Omar Al-Safi]

Hello Chirtsian,

Regarding 2). Honestly, personally I didn't try to configure the proxy to
communicate with the Azure services. However, I have done some research and
it looks like you will need first to build your own HttpClient with the
proxy settings as stated here
.
Once you have done that, you will need to build your own BlobServiceClient
with your custom HttpClient and then use it in the endpoint parameters

.

Regards,
Omar


On Tue, Feb 23, 2021 at 11:27 AM  wrote:

> Hi,
> Thx for the quick response.
> ad.1) will do that thx
> ad.2)
> it is just a simple http/https proxy. We require a proxy to communicate
> with the outside world.
> Tried to set the options in the jvm via -Dhttps.proxyHost/
> -Dhttps.proxyPort without any luck.
> Is it possible that the component just ignores the jvm options? Seems
> unlikely...
>
> Regards Christian
>
> -Original Message-
> From: Claus Ibsen 
> Sent: Dienstag, 23. Februar 2021 06:40
> To: users@camel.apache.org
> Subject: {EXT} Re: Bug camel-azure-storage-blob 3.7.0
>
> Hi Christian
>
> Welcome to the Camel community.
>
> Ad 1)
> Yes that seems like a little bug. You are welcome to create a JIRA ticket.
>
> You can try as a workaround in a custom processor before calling the
> azure, to convert the message body to a FileInputStream.
>
> Ad 2)
> What kind of proxy would that be? I have not heard of any cloud services
> that go via a proxy before.
>
>
>
>
> On Mon, Feb 22, 2021 at 7:10 PM  wrote:
>
> > Hi Guys,
> >
> > First of all thanks for your great work.
> > We are having a great time with camel and are looking forward to
> > contribute when we are more experienced with the framework!
> >
> > It would be cool if someone can help us with our current issue:
> > We are currently trying to integrate azure cloud by using the
> > *camel-azure-storage-blob* component
> >
> > 
> > org.apache.camel.springboot
> > camel-azure-storage-blob-starter
> >
> > 3.7.0
> > 
> >
> >
> > 1.) Everything works fine until we activate *stream caching* and camel
> > starts to spool the payload. In this case we get the following error:
> >
> > Caused by: java.lang.IllegalArgumentException: InputStream of body
> > exchange does not support mark/rest operations.
> >
> > at
> > org.apache.camel.component.azure.storage.blob.BlobStreamAndLength.crea
> > teBlobStreamAndLengthFromExchangeBody(BlobStreamAndLength.java:52)
> >
> > at
> > org.apache.camel.component.azure.storage.blob.operations.BlobOperation
> > s.uploadBlockBlob(BlobOperations.java:181)
> >
> > I have looked into the source and think that the component does not
> > load the stream properly from the FileInputStreamCache object. Am I
> > missing something or is this actually a bug (missing feature) in the
> component?
> >
> > 2.) Additionally I would like to force the component to communicate
> > over a proxy with the azure cloud. However I did not find anything
> > regarding setting a proxy in camel-azure-storage-blob component?
> > Not possible? Does there exist a workaround.
> >
> > Regards Christian
> >
> >
> >
> > 
> >
> >
> > *Dipl. Ing. Christian* *Müller*
> > Software Engineer
> > Business Automation & Integration
> >
> > *Mobile:*
> >
> > +43 664 88748100
> >
> > *E-Mail:*
> >
> > christian.muel...@spar-ics.com
> >
> >
> >
> >
> > *SPAR Business Services GmbH *Information & Communication Services
> > Europastrasse 3, 5015 Salzburg, Austria
> >
> >
> >
> >
> >
> > Sollten Sie diese E-Mail unbeabsichtigt bzw. irrtümlich erhalten
> > haben, so weisen wir Sie darauf hin, dass gemäß § 93 Abs 4 TKG der
> > Inhalt sowie die Tatsache des Empfangs dieser E-Mail weder
> > aufgezeichnet noch verwertet oder Unbefugten mitgeteilt werden dürfen.
> > Wir ersuchen Sie, die Nachricht von Ihrem System zu löschen und sich mit
> uns in Verbindung zu setzen.
> >
> > If you have received this email accidentally or in error, we point out
> > that, in accordance with § 93 para. 4 TKG (Telecommunications Act),
> > the contents of this email and the fact of its receipt must not be
> > recorded, exploited or communicated to unauthorized persons. We ask
> > you to delete the message from your system and to contact us.
> >
> >
> >
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: Azure Blob ignores LIST_BLOB_OPTIONS

[Omar Al-Safi]

Glad to hear that!

However, if you have an idea how to handle these listBlobs, please feel
free to send a PR with your proposed fix.

Regards,
Omar

On Tue, Jan 26, 2021 at 2:14 PM Mark Andreev  wrote:

> I found a solution for this issue. When I use Azure Api directly and
> operate with Page all works. It is useful when your azure blob storage
> contains a lot of blobs (in my case ~250k).
>
> from("direct:list")
> .routeId("listBlobs")
> .process(exchange -> {
>   BlobServiceClient blobServiceClient = new
> BlobServiceClientBuilder()
>   .endpoint(String
>   .format("https://%s.blob.core.windows.net;,
> AzureBlobConfigHelper.ACCOUNT))
>   .credential(new
> StorageSharedKeyCredential(AzureBlobConfigHelper.ACCOUNT,
>   AzureBlobConfigHelper.ACCESS_KEY))
>   .buildClient();
>   BlobContainerClient containerClient =
>
>
> blobServiceClient.getBlobContainerClient(AzureBlobConfigHelper.BLOB_CONTAINER_NAME);
>   ListBlobsOptions blobsOptions = new
> ListBlobsOptions().setMaxResultsPerPage(5);
>   Duration duration = Duration.of(10, ChronoUnit.SECONDS);
>
>   Iterator
>   iterator = containerClient.listBlobs(blobsOptions,
> duration).iterator();
>   exchange.getIn().setBody(iterator);
> })
> .loopDoWhile(exchange -> {
>   Iterator iterator =
>   (Iterator)
> exchange.getIn().getBody(Iterator.class);
>   return iterator.hasNext();
> })
> .process(exchange -> {
>   Iterator iterator =
>   (Iterator)
> exchange.getIn().getBody(Iterator.class);
>   exchange.getIn().setBody(iterator.next());
> })
> .process(exchange -> {
>       BlobItem item = exchange.getIn().getBody(BlobItem.class);
>   log.info(item);
> })
> .end();
>
> On Tue, 26 Jan 2021 at 13:57, Omar Al-Safi  wrote:
>
> > Hi,
> >
> > Actually you are doing nothing wrong here. Apparently when you set
> > maxResultsPerPage, in Azure client, that means it will return pageable
> > list, now in every pagelist it will return the max blobs that you set
> > earlier in `maxResultsPerPage` which is IMO pretty weird from Azure's
> side.
> > Now in the Camel component side, we just collect all that list of blobs
> in
> > the pages into one list and return that in the body and hence the reason
> > why you see no effects. I hope Azure will improve the API in the future
> to
> > allow alternative options as max results regardless of the pageable list.
> >
> > Regards,
> > Omar
> >
> > On Mon, Jan 25, 2021 at 8:04 PM Mark Andreev 
> > wrote:
> >
> > > Hi, I use Spring 2.4.2 with camel-azure-storage-blob 3.7.0.
> > >
> > > When I set BlobConstants.LIST_BLOB_OPTIONS as "new
> > > ListBlobsOptions().setMaxResultsPerPage(5)" driver returns all entities
> > > from blob.
> > >
> > > from("direct:list")
> > > .routeId("listBlobs")
> > > .setHeader(BlobConstants.LIST_BLOB_OPTIONS)
> > >
> > > .constant(new ListBlobsOptions().setMaxResultsPerPage(5))
> > > .to(
> > > String.format(
> > > "
> > >
> > >
> >
> azure-storage-blob://%s/%s?accessKey=RAW(%s)=listBlobs=true
> > > ",
> > > ACCOUNT,
> > > BLOB_CONTAINER_NAME,
> > > ACCESS_KEY
> > > )
> > > )
> > > .process(exchange -> {
> > > int itemsCount = exchange.getIn().getBody(List.class).size();
> > > log.info(String.format("Items count = '%d'", itemsCount));
> > > });
> > >
> > > How should I use this API to fetch only 5 items?
> > >
> > > Full example:
> > >
> > >
> >
> https://github.com/mrk-andreev/example-camel-azure-blob-pagination-bug/blob/master/src/main/java/name/mrkandreev/camel/MyApplication.java
> > > --
> > > Best regards,
> > > Mark Andreev
> > >
> >
>
>
> --
> Best regards,
> Mark Andreev
>

Re: Azure Blob ignores LIST_BLOB_OPTIONS

[Omar Al-Safi]

Hi,

Actually you are doing nothing wrong here. Apparently when you set
maxResultsPerPage, in Azure client, that means it will return pageable
list, now in every pagelist it will return the max blobs that you set
earlier in `maxResultsPerPage` which is IMO pretty weird from Azure's side.
Now in the Camel component side, we just collect all that list of blobs in
the pages into one list and return that in the body and hence the reason
why you see no effects. I hope Azure will improve the API in the future to
allow alternative options as max results regardless of the pageable list.

Regards,
Omar

On Mon, Jan 25, 2021 at 8:04 PM Mark Andreev  wrote:

> Hi, I use Spring 2.4.2 with camel-azure-storage-blob 3.7.0.
>
> When I set BlobConstants.LIST_BLOB_OPTIONS as "new
> ListBlobsOptions().setMaxResultsPerPage(5)" driver returns all entities
> from blob.
>
> from("direct:list")
> .routeId("listBlobs")
> .setHeader(BlobConstants.LIST_BLOB_OPTIONS)
>
> .constant(new ListBlobsOptions().setMaxResultsPerPage(5))
> .to(
> String.format(
> "
>
> azure-storage-blob://%s/%s?accessKey=RAW(%s)=listBlobs=true
> ",
> ACCOUNT,
> BLOB_CONTAINER_NAME,
> ACCESS_KEY
> )
> )
> .process(exchange -> {
> int itemsCount = exchange.getIn().getBody(List.class).size();
> log.info(String.format("Items count = '%d'", itemsCount));
> });
>
> How should I use this API to fetch only 5 items?
>
> Full example:
>
> https://github.com/mrk-andreev/example-camel-azure-blob-pagination-bug/blob/master/src/main/java/name/mrkandreev/camel/MyApplication.java
> --
> Best regards,
> Mark Andreev
>

Re: Camel-K / Kafka / Avro: something broke overnight

[Omar Al-Safi]

Hi Mark,

indeed it looks like the additionalProperties option is broken in 3.7.0. I
am pushing a fix for both branches, 3.7.x and master branch.

Regards,
Omar


On Thu, Jan 14, 2021 at 6:41 PM Andrea Cosentino  wrote:

> I don't think it is feasible from our side to get to 3.6.0, it's up to you.
>
> Can you try to set the additionalProperties at component level like this
> example?
>
>
> https://github.com/apache/camel/blob/master/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/KafkaComponentTest.java#L63
>
> Il giorno gio 14 gen 2021 alle ore 17:28 mark  ha
> scritto:
>
> > Hello,
> > Until yesterday we were using Camel-K to send Avro-encoded messages to
> > Kafka using code of the form,
> >
> > .to("kafka:{{topic}}?brokers={{kafka-bootstrap}}"
> >   +
> >
> >
> "=org.apache.kafka.common.serialization.StringSerializer"
> >   +
> > "=io.apicurio.registry.utils.serde.AvroKafkaSerializer"
> >   + "=" +
> > schemaRegistry
> >
> > Running this today, our Integration had picked up camel-kafka 3.7.0 and
> > issue https://issues.apache.org/jira/browse/CAMEL-15770 which changed
> the
> > first two properties to keySerializer and valueSerializer. However
> > adjusting the code to,
> >
> > .to("kafka:{{topic}}?brokers={{kafka-bootstrap}}"
> > +
> > "=org.apache.kafka.common.serialization.StringSerializer"
> > +
> > "=io.apicurio.registry.utils.serde.AvroKafkaSerializer"
> > + "=" + schemaRegistry
> >
> > Leaves us no longer able to set additionalProperties. Attempts such as
> > adding
> >
> > -d mvn:org.apache.camel/camel-kafka:[3.6.0]
> >
> > have no effect: we always get camel-kafka 3.7 and cannot pin the
> > Integration to 3.6 or below. The relevant stack trace is below - please
> can
> > anyone help us either adjust the apicurio.registry.url for the new
> library,
> > or pin things back to 3.6 please?
> >
> > [1] Caused by: org.apache.kafka.common.KafkaException: Failed to
> construct
> > kafka producer
> > [1] at
> >
> >
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:434)
> > [1] at
> >
> >
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:298)
> > [1] at
> >
> >
> org.apache.camel.component.kafka.KafkaProducer.doStart(KafkaProducer.java:115)
> > [1] at
> > org.apache.camel.support.service.BaseService.start(BaseService.java:115)
> > [1] at
> >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:84)
> > [1] at
> >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.internalAddService(AbstractCamelContext.java:1425)
> > [1] at
> >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.addService(AbstractCamelContext.java:1343)
> > [1] at
> > org.apache.camel.processor.SendProcessor.doStart(SendProcessor.java:236)
> > [1] at
> > org.apache.camel.support.service.BaseService.start(BaseService.java:115)
> > [1] at
> >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:84)
> > [1] at
> >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:101)
> > [1] at
> >
> >
> org.apache.camel.processor.errorhandler.RedeliveryErrorHandler.doStart(RedeliveryErrorHandler.java:1487)
> > [1] at
> >
> >
> org.apache.camel.support.ChildServiceSupport.start(ChildServiceSupport.java:60)
> > [1] ... 37 more
> > [1] Caused by: java.lang.IllegalArgumentException: Missing registry base
> > url, set apicurio.registry.url
> > [1] at
> >
> >
> io.apicurio.registry.utils.serde.AbstractKafkaSerDe.configure(AbstractKafkaSerDe.java:120)
> > [1] at
> >
> >
> io.apicurio.registry.utils.serde.AbstractKafkaStrategyAwareSerDe.configure(AbstractKafkaStrategyAwareSerDe.java:75)
> > [1] at
> >
> >
> io.apicurio.registry.utils.serde.AvroKafkaSerializer.configure(AvroKafkaSerializer.java:78)
> > [1] at
> >
> >
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:369)
> > [1] ... 49 more
> >
> > My dependencies are currently,
> >
> > // camel-k:
> > dependency=mvn:org.apache.camel.quarkus/camel-quarkus-kafka:1.5.0
> > // camel-k: dependency=mvn:org.apache.avro/avro:1.10.1
> > // camel-k: dependency=mvn:org.glassfish.jersey.core/jersey-common:2.22.2
> > // camel-k:
> > dependency=mvn:io.apicurio/apicurio-registry-utils-serde:1.3.2.Final
> >
> > Many thanks in advance,
> > Regards,
> >
> > Mark
> >
>

Re: How to produce messages to Azure SQL DB in Camel

[Omar Al-Safi]

Hi,

As mentioned here
<https://camel.apache.org/components/latest/sql-component.html>, both use
JDBC driver meanwhile Camel SQL provides better support for SQL queries in
the endpoint. Personally, it is more what you'd feel more comfortable with,
try both and see what component that can fit your needs better.

Regards,
Omar

On Fri, Dec 11, 2020 at 6:58 PM SRIKANT MVS  wrote:

> Thanks for your reply, how better is it to use the Camel SQL compared to
> Camel JDBC.
>
> -Regards
> Srikant
>
> On Thu, 10 Dec, 2020, 10:53 Omar Al-Safi,  wrote:
>
> > Hi,
> >
> > To write your events to your target Azure SQL server database, you can
> > use Camel
> > JDBC <https://camel.apache.org/components/latest/jdbc-component.html>
> > component since Azure SQL supports JDBC uses JDBC driver to connect to
> the
> > database.
> > Plus Camel Debezium SQL Server, I'd recommend using something like Azure
> > Eventhub as event messaging medium in order to retain the events for some
> > period of time in case of failures .. etc, hence your pipeline may look
> > like this:
> > Azure SQL DB (live) -> Camel Debezium SQL Server -> Azure Eventhub ->
> Camel
> > JDBC -> Azure SQL DB (replication).
> >
> > There are few examples that can help you to get started:
> >
> >- Debezium with AWS Kinesis and Cassandra Example
> ><
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium
> > >
> >.
> >- Debezium with Azure Eventhubs with Azure Storage Blob Example.
> ><
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium-eventhubs-blob
> > >
> >- Camel JDBC Example
> ><
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-jdbc
> > >
> >.
> >
> > I hope it helps!
> >
> > Regards,
> > Omar
> >
> > On Thu, Dec 10, 2020 at 8:47 AM SRIKANT MVS 
> wrote:
> >
> > > Team,
> > > I have a usecase where I need to replicate data for a specific duration
> > > from one Azure SQL DB (subscription1) to another Azure SQL DB
> > > (supbscription2).
> > >
> > > The source DB has live data with continuous inserts, updates and
> deletes
> > > possibility. Henceforth, we will miss the updates while performing DB
> > copy
> > > operations.
> > >
> > > For consuming the messages, I am considering to use Debizium SQL Server
> > > Connector
> > > <
> > >
> >
> https://camel.apache.org/components/latest/debezium-sqlserver-component.html
> > > >,
> > > but this component
> > >
> > > *supports only the consumer functionality*
> > >
> > > Which camel component can I use to produce the same event to the
> > > destination DB ??
> > >
> > > -Srikant
> > >
> >
>

Re: How to produce messages to Azure SQL DB in Camel

[Omar Al-Safi]

Hi,

To write your events to your target Azure SQL server database, you can
use Camel
JDBC 
component since Azure SQL supports JDBC uses JDBC driver to connect to the
database.
Plus Camel Debezium SQL Server, I'd recommend using something like Azure
Eventhub as event messaging medium in order to retain the events for some
period of time in case of failures .. etc, hence your pipeline may look
like this:
Azure SQL DB (live) -> Camel Debezium SQL Server -> Azure Eventhub -> Camel
JDBC -> Azure SQL DB (replication).

There are few examples that can help you to get started:

   - Debezium with AWS Kinesis and Cassandra Example
   

   .
   - Debezium with Azure Eventhubs with Azure Storage Blob Example.
   

   - Camel JDBC Example
   

   .

I hope it helps!

Regards,
Omar

On Thu, Dec 10, 2020 at 8:47 AM SRIKANT MVS  wrote:

> Team,
> I have a usecase where I need to replicate data for a specific duration
> from one Azure SQL DB (subscription1) to another Azure SQL DB
> (supbscription2).
>
> The source DB has live data with continuous inserts, updates and deletes
> possibility. Henceforth, we will miss the updates while performing DB copy
> operations.
>
> For consuming the messages, I am considering to use Debizium SQL Server
> Connector
> <
> https://camel.apache.org/components/latest/debezium-sqlserver-component.html
> >,
> but this component
>
> *supports only the consumer functionality*
>
> Which camel component can I use to produce the same event to the
> destination DB ??
>
> -Srikant
>

Re: Camel Quarkus and Picocli

[Omar Al-Safi]

Hi Mikael,

I did something before in Picocli, So basically, in the call method that
implements Callable,

I guess here you will need start your Camel Quarkus context. I hope it
helps.

Regards,
Omar

On Tue, Dec 1, 2020 at 8:35 PM Mikael Andersson Wigander
 wrote:

> Fooled myself. Thought it worked but it doesn't.
> Back to the original question: how get a Picocli work with a Camel in
> Quarkus
>
> /M
>
> ‐‐‐ Original Message ‐‐‐
>
> On Tuesday, December 1st, 2020 at 19:49, Mikael Andersson Wigander <
> mikael.andersson.wigan...@pm.me> wrote:
>
> > Nevermind, I just removed the class with QuarkusMain and adding Picocli
> to the Camel component instead
> >
> > /M
> >
> > ‐‐‐ Original Message ‐‐‐
> >
> > On Tuesday, December 1st, 2020 at 19:34, Mikael Andersson Wigander
> mikael.andersson.wigan...@pm.me.INVALID wrote:
> >
> > > Anyone managed to create an application in Quarkus with Camel and
> Picocli?
> > >
> > > The different types of command modes makes everything confusing.
> > >
> > > Thx
> > >
> > > /M
>

Re: [ANNOUNCEMENT] Migration Tool Camel 2 --> Camel 3

[Omar Al-Safi]

Hi John,

Awesome to see this and I think it is very helpful! Thank you .

Regards,
Omar

On Thu, Sep 3, 2020 at 9:33 AM Claus Ibsen  wrote:

> Hi John
>
> Thanks for sharing this. Its great to see other kind of tooling for
> Apache Camel.
> Great and detailed blog post. And good to hear there will be added
> more rules for the upcoming Camel 3.x releases.
>
> I added a link to your blog from the tooling section at
> https://camel.apache.org/community/articles/
>
> To help spread the word about this tool, I wonder if it would be
> worthwhile and you would be willing to post the blog on the Camel
> website in its blog system.
> Then it will be shown on the front page under news.
>
> The blog system uses mark down format, and you basically do a PR
> against camel-website
> https://github.com/apache/camel-website
>
> On Wed, Sep 2, 2020 at 11:47 AM John Poth  wrote:
> >
> > Hello everyone,
> >
> > Matej and I have been working on Windup rules [0] to help people estimate
> > their migration from Camel 2 to Camel 3. When the tool is run it will
> > generate a report listing the different issues and assign story points to
> > each issue. Here are some sample reports for source codes of Camel books
> if
> > you want to check it out:
> >
> > Camel In Action, 2nd Edition:
> > http://windup.surge.sh/cia2/reports/migration_issues.html
> > Apache Camel Developer's Cookbook, 2nd Edition:
> > http://windup.surge.sh/cookbook/reports/migration_issues.html
> > Mastering Apache Camel:
> > http://windup.surge.sh/mastering/reports/migration_issues.html
> >
> > If you are really interested in this, I wrote a blogpost about it [1]
> >
> > Thanks!
> >
> > [0] https://github.com/windup/windup
> > [1]
> >
> https://jpoth.blogspot.com/2020/09/upgrading-to-camel-300-with-windup.html
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: Camel website: documentation or projects

[Omar Al-Safi]

Indeed, I think documentation serves what I would expect from that page.

Regards,
Omar

On Thu, Jun 18, 2020 at 8:59 AM Ajmera, Hemang C 
wrote:

> Hi
>Earlier in UI the mouse over on menu would should be direct link for
> Components documents Now mouse over is not working. May be we can
> enable that once we migrate to one page... That will help in quick
> navigation.
>
>
> Thanks and Regards,
> Hemang Ajmera
>
> -Original Message-
> From: Jean-Baptiste Onofre 
> Sent: 18 June 2020 12:23
> To: users@camel.apache.org
> Cc: Dev 
> Subject: Re: Camel website: documentation or projects
>
> I agree with Andrea. Single page and links to docs make sense.
>
> Regards
> JB
>
> > Le 18 juin 2020 à 08:51, Andrea Cosentino  a écrit :
> >
> > I'd say this should be under Documentation and I'd expect to find
> > description of projects and links to docs, I guess.
> >
> > One page it's for sure more clear than two.
> >
> > Thanks for your time on this :-)
> >
> > Il giorno mer 17 giu 2020 alle ore 23:05 Zoran Regvart
> >  ha scritto:
> >
> >> Hi Cameleers,
> >> we have a discussion on pull request #397[1] about merging the
> >> Documentation and Projects sections on the website. This came about
> >> from a fair bit of duplication we currently have between those two
> >> pages and we're trying to consolidate and simplify.
> >>
> >> We're looking for community feedback on this:
> >> - would you expect this unified page to be under "Projects" or under
> >> "Documentation"?
> >> - what do you expect to find on this page (description, links, etc?)
> >>
> >> thanks :)
> >>
> >> zoran
> >>
> >> [1] https://github.com/apache/camel-website/pull/397
> >> --
> >> Zoran Regvart
> >>
>
>

Re: Passwords in Camel endpoint URIs and limitations of RAW syntax

[Omar Al-Safi]

Hi Florian,

As Fabry mentioned, it would be worth checking to use config file to
achieve this. Here is an example:
https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium

Regards,
Omar

On Thu, Jun 4, 2020 at 10:56 AM FabryProg  wrote:

> Hello Florian,
>
> Did you try to save the password into a variable / parameter / config file
> and lookup it into the URI?
>
> Kind regards!
>
> Il giorno gio 4 giu 2020 alle ore 10:50 Florian Patzl <
> florian.pa...@evolit.com> ha scritto:
>
> > Hello Ralf,
> > thanks for your response. No, I didn't mention that in my description.
> :-)
> > URL encoding would be my preferred solution, too, but unfortunately that
> > does not seem to prevent the problems with passwords containing ")&".
> > Unless something about my encoding is wrong.
> >
> > For example, given a password "pwd)=b", both with and without RAW(...)
> > the result is wrong:
> >
> > TRACE o.a.c.i.engine.AbstractCamelContext - Getting endpoint with raw
> uri:
> > pop3://localhost:3110/?username=test2=RAW%28pwd%29%26a%3Db%29,
> > normalized uri:
> > pop3://localhost:3110/?a=b%29=RAW(pwd)=test2
> >
> > TRACE o.a.c.i.engine.AbstractCamelContext - Getting endpoint with raw
> uri:
> > pop3://localhost:3110/?username=test2=pwd%29%26a%3Db, normalized
> > uri: pop3://localhost:3110/?a=b=pwd%29=test2
> >
> > I'm currently testing on 3.3.0.
> >
> > Best Regards,
> > Florian
> > 
> > From: Claussnitzer, Ralf 
> > Sent: Thursday, June 4, 2020 08:13
> > To: users@camel.apache.org 
> > Subject: Re: Passwords in Camel endpoint URIs and limitations of RAW
> syntax
> >
> > Hi Florian,
> >
> > I may have missed the answer to my questions in your detailed problem
> > description. But how is this not solved by URL-Encoding?
> > There was once a bug with URL encodings in Camel. Does this bug still
> > exist? What version of Camel are you using?
> >
> > -Ralf
> > 
> > From: Florian Patzl 
> > Sent: Wednesday, June 3, 2020 2:49 PM
> > To: users@camel.apache.org
> > Subject: Passwords in Camel endpoint URIs and limitations of RAW syntax
> >
> > Hello,
> > I'm trying to figure out the best way to handle passwords in Camel
> > endpoint URIs in my application.
> > I know the topic has been cause for Stack Overflow posts, JIRA entries
> and
> > mails but I'm still not sure I've got everything right.
> > Sorry for the big wall of text, but I think I should explain what exactly
> > I've tried and found out on the topic.
> >
> > The main problem is that the reserved URI characters '+' and '&' (plus
> and
> > ampersand) cause parsing problems in Camel endpoint URIs.
> > '+' is replaced by a blank, and '&' is treated as the delimiter to the
> > next parameter.
> > An example URI for the password "pwd2+2":
> > pop3://localhost:3110/?username=test2=pwd2%2B2
> >
> > A relevant post is here:
> >
> >
> https://stackoverflow.com/questions/11018987/camel-how-to-include-an-ampersand-as-data-in-a-uri-not-as-a-delimiter/34926623#34926623
> >
> >
> > Now, the solution in documentation is using the RAW(...) syntax:
> >
> >
> https://camel.apache.org/manual/latest/faq/how-do-i-configure-endpoints.html#HowdoIconfigureendpoints-Configuringparametervaluesusingrawvalues
> > So for example:
> > pop3://localhost:3110/?username=test2=RAW(pwd2+2)
> >
> > Using that feature means we can no longer treat Camel URIs as pure URIs
> in
> > our application, because the '+' of the password must not be escaped for
> > this to work.
> > But if there's no way around that, we can deal with that.
> >
> > However, when trying the limits of the RAW(...) syntax, we noticed that
> it
> > can not parse passwords that contain ')&'.
> > This was covered in the following JIRA issue, and since then there is
> > support for an alternative syntax using curly braces: RAW{...}, that I
> > didn't find in documentation:
> > https://issues.apache.org/jira/browse/CAMEL-12982
> > The last comment provides a pretty detailed summary of the options and
> > limits.
> >
> >
> > The alternative RAW{...} syntax works fine, except for a minor flaw: It
> > breaks URI sanitizing.
> > For example, Camel leaks the '&2' portion of the password 'pwd2&2' in log
> > entries like:
> > pop3://localhost:3110/?password=xx&2%7D=test2
> >
> > The same problem existed for the RAW(...) syntax:
> > https://issues.apache.org/jira/browse/CAMEL-11269
> > So the fix should be rather simple, I will check whether there's already
> > an issue for that and might even be able to submit a PR for that.
> >
> > But, more importantly: By checking the passwords for ')&' and '}&' and
> > dynamically deciding the RAW syntax to use, we should be able to support
> > any password *except* if they contain both ')&' and '}&'.
> > That is a weird constraint for passwords, but should be justifiable as
> > technical limitation.
> >
> >
> > As an alternative to all of this, I sometimes saw the suggestion to
> > configure the component 

Re: Camel Salesforce Component can't find Platform Event object

[Omar Al-Safi]

Hi Anne,

Which Camel version do you use here?

Regards,
Omar

On Thu, May 21, 2020 at 8:47 AM Anne Nguyen  wrote:

> 0
>
> I have used camel-salesforce-maven-plugin to generate the DTO and have
> successfully created a Contact record. However, when I tried to emit an
> event following the docs on:
>
> https://camel.apache.org/components/latest/salesforce-component.html
>
> I received the following error:
>
> 2020-05-21 13:30:47.844 ERROR 4276 --- [ent@3c50ad4b-22]
> o.a.c.p.e.DefaultErrorHandler: Failed delivery for (MessageId:
> ID-DESKTOP-UO2CP5I-1590033642730-0-7 on ExchangeId:
> ID-DESKTOP-UO2CP5I-1590033642730-0-7). Exhausted after delivery attempt: 1
> caught: org.apache.camel.component.salesforce.api.NoSuchSObjectException:
> {errors:[{"errorCode":"NOT_FOUND","message":"The requested resource does
> not exist","fields":null}],statusCode:404}
>
> Message History (complete message history is disabled)
>
> ---
> RouteId  ProcessorId  Processor
>Elapsed (ms)
> [moodle-event  ] [moodle-event  ] [from[timer://hello?period=2000]
>   ] [81]
> ...
> [moodle-event  ] [to3   ]
> [salesforce:createSObject?sObjectName=Moodle__e
>] [ 0]
>
> Stacktrace
>
> ---
>
> org.apache.camel.component.salesforce.api.NoSuchSObjectException:
> {errors:[{"errorCode":"NOT_FOUND","message":"The requested resource does
> not exist","fields":null}],statusCode:404}
> at
>
> org.apache.camel.component.salesforce.internal.client.DefaultRestClient.createRestException(DefaultRestClient.java:110)
> ~[camel-salesforce-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
>
> org.apache.camel.component.salesforce.internal.client.AbstractClientBase$1.onComplete(AbstractClientBase.java:231)
> ~[camel-salesforce-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
> at
>
> org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:218)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:210)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:543)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:523)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.HttpReceiver.responseSuccess(HttpReceiver.java:486)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.messageComplete(HttpReceiverOverHTTP.java:326)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
> org.eclipse.jetty.http.HttpParser.parseFields(HttpParser.java:1161)
> ~[jetty-http-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1534)
> ~[jetty-http-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.parse(HttpReceiverOverHTTP.java:200)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:141)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:75)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:133)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:156)
> ~[jetty-client-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
> org.eclipse.jetty.io
> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> ~[jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> ~[jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
>
> org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:543)
> ~[jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
> org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:398)
> ~[jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
> org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
> ~[jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
> at
> 

Re: Google pubsub component broken in 3.2.0 ?

[Omar Al-Safi]

Hi Jean,

I recall there was major work being done on this component and indeed, it
looks to me like a bug with the associated work that was being done. Can
you please file this issue as bug report under
https://issues.apache.org/jira/browse/CAMEL in order to be investigated
further?

Regards,
Omar

On Thu, May 14, 2020 at 12:58 PM Jean-Pierre Matsumoto 
wrote:

> Hi all
>
> Previously I used Camel 3.0.0 for my mixture of google-pubsub and
> google-mail routes. I've tried to update to Camel 3.2.0 but pubsub part is
> not working, with no error message.
>
> I've traced with debugger. I think the issue is with bad version of Guava
> library. Anyway, here is a simple route to reproduce:
>
> @Component
> public class SimplePubsubRouteBuilder extends RouteBuilder {
>   @Override
>   public void configure() throws Exception {
> from("google-pubsub://project-id:subs-name")
>   .to("log:myLog?showHeaders=true");
>   }
> }
>
> Result with Camel 3.2.0: nothing happens, no errors even with DEBUG log
> level.
>
> Result with Camel 3.1.0: error 404 as expected since Pubsub subscription
> path is invalid.
>
> My platform: macOS 10.15.4 with openjdk 11.0.6 2020-01-14 (but I've been
> able to reproduce on Ubuntu 16, same jdk).
>
> I've tried to fix the issue myself with no success.
>
> Let me know if I can help more.
>
> Thanks
> Jean-Pierre
>

Re: Camel-kafka integration issue

[Omar Al-Safi]

You can take a look at this example

which
explains this in good details.

Regards,
Omar

On Tue, Mar 31, 2020 at 2:20 PM Andrea Cosentino  wrote:

> You need to send something to direct. As it is you're not sending messages
> to your route.
>
> Use a timer.
>
> Il giorno mar 31 mar 2020 alle ore 14:18 Sanu Sandesh <
> muralisandes...@gmail.com> ha scritto:
>
> > Team,
> >
> > I am trying to integrate Camel with Kafka. Written simple application
> with
> > camelcontext initialized in main() program & added my routebuilder with
> > below code:
> >
> >  from("direct:start")
> > .setBody(constant("Message from Camel")) // Message to
> send
> > .setHeader(KafkaConstants.KEY, constant("Camel")) // Key
> of
> > the message
> > .to("kafka:test?brokers=localhost:2181");
> >
> > where my topic is : test
> >
> > This code not sending message to my kafka consumer. where as plain kafka
> > code is sending. I am trying to see consumer using CLI.
> >
> > Need assistance pls.
> >
> > REgards,
> > Sandesh
> >
>

Re: Camel + Drill + Parquet

[Omar Al-Safi]

No worries Ron! And enjoy the ride :)

Regards,
Omar

On Thu, Feb 13, 2020 at 6:39 AM Ron Cecchini 
wrote:

> Thanks, Omar.
>
> As it turns out, Parquet is not the way to go since it looks like it is
> geared more toward data warehousing, whereas I need to persist streaming
> data - and from what I can gather, I would need the overhead of Spark or
> Hive to accomplish that with Parquet (appending to a growing Parquet file).
>
> *However*, it looks like Apache Kudu is exactly what we need.  And not
> only does Camel already provide a Kudu component, as coincidence would have
> it it looks like you co-authored.  Awesome!
>
> Moreover, Kudu takes just a Map as input, and not an Avro formatted
> message or whatever like Parquet.  So migrating this Kafka->Mongo route to
> Kafka->Kudu is almost trivial.
>
> Anyway, time to bump up my Camel version to 3.0.1 and give Kudu a whirl...
>
> Thanks again.
>
> > On February 12, 2020 at 4:33 AM Omar Al-Safi  wrote:
> >
> >
> > Hi Ron,
> >
> > By reading some introduction in Apache Drill, I'd say the file component
> > would be more suitable to write parquet files.
> > In regards to parquet and Camel, we don't have an example for it but the
> > way I see it, you are heading into the right direction by creating a
> > processor to convert the data to parquet format.
> > However, we do have an open feature request
> > <https://issues.apache.org/jira/browse/CAMEL-13573> to add parquet data
> > format, we would love to see some contributions to add this to Camel :) .
> >
> > Regards,
> > Omar
> >
> >
> > On Tue, Feb 11, 2020 at 11:37 PM Ron Cecchini 
> > wrote:
> >
> > > Hi, all.  I'm just looking for quick guidance or confirmation that I'm
> > > going in the right direction here:
> > >
> > > - There's a small Kotlin service that uses Camel to read from Kafka and
> > > write to Mongo.
> > > - I need to replace Mongo with Apache Drill and write Parquet files to
> the
> > > file system.
> > >   (I know nothing about Parquet but I know a little bit about Drill.)
> > >
> > > - This service isn't used to do any queries, it's just for persisting
> data.
> > >   So, given that, and the fact that Drill is just a query engine, I
> really
> > > can't use the "Drill" component for anything.
> > >
> > > - But there is that "HDFS" component that I think I can use?
> > >   Or maybe the "File" component is better here?
> > >
> > > So my thinking is that I just need to:
> > >
> > > 1. write a Processor to transform the JSON data into Parquet
> > >(and keep in mind that I know nothing about Parquet...)
> > >
> > > 2. use the HDFS (or File) component to write it to a file
> > >(I think there's some Parquet set up to do (?) outside the scope of
> > > this service, but that's another matter...)
> > >
> > > Seems pretty straight-forward.  Does that sound reasonable?
> > >
> > > Are there any Camel examples I can look at?  The Google machine seems
> to
> > > not find anything related to Camel and Parquet...
> > >
> > > Thank you so much!
> > >
> > > Ron
> > >
>

Re: Camel + Drill + Parquet

[Omar Al-Safi]

Hi Ron,

By reading some introduction in Apache Drill, I'd say the file component
would be more suitable to write parquet files.
In regards to parquet and Camel, we don't have an example for it but the
way I see it, you are heading into the right direction by creating a
processor to convert the data to parquet format.
However, we do have an open feature request
 to add parquet data
format, we would love to see some contributions to add this to Camel :) .

Regards,
Omar


On Tue, Feb 11, 2020 at 11:37 PM Ron Cecchini 
wrote:

> Hi, all.  I'm just looking for quick guidance or confirmation that I'm
> going in the right direction here:
>
> - There's a small Kotlin service that uses Camel to read from Kafka and
> write to Mongo.
> - I need to replace Mongo with Apache Drill and write Parquet files to the
> file system.
>   (I know nothing about Parquet but I know a little bit about Drill.)
>
> - This service isn't used to do any queries, it's just for persisting data.
>   So, given that, and the fact that Drill is just a query engine, I really
> can't use the "Drill" component for anything.
>
> - But there is that "HDFS" component that I think I can use?
>   Or maybe the "File" component is better here?
>
> So my thinking is that I just need to:
>
> 1. write a Processor to transform the JSON data into Parquet
>(and keep in mind that I know nothing about Parquet...)
>
> 2. use the HDFS (or File) component to write it to a file
>(I think there's some Parquet set up to do (?) outside the scope of
> this service, but that's another matter...)
>
> Seems pretty straight-forward.  Does that sound reasonable?
>
> Are there any Camel examples I can look at?  The Google machine seems to
> not find anything related to Camel and Parquet...
>
> Thank you so much!
>
> Ron
>

Re: Proposal : camel-rabbitmq : RabbitMQProducer to override EXCHANGE_NAME/ROUTING_KEY headers if bridgeEndpoint

[Omar Al-Safi]

Hi,

You are welcome to do so. Please raise a JIRA ticket and then send your PR.

Regards,
Omar

On Mon, Jan 20, 2020 at 4:15 PM Jonathan Schoreels <
jonathan.schore...@gmail.com> wrote:

> Hi,
>
> Currently, the camel-rabbitmq endpoints, if set as "bridgeEndpoint",
> ignores the following exchange's headers : EXCHANGE_NAME & ROUTING_KEY, but
> it keeps the previous headers setted, which leak information about the
> origin of the event. For example, if my message comes from Exchange A, I
> transform it and send it to Exchange B, the RabbitMQProducer will ignore
> the headers set by A but still will send them to B, which could leaks some
> internal info of A to the consumer of B.
>
> Wouldn't it be more correct to replace those two headers, if we ignore them
> ? I can do the pull request, but I would like to discuss the idea before.
>
> See RabbitMQProducer:processInOut and processInOnly for related code.
>

Re: Micrometer/Prometheus Metric Count

[Omar Al-Safi]

Hi Steve,

in regards to the JIRA issue you added here, it looks like these metrics
are for MicroProfile component (
https://camel.apache.org/components/latest/microprofile-metrics-component.html),
perhaps you can try with microprofile component instead?

Regards,
Omar

On Sat, Jan 18, 2020 at 12:16 AM Steve Davids  wrote:

> I have recently migrated to Camel 3.0 (using Spring Boot) and found that
> there was a change in behavior for our metrics which drives some dashboards
> which shows the number of requests in our primary "process" exchange. The
> old code performed a prometheus query which counted the sum of  an earlier
> stage's CamelMessageHistory_seconds_count and compared it to the count of
> the same metric of the primary "process" stage. Camel 2.x seemed to only
> increment the count upon exit of the stage while Camel 3.x seems to
> increment the count upon starting the stage. This nuance throws off our
> timing graphing and we can't simply choose the "next" component in the
> chain because there is global exception handling that might be triggered
> from the process which wouldn't allow us to watch a single component "next
> in line".
>
> Taking a step back, I would like to find out what the best way to get a
> simple count of "in flight" count into Micrometer/Prometheus. Ideally Camel
> would provide this functionality natively and would allow me to filter on
> the nodeId of the process I'm interested in, but that metric doesn't seem
> to be exposed (though it seems to be exposed for other metrics components
> ). I also tried to do:
>
> > .doTry()
> > .toF("micrometer:counter:%s", INFLIGHT_METRIC_NAME)
> > .process(handler)
> > .doFinally()
> > .toF("micrometer:counter:%s?decrement=1", INFLIGHT_METRIC_NAME)
> > .end()
> >
>
> But with that configuration my global exception handling breaks for the
> `process(handler)` code. Does anyone have suggestions as to how to
> accomplish this or if Camel should natively support this use-case?
>
> Thanks,
>
> -Steve
>

Re: [ANNOUNCEMENT] Camel at FOSDEM 2020

[Omar Al-Safi]

Thank you Zoran for organizing. Looking forward to the event!

Regards,
Omar

On Mon, Jan 13, 2020 at 3:31 PM Zoran Regvart  wrote:

> Hi Cameleers,
> short update on the social event. It'll be at A La Mort Subite[1], rue
> Montagne-aux-Herbes Potagères 7 in Brussels[2] on Saturday 1st of February
> first day of FOSDEM (ULB, Av. F. D. Roosevelt 50) from 20h onwards. It's in
> the city center, about 30-40 mins from FOSDEM by public transport[3].
>
> zoran
>
> [1] https://www.alamortsubite.com/en/
> [2] https://www.openstreetmap.org/node/4480414658
> [3] www.stib-mivb.be/tripplanner/
>
>
> On Wed, Jan 8, 2020 at 3:01 PM Zoran Regvart  wrote:
>
> > Hi Cameleers,
> > FOSDEM[1] is free as in beer grassroots event for free software/open
> > source communities to meet, share ideas and collaborate. You'll find
> > over 800 talks spread over 35 rooms of Université libre de Bruxelles
> > campus over the first weekend of February. And building on last year's
> > well-received BoF session Camel is returning to FOSDEM.
> >
> > FOSDEM 2020 is on 1st (from 9:30 to 19:00) and 2nd (from 9:00 to
> > 18:00) of February.
> >
> > Aside from all the interesting talks[2], your favorite even-toed
> > ungulate is going to be represented there.
> >
> > First, at the Integration Process Community stand[3] in the building K
> > both days of the conference. Come by to say hi, learn from the
> > Cameleers about all the new and exciting developments on Camel 3,
> > Camel K, Camel Quarkus and Camel Kafka Connector. There might even be
> > some Camel swag for you to grab :)
> >
> > The Camel BoF session at FOSDEM 2019 was very well received last year
> > and we're repeating it: the Camel birds of a feather[4] will be on
> > Saturday from 11 to 12 in the room H.3242. Let's meet, discuss and
> > exchange ideas. Anyone wishing to share can present on any topic
> > relating to Apache Camel is welcome to do so, though please be mindful
> > of the time - we have one hour in this session and would like that
> > everyone would have the chance to participate.
> >
> > Lastly, we're planning on having a social event on Saturday a chance
> > to wind down and discuss more in one of many Belgian pubs. Watch this
> > space for details on that.
> >
> > I hope to see many of you at FOSDEM 2020!
> >
> > zoran
> >
> > p.s. this is text is also available on the website[5]
> >
> > [1] https://fosdem.org/2020/about/
> > [2] https://fosdem.org/2020/schedule/
> > [3] https://fosdem.org/2020/stands/
> > [4] https://fosdem.org/2020/schedule/event/bof_apache_camel/
> > [5] https://camel.apache.org/blog/FOSDEM-2020/
> > --
> > Zoran Regvart
> >
>
>
> --
> Zoran Regvart
>

Re: Stack trace after upgrading to Camel 3.0

[Omar Al-Safi]

xchange;
> > >>>>> import org.apache.camel.builder.RouteBuilder;
> > >>>>>
> > >>>>>
> > >>>>> public class APIRoute extends RouteBuilder {
> > >>>>>
> > >>>>> @Override
> > >>>>> public void configure() throws Exception {
> > >>>>>
> > >>>>> from("direct:apiCall")
> > >>>>> .log("URL -> ${header.apiURL}")
> > >>>>> .to("log:?level=INFO=true")
> > >>>>> .setHeader(Exchange.HTTP_METHOD, constant("GET"))
> > >>>>> .setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
> > >>>>> .toD("${header.apiURL}")
> > >>>>> .to("log:?level=INFO=true");
> > >>>>>
> > >>>>> }
> > >>>>>
> > >>>>> On Fri, Jan 10, 2020 at 3:54 PM Andrea Cosentino <
> anco...@gmail.com>
> > >>>> wrote:
> > >>>>>> Change camel-core to camel-core-engine.
> > >>>>>>
> > >>>>>> Il giorno ven 10 gen 2020 alle ore 15:48 Tom Coudyzer <
> > >>>> tcdm...@gmail.com>
> > >>>>>> ha scritto:
> > >>>>>>
> > >>>>>>> Hi,
> > >>>>>>>
> > >>>>>>> This is the POM
> > >>>>>>>
> > >>>>>>> http://maven.apache.org/POM/4.0.0;
> > >>>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
> > >>>>>>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> > >>>>>>> http://maven.apache.org/xsd/maven-4.0.0.xsd;>
> > >>>>>>> 4.0.0
> > >>>>>>> com.xyz
> > >>>>>>> camel.samples
> > >>>>>>> 0.0.1-SNAPSHOT
> > >>>>>>> pom
> > >>>>>>>
> > >>>>>>> 
> > >>>>>>>
> UTF-8
> > >>>>>>> 11
> > >>>>>>> 11
> > >>>>>>> 3.0.0
> > >>>>>>> 1.7.29
> > >>>>>>> 5.15.10
> > >>>>>>> 2.10.0
> > >>>>>>> 5.1.6.RELEASE
> > >>>>>>> 2.1.4.RELEASE
> > >>>>>>> 4.12
> > >>>>>>> 1.4.0.Final
> > >>>>>>> 1.5.10
> > >>>>>>> 0.1.1
> > >>>>>>> 
> > >>>>>>>
> > >>>>>>> 
> > >>>>>>>
> > >>>>>>> 
> > >>>>>>> 
> > >>>>>>> org.apache.camel
> > >>>>>>> camel-core
> > >>>>>>> ${camel-version}
> > >>>>>>> 
> > >>>>>>> 
> > >>>>>>> org.apache.camel
> > >>>>>>> camel-jms
> > >>>>>>> ${camel-version}
> > >>>>>>> 
> > >>>>>>> 
> > >>>>>>> org.apache.camel
> > >>>>>>> camel-http
> > >>>>>>> ${camel-version}
> > >>>>>>> 
> > >>>>>>>
> > >>>>>>> 
> > >>>>>>> 
> > >>>>>>> org.apache.activemq
> > >>>>>>> activemq-all
> > >>>>>>> ${activemq-version}
> > >>>>>>> 
> > >>>>>>>
> > >>>>>>> 
> > >>>>>>> com.fasterxml.jackson.core
> > >>>>>>> jackson-databind
> > >>>>>>> ${jackson-version}
> > >>>>>>> 
> > >>>>>>>
> > >>>>>>> 
> > >>>>>>> 
> > >>>>>>> org.springframework
> > >>>>>>> spring-context
> > >>>>>>> ${spring-version}
> > >>>>>>> 
> > >>>>>>> 
> > >>>>>>> org.springframework
> > 

Re: Stack trace after upgrading to Camel 3.0

[Omar Al-Safi]

Hi,

Java 8 should work at the moment. However, at some point in upcoming
versions of 3.x, I think Java 8 support will be dropped.
Referring to your error, looks like your depencides cache are not updated
to camel 3.0, this method only exists in 2.14,x but not in 3.0. You can try
to remove the maven cache folder and rebuild again?

Regards,
Omar

On Fri, Jan 10, 2020 at 3:01 PM Tom Coudyzer  wrote:

> Hi,
>
> Thanks for the reply. Switched to Java 11 (and as told earlier) checked the
> migration guide, but still not working...
>
> Any other ideas?
>
> Thanks a lot!!
>
> Regards,
> /Tom
>
> On Fri, Jan 10, 2020 at 2:52 PM Jeremy Ross 
> wrote:
>
> > Camel 3 requires Java 11, or will soon.
> >
> > Also, make sure you've made necessary changes outlined here
> > https://camel.apache.org/manual/latest/camel-3-migration-guide.html.
> >
> > On Fri, Jan 10, 2020 at 7:49 AM Tom Coudyzer  wrote:
> >
> > > Hi,
> > >
> > > Probably a stupid question but seems can't find a solution. I upgraded
> to
> > > Camel 3.0
> > >
> > > When I run now my test classes (using CamelTestSupport) I get the
> > > stacktrace below
> > > (Note atm still using Java 8)
> > >
> > > Any idea what the reason could be? Couldn't find either anything in the
> > > migration guide in this area.
> > >
> > > Many thanks already and sorry again if this is a stupid question!
> > >
> > > Regards,
> > > /Tom
> > >
> > > Stacktrace
> > >
> > > java.lang.NoSuchMethodError:
> > >
> > >
> >
> org.apache.camel.CamelContext.getManagementMBeanAssembler()Lorg/apache/camel/spi/ManagementMBeanAssembler;
> > > at
> > >
> > >
> >
> org.apache.camel.management.DefaultManagementAgent.doStart(DefaultManagementAgent.java:394)
> > > at
> org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:72)
> > > at
> > >
> > >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:70)
> > > at
> > >
> > >
> >
> org.apache.camel.impl.engine.DefaultManagementStrategy.doStartManagementStrategy(DefaultManagementStrategy.java:200)
> > > at
> > >
> > >
> >
> org.apache.camel.management.JmxManagementStrategy.doStart(JmxManagementStrategy.java:97)
> > > at
> > >
> > >
> >
> org.apache.camel.support.service.ServiceSupport.start(ServiceSupport.java:117)
> > > at
> > >
> > >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.startService(AbstractCamelContext.java:2900)
> > > at
> > >
> > >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.doStartCamel(AbstractCamelContext.java:2522)
> > > at
> > >
> > >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.lambda$doStart$2(AbstractCamelContext.java:2445)
> > > at
> > >
> > >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.doWithDefinedClassLoader(AbstractCamelContext.java:2462)
> > > at
> > >
> > >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.doStart(AbstractCamelContext.java:2443)
> > > at
> > >
> > >
> >
> org.apache.camel.support.service.ServiceSupport.start(ServiceSupport.java:117)
> > > at
> > >
> > >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.start(AbstractCamelContext.java:2352)
> > > at
> > >
> > >
> >
> org.apache.camel.test.junit4.CamelTestSupport.startCamelContext(CamelTestSupport.java:820)
> > > at
> > >
> > >
> >
> org.apache.camel.test.junit4.CamelTestSupport.doSetUp(CamelTestSupport.java:460)
> > > at
> > >
> > >
> >
> org.apache.camel.test.junit4.CamelTestSupport.setUp(CamelTestSupport.java:336)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > > at
> > >
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> > > at
> > >
> > >
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> > > at
> > >
> > >
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> > > at
> > >
> > >
> >
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> > > at
> > >
> >
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> > > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> > > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> > > at
> > >
> > >
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> > > at
> > >
> > >
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> > > at 

Re: building Camel in IntelliJ

[Omar Al-Safi]

Oh yeah I never actually ran mvn:idea, I just open it directly in intellj,
most importantly by submodules. Not sure if that is still valid though

On Thu, Dec 19, 2019, 17:20 Omar Al-Safi  wrote:

> Which camel version are you referring to? Also, I hope you are not opening
> the full project in intellj as that may cause problems. You can try only
> opening the submodules of the project, e.g: only one component to open in
> intellj.
>
> Regards,
> Omar
>
> On Thu, Dec 19, 2019, 17:05 Robin Vanderhallen <
> robinvanderhal...@gmail.com> wrote:
>
>> Hello,
>>
>> I’m having trouble building Camel from source in IntelliJ. I followed the
>> instructions on https://camel.apache.org/manual/latest/building.html
>> However, mvn idea:idea is throwing an exception:
>>
>> Failed to execute goal
>> org.apache.maven.plugins:maven-idea-plugin:2.3-atlassian-10:idea
>> (default-cli) on project camel-parent: Execution default-cli of goal
>> org.apache.maven.plugins:maven-idea-plugin:2.3-atlassian-10:idea failed.:
>> NullPointerException
>>
>> Caused by: java.lang.NullPointerException
>> at org.apache.maven.plugin.idea.IdeaModuleMojo.getLibraryUrl
>> (IdeaModuleMojo.java:1141)
>> at org.apache.maven.plugin.idea.IdeaModuleMojo.rewriteDependencies
>> (IdeaModuleMojo.java:531)
>> at org.apache.maven.plugin.idea.IdeaModuleMojo.rewriteModule
>> (IdeaModuleMojo.java:423)
>> at org.apache.maven.plugin.idea.IdeaMojo.rewriteModule
>> (IdeaMojo.java:236)
>> at org.apache.maven.plugin.idea.IdeaMojo.execute (IdeaMojo.java:216)
>> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
>> (DefaultBuildPluginManager.java:137)
>> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
>> (MojoExecutor.java:210)
>> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
>> (MojoExecutor.java:156)
>> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
>> (MojoExecutor.java:148)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
>> (LifecycleModuleBuilder.java:117)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
>> (LifecycleModuleBuilder.java:81)
>> at
>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
>> (SingleThreadedBuilder.java:56)
>> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
>> (LifecycleStarter.java:128)
>> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
>> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
>> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
>> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
>> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
>> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke
>> (NativeMethodAccessorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke
>> (DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke (Method.java:498)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
>> (Launcher.java:289)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
>> (Launcher.java:229)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
>> (Launcher.java:415)
>> at org.codehaus.plexus.classworlds.launcher.Launcher.main
>> (Launcher.java:356)
>>
>> It looks like the Maven IDEA plugin is no longer maintained.
>>
>> I’m sure there must be other IntelliJ users building Camel. How are you
>> handling this?
>>
>> Best regards,
>>
>> Robin
>>
>>

Re: building Camel in IntelliJ

[Omar Al-Safi]

Which camel version are you referring to? Also, I hope you are not opening
the full project in intellj as that may cause problems. You can try only
opening the submodules of the project, e.g: only one component to open in
intellj.

Regards,
Omar

On Thu, Dec 19, 2019, 17:05 Robin Vanderhallen 
wrote:

> Hello,
>
> I’m having trouble building Camel from source in IntelliJ. I followed the
> instructions on https://camel.apache.org/manual/latest/building.html
> However, mvn idea:idea is throwing an exception:
>
> Failed to execute goal
> org.apache.maven.plugins:maven-idea-plugin:2.3-atlassian-10:idea
> (default-cli) on project camel-parent: Execution default-cli of goal
> org.apache.maven.plugins:maven-idea-plugin:2.3-atlassian-10:idea failed.:
> NullPointerException
>
> Caused by: java.lang.NullPointerException
> at org.apache.maven.plugin.idea.IdeaModuleMojo.getLibraryUrl
> (IdeaModuleMojo.java:1141)
> at org.apache.maven.plugin.idea.IdeaModuleMojo.rewriteDependencies
> (IdeaModuleMojo.java:531)
> at org.apache.maven.plugin.idea.IdeaModuleMojo.rewriteModule
> (IdeaModuleMojo.java:423)
> at org.apache.maven.plugin.idea.IdeaMojo.rewriteModule
> (IdeaMojo.java:236)
> at org.apache.maven.plugin.idea.IdeaMojo.execute (IdeaMojo.java:216)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
> (DefaultBuildPluginManager.java:137)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:210)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:498)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:289)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:229)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:415)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:356)
>
> It looks like the Maven IDEA plugin is no longer maintained.
>
> I’m sure there must be other IntelliJ users building Camel. How are you
> handling this?
>
> Best regards,
>
> Robin
>
>

Re: Camel-RabbitMQ-STOMP

[Omar Al-Safi]

Hi Mahesh,

The error above is related to the initial creation of the Camel route, is
pretty weird though that it can't parse the options although you
correctly spelled them. Which version of Camel do you use?
Also, in regards camel stomp with RabbitMQ, I honestly have no idea if it
is going to work although I don't see why wouldn't work.

Regards,
Omar

On Fri, Nov 22, 2019 at 12:51 PM Mahesh  wrote:

> Hi All,
> I am new to message brokers. I successfully completed a POC using Spring
> Reactor Netty Stomp client (Spring messaging framework) to connect to
> RabbitMQ on port 61613 (port dedicated for STOMP). That was just a POC. Now
> i have to build something concrete to be able to deploy in prod
> environments. I was googling Spring Messaging Vs Spring Integration and
> stumbled upon Apache Camel.
> My question is: Can Apache camel be used to connect to RabbitMQ on port
> 61613.
>
> I read camel documents:
> 1. RabbitMQ component page talks about port 5672 and that is AMQP
> 2. STOMP component page talks about ActiveMQ
>
> I did not see any examples or documentation regarding Camel in conjunction
> with RabbitMQ and STOMP.
>
> I went ahead and tried establishing STOMP connection with RabbitMQ as
> mentioned below:
>
>
> from("stomp:queue:controllerlistener?brokerURL=tcp://localhost:61613=guest=guest").to("stream:out");
>
> I am getting an exception:
>
>  org.apache.camel.FailedToCreateRouteException: Failed to create route
> route1: Route(route1)[[From[stomp:queue:test?brokerURL=tcp://localho...
> because of Failed to resolve endpoint:
>
> stomp://queue:test?brokerURL=tcp%3A%2F%2Flocalhost%3A61613=guest=guest
> due to: Failed to resolve endpoint:
>
> stomp://queue:test?brokerURL=tcp%3A%2F%2Flocalhost%3A61613=guest=guest
> due to: There are 3 parameters that couldn't be set on the endpoint. Check
> the uri if the parameters are spelt correctly and that they are properties
> of the endpoint. Unknown parameters=[{brokerURL=tcp://localhost:61613,
> login=guest, passcode=guest}]
>
> What is the right way to establish STOMP connection with RabbitMQ?
>
> Thanks,
>
> Mahesh
>

Re: Micronaut and Apache Camel

[Omar Al-Safi]

Also you can take a look at Camel with Quarkus
https://github.com/apache/camel-quarkus

On Thu, 31 Oct 2019 at 10:41, Imran Raza Khan 
wrote:

> I am looking for working example of Apache Camel with Micronaut.
>
> <
> https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=webmail
> >
> Virenfrei.
> www.avast.com
> <
> https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=webmail
> >
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>

Camel-netty to handle 1000+ TCP connections concurrently

[Omar Al-Safi]

Hello Camel riders,

I have a general question regarding camel-netty over TCP, has any one
succeed to run camel-netty in production in order to handle over 1000 TCP
connections concurrently? Personally I haven't tested it but I was curious
if someone else has done something similar and can share their feedback on
it perhaps?

Regards,
Omar

Re: Configuring custom Kafka producer/consumer properties on Camel Kafka component/endpoint?

[Omar Al-Safi]

Hi Erik,

To be honest I will keep confluent discussion separate as I personally
don't 100% agree with you and I am afraid this will need entirely separate
thread :P.

Regardimg the maven plugin, this will solve different problem, the problem
of maintaining the configurations with more recent versions.

Yes as Claus mentioned, please create a ticket for it in order to allow the
component to set custom configurations.


Regards,
Omar

On Fri, Oct 18, 2019, 08:59 Erik Boye  wrote:

> Åpen informasjon / Public information
>
> Hi,
>
> Sorry for being offline on this matter
>
> Claus, please do not get political on this.  Confluent Kafka is also
> open source - at least the stuff we are talking about here. And I am pretty
> sure Confluent is one of, if not the, main contributor to Apache Kafka.
> Enterprise open source is the "new way", and you cannot blame others for
> doing the same as Red Hat has been doing successfully for many years! Camel
> should, in my option, provide the extensibility mechanisms to fulfill Camel
> users requirements - even if you consider them to be "proprietary"
> And no, I have absolutely no personal relationship to Confluent. I was
> also a bit surprised when our Kafka Ops tells me I have to use a "schema
> registry" to produce Kafka messages. But I have got used to it, and I even
> like the idea of having some API/schema on the Kafka data streams
>
> A Maven plugin would not solve my original issues - even if it was in
> place when I needed to configure some Confluent specific Kafka properties.
> Or did I misunderstood something? I think we really need a way for users to
> set (additional) Kafka consumer/producer properties that are not known to
> Camel. And other "complex" Camel components do have that. That is my main
> point of view. Do you want me to create an issue for this?
>
> Erik
>
> -Opprinnelig melding-
> Fra: Claus Ibsen 
> Sendt: søndag 13. oktober 2019 18:19
> Til: users@camel.apache.org
> Emne: [SUSPECT URL !]Re: Configuring custom Kafka producer/consumer
> properties on Camel Kafka component/endpoint?
>
> Hi Omar
>
> Yeah the maven plugin idea is really good.
> But maybe we need both.
>
> The maven plugin will only get you the options of the kafka version
> that was used for that release.
> With having a way for end users to set extra options then they could
> work against newer Kafka versions without Camel have to upgrade.
> And maybe also to proprietary options on Confluent.
>
>
> On Sun, Oct 13, 2019 at 2:31 PM Omar Al-Safi  wrote:
> >
> > Hi Claus,
> >
> > I also thought about this, however since Kafka producers/consumers
> options
> > increase and some of them may gets deprecated, I was thinking we can
> have a
> > maven plugin that generate these configurations for us without the need
> to
> > do it manually, it will generate using the configDef (e.g:
> >
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fkafka%2Fblob%2Ftrunk%2Fclients%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkafka%2Fclients%2Fconsumer%2FConsumerConfig.java%23L555data=02%7C01%7CErik.Boye%40statnett.no%7C512ca3c98977460f22d908d74ff923f3%7Ca8d61462f25244b2bf6ad7231960c041%7C0%7C0%7C637065803788184830sdata=S%2BJyTAx8u7V0jow9oCELojh9af5dsgDtxYjRzRz89Lg%3Dreserved=0
> ).
> > I did this for DBZ as I built a maven plugin that scrap all these config
> > and generate the camel configurations, you can take a look at my progress
> > here:
> >
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fomarsmak%2Fcamel%2Fblob%2FCAMEL-14009%2Fcomponents%2Fcamel-debezium%2Fcamel-debezium-maven-plugin%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fcamel%2Fmaven%2FTestMojo.java%23L24data=02%7C01%7CErik.Boye%40statnett.no%7C512ca3c98977460f22d908d74ff923f3%7Ca8d61462f25244b2bf6ad7231960c041%7C0%7C0%7C637065803788184830sdata=hTRBEetZ7zBt%2BoS9fSfbQV94G0KRIudY0WlkDeo50Fs%3Dreserved=0
> .
> > Please take a look and share your thoughts about it.
> >
> > Regards,
> > Omar
> >
> > On Sun, 13 Oct 2019 at 14:12, Claus Ibsen  wrote:
> >
> > > Hi
> > >
> > > Yeah this has come up a few times that Kafka is adding new options, or
> > > more particularly Confluent adds propriety options that they dont give
> > > back to the community.
> > >
> > > Can you create a JIRA ticket.
> > >
> > > Then we can look at adding a way to add custom options. You are of
> > > course welcome to try to implement this. We love contributions.
> > >
> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fcamel%2Fblob%2Fmaster%2FCONT

Re: Dynamically restarting consumer routes with changed URI parameters

[Omar Al-Safi]

Hi Ralf,

If you have these routes as a RouteBuilder beans, I think you can utilize
CamelContext APIs such `addRoute` to add the bean dynamically. Of course
you will need to implement a way to hot reloading of your .class files
whenever you deem to. You can take a look at this test for an example:
https://github.com/apache/camel/blob/b9a3117f19dd19abd2ea8b789c42c3e86fe4c488/core/camel-core/src/test/java/org/apache/camel/builder/AddRoutesAtRuntimeTest.java

Regards,
Omar

On Wed, 16 Oct 2019 at 09:28, Ralf Claussnitzer <
ralf.claussnit...@slub-dresden.de> wrote:

> Hello all,
>
> is there a way to restart defined routes so that the from() URI gets
> updated?
> We have tried several things using the "Controlbus" component, but we
> cannot restart a defined and running route /with changed URI parameters/.
>
> Background (what do we want to achieve):
>
> We have several Kafka Consumer routes which we want to restart with
> different options depending on other events. In particular we want
> to restart a running route and change the URI parameter "seekTo" to
> trigger reprocessing past events.
>
> While reprocessing Kafka events is our main topic at the moment, we
> imagine that restarting a route and recalculate the consumer URI might
> be useful for other use cases as well. Let us know what you think!
>
> Regards,
> Ralf
>
>

Re: Configuring custom Kafka producer/consumer properties on Camel Kafka component/endpoint?

[Omar Al-Safi]

Hi Claus,

I agree with you that we will need both, especially on these configurations
that are related to Confluent platform which are ported back to the
community as you mentioned. I have created a JIRA ticket for it here:
https://issues.apache.org/jira/browse/CAMEL-14062, I will work on it once I
am done from DBZ which is already on the final stage that I can create a PR
for.

Regards,
Omar

On Sun, 13 Oct 2019 at 18:19, Claus Ibsen  wrote:

> Hi Omar
>
> Yeah the maven plugin idea is really good.
> But maybe we need both.
>
> The maven plugin will only get you the options of the kafka version
> that was used for that release.
> With having a way for end users to set extra options then they could
> work against newer Kafka versions without Camel have to upgrade.
> And maybe also to proprietary options on Confluent.
>
>
> On Sun, Oct 13, 2019 at 2:31 PM Omar Al-Safi  wrote:
> >
> > Hi Claus,
> >
> > I also thought about this, however since Kafka producers/consumers
> options
> > increase and some of them may gets deprecated, I was thinking we can
> have a
> > maven plugin that generate these configurations for us without the need
> to
> > do it manually, it will generate using the configDef (e.g:
> >
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java#L555
> ).
> > I did this for DBZ as I built a maven plugin that scrap all these config
> > and generate the camel configurations, you can take a look at my progress
> > here:
> >
> https://github.com/omarsmak/camel/blob/CAMEL-14009/components/camel-debezium/camel-debezium-maven-plugin/src/main/java/org/apache/camel/maven/TestMojo.java#L24
> .
> > Please take a look and share your thoughts about it.
> >
> > Regards,
> > Omar
> >
> > On Sun, 13 Oct 2019 at 14:12, Claus Ibsen  wrote:
> >
> > > Hi
> > >
> > > Yeah this has come up a few times that Kafka is adding new options, or
> > > more particularly Confluent adds propriety options that they dont give
> > > back to the community.
> > >
> > > Can you create a JIRA ticket.
> > >
> > > Then we can look at adding a way to add custom options. You are of
> > > course welcome to try to implement this. We love contributions.
> > > https://github.com/apache/camel/blob/master/CONTRIBUTING.md
> > >
> > > On Tue, Oct 8, 2019 at 9:57 AM Erik Boye 
> wrote:
> > > >
> > > > Hi,
> > > >
> > > > Is there any way to configure custom Kafka producer/consumer
> properties
> > > on the Camel Kafka component/endpoint (without "breaking in")?
> > > > I specifically want to configure the Confluent Kafka property with
> key
> > > 'value.subject.name.strategy':
> > >
> https://docs.confluent.io/current/schema-registry/serializer-formatter.html
> > > >
> > > > Other Camel components (i.e Camel ActiveMQ; destination.consumer) are
> > > made extensible by providing users a way to set "additional
> properties".
> > > Why does not Camel Kafka provide that? Until schemaRegistryURL was
> added as
> > > a configurable property, we had to make our own Camel Kafka Component
> to
> > > make it work ("breaking in"). There should definitely be a way to set
> > > additional/custom properties on Kafka producers/consumers.
> > > >
> > > > Just have a look at the Camel Kafka documentation and do a text
> search
> > > for Confluent:
> > > https://camel.apache.org/components/latest/kafka-component.html ;-)
> > > > Having "native" support for schemas on Kafka records is definitely a
> > > good idea, and I am pretty sure Apache Kafka will move in that
> direction at
> > > some point
> > > >
> > > > Erik
> > > >
> > > > "This email with attachments is solely for the use of the individual
> or
> > > entity to which it is addressed. It may contain confidential or
> privileged
> > > information. If you are not the addressee, please notify the sender and
> > > delete this message and all attachments from your files."
> > >
> > >
> > >
> > > --
> > > Claus Ibsen
> > > -
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
> > >
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: Configuring custom Kafka producer/consumer properties on Camel Kafka component/endpoint?

[Omar Al-Safi]

Hi JB,

Thanks for the feedback. The way I am doing it now in my PoC branch is
that, the maven plugin will generate camel configuration beans when the
component is being built by calling the plugin in the generate-sources
maven phase, therefore the resulting JAR will include these configurations
as java classes in similar way to the property configurator classes being
generated, thus I don't think non-maven users will have a problem since all
configurations being generated when the component has been built.

Regards,
Omar

On Sun, 13 Oct 2019 at 17:09, Jean-Baptiste Onofré  wrote:

> Hi,
>
> IMHO, I think maven plugin is fine if it's still generate a bean. We
> have to provide configuration via a bean and/or URI for most for users.
> We have to think about non-Maven users like Gradle or script users.
>
> Regards
> JB
>
> On 13/10/2019 14:30, Omar Al-Safi wrote:
> > Hi Claus,
> >
> > I also thought about this, however since Kafka producers/consumers
> options
> > increase and some of them may gets deprecated, I was thinking we can
> have a
> > maven plugin that generate these configurations for us without the need
> to
> > do it manually, it will generate using the configDef (e.g:
> >
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java#L555
> ).
> > I did this for DBZ as I built a maven plugin that scrap all these config
> > and generate the camel configurations, you can take a look at my progress
> > here:
> >
> https://github.com/omarsmak/camel/blob/CAMEL-14009/components/camel-debezium/camel-debezium-maven-plugin/src/main/java/org/apache/camel/maven/TestMojo.java#L24
> .
> > Please take a look and share your thoughts about it.
> >
> > Regards,
> > Omar
> >
> > On Sun, 13 Oct 2019 at 14:12, Claus Ibsen  wrote:
> >
> >> Hi
> >>
> >> Yeah this has come up a few times that Kafka is adding new options, or
> >> more particularly Confluent adds propriety options that they dont give
> >> back to the community.
> >>
> >> Can you create a JIRA ticket.
> >>
> >> Then we can look at adding a way to add custom options. You are of
> >> course welcome to try to implement this. We love contributions.
> >> https://github.com/apache/camel/blob/master/CONTRIBUTING.md
> >>
> >> On Tue, Oct 8, 2019 at 9:57 AM Erik Boye  wrote:
> >>>
> >>> Hi,
> >>>
> >>> Is there any way to configure custom Kafka producer/consumer properties
> >> on the Camel Kafka component/endpoint (without "breaking in")?
> >>> I specifically want to configure the Confluent Kafka property with key
> >> 'value.subject.name.strategy':
> >>
> https://docs.confluent.io/current/schema-registry/serializer-formatter.html
> >>>
> >>> Other Camel components (i.e Camel ActiveMQ; destination.consumer) are
> >> made extensible by providing users a way to set "additional properties".
> >> Why does not Camel Kafka provide that? Until schemaRegistryURL was
> added as
> >> a configurable property, we had to make our own Camel Kafka Component to
> >> make it work ("breaking in"). There should definitely be a way to set
> >> additional/custom properties on Kafka producers/consumers.
> >>>
> >>> Just have a look at the Camel Kafka documentation and do a text search
> >> for Confluent:
> >> https://camel.apache.org/components/latest/kafka-component.html ;-)
> >>> Having "native" support for schemas on Kafka records is definitely a
> >> good idea, and I am pretty sure Apache Kafka will move in that
> direction at
> >> some point
> >>>
> >>> Erik
> >>>
> >>> "This email with attachments is solely for the use of the individual or
> >> entity to which it is addressed. It may contain confidential or
> privileged
> >> information. If you are not the addressee, please notify the sender and
> >> delete this message and all attachments from your files."
> >>
> >>
> >>
> >> --
> >> Claus Ibsen
> >> -
> >> http://davsclaus.com @davsclaus
> >> Camel in Action 2: https://www.manning.com/ibsen2
> >>
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: Configuring custom Kafka producer/consumer properties on Camel Kafka component/endpoint?

[Omar Al-Safi]

Hi Claus,

I also thought about this, however since Kafka producers/consumers options
increase and some of them may gets deprecated, I was thinking we can have a
maven plugin that generate these configurations for us without the need to
do it manually, it will generate using the configDef (e.g:
https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java#L555).
I did this for DBZ as I built a maven plugin that scrap all these config
and generate the camel configurations, you can take a look at my progress
here:
https://github.com/omarsmak/camel/blob/CAMEL-14009/components/camel-debezium/camel-debezium-maven-plugin/src/main/java/org/apache/camel/maven/TestMojo.java#L24.
Please take a look and share your thoughts about it.

Regards,
Omar

On Sun, 13 Oct 2019 at 14:12, Claus Ibsen  wrote:

> Hi
>
> Yeah this has come up a few times that Kafka is adding new options, or
> more particularly Confluent adds propriety options that they dont give
> back to the community.
>
> Can you create a JIRA ticket.
>
> Then we can look at adding a way to add custom options. You are of
> course welcome to try to implement this. We love contributions.
> https://github.com/apache/camel/blob/master/CONTRIBUTING.md
>
> On Tue, Oct 8, 2019 at 9:57 AM Erik Boye  wrote:
> >
> > Hi,
> >
> > Is there any way to configure custom Kafka producer/consumer properties
> on the Camel Kafka component/endpoint (without "breaking in")?
> > I specifically want to configure the Confluent Kafka property with key
> 'value.subject.name.strategy':
> https://docs.confluent.io/current/schema-registry/serializer-formatter.html
> >
> > Other Camel components (i.e Camel ActiveMQ; destination.consumer) are
> made extensible by providing users a way to set "additional properties".
> Why does not Camel Kafka provide that? Until schemaRegistryURL was added as
> a configurable property, we had to make our own Camel Kafka Component to
> make it work ("breaking in"). There should definitely be a way to set
> additional/custom properties on Kafka producers/consumers.
> >
> > Just have a look at the Camel Kafka documentation and do a text search
> for Confluent:
> https://camel.apache.org/components/latest/kafka-component.html ;-)
> > Having "native" support for schemas on Kafka records is definitely a
> good idea, and I am pretty sure Apache Kafka will move in that direction at
> some point
> >
> > Erik
> >
> > "This email with attachments is solely for the use of the individual or
> entity to which it is addressed. It may contain confidential or privileged
> information. If you are not the addressee, please notify the sender and
> delete this message and all attachments from your files."
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: netty tcp consumer not reading large payload

[Omar Al-Safi]

I think there is some few information here, like how did you use it, how
does your route looks like ..etc. Also did you take a look at netty example
here
https://github.com/apache/camel/tree/master/examples/camel-example-netty-http
?

On Tue, 8 Oct 2019 at 13:49, Rohan Emmanuel 
wrote:

> Hi,
> I have problem reading large payload using the netty TCP component. The
> payload is getting read in chunks, can you please guide me as to how can I
> read the entire payload
>
> --
> Regards,
> Rohan Emmanuel
>

Re: One CamelContext vs. multiple CamelContexts

[Omar Al-Safi]

Yo be honest, I don't really see where it will help to have multiple
contexts to run route, unless we are talking about fault tolerance and fail
over? IMHO if the external system that you injest from that can support
high throughput and parallelism (e.g: Kafka), maybe that will show good
difference.

Regards,

On Sat, 28 Sep 2019 at 19:19, Ron  wrote:

> Hi, Jan. Thanks for the info.
>
> I didn’t mean multiple CamelContexts per app. I meant multiple (many)
> apps, each with one CamelContext and one Route - versus one app, with one
> CamelContext, with many Routes.
>
> If I were running on a single host, I think I’d want to keep the number of
> apps down and see how much I can put into one CamelContext (one app).
>
> If I’m in a clustered environment, I of course have more flexibility and
> scalability. But even then, I wouldn’t want to create an app (with one
> CamelContext) per Route.
>
> Basically, I’ll just have to experiment. I was looking to see how others
> approached this and what they found. (Mantas said his group put 400 routes
> in a single context!)
>
> Thanks again.
>
> Ron
>
> Sent from Xfinity Connect App
>
> -- Original Message --
>
> From: Jan Bednář
> To: users@camel.apache.org
> Sent: September 28, 2019 at 12:42 PM
> Subject: Re: One CamelContext vs. multiple CamelContexts
>
> Hi Ron, Mixing multiple CamelContext per application is not recomended
> approach. Also support for multiple contexts is removed in Camel 3:
> https://camel.apache.org/manual/latest/camel-3-migration-guide.html#_multiple_camelcontexts_per_application_not_supported
> Dne 27.9.2019 v 21:19 Mantas Gridinas napsal(a): > You're going to be fine.
> My current project runs 400+ routes in single context. > > On Fri, Sep 27,
> 2019 at 7:14 PM Ron Cecchini wrote: >> TL;DR: 1 CamelContext with 100
> Routes vs. 100 CamelContexts each with 1 Route >> >> Say I need to ingest
> data from a hundred sensors or data sources, over TCP or JMS, and get it
> written to a central database or JMS. >> >> The messages are asynchronous
> and don't require a response or any processing. We just have to suck in all
> that data and write it out to a DB or JMS. >> >> It would be really nice to
> keep these 100 very simple routes in a single config / RouteBuilder. But
> that's not the smart thing to do... By the time you reach a 100 routes
> you'd probably need an app server and access to a cluster. But I don't
> think spinning up a new CamelContext / app for 100 single Routes is the way
> to go either. Or maybe it is? Maybe you containerize every single Route
> with Docker and manage it with Kubernetes (or whatever)? >> >> I guess I'm
> just looking to see if anyone has experimented with this and did some
> performance comparisons - like, how many Routes were you able to cram into
> your CamelContext / Spring Boot app before it started degrading? And how
> folks managed a scenario like this where they had to pull in data from many
> sources. >> >> If you don't have a cluster, and have to keep everything on
> a single beefy host, I guess the question is moot and you have to do as
> much as you can in one CamelContext until you hit a scalability limit... >>
> >> Thanks and have a good weekend. >> >> Ron

Re: Best approach to deploy camel code as standalone application

[Omar Al-Safi]

The best way is to follow the documentation in this regards:
https://camel.apache.org/manual/latest/faq/running-camel-standalone.html.
And this is an example for a standalone Camel application:
https://github.com/apache/camel/blob/master/examples/camel-example-debezium/src/main/java/org/apache/camel/example/debezium/DebeziumMySqlConsumerToKinesis.java

Regards,
Omar

On Wed, 25 Sep 2019 at 12:39, Jaiprakash Mewada <
jaiprakash.mew...@hotwaxsystems.com> wrote:

> Hi Team,
>
> I am beginner in Apache Camel. I have ofbiz code running on different jvm.
> I want Camel code to be running on different jvm. I need to send the data
> from ofbiz to third party using camel's route.
>
> It looks like I need to run standalone application of Camel. Can someone
> please guide me, what is the best possible way to implement this ?
>
> And If I implement separate standalone application, then how would I call
> the code of Camel from ofbiz which is running on different jvm ?
>
> --
> *Thanks & Regards,*
> *Jaiprakash Mewada*
> Senior Enterprise Software Engineer
>
> *HotWax Systems*
>

Re: own endpoint exmaple?

[Omar Al-Safi]

Hi Thomas,

May I know what do you want to achieve here? The context of your problem
can help us to know how can you move forward without much complexity
involved.

Regards,
Omar

On Thu, Sep 19, 2019, 13:20  wrote:

> "Scaffolding your component through maven archetype"
>
> I think at the moment I didn’t know much about it.
> I really didn’t want to blow up pom only for a little endpoint. Or did I
> not get it?
>
>
> -Ursprüngliche Nachricht-
> Von: Alex Dettinger 
> Gesendet: Donnerstag, 19. September 2019 10:20
> An: users@camel.apache.org
> Betreff: Re: own endpoint exmaple?
>
> Hi Thomas,
>
>   Did you figured it out ?
>   From the code, it's hard to see what the issue could be: Do you extend
> DefaultComponent ? Do you call setProperties() in createEndpoint() ? and so
> on...
>   If you have not used an archetype yet, then I would direct you back to
> Omar Al-Safi suggestion: Scaffolding your component through maven archetype
> https://camel.apache.org/manual/latest/camel-maven-archetypes.html.
>   You'll then have a proper layout to start on then.
>
> Alex
>
>
> On Tue, Sep 17, 2019 at 12:49 PM  wrote:
>
> > Hi,
> >
> > thanks so far. I didn’t know that I had to explicitely create a
> Component.
> > The direct example helped a lot.
> > But there is this contruct, and it will not work with my code.
> > name is not set. (it has getter and setter)
> >
> > I want that name is "thomas" when route-def is like
> >
> > to("monitoring:thomas")
> >
> > @UriEndpoint(scheme = "monitoring", title = "monitoring",
> > syntax="monitoring:name", label = "Monitoring") public class
> > MonitorEndpoint extends DefaultEndpoint {
> >   @UriPath
> >   @Metadata(required = true)
> >   private String name;
> >
> >   public MonitorEndpoint() {
> >   }
> >
> >   public MonitorEndpoint(final String endpointUri, final Component
> > component) {
> > super(endpointUri, component);
> >   }
> >
> >   @Override
> >   public Consumer createConsumer(Processor processor) throws Exception {
> > // TODO Auto-generated method stub
> > return null;
> >   }
> >
> >
> >
> >
> >
> >
> > -Ursprüngliche Nachricht-
> > Von: Omar Al-Safi 
> > Gesendet: Montag, 16. September 2019 13:05
> > An: users@camel.apache.org
> > Betreff: Re: own endpoint exmaple?
> >
> > I see, yeah we will need to fix this. By the way, you can start
> > scaffolding your component through maven archetype,
> > https://camel.apache.org/manual/latest/camel-maven-archetypes.html, as
> > I recall, it will also add the necessary files to enable camel auto
> > discovery out of the box
> >
> > On Mon, 16 Sep 2019 at 13:01, Alex Dettinger 
> > wrote:
> >
> > > Also, the source code of the direct component could be a good
> > > starter to get inspiration:
> > >
> > > https://github.com/apache/camel/tree/master/components/camel-direct/
> > > sr c/main/java/org/apache/camel/component/direct
> > >
> > > Alex
> > >
> > > On Mon, Sep 16, 2019 at 12:57 PM Steve Huston 
> > > wrote:
> > >
> > > > Is it possible that Spring is not finding your endpoint class?
> > > >
> > > > -Steve
> > > >
> > > > On 9/16/19, 6:55 AM, "thomas.thi...@telekom.de" <
> > > thomas.thi...@telekom.de>
> > > > wrote:
> > > >
> > > > Hi Alex,
> > > >
> > > > 1. I found this webpage but the links (e.g. createProducer())
> > > > goes to "page not found".
> > > >
> > > > 2. Stupid question: how and where to do this?
> > > >
> > > > -Ursprüngliche Nachricht-
> > > > Von: Alex Dettinger 
> > > > Gesendet: Montag, 16. September 2019 12:52
> > > > An: users@camel.apache.org
> > > > Betreff: Re: own endpoint exmaple?
> > > >
> > > > Hi Thomas,
> > > >
> > > >   This link might be of interest:
> > > >
> > > > https://camel.apache.org/manual/latest/writing-components.html
> > > >
> > > >   In your case, I wonder if you missed to create the thomas
> > > > component first.
> > > >
> > > > hth,
> > > > Alex
> > > >
> > > > On Mon, Sep 16, 2019 at 12:44 PM 
> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > can some point me to a working example where an own endpoint
> > > > was created.
> > > > >
> > > > > In route I say:
> > > > >
> > > > > .to("thomas://foo")
> > > > >
> > > > > my Endpoint is
> > > > >
> > > > > @Component
> > > > > @UriEndpoint(scheme = "thomas", title = "Thiele",
> > > > > syntax="thomas:name", label = "Thomas") public class ThomasStep
> > > > > extends DefaultEndpoint {}
> > > > >
> > > > > But I get.
> > > > >
> > > > > thomas://name due to: No component found with scheme: thomas
> > > > >
> > > >
> > > >
> > > >
> > >
> >
>

Re: own endpoint exmaple?

[Omar Al-Safi]

I see, yeah we will need to fix this. By the way, you can
start scaffolding your component through maven archetype,
https://camel.apache.org/manual/latest/camel-maven-archetypes.html, as I
recall, it will also add the necessary files to enable camel auto discovery
out of the box

On Mon, 16 Sep 2019 at 13:01, Alex Dettinger  wrote:

> Also, the source code of the direct component could be a good starter to
> get inspiration:
>
> https://github.com/apache/camel/tree/master/components/camel-direct/src/main/java/org/apache/camel/component/direct
>
> Alex
>
> On Mon, Sep 16, 2019 at 12:57 PM Steve Huston 
> wrote:
>
> > Is it possible that Spring is not finding your endpoint class?
> >
> > -Steve
> >
> > On 9/16/19, 6:55 AM, "thomas.thi...@telekom.de" <
> thomas.thi...@telekom.de>
> > wrote:
> >
> > Hi Alex,
> >
> > 1. I found this webpage but the links (e.g. createProducer()) goes to
> > "page not found".
> >
> > 2. Stupid question: how and where to do this?
> >
> > -Ursprüngliche Nachricht-
> > Von: Alex Dettinger 
> > Gesendet: Montag, 16. September 2019 12:52
> > An: users@camel.apache.org
> > Betreff: Re: own endpoint exmaple?
> >
> > Hi Thomas,
> >
> >   This link might be of interest:
> >   https://camel.apache.org/manual/latest/writing-components.html
> >
> >   In your case, I wonder if you missed to create the thomas component
> > first.
> >
> > hth,
> > Alex
> >
> > On Mon, Sep 16, 2019 at 12:44 PM  wrote:
> >
> > > Hi,
> > >
> > > can some point me to a working example where an own endpoint was
> > created.
> > >
> > > In route I say:
> > >
> > > .to("thomas://foo")
> > >
> > > my Endpoint is
> > >
> > > @Component
> > > @UriEndpoint(scheme = "thomas", title = "Thiele",
> > > syntax="thomas:name", label = "Thomas") public class ThomasStep
> > > extends DefaultEndpoint {}
> > >
> > > But I get.
> > >
> > > thomas://name due to: No component found with scheme: thomas
> > >
> >
> >
> >
>

Re: own endpoint exmaple?

[Omar Al-Safi]

If I understand correctly, you want to create your own component, is that
correct? If that is the case, please check here:
https://camel.apache.org/manual/latest/writing-components.html. Basically
you will need to either:

   - Add add a file to META-INF/services/org/apache/camel/component/FOO to
   enable the auto-discovery per the docs.
   - Or initiate the component manually and add it to camel context, you
   can take a look an example here:
   
https://github.com/apache/camel/blob/master/examples/camel-example-artemis/src/main/java/org/apache/camel/example/artemis/ArtemisMain.java#L39
or
   here:
   
https://github.com/apache/camel/blob/master/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/BaseEmbeddedKafkaTest.java#L81

regards

On Mon, 16 Sep 2019 at 12:44,  wrote:

> Hi,
>
> can some point me to a working example where an own endpoint was created.
>
> In route I say:
>
> .to("thomas://foo")
>
> my Endpoint is
>
> @Component
> @UriEndpoint(scheme = "thomas", title = "Thiele", syntax="thomas:name",
> label = "Thomas")
> public class ThomasStep extends DefaultEndpoint
> {}
>
> But I get.
>
> thomas://name due to: No component found with scheme: thomas
>

Re: OnException block to figured out where the Exception is originated from

[Omar Al-Safi]

If I understood you correctly, you want to know from where this Exchange
which holds the exception, have you tried something like
`exchange.getFromEndpoint` or `exchange.getFromRouteId`? Which as I
understood should gives from which endpoint this exchange has been
originated from. However, I think `Exchange.FAILURE_ENDPOINT` will contain
the information about the last time Camel has sent the exchange to, hence I
think you would see this information after a `to` route. However I am not
100% sure though.

On Wed, 28 Aug 2019 at 07:38, Wang Yan  wrote:

> Hello all,
>
>
> OnException block , the TechnicalException will be caught, this
> exception could come
>
> from Route A or Route B , my question is , Is it possible for on Exception
> block
>
> to figure out where the TechnicalException is originated from ? for
> example from which Route or from which Endpoint?
>
> I tried to use exchange.getProperty(Exchange.FAILURE_ENDPOINT) or
> exchange.getProperty(Exchange.FAILURE_ROUTE_ID)
>
> but they both print out null value
>
> Any hints or suggestions are more than welcome!
>
>
>
>
> onException(TechnicalException.class)
>   .log(LoggingLevel.ERROR, "TechnicalException caught
> globally  ")
>   .redeliveryDelay(3000)
>   .maximumRedeliveries(3)
>   //Exchange.FAILURE_ROUTE_ID
>   .onRedelivery(exchange -> {
>LOG.info("onRedelivery {} , failed endpoint id {}
> exception type {}",
> exchange.getIn().getHeader(Exchange.REDELIVERY_COUNTER)
>  ,exchange.getProperty(Exchange.FAILURE_ENDPOINT) ,
> exchange.getProperty(Exchange.EXCEPTION_CAUGHT).getClass());
>   })
>   .handled(true)
>   .to("activemq:queue:mydeadletterqueue")
>   .log(LoggingLevel.ERROR, "technical exception send to
> mydeadletterqueue")
>   .end();
>
>
> route A
>
> route B
>

Re: Code of calling rest service is not working

[Omar Al-Safi]

I am new to Camel as well, I was figuring things on the way :). First you
need to use a REST component, you can use the REST DSL
, however this is only a DSL not a
actual http server, you will need to include an http server component as
well (e.g:  camel-netty4-http, camel-restlet, camel-spark-rest ..etc), once
you included it, you will need to tell Camel to use the specified component
through the configuration RouteBuilder
(e.g: restConfiguration().component("spark-rest").port(8000);). Also in
your above example, the JVM will terminate immediately, and hence Camel
will stop immediately. Therefore you need to either use the Main class that
is shipped in Camel Core (example
)
or keep the JVM running indefinitely by adding
(Thread.sleep(Integer.MAX_VALUE);)

You can take a look here for an example:
https://www.codepile.net/pile/p4dDJd4x

Please note I also included the camel-spark-rest component in the POM file
as well.

On Mon, 29 Jul 2019 at 14:06, Jaiprakash Mewada <
jaiprakash.mew...@hotwaxsystems.com> wrote:

> Hi Team,
>
> I am beginner in apache camel. I want to make to rest call but not able to
> get expected result. Can someone help me, what exact change I need to do in
> below code in order to receive the json response.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *import org.apache.camel.CamelContext;import
> org.apache.camel.ConsumerTemplate;import org.apache.camel.Exchange;import
> org.apache.camel.Processor;import org.apache.camel.ProducerTemplate;import
> org.apache.camel.builder.RouteBuilder;import
> org.apache.camel.impl.DefaultCamelContext;public class
> CamelHelloWorldExample { public static void main(String[] args) throws
> Exception { CamelContext context = new DefaultCamelContext();
> context.setTracing(true); context.addRoutes(new RouteBuilder() { @Override
> public void configure() throws Exception { System.out.println("---Start
> of RouteBuilder---");
> from("direct:start").setHeader(Exchange.HTTP_METHOD, simple("GET"))
> .to("http://dummy.restapiexample.com/api/v1/employee/330
> ").process(new
> Processor() { public void process(Exchange exchange) throws Exception {
> System.out.println(exchange.getIn().getBody(String.class)); } } )
> .to("seda:end"); System.out.println("---End of
> RouteBuilder---"); } } ); context.start(); ProducerTemplate
> prodTemp = context.createProducerTemplate();
> prodTemp.sendBody("direct:start", "Hello"); ConsumerTemplate consTemp =
> context.createConsumerTemplate(); String msg =
> consTemp.receiveBody("seda:end", String.class); System.out.println(msg);
> }}*
>
> --
> *Thanks & Regards,*
> *Jaiprakash Mewada*
> Senior Enterprise Software Engineer
>
> *HotWax Systems*
>