Re: JMX issues with processor

[Jean-Baptiste Onofre]

Hi Nicolas,

Which camel version are you using ? And which runtime ?

Regards
JB

> Le 14 déc. 2020 à 19:24, CASAUX Nicolas  a 
> écrit :
> 
> Hello !
> 
> I may have found two minor bugs with data available from processors through 
> JMX.
> 
> 1)
> When I use processors like :
> .enrich().constant(direct("myEndpoint").failIfNoConsumers(false))
> in JMX data, the exposed "ExpressionLanguage" is empty, and the "Expression" 
> is something like 
> "org.apache.camel.builder.ExpressionClause@7c098bb3"
> 
> Similar issue happens with pollEnrich, and probably other processors.
> Workaround for that was to add a ".getUri()" :
> 
>
> .enrich().constant(direct("enrichConstant").failIfNoConsumers(false).getUri())
> 
> 2)
> Processor "pollEnrich" says it does not support extended information 
> (SupportExtendedInformation=false), however, it does support it.
> There are indeed some data available with the "extendedInformation" operation.
> Maybe some other processors have the same problem.
> 
> I can of course create jira(s) if it makes sense.
> 
> Regards,
> Nicolas
> 
> 

Re: Are you using binary distribution?

[Tadayoshi Sato]

Hi folks,

I don't use binary distributions, either. It makes sense for runtime
projects such as Apache Karaf, Kafka, Artemis, etc. to provide binary
distributions, but it doesn't for library frameworks like Camel Core, Camel
Quarkus, and CKC.

For Camel-K, however, it has a CLI `kamel` so IMO it still makes sense to
keep binary distributions for it.

Thanks,
Tadayoshi

On Mon, Dec 14, 2020 at 10:58 PM Ajmera, Hemang C 
wrote:

> I am relying on maven for binaries and git for sourcecode. IMHO binary
> distribution may not be needed.
> Github has release feature, where we can have binaries can be downloaded
> from there. Many users might be used to that way of downloading binary, so
> if needed, that can be alternate place for download if anyone needs it.
> However CI/CD pipeline might need updates for github release to work.
>
>
> Thanks and Regards,
> Hemang Ajmera
>
>
> -Original Message-
> From: Zoran Regvart 
> Sent: 14 December 2020 19:08
> To: Dev ; users@camel.apache.org
> Subject: Are you using binary distribution?
>
>
> EXTERNAL SENDER:   Do not click any links or open any attachments unless
> you trust the sender and know the content is safe.
> EXPÉDITEUR EXTERNE:Ne cliquez sur aucun lien et n’ouvrez aucune pièce
> jointe à moins qu’ils ne proviennent d’un expéditeur fiable, ou que vous
> ayez l'assurance que le contenu provient d'une source sûre.
>
> Hi Cameleers,
> we're discussing binary distribution on two issues[1][2]. The binary
> distribution is the tar.gz/ZIP file linked from the Camel website. By ASF
> policy we only ship source code, and the binary distribution is optional.
>
> Back in the dark days, before using build tools that knew about dependency
> management (Maven, Gradle...) folk used to use the binary distribution.
>
> I've found some statistics on downloads/per day, for us[3]/eu[4] and
> created these charts:
>
>
> https://urldefense.com/v3/__https://s.apache.org/camel-dl-us__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDiSwKzT18$
>
> https://urldefense.com/v3/__https://s.apache.org/camel-dl-eu__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDic4MBCm6$
>
> The data is over 2 and 1/4 years, we've had 19.7+-8.8 via US, and
> 20.24+-8.43 in via EU per day. So not that much IMHO.
>
> I'm wondering if anyone is still relying on these, and if so what would a
> binary distribution look like for sub projects? Should we do the same as we
> do for the Camel core?
>
> Please reply on this thread or chime in on those issues for sub-project
> specific concerns.
>
> Thanks :)
>
> zoran
>
> [1]
> https://urldefense.com/v3/__https://github.com/apache/camel-quarkus/issues/2045__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDiZ80_jXD$
> [2]
> https://urldefense.com/v3/__https://github.com/apache/camel-kafka-connector/issues/754__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDiVlO1Pqs$
> [3]
> https://urldefense.com/v3/__https://www-us.apache.org/dyn/stats/camel.log__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDiStt2eyy$
> [4]
> https://urldefense.com/v3/__https://www-eu.apache.org/dyn/stats/camel.log__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDicRsFgPE$
> --
> Zoran Regvart
>


-- 
Tadayoshi Sato

JMX issues with processor

[CASAUX Nicolas]

Hello !

I may have found two minor bugs with data available from processors through JMX.

1)
When I use processors like :
.enrich().constant(direct("myEndpoint").failIfNoConsumers(false))
in JMX data, the exposed "ExpressionLanguage" is empty, and the "Expression" is 
something like 
"org.apache.camel.builder.ExpressionClause@7c098bb3"

Similar issue happens with pollEnrich, and probably other processors.
Workaround for that was to add a ".getUri()" :


.enrich().constant(direct("enrichConstant").failIfNoConsumers(false).getUri())

2)
Processor "pollEnrich" says it does not support extended information 
(SupportExtendedInformation=false), however, it does support it.
There are indeed some data available with the "extendedInformation" operation.
Maybe some other processors have the same problem.

I can of course create jira(s) if it makes sense.

Regards,
Nicolas

Camel and WebFlux issue

[Claudio Tasso]

Hi,
I'm trying to use the integration of Camel with reactive streams in a
Spring project, but I'm stuck.
This is my RouterBuilder:

@Component
public class MyRouteBuilder extends RouteBuilder {
@Override
public void configure() throws Exception {
from("direct:pluto").to("reactive-streams:paperino");
  }
}

and this is my RestController:

@GetMapping(path = "/hello2")
public Flux hello2() {

Publisher paperinoPublisher = camel.fromStream("paperino",
String.class);
Mono bodyToMono = wcBuilder.baseUrl("http://httpbin.org
").build().get().uri("/get").retrieve().bodyToMono(String.class);

Flux flux = Flux.from(paperinoPublisher).doOnNext(s -> log.info("Hello
from flux: {}",s) )
.flatMap(s -> bodyToMono)
.doOnNext(b -> log.info("Response: {}",b));

producer.sendBody("direct:pluto","ciao");
return  flux;
}

When Rest endpoint is called, I got this exception raised by the call to
"sendBody":

Caused by: java.lang.IllegalStateException: The stream has no active
subscriptions
at
org.apache.camel.component.reactive.streams.engine.CamelPublisher.publish(CamelPublisher.java:110)
~[camel-reactive-streams-3.5.0.jar:3.5.0]
at
org.apache.camel.component.reactive.streams.engine.DefaultCamelReactiveStreamsService.sendCamelExchange(DefaultCamelReactiveStreamsService.java:151)
~[camel-reactive-streams-3.5.0.jar:3.5.0]
at
org.apache.camel.component.reactive.streams.ReactiveStreamsProducer.process(ReactiveStreamsProducer.java:52)
~[camel-reactive-streams-3.5.0.jar:3.5.0]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:169)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:404)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60)
~[camel-base-3.5.0.jar:3.5.0]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:147)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:287)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:67)
~[camel-direct-3.5.0.jar:3.5.0]
at
org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:217)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.processor.SharedCamelInternalProcessor$1.process(SharedCamelInternalProcessor.java:111)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.impl.engine.DefaultAsyncProcessorAwaitManager.process(DefaultAsyncProcessorAwaitManager.java:83)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:108)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.impl.engine.DefaultProducerCache.send(DefaultProducerCache.java:189)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:175)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:171)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:152)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.impl.engine.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:186)
~[camel-base-3.5.0.jar:3.5.0]
at
org.apache.camel.impl.engine.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:194)
~[camel-base-3.5.0.jar:3.5.0]
at
eu.tasgroup.camel.CamelRestController.hello2(CamelRestController.java:70)
~[classes/:na]

Obviously the subscription must not be done by my code, because it's Spring
Webflux which subscribes to the returned flux.
Thanks in advance.

Camel SFTP aggregation in streaming mode

[Florian.EIBL]

Dear community,

We have a problem (of understanding) concerning Camel streaming.

In our use case we pick up 3 large files with Camel SFTP component 
(=true=false). We aggregate the 3 files and write the 
aggregation per SFTP to a server. Due to the sizes of the input files, we have 
to stream.
We have 2 approaches, but none works as expected.

1. For the aggregation of the 3 Java InputStreams, we use 
java.io.SequenceInputStream 
(https://docs.oracle.com/javase/7/docs/api/java/io/SequenceInputStream.html)
This means that our Exchange body holds a SequenceInputStream after the 
aggregation. The SFTP producer  accepts the SequenceInputStream, since 
SequenceInputStream extends InputStream, but the SFTP producer does not write 
all 3 streams included in the SequenceInputStream (somehow it only writes one 
of the 3 streams, but all 3 streams are included!).

2. For the aggregation of the 3 Java InputStreams, we use 
org.apache.camel.converter.stream.CachedOutputStream and Camel stream caching. 
This works, but as soon as we use aggregation in our routes, the tmp file for 
the CachedOutputStream, which is created by stream caching, is never deleted.

How would you solve this problem? I would prefer a solution without writing tmp 
files. As we already spent a lot of time on this use case, I would appreciate 
any help. Thank you.

Camel Version: 3.4.2
JDK: 13


Regards,
Flo



[cid:image86f0b9.PNG@1f264e9b.428d9e59]

Florian EIBL
Software Engineer
Business Automation & Integration

Phone:  +43 664 6259733
Mobile: +43 664 6259733
E-Mail: florian.e...@spar-ics.com

SPAR Business Services GmbH
Information & Communication Services
Europastrasse 3, 5015 Salzburg, Austria



[cid:imagee1de8d.PNG@e08ed63b.4cb584cd]

Sollten Sie diese E-Mail unbeabsichtigt bzw. irrtümlich erhalten haben, so 
weisen wir Sie darauf hin, dass gemäß § 93 Abs 4 TKG der Inhalt sowie die 
Tatsache des Empfangs dieser E-Mail weder aufgezeichnet noch verwertet oder 
Unbefugten mitgeteilt werden dürfen. Wir ersuchen Sie, die Nachricht von Ihrem 
System zu löschen und sich mit uns in Verbindung zu setzen.

If you have received this email accidentally or in error, we point out that, in 
accordance with § 93 para. 4 TKG (Telecommunications Act), the contents of this 
email and the fact of its receipt must not be recorded, exploited or 
communicated to unauthorized persons. We ask you to delete the message from 
your system and to contact us.

RE: Are you using binary distribution?

[Ajmera, Hemang C]

I am relying on maven for binaries and git for sourcecode. IMHO binary 
distribution may not be needed. 
Github has release feature, where we can have binaries can be downloaded from 
there. Many users might be used to that way of downloading binary, so if 
needed, that can be alternate place for download if anyone needs it. However 
CI/CD pipeline might need updates for github release to work. 


Thanks and Regards,
Hemang Ajmera


-Original Message-
From: Zoran Regvart  
Sent: 14 December 2020 19:08
To: Dev ; users@camel.apache.org
Subject: Are you using binary distribution?


EXTERNAL SENDER:   Do not click any links or open any attachments unless you 
trust the sender and know the content is safe.
EXPÉDITEUR EXTERNE:Ne cliquez sur aucun lien et n’ouvrez aucune pièce 
jointe à moins qu’ils ne proviennent d’un expéditeur fiable, ou que vous ayez 
l'assurance que le contenu provient d'une source sûre.

Hi Cameleers,
we're discussing binary distribution on two issues[1][2]. The binary 
distribution is the tar.gz/ZIP file linked from the Camel website. By ASF 
policy we only ship source code, and the binary distribution is optional.

Back in the dark days, before using build tools that knew about dependency 
management (Maven, Gradle...) folk used to use the binary distribution.

I've found some statistics on downloads/per day, for us[3]/eu[4] and created 
these charts:

https://urldefense.com/v3/__https://s.apache.org/camel-dl-us__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDiSwKzT18$
https://urldefense.com/v3/__https://s.apache.org/camel-dl-eu__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDic4MBCm6$
 

The data is over 2 and 1/4 years, we've had 19.7+-8.8 via US, and
20.24+-8.43 in via EU per day. So not that much IMHO.

I'm wondering if anyone is still relying on these, and if so what would a 
binary distribution look like for sub projects? Should we do the same as we do 
for the Camel core?

Please reply on this thread or chime in on those issues for sub-project 
specific concerns.

Thanks :)

zoran

[1] 
https://urldefense.com/v3/__https://github.com/apache/camel-quarkus/issues/2045__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDiZ80_jXD$
[2] 
https://urldefense.com/v3/__https://github.com/apache/camel-kafka-connector/issues/754__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDiVlO1Pqs$
[3] 
https://urldefense.com/v3/__https://www-us.apache.org/dyn/stats/camel.log__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDiStt2eyy$
[4] 
https://urldefense.com/v3/__https://www-eu.apache.org/dyn/stats/camel.log__;!!AaIhyw!8ibMqXkFVhQZbXiUiDEZIMhodvHt6ZT0-9hO37pz6vxmkpWyl1rF-cNDicRsFgPE$
--
Zoran Regvart

Are you using binary distribution?

[Zoran Regvart]

Hi Cameleers,
we're discussing binary distribution on two issues[1][2]. The binary
distribution is the tar.gz/ZIP file linked from the Camel website. By
ASF policy we only ship source code, and the binary distribution is
optional.

Back in the dark days, before using build tools that knew about
dependency management (Maven, Gradle...) folk used to use the binary
distribution.

I've found some statistics on downloads/per day, for us[3]/eu[4] and
created these charts:

https://s.apache.org/camel-dl-us
https://s.apache.org/camel-dl-eu

The data is over 2 and 1/4 years, we've had 19.7+-8.8 via US, and
20.24+-8.43 in via EU per day. So not that much IMHO.

I'm wondering if anyone is still relying on these, and if so what
would a binary distribution look like for sub projects? Should we do
the same as we do for the Camel core?

Please reply on this thread or chime in on those issues for
sub-project specific concerns.

Thanks :)

zoran

[1] https://github.com/apache/camel-quarkus/issues/2045
[2] https://github.com/apache/camel-kafka-connector/issues/754
[3] https://www-us.apache.org/dyn/stats/camel.log
[4] https://www-eu.apache.org/dyn/stats/camel.log
--
Zoran Regvart

Re: How to produce messages to Azure SQL DB in Camel

[Omar Al-Safi]

Hi,

As mentioned here
, both use
JDBC driver meanwhile Camel SQL provides better support for SQL queries in
the endpoint. Personally, it is more what you'd feel more comfortable with,
try both and see what component that can fit your needs better.

Regards,
Omar

On Fri, Dec 11, 2020 at 6:58 PM SRIKANT MVS  wrote:

> Thanks for your reply, how better is it to use the Camel SQL compared to
> Camel JDBC.
>
> -Regards
> Srikant
>
> On Thu, 10 Dec, 2020, 10:53 Omar Al-Safi,  wrote:
>
> > Hi,
> >
> > To write your events to your target Azure SQL server database, you can
> > use Camel
> > JDBC 
> > component since Azure SQL supports JDBC uses JDBC driver to connect to
> the
> > database.
> > Plus Camel Debezium SQL Server, I'd recommend using something like Azure
> > Eventhub as event messaging medium in order to retain the events for some
> > period of time in case of failures .. etc, hence your pipeline may look
> > like this:
> > Azure SQL DB (live) -> Camel Debezium SQL Server -> Azure Eventhub ->
> Camel
> > JDBC -> Azure SQL DB (replication).
> >
> > There are few examples that can help you to get started:
> >
> >- Debezium with AWS Kinesis and Cassandra Example
> ><
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium
> > >
> >.
> >- Debezium with Azure Eventhubs with Azure Storage Blob Example.
> ><
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium-eventhubs-blob
> > >
> >- Camel JDBC Example
> ><
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-jdbc
> > >
> >.
> >
> > I hope it helps!
> >
> > Regards,
> > Omar
> >
> > On Thu, Dec 10, 2020 at 8:47 AM SRIKANT MVS 
> wrote:
> >
> > > Team,
> > > I have a usecase where I need to replicate data for a specific duration
> > > from one Azure SQL DB (subscription1) to another Azure SQL DB
> > > (supbscription2).
> > >
> > > The source DB has live data with continuous inserts, updates and
> deletes
> > > possibility. Henceforth, we will miss the updates while performing DB
> > copy
> > > operations.
> > >
> > > For consuming the messages, I am considering to use Debizium SQL Server
> > > Connector
> > > <
> > >
> >
> https://camel.apache.org/components/latest/debezium-sqlserver-component.html
> > > >,
> > > but this component
> > >
> > > *supports only the consumer functionality*
> > >
> > > Which camel component can I use to produce the same event to the
> > > destination DB ??
> > >
> > > -Srikant
> > >
> >
>