Re: sftp endpoint is not as performant as expected

[David Hoffer]

I'm rather new to Camel, I'm assuming that if the SFTP source gets 900
files then the configured
GenericFileExclusiveReadLockStrategy#acquireExclusiveReadLock will get
called 900 times (once per file) every polling cycle.

We too have a custom GenericFileExclusiveReadLockStrategy class that does
not block.  What ours does is maintain a map/cache of of the
lastModifiedTime and fileSize for each of the 900 files.  The call to
acquireExclusiveReadLock() will return true as soon as neither of those
values changed since the last poll cycle.

We have our delay/poll cycle set to 60 seconds.  I'm not sure why such a
large value but we are fine waiting 2 minutes to get each of the 900
files...and by the time 2 minutes go by we will likely have more files in
the SFTP source.  E.g. if we have a constant delay in receiving each file
of 2-3 minutes that is more than fine.  The problem is that on each cycle
it only processes 3-5 files instead of the 900 that should have returned
true from acquireExclusiveReadLock()

What seems to be happening is that Camel is not calling the
acquireExclusiveReadLock() method for each of the 900 files every 60
seconds, rather it slows down and either calls just a few of those 900
files or none.

Any ideas?

-Dave

Btw, I'm  using Camel 2.8.2



On Thu, Nov 12, 2015 at 12:43 PM, pmmerritt  wrote:

> We had a similar issue and traced it down to our usage of
> SftpChangedExclusiveReadLockStrategy which has a default check interval of
> 5
> seconds which causes the polling to slow down waiting on files to finish
> being written, so max you can do is about 1 per 5 seconds. You can change
> the checkInterval so that it is faster, or we ended up writing our own read
> lock strategy that did not block the polling thread
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/sftp-endpoint-is-not-as-performant-as-expected-tp5773654p5773780.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>

Re: sftp endpoint is not as performant as expected

[pmmerritt]

We had a similar issue and traced it down to our usage of
SftpChangedExclusiveReadLockStrategy which has a default check interval of 5
seconds which causes the polling to slow down waiting on files to finish
being written, so max you can do is about 1 per 5 seconds. You can change
the checkInterval so that it is faster, or we ended up writing our own read
lock strategy that did not block the polling thread



--
View this message in context: 
http://camel.465427.n5.nabble.com/sftp-endpoint-is-not-as-performant-as-expected-tp5773654p5773780.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel jetty response body

[souciance]

I figured out what the problem is.

In the REST DSL I turned bindingMode off like:
  

Then it worked. I could get the response back as OK instead of "OK" with
quotes.

Even with bindingMode set to auto, I would get "OK" instead of OK. 

I am not sure if this is a bug or not, at least with bindingMode set to auto
it should not try to convert my response which is just plain string to a
json. I have set the content-type to be text/plain.

Hopefully someone can shed a light on this.

Thanks.



--
View this message in context: 
http://camel.465427.n5.nabble.com/Camel-jetty-response-body-tp5773652p5773712.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: How to improve throughput by grouping messages in a Transaction (in JMS)

[deepak_a]

All,

I read the below link - and it appears this lets me batch messages during
consumption or publication
http://camel.apache.org/sjms.html
i.e. (multiple messages in one transaction).

But unfortunately this does not support XA (I am using Atomikos as
Transaction Manager)

https://issues.apache.org/jira/browse/CAMEL-6108

There is an open ticket for this – so likely to be available in future…..but
no ETA
So long story short – out-of-the box we do not have support for grouping
messages in one transaction.

Has any one found a work around for this? Would using spring-batch be
beneficial?


regards
D





--
View this message in context: 
http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773715.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: How to improve throughput by grouping messages in a Transaction (in JMS)

[Claus Ibsen]

See
http://camel.apache.org/sjms-batch.html

On Thu, Nov 12, 2015 at 12:24 PM, deepak_a  wrote:
> All,
>
> I read the below link - and it appears this lets me batch messages during
> consumption or publication
> http://camel.apache.org/sjms.html
> i.e. (multiple messages in one transaction).
>
> But unfortunately this does not support XA (I am using Atomikos as
> Transaction Manager)
>
> https://issues.apache.org/jira/browse/CAMEL-6108
>
> There is an open ticket for this – so likely to be available in future…..but
> no ETA
> So long story short – out-of-the box we do not have support for grouping
> messages in one transaction.
>
> Has any one found a work around for this? Would using spring-batch be
> beneficial?
>
>
> regards
> D
>
>
>
>
>
> --
> View this message in context: 
> http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773715.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: How to improve throughput by grouping messages in a Transaction (in JMS)

[deepak_a]

Thanks Claus,

The dependency looks exactly same as  SJMS, does this mean  SJMS has been
enhanced/re-branded as  SJMS-batch? in version 2.16?





org.apache.camel
camel-sjms
x.x.x



Also can you pls point me to the Javadoc? I can only see Javadoc available
for 2.15.
I need to check if there is support for XA transactions




--
View this message in context: 
http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773717.html
Sent from the Camel - Users mailing list archive at Nabble.com.

SimpleBuilder usage in camel

[Kasim Sert (Ibtech-Software Infrastructure)]

Hi,

I have following processor, when I run it from my route I get the following 
error.  I know exchange body is not null and you can see it in logs below. What 
is wrong with my usage of SimpleBuilder here ?

public class UpdateCustomerProcessor implements Processor {
public static final Logger log = 
LoggerFactory.getLogger(UpdateCustomerProcessor.class);

public void process(Exchange exchng) throws Exception {
Customer c = (Customer) exchng.getIn().getBody(Object[].class)[0];
System.out.println("Updating customer " + c.getFirstName() + " " + 
c.getLastName());
System.out.println(SimpleBuilder.simple("Hello 
${body.getFirstName()}").evaluate(exchng, String.class));
exchng.getOut().setBody(new Object[] {});
}

}

Updating customer kasim sert
org.apache.cxf.interceptor.Fault: Failed to invoke method: .getFirstName() on 
null due to: org.apache.camel.language.bean.RuntimeBeanExpressionException: 
Failed to invoke method: getFirstName() on null



[Facebook]  [Twitter] 


[https://www.finansbank.com.tr/Disclaimer/BannerImages.aspx?date=12.11.201514:1100]


Bu e-posta'n?n i?erdi?i bilgiler (ekleri dahil olmak ?zere) gizlidir. Onay?m?z 
olmaks?z?n ???nc? ki?ilere a?iklanamaz. Bu mesaj?n g?nderilmek istendi?i ki?i 
de?ilseniz, l?tfen mesaj? sisteminizden derhal siliniz. IBTech A.?. bu mesaj?n 
i?erdi?i bilgilerin do?rulu?u veya eksiksiz oldu?u konusunda bir garanti 
vermemektedir. Bu nedenle bilgilerin ne ?ekilde olursa olsun i?eri?inden, 
iletilmesinden, al?nmas?ndan, saklanmas?ndan sorumlu de?ildir. Bu mesaj?n 
i?eri?i yazar?na ait olup, IBTech A.?.'nin g?r??lerini i?ermeyebilir.

The information contained in this e-mail (including any attachments)is 
confidential. It must not be disclosed to any person without our authority. If 
you are not the intended recipient, please delete it from your system 
immediately. IBTech A.S. makes no warranty as to the accuracy or completeness 
of any information contained in this message and hereby excludes any liability 
of any kind for the information contained therein or for the information 
transmission, reception, storage or use of such in any way whatsoever. Any 
opinions expressed in this message are those of the author and may not 
necessarily reflect the opinions of IBTech A.S.

Re: [ANNOUNCEMENT] Apache Camel 2.14.4 Released

[Andrea Cosentino]

Thank you very much Gregor :-)
--  Andrea Cosentino  -Apache Camel PMC Member  
Email: ancosen1...@yahoo.com  Twitter: @oscerd2  Github: oscerd  


 On Wednesday, November 11, 2015 11:56 PM, Gregor Zurowski 
 wrote:
   

 The Apache Camel community announces the immediate availability of the
new patch release Apache Camel 2.14.4. This release contains 18 fixes
applied in the past few weeks by the community on the Camel 2.14.x
maintenance branch. We expect this to be the last patch release for
the 2.14.x branch.

The artifacts are published and ready for you to download [1] either
from the Apache mirrors or from the Maven Central repository. For more
details please take a look at the release notes [2, 3].

Many thanks to all who made this release possible.

On behalf of the Camel PMC,
Gregor Zurowski

[1] http://camel.apache.org/download.html
[2] http://camel.apache.org/camel-2144-release.html
[3] 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311211=12332478


  

Spring-Boot + Camel + producerTemplate ssh spawning thousands of threads

[codan84]

I did write a simple app using Spring Boot (1.2.7.RELEASE) and Apache Camel
(2.15.0). The app is simple and has only 1 route: a timer will invoke a
method on a bean every 1s. The method invoked will use ProducerTemplate to
ssh into a remote machine, execute a small script, and print out the output
to the console. Simple, right?
However, when profiling this, I can see the number of threads! It seems like
any threads created for the ssh are never killed, but parked instead.
Because of that I run OOM pretty quickly with an error:

Exception in thread "Thread-341" java.lang.OutOfMemoryError: unable to
create new native thread

More description and profiler results can be found on stackoverflow:
http://stackoverflow.com/questions/33671567/spring-boot-camel-producertemplate-memory-leak

I think this is a bug, as from the documentation I can't see that I am using
the ProducerTemplate wrongly.
Or am I? If so, please tell me. If I am doing things right and it seems like
a bug, I will submit it...



--
View this message in context: 
http://camel.465427.n5.nabble.com/Spring-Boot-Camel-producerTemplate-ssh-spawning-thousands-of-threads-tp5773741.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Spring-Boot + Camel + producerTemplate ssh spawning thousands of threads

[Jakub Korab]

To work out if it's your use of the ProducerTemplate, try the following 
route instead and see whether the leak persists:


from("timer://foo?period=1000")
.transform().constant("/home/_username_/some_temp_script.sh")
.to("ssh://_remote.machine.url.here_?username=_username_=#keyPairProvider")
.log("Received: ${body}");

On 12/11/15 14:35, codan84 wrote:

I did write a simple app using Spring Boot (1.2.7.RELEASE) and Apache Camel
(2.15.0). The app is simple and has only 1 route: a timer will invoke a
method on a bean every 1s. The method invoked will use ProducerTemplate to
ssh into a remote machine, execute a small script, and print out the output
to the console. Simple, right?
However, when profiling this, I can see the number of threads! It seems like
any threads created for the ssh are never killed, but parked instead.
Because of that I run OOM pretty quickly with an error:

Exception in thread "Thread-341" java.lang.OutOfMemoryError: unable to
create new native thread

More description and profiler results can be found on stackoverflow:
http://stackoverflow.com/questions/33671567/spring-boot-camel-producertemplate-memory-leak

I think this is a bug, as from the documentation I can't see that I am using
the ProducerTemplate wrongly.
Or am I? If so, please tell me. If I am doing things right and it seems like
a bug, I will submit it...



--
View this message in context: 
http://camel.465427.n5.nabble.com/Spring-Boot-Camel-producerTemplate-ssh-spawning-thousands-of-threads-tp5773741.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: How to improve throughput by grouping messages in a Transaction (in JMS)

[Jakub Korab]

Hi Deepak,

The SJMS Batch component is included in the same library, but it's a 
different Component implementation specifically for consuming batches of 
messages. It works using local JMS transactions only, and does not 
integrate with XA.


Jakub

On 12/11/15 11:59, deepak_a wrote:

Thanks Claus,

The dependency looks exactly same as  SJMS, does this mean  SJMS has been
enhanced/re-branded as  SJMS-batch? in version 2.16?





 org.apache.camel
 camel-sjms
 x.x.x
 



Also can you pls point me to the Javadoc? I can only see Javadoc available
for 2.15.
I need to check if there is support for XA transactions




--
View this message in context: 
http://camel.465427.n5.nabble.com/How-to-improve-throughput-by-grouping-messages-in-a-Transaction-tp5773344p5773717.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: SimpleBuilder usage in camel

[Joakim Bjørnstad]

Hello,

In your example, ${body} is still of type Object[]. So it seems the
SimpleBuilder evaluates it to null since getFirstName() is not a
method on the Object[] in body.

Try:

System.out.println(SimpleBuilder.simple("Hello
${body[0].getFirstName()}").evaluate(exchng, String.class));

On Thu, Nov 12, 2015 at 1:11 PM, Kasim Sert (Ibtech-Software
Infrastructure)  wrote:
> Hi,
>
> I have following processor, when I run it from my route I get the following 
> error.  I know exchange body is not null and you can see it in logs below. 
> What is wrong with my usage of SimpleBuilder here ?
>
> public class UpdateCustomerProcessor implements Processor {
> public static final Logger log = 
> LoggerFactory.getLogger(UpdateCustomerProcessor.class);
>
> public void process(Exchange exchng) throws Exception {
> Customer c = (Customer) exchng.getIn().getBody(Object[].class)[0];
> System.out.println("Updating customer " + c.getFirstName() + " " + 
> c.getLastName());
> System.out.println(SimpleBuilder.simple("Hello 
> ${body.getFirstName()}").evaluate(exchng, String.class));
> exchng.getOut().setBody(new Object[] {});
> }
>
> }
>
> Updating customer kasim sert
> org.apache.cxf.interceptor.Fault: Failed to invoke method: .getFirstName() on 
> null due to: org.apache.camel.language.bean.RuntimeBeanExpressionException: 
> Failed to invoke method: getFirstName() on null
>
>
>
> [Facebook]  [Twitter] 
> 
>
> [https://www.finansbank.com.tr/Disclaimer/BannerImages.aspx?date=12.11.201514:1100]
>
>
> Bu e-posta'n?n i?erdi?i bilgiler (ekleri dahil olmak ?zere) gizlidir. 
> Onay?m?z olmaks?z?n ???nc? ki?ilere a?iklanamaz. Bu mesaj?n g?nderilmek 
> istendi?i ki?i de?ilseniz, l?tfen mesaj? sisteminizden derhal siliniz. IBTech 
> A.?. bu mesaj?n i?erdi?i bilgilerin do?rulu?u veya eksiksiz oldu?u konusunda 
> bir garanti vermemektedir. Bu nedenle bilgilerin ne ?ekilde olursa olsun 
> i?eri?inden, iletilmesinden, al?nmas?ndan, saklanmas?ndan sorumlu de?ildir. 
> Bu mesaj?n i?eri?i yazar?na ait olup, IBTech A.?.'nin g?r??lerini 
> i?ermeyebilir.
>
> The information contained in this e-mail (including any attachments)is 
> confidential. It must not be disclosed to any person without our authority. 
> If you are not the intended recipient, please delete it from your system 
> immediately. IBTech A.S. makes no warranty as to the accuracy or completeness 
> of any information contained in this message and hereby excludes any 
> liability of any kind for the information contained therein or for the 
> information transmission, reception, storage or use of such in any way 
> whatsoever. Any opinions expressed in this message are those of the author 
> and may not necessarily reflect the opinions of IBTech A.S.



-- 
Kind regards
Joakim Bjørnstad

RE: SimpleBuilder usage in camel

[Kasim Sert (Ibtech-Software Infrastructure)]

Thank you it works the way you suggest. By the way I can use the code below to 
get customer pojo. 

Customer c = exchng.getIn().getBody(Customer.class);

But when I use it in the SimpleBuilder i should use with array notation ? I did 
not get the reason well.

Thank you anyway.

-Original Message-
From: Joakim Bjørnstad [mailto:joak...@gmail.com] 
Sent: Thursday, November 12, 2015 3:10 PM
To: users@camel.apache.org
Subject: Re: SimpleBuilder usage in camel

Hello,

In your example, ${body} is still of type Object[]. So it seems the 
SimpleBuilder evaluates it to null since getFirstName() is not a method on the 
Object[] in body.

Try:

System.out.println(SimpleBuilder.simple("Hello
${body[0].getFirstName()}").evaluate(exchng, String.class));

On Thu, Nov 12, 2015 at 1:11 PM, Kasim Sert (Ibtech-Software
Infrastructure)  wrote:
> Hi,
>
> I have following processor, when I run it from my route I get the following 
> error.  I know exchange body is not null and you can see it in logs below. 
> What is wrong with my usage of SimpleBuilder here ?
>
> public class UpdateCustomerProcessor implements Processor {
> public static final Logger log = 
> LoggerFactory.getLogger(UpdateCustomerProcessor.class);
>
> public void process(Exchange exchng) throws Exception {
> Customer c = (Customer) exchng.getIn().getBody(Object[].class)[0];
> System.out.println("Updating customer " + c.getFirstName() + " " + 
> c.getLastName());
> System.out.println(SimpleBuilder.simple("Hello 
> ${body.getFirstName()}").evaluate(exchng, String.class));
> exchng.getOut().setBody(new Object[] {});
> }
>
> }
>
> Updating customer kasim sert
> org.apache.cxf.interceptor.Fault: Failed to invoke method: 
> .getFirstName() on null due to: 
> org.apache.camel.language.bean.RuntimeBeanExpressionException: Failed 
> to invoke method: getFirstName() on null
>
>
>
> [Facebook]  [Twitter] 
> 
>
> [https://www.finansbank.com.tr/Disclaimer/BannerImages.aspx?date=12.11
> .201514:1100] ?date=12.11.201514:1100>
>
>
> Bu e-posta'n?n i?erdi?i bilgiler (ekleri dahil olmak ?zere) gizlidir. 
> Onay?m?z olmaks?z?n ???nc? ki?ilere a?iklanamaz. Bu mesaj?n g?nderilmek 
> istendi?i ki?i de?ilseniz, l?tfen mesaj? sisteminizden derhal siliniz. IBTech 
> A.?. bu mesaj?n i?erdi?i bilgilerin do?rulu?u veya eksiksiz oldu?u konusunda 
> bir garanti vermemektedir. Bu nedenle bilgilerin ne ?ekilde olursa olsun 
> i?eri?inden, iletilmesinden, al?nmas?ndan, saklanmas?ndan sorumlu de?ildir. 
> Bu mesaj?n i?eri?i yazar?na ait olup, IBTech A.?.'nin g?r??lerini 
> i?ermeyebilir.
>
> The information contained in this e-mail (including any attachments)is 
> confidential. It must not be disclosed to any person without our authority. 
> If you are not the intended recipient, please delete it from your system 
> immediately. IBTech A.S. makes no warranty as to the accuracy or completeness 
> of any information contained in this message and hereby excludes any 
> liability of any kind for the information contained therein or for the 
> information transmission, reception, storage or use of such in any way 
> whatsoever. Any opinions expressed in this message are those of the author 
> and may not necessarily reflect the opinions of IBTech A.S.



--
Kind regards
Joakim Bjørnstad


Bu e-posta'nin içerdigi bilgiler (ekleri dahil olmak 
üzere) gizlidir. Onayimiz olmaksizin üçüncü kisilere açiklanamaz. Bu mesajin 
gönderilmek istendigi kisi degilseniz, lütfen mesaji sisteminizden derhal 
siliniz. IBTech A.S. bu mesajin içerdigi bilgilerin dogrulugu veya eksiksiz 
oldugu konusunda bir garanti vermemektedir. Bu nedenle bilgilerin ne sekilde 
olursa olsun içeriginden, iletilmesinden, alinmasindan, saklanmasindan sorumlu 
degildir. Bu mesajin içerigi yazarina ait olup, IBTech A.S.'nin görüslerini 
içermeyebilir.

The information contained in this e-mail (including any 
attachments)is confidential. It must not be disclosed to any person without our 
authority. If you are not the intended recipient, please delete it from your 
system immediately. IBTech A.S. makes no warranty as to the accuracy or 
completeness of any information contained in this message and hereby excludes 
any liability of any kind for the information contained therein or for the 
information transmission, reception, storage or use of such in any way 
whatsoever. Any opinions expressed in this message are those of the author and 
may not necessarily reflect the opinions of IBTech 
A.S.

Messages remain unacknowledged when exception thrown during RabbitMQ InOut

[E Wong]

Hello,

We’re using the new rabbitMQ InOut feature in camel version 2.16.0
(with rabbitMQ v3.5.5) and have set up a flow like the following:

Producer -> route1 -> rabbitMQ -> route2 -> rabbitMQ -> route3 -> service bean

The service bean will return a response which the caller can choose
whether or not to be interested in, so essentially we want the flow to
be able to work asynchronously and synchronously.  Ideally we’d like
to reuse the same routes, where the caller can just indicate whether
it cares about a response by setting the exchange pattern to InOnly or
InOut.

The rabbitMQ to and from endpoints are configured like the following:

/* route1 */
from( "direct:route1" )
.to( 
"rabbitmq://localhost:5672/testExchange?exchangeType=topic=false=false"
);

/* route2 */
from( 
"rabbitmq://localhost:5672/testExchange?queue=queue.test.exchange=topic=false"
+

"=direct=deadLetterTestExchange"
+
"=deadletter.test.exchange=#" +
"=false=true" )
.throwException( new RuntimeException( "test exception" ) );

We have set autoAck=false so that, for an InOnly exchange, if there
are any exceptions, the message will be rejected and pushed to a
deadletter queue.

However for an InOut exchange, if an exception is thrown, given we
have autoAck=false and transferException=true, the original message
will remain unacknowledged forever on rabbitMQ, even though the caller
will receive and handle the exception. If the server is restarted, the
application will attempt to reprocess the message.

We would like to propose a patch to include a basicAck in the rabbitMQ
consumer in this scenario.  A basicAck seems more appropriate (rather
than a rejection) since the caller would be able to handle the
exception and therefore there should be no need to send the message to
a deadletter queue.

//In the RabbitMQConsumer
else if (endpoint.isTransferException() &&
exchange.getPattern().isOutCapable()) {
// the inOut exchange failed so put the exception in the body and send back
msg.setBody(exchange.getException());
exchange.setOut(msg);
endpoint.publishExchangeToChannel(exchange, channel,
properties.getReplyTo());

//Add something like this
if (!consumer.endpoint.isAutoAck()) {
channel.basicAck(deliveryTag, false);
}
}

Would a solution like this be acceptable or was there a specific
reason why no ack or reject was included in this scenario?

Thanks

cxfendpoint consumer fails for certain WS soap calls (2.16.0) - nullpoint exception

[bocamel]

After upgrade to 2.16.0, certain WS soap calls fail with the error below. 
This works with all the combinations of 2.15.x and CXF 3.1.x.  But it does
not work for 2.16.0 and CXF 3.1.x.  Is there any change I need to make for
the CXF endpoint (consumer or producer) for 2.16.0?  Using IDE, I could see
that for some soap calls, the in message body can be converted to String. 
But others it would only show CXFPayload.  These soap calls are almost
identical.

Thanks for any help!

Stacktrace
---
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
store file:
log\acme\uat\.\2015-11-12\failed_messages\20151112-222445902.xml
at
org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:292)
at
org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:277)
at
org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:165)
at
org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:79)
at
org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at
org.apache.camel.processor.SendDynamicProcessor$1.doInAsyncProducer(SendDynamicProcessor.java:124)
at
org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:341)
at
org.apache.camel.processor.SendDynamicProcessor.process(SendDynamicProcessor.java:119)
at
org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:460)
at
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)
at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:87)
at
org.apache.camel.processor.WireTapProcessor$1.call(WireTapProcessor.java:137)
at
org.apache.camel.processor.WireTapProcessor$1.call(WireTapProcessor.java:133)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.camel.InvalidPayloadException: No body available of
type: java.io.InputStream but has value:
org.apache.camel.component.cxf.CxfPayload@297d8e2e of type:
org.apache.camel.component.cxf.CxfPayload on: Message:
org.apache.camel.component.cxf.CxfPayload@297d8e2e. Caused by: Error during
type conversion from type: java.lang.String to the required type:
java.io.InputStream with value
org.apache.camel.component.cxf.CxfPayload@297d8e2e due Problem converting
content to Element. Exchange[ID-John-PCI-54846-1447385077811-0-10][Message:
org.apache.camel.component.cxf.CxfPayload@297d8e2e]. Caused by:
[org.apache.camel.TypeConversionException - Error during type conversion
from type: java.lang.String to the required type: java.io.InputStream with
value org.apache.camel.component.cxf.CxfPayload@297d8e2e due Problem
converting content to Element]
at
org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:101)
at
org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:273)
... 17 more
Caused by: org.apache.camel.TypeConversionException: Error during type
conversion from type: java.lang.String to the required type:
java.io.InputStream with value
org.apache.camel.component.cxf.CxfPayload@297d8e2e due Problem converting
content to Element
at
org.apache.camel.impl.converter.BaseTypeConverterRegistry.createTypeConversionException(BaseTypeConverterRegistry.java:610)
at
org.apache.camel.impl.converter.BaseTypeConverterRegistry.mandatoryConvertTo(BaseTypeConverterRegistry.java:177)
at
org.apache.camel.impl.MessageSupport.getMandatoryBody(MessageSupport.java:99)
... 18 more
Caused by: org.apache.camel.RuntimeCamelException: Problem converting
content to Element
at org.apache.camel.component.cxf.CxfPayload$1.get(CxfPayload.java:101)
at org.apache.camel.component.cxf.CxfPayload$1.get(CxfPayload.java:66)
at
org.apache.camel.component.cxf.converter.NodeListWrapper.item(NodeListWrapper.java:37)
at
org.apache.camel.converter.jaxp.DomConverter.toString(DomConverter.java:73)
at
org.apache.camel.converter.jaxp.DomConverter.toByteArray(DomConverter.java:144)
at
org.apache.camel.converter.jaxp.DomConverter.toInputStream(DomConverter.java:139)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62

Start/Stop CamelContext

[Sashika]

Hi,
I'm in need to load some camel route attributes like file paths etc from the
database and I'm initializing CamelContext from within spring. Also I use java
DSL rather than XML DSL. I need camel to pickup the new values if I change those
in the database.
1. Do I have to restart the camel context in order for the new attributes to
   take effect?
2. If so how to start/stop CamelContext from within a Spring web application

Any help is appreciated.
Regards Sashika.

Re: Spring-Boot + Camel + producerTemplate ssh spawning thousands of threads

[Claus Ibsen]

Maybe your bean with the producer template is not singleton scoped, so
spring creates a new instance per message.

On Thu, Nov 12, 2015 at 4:10 PM, codan84  wrote:
> Hmm Interesting, this indeed fixes the problem. I am forced to use a
> producerTemplate tho... Any ideas what am I doing wrong?
>
>
>
> --
> View this message in context: 
> http://camel.465427.n5.nabble.com/Spring-Boot-Camel-producerTemplate-ssh-spawning-thousands-of-threads-tp5773741p5773747.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Spring-Boot + Camel + producerTemplate ssh spawning thousands of threads

[codan84]

Hmm Interesting, this indeed fixes the problem. I am forced to use a
producerTemplate tho... Any ideas what am I doing wrong?



--
View this message in context: 
http://camel.465427.n5.nabble.com/Spring-Boot-Camel-producerTemplate-ssh-spawning-thousands-of-threads-tp5773741p5773747.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Spring-Boot + Camel + producerTemplate ssh spawning thousands of threads

[codan84]

I have tried similar thing, but with sftp rather than ssh. Everything else is
exactly the same:

producerTemplate.sendBodyAndHeader(
"sftp://_target_machine_url_?username=_username_; +
"=" + sshKeyPath +
"=" + knownHosts +
"=publickey",
new ByteArrayInputStream(new String("Random
junk").getBytes()),
"CamelFileName", "/home/_username_/test_file.txt"
);

And this works fine. So it is not the way I am using ProducerTemplate. It is
a bug with ssh when using producer template, or "requestBody" method...



--
View this message in context: 
http://camel.465427.n5.nabble.com/Spring-Boot-Camel-producerTemplate-ssh-spawning-thousands-of-threads-tp5773741p5773751.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: How to test routes when using another TestRunner

[souciance]

I got it to finally work. Thanks a lot!

Essentially you need to add this.setUp(); in the method that exist in the
class that extends CamelBlueprintTestSupport.

Then, your cucumber methods simply call the method..and that seems to do the
trick.



--
View this message in context: 
http://camel.465427.n5.nabble.com/How-to-test-routes-when-using-another-TestRunner-tp5772687p5773748.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Spring-Boot + Camel + producerTemplate ssh spawning thousands of threads

[codan84]

In that case starting/stopping it should in theory release threads, right?
I tried start->requestBody->stop, without effect, still all threads keep
running. I also tried to instantiate new template for each call, start it
and stop at the end, still the same.



--
View this message in context: 
http://camel.465427.n5.nabble.com/Spring-Boot-Camel-producerTemplate-ssh-spawning-thousands-of-threads-tp5773741p5773750.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Problem with bean method calls and parameter types

[Ryan Moquin]

Hi, I'm using Camel 2.16.0 and trying to invoke a bean method using spring
xml and having some issues.  It seems like any time I try to invoke a bean
method with a camel message body, camel tries to pass it as a string to the
method (even though the method takes the class as an argument) I'm calling
even though the body type is correctly set to the java class type and the
body itself is a class.  I've called bean methods a lot from Camel and
don't remember this being normal behavior?  Or am I wrong?

Caused by: org.apache.camel.component.bean.ParameterBindingException: Error
during parameter binding on method: public java.lang.String
com.test.TestService.retrieve(com.test.bo.TestObject,java.util.Date,com.test.TestParam)
at parameter #0 with type: class com.test.bo.TestObject with value type:
class java.lang.String and value: com.test.bo.TestObject
at
org.apache.camel.component.bean.MethodInfo$2.evaluateParameterValue(MethodInfo.java:582)
...
Caused by: org.apache.camel.NoTypeConversionAvailableException: No type
converter available to convert from type: java.lang.String to the required
type: com.test.bo.TestObject with value com.test.bo.TestObject
at
org.apache.camel.impl.converter.BaseTypeConverterRegistry.mandatoryConvertTo(BaseTypeConverterRegistry.java:1
85)

I'm just calling the bean method like you normally would.  I can log the
value of a property of the java class in the body using a log statement
right before I do the invocation of the method.  I don't do anything
special when calling the method other than putting the first parameter as:
${body}

Is this by design in Camel and I never noticed?  If so, is there any way
around it?

Thanks for any help!
Ryan

Re: How to consume selected files from FTP endpoint?

[Jack Frosch]

Thanks Claus.

I’ll take another run at this.

—
Jack Frosch







On 11/12/15, 10:08 AM, "Claus Ibsen"  wrote:

>The filter should work while scanning for files - not after. eg if you
>configure the filter on the ftp endpoint. There is a bunch of options
>for filtering with include/exclude/ant/filter etc.
>
>As ftp extends file you can find all the options in the file docs
>http://camel.apache.org/file2
>
>On Thu, Nov 12, 2015 at 5:04 PM, Jack Frosch  wrote:
>> Our application consumes files from a folder on a supplier's FTP server.  
>> Unfortunately, the supplier uses the same folder for documents intended to 
>> be consumed by other applications. We’d like to consume (i.e. read, then 
>> delete or move) our files which can be identified by file name pattern, 
>> leaving the other files we don’t care about in the folder.
>>
>> I was hoping a filter would do this, but it seems the filter gets applied 
>> after the file is consumed.
>>
>> I’m sure I’m overlooking something, but am Googled out. :-)
>>
>> Is there a way to configure the FTP endpoint using Camel 2.15.1 to only 
>> consume a subset of files from a folder on an FTP server?
>>
>> Thanks in advance!
>>
>> —
>> Jack
>>
>
>
>
>-- 
>Claus Ibsen
>-
>http://davsclaus.com @davsclaus
>Camel in Action 2: https://www.manning.com/ibsen2

Re: How to consume selected files from FTP endpoint?

[Claus Ibsen]

The filter should work while scanning for files - not after. eg if you
configure the filter on the ftp endpoint. There is a bunch of options
for filtering with include/exclude/ant/filter etc.

As ftp extends file you can find all the options in the file docs
http://camel.apache.org/file2

On Thu, Nov 12, 2015 at 5:04 PM, Jack Frosch  wrote:
> Our application consumes files from a folder on a supplier's FTP server.  
> Unfortunately, the supplier uses the same folder for documents intended to be 
> consumed by other applications. We’d like to consume (i.e. read, then delete 
> or move) our files which can be identified by file name pattern, leaving the 
> other files we don’t care about in the folder.
>
> I was hoping a filter would do this, but it seems the filter gets applied 
> after the file is consumed.
>
> I’m sure I’m overlooking something, but am Googled out. :-)
>
> Is there a way to configure the FTP endpoint using Camel 2.15.1 to only 
> consume a subset of files from a folder on an FTP server?
>
> Thanks in advance!
>
> —
> Jack
>



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

How to consume selected files from FTP endpoint?

[Jack Frosch]

Our application consumes files from a folder on a supplier's FTP server.  
Unfortunately, the supplier uses the same folder for documents intended to be 
consumed by other applications. We’d like to consume (i.e. read, then delete or 
move) our files which can be identified by file name pattern, leaving the other 
files we don’t care about in the folder.

I was hoping a filter would do this, but it seems the filter gets applied after 
the file is consumed.

I’m sure I’m overlooking something, but am Googled out. :-)

Is there a way to configure the FTP endpoint using Camel 2.15.1 to only consume 
a subset of files from a folder on an FTP server?

Thanks in advance!

—
Jack

Re: Spring-Boot + Camel + producerTemplate ssh spawning thousands of threads

[codan84]

I tried your suggestion:

ProducerTemplate producerTemplate =
camelContext.createProducerTemplate();
producerTemplate.start();
String response = producerTemplate.requestBody("ssh://",
String.class);
producerTemplate.stop();

There is no difference. Still the number of threads grows. 
I would disagree that this is not a bug. Especially with a Spring-Boot
application, where the app will usually run on a server forever, this is an
issue. It is unfortunate how Camel handles non-singleton endpoints, but it
should either be very well documented (bold, red etc), or a different
approach to non-singleton endpoints worked out.



--
View this message in context: 
http://camel.465427.n5.nabble.com/Spring-Boot-Camel-producerTemplate-ssh-spawning-thousands-of-threads-tp5773741p5773755.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Spring-Boot + Camel + producerTemplate ssh spawning thousands of threads

[codan84]

Indeed the ClientSession of SshClient is being started, but never finished.
Here are logs:

2015-11-12 16:39:08 [sshd-SshClient[68c05218]-nio2-thread-1] INFO 
o.a.s.c.session.ClientSessionImpl - Client session created
2015-11-12 16:39:08 [sshd-SshClient[68c05218]-nio2-thread-2] INFO 
o.a.s.c.session.ClientSessionImpl - Server version string:
SSH-1.99-OpenSSH_3.9p1
2015-11-12 16:39:08 [sshd-SshClient[68c05218]-nio2-thread-3] INFO 
o.a.s.c.session.ClientSessionImpl - Kex: server->client aes128-ctr hmac-sha1
none
2015-11-12 16:39:08 [sshd-SshClient[68c05218]-nio2-thread-3] INFO 
o.a.s.c.session.ClientSessionImpl - Kex: client->server aes128-ctr hmac-sha1
none
2015-11-12 16:39:09 [sshd-SshClient[68c05218]-nio2-thread-5] WARN 
o.a.s.c.k.AcceptAllServerKeyVerifier - Server at *** presented unverified
DSA key: ***

Script output :)


This just repeats for every call.



--
View this message in context: 
http://camel.465427.n5.nabble.com/Spring-Boot-Camel-producerTemplate-ssh-spawning-thousands-of-threads-tp5773741p5773762.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Spring-Boot + Camel + producerTemplate ssh spawning thousands of threads

[Jakub Korab]

When you use the SSH producer endpoint in a to(..) statement, one 
instance of the endpoint is created within the route, its lifecycle is 
tied to the CamelContext and any resources used by it will be cleaned up 
at shutdown. The endpoint creates an instance of an SshClient 
(underlying library), and closes it when the context shuts down.


When you use it through a ProducerTemplate, each time you access the 
send() method it creates a new instance of the SshEndpoint, allocating 
memory on the way by creating a new underlying SshClient instance. This 
behaviour is dictated by SshEndpoint. isSingleton(), which returns false 
as the underlying client library is not thread-safe. You need to close 
the ProducerTemplate in order to close the SshClient instance (note, 
nothing to do with the SSH connection itself).


As a workaround, you might consider having the CamelContext injected 
into your bean, and for each request do:


ProducerTemplate template = context.createProducerTemplate();
template.sendBody(...);
template.close();

It's expensive, but it will do the job.

I don't think this is a bug, if seems to be a side-effect of how a 
ProducerTemplate works with non-singleton endpoints. If there were 
multiple threads, then the ProducerTemplate would have to create 
multiple endpoints for the underlying library to work correctly, and for 
the threads to not trip over each other. In your case, only one thread 
will ever be using that one ProducerTemplate, so it doesn't make sense. 
The ProducerTemplate can't know in advance how many threads will be 
accessing it.


On 12/11/15 15:10, codan84 wrote:

Hmm Interesting, this indeed fixes the problem. I am forced to use a
producerTemplate tho... Any ideas what am I doing wrong?



--
View this message in context: 
http://camel.465427.n5.nabble.com/Spring-Boot-Camel-producerTemplate-ssh-spawning-thousands-of-threads-tp5773741p5773747.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Create and call a route from Processor

[Ishada]

I googled out but I could not find any single insight about the problem.
which is as follows

Get a payload from JMS queue.
get the details of the FTP server from the payload and archive all the
files to a different location.
Do some Processing on the payload data and store it as a text files on
the local system.

As it can be seen that the 2nd step is nothing but starting a different
route if I am not mistaken. I thought of adding a processor which would
start a route to consume all the files from the ftp location and producing
it to some endpoint. This step it has to do nothing with the exchange which
I receive from the first step. So is there a way we can implement this
functionality.

I have completed 1st and 3rd step as it is easy and there are many examples
all over the internet.

If you have come across this situation, please guide me.




--
View this message in context: 
http://camel.465427.n5.nabble.com/Create-and-call-a-route-from-Processor-tp5773761.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: [ANNOUNCEMENT] Apache Camel 2.14.4 Released

[Christian Müller]

Great!

On Wed, Nov 11, 2015 at 11:55 PM, Gregor Zurowski 
wrote:

> The Apache Camel community announces the immediate availability of the
> new patch release Apache Camel 2.14.4. This release contains 18 fixes
> applied in the past few weeks by the community on the Camel 2.14.x
> maintenance branch. We expect this to be the last patch release for
> the 2.14.x branch.
>
> The artifacts are published and ready for you to download [1] either
> from the Apache mirrors or from the Maven Central repository. For more
> details please take a look at the release notes [2, 3].
>
> Many thanks to all who made this release possible.
>
> On behalf of the Camel PMC,
> Gregor Zurowski
>
> [1] http://camel.apache.org/download.html
> [2] http://camel.apache.org/camel-2144-release.html
> [3]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311211=12332478
>