Re: Component "camel-ssh" broken? - NoSuchMethodError: ConnectFuture.getSession()
Camel 3.7.x doesn't support Karaf 4.3.0, the last version we tested is 4.2.9 Il giorno mar 26 gen 2021 alle ore 22:00 Schulze, Jan < jan.schu...@uni-tuebingen.de> ha scritto: > Hi Claus, > > > thanks for your reply. I forgot to mention, that I am using Camel in Karaf. > > Along with Camel I also upgraded Karaf (4.2.9 => 4.3.0). > > > When using Camel 2.7.1 with Karaf 4.2.9, the camel-ssh component is > working without problems. > > > Also, the camel features URLs for installing Camel into Karaf both specify > mvn:org.apache.sshd/sshd-core/2.0.0 as dependency for camel-ssh > > > https://repo1.maven.org/maven2/org/apache/camel/karaf/apache-camel/3.4.0/apache-camel-3.4.0-features.xml > > > https://repo1.maven.org/maven2/org/apache/camel/karaf/apache-camel/3.7.1/apache-camel-3.7.1-features.xml > > > So it seems to be a Karaf-related issue. > > > Using "feature:install camel-ssh" results in the following in a clean > Karaf 4.2.9: > > > karaf@root()> bundle:list -t 0 | grep -i ssh > 35 x Active x 30 x 4.2.9x Apache Karaf :: Shell :: SSH > 38 x Active x 30 x 1.7.0x Apache Mina SSHD :: Core > 79 x Active x 50 x 3.7.1x camel-ssh > 101 x Active x 50 x 2.0.0x Apache Mina SSHD :: Core > > > And in a clean Karaf 4.3.0 it results in: > > karaf@root()> bundle:list -t 0 | grep -i ssh > 44 x Active x 30 x 4.3.0 x Apache Karaf :: Shell :: SSH > 46 x Active x 30 x 2.5.1 x Apache Mina SSHD :: OSGi > 47 x Active x 30 x 2.5.1 x Apache Mina SSHD :: SCP > 48 x Active x 30 x 2.5.1 x Apache Mina SSHD :: SFTP > 121 x Active x 50 x 3.7.1 x camel-ssh > > > I'm too tired right now to persue this any further. Maybe I can resolve it > tomorrow. > > > > Regards > -- > Jan Schulze > Eberhard Karls Universität Tübingen > > Von: Claus Ibsen > Gesendet: Dienstag, 26. Januar 2021 13:26:16 > An: users@camel.apache.org > Betreff: Re: Component "camel-ssh" broken? - NoSuchMethodError: > ConnectFuture.getSession() > > Hi > > Thanks for reporting. Can you create a JIRA ticket. > And would you be able to try test with switching to use mina 2.0 JAR > on the classpath but keep using the 3.7.1 camel version. > > On Tue, Jan 26, 2021 at 1:20 PM Schulze, Jan > wrote: > > > > Hi, > > > > "camel-ssh" component is throwing a CamelExecutionException when it is > used to write a file via SSH. > > It used to be working in Camel 3.4.0, but after upgrading to 3.7.1, I > observe the following stack trace: > > > > org.apache.camel.CamelExecutionException: Exception occurred during > execution on the exchange: Exchange[F2205BFA0B31B1C-] > > at > org.apache.camel.CamelExecutionException.wrapCamelExecutionException(CamelExecutionException.java:45) > ~[!/:3.7.1] > > at > org.apache.camel.support.DefaultExchange.setException(DefaultExchange.java:425) > ~[!/:3.7.1] > > at > org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:69) > ~[!/:3.7.1] > > at > org.apache.camel.processor.SendProcessor.lambda$process$2(SendProcessor.java:188) > ~[!/:3.7.1] > > at > org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:317) > ~[!/:3.7.1] > > at > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:187) > ~[!/:3.7.1] > > at > org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.doRun(RedeliveryErrorHandler.java:714) > [!/:3.7.1] > > at > org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.run(RedeliveryErrorHandler.java:623) > [!/:3.7.1] > > at > org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148) > [!/:3.7.1] > > at > org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60) > [!/:3.7.1] > > at > org.apache.camel.processor.Pipeline.process(Pipeline.java:147) [!/:3.7.1] > > at > org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:312) > [!/:3.7.1] > > at > org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:207) > [!/:3.7.1] > > at > org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76) > [!/:3.7.1] > > at java.util.TimerThread.mainLoop(Timer.java:556) [?:?] > > at java.util.TimerThread.run(Timer.java:506) [?:?] > > Caused by: java.lang.NoSuchMethodError: > 'org.apache.sshd.client.session.ClientSession > org.apache.sshd.client.future.ConnectFuture.getSession()' > > at > org.apache.camel.component.ssh.SshHelper.sendExecCommand(SshHelper.java:84) > ~[?:?] > > at > org.apache.camel.component.ssh.SshProducer.process(SshProducer.java:74) > ~[?:?] > > at >
AW: Component "camel-ssh" broken? - NoSuchMethodError: ConnectFuture.getSession()
Hi Claus,
thanks for your reply. I forgot to mention, that I am using Camel in Karaf.
Along with Camel I also upgraded Karaf (4.2.9 => 4.3.0).
When using Camel 2.7.1 with Karaf 4.2.9, the camel-ssh component is working
without problems.
Also, the camel features URLs for installing Camel into Karaf both specify
mvn:org.apache.sshd/sshd-core/2.0.0 as dependency for camel-ssh
https://repo1.maven.org/maven2/org/apache/camel/karaf/apache-camel/3.4.0/apache-camel-3.4.0-features.xml
https://repo1.maven.org/maven2/org/apache/camel/karaf/apache-camel/3.7.1/apache-camel-3.7.1-features.xml
So it seems to be a Karaf-related issue.
Using "feature:install camel-ssh" results in the following in a clean Karaf
4.2.9:
karaf@root()> bundle:list -t 0 | grep -i ssh
35 x Active x 30 x 4.2.9x Apache Karaf :: Shell :: SSH
38 x Active x 30 x 1.7.0x Apache Mina SSHD :: Core
79 x Active x 50 x 3.7.1x camel-ssh
101 x Active x 50 x 2.0.0x Apache Mina SSHD :: Core
And in a clean Karaf 4.3.0 it results in:
karaf@root()> bundle:list -t 0 | grep -i ssh
44 x Active x 30 x 4.3.0 x Apache Karaf :: Shell :: SSH
46 x Active x 30 x 2.5.1 x Apache Mina SSHD :: OSGi
47 x Active x 30 x 2.5.1 x Apache Mina SSHD :: SCP
48 x Active x 30 x 2.5.1 x Apache Mina SSHD :: SFTP
121 x Active x 50 x 3.7.1 x camel-ssh
I'm too tired right now to persue this any further. Maybe I can resolve it
tomorrow.
Regards
--
Jan Schulze
Eberhard Karls Universität Tübingen
Von: Claus Ibsen
Gesendet: Dienstag, 26. Januar 2021 13:26:16
An: users@camel.apache.org
Betreff: Re: Component "camel-ssh" broken? - NoSuchMethodError:
ConnectFuture.getSession()
Hi
Thanks for reporting. Can you create a JIRA ticket.
And would you be able to try test with switching to use mina 2.0 JAR
on the classpath but keep using the 3.7.1 camel version.
On Tue, Jan 26, 2021 at 1:20 PM Schulze, Jan
wrote:
>
> Hi,
>
> "camel-ssh" component is throwing a CamelExecutionException when it is used
> to write a file via SSH.
> It used to be working in Camel 3.4.0, but after upgrading to 3.7.1, I observe
> the following stack trace:
>
> org.apache.camel.CamelExecutionException: Exception occurred during execution
> on the exchange: Exchange[F2205BFA0B31B1C-]
> at
> org.apache.camel.CamelExecutionException.wrapCamelExecutionException(CamelExecutionException.java:45)
> ~[!/:3.7.1]
> at
> org.apache.camel.support.DefaultExchange.setException(DefaultExchange.java:425)
> ~[!/:3.7.1]
> at
> org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:69)
> ~[!/:3.7.1]
> at
> org.apache.camel.processor.SendProcessor.lambda$process$2(SendProcessor.java:188)
> ~[!/:3.7.1]
> at
> org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:317)
> ~[!/:3.7.1]
> at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:187)
> ~[!/:3.7.1]
> at
> org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.doRun(RedeliveryErrorHandler.java:714)
> [!/:3.7.1]
> at
> org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.run(RedeliveryErrorHandler.java:623)
> [!/:3.7.1]
> at
> org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148)
> [!/:3.7.1]
> at
> org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60)
> [!/:3.7.1]
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:147)
> [!/:3.7.1]
> at
> org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:312)
> [!/:3.7.1]
> at
> org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:207)
> [!/:3.7.1]
> at
> org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76)
> [!/:3.7.1]
> at java.util.TimerThread.mainLoop(Timer.java:556) [?:?]
> at java.util.TimerThread.run(Timer.java:506) [?:?]
> Caused by: java.lang.NoSuchMethodError:
> 'org.apache.sshd.client.session.ClientSession
> org.apache.sshd.client.future.ConnectFuture.getSession()'
> at
> org.apache.camel.component.ssh.SshHelper.sendExecCommand(SshHelper.java:84)
> ~[?:?]
> at
> org.apache.camel.component.ssh.SshProducer.process(SshProducer.java:74) ~[?:?]
> at
> org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66)
> ~[!/:3.7.1]
> ... 13 more
>
> The Camel route for writing the file is very simple (and has not changed
> during the Camel upgrade):
>
> .setHeader("currentDate", simple("${date:now:-MM-dd}", String.class))
>
parameter types in RouteTemplateDefinition
Hi all, I’ve a Question About the usage of Route templates (Camel-3.7.0). As far as I see, it is possible to provide every kind of object as a paramter. But within the template, it seems to me, that those Parameters are only processed as Strings. So my question: Is there a way/best practice to provide object parameters, to e.g. a startup order or route configurable error handler could be provided? Maybe I miss something, but I could not find anything about it within the documentation (https://camel.apache.org/manual/latest/route-template.html)? Btw: Using the route templates is a really great feature! Thanks, Joerg
Re: migration to Camel 3.7
Seems like a bug
Here is a full dependencyManagement and dependencies parts of pom.xml made
according to https://camel.apache.org/camel-spring-boot/latest/index.html
org.apache.camel.springboot
camel-spring-boot-bom
${camel-version}
pom
import
org.apache.camel.springboot
camel-spring-boot-starter
org.apache.camel.springboot
camel-activemq-starter
The main method looks like
@SpringBootApplication
@ImportResource("classpath:camel.xml")
public class MyApplication {
public static void main(String[] args) {
SpringApplication.run(MyApplication.class, args);
}
}
Logs, where I have camel got started twice with 2 routes and 0 routes
respectively
2021-01-26 20:23:00.242 INFO 27189 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.7.1 (camel) is
starting
2021-01-26 20:23:00.244 INFO 27189 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : StreamCaching is not in use. If
using streams then it's recommended to enable stream caching. See more details
at http://camel.apache.org/stream-caching.html
2021-01-26 20:23:00.272 INFO 27189 --- [ main]
c.s.b.CamelSpringBootApplicationListener : Starting CamelMainRunController to
ensure the main thread keeps running
2021-01-26 20:23:00.275 INFO 27189 --- [ main]
o.a.c.i.e.InternalRouteStartupManager: Route: my.route started and
consuming from: file:///home/camel/in
2021-01-26 20:23:00.276 INFO 27189 --- [ main]
o.a.c.i.e.InternalRouteStartupManager: Route: another.route started and
consuming from: direct://routing
2021-01-26 20:23:00.279 INFO 27189 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Total 2 routes, of which 2 are
started
2021-01-26 20:23:00.280 INFO 27189 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.7.1 (camel) started
in 38ms
2021-01-26 20:23:00.292 INFO 27189 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.7.1 (camel) is
starting
2021-01-26 20:23:00.293 INFO 27189 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : StreamCaching is not in use. If
using streams then it's recommended to enable stream caching. See more details
at http://camel.apache.org/stream-caching.html
2021-01-26 20:23:00.293 INFO 27189 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Total 0 routes, of which 0 are
started
2021-01-26 20:23:00.293 INFO 27189 --- [ main]
o.a.c.impl.engine.AbstractCamelContext : Apache Camel 3.7.1 (camel) started
in 1ms
2021-01-26 20:23:00.300 INFO 27189 --- [ main] as2.MyApplication
: Started MyApplication in 4.558 seconds (JVM running for
5.694)
The problem disappears if I remove ImportResource annotation, but how one could
set onException or InterceptFrom blocks in this case?
От: Иванов Григорий Олегович
Отправлено: 26 января 2021 г. 17:28:58
Кому: users@camel.apache.org
Тема: Re: migration to Camel 3.7
I figured out it is related to having camelContext described in camel.xml along
with another beans and for example camel-activemq-starter dependency in pom.xml
The one way to eliminate camelContext reloading is to drop camelContext
description at all, xml routes then could be placed in classpath directory to
be load. But I have interceptFrom, onException and others described in my
camelContext bean, where should I move it now?
Or, I noticed using the camel-activemq from org.apache.camel instead of
camel-activemq-starter from org.apache.camel.springboot also helps, would it be
correct?
Please, help
От: Иванов Григорий Олегович
Отправлено: 23 декабря 2020 г. 15:03:34
Кому: users@camel.apache.org
Тема: migration to Camel 3.7
Hi,
I have an issue when migrated from 3.5 to 3.7 Camel
According to logs there are 4 additional empty (0 routes) camel contexts being
started (camel-1 to camel-4)
Please tell if it is a feature and if so what is its purpose? Should I care
about these contexts or somehow treat them?
Camel 3.7.0, Spring Boot 2.4.1
Here is the logs, thanks in advance!
14:46:01.152 [main ] INFO pl.engine.AbstractCamelContext - Total 51
routes, of which 51 are started
14:46:01.152 [main ] INFO pl.engine.AbstractCamelContext - Apache
Camel 3.7.0 (camel) started in 3s362ms
14:46:01.193 [main ] INFO nagement.JmxManagementStrategy - JMX is
enabled
14:46:01.194 [main ] INFO pl.engine.AbstractCamelContext - Apache
Camel 3.7.0 (camel) is starting
14:46:01.195 [main ] WARN JmxManagementLifecycleStrategy - This
CamelContext(camel) will be registered using the name: camel-1 due to clash
with an existing name already registered in MBeanServer.
14:46:01.215 [main ] INFO
Re: Azure Blob ignores LIST_BLOB_OPTIONS
I suggest publishing this work around in camel-examples (
https://github.com/apache/camel-examples/pull/32). If you have any comments
to my MR, you are welcome.
On Tue, 26 Jan 2021 at 19:53, Omar Al-Safi wrote:
> Glad to hear that!
>
> However, if you have an idea how to handle these listBlobs, please feel
> free to send a PR with your proposed fix.
>
> Regards,
> Omar
>
> On Tue, Jan 26, 2021 at 2:14 PM Mark Andreev
> wrote:
>
> > I found a solution for this issue. When I use Azure Api directly and
> > operate with Page all works. It is useful when your azure blob storage
> > contains a lot of blobs (in my case ~250k).
> >
> > from("direct:list")
> > .routeId("listBlobs")
> > .process(exchange -> {
> > BlobServiceClient blobServiceClient = new
> > BlobServiceClientBuilder()
> > .endpoint(String
> > .format("https://%s.blob.core.windows.net;,
> > AzureBlobConfigHelper.ACCOUNT))
> > .credential(new
> > StorageSharedKeyCredential(AzureBlobConfigHelper.ACCOUNT,
> > AzureBlobConfigHelper.ACCESS_KEY))
> > .buildClient();
> > BlobContainerClient containerClient =
> >
> >
> >
> blobServiceClient.getBlobContainerClient(AzureBlobConfigHelper.BLOB_CONTAINER_NAME);
> > ListBlobsOptions blobsOptions = new
> > ListBlobsOptions().setMaxResultsPerPage(5);
> > Duration duration = Duration.of(10, ChronoUnit.SECONDS);
> >
> > Iterator
> > iterator = containerClient.listBlobs(blobsOptions,
> > duration).iterator();
> > exchange.getIn().setBody(iterator);
> > })
> > .loopDoWhile(exchange -> {
> > Iterator iterator =
> > (Iterator)
> > exchange.getIn().getBody(Iterator.class);
> > return iterator.hasNext();
> > })
> > .process(exchange -> {
> > Iterator iterator =
> > (Iterator)
> > exchange.getIn().getBody(Iterator.class);
> > exchange.getIn().setBody(iterator.next());
> > })
> > .process(exchange -> {
> > BlobItem item = exchange.getIn().getBody(BlobItem.class);
> > log.info(item);
> > })
> > .end();
> >
> > On Tue, 26 Jan 2021 at 13:57, Omar Al-Safi wrote:
> >
> > > Hi,
> > >
> > > Actually you are doing nothing wrong here. Apparently when you set
> > > maxResultsPerPage, in Azure client, that means it will return pageable
> > > list, now in every pagelist it will return the max blobs that you set
> > > earlier in `maxResultsPerPage` which is IMO pretty weird from Azure's
> > side.
> > > Now in the Camel component side, we just collect all that list of blobs
> > in
> > > the pages into one list and return that in the body and hence the
> reason
> > > why you see no effects. I hope Azure will improve the API in the future
> > to
> > > allow alternative options as max results regardless of the pageable
> list.
> > >
> > > Regards,
> > > Omar
> > >
> > > On Mon, Jan 25, 2021 at 8:04 PM Mark Andreev
> > > wrote:
> > >
> > > > Hi, I use Spring 2.4.2 with camel-azure-storage-blob 3.7.0.
> > > >
> > > > When I set BlobConstants.LIST_BLOB_OPTIONS as "new
> > > > ListBlobsOptions().setMaxResultsPerPage(5)" driver returns all
> entities
> > > > from blob.
> > > >
> > > > from("direct:list")
> > > > .routeId("listBlobs")
> > > > .setHeader(BlobConstants.LIST_BLOB_OPTIONS)
> > > >
> > > > .constant(new ListBlobsOptions().setMaxResultsPerPage(5))
> > > > .to(
> > > > String.format(
> > > > "
> > > >
> > > >
> > >
> >
> azure-storage-blob://%s/%s?accessKey=RAW(%s)=listBlobs=true
> > > > ",
> > > > ACCOUNT,
> > > > BLOB_CONTAINER_NAME,
> > > > ACCESS_KEY
> > > > )
> > > > )
> > > > .process(exchange -> {
> > > > int itemsCount = exchange.getIn().getBody(List.class).size();
> > > > log.info(String.format("Items count = '%d'", itemsCount));
> > > > });
> > > >
> > > > How should I use this API to fetch only 5 items?
> > > >
> > > > Full example:
> > > >
> > > >
> > >
> >
> https://github.com/mrk-andreev/example-camel-azure-blob-pagination-bug/blob/master/src/main/java/name/mrkandreev/camel/MyApplication.java
> > > > --
> > > > Best regards,
> > > > Mark Andreev
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Mark Andreev
> >
>
--
Best regards,
Mark Andreev
Re: Azure Blob ignores LIST_BLOB_OPTIONS
Glad to hear that!
However, if you have an idea how to handle these listBlobs, please feel
free to send a PR with your proposed fix.
Regards,
Omar
On Tue, Jan 26, 2021 at 2:14 PM Mark Andreev wrote:
> I found a solution for this issue. When I use Azure Api directly and
> operate with Page all works. It is useful when your azure blob storage
> contains a lot of blobs (in my case ~250k).
>
> from("direct:list")
> .routeId("listBlobs")
> .process(exchange -> {
> BlobServiceClient blobServiceClient = new
> BlobServiceClientBuilder()
> .endpoint(String
> .format("https://%s.blob.core.windows.net;,
> AzureBlobConfigHelper.ACCOUNT))
> .credential(new
> StorageSharedKeyCredential(AzureBlobConfigHelper.ACCOUNT,
> AzureBlobConfigHelper.ACCESS_KEY))
> .buildClient();
> BlobContainerClient containerClient =
>
>
> blobServiceClient.getBlobContainerClient(AzureBlobConfigHelper.BLOB_CONTAINER_NAME);
> ListBlobsOptions blobsOptions = new
> ListBlobsOptions().setMaxResultsPerPage(5);
> Duration duration = Duration.of(10, ChronoUnit.SECONDS);
>
> Iterator
> iterator = containerClient.listBlobs(blobsOptions,
> duration).iterator();
> exchange.getIn().setBody(iterator);
> })
> .loopDoWhile(exchange -> {
> Iterator iterator =
> (Iterator)
> exchange.getIn().getBody(Iterator.class);
> return iterator.hasNext();
> })
> .process(exchange -> {
> Iterator iterator =
> (Iterator)
> exchange.getIn().getBody(Iterator.class);
> exchange.getIn().setBody(iterator.next());
> })
> .process(exchange -> {
> BlobItem item = exchange.getIn().getBody(BlobItem.class);
> log.info(item);
> })
> .end();
>
> On Tue, 26 Jan 2021 at 13:57, Omar Al-Safi wrote:
>
> > Hi,
> >
> > Actually you are doing nothing wrong here. Apparently when you set
> > maxResultsPerPage, in Azure client, that means it will return pageable
> > list, now in every pagelist it will return the max blobs that you set
> > earlier in `maxResultsPerPage` which is IMO pretty weird from Azure's
> side.
> > Now in the Camel component side, we just collect all that list of blobs
> in
> > the pages into one list and return that in the body and hence the reason
> > why you see no effects. I hope Azure will improve the API in the future
> to
> > allow alternative options as max results regardless of the pageable list.
> >
> > Regards,
> > Omar
> >
> > On Mon, Jan 25, 2021 at 8:04 PM Mark Andreev
> > wrote:
> >
> > > Hi, I use Spring 2.4.2 with camel-azure-storage-blob 3.7.0.
> > >
> > > When I set BlobConstants.LIST_BLOB_OPTIONS as "new
> > > ListBlobsOptions().setMaxResultsPerPage(5)" driver returns all entities
> > > from blob.
> > >
> > > from("direct:list")
> > > .routeId("listBlobs")
> > > .setHeader(BlobConstants.LIST_BLOB_OPTIONS)
> > >
> > > .constant(new ListBlobsOptions().setMaxResultsPerPage(5))
> > > .to(
> > > String.format(
> > > "
> > >
> > >
> >
> azure-storage-blob://%s/%s?accessKey=RAW(%s)=listBlobs=true
> > > ",
> > > ACCOUNT,
> > > BLOB_CONTAINER_NAME,
> > > ACCESS_KEY
> > > )
> > > )
> > > .process(exchange -> {
> > > int itemsCount = exchange.getIn().getBody(List.class).size();
> > > log.info(String.format("Items count = '%d'", itemsCount));
> > > });
> > >
> > > How should I use this API to fetch only 5 items?
> > >
> > > Full example:
> > >
> > >
> >
> https://github.com/mrk-andreev/example-camel-azure-blob-pagination-bug/blob/master/src/main/java/name/mrkandreev/camel/MyApplication.java
> > > --
> > > Best regards,
> > > Mark Andreev
> > >
> >
>
>
> --
> Best regards,
> Mark Andreev
>
Re: migration to Camel 3.7
I figured out it is related to having camelContext described in camel.xml along with another beans and for example camel-activemq-starter dependency in pom.xml The one way to eliminate camelContext reloading is to drop camelContext description at all, xml routes then could be placed in classpath directory to be load. But I have interceptFrom, onException and others described in my camelContext bean, where should I move it now? Or, I noticed using the camel-activemq from org.apache.camel instead of camel-activemq-starter from org.apache.camel.springboot also helps, would it be correct? Please, help От: Иванов Григорий Олегович Отправлено: 23 декабря 2020 г. 15:03:34 Кому: users@camel.apache.org Тема: migration to Camel 3.7 Hi, I have an issue when migrated from 3.5 to 3.7 Camel According to logs there are 4 additional empty (0 routes) camel contexts being started (camel-1 to camel-4) Please tell if it is a feature and if so what is its purpose? Should I care about these contexts or somehow treat them? Camel 3.7.0, Spring Boot 2.4.1 Here is the logs, thanks in advance! 14:46:01.152 [main ] INFO pl.engine.AbstractCamelContext - Total 51 routes, of which 51 are started 14:46:01.152 [main ] INFO pl.engine.AbstractCamelContext - Apache Camel 3.7.0 (camel) started in 3s362ms 14:46:01.193 [main ] INFO nagement.JmxManagementStrategy - JMX is enabled 14:46:01.194 [main ] INFO pl.engine.AbstractCamelContext - Apache Camel 3.7.0 (camel) is starting 14:46:01.195 [main ] WARN JmxManagementLifecycleStrategy - This CamelContext(camel) will be registered using the name: camel-1 due to clash with an existing name already registered in MBeanServer. 14:46:01.215 [main ] INFO pl.engine.AbstractCamelContext - StreamCaching is not in use. If using streams then it's recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html 14:46:01.221 [main ] INFO pl.engine.AbstractCamelContext - Total 0 routes, of which 0 are started 14:46:01.223 [main ] INFO pl.engine.AbstractCamelContext - Apache Camel 3.7.0 (camel) started in 29ms 14:46:01.236 [main ] INFO nagement.JmxManagementStrategy - JMX is enabled 14:46:01.236 [main ] INFO pl.engine.AbstractCamelContext - Apache Camel 3.7.0 (camel) is starting 14:46:01.236 [main ] WARN JmxManagementLifecycleStrategy - This CamelContext(camel) will be registered using the name: camel-2 due to clash with an existing name already registered in MBeanServer. 14:46:01.262 [main ] INFO pl.engine.AbstractCamelContext - StreamCaching is not in use. If using streams then it's recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html 14:46:01.280 [main ] INFO pl.engine.AbstractCamelContext - Total 0 routes, of which 0 are started 14:46:01.283 [main ] INFO pl.engine.AbstractCamelContext - Apache Camel 3.7.0 (camel) started in 47ms 14:46:01.294 [main ] INFO nagement.JmxManagementStrategy - JMX is enabled 14:46:01.295 [main ] INFO pl.engine.AbstractCamelContext - Apache Camel 3.7.0 (camel) is starting 14:46:01.298 [main ] WARN JmxManagementLifecycleStrategy - This CamelContext(camel) will be registered using the name: camel-3 due to clash with an existing name already registered in MBeanServer. 14:46:01.347 [main ] INFO pl.engine.AbstractCamelContext - StreamCaching is not in use. If using streams then it's recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html 14:46:01.356 [main ] INFO pl.engine.AbstractCamelContext - Total 0 routes, of which 0 are started 14:46:01.356 [main ] INFO pl.engine.AbstractCamelContext - Apache Camel 3.7.0 (camel) started in 61ms 14:46:01.396 [main ] INFO nagement.JmxManagementStrategy - JMX is enabled 14:46:01.398 [main ] INFO pl.engine.AbstractCamelContext - Apache Camel 3.7.0 (camel) is starting 14:46:01.399 [main ] WARN JmxManagementLifecycleStrategy - This CamelContext(camel) will be registered using the name: camel-4 due to clash with an existing name already registered in MBeanServer. 14:46:01.451 [main ] INFO pl.engine.AbstractCamelContext - StreamCaching is not in use. If using streams then it's recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html 14:46:01.463 [main ] INFO pl.engine.AbstractCamelContext - Total 0 routes, of which 0 are started 14:46:01.463 [main ] INFO pl.engine.AbstractCamelContext - Apache Camel 3.7.0 (camel) started in 64ms 14:46:01.474 [main ] INFO phere.sample.CamelApplication - Started CamelApplication in 13.587 seconds (JVM running for 16.829)
Re: Azure Blob ignores LIST_BLOB_OPTIONS
I found a solution for this issue. When I use Azure Api directly and
operate with Page all works. It is useful when your azure blob storage
contains a lot of blobs (in my case ~250k).
from("direct:list")
.routeId("listBlobs")
.process(exchange -> {
BlobServiceClient blobServiceClient = new
BlobServiceClientBuilder()
.endpoint(String
.format("https://%s.blob.core.windows.net;,
AzureBlobConfigHelper.ACCOUNT))
.credential(new
StorageSharedKeyCredential(AzureBlobConfigHelper.ACCOUNT,
AzureBlobConfigHelper.ACCESS_KEY))
.buildClient();
BlobContainerClient containerClient =
blobServiceClient.getBlobContainerClient(AzureBlobConfigHelper.BLOB_CONTAINER_NAME);
ListBlobsOptions blobsOptions = new
ListBlobsOptions().setMaxResultsPerPage(5);
Duration duration = Duration.of(10, ChronoUnit.SECONDS);
Iterator
iterator = containerClient.listBlobs(blobsOptions,
duration).iterator();
exchange.getIn().setBody(iterator);
})
.loopDoWhile(exchange -> {
Iterator iterator =
(Iterator) exchange.getIn().getBody(Iterator.class);
return iterator.hasNext();
})
.process(exchange -> {
Iterator iterator =
(Iterator) exchange.getIn().getBody(Iterator.class);
exchange.getIn().setBody(iterator.next());
})
.process(exchange -> {
BlobItem item = exchange.getIn().getBody(BlobItem.class);
log.info(item);
})
.end();
On Tue, 26 Jan 2021 at 13:57, Omar Al-Safi wrote:
> Hi,
>
> Actually you are doing nothing wrong here. Apparently when you set
> maxResultsPerPage, in Azure client, that means it will return pageable
> list, now in every pagelist it will return the max blobs that you set
> earlier in `maxResultsPerPage` which is IMO pretty weird from Azure's side.
> Now in the Camel component side, we just collect all that list of blobs in
> the pages into one list and return that in the body and hence the reason
> why you see no effects. I hope Azure will improve the API in the future to
> allow alternative options as max results regardless of the pageable list.
>
> Regards,
> Omar
>
> On Mon, Jan 25, 2021 at 8:04 PM Mark Andreev
> wrote:
>
> > Hi, I use Spring 2.4.2 with camel-azure-storage-blob 3.7.0.
> >
> > When I set BlobConstants.LIST_BLOB_OPTIONS as "new
> > ListBlobsOptions().setMaxResultsPerPage(5)" driver returns all entities
> > from blob.
> >
> > from("direct:list")
> > .routeId("listBlobs")
> > .setHeader(BlobConstants.LIST_BLOB_OPTIONS)
> >
> > .constant(new ListBlobsOptions().setMaxResultsPerPage(5))
> > .to(
> > String.format(
> > "
> >
> >
> azure-storage-blob://%s/%s?accessKey=RAW(%s)=listBlobs=true
> > ",
> > ACCOUNT,
> > BLOB_CONTAINER_NAME,
> > ACCESS_KEY
> > )
> > )
> > .process(exchange -> {
> > int itemsCount = exchange.getIn().getBody(List.class).size();
> > log.info(String.format("Items count = '%d'", itemsCount));
> > });
> >
> > How should I use this API to fetch only 5 items?
> >
> > Full example:
> >
> >
> https://github.com/mrk-andreev/example-camel-azure-blob-pagination-bug/blob/master/src/main/java/name/mrkandreev/camel/MyApplication.java
> > --
> > Best regards,
> > Mark Andreev
> >
>
--
Best regards,
Mark Andreev
Re: Component "camel-ssh" broken? - NoSuchMethodError: ConnectFuture.getSession()
Hi
Thanks for reporting. Can you create a JIRA ticket.
And would you be able to try test with switching to use mina 2.0 JAR
on the classpath but keep using the 3.7.1 camel version.
On Tue, Jan 26, 2021 at 1:20 PM Schulze, Jan
wrote:
>
> Hi,
>
> "camel-ssh" component is throwing a CamelExecutionException when it is used
> to write a file via SSH.
> It used to be working in Camel 3.4.0, but after upgrading to 3.7.1, I observe
> the following stack trace:
>
> org.apache.camel.CamelExecutionException: Exception occurred during execution
> on the exchange: Exchange[F2205BFA0B31B1C-]
> at
> org.apache.camel.CamelExecutionException.wrapCamelExecutionException(CamelExecutionException.java:45)
> ~[!/:3.7.1]
> at
> org.apache.camel.support.DefaultExchange.setException(DefaultExchange.java:425)
> ~[!/:3.7.1]
> at
> org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:69)
> ~[!/:3.7.1]
> at
> org.apache.camel.processor.SendProcessor.lambda$process$2(SendProcessor.java:188)
> ~[!/:3.7.1]
> at
> org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:317)
> ~[!/:3.7.1]
> at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:187)
> ~[!/:3.7.1]
> at
> org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.doRun(RedeliveryErrorHandler.java:714)
> [!/:3.7.1]
> at
> org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.run(RedeliveryErrorHandler.java:623)
> [!/:3.7.1]
> at
> org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148)
> [!/:3.7.1]
> at
> org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60)
> [!/:3.7.1]
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:147)
> [!/:3.7.1]
> at
> org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:312)
> [!/:3.7.1]
> at
> org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:207)
> [!/:3.7.1]
> at
> org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76)
> [!/:3.7.1]
> at java.util.TimerThread.mainLoop(Timer.java:556) [?:?]
> at java.util.TimerThread.run(Timer.java:506) [?:?]
> Caused by: java.lang.NoSuchMethodError:
> 'org.apache.sshd.client.session.ClientSession
> org.apache.sshd.client.future.ConnectFuture.getSession()'
> at
> org.apache.camel.component.ssh.SshHelper.sendExecCommand(SshHelper.java:84)
> ~[?:?]
> at
> org.apache.camel.component.ssh.SshProducer.process(SshProducer.java:74) ~[?:?]
> at
> org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66)
> ~[!/:3.7.1]
> ... 13 more
>
> The Camel route for writing the file is very simple (and has not changed
> during the Camel upgrade):
>
> .setHeader("currentDate", simple("${date:now:-MM-dd}", String.class))
> .setHeader("filename", simple("/tmp/test.text", String.class))
> .setBody(simple("echo ${header.currentDate} > ${header.filename};"))
> .to("ssh://username:password@hostname")
>
> It seems, that some changes to Mina SSHD in
> "org.apache.sshd.client.future.ConnectFuture.java" related to the following
> JIRA are responsible:
> https://issues.apache.org/jira/browse/SSHD-842
>
> Camel 3.4.0 (working) uses Mina 2.0.0, whereas Camel 3.7.1 uses Mina 2.5.1.
> The changes from the mentioned JIRA are in effect since Mina 2.1.0.
>
> Can anyone confirm?
>
>
> Regards
> --
> Jan Schulze
> Eberhard Karls Universität Tübingen
--
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Component "camel-ssh" broken? - NoSuchMethodError: ConnectFuture.getSession()
Hi,
"camel-ssh" component is throwing a CamelExecutionException when it is used to
write a file via SSH.
It used to be working in Camel 3.4.0, but after upgrading to 3.7.1, I observe
the following stack trace:
org.apache.camel.CamelExecutionException: Exception occurred during execution
on the exchange: Exchange[F2205BFA0B31B1C-]
at
org.apache.camel.CamelExecutionException.wrapCamelExecutionException(CamelExecutionException.java:45)
~[!/:3.7.1]
at
org.apache.camel.support.DefaultExchange.setException(DefaultExchange.java:425)
~[!/:3.7.1]
at
org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:69)
~[!/:3.7.1]
at
org.apache.camel.processor.SendProcessor.lambda$process$2(SendProcessor.java:188)
~[!/:3.7.1]
at
org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:317)
~[!/:3.7.1]
at
org.apache.camel.processor.SendProcessor.process(SendProcessor.java:187)
~[!/:3.7.1]
at
org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.doRun(RedeliveryErrorHandler.java:714)
[!/:3.7.1]
at
org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.run(RedeliveryErrorHandler.java:623)
[!/:3.7.1]
at
org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148)
[!/:3.7.1]
at
org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60)
[!/:3.7.1]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:147)
[!/:3.7.1]
at
org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:312)
[!/:3.7.1]
at
org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:207)
[!/:3.7.1]
at
org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76)
[!/:3.7.1]
at java.util.TimerThread.mainLoop(Timer.java:556) [?:?]
at java.util.TimerThread.run(Timer.java:506) [?:?]
Caused by: java.lang.NoSuchMethodError:
'org.apache.sshd.client.session.ClientSession
org.apache.sshd.client.future.ConnectFuture.getSession()'
at
org.apache.camel.component.ssh.SshHelper.sendExecCommand(SshHelper.java:84)
~[?:?]
at
org.apache.camel.component.ssh.SshProducer.process(SshProducer.java:74) ~[?:?]
at
org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66)
~[!/:3.7.1]
... 13 more
The Camel route for writing the file is very simple (and has not changed during
the Camel upgrade):
.setHeader("currentDate", simple("${date:now:-MM-dd}", String.class))
.setHeader("filename", simple("/tmp/test.text", String.class))
.setBody(simple("echo ${header.currentDate} > ${header.filename};"))
.to("ssh://username:password@hostname")
It seems, that some changes to Mina SSHD in
"org.apache.sshd.client.future.ConnectFuture.java" related to the following
JIRA are responsible:
https://issues.apache.org/jira/browse/SSHD-842
Camel 3.4.0 (working) uses Mina 2.0.0, whereas Camel 3.7.1 uses Mina 2.5.1.
The changes from the mentioned JIRA are in effect since Mina 2.1.0.
Can anyone confirm?
Regards
--
Jan Schulze
Eberhard Karls Universität Tübingen
Re: Azure Blob ignores LIST_BLOB_OPTIONS
Hi,
Actually you are doing nothing wrong here. Apparently when you set
maxResultsPerPage, in Azure client, that means it will return pageable
list, now in every pagelist it will return the max blobs that you set
earlier in `maxResultsPerPage` which is IMO pretty weird from Azure's side.
Now in the Camel component side, we just collect all that list of blobs in
the pages into one list and return that in the body and hence the reason
why you see no effects. I hope Azure will improve the API in the future to
allow alternative options as max results regardless of the pageable list.
Regards,
Omar
On Mon, Jan 25, 2021 at 8:04 PM Mark Andreev wrote:
> Hi, I use Spring 2.4.2 with camel-azure-storage-blob 3.7.0.
>
> When I set BlobConstants.LIST_BLOB_OPTIONS as "new
> ListBlobsOptions().setMaxResultsPerPage(5)" driver returns all entities
> from blob.
>
> from("direct:list")
> .routeId("listBlobs")
> .setHeader(BlobConstants.LIST_BLOB_OPTIONS)
>
> .constant(new ListBlobsOptions().setMaxResultsPerPage(5))
> .to(
> String.format(
> "
>
> azure-storage-blob://%s/%s?accessKey=RAW(%s)=listBlobs=true
> ",
> ACCOUNT,
> BLOB_CONTAINER_NAME,
> ACCESS_KEY
> )
> )
> .process(exchange -> {
> int itemsCount = exchange.getIn().getBody(List.class).size();
> log.info(String.format("Items count = '%d'", itemsCount));
> });
>
> How should I use this API to fetch only 5 items?
>
> Full example:
>
> https://github.com/mrk-andreev/example-camel-azure-blob-pagination-bug/blob/master/src/main/java/name/mrkandreev/camel/MyApplication.java
> --
> Best regards,
> Mark Andreev
>
