I am using Camel 2.15.2.
I am trying to download files from a S3 bucket at regular interval using the
following aw-s3 route. String bucketName = "camel-lombok"; final String prefix
= "out";
// Use RAW url encode the secretKey. String awsURLTemplate =
"aws-s3://%s?accessKey=%s&secretKey=RAW(%s)®ion=eu-west-1&prefix=%s&deleteAfterRead=false&delay=5000&maxMessagesPerPoll=5";
final String awsURL = String.format(awsURLTemplate, bucketName, accessKey,
secretKey, prefix);
DefaultCamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() { public void configure() { // Only
download {bucket}/out/**.txt}.
from(awsURL).filter().simple("${header.camelawss3key} regex '.*\\.txt'")
.setHeader(Exchange.FILE_NAME,
simple("${header.camelawss3key}")).to("file:/home/matmsh/tmp/camel/in");
} });
context.start();
Thread.sleep(600000);
But I get following timeout exception after about 1 minute.
216 [main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.15.2
(CamelContext: camel-1) is starting217 [main] INFO
org.apache.camel.management.ManagedManagementStrategy - JMX is enabled474
[main] INFO org.apache.camel.impl.converter.DefaultTypeConverter - Loaded 233
type converters1703 [main] INFO org.apache.camel.impl.DefaultCamelContext -
AllowUseOriginalMessage is enabled. If access to the original message is not
needed, then its recommended to turn this option off as it may improve
performance.1703 [main] INFO org.apache.camel.impl.DefaultCamelContext -
StreamCaching is not in use. If using streams then its recommended to enable
stream caching. See more details at
http://camel.apache.org/stream-caching.html1763 [main] INFO
org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming
from:
Endpoint[aws-s3://camel-lombok?accessKey=XXXX&delay=5000&deleteAfterRead=false&maxMessagesPerPoll=5&prefix=out®ion=eu-west-1&secretKey=xxxxxx]1763
[main] INFO org.apache.camel.impl.DefaultCamelContext - Total 1 routes, of
which 1 is started.1764 [main] INFO org.apache.camel.impl.DefaultCamelContext -
Apache Camel 2.15.2 (CamelContext: camel-1) started in 1.549 seconds[ead #0 -
aws-s3://camel-lombok] AmazonHttpClient INFO Unable to execute HTTP request:
Timeout waiting for connection from
poolorg.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for
connection from pool at
org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:226)[httpclient-4.3.4.jar:4.3.4]
at
org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:195)[httpclient-4.3.4.jar:4.3.4]
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_71]
at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_71] at
com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)[aws-java-sdk-1.8.9.1.jar:]
at com.amazonaws.http.conn.$Proxy17.getConnection(Unknown Source)[:] at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:423)[httpclient-4.3.4.jar:4.3.4]
at
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)[httpclient-4.3.4.jar:4.3.4]
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)[httpclient-4.3.4.jar:4.3.4]
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)[httpclient-4.3.4.jar:4.3.4]
at
com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:464)[aws-java-sdk-1.8.9.1.jar:]
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:273)[aws-java-sdk-1.8.9.1.jar:]
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3660)[aws-java-sdk-1.8.9.1.jar:]
at
com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1133)[aws-java-sdk-1.8.9.1.jar:]
at
com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1005)[aws-java-sdk-1.8.9.1.jar:]
at
org.apache.camel.component.aws.s3.S3Consumer.createExchanges(S3Consumer.java:114)[camel-aws-2.15.2.jar:2.15.2]
at
org.apache.camel.component.aws.s3.S3Consumer.poll(S3Consumer.java:95)[camel-aws-2.15.2.jar:2.15.2]
at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)[camel-core-2.15.2.jar:2.15.2]
at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)[camel-core-2.15.2.jar:2.15.2]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_71]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)[:1.7.0_71]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)[:1.7.0_71]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)[:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_71]
at java.lang.Thread.run(Thread.java:745)[:1.7.0_71]
Thank advance for any assistance !Shing