Re: camel-spring-boot-starter can´t starts CamelContext on lastest SpringBoot versions

[Zoran Regvart]

Hi Garry,
the issue you're describing seems to correlate with a change in the
way we approach startup/shutdown of Camel context in Spring (Boot)
applications. For context you can read through CAMEL-11261[1] and
follow links from there.

It seems that you might be performing configuration in an uncommon
way. If you can minimize your example to just show the issue you're
experiencing and share the project code, we can then discuss if this
is something that needs to be addressed in Camel.

zoran

[1] https://issues.apache.org/jira/browse/CAMEL-11261

On Tue, Apr 3, 2018 at 7:13 PM, Garry Dias  wrote:
> Hi guys
>
> Just to warn you,
>
> Problem: CamelContext not started and then RouteBuilder implementations not
> performed
>
> Facts: camel-spring-boot-starter:2.19.5 is the last Camel version I could
> mix with spring-boot-starter-parent 1.5.10.RELEASE and see my CamelContext
> starting. It doesn´t works on 2.20.x and 2.21.x.
>
> My application startup log in debug mode is attached.



-- 
Zoran Regvart

RE: Camel FTP Producer catches errors silently

[Florian Posch]

I believe we are bit smarter now. We are consuming from JMS with 
concurrentConsumers and so it is possible the the same FTP endpoint is served 
twice.

With more verbose logging, it seems as if the connection is somehow shared and 
the GOODBYE initiated by the operation finishing first kills the second 
connection as well.

Is this a race condition or do we need to make sure that concurrency isn't 
possible at this level?

-Ursprüngliche Nachricht-
Von: Florian Posch [mailto:florian.po...@performgroup.com]
Gesendet: Mittwoch, 4. April 2018 16:36
An: users@camel.apache.org
Betreff: Camel FTP Producer catches errors silently

Hi.



We are using Camel 2.20.2 to transfer files to remote servers.



Our route is built like that - sorry that I can't provide a fully featured 
application and all beans.



   from("direct:chunk.process")

   
.routeId(DEFAULT_ROUTE_ID_STRING)

   
.process("feedNotificationProcessor")

 
.onException(Exception.class).handled(false)

   .end()


.recipientList(exchangeProperty(RECIPIENTLIST_HEADER).tokenize(RECIPIENTLIST_SEPARATOR))

   
.parallelProcessing().timeout(4)

   
.executorService(customThreadPoolExecutor)


.streaming().aggregationStrategyRef("recipientAggregationStrategy")

   
.to("direct:chunk.evaluate");



We are dispatching to (multiple) dynamic FTP producers with routes like that:



ftp://myHost/?fileName=myfile.xmlpassword=secretbinary=falsepassiveMode=falseusername=userdisconnect=truefastExistsCheck=trueftpClient.dataTimeout=3ftpClient.defaultTimeout=3ftpClient.connectTimeout=3soTimeout=3stepwise=false



When something fails, we would expect that we recognize that in the 'aggregate' 
callback of our aggregation strategy call with 'CamelFailureEndpoint' and 
'CamelExceptionCaught' properties set, so that we can retry or report failure 
to our system.



However, in certain cases it appears that we only get a WARN logging but no 
error set in the Exchange



   [2018-04-03 14:38:21,426][[WARN 
][org.apache.camel.component.file.remote.RemoteFileProducer][[]][2188110951-169-19445712825]
 Exception occurred during disconnecting from: ftp://myhost/? 
binary=false=true=true=3=3=3=30720=307
 
20=false=false=xx=3=false=user
 File operation failed:  Connection closed without indication.. Code: 221



It looks as if this just happens on cleanup of connections after upload but our 
recipient confirmed that the corresponding FTP session dropped just after 
creating the data connection



2018-04-03 14:38:21,372 [99773] : decoded 'myfile.xml' into 
'myfile.xml'

2018-04-03 14:38:21,372 [99773] : dispatching PRE_CMD command 'STOR 
myfile.xml' to mod_exec.c

2018-04-03 14:38:21,372 [99773] : dispatching PRE_CMD command 'STOR 
myfile.xml' to mod_rewrite.c

2018-04-03 14:38:21,372 [99773] : dispatching PRE_CMD command 'STOR 
myfile.xml' to mod_tls.c

2018-04-03 14:38:21,372 [99773] : dispatching PRE_CMD command 'STOR 
myfile.xml' to mod_core.c

2018-04-03 14:38:21,372 [99773] : dispatching PRE_CMD command 'STOR 
myfile.xml' to mod_core.c

2018-04-03 14:38:21,372 [99773] : dispatching PRE_CMD command 'STOR 
myfile.xml' to mod_ratio.c

2018-04-03 14:38:21,372 [99773] : dispatching PRE_CMD command 'STOR 
myfile.xml' to mod_quotatab.c

2018-04-03 14:38:21,372 [99773] : dispatching PRE_CMD command 'STOR 
myfile.xml' to mod_xfer.c

2018-04-03 14:38:21,372 [99773] : decoded 'myfile.xml' into 
'myfile.xml'

2018-04-03 14:38:21,374 [99773] : dispatching CMD command 'STOR 
myfile.xml' to mod_xfer.c

2018-04-03 14:38:21,397 [99773] : 150 Opening ASCII mode data 
connection for myfile.xml

2018-04-03 14:38:21,430 [99773] : decoded '/myfile.xml' into 
'/myfile.xml'

2018-04-03 14:38:21,430 [99773] : dispatching POST_CMD_ERR command 
'QUIT /myfile.xml' to mod_exec.c

2018-04-03 14:38:21,430 [99773] : dispatching LOG_CMD_ERR command 
'QUIT /myfile.xml' to mod_log.c

2018-04-03 14:38:21,430 [99773] : dispatching LOG_CMD_ERR command 
'QUIT /myfile.xml' to mod_core.c



Is this intended behavior of Camel? it is quite inconvenient to miss out failed 
uploads due to that (not being aware of any errors except for logging)



Kind regards,

Florian

CONFIDENTIALITY: This email and any files transmitted with it are confidential, 
may be legally privileged and are intended solely for the use of the individual 
or entity to whom they are addressed. If this has come to you in error, you 
must not copy, distribute, disclose or use any of the information it contains. 
Please 

Concurrent FTP endpoints close each others session

[Florian Posch]

Hi,

We have a Camel application which is running multiple FTP consumers to the same 
server and it seems as if one transaction failing affects others, like Client A 
sending GOODBYE kills Client B still running STOR (please see the attached log).

[2018-04-05 
09:53:28,718][2188512939-11851-19450835297][TRACE][org.apache.camel.component.file.remote.FtpOperations][[]][2188512939-11851-19450835297]
 doStoreFile(file1.xml)
[2018-04-05 
09:53:28,721][2188512939-11851-19450835297][DEBUG][org.apache.camel.component.file.remote.FtpOperations][[]][2188512939-11851-19450835297]
 About to store file: file1.xml using stream: 
java.io.ByteArrayInputStream@10adca16
[2018-04-05 
09:53:28,721][2188512939-11851-19450835297][TRACE][org.apache.camel.component.file.remote.FtpOperations][[]][2188512939-11851-19450835297]
 Client storeFile: file1.xml
[2018-04-05 
09:53:28,786][2188512931-11851-19450835141][TRACE][org.apache.camel.component.file.remote.FtpOperations][[]][2188512931-11851-19450835141]
 Client disconnect
[2018-04-05 
09:53:28,786][2188512931-11851-19450835141][DEBUG][org.apache.camel.component.file.remote.RemoteFileProducer][[]][2188512931-11851-19450835141]
 Disconnecting from: 
ftp://secrethost:21/?binary=false=true=true=Override=3=3=3=30720=30720=false=true=xx=1000=3=false=user
[2018-04-05 
09:53:28,786][2188512931-11851-19450835141][TRACE][org.apache.camel.component.file.remote.FtpOperations][[]][2188512931-11851-19450835141]
 Client logout
[2018-04-05 
09:53:29,135][2188512931-11851-19450835141][TRACE][org.apache.camel.component.file.remote.FtpOperations][[]][2188512931-11851-19450835141]
 Client disconnect
[2018-04-05 
09:53:29,135][2188512939-11851-19450835297][DEBUG][org.apache.camel.component.file.remote.RemoteFileProducer][[]][2188512939-11851-19450835297]
 Exception occurred during stopping: File operation failed: null Socket closed. 
Code: 221

Doing the same thing with Http4 works because it does connection pooling.

Kind regards,
Florian
CONFIDENTIALITY: This email and any files transmitted with it are confidential, 
may be legally privileged and are intended solely for the use of the individual 
or entity to whom they are addressed. If this has come to you in error, you 
must not copy, distribute, disclose or use any of the information it contains. 
Please notify the sender immediately and delete them from your system. PRIVACY: 
Perform Media Services Ltd (a member of the Perform Group, with Companies House 
registration number 3426471) may monitor email traffic data and also the 
content of email for the purposes of security and staff training. For 
information on how the Perform Group processes your information, please refer 
to http://www.performgroup.com/privacy-notice/. SECURITY: Please be aware that 
emails are not secure and may contain viruses. AUTHORITY: Any views or opinions 
expressed in this email are solely those of the sender and do not necessarily 
represent those of the Perform Group. COPYRIGHT: Copyright of this email and 
any attachments belongs to Perform Group Limited, Companies House registration 
number 6324278. Further information about Perform Group is available at 
www.performgroup.com

Re: camel-spring-boot-starter can´t starts CamelContext on lastest SpringBoot versions

[Claus Ibsen]

Hi

We cannot see your attached debug log file. However I encourage you to
take a look at the many camel spring boot examples that are shipped
and see how they work, and they are good/best practice.

https://github.com/apache/camel/tree/master/examples#examples

If its still a problem you may need to maybe post your sample
application somewhere, or tell more about how you use Camel in your
spring boot application

On Tue, Apr 3, 2018 at 8:13 PM, Garry Dias  wrote:
> Hi guys
>
> Just to warn you,
>
> Problem: CamelContext not started and then RouteBuilder implementations not
> performed
>
> Facts: camel-spring-boot-starter:2.19.5 is the last Camel version I could
> mix with spring-boot-starter-parent 1.5.10.RELEASE and see my CamelContext
> starting. It doesn´t works on 2.20.x and 2.21.x.
>
> My application startup log in debug mode is attached.



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Recommendation/Best Practice for shutting down Camel Spring Boot App

[Zoran Regvart]

Hi Jonathan,

On Wed, Apr 4, 2018 at 10:20 PM, Jonathan Cook
 wrote:
> But it isn't clear, do I have to create some kind of custom shutdown hook
> which I can call externally and stop the camel context or shut the routes
> down gracefully or does something in Camel Spring Boot do this for me?

When auto-configured under Spring Boot, or manually configured using
SpringCamelContext, Camel's context startup/shutdown is tied to Spring
context startup/shutdown.

> To shut it down I was thinking of another shell script but don't simply want
> to kill the process. I've see Spring Boot Actuator provides a way via http
> but my app is not a webapp in a fat jar, just a standalone java app. I've
> also seen Spring Boot has a ApplicationPidFileWriter which writes the pid to
> a file but again this approach uses a brute force kill.

I think you should be able just to use `kill` without specifying
signal (that is SIGTERM or -15), or `kill -2` to use SIGINT signal,
equivalent of sending CTRL+C.

zoran
-- 
Zoran Regvart