I believe we are bit smarter now. We are consuming from JMS with concurrentConsumers and so it is possible the the same FTP endpoint is served twice.
With more verbose logging, it seems as if the connection is somehow shared and the GOODBYE initiated by the operation finishing first kills the second connection as well. Is this a race condition or do we need to make sure that concurrency isn't possible at this level? -----Ursprüngliche Nachricht----- Von: Florian Posch [mailto:florian.po...@performgroup.com] Gesendet: Mittwoch, 4. April 2018 16:36 An: users@camel.apache.org Betreff: Camel FTP Producer catches errors silently Hi. We are using Camel 2.20.2 to transfer files to remote servers. Our route is built like that - sorry that I can't provide a fully featured application and all beans. from("direct:chunk.process") .routeId(DEFAULT_ROUTE_ID_STRING) .process("feedNotificationProcessor") .onException(Exception.class).handled(false) .end() .recipientList(exchangeProperty(RECIPIENTLIST_HEADER).tokenize(RECIPIENTLIST_SEPARATOR)) .parallelProcessing().timeout(40000) .executorService(customThreadPoolExecutor) .streaming().aggregationStrategyRef("recipientAggregationStrategy") .to("direct:chunk.evaluate"); We are dispatching to (multiple) dynamic FTP producers with routes like that: ftp://myHost/?fileName=myfile.xml&password=secret&binary=false&passiveMode=false&username=user&disconnect=true&fastExistsCheck=true&ftpClient.dataTimeout=30000&ftpClient.defaultTimeout=30000&ftpClient.connectTimeout=30000&soTimeout=30000&stepwise=false When something fails, we would expect that we recognize that in the 'aggregate' callback of our aggregation strategy call with 'CamelFailureEndpoint' and 'CamelExceptionCaught' properties set, so that we can retry or report failure to our system. However, in certain cases it appears that we only get a WARN logging but no error set in the Exchange [2018-04-03 14:38:21,426][[WARN ][org.apache.camel.component.file.remote.RemoteFileProducer][[]][2188110951-169-19445712825] Exception occurred during disconnecting from: ftp://myhost/? binary=false&disconnect=true&fastExistsCheck=true&ftpClient.connectTimeout=30000&ftpClient.dataTimeout=30000&ftpClient.defaultTimeout=30000&ftpClient.receiveDataSocketBufferSize=30720&ftpClient.sendDataSocketBufferSize=307 20&ftpClient.strictReplyParsing=false&passiveMode=false&password=xxxxxx&soTimeout=30000&stepwise=false&username=user File operation failed: Connection closed without indication.. Code: 221 It looks as if this just happens on cleanup of connections after upload but our recipient confirmed that the corresponding FTP session dropped just after creating the data connection 2018-04-03 14:38:21,372 [99773] <encode:5>: decoded 'myfile.xml' into 'myfile.xml' 2018-04-03 14:38:21,372 [99773] <command:7>: dispatching PRE_CMD command 'STOR myfile.xml' to mod_exec.c 2018-04-03 14:38:21,372 [99773] <command:7>: dispatching PRE_CMD command 'STOR myfile.xml' to mod_rewrite.c 2018-04-03 14:38:21,372 [99773] <command:7>: dispatching PRE_CMD command 'STOR myfile.xml' to mod_tls.c 2018-04-03 14:38:21,372 [99773] <command:7>: dispatching PRE_CMD command 'STOR myfile.xml' to mod_core.c 2018-04-03 14:38:21,372 [99773] <command:7>: dispatching PRE_CMD command 'STOR myfile.xml' to mod_core.c 2018-04-03 14:38:21,372 [99773] <command:7>: dispatching PRE_CMD command 'STOR myfile.xml' to mod_ratio.c 2018-04-03 14:38:21,372 [99773] <command:7>: dispatching PRE_CMD command 'STOR myfile.xml' to mod_quotatab.c 2018-04-03 14:38:21,372 [99773] <command:7>: dispatching PRE_CMD command 'STOR myfile.xml' to mod_xfer.c 2018-04-03 14:38:21,372 [99773] <encode:5>: decoded 'myfile.xml' into 'myfile.xml' 2018-04-03 14:38:21,374 [99773] <command:7>: dispatching CMD command 'STOR myfile.xml' to mod_xfer.c 2018-04-03 14:38:21,397 [99773] <response:1>: 150 Opening ASCII mode data connection for myfile.xml 2018-04-03 14:38:21,430 [99773] <encode:5>: decoded '/myfile.xml' into '/myfile.xml' 2018-04-03 14:38:21,430 [99773] <command:7>: dispatching POST_CMD_ERR command 'QUIT /myfile.xml' to mod_exec.c 2018-04-03 14:38:21,430 [99773] <command:7>: dispatching LOG_CMD_ERR command 'QUIT /myfile.xml' to mod_log.c 2018-04-03 14:38:21,430 [99773] <command:7>: dispatching LOG_CMD_ERR command 'QUIT /myfile.xml' to mod_core.c Is this intended behavior of Camel? it is quite inconvenient to miss out failed uploads due to that (not being aware of any errors except for logging) Kind regards, Florian CONFIDENTIALITY: This email and any files transmitted with it are confidential, may be legally privileged and are intended solely for the use of the individual or entity to whom they are addressed. If this has come to you in error, you must not copy, distribute, disclose or use any of the information it contains. Please notify the sender immediately and delete them from your system. PRIVACY: Perform Media Services Ltd (a member of the Perform Group, with Companies House registration number 3426471) may monitor email traffic data and also the content of email for the purposes of security and staff training. For information on how the Perform Group processes your information, please refer to https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.performgroup.com%2Fprivacy-notice%2F&data=02%7C01%7Cflorian.posch%40performgroup.com%7C9f3d151b8f7e440a853e08d59a39630f%7C30459df51e534d8ba1620ad2348546f1%7C1%7C0%7C636584493622620078&sdata=%2FgychTG0%2FRoJR9TveIdGRrv0h4RtQJH96tIXRqyqooU%3D&reserved=0. SECURITY: Please be aware that emails are not secure and may contain viruses. AUTHORITY: Any views or opinions expressed in this email are solely those of the sender and do not necessarily represent those of the Perform Group. COPYRIGHT: Copyright of this email and any attachments belongs to Perform Group Limited, Companies House registration number 6324278. Further information about Perform Group is available at https://emea01.safelinks.protection.outlook.com/?url=www.performgroup.com&data=02%7C01%7Cflorian.posch%40performgroup.com%7C9f3d151b8f7e440a853e08d59a39630f%7C30459df51e534d8ba1620ad2348546f1%7C1%7C0%7C636584493622620078&sdata=G1BkT7NNPfHpomFZvFU95Dj4JWHf2ivhI4JGxVU3LDY%3D&reserved=0 CONFIDENTIALITY: This email and any files transmitted with it are confidential, may be legally privileged and are intended solely for the use of the individual or entity to whom they are addressed. If this has come to you in error, you must not copy, distribute, disclose or use any of the information it contains. Please notify the sender immediately and delete them from your system. PRIVACY: Perform Media Services Ltd (a member of the Perform Group, with Companies House registration number 3426471) may monitor email traffic data and also the content of email for the purposes of security and staff training. For information on how the Perform Group processes your information, please refer to http://www.performgroup.com/privacy-notice/. SECURITY: Please be aware that emails are not secure and may contain viruses. AUTHORITY: Any views or opinions expressed in this email are solely those of the sender and do not necessarily represent those of the Perform Group. COPYRIGHT: Copyright of this email and any attachments belongs to Perform Group Limited, Companies House registration number 6324278. Further information about Perform Group is available at www.performgroup.com
