Hi Raymond, that `reuseSession=true` setting would have to rely on subclassing the FTPS implementation in commons-net and still do reflection tricks that would tie it to a specific JDK implementation of TLS. Not sure that this is something for Camel to solve.
It's often that we find ourselves in situations like this, we can make something work but the cost of making it work is in all the future maintenance that has to be done to make this work in the future. I think that this change needs to be made in commons-net or we need to find and migrate to a different FTP/FTPS/SFTP library in camel-ftp that supports this and all other currently supported features -- this might not be such a small undertaking... (2c) zoran On Thu, Feb 21, 2019 at 8:38 PM ski n <[email protected]> wrote: > > Hi Zoran, > > That's a great tip and I think you are right. Just setting the system > property didn't work however. Probably because Camel's own implementation. > For example one can set the SSL/TLS version from system property, but in > Camel this a separate property. It might work when using the "ftpclient" > property which also let you use a custom > org.apache.commons.net.ftp.FTPClient instance. This however is a little > against using the Camel DSL and configuration. Then one might write a > custom bean or use a third-party tool which acts as a proxy in between, but > these both are kind of nasty. Hope in the future there will a property add > "reuseSession=true" the FTPS component that would probably more elegant. > > Raymond > > > > > > Op do 21 feb. 2019 om 14:29 schreef Zoran Regvart <[email protected]>: > > > Hi Raymond, > > from what I understand the security measure in FileZilla requires that > > the same TLS session is used for the control and data connections, am > > I correct? > > > > In your research have you come across this StackOverflow post? > > > > > > https://stackoverflow.com/questions/32398754/how-to-connect-to-ftps-server-with-data-connection-using-same-tls-session > > > > I'm not sure if that would be a good fit to implement in Camel, it > > seems to rely on a Java system property > > `jdk.tls.useExtendedMasterSecret` being set to `false` on newer JVMs > > and also reflection tricks to fool the TLS session cache. > > > > I think it might be against the design of JSSE to reuse the same SSL > > session across different ports. > > > > zoran > > > > On Wed, Feb 20, 2019 at 8:04 PM ski n <[email protected]> wrote: > > > > > > I looked again into this issue. The issue occurs, because of TLS session > > > resumption. There is an option enabled by default on the FileZilla server > > > to avoid an exploit. See this forum post: > > > > > > https://forum.filezilla-project.org/viewtopic.php?p=137191#p137191 > > > > > > The proposed solution on StackOverflow is to turn off this security > > option: > > > > > > > > https://stackoverflow.com/questions/35061145/ftpsclient-file-upload-and-download-always-size-0-and-exception > > > > > > However, this isn't desirable and is not always an option because the > > > server-side cannot be changed. See also this question: > > > > > > > > https://stackoverflow.com/questions/50852231/camel-ftps-how-enable-tls-resumption > > > > > > As the questioner wrote there's currently no way to enable TLS enable > > from > > > Camel FTPS options. This issue can easily be reproduced by installing a > > > FileZilla server with FTPS with default options. > > > > > > Raymond > > > > > > > > > > > > Op ma 11 feb. 2019 om 22:08 schreef ski n <[email protected]>: > > > > > > > Yes, I have JCE accessible in the JVM. I am using latest Java 8 Update > > > > 201. > > > > > > > > As in this anser stackoverflow the unlimited shouldn't be set anymore: > > > > > > > > > > > > > > https://golb.hplar.ch/2017/10/JCE-policy-changes-in-Java-SE-8u151-and-8u152.html > > > > > > > > To be sure I did enable crypto.policy=unlimited in the security file. > > But > > > > this hadn't any effect. > > > > > > > > Raymond > > > > > > > > > > > > Op ma 11 feb. 2019 om 14:43 schreef Björn Þór Jónsson <[email protected]>: > > > > > > > >> Do you have the Java Cryptography Extension (JCE) accessible to your > > JVM ? > > > >> > > > >> In one case Camel had trouble communicating with an FTPS server which > > was > > > >> using a cipher not available to the default Java installation and I > > solved > > > >> it by installing that extension. > > > >> > > > >> /Björn > > > >> > > > >> On 10/02/2019, 19:09, "ski n" <[email protected]> wrote: > > > >> > > > >> I'm trying to send some text files to a FTPS server. The FTPS > > server > > > >> has > > > >> following specifications: > > > >> > > > >> 1) FTP Type: FileZilla Server > > > >> 2) Cryptographic protocol = TLS/SSL Explicit encryption, TLSv1.2 > > > >> 3) Encryption algorithm = TLSv1/SSLv3: > > ECDHE-RSA-AES256-GCM-SHA384, > > > >> 2048 > > > >> bit RSA, ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA > > > >> Enc=AESGCM(256) Mac=AEAD > > > >> 4) Mode: Passive > > > >> 5) Port: 21 (NAT ports 50000-500010) > > > >> > > > >> I can access and send the files with WinSCP or FilleZilla client. > > When > > > >> using the FTPS Component of Camel (using a TrustStore and set > > > >> passivemode > > > >> to true) the file is created within the FTPS Server directory, > > > >> however with > > > >> 0 kb. No data is send. In the logs I got the following error: > > > >> > > > >> Remote host closed connection during handshake. Code: 150 > > > >> Writing file failed with: File operation failed: 150 Opening data > > > >> channel > > > >> for file upload to server of "/file.csv" > > > >> > > > >> Stacktrace > > > >> > > > >> > > --------------------------------------------------------------------------------------------------------------------------------------- > > > >> > > > >> > > org.apache.camel.component.file.GenericFileOperationFailedException: > > > >> File > > > >> operation failed: 150 Opening data channel for file upload to > > server > > > >> of > > > >> "/file.csv" > > > >> Remote host closed connection during handshake. Code: 150 > > > >> at > > > >> > > > >> > > org.apache.camel.component.file.remote.FtpOperations.doStoreFile(FtpOperations.java:710) > > > >> at > > > >> > > > >> > > org.apache.camel.component.file.remote.FtpOperations.storeFile(FtpOperations.java:615) > > > >> at > > > >> > > > >> > > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305) > > > >> at > > > >> > > > >> > > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169) > > > >> at > > > >> > > > >> > > org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:57) > > > >> at > > > >> > > > >> > > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61) > > > >> at > > > >> > > > >> > > org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:178) > > > >> at > > > >> > > > >> > > org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:445) > > > >> at > > > >> > > > >> > > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:173) > > > >> at > > > >> > > > >> > > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) > > > >> at > > > >> > > > >> > > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) > > > >> at > > > >> > > > >> > > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) > > > >> at > > > >> > > > >> > > org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109) > > > >> at > > > >> > > > >> > > org.apache.camel.processor.MulticastProcessor.doProcessParallel(MulticastProcessor.java:860) > > > >> at > > > >> > > > >> > > org.apache.camel.processor.MulticastProcessor.access$200(MulticastProcessor.java:86) > > > >> at > > > >> > > > >> > > org.apache.camel.processor.MulticastProcessor$1.call(MulticastProcessor.java:330) > > > >> at > > > >> > > > >> > > org.apache.camel.processor.MulticastProcessor$1.call(MulticastProcessor.java:316) > > > >> at java.util.concurrent.FutureTask.run(Unknown Source) > > > >> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown > > > >> Source) > > > >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown > > > >> Source) > > > >> at java.lang.Thread.run(Unknown Source) > > > >> Suppressed: > > > >> > > org.apache.camel.component.file.GenericFileOperationFailedException: > > > >> File > > > >> operation failed: 150 Opening data channel for file upload to > > server > > > >> of > > > >> "/file.csv" > > > >> Remote host closed connection during handshake. Code: 150 > > > >> ... 21 common frames omitted > > > >> Caused by: javax.net.ssl.SSLHandshakeException: Remote host > > closed > > > >> connection during handshake > > > >> at sun.security.ssl.SSLSocketImpl.readRecord(Unknown > > Source) > > > >> at > > > >> sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown > > > >> Source) > > > >> at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown > > > >> Source) > > > >> at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown > > > >> Source) > > > >> at > > > >> > > > >> > > org.apache.commons.net.ftp.FTPSClient._openDataConnection_(FTPSClient.java:646) > > > >> at > > > >> > > org.apache.commons.net.ftp.FTPClient._storeFile(FTPClient.java:653) > > > >> at > > > >> > > org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:639) > > > >> at > > > >> > > org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:2030) > > > >> at > > > >> > > > >> > > org.apache.camel.component.file.remote.FtpOperations.doStoreFile(FtpOperations.java:685) > > > >> ... 20 common frames omitted > > > >> Caused by: java.io.EOFException: SSL peer shut down > > incorrectly > > > >> at sun.security.ssl.InputRecord.read(Unknown Source) > > > >> ... 29 common frames omitted > > > >> Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed > > > >> connection during handshake > > > >> at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source) > > > >> at > > sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown > > > >> Source) > > > >> at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown > > Source) > > > >> at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown > > Source) > > > >> at > > > >> > > > >> > > org.apache.commons.net.ftp.FTPSClient._openDataConnection_(FTPSClient.java:646) > > > >> at > > > >> org.apache.commons.net.ftp.FTPClient._storeFile(FTPClient.java:653) > > > >> at > > > >> org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:639) > > > >> at > > > >> org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:2030) > > > >> at > > > >> > > > >> > > org.apache.camel.component.file.remote.FtpOperations.doStoreFile(FtpOperations.java:685) > > > >> ... 20 common frames omitted > > > >> Caused by: java.io.EOFException: SSL peer shut down incorrectly > > > >> at sun.security.ssl.InputRecord.read(Unknown Source) > > > >> ... 29 common frames omitted > > > >> > > > >> - Camel is run with Java 8 Update 201. > > > >> - The certificated was downloaded with OpenSSL and successfully > > > >> imported in > > > >> the truststore with Keytool > > > >> > > > >> When searching for this type of error I found the following > > resources: > > > >> > > > >> > > > >> > > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.jboss.org%2Fthread%2F247957&data=02%7C01%7Cbthj%40origo.is%7Cf6e9e3d54e5847d8ff3c08d68f8b4402%7C470852d578e34ee3bee9e9638b276d14%7C0%7C0%7C636854225647654211&sdata=BnB7mypukyWdnJOa86xiPXN78aAL3twZ4jFSSOaNn%2BA%3D&reserved=0 > > > >> > > > >> > > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F21245796%2Fjavax-net-ssl-sslhandshakeexception-remote-host-closed-connection-during-handsh&data=02%7C01%7Cbthj%40origo.is%7Cf6e9e3d54e5847d8ff3c08d68f8b4402%7C470852d578e34ee3bee9e9638b276d14%7C0%7C0%7C636854225647654211&sdata=8FjzMN73kfXTGPWf22%2Bo8vFcPuofomKtzNCt4GBbPaA%3D&reserved=0 > > > >> > > > >> These sources recommended that following actions: > > > >> > > > >> 1) Setting the-Dhttps.protocols parameter at Java: > > > >> > > > >> I tried the following: > > > >> > > > >> -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 > > > >> > > > >> -Dhttps.protocols=TLSv1.1,TLSv1.2 > > > >> > > > >> -Dhttps.protocols=TLSv1.2 > > > >> > > > >> 2) Set RCE (*Java Cryptography Extension*) to unlimited in > > > >> java.security file > > > >> > > > >> 3) Try settings passiveMode=false and set the parameter > > ActivePorts > > > >> (this causes an "Accepted time out") and I also tried it with > > > >> passiveMode=true > > > >> > > > >> All actions still gave the same result. When running the Camel > > > >> application with parameter: > > > >> > > > >> -Djavax.net.debug=all > > > >> > > > >> I see the failure during WRITE: > > > >> > > > >> Camel (camel-1) thread #2 - Multicast, WRITE: TLSv1.2 Handshake, > > > >> length = 96 > > > >> [Raw write]: length = 101 > > > >> 0000: 16 03 03 00 60 88 00 8B E3 1B DD 54 7B DE 7A 01 > > > >> ....`......T..z. > > > >> 0010: 35 A6 44 CC AF 6A 4C 4F A6 AB C6 9E 04 FA F5 0F > > > >> 5.D..jLO........ > > > >> 0020: 61 CF AF CD FC B1 E4 F2 01 0D 15 33 18 C3 FA B7 > > > >> a..........3.... > > > >> 0030: 34 6A A5 53 60 4A E9 B3 63 99 31 98 F6 97 A7 81 > > > >> 4j.S`J..c.1..... > > > >> 0040: AB 9D 83 D4 7B C4 F8 3B D0 54 45 91 E4 13 58 C6 > > > >> .......;.TE...X. > > > >> 0050: AA 20 CC 81 ED 62 C0 FD F3 D8 90 53 85 DF 7D 0E . > > > >> ...b.....S.... > > > >> 0060: A6 E7 E0 AC 51 ....Q > > > >> Camel (camel-1) thread #2 - Multicast, received EOFException: > > error > > > >> Camel (camel-1) thread #2 - Multicast, handling exception: > > > >> javax.net.ssl.SSLHandshakeException: Remote host closed connection > > > >> during handshake > > > >> %% Invalidated: [Session-2, > > TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384] > > > >> Camel (camel-1) thread #2 - Multicast, SEND TLSv1.2 ALERT: fatal, > > > >> description = handshake_failure > > > >> > > > >> I have three questions based on this: > > > >> > > > >> 1) How to debug this further? Did I miss an option of the FTPS > > > >> component? Does this happen because of muliple session/threads? > > > >> > > > >> 2) How can it be that the message/file is consumed, then there is > > > >> failure, but I don't see the message in the error endpoint? Camel > > > >> statistics > > > >> says the message is completed, but it's not really transfered > > (it's > > > >> gone at consumer and producer). > > > >> > > > >> 3) Isn't there a way that Camel can handle certificates just like > > > >> clients like WinSCP or FileZilla do? This opposed of doing it > > manually > > > >> (a) create truststore, b) download certificatei > > > >> > > > >> and c) import certificate in truststore and d) link Camel route to > > > >> truststore? > > > >> > > > >> Raymond > > > >> > > > >> > > > >> > > > > > > > > -- > > Zoran Regvart > > -- Zoran Regvart
