Hi Kenneth, Of course there are FTP-capable (reverse) proxies out there. Maybe there are no free ones; that´s different...
Regarding when to close the data channel, that should be easy: if the FTP server closes the data channel, the proxy server should close it as well... Is there a reason why you can´t do that? It´s been a while since I developed our small proxy but I´m quite sure it was enough for us. You mention you´ve been debugging the issue for 6 weeks but only recently (the last day?) realised that the data channel was not closed by the proxy. I think you need some way to get a better insight on what the proxy is doing. My recommendation is a tcpdump network trace from the Proxy server itself so you can capture the traffic between the FTP server and the proxy, as well as the traffic between the proxy and the client. If I have understood your problem correctly, you think the "unknown internal server error" only happens when there´s a proxy between client and server (although that should be transparent for the server). It´s of course a potential explanation, but in all honesty, I haven´t even seen any evidence that the FTPServer is the one closing the control channel. If the network traces say that it´s definitely the FTPServer deciding to close the connection and the SFTP server logs don´t show anything, I believe you´ll have to debug the FTPServer to see what´s going on. I´m sorry we cannot be more helpful. It was with a previous version of FTPServer, but we have used a proxy to forward FTPS traffic to Mina Ftpserver so I´m hoping you´ll finally get to the bottom of this. Best regards, David El jue., 22 ago. 2019 a las 21:15, Kenneth Taylor (< kenneth.tay...@dataexpress.com>) escribió: > David, > > When you say "a proxy that understands FTP", there is no such proxy. > Remember we are doing reverse proxy, not proxy. And the proxy we are using > is one we wrote. And we do have to tell it when to close the data > channel. There is nothing to magically do that. We have now changed our > server Ftplet.afterCommand() to look for the 226 reply and to call the > proxy to close the data chnnel. So that part is fixed. > > But the client still fails on MLSD response. It does not understand what > is coming from the proxy I guess, but its identical to what's coming from > the server. So the only thing that could be causing the problem is that > Apache is closing the command channel, which we see it doing, and it should > not be doing that. We set all the timeouts to large values so that’s not > it. > > I think we've stumbled upon a bug. We've been debugging this for 6 weeks > so we didn't come to this conclusion lightly. Could there be a hard coded > timeout that we just happen to be tripping because the proxy imparts a > small delay? I did see in the debugger a write timeout that users don't > have access to and it was set to 60. Our entire test session only lasts > about 3 seconds. > > Our proxy is very simple. Its just a client side socket and a server side > socket with stream copiers for the 4 streams. Nothing magical. It should > be completely transparent. The stream copiers are doing line-based IO to > match FTP with auto-flush on. We know the socket close logic is at the > heart of the problem. Apache is closing the command socket and I'll bet > there is some internal hidden error causing that. > > Thanks. > Ken > > -----Original Message----- > From: David Latorre <dvl...@gmail.com> > Sent: Thursday, August 22, 2019 12:51 PM > To: ftpserver-users@mina.apache.org > Subject: Re: FTP Data Channels Through Reverse Proxy > > From your first line I understand that the problem is with the proxy > that's not closing the data channel, isn't it? > > I guess that for a proxy that understands FTP ,you shouldn't do anything > to ensure the proxy closes the channel when the transfer is done. > > It's not clear from your message but I would think that if the connection > gets closed it will be due to some inactivity timeout being reached. Other > than the idle time, I cannot understand why the control channel would be > closed, I can guarantee it's never happened to us. > > Overriding PASV to provide a different IP address works fine and is > correctly supported. I don't know why you want to capture anything after > MLSD completes. If the data channel is closed the FTP proxy will close it's > own data channel to the client. If it doesn't, you need to see how to fix > the proxy. Why would you try to workaround it for just the MLSD command? > don't you ever need to download files? > I wouldn't try to find a workaround in any case to be honest. > > > Best regards, > David > > On Thu, 22 Aug 2019, 17:33 Kenneth Taylor, <kenneth.tay...@dataexpress.com > > > wrote: > > > It looks like the problem is in the closing of the data channel. Its > > happening on the server but not on the proxy so the client hangs > > waiting for that to happen, unless the command channel closes. > > Apache, for some reason, closes the command channel after the "226 > Closing data connection" > > is sent. Why? I found this by preventing the command channel from > > closing on the proxy, and that caused the client to hang after > > receiving the MLSD response. So the previous behavior was misleading > > because Apache closing the command channel was being seen by the > > client which was giving up the session entirely but complaining about > the MLSD failing. > > > > So is this a bug in Apache or just a missing piece? How would I catch > > the > > 226 response so I could tell the proxy to close the DC? > > > > We can't use afterCommand() for PASV because that would cause Apache > > to open its own DC. So it seems like we have to do an afterCommand() > > for MLSD, but then we'd have to know this was for a DC and close that > > DC instead of the Command Channel. Ugh! > > > > Thanks. > > Ken > > > > -----Original Message----- > > From: Kenneth Taylor <kenneth.tay...@dataexpress.com> > > Sent: Thursday, August 22, 2019 9:01 AM > > To: ftpserver-users@mina.apache.org > > Subject: RE: FTP Data Channels Through Reverse Proxy > > > > Thanks for the response David. I have used 3 different clients > > (FileZilla, WinSCP and custom) and they all respond the same. I can > > see the MLSD response on the Proxy and that its sending it to the client. > > That's why we are stumped. We cannot see any difference between Proxy > > and No Proxy. But Apache is closing the Data Channel & Session after > > sending the MLSD response, and it should not. Connecting direct, the > > Data Channel does not get closed, but I suppose that could be because > > Apache is detecting some kind of error that it is not reporting. I've > > turned the logs all the way up and don't see anything that would help. > > > > A network trace might be a good idea. We will try a wireshark scan. > > > > One thing that just occurred to me is that Apache may be opening more > > than one Data Channel and sending the MLSD response down the wrong > > pipe. Not sure why it would do that since our PASV handler is a > > direct copy of the Apache one (PASV), except with the change of IP and > Port in the reponse. > > Our Ftplet returns the ResultType of SKIP in the beforeCommand() for > > PASV which should stop Apache from opening its own data channel, right? > > > > Has Apache ever tested this? I would think putting an FTP Server > > behind a reverse proxy would be VERY common. Our customers demand it. > > > > I will have to look at the FTP RFCs again but I don't remember if any > > messages are supposed to go through the data channel from the client > > to the server. Could it be that Apache is expecting something on that > > stream, even if its just an ACK or disconnect? > > > > Thanks. > > Ken > > > > -----Original Message----- > > From: David Latorre <dvl...@gmail.com> > > Sent: Wednesday, August 21, 2019 6:09 PM > > To: ftpserver-users@mina.apache.org > > Subject: Re: FTP Data Channels Through Reverse Proxy > > > > Ah sorry, > > I was rechecking something and made a mistake regarding the IP > > address in the PASV response. In any case, unless you have proof (logs > > from the server or network traces of the server <-> proxy bit) that > > the server returns invalid data in the proxy case, I think you should > > focus on verifying why the client is failing and, if it is caused by > > an invalid response, why the proxy is generating it. > > > > > > > > > > On Thu, 22 Aug 2019, 00:03 David Latorre, <dvl...@gmail.com> wrote: > > > > > Hi Kenneth, > > > > > > Shouldn't you receive the proxy address as a response to PASV when > > > going through the proxy? Otherwise it would bypass the proxy, but > > > I'm assuming you should not connect directly from the client to the > > > server and that's why you need a proxy. I'm a bit puzzled since you > > > state that the proxy receives the response to MLSD but the IP that > > > the client sees for PASV is the same in both cases. > > > > > > Your client log for the proxy case does not show any listing as a > > > response for MLSD, so it's difficult to know what's going on. If you > > > cannot get any logs for that, get some network traces with tcpdump > > > and check what's going on. Enabling more detailed logging in the > > > client might also help (you can configure that in the Filezilla > settings). > > > Server logs might also prove useful. > > > > > > In any case I don't see how this would be caused by Ftpserver with > > > the information you've given. > > > > > > > > > > > > On Wed, 21 Aug 2019, 21:56 Kenneth Taylor, > > > <kenneth.tay...@dataexpress.com> > > > wrote: > > > > > >> We are trying to get Apache FTP Server to work behind a reverse proxy. > > >> The only thing not working is the data channels. We use an Ftplet > > >> to intercept and handle the PASV command and substitute our proxy’s > > >> IP and port in the PASV response. That all seems to work fine. We > > >> are testing it with FileZilla client and our server works perfectly > > >> when connecting directly to the server but fails when going through > > >> the > > proxy. > > >> > > >> > > >> > > >> We can see the data connections being established on both the > > >> server and the proxy. Our proxy logs every forwarded message and we > > >> can see that all client and server messages are being forwarded both > ways. > > >> > > >> > > >> > > >> The problem is that after the PASV command happens the next comand > > >> is MLSD and we can see the server sending the correct response and > > >> the proxy is forwarding the listing. But the client interprets the > > >> response incorrectly and closes the session. After looking at this > > >> for many many hours we’re stumped. > > >> > > >> > > >> > > >> Our Proxy is a very simple IO streams forwarder, with a control > > >> connection to tell the Proxy when to open data channels that > > >> forward back to the data channel opened by Apache on the Server. > > >> > > >> > > >> > > >> This is the client log when connecting direct: > > >> > > >> > > >> > > >> 14:10:44 Command: PASV > > >> > > >> 14:10:44 Response: 227 Entering Passive Mode (192,168,0,173,40,122) > > >> > > >> 14:10:44 Trace: Binding data connection source IP to control > > >> connection source IP 192.168.0.92 > > >> > > >> 14:10:44 Command: MLSD > > >> > > >> 14:10:44 Response: 150 File status okay; about to open data > connection. > > >> > > >> 14:10:44 Response: 226 Closing data connection. > > >> > > >> 14:10:44 Listing: Size=0;Modify=20190814174646.670;Type=dir; > > Monthly > > >> > > >> 14:10:44 Listing: Size=0;Modify=20190814180307.721;Type=dir; > Weekly > > >> > > >> 14:10:44 Listing: Size=0;Modify=20190814174833.915;Type=dir; > Daily > > >> > > >> 14:10:44 Listing: Size=0;Modify=20190814175140.777;Type=dir; > > Archive > > >> > > >> 14:10:44 Status: Directory listing of "/" successful > > >> > > >> > > >> > > >> This is the client log when connecting through the proxy: > > >> > > >> > > >> > > >> 14:37:16 Command: PASV > > >> > > >> 14:37:16 Response: 227 Entering Passive Mode (192,168,0,91,202,228) > > >> > > >> 14:37:16 Trace: Binding data connection source IP to control > > >> connection source IP 192.168.0.173 > > >> > > >> 14:37:16 Command: MLSD > > >> > > >> 14:37:16 Response: 150 File status okay; about to open data > connection. > > >> > > >> 14:37:16 Response: 226 Closing data connection. > > >> > > >> 14:37:16 Error: Disconnected from server: ECONNABORTED - > > Connection > > >> aborted > > >> > > >> 14:37:16 Error: Failed to retrieve directory listing > > >> > > >> > > >> > > >> One weird thing we see is that the “226 Closing data connection” > > >> shows up on the proxy after the MLSD listing is returned. When > > >> connecting direct it is before the listing in the client log, but > > >> in all cases these messages are coming from the same server so that > > >> may be > > a logging anomaly. > > >> > > >> > > >> > > >> Thanks for any help. > > >> > > >> Ken > > >> > > >> > > >> > > >> Disclaimer: This email from DMBGroup LLC, DMB Consulting Services > > >> LLC, or the personnel associated with either entity (collectively > > >> "*DMB*") and attachments, contain *CONFIDENTIAL, PRIVILEGED AND > > >> PROPRIETARY *information for exclusive use of the addressee > > >> individual(s) or entity. Unauthorized viewing, copying, disclosure, > > >> distribution or use of this e-mail or attachments may be subject to > > >> legal restriction or sanction. If received in error, notify sender > > >> immediately by return e-mail and delete original message and > > >> attachments. Nothing contained in this e-mail or attachments shall > > satisfy the requirements for a writing unless specifically stated. > > >> Nothing contained herein shall constitute a contract or electronic > > >> signature under the Electronic Signatures in Global and National > > >> Commerce Act, any version of the Uniform Electronic Transactions > > >> Act or any other statute governing electronic transactions. > > >> Opinions and statements expressed in this e-mail and any > > >> attachments are those of the individual sender and not necessarily > > >> of DMB. DMB does not guarantee this e-mail transmission is secured, > error or virus-free. > > >> Neither DMB nor the sender of this e-mail accepts liability for > > >> errors or omissions in the contents of this e-mail, which arise as > > >> a > > result of e-mail transmission. . > > >> > > > > > > > Disclaimer: This email from DMBGroup LLC, DMB Consulting Services LLC, > > or the personnel associated with either entity (collectively "DMB") > > and attachments, contain CONFIDENTIAL, PRIVILEGED AND PROPRIETARY > > information for exclusive use of the addressee individual(s) or > > entity. Unauthorized viewing, copying, disclosure, distribution or use > > of this e-mail or attachments may be subject to legal restriction or > > sanction. If received in error, notify sender immediately by return > > e-mail and delete original message and attachments. Nothing contained > > in this e-mail or attachments shall satisfy the requirements for a > writing unless specifically stated. > > Nothing contained herein shall constitute a contract or electronic > > signature under the Electronic Signatures in Global and National > > Commerce Act, any version of the Uniform Electronic Transactions Act > > or any other statute governing electronic transactions. Opinions and > > statements expressed in this e-mail and any attachments are those of > > the individual sender and not necessarily of DMB. DMB does not > > guarantee this e-mail transmission is secured, error or virus-free. > > Neither DMB nor the sender of this e-mail accepts liability for errors > > or omissions in the contents of this e-mail, which arise as a result of > e-mail transmission. . > > > > > > Disclaimer: This email from DMBGroup LLC, DMB Consulting Services LLC, > > or the personnel associated with either entity (collectively "DMB") > > and attachments, contain CONFIDENTIAL, PRIVILEGED AND PROPRIETARY > > information for exclusive use of the addressee individual(s) or > > entity. Unauthorized viewing, copying, disclosure, distribution or use > > of this e-mail or attachments may be subject to legal restriction or > > sanction. If received in error, notify sender immediately by return > > e-mail and delete original message and attachments. Nothing contained > > in this e-mail or attachments shall satisfy the requirements for a > writing unless specifically stated. > > Nothing contained herein shall constitute a contract or electronic > > signature under the Electronic Signatures in Global and National > > Commerce Act, any version of the Uniform Electronic Transactions Act > > or any other statute governing electronic transactions. Opinions and > > statements expressed in this e-mail and any attachments are those of > > the individual sender and not necessarily of DMB. DMB does not > > guarantee this e-mail transmission is secured, error or virus-free. > > Neither DMB nor the sender of this e-mail accepts liability for errors > > or omissions in the contents of this e-mail, which arise as a result of > e-mail transmission. . > > > > > > Disclaimer: This email from DMBGroup LLC, DMB Consulting Services LLC, or > the personnel associated with either entity (collectively "DMB") and > attachments, contain CONFIDENTIAL, PRIVILEGED AND PROPRIETARY information > for exclusive use of the addressee individual(s) or entity. Unauthorized > viewing, copying, disclosure, distribution or use of this e-mail or > attachments may be subject to legal restriction or sanction. If received in > error, notify sender immediately by return e-mail and delete original > message and attachments. Nothing contained in this e-mail or attachments > shall satisfy the requirements for a writing unless specifically stated. > Nothing contained herein shall constitute a contract or electronic > signature under the Electronic Signatures in Global and National Commerce > Act, any version of the Uniform Electronic Transactions Act or any other > statute governing electronic transactions. Opinions and statements > expressed in this e-mail and any attachments are those of the individual > sender and not necessarily of DMB. DMB does not guarantee this e-mail > transmission is secured, error or virus-free. Neither DMB nor the sender of > this e-mail accepts liability for errors or omissions in the contents of > this e-mail, which arise as a result of e-mail transmission. . > >
