Re: [twsocket] FTP Client
I see no problems here. I use Idle and Connect timeouts in production (for plain sockets), they work fine. Protocol-specific components expose TWSocket members so user can add any handlers he wants. This option might seem slighly leaking abstraction but it will work just now, without any changes to ICS. I agree that OnTimeout event should be added to protocol-specific components but necessity of under-the-hood actions is doubtable. At least it should be configurable. Windows will still be trying to connect until it's internal timeout, and probably would not allow a second connection attempt until the first had finished. Yep, of course there could be only one connect attempt per socket handle. On timeout the socket must be closed, created again and then connected. Yes, I think this is not a simple solution. In the past when I have created a timer which times out before the network and thenFTP.Abort this caused me a problem with messages like "Component Not Ready", until the whole thing had completed. So I have just left it all in its default state. Graham Why not? I think it does exactly what you need. -- A.S. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client
Yes, I think this is not a simple solution. In the past when I have created a timer which times out before the network and then FTP.Abort this caused me a problem with messages like "Component Not Ready", until the whole thing had completed. So I have just left it all in its default state. Graham On 20/11/2017 10:25, Angus Robertson - Magenta Systems Ltd wrote: ICS sockets have built-in timeout feature, just define BUILTIN_TIMEOUT and use TimeoutIdle / TimeoutConnect / OnTimeout. I suspect TimeoutConnect has never been tested, it is not implemented in any ICS protocol components or samples. Components like FTP would need the OnTimeout event to be added, which would then try to abort the connection. This might be successful for wsocket, but Windows will still be trying to connect until it's internal timeout, and probably would not allow a second connection attempt until the first had finished. Timeouts are a fact of life with TCP/IP, and difficult to fight. Because the built-in timeout stuff was not enabled by default until 2011, most protocol components and/or applications have their own timeout implementations, and generally we don't mess with stable and tested code, unless doing so adds significant benefits. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client
> ICS sockets have built-in timeout feature, just define > BUILTIN_TIMEOUT and use TimeoutIdle / TimeoutConnect / OnTimeout. I suspect TimeoutConnect has never been tested, it is not implemented in any ICS protocol components or samples. Components like FTP would need the OnTimeout event to be added, which would then try to abort the connection. This might be successful for wsocket, but Windows will still be trying to connect until it's internal timeout, and probably would not allow a second connection attempt until the first had finished. Timeouts are a fact of life with TCP/IP, and difficult to fight. Because the built-in timeout stuff was not enabled by default until 2011, most protocol components and/or applications have their own timeout implementations, and generally we don't mess with stable and tested code, unless doing so adds significant benefits. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client
OK, thanks for your help. Graham On 17/11/2017 17:26, Angus Robertson - Magenta Systems Ltd wrote: If I try and connect to a FTP server and that computer isn't even switch on, the FTP client gives up after about 25 seconds. Is this timeout goverened by the FTP component or by something deeper down in Windows networking. I was just wondering if I could shorten this timeout period. This timeout is standard TCP/IP stuff, very hard to change, buried in the Windows registry and will effect all applications. The only simple solution is to ping the remote server first, which can have a variable timeout of milliseconds upwards. But not all servers respond to ping and sometimes firewalls block it deliberately, like mine. Look at the OverbyteIcsPingTst.dpr sample, and TPingThread in particular. The FTP Client "Timeout" property doesn't seem to have any effect on this, so the next question would be as to what this does affect? Timeout during a connection, when no traffic is moving. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client
> If I try and connect to a FTP server and that computer isn't even > switch on, the FTP client gives up after about 25 seconds. Is > this timeout goverened by the FTP component or by something > deeper down in Windows networking. I was just wondering if I > could shorten this timeout period. This timeout is standard TCP/IP stuff, very hard to change, buried in the Windows registry and will effect all applications. The only simple solution is to ping the remote server first, which can have a variable timeout of milliseconds upwards. But not all servers respond to ping and sometimes firewalls block it deliberately, like mine. Look at the OverbyteIcsPingTst.dpr sample, and TPingThread in particular. > The FTP Client "Timeout" > property doesn't seem to have any effect on this, so the next > question would be as to what this does affect? Timeout during a connection, when no traffic is moving. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] FTP Client
Hello, If I try and connect to a FTP server and that computer isn't even switch on, the FTP client gives up after about 25 seconds. Is this timeout goverened by the FTP component or by something deeper down in Windows networking. I was just wondering if I could shorten this timeout period. The FTP Client "Timeout" property doesn't seem to have any effect on this, so the next question would be as to what this does affect? Regards Graham -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client - Port Number
I have a user who wants to change the FTP Port number to something like 14581. Component wise there is no problem using strange ports, I use 2100 for testing stuff. But NAT routers may be less forgiving, they need to modify IP addresses in the control channel to forward packets and know which port to open for the data connection, and expect port 21. This is mainly a server issue, the client can use passive mode. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] FTP Client - Port Number
I have a user who wants to change the FTP Port number to something like 14581. At the client end of the operation, is this just a case of changing: Ftp1.Port := 21; To Ftp1.Port := 14581 Or is there some gotcha I am not aware of. Regards Graham -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client - reconnecting
Angus, trying to change the code to use your components. Does the TMagFtp component support a CopyType := FCTypeDates ; if so, how does one set the dates as fMagFileCopy is declared as private so these aren't accessible ? UseUTC := true // whether to compare file timestamps in UTC or local time CopyLTime := Now - 365; CopyHiDT:= Now + 1; Regards David -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client - reconnecting
Angus, trying to change the code to use your components. Does the TMagFtp component support a CopyType := FCTypeDates ; Sorry, seems like I never supported specific date ranges for FTP which is mostly used to synchronise remote and local directories, or for specific files. if so, how does one set the dates as fMagFileCopy is declared as private so these aren't accessible ? You'd have to create new published properties for the component and makes sure they are passed when comparing the directories. Beware none of my demo or current live applications use date ranges, that feature may have got implemented for a private project 10 years ago but has certainly not been tested in recent years. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client - reconnecting
Angus Now trying with your components and although it's much better behaved and fails gracefully, it still fails (in my app or the supplied demo app). However as both your app and my app fail with my hosted FTP server and both work with your FTP server, I'm going to assume that there's something amiss with the ISP's FTP offering. Any clues in the log as to what it could be ? This is the end of the log - though it doesn't always die in the same place - sometimes it's during the directory listing stage. Source Directory: /pub/HireTRACK NX/autoupdates/ Target Directory: c:\FTP_test Source Files Skipped 0 Selected Total Files 1139, Total Size 700,295,024 Started FTP Download from: ftp.navigator.co.uk CWD /pub/HireTRACK NX/autoupdates/HireTrack NX/3.0.1.151/ 250 CWD command successful PWD 257 /pub/HireTRACK NX/autoupdates/HireTrack NX/3.0.1.151 is the current directory MLST HireTrack NX.CAB modify=20130803144603;perm=adfr;size=12325384;type=file;unique=811U141EC093;UNIX.group=697921;UNIX.mode=0644;UNIX.owner=10749373; HireTrack NX.CAB 250 End of list Downloading File: /pub/HireTRACK NX/autoupdates/HireTrack NX/3.0.1.151/HireTrack NX.CAB to c:\FTP_test\HireTrack NX\3.0.1.151\HireTrack NX.CAB, Size 12,325,384 Saved File Resume Info c:\ftp_test\hiretrack nx\3.0.1.151\hiretrack nx_cab.ftp PASV 227 Entering Passive Mode (205,196,214,163,235,210). 15:16:02:153 ! Data Socket Connect 15:16:02:153 011A0800 Socket handle created 324 15:16:02:169 TWSocket will connect to 205.196.214.163:60370 RETR HireTrack NX.CAB 15:16:02:325 ! Data Session Connected (Get) 150 Opening BINARY mode data connection for HireTrack NX.CAB (12325384 bytes) FTP Session Closed 15:16:04:263 011A0800 TCustomWSocket.Shutdown 1 324 15:16:04:263 ! Data Session closed 15:16:04:263 011A0040 TCustomWSocket.Shutdown 1 292 Downloaded File c:\ftp_test\hiretrack nx\3.0.1.151\hiretrack nx_cab.tmp, Size 321,058 Request Failed: Partial File Downloaded, Resume Allowed, Expected File Size 12,325,384, Actual Size 321,058 Download Failed: 500 Control connection closed - Winsock - No Error (#0) No Connection to Server, Cancelled Downloading Files Task Result: Failed No Connection to Server, Cancelled Downloading Files David -- David Rose Navigator Systems Ltd, Network Business Centre, 9 River Street Place, Julian Road, Bath BA1 2RS Main Tel: +44 (0)207 183 0011 Fax: +44 (0)1225 464984 Company Email: da...@navigator.co.uk (Private e-mail on request) CONFIDENTIALITY NOTICE The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error please e-mail the sender by replying to this message. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client - reconnecting
*Subject:* Re: [twsocket] FTP Client - reconnecting Now trying with your components and although it's much better behaved and fails gracefully, it still fails (in my app or the supplied demo app). However as both your app and my app fail with my hosted FTP server and both work with your FTP server, I'm going to assume that there's something amiss with the ISP's FTP offering. Any clues in the log as to what it could be ? FTP Session Closed Downloaded File c:\ftp_test\hiretrack nx\3.0.1.151\hiretrack nx_cab.tmp, Size 321,058 Request Failed: Partial File Downloaded, Resume Allowed, Expected File Size 12,325,384, Actual Size 321,058 Looks like the session closed after two seconds of downloading, once 300K of data had been done. No reason for the server or client to close during a data transfer, it could be a firewall somewhere closing the TCP connection. My own firewall does this if it finds something nasty part way through the file, it's been deleting mails supposedly from the taxman at gateway.gov.uk today which are actually viruses, the emails are very authentic since I had a real one yesterday when I submitted taxes online, and then got a reply to the email address less than 12 hours later that was almost the same except for a zip attachment. Or maybe the FTP server has a badly implemented session timeout? Can you get the server logs or is not your server? Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client - reconnecting
Angus Turned everything firewall like off and it makes no difference, and I assume that wouldn't explain why when downloading the same root folder each time, it would fail in a different place (sometimes not at all). I'm pretty sure it's something server side, but it's hosted by big US ISP, so not much chance of getting logs. I'll contact their support though and see if I can get anything. Thanks David On 07/11/2013 18:02, Angus Robertson - Magenta Systems Ltd wrote: Or maybe the FTP server has a badly implemented session timeout? Can you get the server logs or is not your server? -- David Rose Navigator Systems Ltd, Network Business Centre, 9 River Street Place, Julian Road, Bath BA1 2RS Main Tel: +44 (0)207 183 0011 Fax: +44 (0)1225 464984 Company Email: da...@navigator.co.uk (Private e-mail on request) CONFIDENTIALITY NOTICE The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error please e-mail the sender by replying to this message. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client - reconnecting
On 07/11/2013 16:15, Angus Robertson - Magenta Systems Ltd wrote: Angus, trying to change the code to use your components. Does the TMagFtp component support a CopyType := FCTypeDates ; Sorry, seems like I never supported specific date ranges for FTP which is mostly used to synchronise remote and local directories, or for specific files. The following code inserted in to TMagFTP.FtpDownFiles seems to do the job I need if (CopyType = FCTypeDates) and (FEarliestDate 0) then begin for i := 0 to Pred(TotSrcFiles) do begin SrcFileRec := SrcFileList[i]; with SrcFileRec^ do begin if FrFileCopy = FCStateSelect then begin if FrFileUDT FEarliestDate then begin FrFileCopy := FCStateNone; Dec(fTotProcFiles); end; end; end; end; if fTotProcFiles = 0 then begin result := TaskResOKNone; fReqResponse := 'No Specific Source Files Selected to Download'; Exit; end; end; David -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Client - reconnecting
I don't think so (though I will double check), as this is happening about 30 seconds in to the process, I've already iterated through all the directories and put together a list of files I need to download and downloaded quite a few of them. Never the less, your log shows a connected event, followed by the prompts for a new connection to a FTP server, so maybe you disconnected earlier and reconnected somehow? I tested earlier with your DUNmanager as I read that it uses the TMagFtp and that had a similar problem (though it could be because I hadn't set something correctly) I tried the View Remote Files option, with all logging turned on. After a couple of hundred files it gave up too. PASV 227 Entering Passive Mode (205,196,214,163,179,155). MLSD /pub/HireTRACK NX/autoupdates/navigator_nxserver/3.0.1.67/ FTP Session Closed Directory Listing Abandoned, Not Connected Directory Listing Abandoned, Failed Response QUIT Task Result: Failed Request Failed: No Directory Listing It's possible you have reached a large directory that is taking a long time to process, and the session is timing out. I got caught myself the other day with a large download that took longer than 60 seconds to compress using Z Mode, and the session closed before the server responds. Try increasing the Abort Timeout to 3 or 4 minutes and retries to 2 or 3 above it. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
