Re: [twsocket] HTTPcli: source path question
Francois wrote: In HTTP world, there is no real directory concept. There are only documents. It happens that some webservers, if configured so could display a directory content if the default document is missing. That directory content is a HTML page built automatically by the webserver. Yes, I've realized it already This is not always the case. I would not rely on that behaviour. Zvone wrote: So you cannot really know how folders are structured on the server is just by looking at the URL. Sad :( That's what I was afraid of... Well, then I have a question: maybe you have some ideas of how to organize recursive download: for example, if user started to download www.example.com/path/index.html, we should also accept www.example.com/path/logo.jpg and so on, but not www.example.com/index.php. If user started www.example.com/path/foo, we should accept www.example.com/path/foo/index.php but NOT www.example.com/path/bar.jpg. Applications like Wget do support this behavior but the question is how they do it. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Experimental built-in throttle and timeout changed again
Hi, Just checked-in rev. #588-590, a rework of the experimental throttle and timeout features. Log: TFtpClient - If conditional BUILTIN_THROTTLE is defined the bandwidth control uses TWSocket's built-in throttle code rather than TFtpClient's. Files: U trunk/Delphi/Vc32/OverbyteIcsFtpCli.pas Log: Reworked the experimental timeout and throttle code. Method names of TCustomTimeoutWSocket **changed**, they all got prefix Timeout. Removed the crappy TCustomTimerWSocket class, both throttle and timeout use their own TIcsThreadTimer instance now. Files: U trunk/Delphi/Vc32/OverbyteIcsWSocket.pas Log: Added boolean property TIcsThreadTimer.KeepThreadAlive. If it's set prevents the underlying, shared thread object from being freed when its reference count becomes 0. This fixes a serious performance leak when just a single TIcsThreadTimer instance was used which was created/freed, enabled/unabled or its interval or event handler were changed frequently. Best performance is achieved by setting this property to TRUE. Files: U trunk/Delphi/Vc32/OverbyteIcsThreadTimer.pas -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTPcli: source path question
Hello Anton, You must parse the HTML for this. We use a Delphi HTML parser which I downloaded from sourceforge for this but sometimes it raises an exception. Search for that and if you cannot find it I will do my best to search it for you in our projects... Regards, SZ On Wed, Sep 8, 2010 at 10:15 AM, Anton S. an...@rambler.ru wrote: Francois wrote: In HTTP world, there is no real directory concept. There are only documents. It happens that some webservers, if configured so could display a directory content if the default document is missing. That directory content is a HTML page built automatically by the webserver. Yes, I've realized it already This is not always the case. I would not rely on that behaviour. Zvone wrote: So you cannot really know how folders are structured on the server is just by looking at the URL. Sad :( That's what I was afraid of... Well, then I have a question: maybe you have some ideas of how to organize recursive download: for example, if user started to download www.example.com/path/index.html, we should also accept www.example.com/path/logo.jpg and so on, but not www.example.com/index.php. If user started www.example.com/path/foo, we should accept www.example.com/path/foo/index.php but NOT www.example.com/path/bar.jpg. Applications like Wget do support this behavior but the question is how they do it. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] 535 SSL handshake failed. Error #1
Hi, i've a problem with on a customer pc. When i try to open a connection over tls layer, i recive the error 535 SSL handshake failed. Error #1. I use the last ICS package and delphi 2010, i use OverbyteIcsSslFtpTst.exe for this test. Anyone can help me? Best regards daniele This is IcsLog: 09.42.49.399 TWSocket will connect to 95.110.201.126:21 09.42.49.399 00A6D550 TryToSend 212 09.42.49.399 00A6D550 TriggerDataSent 212 09.42.49.649 |220-- Welcome to Pure-FTPd [privsep] [TLS] --| 09.42.49.649 |220-You are user number 2 of 80 allowed.| 09.42.49.649 |220-Local time is now 09:42. Server port: 21.| 09.42.49.649 |220-IPv6 connections are also welcome on this server.| 09.42.49.659 |220 You will be disconnected after 20 minutes of inactivity.| 09.42.50.711 00A6D550 PutDataInSendBuffer 212 len 27 [1] 09.42.50.711 00A6D550 TryToSend 212 09.42.50.711 00A6D550 TryToSend 212 09.42.50.711 00A6D550 TriggerDataSent 212 09.42.50.831 |331 User xxx OK. Password required| 09.42.51.912 00A6D550 PutDataInSendBuffer 212 len 13 [2] 09.42.51.912 00A6D550 TryToSend 212 09.42.51.912 00A6D550 TryToSend 212 09.42.51.912 00A6D550 TriggerDataSent 212 09.42.52.143 |230-User xxx has group access to: easwebjv | 09.42.52.143 |230-OK. Current restricted directory is /| 09.42.52.143 |230 38714 Kbytes used (2%) - authorized: 1638400 Kb| 09.42.54.586 00A6D550 PutDataInSendBuffer 212 len 8 [3] 09.42.54.586 00A6D550 TryToSend 212 09.42.54.586 00A6D550 TryToSend 212 09.42.54.586 00A6D550 TriggerDataSent 212 09.42.54.706 |200 TYPE is now 8-bit binary| 09.42.56.599 00A6D550 PutDataInSendBuffer 212 len 10 [4] 09.42.56.599 00A6D550 TryToSend 212 09.42.56.599 00A6D550 TryToSend 212 09.42.56.599 00A6D550 TriggerDataSent 212 09.42.56.719 |234 AUTH TLS OK.| 09.42.56.719 00A6D550 StartSslHandshake 212 09.42.56.919 00A6D550 InitSSLConnection 212 09.42.56.919 00A6D550 BIO_ctrl(sslbio, BIO_C_SET_SSL, BIO_NOCLOSE, 0x1085A70) = 1 [5] 09.42.56.919 ICB SSL_CB_HANDSHAKE_START 09.42.56.919 ICB SSL_connect: before/connect initialization 09.42.56.919 ICB SSL_connect: SSLv2/v3 write client hello A 09.42.56.919 ICB SSL_connect: error in SSLv2/v3 read server hello A 09.42.56.919 00A6D550 BIO_read(sslbio, 0x1, 0) = -1 [6] 09.42.56.919 00A6D550 BIO_should_retry(sslbio) = 1 [7] 09.42.56.919 00A6D550 TriggerEvent sslFdRead 212 09.42.56.919 00A6D550 TriggerEvent sslFdWrite 212 09.42.56.919 SslAsyncSelect 212, 1 FD_READ 09.42.56.919 00A6D550 TCustomSslWSocket.Do_FD_READ 212 09.42.56.919 00A6D550 BIO_ctrl_get_read_request(nbio) = 7 [8] 09.42.56.919 00A6D550 Winsock recv( 212, 0x12DD44, 7, 0) = -1 [9] 09.42.56.919 00A6D550 TriggerEvents 212 SslState: SSL_ST_INIT // MayFD_Read=0 MayDoRecv=-1 MayFD_Write=-1 MaySslTryToSend=-1 bSslAllSent=0 bAllSent=-1 09.42.56.919 00A6D550 BIO_ctrl_pending(nbio) = 90 [10] 09.42.56.919 00A6D550 BIO_ctrl_get_write_guarantee(nbio) = 4096 [11] 09.42.56.919 SslAsyncSelect 212, 2 FD_WRITE 09.42.56.919 00A6D550 TCustomSslWSocket.Do_FD_WRITE 212 09.42.56.919 00A6D550 BIO_ctrl_pending(nbio) = 90 [12] 09.42.56.919 00A6D550 BIO_read(nbio, 0x12BD60, 90) = 90 [13] 09.42.56.919 00A6D550 my_RealSend (0xD4, 1228128, 90) = 90 [14] 09.42.56.919 00A6D550 BIO_ctrl_pending(nbio) = 0 [15] 09.42.56.919 00A6D550 TriggerEvents 212 SslState: SSL_ST_INIT // MayFD_Read=0 MayDoRecv=-1 MayFD_Write=-1 MaySslTryToSend=-1 bSslAllSent=0 bAllSent=-1 09.42.56.919 00A6D550 BIO_ctrl_pending(nbio) = 0 [16] 09.42.56.919 00A6D550 BIO_ctrl_get_write_guarantee(nbio) = 4096 [17] 09.42.56.919 00A6D550 TCustomSslWSocket.Do_FD_WRITE 212 09.42.56.919 00A6D550 BIO_ctrl_pending(nbio) = 0 [18] 09.42.56.919 00A6D550 BIO_read(nbio, 0x12BD6C, 0) = 0 [19] 09.42.56.919 00A6D550 TriggerEvents 212 SslState: SSL_ST_INIT // MayFD_Read=0 MayDoRecv=-1 MayFD_Write=-1 MaySslTryToSend=-1 bSslAllSent=0 bAllSent=-1 09.42.56.919 00A6D550 BIO_ctrl_pending(nbio) = 0 [20] 09.42.56.919 00A6D550 BIO_ctrl_get_write_guarantee(nbio) = 4096 [21] 09.42.56.919 00A6D550 TCustomSslWSocket.Do_FD_WRITE 212 09.42.56.919 00A6D550 BIO_ctrl_pending(nbio) = 0 [22] 09.42.56.919 00A6D550 BIO_read(nbio, 0x12BD6C, 0) = 0 [23] 09.42.56.919 00A6D550 TriggerEvents 212 SslState: SSL_ST_INIT // MayFD_Read=0 MayDoRecv=-1 MayFD_Write=-1 MaySslTryToSend=-1 bSslAllSent=0 bAllSent=-1 09.42.56.919 00A6D550 BIO_ctrl_pending(nbio) = 0 [24] 09.42.56.919 00A6D550 BIO_ctrl_get_write_guarantee(nbio) = 4096 [25] 09.42.57.040 00A6D550 TCustomSslWSocket.Do_FD_READ 212 09.42.57.040 00A6D550 BIO_ctrl_get_read_request(nbio) = 7 [26] 09.42.57.040 00A6D550 Winsock recv( 212, 0x12DD6C, 7, 0) = 7 [27] 09.42.57.040 00A6D550 BIO_write(nbio, 0x12DD6C, 7) = 7 [28] 09.42.57.040 00A6D550 BIO_ctrl(nbio, BIO_CTRL_FLUSH, 0, 0x0) = 1 [29] 09.42.57.040 ICB SSL3 alert read fatal unknown 09.42.57.040 ICB SSL_connect: error in SSLv2/v3 read server hello A 09.42.57.040 00A6D550 BIO_read(sslbio, 0x1, 0) = -1 [30] 09.42.57.040 00A6D550
Re: [twsocket] PASV fallback to public IP
I have a nagging feeling that NAT address manipulation may only happen with FTP clients, if it fails then people use passive mode. This issue happens in passive mode. When FTP client sends PASV command it gets a response which contains private IP address... Adding the same feature as FileZilla FTP client is not hard, since the server public IP address is available from the socket. Doing the same on an FTP server is much harder, and really needs a public STUN server (as used for SIP for the same reason). ... so I guess only replacing IP address given by server in response to PASV with the public one (the one used to connect to the FTP server) should do the trick (at least in this case). This does not need to be automatic or fancy, I guess something like a property OverridePASVIP would be OK - it would force ICS to use server IP plus port given in PASV response. Best regards Kristof -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] PASV fallback to public IP
If it is all the NAT to blame, how could NAT devices translate the FTPS PASV responses? SZ On Wed, Sep 8, 2010 at 1:03 PM, Kristof Gajsek kris...@cyberkiko.comwrote: I have a nagging feeling that NAT address manipulation may only happen with FTP clients, if it fails then people use passive mode. This issue happens in passive mode. When FTP client sends PASV command it gets a response which contains private IP address... Adding the same feature as FileZilla FTP client is not hard, since the server public IP address is available from the socket. Doing the same on an FTP server is much harder, and really needs a public STUN server (as used for SIP for the same reason). ... so I guess only replacing IP address given by server in response to PASV with the public one (the one used to connect to the FTP server) should do the trick (at least in this case). This does not need to be automatic or fancy, I guess something like a property OverridePASVIP would be OK - it would force ICS to use server IP plus port given in PASV response. Best regards Kristof -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] PASV fallback to public IP
I have a nagging feeling that NAT address manipulation may only happenwith FTP clients, if it fails then people use passive mode. This issue happens in passive mode. When FTP client sends PASV command it gets a response which contains private IP address... Irrelevant, we are talking about two NAT routers here, the client is almost certainly behind a NAT router using a private IP, and the server is behind a second NAT router. In an ideal world, both routers would be changing the private IP to public IPs, and FTP would just work. Using passive mode gets around the client NAT router, but not the server NAT router. My first example is the ICS FTP client behind a NAT router, accessing an ICS FTP server a public IP. The client sends a port command with a private IP: 00:08:07 Downloading File: /info-2010-09-07.txt 00:08:07 PORT 192,168,1,119,236,41 00:08:07 200 Port command successful. but the server receives the command with the public IP and the same port, because it's been translated by the client NAT router. 00:08:06 angussha1 [217.146.115.81] [288] PORT 217,146,115,81,236,41 00:08:06 angussha1 [217.146.115.81] [288] 200 Port command successful. I'm not using passive mode, because the NAT router is working properly and manipulating the control channel. Note it can not do this with SSL due to encryption which is why passive mode is needed. My second example is accessing the ICS FTP server behind a NAT router, from an ICS FTP client on the public server. Non passive mode works immediately, because there is no NAT. With the client in passive mode , it gets this response from the FTP server behind NAT with the public IP: PASV 227 Entering Passive Mode (217,146,115,84,82,9). but the server actually sent a private IP, which has been modified by the NAT router: 12:46:30 angusadmin [217.146.102.131] [11] PASV 12:46:30 angusadmin [217.146.102.131] [11] 227 Entering Passive Mode (192,168,1,63,82,9). So my original hypothesis that an FTP server behind a proper NAT router will work without needing any special commands or manipulation in the client or server is correct. I'm using a Sonicwall TZ200 router and firewall. However I've not yet tested FTP behind two NAT routers. If anyone wants to test against the latest ICS FTP server either on the public or NAT address, please email and I'll give you logins. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Multiple NICs question
Hello, I have a customer who wishes to use our ICS-based product to cache/proxy between connection in two different NICs with different IPs. The listening part is trivial. He says Winsock cannot automatically route the traffic to the second NIC outbound port when a public IP is destinated. I have no clue why! Is there something I am missing on this? Regards, SZ -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] HTTPcli: source path question
Well, then I have a question: maybe you have some ideas of how to organize recursive download: for example, if user started to download www.example.com/path/index.html, we should also accept www.example.com/path/logo.jpg and so on, but not www.example.com/index.php. If user started www.example.com/path/foo, we should accept www.example.com/path/foo/index.php but NOT www.example.com/path/bar.jpg. Applications like Wget do support this behavior but the question is how they do it. HTTP reply consists of header and document. In header you can find useful info about the type of the document being served. Wget uses this info to determine filename and hint the directory structure. It parses HTML but not in a way that it creates a folder structure. Rather it creates a browsable structure that you can open in your web browser. Basically for each document you receive you have to scan for a href=link links (and possibly also CSS-based links) and internally in your program organize them into folder structure. You also need to look at base link in html header if it exists. To create browsable structure sometimes also a href links in downloaded documents need to be modified as well, to point to different location. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] SSL OnSslVerifyPeer vs. OnSslHandshakeDone event
Kurt,
I'm a bit puzzled about what the exact purpose of the HandshakeDone
event is then.
Is it to verify that the SSL connection is now complete with(out)
errors ?
When it triggers without error the certificate chain verification
completed successfully. In case of option SslVerifyPeer is set it
is the your responsibility to do a PostConnectionCheck.
If the SslSession was reused or SslVerifyPeer isn't set this
check is not required of course.
And why is the certificate sent along as a param in this event too ?
The peer certificate object is required for the PostConnectionCheck,
it also has a property VerifyResult which should be X509_V_OK in case
of ErrCode = 0.
PeerCert.PostConnectionCheck(DNS name);
If PostConnectionCheck failed and you set var Disconnect to TRUE
and the connection will be closed delayed. Do not call Close.
Description of PostConnectionCheck:
{ Now to the PostConnectionCheck, a very important security check!
Our application will be vulnerable if we do not check the peer
certificate beyond verification of the chain. Nothing prevents an
attacker from getting his own certificate signed by one of our trusted
CAs and then hijacking all our sessions. We thward this kind of
masquerade by tying the certificate to some information unique to the
machine. In SSL this information is one or multiple full qualified
domain names (FQDN) also called DNS names stored in certificate's
commonName field(s) of the subjectName field. Since X.509v3 the
subjectAltName extension allows to hold the FQDN as well as other
identifying information such as the IP address.
We use function PostConnectionCheck to perform these checks for us. }
Is it safe to handle just one of these events, and if not what to
check for in each ?
Yes, it's safe to only handle OnSslHandshakeDone.
--
Arno Garrels
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
Re: [twsocket] PASV fallback to public IP
Angus Robertson - Magenta Systems Ltd wrote: Or simply: ?php echo $_SERVER[REMOTE_ADDR]; This still needs be running on a public server somewhere! I don't have PHP on mine. BTW: The NAT trouble will stop with IPv6. And introduce lots of new problems instead. My new Sonicwall pass IPv6, but not process it. No problem with FritzBox, it blocks IPv6 unless I explicitly unblock certain IPs. I currently use a free SixXS tunnel with Heartbeat support (http://www.sixXS.net) and got my own subnet, FritzBox handles this fine as well, no additional software required. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] 535 SSL handshake failed. Error #1
09.42.57.040 00A6D550 212 [32] error:14077447:SSL routines:SSL23_GET_SERVER_HELLO:reason(1095) Error number 1095 seems to mean const SSL_R_KRB5_C_GET_CRED which has been changed from 1095 to 287 in OpenSSL 0.9.8a to 0.9.8b. Dunno the meaning of this error, may have to do with Kerberos. Hi Arno, thank you for your answer. LibEay32.dll is 0.9.8e and is the same on ftp server. From my pc work fine with same dll. Can you give me an idea for investigate? In my lan there is any kerberos's server. best regards daniele barbato -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] 535 SSL handshake failed. Error #1
Hello, Svemu - Reparto Sviluppo wrote: 09.42.57.040 00A6D550 212 [32] error:14077447:SSL routines:SSL23_GET_SERVER_HELLO:reason(1095) Error number 1095 seems to mean const SSL_R_KRB5_C_GET_CRED which has been changed from 1095 to 287 in OpenSSL 0.9.8a to 0.9.8b. Dunno the meaning of this error, may have to do with Kerberos. LibEay32.dll is 0.9.8e and is the same on ftp server. It might be that the application loaded some incompatible OpenSSL libraries unless the full path and filenames are specified. Quote from thread New DLL hijacking vulnerability KB 2269637: The DLL names are globally writable typed constants, set their values before the OpenSSL libraries are loaded. OSSL is dynamically loaded at runtime, that is when the first OpenSSL function is called. In order to enforce a load call TSslContext.InitContext or set TSslDynamicLock/TSslStaticLock.Enabled to TRUE. I prefer this anyway since the load errors don't raise somewhere but where I can handle them easily: try GSSLEAY_DLL_Name := full path and filename; GLIBEAY_DLL_Name := full path and filename; MySslContext.InitContext; // loads the libraries and initializes the SslContext except // Something went wrong, log and handle it. end; From my pc work fine with same dll. I just tested from here with the demo OverbyteIcsSslFtpTst.exe and that works for me as well. Can you give me an idea for investigate? As I understand, your customer uses your application rather than OverbyteIcsSslFtpTst.exe. If so, I would compare all SSL settings of your application with the demo settings. Or you could ask for a reason of error error:14077447:SSL routines:SSL23_GET_SERVER_HELLO:reason(1095) in the OpenSSL mailing list. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] 535 SSL handshake failed. Error #1
From: Arno Garrels arno.garr...@gmx.de It might be that the application loaded some incompatible OpenSSL libraries unless the full path and filenames are specified. try GSSLEAY_DLL_Name := full path and filename; GLIBEAY_DLL_Name := full path and filename; MySslContext.InitContext; // loads the libraries and initializes the SslContext except // Something went wrong, log and handle it. end; ok, tomorrow morning i try this. i'm sure that in the folder of OverbyteIcsSslFtpTst.exe, the dll are 0.9.8e and with ProcessMonitor.exe i see that they are loaded this morning i've see that if i use OverbyteIcsHttpsTst.exe from the customer's pc, ssl work fine. TSSLContext is different between ftp and http? As I understand, your customer uses your application rather than OverbyteIcsSslFtpTst.exe. If so, I would compare all SSL settings of your application with the demo settings. yes, my customer use my application but for this test, i use OverbyteIcsSslFtpTst.exe on the customer's pc. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
