Re: [twsocket] constant redeclared

[Angus Robertson - Magenta Systems Ltd]

> Just to inform that these constants are redeclared.
> I could not determine if this was intentional.
> Is not this a source of error?

Not intentional, I've created hundreds of private keys and certificates
this year and have not seen any problems.  

But I will see which is correct, EVP_MAX_KEY_LENGTH may have changed in
the last 10 years, and fix the duplicates. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Encrypt/decrypt file with RSA

[Angus Robertson - Magenta Systems Ltd]

> It is my first attempt to encrypt and decrypt a short message in
> a file with RSA.

The OverbyteIcsPemTool does this, look on the Extras menu.  

There are various functions for encryption and decryption in
OverbyteIcsSslX509Utils, not currently built as a component, but all
can be tested from OverbyteIcsPemTool.  

However I've never really looked at them, they really need some clean
up. 

Angus
 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Sending data from multiple threads on SSL doesn't work

[Angus Robertson - Magenta Systems Ltd]

> It happens if I'm sending from a thread or even only from the 
> main thread (from a timer), please check this demo app:

I look very briefly at your demo, it uses the non-thread aware socket
server, and creates one single thread to send data to all clients, with
no locking attempts or even a message loop in the thread.  

Your application should be based on TSslWSocketThrdServer which creates
one thread per client, and handles all the threaded messages correctly.
You should not need to create any threads yourself, although I guess
extra threads within a client would not harm provided they do not use
ICS functions. 

As I said before, I've never used TSslWSocketThrdServer myself, and it
does not yet support multi-listen or advanced SSL stuff like IcsHosts
which simplify SSL programming considerable. 

Angus







-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Sending data from multiple threads on SSL doesn't work

[Angus Robertson - Magenta Systems Ltd]

> I didn't change any ICS code, but only my derived class... do you 
> think that by always adding to the send buffer PutDataInSendBuffer) 
> and posting FD_WRITE instead of calling TryToSend could cause any
> issues? 

There are often several ways of doing something, if extensive testing
does not show any problems, then it works.  

But you have still not answered my question about SSL version or the
locking unit.  ICS V8.53 came with two versions of OpenSSL.  It would
be worth testing both 1.0.2 and 1.1.0 to see if the behaviour is
different, since threading support changed completely between them. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Sending data from multiple threads on SSL doesn't work

[Angus Robertson - Magenta Systems Ltd]

> I understand that ICS is thread safe when sending data because 
> adding and removing data to send buffers are locked with critical
> sections..  but maybe OpenSSL does not like when we send data
> from two different threads?

OpenSSL is thread safe, or at least has been in the past, I've tested
an FTP application sending hundreds of SSL sessions each using a
separate thread.  

But OpenSSL has changed a lot in the last two years, there is a ICS
unit OverbyteIcsSslThrdLock which used to provide OpenSSL with thread
locking for 1.0.2 and earlier, but is no longer used for 1.1.0 and
later since OpenSSL does locking internally.  

I've not tested heavy thread use for a while, since it is rarely
necessary with ICS, it just complicates applications and introduces
problems. 

Which OpenSSL and ICS versions are you using?

Angus
 



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Android

[Angus Robertson - Magenta Systems Ltd]

> Means if there would be some sort of AnsiString support say for 
> Linux one could continue to work on it because the amount of work 
> needed would be way less?

Yes, if a new version of Delphi restored base1 AnsiStrings, life would
be much easier, avoiding our own AnsiString units and changes
everywhere to use them.  

OS-X was supported by Arno which is effectively Linux already, however
I don't have any Apple Macs without which the mobile version can not be
built.  

I do have Android but I have too many other things to do than learn
Linux from scratch, at my age!  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Android

[Angus Robertson - Magenta Systems Ltd]

> Now my other topic: what became of the idea to support Android 
> which came up lately? 

Effort on mobile platforms and Linux stalled due the massive changes
needed to replace missing AnsiStrings.  Also little demand.

> The issue is, that since Android 6 Google switched from OpenSSL 
> to its own BoringSSL implementation and Indy still cannot use
> that one.

Indy still uses OpenSSL 1.0.2 for which all support will be
discontinued at the end of 2019.  

ICS supports 1.0.2, 1.1.0 and 1.1.1, the latter is needed for TLSv1.3.


BoringSSL is based on OpenSSL 1.1.0 with similar API exports, but I
believe is simplified removing a lot of legacy OpenSSL stuff, looked at
it a while ago, not closely. 

ICS uses an API exports table to load different exports from different
OpenSSL DLL versions, so adding BoringSSL would be relatively easy,
provided nothing we need is actually missing, and that the behaviour of
the API is mostly the same.  

> Maybe it could be helpful for both sides if forces would be joined 
> on the TLS matter (requires of course implementing the other
> stuff), means if a cooperation between Remy Lebeau from Indy and
> folks from ICS working on Android formed some sort of sub project
> to do some of the work requiredto interface BoringSSL in a joint
> manner.

It would be possible for Indy to replace it's OpenSSL interface unit
with our version, although we rename all exports which would either
mean separate versions for Indy renaming other units to match.  

I could assist in a Windows migration to Indy, but would not get
involved in Android testing, I don't have the time for new development
environments.  

Angus
 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] SSL certificate automatic ordering and installation

[Angus Robertson - Magenta Systems Ltd]

Almost the end of six months of changes, the ICS SSL web and proxy
servers
can now automatically order and install SSL certificates if so
required,
from various suppliers, including free certificates from Let's Encrypt,
and
commercial certificates for DigiCert, Comodo, Thawte and GeoTrust from
CertCentre AG.  For automated ordering, Domain Validation is used which
means the web server must be accessible from the public internet by all
the
host names for which an SSL certificate is being ordered. See
OverbyteIcsSslX509Certs.pas fore more info.

SSL certificate ordering is built into TWSocketServer using IcsHosts,
and
any servers based on these can add ordering with minimal new code.

The OverbyteIcsSslMultiWebServ and OverbyteIcsProxySslServer samples
can
order SSL certificates provided they are accessible from the public
internet and all the INI file settings are updated accordingly.  The
OverbyteIcsX509CertsTst sample includes a built in web server and will
order certificates on it's own or for other web servers.

All the planned code for this release is done, I've run out of time for
documentation until next week, and there is a problem building the FMX
libraries currently only done for 10.2, causing VCL to be dragged in,
can
not find why.  Was not intending to support FMX, but TWSocketServer
supports
it, which dragged in several units not previously supported for FMX.
So please only build for VCL at present.  

Currently you need to download from SVN or the nightly zip to get this
functionality, there will be a final release within two weeks once more
documentation is completed and some extra REST examples added. 

All four of my public web servers installed their own Let's Encrypt
certificates last week, so this release is ready for use.  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Extended Validation (EV) SSL certificate

[Angus Robertson - Magenta Systems Ltd]

(Comodo advert removed)

> Have you thought about supporting it in the future?

ICS has supported SSL and EV SSL for 15 years.  

The release of ICS due next week will automatically buy and install SSL
EV certificates, but they cost from £200-500 per year for a single
domain, and generally only major e-commerce and financial service sites
use them. 

What exactly is ICS missing? 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] HtmlSmtpCli blocking

[Angus Robertson - Magenta Systems Ltd]

> I have an old D5 application which can send html-emails with 
> attachments. When the "SmtpCli.Data" function is called the 
> application is blocked until all the data is sent. The connection 
> is over SSL if that can change something. Is this normal or am I 
> doing something wrong.

It should not be blocking, since the component processes one text line
at a time, through the onGetData event, which is uses to fill a buffer,
sends when full, and triggers for the next buffer full, like most ICS
applications. 

It is CPU intensive, and might not be particularly efficient when
sending massive MIME attachments, but is not blocking as such. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] OpenSSL 1.1.1 support

[Angus Robertson - Magenta Systems Ltd]

Two new zips for Win32 and Win64 versions of OpenSSL 1.1.1 can now be
downloadable from the Wiki at:

http://wiki.overbyte.eu/wiki/index.php/ICS_Download

The DLLs are also included in the ICS distribution SVN and overnight
zip.  A formal release is planned for later this month. 

OpenSSL 1.1.1 is the latest long term support version and adds support
for TLSv1.3 RFC8446 and various new cryptographic private key and hash
digest types. Delphi applications require ICS V8.57 or later. This
version will be supported until 2023.  

Support for OpenSSL 1.1.0 will cease in 12 months, and for 1.0.2 at the
end of 2019.  

More details of OpenSSL 1.1.1 are at:  

https://www.openssl.org/blog/blog/2018/09/11/release111/

ICS V8.57 supports TLSv1.3 for servers and clients, although I still
need to look at supporting some of the new features.  We also support
several of the new cryptographic algorithms like Digest_sha3xx,
Digest_shakexx, PrivKeyEd25519 and PrivKeyRsaPssxx, although not all
are properly tested yet. 

Angus


 


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] HttpCli Request in Sync mode

[Angus Robertson - Magenta Systems Ltd]

> In thread mode, the DoRequestSync of httpcli and sslhttpcli will 
> spend more than 1000ms to wait for the http state chage, i think 
> the reason is the MsgWaitForMultipleObjects dont get QS_ALLINPUT 
> event in thread mode,

I rarely use threaded mode for ICS components, except blocking stuff
like ping, so don't really have experience to make recommendations
about changes, or be able to easily test them. 

However my derived TMagHttp and TMagFTP components override the
DoRequestSync method with a simplified version that removes
MsgWaitForMultipleObjects completely.  

I did this many years ago because it made applications more responsive
at the expense of customers complaining by applications often seemed to
use 90% or more CPU.  

But this was always a fallacy, it means they used lots of idle CPU but
did not stop any other application taking precedence. Now all our PCs
have multiple Cores, it means the application use 50% or 25% of CPU
instead for the few moments while DoRequestSync waits.  

A quick look at MSDN suggests we could use mask QS_ALLEVENTS instead of
QS_ALLINPUT to also handle 'a message sent by another thread or
application is in the queue'.  

And/or we could make dwWaitMs configurable, including allowing zero to
bypass it completely as I effectively do in my derived components. 

Any opinions from those that use threads? 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ICS Sockets for sending receiving files

[Angus Robertson - Magenta Systems Ltd]

> I want to use ICS Sockets to send/receive files but can't see any 
> example. All examples are for Text.After working on developing my 
> application with FTP and getting to production stage I get to 
> know that AntiViruses hate FTP and they just throw my application 
> out. Therefore I am looking at Basic Sockets for sending files.

Most file transfers today are carried out using HTTP protocols, a web
client sending and receiving from a web server, and the ICS components
do that, there are examples. 

If you want something simple, look at the samples OverbyteIcsRecv.dpr
and OverbyteIcsSender.dpr which send a file. 

But you may end up re-inventing FTP if you want to send multiple files.


Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] DnsResult hook for http and https connection

[Angus Robertson - Magenta Systems Ltd]

> Is there a better way to hook the DnsResult for THttpCli and 
> TSslHttpCli?

There is new version in SVN, will get zipped this evening, that adds a
new event, which will allow you to update DnsResult.  

>From the change log:

When a host name is looked-up to get an IP address from a DNS server,
sometimes more than one address is offered, perhaps for alternate load
sharing servers or both IPV4 and IPV6.  Currently ICS clients choose
the first IP address from DnsResultList (in wsocket) as DnsResult.

This change adds an OnSelectDns to the HTTP client so the application
can choose one of the alternate IP addresses, or retry with an
alternate on failure.  A similar event will eventually be added to
other clients.  This may be tested with www.google.co.uk which returns
two IP addresses.

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] OT: New Roadmap

[Angus Robertson - Magenta Systems Ltd]

There is a new roadmap of forthcoming products, the next being RAD
Studio 'Carnival' 10.3 due in late 2018, so ICS will soon start
including references to 10.3. 

https://community.embarcadero.com/article/news/16638-rad-studio-august-2
018-roadmap

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] DnsResult hook for http and https connection

[Angus Robertson - Magenta Systems Ltd]

> Is there a better way to hook the DnsResult for THttpCli and 
> TSslHttpCli?

There should really be an event to allow choice of which DNS result to
use, specifically where several IP addresses are returned.  We
currently use the first and never try any alternates, so may fail if
the first is dead.  

Not quite sure if this should be component or application level, since
it really requires retries, perhaps something that could be added to
the new TSslHttpRest component, rather than changing the lower level
ones. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Efficient bulk dns lookup

[Angus Robertson - Magenta Systems Ltd]

> > So, if I create more than 200 http instances (with TCP sockets 
> > inside) I will  have same limit like with UDP, right ? And no 
> > see any speed difference !

Did you test the performance of DNS over HTTP (DOH)?  

It is becoming more mainstream with an RFC on the way, it's now
available in Firefox for testing.

https://github.com/curl/curl/wiki/DNS-over-HTTPS

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] TLSv1.3 published

[Angus Robertson - Magenta Systems Ltd]

> As I'm a neophyte on TLS evolution, could you make a small 
> summary of what v1.3 brings in?

Briefly, TLSv1.3 is simplified and faster with fewer options, and more
secure (harder to intercept with Wireshark and other tools) than
TLSv1.2.  From Wikipedia:

Major differences from TLS 1.2 include:

- Separating key agreement and authentication algorithms from the
cipher suites
- Removing support for weak and lesser-used named elliptic curves
- Removing support for MD5 and SHA-224 cryptographic hash functions
- Requiring digital signatures even when a previous configuration is
used
- Integrating HKDF and the semi-ephemeral DH proposal
- Replacing resumption with PSK and tickets
- Supporting 1-RTT handshakes and initial support for 0-RTT
- Mandating perfect forward secrecy, by means of using ephemeral keys
during the (EC)DH key agreement
- Dropping support for many insecure or obsolete features including
compression, renegotiation, non-AEAD ciphers, non-PFS key exchange
(among which static RSA and static DH key exchanges), custom DHE groups,
EC point format negotiation, Change Cipher Spec protocol, Hello message
UNIX time, and the length field AD input to AEAD ciphers
- Prohibiting SSL or RC4 negotiation for backwards compatibility
- Integrating use of session hash
- Deprecating use of the record layer version number and freezing the
number for improved backwards compatibility
- Moving some security-related algorithm details from an appendix to
the specification and relegating ClientKeyShare to an appendix
- Addition of the ChaCha20 stream cipher with the Poly1305 message
authentication code
- Addition of the Ed25519 and Ed448 digital signature algorithms
- Addition of the x25519 and x448 key exchange protocols

>From an ICS perspective, most of this is transparent, unless you
specify specific ciphers when you need to add new TLSv1.3 versions (max
six) which ICS servers with IcsHosts do automatically.  

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] TLSv1.3 published

[Angus Robertson - Magenta Systems Ltd]

Hopefully the last OpenSSL beta release of 1.1.1 today, with the final
release in September.   

OpenSSL 1.1.1-pre9 (beta) is available for download as Win32 or Win64,
needs the overnight/SVN ICS V8.57 version to load.  

http://wiki.overbyte.eu/wiki/index.php/ICS_Download

This beta supports TLSv1.3 final RFC8446, although neither the latest
Firefox or Chrome betas I have support it, yet.

I believe ICS now fully supports TLSv1.3, I have web, mail and FTP
clients and servers sending data to each other using TLSv1.3
and the browser demo sample can open a few sites using TLSv1.3 final
RFC8446. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Freeze on TWSocketServer close

[Angus Robertson - Magenta Systems Ltd]

> *Subject:* [twsocket] Freeze on TWSocketServer close
> *From:* Engi 
> *To:* "ICS support mailing," 
> *Date:* Fri, 17 Aug 2018 15:38:15 +0200
> 
> Hi,
> 
> My program (a service) hosts 4 TWSocketServers in main thread.
> Sometimes, when I ask the service to stop, the program freeze 
> after I call TWSocketServer Close function

You are assuming all sockets disconnect instantly and are the freeing
the component, when activity may still be happening.  

You should wait until there are zero clients on the server before
freeing it.  Stop listening first, so new clients can not connect while
you wait, at least one minute or more. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] OpenSSL 1.1.0i and 1.0.2p support

[Angus Robertson - Magenta Systems Ltd]

Four new zips for Win32 and Win64 versions of OpenSSL 1.1.0i and 1.0.2p
can now be downloadable from the Wiki at:

http://wiki.overbyte.eu/wiki/index.php/ICS_Download

I have not added these to SVN since OpenSSL 1.1.1 is due to be released
within the next month and will replace 1.1.0.  

There are two low priority security fixes:

Client DoS due to large DH parameter (CVE-2018-0732)
Cache timing vulnerability in RSA Key Generation (CVE-2018-0737)

And a number of bug fixes, including one relating to checking the dates
in X509 SSL certificates to fix bugs with dates beyond 2050 and with
fractional times.  

Cisco is concerned that this change may cause a problem with
certificates embedded in appliances without accurate time, which is
still being investigated.  Essentially the time standard was clarified
10 years ago, and old kit may still be out whose certificates may now
fail validation.
 
Angus 




-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Efficient bulk dns lookup

[Angus Robertson - Magenta Systems Ltd]

> So, if I create more than 200 http instances (with TCP sockets 
> inside) I will  have same limit like with UDP, right ? And no see
> any speed difference !

TCP is a streaming protocol, you open the socket once and can then send
multiple HTTPS DNS requests without connecting again and starting
another thread.  

So it should be more efficient than UDP.  I believe normal DNS servers
will accept TCP requests as well as UDP, they use TCP for zone
transfers, but not sure if they close the TCP connection after a
response or allow multiple requests. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Efficient bulk dns lookup

[Angus Robertson - Magenta Systems Ltd]

> I think Cloudflare not allow me millions dns lookups for free ))

You can search their terms and conditions, but 1.1.1.1 is free. 
It should not take you long to write code to try DNS over HTTPS and
find out how many lookups you can manage per second.  If it blocks you,
then you are correct.  
 
> >is this a server or workstation?
> Workstation. Windows 10 x64.

Workstation versions of Windows are not designed for this/ 

> I will have same slowdown/limits if I create several hundreds 
> TSmtpClient instances ?

It's not the total number of sockets that is an issue, it's how many
new ones you can open per second.  It's even fewer with SSL, which many
SMTP servers prefer.  
 
> How to avoid it ? Use IOCP ?

Windows Server?  But Microsoft writes IIS as a kernel driver for
performance.  And I simply don't have experience of testing massive
load applications, only light load with a couple of thousand
connections.  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Efficient bulk dns lookup

[Angus Robertson - Magenta Systems Ltd]

> The new HTTPS REST OverbyteIcsHttpRestTst sample allows you to 
> make Json DNS look-up requests to Cloudflare.  

In the sample, set URL: https://cloudflare-dns.com/dns-query

Then three REST parameters in the grid, all raw: 

name=www.bbc.co.uk
type=A
ct=application/dns-json

set GET and URL encoded, and you will get back some Json arrays,
including:

Answer":[{"name": "www.bbc.co.uk.", "type": 5, "TTL": 33, "data":
"www.bbc.net.uk."},{"name": "www.bbc.net.uk.", "type": 1, "TTL": 32,
"data": "212.58.244.71"},{"name": "www.bbc.net.uk.", "type": 1, "TTL":
32, "data": "212.58.249.213"}]}

which you can parse to get your IPs, the API will also work with
Wireformat, which is how UDP DNS works.  

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Efficient bulk dns lookup

[Angus Robertson - Magenta Systems Ltd]

> I need to perform bulk dns lookup (for example get A and MX 
> records for several millions domains).
> 
> How to use TDnsQuery most effective:
> It's possible to reach at least 7000-8000 lookups per second ?
> I use about 35 public DNS servers to perform my queries.

You are probably reaching windows limits here, is this a server or
workstation?  The more expensive Windows server versions are able to
handle more traffic than normal workstations. 

Each async UDP or TCP connection causes windows to start a thread, and
this is an overhead probably causing your slow down after 200 instances.


You may also be reaching the limits of what DNS servers can support,
have you tried the Google and Cloudflare versions, they are scaled for
massive usage.

You might want to experiment with DNS over HTTP, which will open one
SSL connection and then allow repeated requests over HTTP, without
needing a thread each time. Look at Cloudflare 1.1.1.1:

https://developers.cloudflare.com/1.1.1.1/dns-over-https/

The new HTTPS REST OverbyteIcsHttpRestTst sample allows you to make
Json DNS look-up requests to Cloudflare.  

I have a vague plan to create a new HTTP DNS component for ICS allowing
WSocket to use this instead of native Windows stuff, it just needs
TDnsQuery being updated to use REST, but this won't happen for many
months, too many other unfinished projects. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Erro in TService with TWSocketServer component in

[Angus Robertson - Magenta Systems Ltd]

> Could you inform me which ICS and Rad Studio XE version are you 
> using ?

All versions of ICS and all versions of Delphi since 7. 

But you should be using the latest stable version which is V8.53 from:

http://wiki.overbyte.eu/wiki/index.php/ICS_Download

> and Could you give me sample source ?

There is an ICS windows service sample: 
\Samples\delphi\SocketDemos\OverbyteIcsSvcTcp.dpr

Note it's not been tested for many years and is very basic. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] using Twsocket with a few thousand clients

[Angus Robertson - Magenta Systems Ltd]

> Is there any clear demo to run tcp server in threaded manner ? 
> The demos in the source confused me too much is there any updated
> demo to better serv tcp with threading as you suggested ?

Don't worry about a threaded server until you reach a slow down on your
non-threaded version, get the basics working first.  

ICS does have an old demo that uses one thread per client, but not one
that uses one thread per x hundred clients.  And using threads slows
down applications, avoid if possible. 

The simplest 'threaded' version is two copies running on two servers
for load sharing and redundancy.  

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Erro in TService with TWSocketServer component in ICS V8

[Angus Robertson - Magenta Systems Ltd]

> I'm trying to make Service application using  TWSocketServer 
> component in ICS V8 Revised: October 2, 2013 on RAD studio XE3

Why are you using a five year old version?

> I just made simple service program which have just TWsocketServer 
> component only but when  I install it with  "testsvc /INSTALL",
> it show "application cannot be  run properly" and fail to install

I've had ICS service applications built with every version of ICS for
15 years running continually on multiple servers. 

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] using Twsocket with a few thousand clients

[Angus Robertson - Magenta Systems Ltd]

> Its stopped from listening when I start loop through clients and 
> send back data to each one of connected cient
> 
> I have a similar server that created with indy with the same code 
> logic and can run 1200 clients per port .

The ICS server has been tested with over 1,200 simultaneous clients in
a single thread, look back in the mailing list, search for ComGen which
uses the TMagIpLog component I mentioned. 

So the problem is your use of the ICS code, no error handling or
logging, so you have no idea why it is not responding. 

But you never mentioned whether your 40 clients were simultaneous or
sequential.  TWSocketServer is a single thread server.  If you run long
blocking SQL operations (more than 100ms) responding to clients, then
the server is unable to process other clients at that moment and
everything will slow down. But without any logging, you currently have
no idea what is blocking your server, or even how many clients are
connected.   

My ICS web server logs it's SQL activity, so I know it's not taking too
long:

07:55:01 EXEC post_codes_lst_part2 'GU84DN'
07:55:01 SQL proc took 31ms
07:55:01 SQL Search Post Codes: GU84DN - Records Returned 1

If SQL is slow, the solution is run your SQL operation is a client
thread, so the server can keep responding, look at the ICS web server
which uses a thread to compress files and calculate CRCs to avoid
blocking other clients.  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] using Twsocket with a few thousand clients

[Angus Robertson - Magenta Systems Ltd]

>  i have created a tcp server using Twsocket but after 40 clients 
> connected the server stopped from listing

How do you know that, I don't see anything in your code to check if the
server is listening.  

And there is no error handling or logging, so you don't really know
what is happening with your server.  For instance you ignore Error in
ClientConnect and ClientDisconnect.  

If you want to make your life easier, try using my TMagIpLog component
from: 

https://www.magsys.co.uk/delphi/magics.asp

which hides most of the server code from you, and just calls events
when data is available from a new remote client.  There is a demo which
you can run to listen to your existing remote clients and make a single
line response, obviously you need more code to do it properly.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] TLSv1.3 published

[Angus Robertson - Magenta Systems Ltd]

Transport Layer Security (TLS) Protocol Version 1.3 has finally been
published as RFC8446.

https://tools.ietf.org/rfc/rfc8446.txt

OpenSSL will be doing a final beta of 1.1.1 shortly followed by the
final release later this month.  New versions of ICS will be needed for
both of these.  A new planned release of older versions of OpenSSL is
also due next week, but may be delayed so they all come out together.  

Expect to see lots of new application releases with support for TLSv1.3.
While many already support earlier drafts of TLSv1.3, they were all
draft specific and will need a final version to support RFC8446.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] Please ignore

[Angus Robertson - Magenta Systems Ltd]

Just checking the mailing list is still behaving itself, and whether it
can cope with the anti-spam DMARC (Domain-based Message Authentication,
Reporting and Conformance) policies. 

Angus
 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] FW: TSslFtpClient...

[Angus Robertson - Magenta Systems Ltd]

> Needs a space before 'bytes...
> 
> Buffer:= IntToKByte(FByteCount) + ' bytes received/sent in ';

Which would then say '14.5M bytes received/sent' rather than 14.5Mbytes
at present.  

Ideally the function would be more intelligent to allow for small and
large figures.  

But if you don't like the formatting, you can format your messages.  

You are also using a two year old version, while the FTP unit has not
really changed in that time, there have been a vast number of SSL and
other improvements in that time.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] FW: TSslFtpClient...

[Angus Robertson - Magenta Systems Ltd]

> I set up the properties and called SslFtpClient1.Receive;

Which does not allow to control anything or make choices about what
server commands to use, or indeed do more than a single file.  

You should use ConnectFeat, check the FEAT response for capabilities, 
then download your files. 

> >ProtLevel C or P means Clear or Private,
> OK, so if P then the control AND data are encrypted.
> I assume servers support C and P, generally?

The FEAT response will tell you, ie AUTH TLS;TLS-C;SSL;TLS-P;

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] FW: TSslFtpClient...

[Angus Robertson - Magenta Systems Ltd]

> Altered the settings to support FTPS Explicit SSL/TLS, set the
> TSslFtpClient.SslType to sslTypeAuthTls, and getting the file 
> worked.
> 
> Altered the setting TSslFtpClient.SslType to sslTypeAuthSsl, and 
> getting the file failed.

sslTypeAuthTls and sslTypeAuthSsl do the same thing but using different
FTP commands, AUTH TLS or AUTH SSL, depending on what the server
requires, TLS is the modern version, SSL the ancient version.  A server
usually supports only one, as listed in FEAT, TLS is the latest.  You
should have done a FEAT command to find out what the server needs. 

ProtLevel C or P means Clear or Private, and determines whether the
data connections are SSL protected, which is optional.  Protecting the
control channel is important because of logins and file names, but
files are often encrypted already. 

This was all designed 20 years ago when processors were slow and memory
expensive, making encryption also expensive.

Angus




-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Idle Socket is closed even with KeepAlive under windows

[Angus Robertson - Magenta Systems Ltd]

(reply from original poster)

Thanks for taking your time to help me!
>> I working on a device communication protocol using TCP with
>> TWSocketServer and TWSocket. (Delphi XE Project)
>> Small description: This device
> Sorry don't understand what you are doing.
>
> Is this two ICS applications talking together as client and server,
or
> a physical device opening client connections to an ICS server?
It's a communication device ( Mifare reader ) in one side. On the other

side my windows service with TWSocketServer.
This device has 2 channels. The first channel connects and send initial

data that I must respond. If the communication is right then the second

channel kicks in and I must respond too. Only then the "session" is 
opened and I can start sending usefull commands.
The first channel is very active and there's always some command or 
status sent to my server.
The second channel on the other hand goes "idle" as soon as ended the 
initial communication, and can stay like that "for hours" (according to

the vendor), but after 72 - 120 s the channels get closed and the
"close 
request" is originated from "my side". Once the second channel gets 
closed, the device also closes channel 1 and I must restart the 
initialization protocol.
Since the close request is originating from my side, it must be either 
my service or windows.
Talking to the vendor they suggested me to configure SO_KEEPALIVE on my

side. That's what i did, but still the connection closes after the same

amount of time ( 72-120s). In the log I sent, notice that the normal 
communication goes from 12000 (external device port ) -> 3424 (internal

device port) or from 12000 -> 3423. After a little more than a minute 
from ports 59216 ( SO port, each time that one changes ) a FIN is sent.

But it's not me (or TWSocketserver).

>
> ICS does not have any automatic time outs, except in some of the
> application servers, it's always obvious being a timer checking for
> connection durations.  But not in SocketServer itself.
>
> My long experience is sockets will stay open for days.  Never tried
to
> use TCP KeepAlive, but from what you say it is being sent.
As suggested by the device vendor I managed to set ICS so the socket 
gets configured with SO_KEEPALIVE.

20507 1164.435227 192.168.1.76 192.168.1.250 TCP 55 [TCP Keep-Alive] 
12000 _ 3424 [ACK] Seq=6 Ack=17 Win=65536 Len=1
20508 1164.436457 192.168.1.250 192.168.1.76 TCP 66 [TCP Keep-Alive ACK]

3424 _ 12000 [ACK] Seq=17 Ack=7 Win=5840 Len=0 SLE=6 SRE=7

This is happening every 30s, so I guess the configuration I did is 
working. But still the connection is closed, and it's not by my 
application, but it's from my side. So I guess windows is closing it. I

already deactivated my anti-virus and firewall as suggested by the 
vendor support team.But still no joy.

With this particular vendor there's no "KeepAlive" command in their 
protocol, and they said SO_KEEPALIVE is the way to go.
The funny thing is that channel 1 sends from time to time an IsAlive 
command that I must respond. Channel 2 just keeps idle until it gets
closed.
>
> Angus
>
>
>

-- 
*Clément G. Doss*

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Idle Socket is closed even with KeepAlive under windows

[Angus Robertson - Magenta Systems Ltd]

> I working on a device communication protocol using TCP with 
> TWSocketServer and TWSocket. (Delphi XE Project)
> Small description: This device 

Sorry don't understand what you are doing.  

Is this two ICS applications talking together as client and server, or
a physical device opening client connections to an ICS server? 

ICS does not have any automatic time outs, except in some of the
application servers, it's always obvious being a timer checking for
connection durations.  But not in SocketServer itself.  

My long experience is sockets will stay open for days.  Never tried to
use TCP KeepAlive, but from what you say it is being sent.  

Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] IPv6 support for Socks5 and Http proxy

[Angus Robertson - Magenta Systems Ltd]

> I added IPv6 support for TCustomSocksWSocket and 
> TCustomHttpTunnelWSocket

Thanks, your changes have been merged and are now in SVN.

I can not easily test SOCKS and tunnelling, so can you please confirm
these changes work this week, otherwise I'll need to revert to the
older version. You need to test both IPv4 and IPv6 to make sure is it
no worse.   

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Error updating to last revision

[Angus Robertson - Magenta Systems Ltd]

> I'm using RAD Studio 2009 and ICS from repository updated manualy 
> (using subversion).

Thanks, however while the latest ICS is developed and tested using
Delphi 2007 and 10.2 and should be compatible with Delphi 2009, I don't
use C++ nor test with 2009.   

Several users here are using the latest ICS with modern versions of C++
10, 10.1 and 10.2, it is possible no-one is using a 10 year old version,
so we can not support it. 

The errors you reported all come from the OverbyteIcsMsSslUtils unit
which uses OverbyteIcsWinCrypt and OverbyteIcsCryptUiApi, so you can
try removing those from the library package and rebuilding,

Those units are needed by some SSL samples but I don't believe by any
core components.  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Error updating to last revision

[Angus Robertson - Magenta Systems Ltd]

> I'm trying to update ICS component to latest revision and get 
> errors when building design time component (no problems with run 
> time component).
> Restoring my old revision (1 year old) I can build again both 
> components,

You don't say what versions fail or work, the real latest is V8.55
which can be downloaded from: 

http://wiki.overbyte.eu/arch/icsv8w.zip

The errors you shown relate to C++ which is not officially supported,
but there have been several fixes for it in the last year. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] OverbyteICSSmtpProt IFNDEF missing?

[Angus Robertson - Magenta Systems Ltd]

> I'm wondering if there is some minor mistake in the 
> OverbyteSmtpProt.pas file of the ICS library.
> I would expect that here also a check for the NO_DEBUG_LOG define 
> is required.

This is now fixed in SVN, also for POP3 and three more recent units
that are all fault.  Finally built the library with NO_DEBUG_LOG. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] OverbyteICSSmtpProt IFNDEF missing?

[Angus Robertson - Magenta Systems Ltd]

> Shouldn't be there a similar construction later in the file at 
> line 1167 where it now only says:
> 
> property IcsLogger;  { V8.03 }
> 
> I would expect that here also a check for the NO_DEBUG_LOG define 
> is required.

Yes, this should be conditional code, however the SSL samples and all
my own applications use the IcsLogger, so I'm afraid I never test ICS
without SSL or logging.  

I've even just fixed an SSL bug caused by logging being optional.  

Ideally we should get rid of many of these IFDEF/IFNDEF to simplify the
code.  There are really a hang over to 10 to 15 years ago when memory
was still very limited.   

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ICS V8.52 allows testing of SSL TLSv1.3 protocol

[Angus Robertson - Magenta Systems Ltd]

Another planned OpenSSL beta release this week.  

OpenSSL 1.1.1-pre8 (beta) is available for download, needs the
overnight/SVN ICS V8.55 version to load.  

This beta supports TLSv1.3 draft 28, which should be published as
RFC8446 shortly, followed by one more OpenSSL beta, before the final
long term support release of OpenSSL 1.1.1.  

I believe ICS now fully supports TLSv1.3, I have web, mail and FTP
clients and servers sending data to each other using TLSv1.3
and the browser demo sample can open several sites using TLSv1.3. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] New version

[Angus Robertson - Magenta Systems Ltd]

> I read on some website that the next version of ICS will be 
> compatible with Android. Already have some forecast of the next 
> version? 

Sorry, very little effort has gone into an Android version, it's really
needs substantial financial or manpower support from a company that
needs it to justify the effort.  

ICS is mostly a volunteer run project, we simply don't have the spare
time for new projects. 

Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] TLSv1 deprecated

[Angus Robertson - Magenta Systems Ltd]

>From 30 June 2018, TLSv1 is being deprecated by the PCI Data Security
Standard (PCI DSS) for safeguarding online payment data.

Even if your servers do not accept online payments, most security
scanning software will expect this standard.  Note the TLS level is
primarily determined by the server, provided the client supports modern
protocols.  

Windows Internet Explorer 10 and earlier on Vista and XP and Android
4.3 and earlier only support TLSv1, but MSIE 11 on Windows 7 and later
supports TLSv1.2, as does Firefox.  

So ICS web, FTP, mail, proxy and other servers should be configured to
support TLSv1.1 as a minimum.  This can be done using the SslContext,
setting SslMinVersion := sslVerTLS1_1; for ICS V8.27 and later (two
years ago).  If you are on an earlier version, you should really update
to the latest ICS, SSL is improving all the time.  

If the server uses new IcsHosts configuration, setting SslSrvSecurity
to sslSrvSecInter or sslSrvSecInterFS now sets minimum TLSv1.1, and
this is also the default if not otherwise set.  

Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ICS V8.52 allows testing of SSL TLSv1.3 protocol

[Angus Robertson - Magenta Systems Ltd]

I've fixed the ICS bug that stopped ICS servers negotiating TLSv1.3
connections with the last couple of OpenSSL betas, which sent new
callback messages.  Also client session caching should work now for
TLSv1.3.  

SVN is updated, the overnight zip later this evening. You also need
OpenSSL 1.1.1-pre7 (beta) from the wiki page. 

I've successfully tested ICS HTTP and FTP servers and clients talking
TLSv1.3 together, not tried SMTP yet.  

Firefox version 61 now supports the latest TLSv1.3 draft 28 version by
default, and successfully accesses the ICS web server.  It is currently
beta, from https://www.mozilla.org/en-US/firefox/61.0beta/releasenotes/

The final TLSv1.3 RFC is currently with the RFC editors in the
publication queue.  

Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ICS V8.52 allows testing of SSL TLSv1.3 protocol

[Angus Robertson - Magenta Systems Ltd]

Another planned OpenSSL beta release this week.  

OpenSSL 1.1.1-pre7 (beta) is available for download, needs the
overnight/SVN ICS V8.54 version to load.  

This beta support TLSv1.3 draft 28, which is hopefully he final version
and is supported by several test web sites.  

Unfortunately TLSv1.3 in ICS is still partially broken, we are able to
reach several test TLSv1.3 sites but my ICS built web server still
connects with TLSv1.2.  Will investigate properly next week. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] FTP upload speed

[Angus Robertson - Magenta Systems Ltd]

> I have this in my code (I think lifted from your example)

You really don't need any code, the default is no bandwidth control.  

> Not sure what else to do to try and diagnose.

Try a different server, such as mine which is the ICS FTP server.  I'll
email a login. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] FTP upload speed

[Angus Robertson - Magenta Systems Ltd]

> Is there any upload speed throttling built in to the TMagFTP 
> component (or Overbyte one) ?

All ICS components have optional bandwidth limiting, it's part of
WSocket, on the XferDemo3 it's set by the Bandwidth Limit field which
should generally be zero.  

I do know from experience that ICS does not offer the maximum possible
transfer speed under Windows that is available with a blocking file
transfer API (we don't use), but that does handle SSL or reporting
(without using threads).  

I do daily FTP downloads from some of my public servers to make sure
everything is running OK, and this is typical: 

https://www.magsys.co.uk/download/testing/speed50meg.zip
Download OK, size: 48.8 Mbytes, duration 0:11, average speed 4.07M/sec

My public servers are only on 100Mbit switches and through an old slow
firewall, so they go a lot faster, not designed for speed tests...

Uploads are a little slower, but they are limited by my FTTC connection
to 20Mb/s: 

Upload OK: magtelecom.zip, size: 75.8 Mbytes, duration 0:36, average
speed 2.09M/sec

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Angus TLS certificate works

[Angus Robertson - Magenta Systems Ltd]

> Use a public domain and certificate that does not generally 
> have any public facing servers, except for port 80 when you 
> collect the certificate.  

You can also authenticate commercial domain validated certificates by
email so don't need a public server at all, ie use
internal.mydomain.com validated against ad...@mydomain.com. 
 
> The PemTool sample can be used to sign certificates as a CA, but 
> it is quite complex 

This is really something that should be in the new TSslX509Certs
component and sample, it's very little extra code.  I'll add a new tab
to the sample 'Private CA' which will create and sign certificates
locally.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Major new automatic SSL/TLS X509 certificate component

[Angus Robertson - Magenta Systems Ltd]

> *Subject:* Re: [twsocket] Major new automatic SSL/TLS X509 
> I'm a litte affraid to update, I have so many systems now running 
> with ICS and if something breaks.

I do try to avoid changes that are not backward compatible, but do
sometimes get caught out moving utility functions from protocol
specific components into shared units so they can be more easily used
in other units without dragging in unnecessary protocol stuff.  

Thus some sample changes this week, but none took more than 10 seconds
to fix.   

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] Major new automatic SSL/TLS X509 certificate component

[Angus Robertson - Magenta Systems Ltd]

SVN and the overnight zip have been updated with a new TSslX509Certs
component that automatically downloads SSL/TLS X509 certificates from
various suppliers, and a new sample application to test it. This is
still a beta component, with a number of planned improvements that
should be done in June, but is fully functional now for free single
domain certificates from Let's Encrypt and commercial certifications
from CertCentre AG.

The new component uses the recent REST, OAuth2 and Jose units, which
also have minor changes and fixes from practical use. The Let's Encrypt
functions use Json REST and Json Web Signatures signed by private keys,
while the CertCentre AG functions use OAuth2 with unsigned Json REST
requests.  There is a new sample that will order and download
certificates, but currently none of this is integrated into
SocketServer, which will happen in June so the process is automatic in
background. 

  
Overview


SSL/TLS X509 certificates
-

There are effectively three classes of SSL/TLS X509 certificates,
Domain Validated, Organisation Validated and Extended Validated, in
order of cost and benefit, usually with three variations, single domain,
multiple domains (SANs), and wildcard.  Adding multiple domains to a
certificate can ease administration and is cheaper than multiple
certificates, wild card means any subdmains usually for the cost of
about six single domains.

Domain Validated certificate issuance is mostly automated so they are
cheap (or free), using one of three challenge methods: file validation
where the supplier checks for a specific file under the domain, usually
http://domain/.well-known/file, domain validation where a special DNS
record is created that can be accessed by the supplier, and email
validation where an email is sent to a predefined address at the domain,
ie admin@domain, with a supplier link that must be clicked to confirm
receipt and domain ownership.  A fourth method Server Name Indication
(SNI) validation has been used in the past which allows an SSL server
to check a special domain name, but security weakness in some
implementations mean this is not currently used.

File and domain validation challenges can both be automated, file is
easiest using a simple HTTP server, while domain validation is
dependent on being able to access and control the DNS server of which
there are many different products.  Note file validation challenges are
not instant, the supplier may have a queue of challenges waiting to the
tested, but usually happen without a couple of minutes. Applications
need to be aware the wait may be longer.

Organisation and Extended Validated certificates can be ordered online,
but require manual validation that the company or organisation legally
exists and is entitled to use the domain name, which may take several
days or weeks for extended validation if legal evidence is required.
Once approved, the certificate and be downloaded automatically.


TSslX509Certs Overview
--

The TSslX509Certs component automatically downloads SSL/TLS X509
certificates from various suppliers, including free certificates from
Let's Encrypt, and commercial certificates from CertCentre AG and
Servertastic (not done yet).  The component automates the process from
creating a new private key and certificate request, placing the order,
arranging for domain validated certificates to be checked by various
challenge methods, collecting the certificate and intermediate,
creating PEM and PKC12 bundle files with the private key, then copying
the files to the web server ready for automatic installation.

The component supports automated file challenge for Domain Validated
certificates, initially using an external HTTP server to which files
may be copied using UNC shares, but the next release will support a
built-in HTTP server as well.  The application can also use an FTP
server to copy files to an external HTTP server. DNS challenge
currently require the application to update the DNS server, likewise
email challenge needs an external email application.

The component supports the Acme V1 and V2 protocols as implemented by
Let's Encrypt to download free domain validated certificates, beware V1
bears little resemblance to any of the Acme Internet Draft
specifications, V2 is much closer to draft 10 but only implemented
sufficiently for Let's Encrypt, V2 is designed to also handle
commercial certificates which are more complicated to process. Note
that Acme V1 has been superseded by V2, and was only supported because
V2 did not go public until March 2018.

You don't need to register with Let's Encrypt, but it only supplies
domain validated certificates so the domains for which you order
certificates must already be registered and have DNS pointing to a live
HTTP (not HTTPS) server where the component can copy a challenge file
for Let's Encrypt to access to prove domain ownership.  Let's Encrypt
also supports DNS challenges but this is not 

[twsocket] Major new REST and OAuth components

[Angus Robertson - Magenta Systems Ltd]

SVN and the overnight zip have been updated with three new major units
and two new demos for REST, OAuth2, Jose and Json, a fourth new unit
and sample for getting X509 certificates automatically will follow 
soon. Please note these new units should be considered early betas, 
and may change to add more functionality as they are used in real REST
applications, including the new X509 cert component.

OverbyteIcsSslHttpRest.pas
HTTPS REST functions for making REST (REpresentional State 
Transfer) client requests.The SslHttpRest component is a high level 
version of THttpCli that bundles all the extra components for extra 
functionality, including SSL configuration and certificate 
validation with a root bundle, SSL session caching, content 
compression, content code page decoding, persistent cookies, Json 
handling, logging, client SSL certificate. Includes functions for 
Auth2 authentication. OAuth1 is pending (need Twitter account).

Includes new components:

TRestParams - Defines a collection of  REST parameters and allows them
to be saved as URL encoded or Json.

TSslHttpRest - This descends from THttpCli, and publishes all it's
properties and events with additional methods and properties for 
making REST (REpresentional State Transfer) client requests.  Using
TSslHttpRest in new applications requires only a few lines of code and
no event handlers to make SSL REST requests instead of much more code
and many event handlers for THttpCli.

TSimpleWebSrv - This is a simple web server primarily designed for
accepting HTTP requests from REST servers which don't expect real 
pages to be sent, but also for .well-known responses generated by
applications. Note this web server does not support SSL since that
would require certificates.

TRestOAuth - This for handling 0Auth authorization to web apps, by
several means.  Beware OAuth is really a concept with differing
implementations, so that implementation may not always be straight
forward.  OAuth1 and 1A were originally developed for Twitter and use
cryptography, OAuth2 is a simpler and easier to implement version now
widely used by most cloud services without any cryptography (other 
than SSL).


OverbyteIcsSslJose.pas
JOSE - Json Object Signing and Encryption, used for: JWS (Json Web
Signing), JWT (Json Web Tokens), JWK (Json Web Key) and JWE (Json Web
Encryption).  Variously used by OAuth1, ACME and other protocols.
Includes OpenSSL Message Authentication Code functions used for 
signing JOSE structures with secret or private/public keys.


OverbyteIcsSuperObject.pas
An ICS version of Super Object Toolkit for Json, updated for the
latest versions of Delphi, supported by Delphi 7 and later.  Recent
Delphi versions do include Json and ideally this would be used by ICS
where possible, but that will add complexity to ICS.

Sorry, currently only packages for Delphi 2007 and 10.2 Tokyo FMX/VCL
have been updated, there are dozens that need doing.  

Any feedback from developers working with REST applications would be
appreciated.  These components are just my best effort attempt to
specifically access REST apps from two suppliers of SSL certificates,
and there may be other REST functionality that could usefully be added.


Angus
  
  




-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ICS V8.52 allows testing of SSL TLSv1.3 protocol

[Angus Robertson - Magenta Systems Ltd]

Another planned OpenSSL beta release this week.  

OpenSSL 1.1.1-pre6 (beta) is available for download, needs the
overnight/SVN ICS V8.54 version to load.  

This beta support TLSv1.3 draft 26, which is increasingly being
supported by several test web sites.  

Unfortunately something in ICS has got broken with this build, or for
some other reason, and we currently seem unable to access some TLSv1.3
servers, including the ICS built server, yet some others still work.
Still investigating, but may be several days, other more important
things to finish first. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] Testing the list

[Angus Robertson - Magenta Systems Ltd]

The mailing list has been down for a week or more, but should be
working again if this message appears...

Angus


 Original Message 

*Subject:* Re: [twsocket] C++Builder XE7. Just need TPing
*From:* E P 
*To:* twsocket@lists.elists.org
*Date:* Thu, 19 Apr 2018 12:08:08 -0400

Hi everyone!

Since there's no preset C++Builder XE7 package and I only need to try
TPing. I'm trying to install it but it also needs the
OverbyteIcsWndControl.pas and OverbyteIcsWinsock.pas.

In building the OverbyteIcsWndControl.pas file it complains about the
Forms.dcu 
[DCC Fatal Error] OverbyteIcsWndControl.pas(149): F1026 File not found:
'C:\My Documents\Embarcadero\Studio\Projects\Forms.dcu'

Seems similar to the problem from an older thread:
http://lists.elists.org/pipermail/twsocket/2017-November/047308.html

Any suggetions on how to proceed?

Thanks for your help and your time.
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ICS V8.52 allows testing of SSL TLSv1.3 protocol

[Angus Robertson - Magenta Systems Ltd]

Another planned OpenSSL beta release yesterday.  

OpenSSL 1.1.1-pre5 (beta) is available for download, needs the
overnight/SVN ICS V8.54 version to load.  

This beta support TLSv1.3 draft 26, which is increasingly being
supported by several test web sites.  

Building the latest Browser sample with this build will now open:

https://tls13.crypto.mozilla.org/

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] ICS V8.53 is now latest stable release

[Angus Robertson - Magenta Systems Ltd]

ICS V8.53 has replaced V8.50 as the latest stable release. and can be
downloaded from:

http://wiki.overbyte.eu/wiki/index.php/ICS_Download

There are a lot of changes in readiness for OpenSSL 1.1.1 due to be
released in May, this version only supports the latest beta version and
another ICS release will be needed for the final release.  

This version adds C++ packages for Delphi 10.2 Tokyo (but there are not
yet any for XE4 to 10.1). 

There are some new SSL root certificates, and a new much larger bundle
file RootCABundle.pem with about 280 root certificates. 

The HTTP client has a new ExtraHeaders property to simplify adding
extra headers to a request (previously done using onBeforeHeaderSend
event).

Improved display of SSL certificates and SSL handshake results.  

Updated the bundled OpenSSL files with the latest security fixes.
Long term stable to OpenSSL version 1.0.2o. Latest release to OpenSSL
version 1.1.0h.

Various minor fixes. 

The FrameBrowserIcs Demo was brought up to date with the latest
components.  FrameBrowserIcs is an example internet browser using the
HtmlViewer component from https://github.com/BerndGabriel/HtmlViewer
(11.8) which must
be installed before this sample can be opened.  Beware HtmlViewer 11.8
only appears to support up to Delphi 10 Seattle and would not build in
10.2 Tokyo.

An executable version may be downloaded at: http://wiki.overbyte.
eu/arch/FrameBrowserIcs.zip

Angus





-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Trying to install latest overnight into Tokyo 10.2.3 Rad Studio

[Angus Robertson - Magenta Systems Ltd]

> Hi folks, having a bit of trouble installing the overnight into 
> 10.2.3 Tokyo. I can build all the runtimes (VCL and Fmx) without 
> incident per the instructions, but when I attempt to install any 
> of the design time components, I get an error saying it can't 
> load the package, "The specified module could not be found".

I installed the latest ICS into 10.2.3 Tokyo OK, there were no changes
needed over earlier 10.2s.  I just use the install project, and the
Build Groups pane, click Build Current Group and it compiles all
versions and both Win platforms.  Then have to install the packages
manually.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ICS V8.52 allows testing of SSL TLSv1.3 protocol

[Angus Robertson - Magenta Systems Ltd]

Another planned OpenSSL beta release yesterday.  

OpenSSL 1.1.1-pre4 (beta) is available for download, needs the
overnight/SVN ICS V8.53 version to load.  

This beta support TLSv1.3 draft 26.

TLSv1.3 is currently at draft 28 and has been submitted for final
publication, but there are no protocol changes since draft 26. 

The final version of 1.1.1 is currently planned for 15th May 2018 after
two more betas, provided the RFC editors have finished their stuff and
TLSv1.3 is formally published.  

Beware, at the time of writing no browsers are yet compatible with
TLSv1.3 draft 26 or 28, they still connect with TLSv1.2.  

ICS now supports client and server connections with TLSv1.3, I've fixed
a false renegotiation issue that stopped clients connecting, there is
no renegotiation with TLSv1.3.  I've temporarily stopped external
session caching which has changed for TLSv1.3, this will be fixed soon.


Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ICS V8.52 allows testing of SSL TLSv1.3 protocol

[Angus Robertson - Magenta Systems Ltd]

Another planned OpenSSL beta release today, which is now feature locked
so no more DLL exports arriving or disappearing.  

OpenSSL 1.1.1-pre3 (beta) is available for download, needs the
overnight/SVN ICS V8.53 version to load.

This beta support TLSv1.3 draft 26.

TLSv1.3 is currently at draft 28 and has been submitted for final
publication, but there are no protocol changes since draft 26. 

Beware, at the time of writing no browsers are yet compatible with
TLSv1.3 draft 26, they still connect with TLSv1.2.  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] SSL trusted root certificate bundles

[Angus Robertson - Magenta Systems Ltd]

The whole SSL certificate verification process depends upon finding a
trusted root certificate that signed the next certificate up the chain,
and so on.  But who chooses those trusted roots?  

The answer is the author of the application that accepts an SSL
certificate, or maybe the operating system the application is running
on, or the SSL library used to build the application. 

The bad news is that root certificates come and go, old roots stop
being accepted for various reasons, and new roots are introduced for
new security standards or new businesses.  

Historically most roots were RSA Sha1 digest signed and most still are,
despite Sha1 being banned for new certificates.  Increasingly Sha256
and ECC root certificates are being introduced and required by new
intermediate certificates.

So SSL trusted root certificate bundles and always changing, annually
perhaps for major changes, although Microsoft officially Windows roots
every two months.  

So there is a maintenance issue with root bundles, choosing what to
include and keeping them updated. Another developer got frustrated with
updating roots, and created a Git repository: 

https://github.com/nabla-c0d3/trust_stores_observatory

which contains a few hundred root certificates and lists of which trust
store contain which roots, currently listing Apple iOS and macOS (173),
Google (135), Microsoft Windows (294) and Mozilla (133). 

For the last few years, I have been manually updating three trust
stores for ICS:

1 - TrustedCABundle.pem is a small root certificate bundle, currently
52 roots, included in the Samples/Delphi/SslInternet/ directory.
Applications needing to check root certificates can copy this file. 

2 - A smaller list of 34 hard coded roots in
OverbyteIcsSslX509Utils.pas that allows major roots to be checked
without needing to include an extra external file in applications. 

3 - RootCaCertsBundle.pem which I distribute with my own applications,
and is a much larger root certificate bundle, about 280 roots including
many country specific roots, created using the PemTool by extracting
Windows root certificates.   I've added this to the ICS V8.53 for those
that need more roots. 

I'm now looking at a better way of maintaining the ICS root stores,
probably by using Trust Stores Observatory data, although it's
difficult to choose which root store to use, all contain more than our
smallest store.  

There have been changes to all three of these root store this year,
adding new root certificates.  ICS V8.52 added DigiCert Global Root G2
and G3 root certificates. DigiCert now owns Symantec, Thawte, RapidSSL,
Geotrust and existing certificates are being re-issued signed by
DigiCert roots, so older root stores will stop trusted them.  ICS V8.53
last week added GlobalSign Root CA - R2 and GlobalSign ECC Root CA - R5
root certificates, the R2 root is old but was missing from our smaller
stores and is now used by Google. 

So updating your root stores is important, if you don't want your
applications to stop trusting lots of common web sites. 

Angus








 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Connecting to a webserver via https...

[Angus Robertson - Magenta Systems Ltd]

> try to insert Sleep after SslHttpCli.Get:
> while not terminated do
>   begin
> SslHttpCli.URL := ...
> SslHttpCli.Get;
> Sleep(0/10/50/100/...);
>   end;

Loops are not necessary, ICS is event driven, when one page is received
(OnRequestDone event) you use that to trigger the next GET, with a
windows message.  

Sleep just slows everything down.  

ICS CPU usage will be in DoRequestSync which is called after every sync
request looping the message loop.  But the CPU usage often shown by
tools is not real, it does not slow down other applications.  And ICS
will normally only be using a single thread, and most CPUs now have
four CPUs so never more than 25%.   

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Connecting to a webserver via https...

[Angus Robertson - Magenta Systems Ltd]

> If you running in a thread you  have to write a message pump. Now 
> the only thing you do is executing a Get. But this has no 
> difference in http or https.

> > I used Indy-Components for a long time, i stucked when i tried 
> > to access an AXIS-Camera with a very high resolution. 

Indeed, Indy is blocking so you need to use threads all over the place,
whereas ICS is event driven and will handle several hundred
simultaneous sessions in a single thread. At least provided those
connections are not doing other blocking things like writing to a
database. 

The camera says you have an authentication problem, so I'd look at the
code in SslHttpCliBeforeAuth and see how it differs from the non-SSL
version.  

You can also the onCommand event to log what headers are actually being
sent to the camera, I don't understand what you trying to do in
BeforeAuth and it might not be producing the result you expect. 

The latest OverbyteIcsHttpProt in SVN and the overnight zip has a new
HTTP property, ExtraHeaders,to simplify adding extra headers to
a request (previously done using onBeforeHeaderSend event), and that is
specifically for strange authentication purposes like OAuth2. 

Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ICS V8.52 allows testing of SSL TLSv1.3 protocol

[Angus Robertson - Magenta Systems Ltd]

OpenSSL 1.1.1-pre2 (alpha) is now available for download, needs the
overnight/SVN ICS version to load.

Still seems to have TLSv1.3 draft 23 according to the literals,
although I thought they had updated the master to draft 24. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] unable to link design time CB2007 as missing external file 'crypt32'

[Angus Robertson - Magenta Systems Ltd]

> These appear to external references to crypt32, yet I don't have 
> that file (I believe).

OverbyteIcsMsSslUtils is used to verify SSL certificates using the
Windows certificate store, if you don't need that functionality you can
remove it.  Most SSL samples don't use it.  

But if you do use it, it needs several other units: 

OverbyteIcsWinCrypt.pas, OverbyteIcsJwaWinCrypt.inc, and
OverbyteIcsCryptuiApi.pas, there were Jedi files that Arno copied for
our use, and I renamed to ICS format to avoid conflicts. These units
import from Cryptui.dll, but not crypt32 to my knowledge. 

None of this stuff has changed in a few years, except
OverbyteIcsCryptuiApi.pas which says: 

5 Oct 2017 - only compile for Windows   

and that V8.50 or later, so maybe look there. 

Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Bug report. TSslSmptCli. Memory corruption on second connection after the first connection is aborted.

[Angus Robertson - Magenta Systems Ltd]

> After some investigation I've found the breakage commit 
> So, I propose this fix.

I use the latest SMPT and POP3 components every few minutes and am not
aware of any current issues with their proper use.   

But I've printed your email, and if I can reproduce your problem I will
look at your fix, sorry it may be some weeks, very busy.

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Using the SslFTPClient

[Angus Robertson - Magenta Systems Ltd]

> In response to the FEAT request, some servers just return "PROT" 
> and the current test for PROT P will always fail.
> This is the minor change I have made to the FTP Client. It would 
> be nice if this could be added so I don't have to modify the 
> source after each update:

I'll put it on my list, but may not be quick, very busy.  The FTP stuff
has not changed for two years except for a new Delphi release. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Building OpenSSL

[Angus Robertson - Magenta Systems Ltd]

> Also any chance that wiki.overbyte.eu will be changed to work 
> over https? I'm kind of wary of downloading OpenSSL binaries over 
> unencrypted connection.

The Overbyte web sites do need to support SSL in the next few months
otherwise Chrome will refuse to open them. 

But there is no real security issue.  Our OpenSSL binaries are all
digitally code signed so you can check they are unaltered, as can your
own ICS applications - which is where the real security check is needed
in case they are substituted or hacked.  

Rui may be able to answer your other questions, he builds the DLLs, or
you ask in the openssl-users mailing list where the OpenSSL developers
answer such questions regularly.  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Using the SslFTPClient

[Angus Robertson - Magenta Systems Ltd]

> The only reason I mentioned it was, as you may remember, my 
> mission is to connect to a Linux FTP server using AUTH TLS. 
> Everything is fine except if I select "Passive" mode and 
> PROT=Private, I cannot login. I get SSL handshake errors. They 
> suggested that I should use BLOCK mode in this instance. 

My reading suggests virtually no FTP clients or servers support MODE B.
The header is only three bytes and no file name, so no idea how
multiple files would be sent. 

It certainly will have no effect on SSL connections. 

You can test SSL against ftp.icstest.org, which is my ICS FTP server
running the latest stuff.  I can give you can account for uploads if
you need it.  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Using the SslFTPClient

[Angus Robertson - Magenta Systems Ltd]

> I see that the component supports MODE S and MODE Z. Is there any 
> way it can, or will in the future, support MODE B (Block)?

Block mode goes back to the RFC959 written over 40 years ago, before
the internet really existed and FTP was used by mainframe computers.  

I've never really come across it until you mentioned it, but a little
Googling suggests IIS FTP7 does actually support it, and it may be
beneficial for lots of small files avoiding lots of separate data
connections (particularly with SSL connection overhead). 

So perhaps it is a good idea to add to ICS, not easy and I certainly
won't have time for several months, too busy adding Jose stuff like JWK,
JWS, JWT, ACME and OAuth2, mostly for Let's Encrypt SSL certificates,
then OpenSSL 1.1.1 and TLS/1.3, never mind my own projects.  

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Using the SslFTPClient

[Angus Robertson - Magenta Systems Ltd]

> I have set Active Mode and the Data Port Range to 1..10010
> I connect to the Demo Server and click on the "Get" button.
> I have a breakpoint where the Data Port number is allocated so I 
> can see it increment through the above range.
> When it gets to 10010 it loops back to 1.
> The next click on the "Get" button fails with the error that this 
> port (1) is in use.

This is standard windows behaviour, established ports do not close
instantly you close a connection, they go into time_wait state for
about 60 seconds in case packets are still floating around.  

So you probably clicked past your range of 10 in one minute or so.
Just make the range 100 or 1,000. 

Finding some users ports (1,024 up) in use is inevitable because so
many applications use them for internal purposes, so you need to always
retry failed connection, repeatedly.  Also better to choose an obscure
range that others are less likely to be using than around 10,000.

Which is exactly why many people use my TMagFtp component, all this
stuff is long known and handled.  
 
Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Active, Passive and Data Port Range

[Angus Robertson - Magenta Systems Ltd]

> Am I correct in assuming the following:
> 
> In Active mode, the Data Port Range is set by the FTP client (if 
> required)
> 
> In Passive Mode, the Data Port Range is set by the FTP server (if 
> required)

The ICS FTP server has no concept of restricting Data Port Range, no
idea if other products do.  

The ICS FTP client does allow a port restriction, but I don't believe
Passive mode makes any difference, since the client either opens the
data channel or tells the server to open it on a specific port.  

But it's 10 years since I really played with FTP, and I'm hazy on this
stuff. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Using the SslFTPClient

[Angus Robertson - Magenta Systems Ltd]

> It would appear that this is a Windows Firewall issue after all. 
> I am using Windows 10 and if I turn off the Private AND Public 
> Firewalls, the problem of "peer reset" after a certain number of 
> transfers does not occur.

Is Windows Firewall aware of your application?  

You can add it to the exception list so the firewall ignores it.  Or
open the ports you needs, which is harder for FTP due to the data
channel. 

https://msdn.microsoft.com/en-us/library/aa366421.aspx

I wrote a MagFirewall component a couple of years ago but have not yet
used it in it's intended application, so not sure how much I ever
tested it.  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Using the SslFTPClient

[Angus Robertson - Magenta Systems Ltd]

> Any clues as to whether or not there is something wrong with the 
> ICS components or is it my setup?

Sorry no, I download hundreds of files a day from my servers using
secure FTP, totalling hundreds of megs, and problems are very rare. 

I suggest you try and use FTP with a single component, instead of two.


Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] LocalIPList on Mac

[Angus Robertson - Magenta Systems Ltd]

> I attached a spreadsheet with the results. 

The mailing list does not widely distribute massive attachments, it's
low volume text only, please email it directly. 

I can fix the Windows issues, but you'll need to look at Mac stuff. 

Angus



 Original Message 

*Subject:* Re: [twsocket] LocalIPList on Mac
*From:* "Bill Florac" <bill.flo...@billflorac.com>
*To:* "'ICS support mailing'" <twsocket@lists.elists.org>
*Date:* Sun, 21 Jan 2018 00:18:01 -0600

Angus,

I finally got around to test this. I attached a spreadsheet with the
results. I ran LocalIPList() with each socket family option. I also
showed
the results of IcsGetInterfaceList(). 

PC is running windows 10, Mac is running High Sierra

LocalIPList:
MAC
- Missing second IPv4 address if called with no parameters
- Returns duplicate entries

PC
- Missing second IPv6 address 

BOTH
Returns IPv6 link-local address but not IPv4. 

IcsGetInterfaceList:
PC
- Missing IPv6 addresses


So if you don't care about IPv6, link-local and filter any duplicates,
it
seems to work!

Bill


-Original Message-
From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of
Angus
Robertson - Magenta Systems Ltd
Sent: Wednesday, January 3, 2018 6:32 AM
To: twsocket@lists.elists.org
Subject: Re: [twsocket] LocalIPList on Mac

> 1) It seems that IcsGetInterfaceList() would fail if, for some odd 
> reason, the computer only had IPv6 interfaces.

This would only be on Windows not MacOS, and I suspect it's impossible
to
install Windows without IPv4, too much stuff is dependent on it.  

> 2) This is my function to replace LocalIpList(). 

Not sure we need to replace it, what is the advantage of using
IcsGetInterfaceList over the existing code (but see below)? 

> 3) As to the original LocalIPList() function, it seems that the code 
> at the top where it checks for ASocketFamily is not needed code in
the 
> "else" section handles IPv4 just fine

The 'else' stuff only works on Windows Vista and later, but not on
Windows
XP which still needs GetHostByName.  While we don't officially support
Windows XP or 2000, we also don't want to deliberately break ICS from
working on them.  

But since GetHostByName is deprecated since Vista, I've changed the
code so
it's now only used for Windows XP and earlier, and never for MacOS,
which
should resolve your original problem.  Or are you saying that
GetAddrInfo
only returns a single IP on MacOS as well?  

Testing on Windows 10, I see multiple IPv4 and IPv6 addresses using
both LocalIpList and IcsGetInterfaceList, and no duplicates.   

Angus

--
To unsubscribe or change your settings for TWSocket mailing list please
goto
http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Using the SslFTPClient

[Angus Robertson - Magenta Systems Ltd]

> I have 2 ftp components and both are logged into the server. The 
> first just does a LIST command every 3 seconds. The second does 
> the GET commands.
> After 198 GETs I receive and error 10054. I re-connect both ftp 
> clients and after a further 198 GETs the same thing happens again.

So you have continuous open control channels, and do 198 SSL gets, each
will open and close a new data channel, with a new SSL negotiation and
session each time.  

There are no known restrictions in ICS over number of SSL sessions that
can be opened, I've done thousands in a single thread.  

However ICS does have various SSL session caching options, and
depending on which you are using that might be the issue.  Never really
looked closely at session caching, it just seems to work. 

So most likely the FTP server SSL implementation is bad.  Have you
tried downloading 250 files using FileZilla and SSL?  Should replicate
what your application is doing. 

I would say long idle FTP sessions are not a good idea, servers like to
timeout long sessions and are not usually tested for long idle periods.
So having your application periodically log-off and on again may help
long term, you'll need that anyway to cope with bad internet
connections. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] LocalIPList on Mac

[Angus Robertson - Magenta Systems Ltd]

> 1) It seems that IcsGetInterfaceList() would fail if, for some 
> odd reason, the computer only had IPv6 interfaces. 

This would only be on Windows not MacOS, and I suspect it's impossible
to install Windows without IPv4, too much stuff is dependent on it.  

> 2) This is my function to replace LocalIpList(). 

Not sure we need to replace it, what is the advantage of using
IcsGetInterfaceList over the existing code (but see below)? 

> 3) As to the original LocalIPList() function, it seems that 
> the code at the top where it checks for ASocketFamily is not
> needed code in the "else" section handles IPv4 just fine

The 'else' stuff only works on Windows Vista and later, but not on
Windows XP which still needs GetHostByName.  While we don't officially
support Windows XP or 2000, we also don't want to deliberately break
ICS from working on them.  

But since GetHostByName is deprecated since Vista, I've changed the
code so it's now only used for Windows XP and earlier, and never for
MacOS, which should resolve your original problem.  Or are you saying
that GetAddrInfo only returns a single IP on MacOS as well?  

Testing on Windows 10, I see multiple IPv4 and IPv6 addresses using
both LocalIpList and IcsGetInterfaceList, and no duplicates.   

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Using the SslFTPClient

[Angus Robertson - Magenta Systems Ltd]

> Hold fire on what I said before. I am still in learning mode as 
> this is all new to me.

I would suggest you at least try the compiled version of my high level
TMagFtp component from: 

https://www.magsys.co.uk/delphi/magxfer.asp

which handles all this stuff automatically.  If TMagFtp works, then you
can look at the source to see why.  

> The one thing I have found is that when I execute the featAsync, 
> I do not see the PROT in the list of supported extensions. When I 
> look at the source code, you are looking for "PROT C" or "PROT 
> P". This server only returns "PROT".

This may be irrelevant, the feature list is designed to tell you which
commands you can use, they don't stop you using them anyway (probably,
not looked at the code in a while).  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Using the SslFTPClient

[Angus Robertson - Magenta Systems Ltd]

> I use the 01cert.pem etc. as example files

SSL client applications don't need certificate or private key files,
unless the server is high security and requires secure identification
of remote users, like access to corporate LANs, money transfer services.
You only need a root CA file bundle, and ICS now bundles one in wsocket
as constants.  

> when I look at the data being transferred with Wireshark, I see 
> that everything is encrypted except for the response to the 
> DirAsync command.

Should be easy to answer, but actually not.  Each data connection uses
a separate TCP connection that requires a new SSL handshake, which is
quite expensive in packets.  

So data encryption is optional, defined by the PROT command, C means
clear, P means private which in modern terms means SSL.  Clear might be
acceptable if the files are already encrypted and the overhead of extra
encryption is not required.  

Remember this stuff was devised back in 1997 when computing power and
memory where expensive, and no-one envisaged how it might be
implemented in the future.  

Assuming you have set protection to P, the ICS client should be opening
an SSL connection for directory related commands, there are several.  I
have a log that shows 

04:05:00  > XDMLSD -R
004:05:00  ! SSL Connected OK with TLSv1.2, cipher
ECDHE-RSA-AES256-GCM-SHA384, key auth RSA, key exchange ECDH,
encryption AESGCM(256), message auth AEAD

So it's working with that particular directory command (only supported
by the ICS FTP server).  But I can no say it works will all versions,
some of which use the control channel instead (which should be
encrypted anyway).  

Assuming you are logging the SSL handshake event and FTP commands, your
application should tell you want is encrypted and how.  

This is all down the client, not the server. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] LocalIPList on Mac

[Angus Robertson - Magenta Systems Ltd]

> It seems that LocalIPList on a Mac (POSIX) returns only a single 
> IP address
> Note that it seems that gethostbyname() is also a documented as 
> depreciated function.
> I switched and used IcsGetInterfaceList() which seems to work. I 
> can toss out the AF_INET6 records and the local loop back. Note
> the local loop back is only return on the Mac, not windows.

The low level code in wsocket is quite convoluted and hard to follow,
due to support for multiple OSs and platforms.  

IcsGetInterfaceList does not currently appear to be used by any ICS
components or samples, but I assume was well tested when originally
written, and it was an oversight to not update wsocket to use it. 

If you let me have your modified unit, I'll update it in SVN.  But
please do test it carefully, to make sure it works as expected. 

Angus
 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] What's wrong in my code with SSL on some web site?

[Angus Robertson - Magenta Systems Ltd]

> In C++Builder 6.0 you can use (u)int64_t in your code

This was 12 years ago now, but I think the main issue was 64-bit
streams which did not work until Delphi 7, and I assume that is after
CB6.  But I never worked on C++. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] What's wrong in my code with SSL on some web site?

[Angus Robertson - Magenta Systems Ltd]

> I suppose that version 8 doesn't works with the old C++ Builder 
> 6.0 (I can't change this, for now), so I have no choice.

I think you are stuck with ICS v5.  ICS v6 in 2012 ceased support for
CB6, with CB2006 the oldest, which we still support with v8.  

New versions have new language features which are essential, like int64.


Many people would prefer us only to support modern unicode stuff with
generics, but I still use Delphi 2007 for all my own applications so
that will continue to be supported while I'm around. 

Angus

 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] What's wrong in my code with SSL on some web site?

[Angus Robertson - Magenta Systems Ltd]

> I'm trying to post some data to some of my https web sites (for 
> sample https://www.millesimo.com/index.php) but I always get the 
> error "SSL handshake failed". My code works well with other sites 
> (https://www.microsoft.com/it-it, for sample).
> 
> ICS: r20 | francois | 2009-09-10 07:50:47 +0100 (Thu, 10 Sep 2009)
> ssleay32.dll/libeay32.dll 0.9.8e

OpenSSL 0.9.8 has been obsolete for several years, likewise the version
of ICS you are using is ancient. 

SSL protocols change with time, old ones are removed due to security
risks and newer ones added in newer versions.  Web sites likewise stop
supporting ancient risky protocols.  

So you need to update to ICS v8, and at least OpenSSL 1.0.2, ideally
1.1.0 which is current.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] I like to send emails with ssl/01cert.pem but certificate too weak according gmail, any solution?

[Angus Robertson - Magenta Systems Ltd]

> What happens if this value is left blank as well -- does it skip 
> checking validity of CA? Is there any way to have the components 
> instead use the CA roots that Windows maintains in the 
> Certificate Store?

Yes, the sample OverbyteIcsMsVerify.dpr will Verify a certificate chain
using the class TMsCertChainEngine which uses MS crypto API and the
Microsoft root store.   You need to add extra code to the
onSslHandshakeDone event to ignore the OpenSSL result and call the
engine instead.  

All my own client application and ICS components have options for both,
look at TMagIpLog at: 

https://www.magsys.co.uk/delphi/magics.asp

which also shows better ways of displaying certificate information from
newer ICS versions. 

> My concern is that installing a 
> TrustedCABundle.pem file along with an application would lead to 
> problems with it going stale.

Root certificates mostly have a very long life and major new ones are
quite rare, although some do go out of favour, like Startcom currently
which is closing down.  

But there are hundreds of root certificates, many small countries want
to issue their own, and out bundle does not include many of those.
Windows should automatically download missing roots from Windows Update
during validation, but this may be slow.  

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] I like to send emails with ssl/01cert.pem but certificate too weak according gmail, any solution?

[Angus Robertson - Magenta Systems Ltd]

> I_m trying to send email via gmail.com. Which perfectly worked 
> since a while ago.
> but now it seems gmail (I think this is the reason) is rejecting 
> the 01cert.pem certificate Because the encryption is too weak.

Client SSL applications rarely need to send certificates, so just leave
SslCertFile and SslPrivKeyFile blank and everything should work.  

The confusion here is that the SslContext is used for both client and
server applications, and it's not obvious which properties relate only
to servers, clients, or both.  Worse, most of the samples just offer
all the options without explaining which are needed or why. 

Some background. 

SSL certificates have two purposes, identifying a host and encrypting
data.  

For SSL to work, a certificate and private key are needed to encrypt
the data, the certificate is sent to the client and includes the public
key so the client can negotiate encryption keys and can decode the data.


The only time a client needs a certificate is if it needs to identify
itself to a server, mostly when using VPN to access remote networks,
sometimes for servers with highly valuable information (like SWIFT
international money transfer).  It is the server that demands a client
certificate, before allowing data to be sent.  

In your case, newer versions of OpenSSL have higher minimum
requirements for certificates, and 01cert.pem is probably very old.
Note current ICS release have one dated 2016 which should not give a
problem, but is still only required for server samples. 

Clients do still need Certificate Authority roots to be able to check
the server is sending a valid certificate, so leave: 

SslCAFile := GlobalUserSettings.fCertDir + '\TrustedCABundle.pem';


Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] THttpCli and Socks Proxy

[Angus Robertson - Magenta Systems Ltd]

> Btw, isn't this line
> > FCtrlSocket.SslEnable := ((FProxy = '') and (FProtocol = 
> > 'https'));
> silently drops security transfer? 

A little googling suggests the SOCKS protocols do not officially
support SSL, although some implementations like Wingate will accept an
SSL/TLS connection before the SOCKS protocol is negotiated.  

But that would not be end to end SSL, since the proxy would be making a
non-HTTP connection.  

There are a couple of SSL SOCKS projects on Github, but not looked to
see what they offer.  

My current feeling is we should fix SOCKS if it has got broken by new
ICS releases, but any SOCKS improvements will need to be commercially
organised. 

Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] THttpCli and Socks Proxy

[Angus Robertson - Magenta Systems Ltd]

> So today I have dived into OverByteIcsHttpProt.pas and made a few 
> small hacks as under to bypass the DNS lookup:

Thanks, I will look at this, but not this month. Stuff like this is
easy, it only got broken because SOCKS got forgotten.  

> For me, a cleaner way would be to get the IP address from the 
> proxy server relating to the existing WSocket connection, but I 
> don't know how to go about it. 

ICS does have a DNS lookup component that will request full domain
details from any DNS server, like the MX records to send email, look at
the sample OverbyteIcsNsLookup.dpr. 

But this assumes the SOCKS proxy offers a DNS server, which is easy to
test with the sample.  Since passing a host name is part of the SOCKS
protocol, they might not want you to know the final IP address.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] THttpCli and Socks Proxy

[Angus Robertson - Magenta Systems Ltd]

> I was quickly able to implement the telnet part using a TWSocket 
> component. This works successfully through the Socks proxy and I 
> am very happy with it. A particular point to note is that DNS 
> resolution is done by the proxy machine, which is a requirement 
> of the project.
> 
> For the http part I am looking at using a THttpCli component. I 
> am experimenting using the OverbyteIcsHttpGet demo, setting up 
> the Socks properties on HttpCli1. It works if I provide the 
> actual IP address of the remote server, but not when I enter a 
> url. It appears to use the local machine for the DNS lookup, not 
> the proxy.
> 
> Is there some way I can encourage it to use the proxy for DNS 
> resolution? Alternatively, how can I get the remote IP address 
> from the existing TWSocket connection? (PeerAddr only gives me 
> the IP of the proxy server).

Don't know much about Socks proxies, not used one for many years.  

But a quick look at Wikipedia suggests SOCKS4a and SOCKS5 allow a
domain name to be passed in the protocol setup rather than an IP
address which is how sockets are normally opened.  

Normally ICS components do a DNS lookup first, then use that IP to open
a socket.  For SOCKS, the lookup would need to be skipped, and it's
possible WSocket is doing that, but the HTTP client is not, which would
explain your experience.  This might have worked in the distant past,
but got broken as other improvements were made, and no-one noticed
because SOCKS use is rare.  

But I've not looked at any source code and will not be able to test any
of this for a couple of weeks.  

Maybe someone else is actually using SOCKS with ICS?

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] FTP Client

[Angus Robertson - Magenta Systems Ltd]

> ICS sockets have built-in timeout feature, just define 
> BUILTIN_TIMEOUT and use TimeoutIdle / TimeoutConnect / OnTimeout.

I suspect TimeoutConnect has never been tested, it is not implemented
in any ICS protocol components or samples.  

Components like FTP would need the OnTimeout event to be added, which
would then try to abort the connection.  This might be successful for
wsocket, but Windows will still be trying to connect until it's
internal timeout, and probably would not allow a second connection
attempt until the first had finished.  

Timeouts are a fact of life with TCP/IP, and difficult to fight. 

Because the built-in timeout stuff was not enabled by default until
2011, most protocol components and/or applications have their own
timeout implementations, and generally we don't mess with stable and
tested code, unless doing so adds significant benefits. 

Angus
 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Install V8.50 on C++ Builder 10.2.1

[Angus Robertson - Magenta Systems Ltd]

> So I can send you the complete package containing project group 
> and all six package project files. 

Thanks, please send to my email, not the list.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] FTP Client

[Angus Robertson - Magenta Systems Ltd]

> If I try and connect to a FTP server and that computer isn't even 
> switch on, the FTP client gives up after about 25 seconds. Is 
> this timeout goverened by the FTP component or by something 
> deeper down in Windows networking. I was just wondering if I 
> could shorten this timeout period. 

This timeout is standard TCP/IP stuff, very hard to change, buried in
the Windows registry and will effect all applications. 

The only simple solution is to ping the remote server first, which can
have a variable timeout of milliseconds upwards.  But not all servers
respond to ping and sometimes firewalls block it deliberately, like
mine. 

Look at the OverbyteIcsPingTst.dpr sample, and TPingThread in
particular. 

> The FTP Client "Timeout" 
> property doesn't seem to have any effect on this, so the next 
> question would be as to what this does affect?

Timeout during a connection, when no traffic is moving. 

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] TWSocket...

[Angus Robertson - Magenta Systems Ltd]

> >That is 25 versions old, lines move.  What function? 
> function TCustomWSocket.GetPeerAddr: String;

Thanks, literal corrected. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Twsocket Problem over Internet.

[Angus Robertson - Magenta Systems Ltd]

> anything is wrong ?

Do you mean your code does not work correctly.  

We have made various suggestions about how it should be written to work
correctly, but you have yet to implement any of them.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] TWSocket...

[Angus Robertson - Magenta Systems Ltd]

> If I call TWSocket .Connect and Windows times out (about 25 
> seconds), I get the TWSocket .SessionClosed callback. How do I
> determine it was a timeout and not just a normal close? 

The onSessionConnected event is always called after Connect, with an
Error number.  If not zero, connection failed.   

onSessionConnected will be called before onSessionClosed, but the
latter does not know if the connection was OK or failed. 

> Line 8249, Version 8.26

That is 25 versions old, lines move.  What function? 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Twsocket Problem over Internet.

[Angus Robertson - Magenta Systems Ltd]

> I try send by block and have the same problem.
> Sleep(100);

If you changed that to Sleep(5000) it would probably work, but very
slowly   

As I said originally, you should be using the OnDataSent to determine
when the data is sent successfully, not just waiting and assuming the
internet is working perfectly and instantly. 

Your receiving is also likely to fail, LineMode should not be used when
you expect binary data, and makes a program more complex than necessary
swapping back and forward while receiving, checking line end is not
very hard. 

And you need to loop to keep receiving until the whole image has been
arrived, it might take minutes on a slow connection.  And handle the
connection breaking before it's complete. 

Angus



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Twsocket Problem over Internet.

[Angus Robertson - Magenta Systems Ltd]

> HI for all, i do a prototype to transfer a imagem overt internet 
> with Twsocket,
> but in local network works fine, but when i test with a remote 
> computer in internet connection
> the file is sended but dont receive or receive a image with lost 
> collor.

Your code assumes that Send() will cope with an unlimited amount of
data, when there are limits due to the internal buffers.

For large sizes, say over 32K, you need to send the data in blocks,
waiting until each has been sent, before sending the next.  This is
illustrated in several sample applications, with each new block being
sent when the OnDataSent event is called.

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] OpenSSL 1.1.0g and 1.0.2m support

[Angus Robertson - Magenta Systems Ltd]

Four new zips for Win32 and Win64 versions of OpenSSL 1.1.0g and 1.0.2m
can now be downloadable from the Wiki at:

http://wiki.overbyte.eu/wiki/index.php/ICS_Download

PLEASE NOTE the EU in the URL, not BE, this is a new wiki server.  

There is one moderate security fix relating to Intel Broadwell (5th
generation) and later or AMD Ryzen processors, and one low risk
security fix relating to malformed certificates.  

Angus


-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] Arno Garrels RIP

[Angus Robertson - Magenta Systems Ltd]

Many of you will remember Arno Garrels, who was a major contributor to
ICS for many years.  

In April 2014, he privately told Francois and I that he was being
treated for cancer, we heard from him a couple more times, but then
nothing.  

I've just found an obituary from a old friend saying he died
unexpectedly on 9th August 2014.  It is in German, but Google Translate
makes it readable.  

http://hortus-conclusus.berlin/tag/arno-garrels/

Arno did much of the ICS SSL support, unicode, Win64, C++, Firemonkey,
MacOS, and IPv6, and added many new units and demos.  His absence is
the reason development of some of these has stalled.  

Arno's lasting legacy is all that ICS code that many of us rely on
daily. 

Angus







-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] tlsv1 unrecognized name

[Angus Robertson - Magenta Systems Ltd]

> A customer is reporting an issue. Perhaps someone understands 
> this error better than I do:
> Error: 10053, Status: 404, Reason: SSL handshake failed - 
> error:14094458:SSL
> routines:ssl3_read_bytes:tlsv1 unrecognized name

SSL handshake failures are difficult to diagnose, despite the seemingly
extensive error messages.   

Most of the time, handshake errors are down to incompatible ciphers and
protocols, the server might have ancient support and the client demands
modern protocols, or vice versa.   

'unrecognized name' might relate to Server Name Indication which was
not supported with SSL, only TLS.  

The best diagnostic is put the remote host name into an SSL server test
tool, like:  

https://www.ssllabs.com/ssltest/

which will throw hundreds of SSL packets are the server and generate an
extensive report it's capabilities, or lack of them.  This will almost
certainly explain what is wrong.  

Otherwise you need to use Wireshark or something to examine the actual
handshake packets and see what is missing, not trivial.   

OpenSSL 1.1.0 includes better diagnostics allowing the handshake
packets to be logged by ICS, after decryption, but ICS does not yet
have the protocol analysis that Wireshark has to decode the packets. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] Wiki...

[Angus Robertson - Magenta Systems Ltd]

> Is it just me or has the Wiki 
> (http://wiki.overbyte.eu/wiki/index.php) disappeared?

It was fine on Tuesday, but the VM has probably run out of disk space
(again) so is currently dead, only Francois can fix it. 

As always, the latest overnight zip and SSL files may be downloaded
from my web site where SVN is hosted: 

https://www.magsys.co.uk/delphi/magics.asp

The zip is currently the same as the latest stable version V8.50, I
copied it on Monday and there are no changes since then.  Only
difference is I removed the massive SVN change log to save space. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be