Re: [twsocket] SSL post extremely slow

[brian -]

Here we go,


*98 SE (Delphi7/v5)* VM
ICS: Upload Duration: 34.67 secs/Speed: 131,341 chars/sec
SYN: Upload Duration: 9.05 secs/Speed: 503,314 chars/sec

*2000* VM
ICS: Upload Duration: 34.38 secs/Speed: 132,475 chars/sec
SYN: Upload Duration: 4.67 secs/Speed: 974,531 chars/sec

*XP* VM
ICS: Upload Duration: 34.47 secs/Speed: 132,114 chars/sec
SYN: Upload Duration: 4.7 secs/Speed: 968,109 chars/sec

*Vista* VM
ICS: Upload Duration: 47.39 secs/Speed: 96,093 chars/sec
SYN: Upload Duration: 2.22 secs/Speed: 2,051,345 chars/sec

*2008 R2* VM
ICS: Upload Duration: 34.41 secs/Speed: 132,356 chars/sec
SYN: Upload Duration: 1.97 secs/Speed: 2,312,842 chars/sec

*Win7 (main pc)*
ICS: Upload Duration: 34.39 secs/132,417 chars/sec
SYN: Upload Duration: 2.03 secs/Speed: 2,241,135 chars/sec

*Win8* VM
ICS: Upload Duration: 2.02 secs/Speed: 2,258,922 chars/sec
SYN: Upload Duration: 1.83 secs/Speed: 2,489,878 chars/sec

*Win8.1* VM
ICS: Upload Duration: 1.84 secs/Speed: 2,469,624 chars/sec
SYN: Upload Duration: 1.86 secs/Speed: 2,448,380 chars/sec

*2012 R2* VM
ICS: Upload Duration: 4.58 secs/Speed: 994,537 chars/sec
SYN: Upload Duration: 4.53 secs/Speed: 1,004,629 chars/sec

*Win10* *(main pc)*
ICS: Upload Duration: 1.88 secs/ 2,427,498 chars/sec
SYN: Upload Duration: 2.02 secs/Speed: 2,257,802 chars/sec

*Ubuntu16* *(main pc)*
ICS: Upload Duration: 1.41 secs/Speed: 3,236,664 chars/sec
SYN: Upload Duration: 1.41 secs/Speed: 3,234,365 chars/sec


On Thu, Apr 28, 2016 at 12:51 AM, RTT  wrote:

> On 27/04/2016 18:12, Angus Robertson - Magenta Systems Ltd wrote:
>
>> WinInet is a special case and probably handles TCP/IP as a driver or
>> something we can not do.
>>
>
> Maybe similar performance can be achieved with the Winsock TransmitFile
> and TransmitPackets functions.
>
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[RTT]

On 27/04/2016 18:12, Angus Robertson - Magenta Systems Ltd wrote:

WinInet is a special case and probably handles TCP/IP as a driver or
something we can not do.


Maybe similar performance can be achieved with the Winsock TransmitFile 
and TransmitPackets functions.



--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] SSL post extremely slow

[Tim Hyde]

I am assuming that you do not have the QOS Packet Scheduler service
installed (under Local Area Connection Properties for the LAN adapter) -
this has always screwed things up for me over the years.
 
Regards,

Tim 

-Original Message-
From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of
twsocket-requ...@lists.elists.org
Sent: 27 April 2016 19:47
To: twsocket@lists.elists.org
Subject: TWSocket Digest, Vol 650, Issue 3

Send TWSocket mailing list submissions to
twsocket@lists.elists.org

To subscribe or unsubscribe via the World Wide Web, visit
http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
or, via email, send a message with subject or body 'help' to
twsocket-requ...@lists.elists.org

You can reach the person managing the list at
twsocket-ow...@lists.elists.org

When replying, please edit your Subject line so it is more specific than
"Re: Contents of TWSocket digest..."

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

On Wed, Apr 27, 2016 at 8:46 PM, brian -  wrote:

>
>
>> I still have some Windows 7 VMs, so will do more testing next week,
>> once the new OpenSSL stuff is finished.
>>
>
>
> These tests on VMs are flawed though, as observed in all my testing,
> should be done on a real W7 host.
>


Actually nvm that, I'm able to replicate the issue in VMs by changing the
default Network adapter configuration from NAT to BRIDGED (vmware). I'll
try to test W8/8.1/Vista/2008
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

On Wed, Apr 27, 2016 at 8:28 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

>
> I still have some Windows 7 VMs, so will do more testing next week,
> once the new OpenSSL stuff is finished.
>


These tests on VMs are flawed though, as observed in all my testing, should
be done on a real W7 host.
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[RTT]

On 27/04/2016 18:12, Angus Robertson - Magenta Systems Ltd wrote:

Does not really make sense to me, but could be down to improved async
winsock with Windows 10.

Maybe related?
https://blogs.windows.com/buildingapps/2015/07/02/networking-api-improvements-in-windows-10/
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> ** Fresh Win7 on my PC: *
> - ICS: Upload Duration: 34.39 secs/132,417 chars/sec
> - SYN: Upload Duration: 2.03 secs/Speed: 2,241,135 chars/sec
> 
> ** Fresh Win10 on my PC**
> - ICS: Upload Duration: 1.88 secs/ 2,427,498 chars/sec
> - SYN: Upload Duration: 2.02 secs/Speed: 2,257,802 chars/sec

Does not really make sense to me, but could be down to improved async
winsock with Windows 10.  ICS always uses async, Indy and Synapse are
blocking sync calls using threads. Windows must be creating a thread
itself to offer async, and maybe this is now better.  

WinInet is a special case and probably handles TCP/IP as a driver or
something we can not do.   

Microsoft does mess this stuff up, a few years ago a new ADO/SQL client
introduced a handle leak for each async call, took me six month to
convince them, another six months for a fix, by which time I changed my
application to use a thread instead so I did not run out of memory when
250,000 handles were allocated.  

> As for the GET, the 2 ICS links seem a bit slower than the IIS.
> Did a GET test on a 5MB file: 
> http://download.thinkbroadband.com/5MB.zip
> Both ICS and SYN took around 4.1-4.3s, no issues with http 
> downloads I think.

As I would expect, IIS uses Kernel mode drivers for performance, ICS is
stuck with winsock API so is slightly slower. 

One puzzle here is your testing of www1 is to Windows 2008 which is
effectively Vista (the server has been running with the ICS SVN since
November 2008, down time 1 hour since then), so should be slow with
async winsock if Windows 7 is slow.  You can also test uploads to www
which is Windows 2012 R2 and only a year old, but is not the latest ICS
since it's my main live server and the new OpenSSL is not stable yet. 

Angus
 



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Alright I'm back with some interesting results that confirm my suspicions
(actually meant 300/30 Mbit btw)

* All VM tests (xp/w7/w10/ubuntu) showed similar results around 0.9..1MB/s
upload, except when using a UK VPN: ICS was again slow.

* Win XP on a netbook on wifi (Asus eee901): ICS: 65.8s. SYN: 6.5s

** Fresh Win7 on my PC: *
- ICS: Upload Duration: 34.39 secs/132,417 chars/sec
- SYN: Upload Duration: 2.03 secs/Speed: 2,241,135 chars/sec

** Fresh Win10 on my PC**
- ICS: Upload Duration: 1.88 secs/ 2,427,498 chars/sec
- SYN: Upload Duration: 2.02 secs/Speed: 2,257,802 chars/sec

** Fresh Ubuntu16 on my PC: (wine)*
- ICS: Upload Duration: 1.41 secs/Speed: 3,236,664 chars/sec
- SYN: Upload Duration: 1.41 secs/Speed: 3,234,365 chars/sec
- Firefox: Upload Duration: 1.35 secs/Speed: 3,385,863 chars/sec

As for the GET, the 2 ICS links seem a bit slower than the IIS.
Did a GET test on a 5MB file: http://download.thinkbroadband.com/5MB.zip
Both ICS and SYN took around 4.1-4.3s, no issues with http downloads I
think.

My conclusion so far:

Up to Windows 7, ICS is interacting with Winsock in a manner that causes
such extreme low post speeds, and it seems to be "fixed" in Windows 10
(possibly since W8?)
The VM tests showed similar results on all platforms, most likely due to
~negating the "wrong" way ICS is interacting with the server, possibly
something like.. VM->ICS posts data to locahost->VMware bridge with host
captures this data and sends it a different way to the actual server,
negating the root ICS issue, sending data its own way no matter the guest
platform.

Also interesting how much faster the Linux tests were :O

Did you compile you application with "release" config, and optimization and
> without any debugging tool enabled such as madExcept or FastMM debug mode ?
>

Yeah, you can check in the test app I linked a few posts earlier,
release/no exception handler/no FastMM.

edit: I'm about to install a fresh Win8 now, will post back.

On Wed, Apr 27, 2016 at 2:05 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > So I'm kinda baffled by these results, not sure what to make of it.
>
> I do know from long experience that ruling out a single PC is the first
> thing I do when testing stuff like this.  But even then, sometimes
> results simply can never be explained.
>
> I have speed test downloads from my public servers every morning, and
> have known for a long time that HTTP is sometimes slower than it should
> be, although today it seems rather good.  I have two broadband
> connections, 40/10 and 152/12 but use the former for most speed tests,
> although I did download 10.1 Berlin ISO at 100Mb last week which would
> excellent.
>
> You have only been doing uploads so far, but I will be mainly testing
> downloads when I fix the buffers, using:
>
> http://www.telecom-tariffs.co.uk/testing/speed50meg.zip
> http://www.magsys.co.uk/download/testing/speed50meg.zip
> http://www1.telecom-tariffs.co.uk/testing/speed50meg.zip
> http://www1.magsys.co.uk/download/testing/speed50meg.zip
>
> The first URL is ICS, second IIS, www is Windows 2012 R2, www1 is 2008.
> As I said earlier, these servers are behind a slowish firewall since I
> get continual hacking attempts and Windows servers are so open to
> attack, the set-up was never designed for high speed testing.
>
> Angus
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Yeah, I'm on 100/30 Mbit fiber, Windows 7 x64 and the latest V8 SVN with
Delphi Seattle. Some days ago I tried an old V5 with Delphi7 to rule out
this problem having been recently introduced, but I observed the same
results. Since you mentioned Windows 10, I thought of making a few extra
tests with various OS's and conditions, with some odd results:

Win7 default:
ICS: Upload Duration: 34.27 secs. Speed: 132,897 chars/sec
SYN: Upload Duration: 2.13 secs. Speed: 2,142,044 chars/sec

Few virtual machines with VMware:
Win7 VM: ICS: Speed: 931,095 chars/sec. SYN: Speed: Speed: 933,959 chars/sec
Win7 VM on UK VPN: ICS: Speed: 88,586 chars/sec. SYN: Speed: 869,745
chars/sec
Win XP VM: Speed: 939,935 chars/sec. SYN: Speed: 943,049 chars/sec
Win10 VM: Speed: Speed: 936,841 chars/sec. Speed: 933,959 chars/sec

Asked a german friend to do the same test, to rule out ISP issues, with him
being on w7:
ICS: Speed: 180,234 chars/sec
Synapse: Speed: 692,200 chars/sec

Tried with my phone's 4G: (shared connection through USB)
Firefox: Upload Duration: 2.72 secs. Speed: 1,674,259 chars/sec
ICS: Upload Duration: 53.56 secs. Speed: 85,021 chars/sec
SYN: Upload Duration: 2.25 secs. Speed: 2,023,095 chars/sec



*Kinda unrelated, but noticed also a weird result from various
browsers:Chrome: Speed: 621,280 chars/sec*

*Firefox: Speed: 1,844,466 chars/sec*
*IE11: Speed: 2,206,389 chars/sec*


So I'm kinda baffled by these results, not sure what to make of it. Ruled
out this being an issue with w7/intel driver (my friend seems to have a
board with intel NIC as well) by testing with 4G. Ruled out ISP issue by
having my friend in Germany do the same test. But then why are the VMs
having those results? (same speed ics/syn). Also the VPN on the w7 VM
experiencing the same issue on ICS is very very weird. Could it be an issue
specifically in my machine? but then having my friend experience the same
is quite a coincidence, and it wouldn't explain why SYN works just fine.

Next I will test on a clean install of both W7 and W10 without a VM on my
machine, I'll post back with results (probably a few hours, gonna install
them on a spare 5400rpm 2,5" disk).

On Wed, Apr 27, 2016 at 9:47 AM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > Memo isn't touched during the progress, only the progressbar.
> > Removed OnSenData event, same results.
>
> You appear to be on a Spanish RIMA (Red IP Multi Acceso) network,
> although trace route does not complete.
>
> Trace Route to: 83.39.166.229
> Can Not Ping Host (83.39.166.229) : Request timed out
>  1 0ms  Unexpected Family  MAGPUB1
>  2 0ms  217.146.102.129217.146.102.129
>  3 12ms 217.146.96.72  gi0.man-gw2.merula.net
>  4 23ms 212.187.137.161te-8-3-215.car2.manchester1.level3.net
>  5 19ms 212.187.160.230telefonica.edge6.london1.level3.net
>  6 50ms 84.16.13.25 xe-grtbcntb1.net.telefonicaglobalsolutions.com
>  7 54ms 94.142.103.178  x0-grtbcntb1.net.telefonicaglobalsolutions.com
>  8 60ms 80.58.88.210210.red-80-58-88.staticip.rima-tde.net
>  9 Request timed out
>
> Your best speed was 1,928,838 chars/sec which suggests you have at
> least a 20Mb/sec upload speed which is about double the speed from my
> office.  This old public server is still on 100Mb/sec ethernet since
> the firewall is quite slow.
>
> The only thing that can explain the speed difference from the same PC
> is TCP/IP packet size, somehow the ICS application is sending more
> smaller packets than some others, and that is probably explained by the
> 8K buffer size currently used by the HTTP client, although it's
> surprising that I get 10 times your speed with the same software,
> assuming you are using Windows 10 and ICS V8.27 which you have never
> mentioned.
>
> Unless ICS is setting some strange TCP setting, buffer size is the only
> thing that can be changed, which I will look at next month, as I've
> said before.  That should allow Windows to send larger IP packets,
> which should help speed.
>
> Unless you have a local firewall or proxy that is crippling speeds.
>
> Angus
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> Memo isn't touched during the progress, only the progressbar. 
> Removed OnSenData event, same results.

You appear to be on a Spanish RIMA (Red IP Multi Acceso) network,
although trace route does not complete.  

Trace Route to: 83.39.166.229
Can Not Ping Host (83.39.166.229) : Request timed out
 1 0ms  Unexpected Family  MAGPUB1
 2 0ms  217.146.102.129217.146.102.129
 3 12ms 217.146.96.72  gi0.man-gw2.merula.net
 4 23ms 212.187.137.161te-8-3-215.car2.manchester1.level3.net
 5 19ms 212.187.160.230telefonica.edge6.london1.level3.net
 6 50ms 84.16.13.25 xe-grtbcntb1.net.telefonicaglobalsolutions.com
 7 54ms 94.142.103.178  x0-grtbcntb1.net.telefonicaglobalsolutions.com
 8 60ms 80.58.88.210210.red-80-58-88.staticip.rima-tde.net
 9 Request timed out

Your best speed was 1,928,838 chars/sec which suggests you have at
least a 20Mb/sec upload speed which is about double the speed from my
office.  This old public server is still on 100Mb/sec ethernet since
the firewall is quite slow. 

The only thing that can explain the speed difference from the same PC
is TCP/IP packet size, somehow the ICS application is sending more
smaller packets than some others, and that is probably explained by the
8K buffer size currently used by the HTTP client, although it's
surprising that I get 10 times your speed with the same software,
assuming you are using Windows 10 and ICS V8.27 which you have never
mentioned.

Unless ICS is setting some strange TCP setting, buffer size is the only
thing that can be changed, which I will look at next month, as I've
said before.  That should allow Windows to send larger IP packets,
which should help speed. 

Unless you have a local firewall or proxy that is crippling speeds.

Angus
 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be