Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> Angus, could you share the upload script on your server?

It's not a script, the server uses the ICS web application server
component, and the upload page is Delphi code, based on the upload demo
page in the ICS web server sample. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Angus, could you share the upload script on your server?

On Thu, May 12, 2016 at 5:35 PM, RTT  wrote:

>
>
> Either way, if the problem is the async nature of ICS,
>> shouldn't an alternate method be provided?
>>
>
> Try with the TransmitFile function
>
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms740565(v=vs.85).aspx
>
> You may create a descendant of the THttpCli, override the SocketDataSent
> method and use the TransmitFile from there.
> Check the Indy IdStackWindows.pas for code (function ServeFile) that uses
> the TransmitFile.
> If this work, maybe it can be integrated in the ICS code.
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[RTT]

Either way, if the problem is the async nature of ICS,
shouldn't an alternate method be provided?


Try with the TransmitFile function
https://msdn.microsoft.com/en-us/library/windows/desktop/ms740565(v=vs.85).aspx

You may create a descendant of the THttpCli, override the SocketDataSent 
method and use the TransmitFile from there.
Check the Indy IdStackWindows.pas for code (function ServeFile) that 
uses the TransmitFile.

If this work, maybe it can be integrated in the ICS code.
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

On Thu, May 12, 2016 at 4:24 PM, RTT  wrote:

>
> If that's the case, using SYN, that is sync, to compare against the async
> ICS is not of any help. You need to compare with another async component.



I think all other libraries are blocking, Indy, Synapse, Delphi's own
tcp/ip stuff. Either way, if the problem is the async nature of ICS,
shouldn't an alternate method be provided? We are not talking about a
little difference here, it's huge. Also I'm not sure it makes much sense to
stay async with current hardware/software, things have changed, and it's
very easy to use a blocking socket in a thread to achieve the same.
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Checked again just now, w7:

ICS with 64k buffer: Upload Duration: 33.2 secs / Speed: 137,151 chars/sec
SYN: Upload Duration: 2 secs / Speed: 2,275,855 chars/sec

On Thu, May 12, 2016 at 4:04 PM, brian -  wrote:

> Erm, I thought we all reached a clear conclusion that the issue happens
> from w98 to w7, and whatever causes the problem seems fixed from w8
> onwards. Since all the tests were compared with a different tcp/ip library
> at the exact same time of performing the ICS tests, I think we can rule out
> random temporal issues.
>
> On Thu, May 12, 2016 at 3:48 PM, Angus Robertson - Magenta Systems Ltd <
> an...@magsys.co.uk> wrote:
>
>> > GET seems fine though. You mentioned changing the buffer size,
>> > where would you do that?
>>
>> I've not yet looked closely at the code, once SSL is done.
>>
>> > I tried changing 8k to 64k in OverbyteIcsHttpProt.pas here,
>> > but it made no difference:
>>
>> But then you discovered the low speeds only happened on certain
>> platforms, not all platforms.  Which is exactly my point, modern
>> hardware and perhaps the OS, can effect performance dramatically, as
>> can the internet generally.
>>
>> I do daily speed tests at 5:30am downloading HTTP files from several of
>> my servers, and a few weeks ago the speed tripled without any changes
>> of client or server software or internet access, but some days the
>> speed is back to the original slower pace.  It could be windows updates,
>> my Sonicwall firewalls behaving strangely, or the broadband wholesaler
>> throttling my connection.  Sometimes this stuff can not be rationally
>> explained.
>>
>> Angus
>>
>>
>>
>>
>> --
>> To unsubscribe or change your settings for TWSocket mailing list
>> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
>> Visit our website at http://www.overbyte.be
>>
>
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

GET seems fine though. You mentioned changing the buffer size, where would
you do that? I tried changing 8k to 64k in OverbyteIcsHttpProt.pas here,
but it made no difference:

~4617
procedure THttpCli.SocketDataSent(Sender : TObject; ErrCode : Word);
var
Len : Integer;
begin
if not FAllowedToSend then
Exit;

if Length(FSendBuffer) = 0 then
SetLength(FSendBuffer, 8192);
Len := FSendStream.Read(FSendBuffer[0], Length(FSendBuffer));



On Thu, May 12, 2016 at 2:05 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > Note that the original subject is a bit misleading, the issue is
> > not about ssl POST, but POST in general, whether on http or https.
>
> And my comments were mostly related to GET/POST in general, not
> specifically SSL which one might expect to slow things down, but not
> necessarily.
>
> Angus
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Note that the original subject is a bit misleading, the issue is not about
ssl POST, but POST in general, whether on http or https.

On Thu, May 12, 2016 at 12:13 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > I still have some Windows 7 VMs, so will do more testing next
> > week, once the new OpenSSL stuff is finished.
>
> Not done any changes or real testing yet, except to confirm that HTTPS
> downloads are currently the same speed as HTTP, although one would
> expect the encryption overhead to slow things down.
>
> I have just discovered that all SSL traffic seems to use a fixed 4K
> buffer in wsocket, which I'll make variable sized.
>
> I suspect one reason we are currently seeing good speeds even with low
> buffer sizes is simply that the modern hardware we now use is so
> powerful and it can cope with programming deficiencies without
> performance penalty.
>
> All my clients and servers (bar one) are 64-bit OSs with 16G of memory
> and multi-core i5 or Xeon, my main development PC is SSD drives, less
> important for the servers since files are RAM cached.  And the latest
> CPUs have instructions to speed up SSL encryption.
>
> But this was all different even five years ago and if repeat testing on
> older hardware, I'm sure it will be much slower and the benefits of
> larger buffers and buffered file handling more obvious.  Not sure it's
> worth the effort...
>
> Angus
>
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Here we go,


*98 SE (Delphi7/v5)* VM
ICS: Upload Duration: 34.67 secs/Speed: 131,341 chars/sec
SYN: Upload Duration: 9.05 secs/Speed: 503,314 chars/sec

*2000* VM
ICS: Upload Duration: 34.38 secs/Speed: 132,475 chars/sec
SYN: Upload Duration: 4.67 secs/Speed: 974,531 chars/sec

*XP* VM
ICS: Upload Duration: 34.47 secs/Speed: 132,114 chars/sec
SYN: Upload Duration: 4.7 secs/Speed: 968,109 chars/sec

*Vista* VM
ICS: Upload Duration: 47.39 secs/Speed: 96,093 chars/sec
SYN: Upload Duration: 2.22 secs/Speed: 2,051,345 chars/sec

*2008 R2* VM
ICS: Upload Duration: 34.41 secs/Speed: 132,356 chars/sec
SYN: Upload Duration: 1.97 secs/Speed: 2,312,842 chars/sec

*Win7 (main pc)*
ICS: Upload Duration: 34.39 secs/132,417 chars/sec
SYN: Upload Duration: 2.03 secs/Speed: 2,241,135 chars/sec

*Win8* VM
ICS: Upload Duration: 2.02 secs/Speed: 2,258,922 chars/sec
SYN: Upload Duration: 1.83 secs/Speed: 2,489,878 chars/sec

*Win8.1* VM
ICS: Upload Duration: 1.84 secs/Speed: 2,469,624 chars/sec
SYN: Upload Duration: 1.86 secs/Speed: 2,448,380 chars/sec

*2012 R2* VM
ICS: Upload Duration: 4.58 secs/Speed: 994,537 chars/sec
SYN: Upload Duration: 4.53 secs/Speed: 1,004,629 chars/sec

*Win10* *(main pc)*
ICS: Upload Duration: 1.88 secs/ 2,427,498 chars/sec
SYN: Upload Duration: 2.02 secs/Speed: 2,257,802 chars/sec

*Ubuntu16* *(main pc)*
ICS: Upload Duration: 1.41 secs/Speed: 3,236,664 chars/sec
SYN: Upload Duration: 1.41 secs/Speed: 3,234,365 chars/sec


On Thu, Apr 28, 2016 at 12:51 AM, RTT  wrote:

> On 27/04/2016 18:12, Angus Robertson - Magenta Systems Ltd wrote:
>
>> WinInet is a special case and probably handles TCP/IP as a driver or
>> something we can not do.
>>
>
> Maybe similar performance can be achieved with the Winsock TransmitFile
> and TransmitPackets functions.
>
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[RTT]

On 27/04/2016 18:12, Angus Robertson - Magenta Systems Ltd wrote:

WinInet is a special case and probably handles TCP/IP as a driver or
something we can not do.


Maybe similar performance can be achieved with the Winsock TransmitFile 
and TransmitPackets functions.



--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] SSL post extremely slow

[Tim Hyde]

I am assuming that you do not have the QOS Packet Scheduler service
installed (under Local Area Connection Properties for the LAN adapter) -
this has always screwed things up for me over the years.
 
Regards,

Tim 

-Original Message-
From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of
twsocket-requ...@lists.elists.org
Sent: 27 April 2016 19:47
To: twsocket@lists.elists.org
Subject: TWSocket Digest, Vol 650, Issue 3

Send TWSocket mailing list submissions to
twsocket@lists.elists.org

To subscribe or unsubscribe via the World Wide Web, visit
http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
or, via email, send a message with subject or body 'help' to
twsocket-requ...@lists.elists.org

You can reach the person managing the list at
twsocket-ow...@lists.elists.org

When replying, please edit your Subject line so it is more specific than
"Re: Contents of TWSocket digest..."

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

On Wed, Apr 27, 2016 at 8:46 PM, brian -  wrote:

>
>
>> I still have some Windows 7 VMs, so will do more testing next week,
>> once the new OpenSSL stuff is finished.
>>
>
>
> These tests on VMs are flawed though, as observed in all my testing,
> should be done on a real W7 host.
>


Actually nvm that, I'm able to replicate the issue in VMs by changing the
default Network adapter configuration from NAT to BRIDGED (vmware). I'll
try to test W8/8.1/Vista/2008
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

On Wed, Apr 27, 2016 at 8:28 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

>
> I still have some Windows 7 VMs, so will do more testing next week,
> once the new OpenSSL stuff is finished.
>


These tests on VMs are flawed though, as observed in all my testing, should
be done on a real W7 host.
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[RTT]

On 27/04/2016 18:12, Angus Robertson - Magenta Systems Ltd wrote:

Does not really make sense to me, but could be down to improved async
winsock with Windows 10.

Maybe related?
https://blogs.windows.com/buildingapps/2015/07/02/networking-api-improvements-in-windows-10/
--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> ** Fresh Win7 on my PC: *
> - ICS: Upload Duration: 34.39 secs/132,417 chars/sec
> - SYN: Upload Duration: 2.03 secs/Speed: 2,241,135 chars/sec
> 
> ** Fresh Win10 on my PC**
> - ICS: Upload Duration: 1.88 secs/ 2,427,498 chars/sec
> - SYN: Upload Duration: 2.02 secs/Speed: 2,257,802 chars/sec

Does not really make sense to me, but could be down to improved async
winsock with Windows 10.  ICS always uses async, Indy and Synapse are
blocking sync calls using threads. Windows must be creating a thread
itself to offer async, and maybe this is now better.  

WinInet is a special case and probably handles TCP/IP as a driver or
something we can not do.   

Microsoft does mess this stuff up, a few years ago a new ADO/SQL client
introduced a handle leak for each async call, took me six month to
convince them, another six months for a fix, by which time I changed my
application to use a thread instead so I did not run out of memory when
250,000 handles were allocated.  

> As for the GET, the 2 ICS links seem a bit slower than the IIS.
> Did a GET test on a 5MB file: 
> http://download.thinkbroadband.com/5MB.zip
> Both ICS and SYN took around 4.1-4.3s, no issues with http 
> downloads I think.

As I would expect, IIS uses Kernel mode drivers for performance, ICS is
stuck with winsock API so is slightly slower. 

One puzzle here is your testing of www1 is to Windows 2008 which is
effectively Vista (the server has been running with the ICS SVN since
November 2008, down time 1 hour since then), so should be slow with
async winsock if Windows 7 is slow.  You can also test uploads to www
which is Windows 2012 R2 and only a year old, but is not the latest ICS
since it's my main live server and the new OpenSSL is not stable yet. 

Angus
 



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Alright I'm back with some interesting results that confirm my suspicions
(actually meant 300/30 Mbit btw)

* All VM tests (xp/w7/w10/ubuntu) showed similar results around 0.9..1MB/s
upload, except when using a UK VPN: ICS was again slow.

* Win XP on a netbook on wifi (Asus eee901): ICS: 65.8s. SYN: 6.5s

** Fresh Win7 on my PC: *
- ICS: Upload Duration: 34.39 secs/132,417 chars/sec
- SYN: Upload Duration: 2.03 secs/Speed: 2,241,135 chars/sec

** Fresh Win10 on my PC**
- ICS: Upload Duration: 1.88 secs/ 2,427,498 chars/sec
- SYN: Upload Duration: 2.02 secs/Speed: 2,257,802 chars/sec

** Fresh Ubuntu16 on my PC: (wine)*
- ICS: Upload Duration: 1.41 secs/Speed: 3,236,664 chars/sec
- SYN: Upload Duration: 1.41 secs/Speed: 3,234,365 chars/sec
- Firefox: Upload Duration: 1.35 secs/Speed: 3,385,863 chars/sec

As for the GET, the 2 ICS links seem a bit slower than the IIS.
Did a GET test on a 5MB file: http://download.thinkbroadband.com/5MB.zip
Both ICS and SYN took around 4.1-4.3s, no issues with http downloads I
think.

My conclusion so far:

Up to Windows 7, ICS is interacting with Winsock in a manner that causes
such extreme low post speeds, and it seems to be "fixed" in Windows 10
(possibly since W8?)
The VM tests showed similar results on all platforms, most likely due to
~negating the "wrong" way ICS is interacting with the server, possibly
something like.. VM->ICS posts data to locahost->VMware bridge with host
captures this data and sends it a different way to the actual server,
negating the root ICS issue, sending data its own way no matter the guest
platform.

Also interesting how much faster the Linux tests were :O

Did you compile you application with "release" config, and optimization and
> without any debugging tool enabled such as madExcept or FastMM debug mode ?
>

Yeah, you can check in the test app I linked a few posts earlier,
release/no exception handler/no FastMM.

edit: I'm about to install a fresh Win8 now, will post back.

On Wed, Apr 27, 2016 at 2:05 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > So I'm kinda baffled by these results, not sure what to make of it.
>
> I do know from long experience that ruling out a single PC is the first
> thing I do when testing stuff like this.  But even then, sometimes
> results simply can never be explained.
>
> I have speed test downloads from my public servers every morning, and
> have known for a long time that HTTP is sometimes slower than it should
> be, although today it seems rather good.  I have two broadband
> connections, 40/10 and 152/12 but use the former for most speed tests,
> although I did download 10.1 Berlin ISO at 100Mb last week which would
> excellent.
>
> You have only been doing uploads so far, but I will be mainly testing
> downloads when I fix the buffers, using:
>
> http://www.telecom-tariffs.co.uk/testing/speed50meg.zip
> http://www.magsys.co.uk/download/testing/speed50meg.zip
> http://www1.telecom-tariffs.co.uk/testing/speed50meg.zip
> http://www1.magsys.co.uk/download/testing/speed50meg.zip
>
> The first URL is ICS, second IIS, www is Windows 2012 R2, www1 is 2008.
> As I said earlier, these servers are behind a slowish firewall since I
> get continual hacking attempts and Windows servers are so open to
> attack, the set-up was never designed for high speed testing.
>
> Angus
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Yeah, I'm on 100/30 Mbit fiber, Windows 7 x64 and the latest V8 SVN with
Delphi Seattle. Some days ago I tried an old V5 with Delphi7 to rule out
this problem having been recently introduced, but I observed the same
results. Since you mentioned Windows 10, I thought of making a few extra
tests with various OS's and conditions, with some odd results:

Win7 default:
ICS: Upload Duration: 34.27 secs. Speed: 132,897 chars/sec
SYN: Upload Duration: 2.13 secs. Speed: 2,142,044 chars/sec

Few virtual machines with VMware:
Win7 VM: ICS: Speed: 931,095 chars/sec. SYN: Speed: Speed: 933,959 chars/sec
Win7 VM on UK VPN: ICS: Speed: 88,586 chars/sec. SYN: Speed: 869,745
chars/sec
Win XP VM: Speed: 939,935 chars/sec. SYN: Speed: 943,049 chars/sec
Win10 VM: Speed: Speed: 936,841 chars/sec. Speed: 933,959 chars/sec

Asked a german friend to do the same test, to rule out ISP issues, with him
being on w7:
ICS: Speed: 180,234 chars/sec
Synapse: Speed: 692,200 chars/sec

Tried with my phone's 4G: (shared connection through USB)
Firefox: Upload Duration: 2.72 secs. Speed: 1,674,259 chars/sec
ICS: Upload Duration: 53.56 secs. Speed: 85,021 chars/sec
SYN: Upload Duration: 2.25 secs. Speed: 2,023,095 chars/sec



*Kinda unrelated, but noticed also a weird result from various
browsers:Chrome: Speed: 621,280 chars/sec*

*Firefox: Speed: 1,844,466 chars/sec*
*IE11: Speed: 2,206,389 chars/sec*


So I'm kinda baffled by these results, not sure what to make of it. Ruled
out this being an issue with w7/intel driver (my friend seems to have a
board with intel NIC as well) by testing with 4G. Ruled out ISP issue by
having my friend in Germany do the same test. But then why are the VMs
having those results? (same speed ics/syn). Also the VPN on the w7 VM
experiencing the same issue on ICS is very very weird. Could it be an issue
specifically in my machine? but then having my friend experience the same
is quite a coincidence, and it wouldn't explain why SYN works just fine.

Next I will test on a clean install of both W7 and W10 without a VM on my
machine, I'll post back with results (probably a few hours, gonna install
them on a spare 5400rpm 2,5" disk).

On Wed, Apr 27, 2016 at 9:47 AM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > Memo isn't touched during the progress, only the progressbar.
> > Removed OnSenData event, same results.
>
> You appear to be on a Spanish RIMA (Red IP Multi Acceso) network,
> although trace route does not complete.
>
> Trace Route to: 83.39.166.229
> Can Not Ping Host (83.39.166.229) : Request timed out
>  1 0ms  Unexpected Family  MAGPUB1
>  2 0ms  217.146.102.129217.146.102.129
>  3 12ms 217.146.96.72  gi0.man-gw2.merula.net
>  4 23ms 212.187.137.161te-8-3-215.car2.manchester1.level3.net
>  5 19ms 212.187.160.230telefonica.edge6.london1.level3.net
>  6 50ms 84.16.13.25 xe-grtbcntb1.net.telefonicaglobalsolutions.com
>  7 54ms 94.142.103.178  x0-grtbcntb1.net.telefonicaglobalsolutions.com
>  8 60ms 80.58.88.210210.red-80-58-88.staticip.rima-tde.net
>  9 Request timed out
>
> Your best speed was 1,928,838 chars/sec which suggests you have at
> least a 20Mb/sec upload speed which is about double the speed from my
> office.  This old public server is still on 100Mb/sec ethernet since
> the firewall is quite slow.
>
> The only thing that can explain the speed difference from the same PC
> is TCP/IP packet size, somehow the ICS application is sending more
> smaller packets than some others, and that is probably explained by the
> 8K buffer size currently used by the HTTP client, although it's
> surprising that I get 10 times your speed with the same software,
> assuming you are using Windows 10 and ICS V8.27 which you have never
> mentioned.
>
> Unless ICS is setting some strange TCP setting, buffer size is the only
> thing that can be changed, which I will look at next month, as I've
> said before.  That should allow Windows to send larger IP packets,
> which should help speed.
>
> Unless you have a local firewall or proxy that is crippling speeds.
>
> Angus
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> Memo isn't touched during the progress, only the progressbar. 
> Removed OnSenData event, same results.

You appear to be on a Spanish RIMA (Red IP Multi Acceso) network,
although trace route does not complete.  

Trace Route to: 83.39.166.229
Can Not Ping Host (83.39.166.229) : Request timed out
 1 0ms  Unexpected Family  MAGPUB1
 2 0ms  217.146.102.129217.146.102.129
 3 12ms 217.146.96.72  gi0.man-gw2.merula.net
 4 23ms 212.187.137.161te-8-3-215.car2.manchester1.level3.net
 5 19ms 212.187.160.230telefonica.edge6.london1.level3.net
 6 50ms 84.16.13.25 xe-grtbcntb1.net.telefonicaglobalsolutions.com
 7 54ms 94.142.103.178  x0-grtbcntb1.net.telefonicaglobalsolutions.com
 8 60ms 80.58.88.210210.red-80-58-88.staticip.rima-tde.net
 9 Request timed out

Your best speed was 1,928,838 chars/sec which suggests you have at
least a 20Mb/sec upload speed which is about double the speed from my
office.  This old public server is still on 100Mb/sec ethernet since
the firewall is quite slow. 

The only thing that can explain the speed difference from the same PC
is TCP/IP packet size, somehow the ICS application is sending more
smaller packets than some others, and that is probably explained by the
8K buffer size currently used by the HTTP client, although it's
surprising that I get 10 times your speed with the same software,
assuming you are using Windows 10 and ICS V8.27 which you have never
mentioned.

Unless ICS is setting some strange TCP setting, buffer size is the only
thing that can be changed, which I will look at next month, as I've
said before.  That should allow Windows to send larger IP packets,
which should help speed. 

Unless you have a local firewall or proxy that is crippling speeds.

Angus
 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[François Piette]

Hello,

You are updating your progress bar and memo in the event, each time it is
called. This probably slow down everything.
Please retest without any GUI interaction during transfer.

--
francois.pie...@overbyte.be
The author of the freeware multi-tier middleware MidWare
The author of the freeware Internet Component Suite (ICS)
http://www.overbyte.be



-Message d'origine-
De : TWSocket [mailto:twsocket-boun...@lists.elists.org] De la part de brian
-
Envoyé : mardi 26 avril 2016 03:58
À : ICS support mailing
Objet : Re: [twsocket] SSL post extremely slow

Did the test with a 4,5MB file, non-SSL

Firefox: 2.6s

Saved Uploaded File OK
New File Name: d:\tempwork\telecom\uploads\20160426-021729_Aion 2016.Apr.21
- 10.57.48.png
File Size: 4,553,987
Upload Duration: 2.6 secs
Speed: 1,754,908 chars/sec
Upload FileName: Aion 2016.Apr.21 - 10.57.48.png
FileTitle: aion ss test
Post URL: http://www1.telecom-tariffs.co.uk/testing/uploadfile.htm

ICS: 34.1s

POST /testing/uploadfile.htm HTTP/1.0
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Referer: http://www1.telecom-tariffs.co.uk/testing/uploadfile.htm
Content-Type: multipart/form-data;
boundary=147712359214935
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101
Firefox/45.0
Host: www1.telecom-tariffs.co.uk
Content-Length: 4554186

34133ms

unit Unit1;
>
> interface
>
> uses
>   Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, 
> System.Classes, Vcl.Graphics,
>   Vcl.Controls, Vcl.Forms, Vcl.Dialogs, OverbyteIcsWndControl,
>   OverbyteIcsHttpProt, Vcl.StdCtrls, Vcl.ComCtrls,
>
>   MultipartBuilder;
>
> type
>   TForm1 = class(TForm)
> Button1: TButton;
> Button2: TButton;
> HttpCli1: THttpCli;
> ProgressBar1: TProgressBar;
> Memo1: TMemo;
> procedure Button1Click(Sender: TObject);
> procedure HttpCli1SendData(Sender: TObject; Buffer: Pointer; Len:
> Integer);
> procedure HttpCli1BeforeHeaderSend(Sender: TObject; const Method:
> string; Headers: TStrings);
> procedure HttpCli1SendEnd(Sender: TObject);
>   private
> { Private declarations }
>   public
> { Public declarations }
>   end;
>
> var
>   Form1: TForm1;
>   x: Int64;
>
> implementation
>
> {$R *.dfm}
>
> procedure TForm1.Button1Click(Sender: TObject); Var fil: AnsiString;
> mp: TMultiPartObject;
> fl: string;
> tm: TMemoryStream;
> Const cT = '147712359214935';
> begin
>   // Fill multipart data+file
>   fl := 'z:\Aion 2016.Apr.21 - 10.57.48.png';
>   mp := TMultiPartObject.Create(cT);
>   tm := TMemoryStream.Create;
>   tm.LoadFromFile(fl);
>   ProgressBar1.Max := tm.Size;
>   With mp do begin
> AddFileFromStream('image',ExtractFileName(fl),'image/png',tm);
> CloseData;
> fil := GetFile;
>   end;
>   mp.Free;
>
>   x := GetTickCount;
>   With HttpCli1 do begin
> url := 'http://www1.telecom-tariffs.co.uk/testing/uploadfile.htm';
> SendStream := TMemoryStream.Create;
> SendStream.Write(fil[1],length(fil));
> SendStream.Seek(0,0);
> RcvdStream := TMemoryStream.Create;
> Timeout := 5;
> Agent := 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) 
> Gecko/20100101 Firefox/45.0';
> ContentTypePost := 'multipart/form-data; boundary='+cT;
> Reference := 
> 'http://www1.telecom-tariffs.co.uk/testing/uploadfile.htm
> ';
> try
>   PostAsync;
> except
>   On E:Exception do ShowMessage(E.Message);
> end;
>   end;
> end;
>
> procedure TForm1.HttpCli1BeforeHeaderSend(Sender: TObject; const Method:
> string; Headers: TStrings);
> begin
>   Memo1.Lines.Add(Headers.Text);
> end;
>
> procedure TForm1.HttpCli1SendData(Sender: TObject; Buffer: Pointer; Len:
> Integer);
> begin
>   ProgressBar1.Position := ProgressBar1.Position + Len; end;
>
> procedure TForm1.HttpCli1SendEnd(Sender: TObject); begin
>   With HttpCli1 do begin
> SendStream.Free;
> RcvdStream.Free;
>   end;
>   Memo1.Lines.Add('');
>   Memo1.Lines.Add(IntToStr(GetTickCount-x)+'ms');
> end;
>
> end.
>


On Mon, Apr 25, 2016 at 7:12 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > Are there any known issues with SslHttpCli for sending data? I've 
> > observed the following:
> > Sending a 3MB picture to imgur:
> > * with ICS/OpenSSL: takes ~27 seconds
>
> Done some testing with a view to investigating this issue, but not 
> currently sure it's worth any more time.
>
> I'm using an updated version of the sample OverbyteIcsHttpPost against 
> the ICS web server (no SSL).
>
> With both running on my PC:
> File Size: 20,450,630
> Upload Duration: 204 msecs

Re: [twsocket] SSL post extremely slow

[RTT]

On 25/04/2016 18:12, Angus Robertson - Magenta Systems Ltd wrote:

Done some testing with a view to investigating this issue, but not
currently sure it's worth any more time.


How about testing against a non-ICS server.
https://posttestserver.com/
https://httpbin.org/
http://putsreq.com/

--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> Are there any known issues with SslHttpCli for sending data? I've 
> observed the following:
> Sending a 3MB picture to imgur:
> * with ICS/OpenSSL: takes ~27 seconds

Done some testing with a view to investigating this issue, but not
currently sure it's worth any more time. 

I'm using an updated version of the sample OverbyteIcsHttpPost against
the ICS web server (no SSL).  

With both running on my PC:
File Size: 20,450,630
Upload Duration: 204 msecs
Speed: 100,248,186 chars/sec
Upload FileName: speed20meg.zip

With the ICS web server running on my public server, over FTTC with a
10Mbit/sec upload speed, from ICS:
File Size: 20,450,630
Upload Duration: 17.56 secs
Speed: 1,164,349 chars/sec
Upload FileName: speed20meg.zip

Using Firefox to the same public server:
File Size: 20,450,630
Upload Duration: 17.53 secs
Speed: 1,166,407 chars/sec
Upload FileName: speed20meg.zip

You can try the public server yourself:

http://www1.telecom-tariffs.co.uk/testing/uploadfile.htm or
https://www1.telecom-tariffs.co.uk/testing/uploadfile.htm

and it will report the upload speeds, as copied above.  Using SSL and
Firefox, got the same speeds exactly, not got an SSL version of the ICS
POST sample, yet.  The web server is running OpenSSL 1.1.0 beta 5 and
unreleased ICS versions, had to change the web server component to get
sensible speed measurements.   

https://www1.telecom-tariffs.co.uk/serverinfo.htm

For completeness, I will make the HTTP client buffers configurable in
size, but I really don't expect to see any real difference in upload
speed, it's seems to be maxed out already.  And this is through two
Sonicwalls that slow things down.  

Angus




-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

You never mentioned you were testing the synchronous methods of the
component, that is not a fair test of ICS performance, you should be
using proper events and the async component methods.


I'm observing the same issue with async calls though, did the same test
from above but this time with PostAsync, waited for RequestDone: 49 seconds.



On Mon, Apr 18, 2016 at 6:06 PM, brian -  wrote:

> Captured with Microsoft Network Monitor, non-SSL post to my server:
>
> Synapse: 3.8s http://pasted.co/1f69a8a8
> ICS: 46.5s
> https://mega.nz/#!b05jyRzT!hLB8_xKHeHJyRb7nAXcQDM2mv0_RW9uf4y-iN3e6gkc
> (too large for text paste sites)
>
> few lines from each:
>
> Frame, time, time offset, process, source, destination, protocol,
> description, conv id
>
> SYN
> 52017:41:32 2016-Apr-1810.0849405Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61535050 - 61536486, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52117:41:32 2016-Apr-1810.0849470Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61536486 - 61537922, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52217:41:32 2016-Apr-1810.1643870Project1.exe**dest
> host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
> DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61391450, Win=1167 (scale
> factor 0x7) = 149376{TCP:75, IPv4:18}
> 52317:41:32 2016-Apr-1810.1644469Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61537922 - 61539358, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52417:41:32 2016-Apr-1810.1644572Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61539358 - 61540794, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52517:41:32 2016-Apr-1810.1647476Project1.exe**dest
> host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
> DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61394322, Win=1167 (scale
> factor 0x7) = 149376{TCP:75, IPv4:18}
> 52617:41:32 2016-Apr-1810.1647476Project1.exe**dest
> host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
> DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61397194, Win=1167 (scale
> factor 0x7) = 149376{TCP:75, IPv4:18}
> 52717:41:32 2016-Apr-1810.1647476Project1.exe**dest
> host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
> DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61402938, Win=1167 (scale
> factor 0x7) = 149376{TCP:75, IPv4:18}
> 52817:41:32 2016-Apr-1810.1647731Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61540794 - 61542230, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52917:41:32 2016-Apr-1810.1647817Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61542230 - 61543666, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
>
> ICS
> 225117:50:17 2016-Apr-18177.9631709Project1.exe
> 192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
> /test/upload.php {HTTP:289, TCP:288, IPv4:18}
> 225217:50:17 2016-Apr-18177.9631750Project1.exe
> 192.168.0.2 **dest host**TCPTCP:[Continuation to
> #2251]Flags=...AP..., SrcPort=58460, DstPort=HTTP(80), PayloadLen=24,
> Seq=458800455 - 458800479, Ack=3147579723, Win=258 (scale factor 0x8) =
> 66048{TCP:288, IPv4:18}
> 225317:50:17 2016-Apr-18177.9631908Project1.exe
> 192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
> /test/upload.php {HTTP:289, TCP:288, IPv4:18}
> 225417:50:17 2016-Apr-18177.9631942Project1.exe
> 192.168.0.2 **dest host**TCPTCP:[Continuation to
> #2253]Flags=...AP..., SrcPort=58460, DstPort=HTTP(80), PayloadLen=24,
> Seq=458801915 - 458801939, Ack=3147579723, Win=258 (scale factor 0x8) =
> 66048{TCP:288, IPv4:18}
> 225517:50:17 2016-Apr-18177.9632252Project1.exe
> 192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
> /test/upload.php {HTTP:289, TCP:288, IPv4:18}
> 225617:50:17 2016-Apr-18177.9632296Project1.exe
> 192.168.0.2 **dest host**TCPTCP:[Continuation to
> #2255]Flags=...AP..., SrcPort=58460, DstPort=HTTP(80), PayloadLen=24,
> Seq=458803375 - 458803399, Ack=3147579723, Win=258 

Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> Captured with Microsoft Network Monitor, non-SSL post to my 
> server:
> Synapse: 3.8s http://pasted.co/1f69a8a8
> ICS: 46.5s

All these measurements have little value when you are not comparing
like with like.  

After my last message, I assume you have redesigned your ICS
application to use async events and not slow sync events.  And what
buffer sizes?  Did you change all the different buffers, including file
I/O?  Or some or none.

For a third time, I will investigate this properly when I have spare
time, probably next month.  

You don't need to keep telling us ICS is slower, we know already.

Angus
 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Captured with Microsoft Network Monitor, non-SSL post to my server:

Synapse: 3.8s http://pasted.co/1f69a8a8
ICS: 46.5s
https://mega.nz/#!b05jyRzT!hLB8_xKHeHJyRb7nAXcQDM2mv0_RW9uf4y-iN3e6gkc (too
large for text paste sites)

few lines from each:

Frame, time, time offset, process, source, destination, protocol,
description, conv id

SYN
52017:41:32 2016-Apr-1810.0849405Project1.exe192.168.0.2
**dest host**TCPTCP:[Continuation to #474]Flags=...A,
SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436, Seq=61535050 - 61536486,
Ack=220751404, Win=258 (scale factor 0x8) = 66048{TCP:75, IPv4:18}
52117:41:32 2016-Apr-1810.0849470Project1.exe192.168.0.2
**dest host**TCPTCP:[Continuation to #474]Flags=...A,
SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436, Seq=61536486 - 61537922,
Ack=220751404, Win=258 (scale factor 0x8) = 66048{TCP:75, IPv4:18}
52217:41:32 2016-Apr-1810.1643870Project1.exe**dest
host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61391450, Win=1167 (scale
factor 0x7) = 149376{TCP:75, IPv4:18}
52317:41:32 2016-Apr-1810.1644469Project1.exe192.168.0.2
**dest host**TCPTCP:[Continuation to #474]Flags=...A,
SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436, Seq=61537922 - 61539358,
Ack=220751404, Win=258 (scale factor 0x8) = 66048{TCP:75, IPv4:18}
52417:41:32 2016-Apr-1810.1644572Project1.exe192.168.0.2
**dest host**TCPTCP:[Continuation to #474]Flags=...A,
SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436, Seq=61539358 - 61540794,
Ack=220751404, Win=258 (scale factor 0x8) = 66048{TCP:75, IPv4:18}
52517:41:32 2016-Apr-1810.1647476Project1.exe**dest
host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61394322, Win=1167 (scale
factor 0x7) = 149376{TCP:75, IPv4:18}
52617:41:32 2016-Apr-1810.1647476Project1.exe**dest
host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61397194, Win=1167 (scale
factor 0x7) = 149376{TCP:75, IPv4:18}
52717:41:32 2016-Apr-1810.1647476Project1.exe**dest
host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61402938, Win=1167 (scale
factor 0x7) = 149376{TCP:75, IPv4:18}
52817:41:32 2016-Apr-1810.1647731Project1.exe192.168.0.2
**dest host**TCPTCP:[Continuation to #474]Flags=...A,
SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436, Seq=61540794 - 61542230,
Ack=220751404, Win=258 (scale factor 0x8) = 66048{TCP:75, IPv4:18}
52917:41:32 2016-Apr-1810.1647817Project1.exe192.168.0.2
**dest host**TCPTCP:[Continuation to #474]Flags=...A,
SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436, Seq=61542230 - 61543666,
Ack=220751404, Win=258 (scale factor 0x8) = 66048{TCP:75, IPv4:18}

ICS
225117:50:17 2016-Apr-18177.9631709Project1.exe
192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
/test/upload.php {HTTP:289, TCP:288, IPv4:18}
225217:50:17 2016-Apr-18177.9631750Project1.exe
192.168.0.2 **dest host**TCPTCP:[Continuation to
#2251]Flags=...AP..., SrcPort=58460, DstPort=HTTP(80), PayloadLen=24,
Seq=458800455 - 458800479, Ack=3147579723, Win=258 (scale factor 0x8) =
66048{TCP:288, IPv4:18}
225317:50:17 2016-Apr-18177.9631908Project1.exe
192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
/test/upload.php {HTTP:289, TCP:288, IPv4:18}
225417:50:17 2016-Apr-18177.9631942Project1.exe
192.168.0.2 **dest host**TCPTCP:[Continuation to
#2253]Flags=...AP..., SrcPort=58460, DstPort=HTTP(80), PayloadLen=24,
Seq=458801915 - 458801939, Ack=3147579723, Win=258 (scale factor 0x8) =
66048{TCP:288, IPv4:18}
225517:50:17 2016-Apr-18177.9632252Project1.exe
192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
/test/upload.php {HTTP:289, TCP:288, IPv4:18}
225617:50:17 2016-Apr-18177.9632296Project1.exe
192.168.0.2 **dest host**TCPTCP:[Continuation to
#2255]Flags=...AP..., SrcPort=58460, DstPort=HTTP(80), PayloadLen=24,
Seq=458803375 - 458803399, Ack=3147579723, Win=258 (scale factor 0x8) =
66048{TCP:288, IPv4:18}
225717:50:17 2016-Apr-18177.9632594Project1.exe
192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
/test/upload.php {HTTP:289, TCP:288, IPv4:18}
225817:50:17 2016-Apr-18177.9632631Project1.exe
192.168.0.2 **dest host**TCPTCP:[Continuation to
#2257]Flags=...AP..., SrcPort=58460, DstPort=HTTP(80), PayloadLen=24,
Seq=458804835 - 458804859, Ack=3147579723, Win=258 (scale factor 0x8) =
66048{TCP:288, IPv4:18}
225917:50:17 2016-Apr-18178.0811555Project1.exe

Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> I think the main issue lies on OverbyteIcsHttpProt around
> 
> while FState <> httpReady do begin
> > {$IFDEF MSWINDOWS}
> > if MsgWaitForMultipleObjects(0, Pointer(nil)^, FALSE, 
> >   1000, QS_ALLINPUT) = WAIT_OBJECT_0 then
> > {$ENDIF}
> >  MessagePump;

You never mentioned you were testing the synchronous methods of the
component, that is not a fair test of ICS performance, you should be
using proper events and the async component methods.   

MsgWaitForMultipleObjects has always been known to reduce performance
in sync applications, I always override it if using sync versions, such
as my TMagFtp and TMagHttp components.  

But without MsgWaitForMultipleObjects, your application will appear to
use 100% CPU (or 50% if two cores, etc) and some users think that is
bad, yet other heavy applications will still run fine.   

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> Do you think there might be an easy solution to this coming soon? 
> the problem is pretty serious, 

I would disagree this a serious problem, it might be annoying that your
ICS implementation runs slower than some other applications, but it's
hardly a major show stopper.  

HTTP was never designed as a high speed upload protocol, the first T
stands for text, FTP is the upload protocol of choice.  Our HTTP
component was designed 20 years ago when high speeds were unheard of
and was not optimised for such use.  

As I said last week, the HTTP client component needs larger buffers and
maybe other minor changes to improve performance, but there are other
ICS priorities that are more important. 

All ICS development is done by volunteers, you need to be patient until
people have spare time to respond to your queries.  

Angus
 

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Some more tests (modified buffer size for the 2nd test)

ICS: ~23s, 377 x 8KB, 55-60ms between packets: http://pastebin.com/Uq5mphUH
ICS @ 64KB: ~22s, 48 x 64KB, 450-470ms between packets:
http://pastebin.com/wTat71Qe
SYN: ~6s, 49 x 64KB, 110-120ms between packets: http://pastebin.com/g5svt11P
Indy: ~9s, 95 x 32KB, 70-80ms between packets: http://pastebin.com/Dt2PydTR

Even with a larger buffer it's still as slow.

On Fri, Apr 15, 2016 at 4:01 AM,  wrote:

> > I updated the FTP client buffers to 32K several years ago for better
> performance.
>
> Yep this was 2011 and it made a huge difference for me.
>
> http://marc.info/?l=twsocket=131146591403948=2
>
> And your response:
>
> Log:
> - Added published property DataSocketSndBufSize and public property
>   DataSocketRcvBufSize. Increase DataSocketSndBufSize in order to
>   make uploads faster. Both values default to value 8192 which is the
>   default winsock size.
>
>
> http://marc.info/?l=twsocket=131152163805666=2
>
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] SSL post extremely slow

[zayin]

> I updated the FTP client buffers to 32K several years ago for better
performance.

Yep this was 2011 and it made a huge difference for me.

http://marc.info/?l=twsocket=131146591403948=2

And your response:

Log:
- Added published property DataSocketSndBufSize and public property
  DataSocketRcvBufSize. Increase DataSocketSndBufSize in order to 
  make uploads faster. Both values default to value 8192 which is the
  default winsock size.


http://marc.info/?l=twsocket=131152163805666=2



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Further testing: to rule out an SSL issue I made a small script to upload
to my server through http, no ssl involved, same 3MB file:

ics 45.8s
indy 4.6s
synapse 4.2s
WinInet 2.2s

On Thu, Apr 14, 2016 at 10:00 AM, Stephen Dickason <sdicka...@elcb.co.za>
wrote:

> I'm not sure if anyone knows, but I can suggest - if you have the time and
> patience to investigate yourself - two tools that I'd use:
> freeware Synopse mORMot framework - has a time Logging profiler
> (synLog.pas) that will show the Delphi program internals times. (It has
> some known naming clashes with some Delphi internals though)
> Magenta Systems Internet Packet Monitoring Components - should allow you
> to check the network traffic coming from your SSL setups (Delphi or
> otherwise) from a separate program and will allow you to try and do some
> deeper investigations as to which part of the communications is taking the
> most time.
>
> Not saying it will be easy and the tools have their own learning curves
> and need for tweaking too.
> HTH
>
>
> Stephen Dickason
> Senior Developer - Managed Services
>   Email: sdicka...@elcb.co.za
>
> ELCB Information Services (Pty) Ltd
> Customer Service Email  e...@elcb.co.za * www.elcb.co.za
> E A S T  L O N D O N
> Tel: +27(43) 704 0700
> Fax: +27(43) 704 0701
> J O H A N N E S B U R G
> Tel: +27(11) 879 6179
> Fax: +27(11) 454 0384
> P O R T  E L I Z A B E T H
> Tel: +27(41) 373 0529
> Fax: +27(86) 650 0135
> Disclaimer
>
> > -Original Message-
> > From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of
> brian
> > -
> > Sent: 14 April 2016 00:26
> > To: ICS support mailing
> > Subject: Re: [twsocket] SSL post extremely slow
> >
> > Synapse (OpenSSL) ~7 seconds.
> >
> > On Wed, Apr 13, 2016 at 2:07 AM, brian - <hikarito...@gmail.com> wrote:
> >
> > > Update: sending through WinInet in my project: 6-7 seconds. What could
> > > be the issue with ICS?
> > >
> > > On Wed, Apr 13, 2016 at 12:47 AM, brian - <hikarito...@gmail.com>
> wrote:
> > >
> > >> Are there any known issues with SslHttpCli for sending data? I've
> > >> observed the following:
> > >>
> > >> Sending a 3MB picture to imgur:
> > >>
> > >> * with ICS/OpenSSL: takes ~27 seconds
> > >> * with Indy/OpenSSL: 8-10 seconds (didn't send correctly though so
> > >> can't confirm 100%, but the post part did take that long nonetheless)
> > >> * another app which I presume uses WinInet rather openssl: 6-7
> > >> seconds
> > >>
> > >> Using OpenSSL 1.0.2f and a very recent ICS from the svn
> > >>
> > >> Any ideas?
> > >>
> > >
> > >
> > --
> > To unsubscribe or change your settings for TWSocket mailing list please
> goto
> > http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> > Visit our website at http://www.overbyte.be
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[Angus Robertson - Magenta Systems Ltd]

> Are there any known issues with SslHttpCli for sending data? 

It seems the component still uses 8K buffers for receiving and sending
data, which was fine 15 years ago when we used slow modems and 10Mb
ethernet, but larger buffers would be more efficient with current
internet speeds.  

I updated the FTP client buffers to 32K several years ago for better
performance.  The HTTP buffer should probably be increased similarly,
or made customised in size to allow the optimum size to be found.  

I'll put it on my long list, unless someone else has the time.  I'm
working on SSL at the moment. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Synapse (OpenSSL) ~7 seconds.

On Wed, Apr 13, 2016 at 2:07 AM, brian -  wrote:

> Update: sending through WinInet in my project: 6-7 seconds. What could be
> the issue with ICS?
>
> On Wed, Apr 13, 2016 at 12:47 AM, brian -  wrote:
>
>> Are there any known issues with SslHttpCli for sending data? I've
>> observed the following:
>>
>> Sending a 3MB picture to imgur:
>>
>> * with ICS/OpenSSL: takes ~27 seconds
>> * with Indy/OpenSSL: 8-10 seconds (didn't send correctly though so can't
>> confirm 100%, but the post part did take that long nonetheless)
>> * another app which I presume uses WinInet rather openssl: 6-7 seconds
>>
>> Using OpenSSL 1.0.2f and a very recent ICS from the svn
>>
>> Any ideas?
>>
>
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Update: sending through WinInet in my project: 6-7 seconds. What could be
the issue with ICS?

On Wed, Apr 13, 2016 at 12:47 AM, brian -  wrote:

> Are there any known issues with SslHttpCli for sending data? I've observed
> the following:
>
> Sending a 3MB picture to imgur:
>
> * with ICS/OpenSSL: takes ~27 seconds
> * with Indy/OpenSSL: 8-10 seconds (didn't send correctly though so can't
> confirm 100%, but the post part did take that long nonetheless)
> * another app which I presume uses WinInet rather openssl: 6-7 seconds
>
> Using OpenSSL 1.0.2f and a very recent ICS from the svn
>
> Any ideas?
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] SSL post extremely slow

[brian -]

Are there any known issues with SslHttpCli for sending data? I've observed
the following:

Sending a 3MB picture to imgur:

* with ICS/OpenSSL: takes ~27 seconds
* with Indy/OpenSSL: 8-10 seconds (didn't send correctly though so can't
confirm 100%, but the post part did take that long nonetheless)
* another app which I presume uses WinInet rather openssl: 6-7 seconds

Using OpenSSL 1.0.2f and a very recent ICS from the svn

Any ideas?
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be