Re: [twsocket] send a binary file via https post

[brian -]

Add *libcrypto-1_1.dll* and *libssl-1_1.dll* to the exe folder, then

Uses
>   OverbyteIcsWndControl, OverbyteIcsHttpProt, OverbyteIcsWSocket;
>


> Var h: TSslHttpCli;
>


>   h := TSslHttpCli.Create(nil);
>   With h do begin
> SslContext := TSslContext.Create(h);
> RcvdStream := TMemoryStream.Create;
> SendStream := TFileStream.Create('c:\pathtofilename.zip',fmOpenRead or
> fmShareDenyWrite);
> ContentTypePost := 'application/octet-stream';
> URL := 'https://...';
> Try
>   Post;
> Except
>   On E:Exception do ShowMessage('Error: '+E.Message);
> End;
> SslContext.Free;

RcvdStream.Free; // result is here, you could do
> TStringList.LoadFromStream.. (seek 0,0 first)
> SendStream.Free;

Free;

 end;



On 11 April 2017 at 09:17, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > has someone a simple example to send a zip-file via HTTPS POST?
>
> ICS contains dozens of sample applications, documented in the readme.
> OverbyteIcsHttpPost.dpr posts a file.
>
> Angus
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] What's the purpose of the Accept parameter in HttpCli.OnCookie event?

[brian -]

Hmm I see. It would be nice to add an option for self-contained cookies
within the httpcli object so it would automatically parse and assign the
received cookies. (like the HTTPsend object on the Synapse library)

On 30 March 2017 at 09:32, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > I thought maybe this was a way to automatically set received
> > cookies to the HttpCli, but upon some testing this wasn't the
> > case. Observing the code there doesn't seem to be any purpose to
> > this variable:
> >
> > else if Field = 'set-cookie' then begin
> > > bAccept := TRUE;
> > > TriggerCookie(Data, bAccept);
>
> bAccept is not used within the component, but TriggerCookie is virtual
> function and there may be descendant components that override the
> function and use it.
>
> I added the OverbyteIcsCookies.pas unit a few years ago which does add
> cookie support with TIcsCookies.  The header says:
>
> Provides a memory store for persistent and non-persistent cookies
> captured from the HTTP SetCookie header, and builds a Cookie header for
> any stored cookies for a URL. Persistent cookies are saved and re-read
> from a simple CSV file. OverbyteIcsHttpTst1.pas shows how to handle
> cookies, very few new lines.
>
> Also look at the browser sample project, which is why I wrote it.
>
> Angus
>
>
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] ***SPAM*** Delphi for Linux and ICS

[Brian Hamilton]

Hi

Twsocket for Linux would be great to see what working :)


On 1/03/2017 2:17 AM, Nicolas MARCHAND wrote:

Angus,

Once again I really thank you for your great job.

All our applications target Windows and we have no plan for Linux.


Le 28/02/2017 à 11:23, Angus Robertson - Magenta Systems Ltd a écrit :

Many will be aware the next release of Delphi will be supporting the
Linux compiler, specifically Linux Intel 64-bit for two operating
systems:

http://blog.marcocantu.com/blog/2017-february-delphi-linux-compiler.html


The Linux version will use the LLVM compiler already used for iOS which
ICS supports, so in theory it should not be a major effort to add Linux
support for ICS, easier than Android.   There are changes, like 1 based
strings instead of 0 for mobile.

Unfortunately, all the ICS iOS development was done by Arno who has not
been able to contribute for a few years, neither I nor Francois have
real Linux experience, and I'm really too old to learn new operating
systems or compilers...

So if ICS is to support Linux, we need some assistance from one or more
people, who probably want to implement Delphi server applications on
Linux.  This work can start immediately with a beta of the next Delphi
release.

Any interest in Linux?

Angus







--
Brian Hamilton
http://www.weather-display.com

--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] Twsocket OSX

[Brian Hamilton]

HI

I use the great ICS
and have had great success with twoscket  now, on windows,I can listen for a 
connection and accept that connection (session available) but the same code on 
a mac (OSX), using delphi, the session is avaialble is never reached
is there something I can change/need to change, etc?
thanks!

--
Brian Hamilton
http://www.weather-display.com

--
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] Delay in getting message back from socket server

[Brian Culverwell]

Hi

Hopefully someone has an idea here.

I have a delphi program that calls a PROGRESS 4gl socket server with a
simple message which returns a port number - using the ICS library to send
the message.

When running the PROGRESS 4gl on windows (even on another server from the
Delphi App), it replies in a millisecond.

When calling the PROGRESS 4gl running on a linux box (exact same code, just
compiled for linux), then the message comes back, but only after approx 9
seconds.

Regards
Brian
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Angus, could you share the upload script on your server?

On Thu, May 12, 2016 at 5:35 PM, RTT  wrote:

>
>
> Either way, if the problem is the async nature of ICS,
>> shouldn't an alternate method be provided?
>>
>
> Try with the TransmitFile function
>
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms740565(v=vs.85).aspx
>
> You may create a descendant of the THttpCli, override the SocketDataSent
> method and use the TransmitFile from there.
> Check the Indy IdStackWindows.pas for code (function ServeFile) that uses
> the TransmitFile.
> If this work, maybe it can be integrated in the ICS code.
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

On Thu, May 12, 2016 at 4:24 PM, RTT  wrote:

>
> If that's the case, using SYN, that is sync, to compare against the async
> ICS is not of any help. You need to compare with another async component.



I think all other libraries are blocking, Indy, Synapse, Delphi's own
tcp/ip stuff. Either way, if the problem is the async nature of ICS,
shouldn't an alternate method be provided? We are not talking about a
little difference here, it's huge. Also I'm not sure it makes much sense to
stay async with current hardware/software, things have changed, and it's
very easy to use a blocking socket in a thread to achieve the same.
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Checked again just now, w7:

ICS with 64k buffer: Upload Duration: 33.2 secs / Speed: 137,151 chars/sec
SYN: Upload Duration: 2 secs / Speed: 2,275,855 chars/sec

On Thu, May 12, 2016 at 4:04 PM, brian - <hikarito...@gmail.com> wrote:

> Erm, I thought we all reached a clear conclusion that the issue happens
> from w98 to w7, and whatever causes the problem seems fixed from w8
> onwards. Since all the tests were compared with a different tcp/ip library
> at the exact same time of performing the ICS tests, I think we can rule out
> random temporal issues.
>
> On Thu, May 12, 2016 at 3:48 PM, Angus Robertson - Magenta Systems Ltd <
> an...@magsys.co.uk> wrote:
>
>> > GET seems fine though. You mentioned changing the buffer size,
>> > where would you do that?
>>
>> I've not yet looked closely at the code, once SSL is done.
>>
>> > I tried changing 8k to 64k in OverbyteIcsHttpProt.pas here,
>> > but it made no difference:
>>
>> But then you discovered the low speeds only happened on certain
>> platforms, not all platforms.  Which is exactly my point, modern
>> hardware and perhaps the OS, can effect performance dramatically, as
>> can the internet generally.
>>
>> I do daily speed tests at 5:30am downloading HTTP files from several of
>> my servers, and a few weeks ago the speed tripled without any changes
>> of client or server software or internet access, but some days the
>> speed is back to the original slower pace.  It could be windows updates,
>> my Sonicwall firewalls behaving strangely, or the broadband wholesaler
>> throttling my connection.  Sometimes this stuff can not be rationally
>> explained.
>>
>> Angus
>>
>>
>>
>>
>> --
>> To unsubscribe or change your settings for TWSocket mailing list
>> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
>> Visit our website at http://www.overbyte.be
>>
>
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

GET seems fine though. You mentioned changing the buffer size, where would
you do that? I tried changing 8k to 64k in OverbyteIcsHttpProt.pas here,
but it made no difference:

~4617
procedure THttpCli.SocketDataSent(Sender : TObject; ErrCode : Word);
var
Len : Integer;
begin
if not FAllowedToSend then
Exit;

if Length(FSendBuffer) = 0 then
SetLength(FSendBuffer, 8192);
Len := FSendStream.Read(FSendBuffer[0], Length(FSendBuffer));



On Thu, May 12, 2016 at 2:05 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > Note that the original subject is a bit misleading, the issue is
> > not about ssl POST, but POST in general, whether on http or https.
>
> And my comments were mostly related to GET/POST in general, not
> specifically SSL which one might expect to slow things down, but not
> necessarily.
>
> Angus
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Note that the original subject is a bit misleading, the issue is not about
ssl POST, but POST in general, whether on http or https.

On Thu, May 12, 2016 at 12:13 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > I still have some Windows 7 VMs, so will do more testing next
> > week, once the new OpenSSL stuff is finished.
>
> Not done any changes or real testing yet, except to confirm that HTTPS
> downloads are currently the same speed as HTTP, although one would
> expect the encryption overhead to slow things down.
>
> I have just discovered that all SSL traffic seems to use a fixed 4K
> buffer in wsocket, which I'll make variable sized.
>
> I suspect one reason we are currently seeing good speeds even with low
> buffer sizes is simply that the modern hardware we now use is so
> powerful and it can cope with programming deficiencies without
> performance penalty.
>
> All my clients and servers (bar one) are 64-bit OSs with 16G of memory
> and multi-core i5 or Xeon, my main development PC is SSD drives, less
> important for the servers since files are RAM cached.  And the latest
> CPUs have instructions to speed up SSL encryption.
>
> But this was all different even five years ago and if repeat testing on
> older hardware, I'm sure it will be much slower and the benefits of
> larger buffers and buffered file handling more obvious.  Not sure it's
> worth the effort...
>
> Angus
>
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Here we go,


*98 SE (Delphi7/v5)* VM
ICS: Upload Duration: 34.67 secs/Speed: 131,341 chars/sec
SYN: Upload Duration: 9.05 secs/Speed: 503,314 chars/sec

*2000* VM
ICS: Upload Duration: 34.38 secs/Speed: 132,475 chars/sec
SYN: Upload Duration: 4.67 secs/Speed: 974,531 chars/sec

*XP* VM
ICS: Upload Duration: 34.47 secs/Speed: 132,114 chars/sec
SYN: Upload Duration: 4.7 secs/Speed: 968,109 chars/sec

*Vista* VM
ICS: Upload Duration: 47.39 secs/Speed: 96,093 chars/sec
SYN: Upload Duration: 2.22 secs/Speed: 2,051,345 chars/sec

*2008 R2* VM
ICS: Upload Duration: 34.41 secs/Speed: 132,356 chars/sec
SYN: Upload Duration: 1.97 secs/Speed: 2,312,842 chars/sec

*Win7 (main pc)*
ICS: Upload Duration: 34.39 secs/132,417 chars/sec
SYN: Upload Duration: 2.03 secs/Speed: 2,241,135 chars/sec

*Win8* VM
ICS: Upload Duration: 2.02 secs/Speed: 2,258,922 chars/sec
SYN: Upload Duration: 1.83 secs/Speed: 2,489,878 chars/sec

*Win8.1* VM
ICS: Upload Duration: 1.84 secs/Speed: 2,469,624 chars/sec
SYN: Upload Duration: 1.86 secs/Speed: 2,448,380 chars/sec

*2012 R2* VM
ICS: Upload Duration: 4.58 secs/Speed: 994,537 chars/sec
SYN: Upload Duration: 4.53 secs/Speed: 1,004,629 chars/sec

*Win10* *(main pc)*
ICS: Upload Duration: 1.88 secs/ 2,427,498 chars/sec
SYN: Upload Duration: 2.02 secs/Speed: 2,257,802 chars/sec

*Ubuntu16* *(main pc)*
ICS: Upload Duration: 1.41 secs/Speed: 3,236,664 chars/sec
SYN: Upload Duration: 1.41 secs/Speed: 3,234,365 chars/sec


On Thu, Apr 28, 2016 at 12:51 AM, RTT  wrote:

> On 27/04/2016 18:12, Angus Robertson - Magenta Systems Ltd wrote:
>
>> WinInet is a special case and probably handles TCP/IP as a driver or
>> something we can not do.
>>
>
> Maybe similar performance can be achieved with the Winsock TransmitFile
> and TransmitPackets functions.
>
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

On Wed, Apr 27, 2016 at 8:46 PM, brian - <hikarito...@gmail.com> wrote:

>
>
>> I still have some Windows 7 VMs, so will do more testing next week,
>> once the new OpenSSL stuff is finished.
>>
>
>
> These tests on VMs are flawed though, as observed in all my testing,
> should be done on a real W7 host.
>


Actually nvm that, I'm able to replicate the issue in VMs by changing the
default Network adapter configuration from NAT to BRIDGED (vmware). I'll
try to test W8/8.1/Vista/2008
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

On Wed, Apr 27, 2016 at 8:28 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

>
> I still have some Windows 7 VMs, so will do more testing next week,
> once the new OpenSSL stuff is finished.
>


These tests on VMs are flawed though, as observed in all my testing, should
be done on a real W7 host.
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Alright I'm back with some interesting results that confirm my suspicions
(actually meant 300/30 Mbit btw)

* All VM tests (xp/w7/w10/ubuntu) showed similar results around 0.9..1MB/s
upload, except when using a UK VPN: ICS was again slow.

* Win XP on a netbook on wifi (Asus eee901): ICS: 65.8s. SYN: 6.5s

** Fresh Win7 on my PC: *
- ICS: Upload Duration: 34.39 secs/132,417 chars/sec
- SYN: Upload Duration: 2.03 secs/Speed: 2,241,135 chars/sec

** Fresh Win10 on my PC**
- ICS: Upload Duration: 1.88 secs/ 2,427,498 chars/sec
- SYN: Upload Duration: 2.02 secs/Speed: 2,257,802 chars/sec

** Fresh Ubuntu16 on my PC: (wine)*
- ICS: Upload Duration: 1.41 secs/Speed: 3,236,664 chars/sec
- SYN: Upload Duration: 1.41 secs/Speed: 3,234,365 chars/sec
- Firefox: Upload Duration: 1.35 secs/Speed: 3,385,863 chars/sec

As for the GET, the 2 ICS links seem a bit slower than the IIS.
Did a GET test on a 5MB file: http://download.thinkbroadband.com/5MB.zip
Both ICS and SYN took around 4.1-4.3s, no issues with http downloads I
think.

My conclusion so far:

Up to Windows 7, ICS is interacting with Winsock in a manner that causes
such extreme low post speeds, and it seems to be "fixed" in Windows 10
(possibly since W8?)
The VM tests showed similar results on all platforms, most likely due to
~negating the "wrong" way ICS is interacting with the server, possibly
something like.. VM->ICS posts data to locahost->VMware bridge with host
captures this data and sends it a different way to the actual server,
negating the root ICS issue, sending data its own way no matter the guest
platform.

Also interesting how much faster the Linux tests were :O

Did you compile you application with "release" config, and optimization and
> without any debugging tool enabled such as madExcept or FastMM debug mode ?
>

Yeah, you can check in the test app I linked a few posts earlier,
release/no exception handler/no FastMM.

edit: I'm about to install a fresh Win8 now, will post back.

On Wed, Apr 27, 2016 at 2:05 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > So I'm kinda baffled by these results, not sure what to make of it.
>
> I do know from long experience that ruling out a single PC is the first
> thing I do when testing stuff like this.  But even then, sometimes
> results simply can never be explained.
>
> I have speed test downloads from my public servers every morning, and
> have known for a long time that HTTP is sometimes slower than it should
> be, although today it seems rather good.  I have two broadband
> connections, 40/10 and 152/12 but use the former for most speed tests,
> although I did download 10.1 Berlin ISO at 100Mb last week which would
> excellent.
>
> You have only been doing uploads so far, but I will be mainly testing
> downloads when I fix the buffers, using:
>
> http://www.telecom-tariffs.co.uk/testing/speed50meg.zip
> http://www.magsys.co.uk/download/testing/speed50meg.zip
> http://www1.telecom-tariffs.co.uk/testing/speed50meg.zip
> http://www1.magsys.co.uk/download/testing/speed50meg.zip
>
> The first URL is ICS, second IIS, www is Windows 2012 R2, www1 is 2008.
> As I said earlier, these servers are behind a slowish firewall since I
> get continual hacking attempts and Windows servers are so open to
> attack, the set-up was never designed for high speed testing.
>
> Angus
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Yeah, I'm on 100/30 Mbit fiber, Windows 7 x64 and the latest V8 SVN with
Delphi Seattle. Some days ago I tried an old V5 with Delphi7 to rule out
this problem having been recently introduced, but I observed the same
results. Since you mentioned Windows 10, I thought of making a few extra
tests with various OS's and conditions, with some odd results:

Win7 default:
ICS: Upload Duration: 34.27 secs. Speed: 132,897 chars/sec
SYN: Upload Duration: 2.13 secs. Speed: 2,142,044 chars/sec

Few virtual machines with VMware:
Win7 VM: ICS: Speed: 931,095 chars/sec. SYN: Speed: Speed: 933,959 chars/sec
Win7 VM on UK VPN: ICS: Speed: 88,586 chars/sec. SYN: Speed: 869,745
chars/sec
Win XP VM: Speed: 939,935 chars/sec. SYN: Speed: 943,049 chars/sec
Win10 VM: Speed: Speed: 936,841 chars/sec. Speed: 933,959 chars/sec

Asked a german friend to do the same test, to rule out ISP issues, with him
being on w7:
ICS: Speed: 180,234 chars/sec
Synapse: Speed: 692,200 chars/sec

Tried with my phone's 4G: (shared connection through USB)
Firefox: Upload Duration: 2.72 secs. Speed: 1,674,259 chars/sec
ICS: Upload Duration: 53.56 secs. Speed: 85,021 chars/sec
SYN: Upload Duration: 2.25 secs. Speed: 2,023,095 chars/sec



*Kinda unrelated, but noticed also a weird result from various
browsers:Chrome: Speed: 621,280 chars/sec*

*Firefox: Speed: 1,844,466 chars/sec*
*IE11: Speed: 2,206,389 chars/sec*


So I'm kinda baffled by these results, not sure what to make of it. Ruled
out this being an issue with w7/intel driver (my friend seems to have a
board with intel NIC as well) by testing with 4G. Ruled out ISP issue by
having my friend in Germany do the same test. But then why are the VMs
having those results? (same speed ics/syn). Also the VPN on the w7 VM
experiencing the same issue on ICS is very very weird. Could it be an issue
specifically in my machine? but then having my friend experience the same
is quite a coincidence, and it wouldn't explain why SYN works just fine.

Next I will test on a clean install of both W7 and W10 without a VM on my
machine, I'll post back with results (probably a few hours, gonna install
them on a spare 5400rpm 2,5" disk).

On Wed, Apr 27, 2016 at 9:47 AM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > Memo isn't touched during the progress, only the progressbar.
> > Removed OnSenData event, same results.
>
> You appear to be on a Spanish RIMA (Red IP Multi Acceso) network,
> although trace route does not complete.
>
> Trace Route to: 83.39.166.229
> Can Not Ping Host (83.39.166.229) : Request timed out
>  1 0ms  Unexpected Family  MAGPUB1
>  2 0ms  217.146.102.129217.146.102.129
>  3 12ms 217.146.96.72  gi0.man-gw2.merula.net
>  4 23ms 212.187.137.161te-8-3-215.car2.manchester1.level3.net
>  5 19ms 212.187.160.230telefonica.edge6.london1.level3.net
>  6 50ms 84.16.13.25 xe-grtbcntb1.net.telefonicaglobalsolutions.com
>  7 54ms 94.142.103.178  x0-grtbcntb1.net.telefonicaglobalsolutions.com
>  8 60ms 80.58.88.210210.red-80-58-88.staticip.rima-tde.net
>  9 Request timed out
>
> Your best speed was 1,928,838 chars/sec which suggests you have at
> least a 20Mb/sec upload speed which is about double the speed from my
> office.  This old public server is still on 100Mb/sec ethernet since
> the firewall is quite slow.
>
> The only thing that can explain the speed difference from the same PC
> is TCP/IP packet size, somehow the ICS application is sending more
> smaller packets than some others, and that is probably explained by the
> 8K buffer size currently used by the HTTP client, although it's
> surprising that I get 10 times your speed with the same software,
> assuming you are using Windows 10 and ICS V8.27 which you have never
> mentioned.
>
> Unless ICS is setting some strange TCP setting, buffer size is the only
> thing that can be changed, which I will look at next month, as I've
> said before.  That should allow Windows to send larger IP packets,
> which should help speed.
>
> Unless you have a local firewall or proxy that is crippling speeds.
>
> Angus
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

You never mentioned you were testing the synchronous methods of the
component, that is not a fair test of ICS performance, you should be
using proper events and the async component methods.


I'm observing the same issue with async calls though, did the same test
from above but this time with PostAsync, waited for RequestDone: 49 seconds.



On Mon, Apr 18, 2016 at 6:06 PM, brian - <hikarito...@gmail.com> wrote:

> Captured with Microsoft Network Monitor, non-SSL post to my server:
>
> Synapse: 3.8s http://pasted.co/1f69a8a8
> ICS: 46.5s
> https://mega.nz/#!b05jyRzT!hLB8_xKHeHJyRb7nAXcQDM2mv0_RW9uf4y-iN3e6gkc
> (too large for text paste sites)
>
> few lines from each:
>
> Frame, time, time offset, process, source, destination, protocol,
> description, conv id
>
> SYN
> 52017:41:32 2016-Apr-1810.0849405Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61535050 - 61536486, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52117:41:32 2016-Apr-1810.0849470Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61536486 - 61537922, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52217:41:32 2016-Apr-1810.1643870Project1.exe**dest
> host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
> DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61391450, Win=1167 (scale
> factor 0x7) = 149376{TCP:75, IPv4:18}
> 52317:41:32 2016-Apr-1810.1644469Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61537922 - 61539358, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52417:41:32 2016-Apr-1810.1644572Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61539358 - 61540794, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52517:41:32 2016-Apr-1810.1647476Project1.exe**dest
> host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
> DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61394322, Win=1167 (scale
> factor 0x7) = 149376{TCP:75, IPv4:18}
> 52617:41:32 2016-Apr-1810.1647476Project1.exe**dest
> host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
> DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61397194, Win=1167 (scale
> factor 0x7) = 149376{TCP:75, IPv4:18}
> 52717:41:32 2016-Apr-1810.1647476Project1.exe**dest
> host**192.168.0.2TCPTCP:Flags=...A, SrcPort=HTTP(80),
> DstPort=58338, PayloadLen=0, Seq=220751404, Ack=61402938, Win=1167 (scale
> factor 0x7) = 149376{TCP:75, IPv4:18}
> 52817:41:32 2016-Apr-1810.1647731Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61540794 - 61542230, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
> 52917:41:32 2016-Apr-1810.1647817Project1.exe
> 192.168.0.2**dest host**TCPTCP:[Continuation to
> #474]Flags=...A, SrcPort=58338, DstPort=HTTP(80), PayloadLen=1436,
> Seq=61542230 - 61543666, Ack=220751404, Win=258 (scale factor 0x8) =
> 66048{TCP:75, IPv4:18}
>
> ICS
> 225117:50:17 2016-Apr-18177.9631709Project1.exe
> 192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
> /test/upload.php {HTTP:289, TCP:288, IPv4:18}
> 225217:50:17 2016-Apr-18177.9631750Project1.exe
> 192.168.0.2 **dest host**TCPTCP:[Continuation to
> #2251]Flags=...AP..., SrcPort=58460, DstPort=HTTP(80), PayloadLen=24,
> Seq=458800455 - 458800479, Ack=3147579723, Win=258 (scale factor 0x8) =
> 66048{TCP:288, IPv4:18}
> 225317:50:17 2016-Apr-18177.9631908Project1.exe
> 192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
> /test/upload.php {HTTP:289, TCP:288, IPv4:18}
> 225417:50:17 2016-Apr-18177.9631942Project1.exe
> 192.168.0.2 **dest host**TCPTCP:[Continuation to
> #2253]Flags=...AP..., SrcPort=58460, DstPort=HTTP(80), PayloadLen=24,
> Seq=458801915 - 458801939, Ack=3147579723, Win=258 (scale factor 0x8) =
> 66048{TCP:288, IPv4:18}
> 225517:50:17 2016-Apr-18177.9632252Project1.exe
> 192.168.0.2 **dest host**HTTPHTTP:HTTP Payload, URL:
> /test/upload.php {HTTP:289, TCP:288, IPv4:18}
>

Re: [twsocket] SSL post extremely slow

[brian -]

**dest
host**192.168.0.2 TCPTCP:Flags=...A, SrcPort=HTTP(80),
DstPort=58460, PayloadLen=0, Seq=3147579723, Ack=458800479, Win=501 (scale
factor 0x7) = 64128{TCP:288, IPv4:18}
226017:50:17 2016-Apr-18178.0811555Project1.exe**dest
host**192.168.0.2 TCPTCP:Flags=...A, SrcPort=HTTP(80),
DstPort=58460, PayloadLen=0, Seq=3147579723, Ack=458801939, Win=501 (scale
factor 0x7) = 64128{TCP:288, IPv4:18}



On Mon, Apr 18, 2016 at 5:14 PM, brian - <hikarito...@gmail.com> wrote:

> Hm not sure if you can see my previous thread messages, I did some little
> analysis:
>
> ICS: ~23s, 377 x 8KB, 55-60ms between packets:
> http://pastebin.com/Uq5mphUH
> ICS @ 64KB: ~22s, 48 x 64KB, 450-470ms between packets:
> http://pastebin.com/wTat71Qe
> SYN: ~6s, 49 x 64KB, 110-120ms between packets:
> http://pastebin.com/g5svt11P
> Indy: ~9s, 95 x 32KB, 70-80ms between packets:
> http://pastebin.com/Dt2PydTR
> WinInet: ~6s, 190 x ~16KB, 5-55ms between packets,
> http://pastebin.com/jZLvua5u
>
> then tried to log timings through various stages of the post, albeit not
> being that helpful, I think the main issue lies on OverbyteIcsHttpProt
> around
>
> while FState <> httpReady do begin
>>   {$IFDEF MSWINDOWS}
>> if MsgWaitForMultipleObjects(0, Pointer(nil)^, FALSE, 1000,
>>  QS_ALLINPUT) = WAIT_OBJECT_0 then
>>   {$ENDIF}
>> MessagePump;
>>
>
> I'll check on wireshark.
>
>
> On Mon, Apr 18, 2016 at 4:44 PM, RTT <p...@sapo.pt> wrote:
>
>> On 18/04/2016 15:15, brian - wrote:
>>
>>> I know ICS is made for free by volunteers, I'm not expecting/demanding a
>>> fix, but I think this is an issue that we could look into to try to find
>>> a
>>> solution/find the core problem together.
>>>
>>
>> To start, you could compare the characteristics of the packets being sent
>> (e.g. using Wireshark), and the times involved, so we can get a general
>> idea of what is being done differently in these HTTP components. Maybe this
>> general analyzes offers the clues for what code needs to be
>> optimized/profiled.
>>
>> --
>> To unsubscribe or change your settings for TWSocket mailing list
>> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
>> Visit our website at http://www.overbyte.be
>>
>
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Some more tests (modified buffer size for the 2nd test)

ICS: ~23s, 377 x 8KB, 55-60ms between packets: http://pastebin.com/Uq5mphUH
ICS @ 64KB: ~22s, 48 x 64KB, 450-470ms between packets:
http://pastebin.com/wTat71Qe
SYN: ~6s, 49 x 64KB, 110-120ms between packets: http://pastebin.com/g5svt11P
Indy: ~9s, 95 x 32KB, 70-80ms between packets: http://pastebin.com/Dt2PydTR

Even with a larger buffer it's still as slow.

On Fri, Apr 15, 2016 at 4:01 AM,  wrote:

> > I updated the FTP client buffers to 32K several years ago for better
> performance.
>
> Yep this was 2011 and it made a huge difference for me.
>
> http://marc.info/?l=twsocket=131146591403948=2
>
> And your response:
>
> Log:
> - Added published property DataSocketSndBufSize and public property
>   DataSocketRcvBufSize. Increase DataSocketSndBufSize in order to
>   make uploads faster. Both values default to value 8192 which is the
>   default winsock size.
>
>
> http://marc.info/?l=twsocket=131152163805666=2
>
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Further testing: to rule out an SSL issue I made a small script to upload
to my server through http, no ssl involved, same 3MB file:

ics 45.8s
indy 4.6s
synapse 4.2s
WinInet 2.2s

On Thu, Apr 14, 2016 at 10:00 AM, Stephen Dickason <sdicka...@elcb.co.za>
wrote:

> I'm not sure if anyone knows, but I can suggest - if you have the time and
> patience to investigate yourself - two tools that I'd use:
> freeware Synopse mORMot framework - has a time Logging profiler
> (synLog.pas) that will show the Delphi program internals times. (It has
> some known naming clashes with some Delphi internals though)
> Magenta Systems Internet Packet Monitoring Components - should allow you
> to check the network traffic coming from your SSL setups (Delphi or
> otherwise) from a separate program and will allow you to try and do some
> deeper investigations as to which part of the communications is taking the
> most time.
>
> Not saying it will be easy and the tools have their own learning curves
> and need for tweaking too.
> HTH
>
>
> Stephen Dickason
> Senior Developer - Managed Services
>   Email: sdicka...@elcb.co.za
>
> ELCB Information Services (Pty) Ltd
> Customer Service Email  e...@elcb.co.za * www.elcb.co.za
> E A S T  L O N D O N
> Tel: +27(43) 704 0700
> Fax: +27(43) 704 0701
> J O H A N N E S B U R G
> Tel: +27(11) 879 6179
> Fax: +27(11) 454 0384
> P O R T  E L I Z A B E T H
> Tel: +27(41) 373 0529
> Fax: +27(86) 650 0135
> Disclaimer
>
> > -Original Message-
> > From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of
> brian
> > -
> > Sent: 14 April 2016 00:26
> > To: ICS support mailing
> > Subject: Re: [twsocket] SSL post extremely slow
> >
> > Synapse (OpenSSL) ~7 seconds.
> >
> > On Wed, Apr 13, 2016 at 2:07 AM, brian - <hikarito...@gmail.com> wrote:
> >
> > > Update: sending through WinInet in my project: 6-7 seconds. What could
> > > be the issue with ICS?
> > >
> > > On Wed, Apr 13, 2016 at 12:47 AM, brian - <hikarito...@gmail.com>
> wrote:
> > >
> > >> Are there any known issues with SslHttpCli for sending data? I've
> > >> observed the following:
> > >>
> > >> Sending a 3MB picture to imgur:
> > >>
> > >> * with ICS/OpenSSL: takes ~27 seconds
> > >> * with Indy/OpenSSL: 8-10 seconds (didn't send correctly though so
> > >> can't confirm 100%, but the post part did take that long nonetheless)
> > >> * another app which I presume uses WinInet rather openssl: 6-7
> > >> seconds
> > >>
> > >> Using OpenSSL 1.0.2f and a very recent ICS from the svn
> > >>
> > >> Any ideas?
> > >>
> > >
> > >
> > --
> > To unsubscribe or change your settings for TWSocket mailing list please
> goto
> > http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> > Visit our website at http://www.overbyte.be
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Synapse (OpenSSL) ~7 seconds.

On Wed, Apr 13, 2016 at 2:07 AM, brian - <hikarito...@gmail.com> wrote:

> Update: sending through WinInet in my project: 6-7 seconds. What could be
> the issue with ICS?
>
> On Wed, Apr 13, 2016 at 12:47 AM, brian - <hikarito...@gmail.com> wrote:
>
>> Are there any known issues with SslHttpCli for sending data? I've
>> observed the following:
>>
>> Sending a 3MB picture to imgur:
>>
>> * with ICS/OpenSSL: takes ~27 seconds
>> * with Indy/OpenSSL: 8-10 seconds (didn't send correctly though so can't
>> confirm 100%, but the post part did take that long nonetheless)
>> * another app which I presume uses WinInet rather openssl: 6-7 seconds
>>
>> Using OpenSSL 1.0.2f and a very recent ICS from the svn
>>
>> Any ideas?
>>
>
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] "an archive extracted daily from the repository" is this updated?

[brian -]

Why isn't http://www.overbyte.be updated or redirected to the Wiki page?

On Wed, Apr 13, 2016 at 4:47 PM, Angus Robertson - Magenta Systems Ltd <
an...@magsys.co.uk> wrote:

> > The ICS page has links to what's suposed to be a daily snapshot
> > of the svn,
> > however, comparing the files from those links to the svn I see
> > they are not
>
> The download page has always been:
> http://wiki.overbyte.be/wiki/index.php/ICS_Download
>
> where you can view the changes log for the latest daily snapshots,
> which was V8.26 on 23rd March 2015, there is nothing newer at the
> moment.
>
> The zip files are rebuilt every night from my SVN server:
>
> IcsV7w.zip 3,880,588A 12/04/2016 22:47:19  \
> IcsV8w.zip 7,409,200A 12/04/2016 22:47:39  \
>
> There is no historic zip archives, although some major versions do get
> saved with versions as 'stable' versions and some are specials for the
> Delphi GetIt package auto installer.
>
> Angus
>
>
>
> --
> To unsubscribe or change your settings for TWSocket mailing list
> please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
> Visit our website at http://www.overbyte.be
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] SSL post extremely slow

[brian -]

Update: sending through WinInet in my project: 6-7 seconds. What could be
the issue with ICS?

On Wed, Apr 13, 2016 at 12:47 AM, brian - <hikarito...@gmail.com> wrote:

> Are there any known issues with SslHttpCli for sending data? I've observed
> the following:
>
> Sending a 3MB picture to imgur:
>
> * with ICS/OpenSSL: takes ~27 seconds
> * with Indy/OpenSSL: 8-10 seconds (didn't send correctly though so can't
> confirm 100%, but the post part did take that long nonetheless)
> * another app which I presume uses WinInet rather openssl: 6-7 seconds
>
> Using OpenSSL 1.0.2f and a very recent ICS from the svn
>
> Any ideas?
>
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] SSL post extremely slow

[brian -]

Are there any known issues with SslHttpCli for sending data? I've observed
the following:

Sending a 3MB picture to imgur:

* with ICS/OpenSSL: takes ~27 seconds
* with Indy/OpenSSL: 8-10 seconds (didn't send correctly though so can't
confirm 100%, but the post part did take that long nonetheless)
* another app which I presume uses WinInet rather openssl: 6-7 seconds

Using OpenSSL 1.0.2f and a very recent ICS from the svn

Any ideas?
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be