Re: [twsocket] TWSocket.Connet/GetHostByName bottleneck

2015-09-18 Thread Angus Robertson - Magenta Systems Ltd 
> At this time I see this way:
> 
> 1. Use TDNSQuery.MXLookup to resolve MX, then TDNSQuery.ALookup to 
> resolve MX's IP. Then supply resolved MX IP to TSmtpCli to avoid 
> GetHostByName.
> 
> 2. Cache both MX and A results in my own DNS cache.

Seems sensible, just make sure you time out the cached entries. 

Most of the client components are designed for that use, not to be used in 
servers.

 
> But maybe you can suggest more efficient way ? Maybe some tuning or 
> secret option in ICS ? 

No secret options.

I have been planning a relaying demo combining the SMTP server I wrote a couple 
of
years ago and Magenta Mail Queue which queues sending, but not got around to 
adding
MX checking to the latter. 

Angus

-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

Re: [twsocket] TWSocket.Connet/GetHostByName bottleneck

2015-09-17 Thread Stephen Dickason 
Just a non-well-informed thought:
Are you not able to cache the results and combine multiple requests that would 
return the same results into a single first call and subsequent requests would 
read from the cache?  That is to say comparing 50 calls to 300 calls, the 
likelihood of more of the requests actually asking the same thing and getting 
the same results is higher, so calling the DNSLookup only once for all the 
unique requests should result in a speed improvement.


Stephen Dickason
Senior Developer - Managed Services
  Email: sdicka...@elcb.co.za

ELCB Information Services (Pty) Ltd
Customer Service Email  e...@elcb.co.za * www.elcb.co.za
E A S T  L O N D O N
Tel: +27(43)  704 0700
Fax: +27(43) 704 0701
J O H A N N E S B U R G
Tel: +27(11) 879 6179
Fax: +27(11) 454 0384
P O R T  E L I Z A B E T H
Tel: +27(41) 373 0529
Fax: +27(86) 650 0135
Disclaimer


-Original Message-
From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of Max 
Terentiev
Sent: 18 September 2015 02:07
To: ICS support mailing
Subject: [twsocket] TWSocket.Connet/GetHostByName bottleneck

Hi,

I develop smtp relay server using ICS. And I notice very big performance hit if 
many TSmtpCli's connects to remote MX server in parallel !

I use TDnsQuery to resolve domain's MX server then supply result to TSmtpCli.

If I use 100+ TSmtpCli instances most of them waits for OnRequestDone after 
TSmtpCli.Connect call.

Looks like problem in TWSocket.DnsLookup because they call 
WSocket_Synchronized_WSAAsyncGetHostByName but MSDN say:

"The WSAAsyncGetHostByName function is not designed to provide parallel 
resolution of several names."

So, if 100 TSmtpCli's perform Connect they can't do it in parallel because 
WSAAsyncGetHostByName works sequentially. So 300+ TSmtpCli's works slower than 
50 TSmtpCli's because waits for WSAAsyncGetHostByName queue.

Can you suggest some workaround to avoid this problem ?

At this time I see this way:

1. Use TDNSQuery.MXLookup to resolve MX, then TDNSQuery.ALookup to resolve MX's 
IP. Then supply resolved MX IP to TSmtpCli to avoid GetHostByName.

2. Cache both MX and A results in my own DNS cache.

But maybe you can suggest more efficient way ? Maybe some tuning or secret 
option in ICS ?

Thanks for help !

---
With best regards, Max Terentiev.
Business Software Products.
AMS Development Team.
supp...@bspdev.com



--
To unsubscribe or change your settings for TWSocket mailing list please goto 
http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be
-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be

[twsocket] TWSocket.Connet/GetHostByName bottleneck

2015-09-17 Thread Max Terentiev 
Hi,

I develop smtp relay server using ICS. And I notice very big performance hit
if many TSmtpCli's connects to remote MX server in parallel !

I use TDnsQuery to resolve domain's MX server then supply result to
TSmtpCli.

If I use 100+ TSmtpCli instances most of them waits for OnRequestDone after
TSmtpCli.Connect call.

Looks like problem in TWSocket.DnsLookup because they call
WSocket_Synchronized_WSAAsyncGetHostByName but MSDN say:

"The WSAAsyncGetHostByName function is not designed to provide parallel
resolution of several names."

So, if 100 TSmtpCli's perform Connect they can't do it in parallel because
WSAAsyncGetHostByName works sequentially. So 300+ TSmtpCli's works slower
than 50 TSmtpCli's because waits for WSAAsyncGetHostByName queue.

Can you suggest some workaround to avoid this problem ?

At this time I see this way:

1. Use TDNSQuery.MXLookup to resolve MX, then TDNSQuery.ALookup to resolve
MX's IP. Then supply resolved MX IP to TSmtpCli to avoid GetHostByName.

2. Cache both MX and A results in my own DNS cache.

But maybe you can suggest more efficient way ? Maybe some tuning or secret
option in ICS ? 

Thanks for help !

---
With best regards, Max Terentiev.
Business Software Products.
AMS Development Team.
supp...@bspdev.com



-- 
To unsubscribe or change your settings for TWSocket mailing list
please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket
Visit our website at http://www.overbyte.be