Hi Angus, I think Cloudflare not allow me millions dns lookups for free ))
>is this a server or workstation? Workstation. Windows 10 x64. >Each async UDP or TCP connection causes windows to start a thread, and >this is an overhead probably causing your slow down after 200 instances Is this true for TCP connections ? I will have same slowdown/limits if I create several hundreds TSmtpClient instances ? How to avoid it ? Use IOCP ? --- With best regards, Max Terentiev. Business Software Products. AMS Development Team. supp...@bspdev.com -----Original Message----- From: TWSocket [mailto:twsocket-boun...@lists.elists.org] On Behalf Of Angus Robertson - Magenta Systems Ltd Sent: Tuesday, August 14, 2018 10:18 AM To: twsocket@lists.elists.org Subject: Re: [twsocket] Efficient bulk dns lookup > I need to perform bulk dns lookup (for example get A and MX > records for several millions domains). > > How to use TDnsQuery most effective: > It's possible to reach at least 7000-8000 lookups per second ? > I use about 35 public DNS servers to perform my queries. You are probably reaching windows limits here, is this a server or workstation? The more expensive Windows server versions are able to handle more traffic than normal workstations. Each async UDP or TCP connection causes windows to start a thread, and this is an overhead probably causing your slow down after 200 instances. You may also be reaching the limits of what DNS servers can support, have you tried the Google and Cloudflare versions, they are scaled for massive usage. You might want to experiment with DNS over HTTP, which will open one SSL connection and then allow repeated requests over HTTP, without needing a thread each time. Look at Cloudflare 1.1.1.1: https://developers.cloudflare.com/1.1.1.1/dns-over-https/ The new HTTPS REST OverbyteIcsHttpRestTst sample allows you to make Json DNS look-up requests to Cloudflare. I have a vague plan to create a new HTTP DNS component for ICS allowing WSocket to use this instead of native Windows stuff, it just needs TDnsQuery being updated to use REST, but this won't happen for many months, too many other unfinished projects. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
