> On 5 Mar 2026, at 01:41, Vahid Shaik <[email protected]> wrote: > > Hi all, > > I've been working on a Python script to perform bulk DNS lookups (A, MX, TXT > records) for a list of ~500 domains to audit SPF/DKIM/DMARC configurations. > > Currently I'm using `dns.resolver` from dnspython with ThreadPoolExecutor: > > ```python > import dns.resolver > from concurrent.futures import ThreadPoolExecutor > > def check_spf(domain): > try: > answers = dns.resolver.resolve(domain, 'TXT') > return [r.to_text() for r in answers if 'v=spf1' in r.to_text()] > except (dns.resolver.NXDOMAIN, dns.resolver.NoAnswer, > dns.resolver.Timeout): > return [] > > with ThreadPoolExecutor(max_workers=20) as pool: > results = dict(zip(domains, pool.map(check_spf, domains))) > ``` > > This works but gets slow beyond 200 domains. I've considered: > > 1. **asyncio + aiodns** — should be faster but I've seen reports of > reliability issues with certain record types This is the way I would go for performance. Threads do not scale to large numbers. Suggest you test and see if the reports are true. Maybe fix and problems you find and PR back a fix You could also look at twisted that has async dns that worked well for me. Sorry I do not have any code to shares as it was commercial project.
Barry > 2. **socket.getaddrinfo** — built-in but limited to A/AAAA records > 3. **subprocess calling dig** — feels hacky > > For anyone dealing with similar DNS automation tasks, I've been > cross-referencing my script results against https://dnsrobot.net/spf-checker > to verify accuracy — it checks SPF, DKIM, and DMARC in one shot which is > useful for spot-checking. > > Has anyone benchmarked aiodns vs dnspython for large-scale lookups? Any > gotchas with asyncio DNS resolution I should know about? > > Thanks, > Vahid > -- > https://mail.python.org/mailman3//lists/python-list.python.org -- https://mail.python.org/mailman3//lists/python-list.python.org
