> > Hey Simon, > > On Thu, 2021-04-01 at 23:55 +0100, Simon Kelley wrote: > > I could do with a handle on exactly how people are configuring dnsmasq > > to do ad blocking. It's not something I have much experience of. > > The situation for Pi-hole (a popular ad blocker based on dnsmasq) is the > following: > > Traditionally, Pi-hole used "addn-hosts" to add HOSTS-like files containing > domains (example: > https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts). This > list contains roughly 80,000 domains. This is doable with dnsmasq on all > platforms having at least 512 MB of memory. However, Pi-hole users > typically want to take it to the extremes. They added more and more lists, > often going beyond the one million domains mark. This became a problem > regarding memory. I don't recall complains about slow replies, though. > > Anyway, as this became more and more an issue and since we wanted to have > something more professional than a text file (so users can easily add > comments, etc.), we amended the dnsmasq code with an interface to a SQLite3 > database holding all domains to be blocked. We also added support for > regular expressions (and hereby wildcards). With this new approach, we > stopped storing anything about blocked domains in dnsmasq's cache: blocked > domains are short-circuited and replied to with a mock answer. They are > never added to the cache. This is done because we allow different lists to > be assigned to different clients so some devices using the DNS server can > be limited further down while other may be fully open on the same process. > > This works really fast because the balanced-tree (B-tree) index on the > domain is very efficient. The tree lives transparently in page cache so > accessing it is very fast even in the 1 mio. range (lookup speed scales > logarithmic, typically < 5 ms on Raspberry Pis for 3mio. blocked domains). > > Note that we are hooking into dnsmasq's code from "outside" to keep changes > in the dnsmasq codebase minimal so we can straightaway apply any patches > from dnsmasq's git. > > So even when this is a bit outside of the current discussion, I thought > it'd be interesting to mention that Pi-hole used to use "addn-hosts" but > stopped to do so some time ago. > > Best, > Dominik > > I was indeed in pi-hole's dnsmasq changes a while go and tried to use it to replace dnsmasq(for its sqlite3, cname etc), however that turns out to be too challenging as they're really geared towards pi-holes specifically. Especially the way it forks dnsmasq. I would be great if pihole's dnsmasq changes can be used standalone(e.g. a dnsmasq variant with sqlite3/cname-nesting etc).
Though the performance issue I mentioned in my last reply about local/address/cname parsing remains the same, you will have to use hosts files to a quick reload when you have large blocklists. Thanks, Gordon
_______________________________________________ Dnsmasq-discuss mailing list Dnsmasq-discuss@lists.thekelleys.org.uk https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss