In the context of Adblock, I noticed that our adblock script can handle
relatively well about 10MB of blockfile which is about 7.8% of the
device RAM (128MB), after that the resolution time increases
exponentially to the point where the DNS resolution times-out and more
importantly the device becomes unstable.
I was trying to understand the root cause on why we couldn't have a
larger blockfile which is compiled in local=/example.com/# format. The
RAM, cache and buffer demand stay relatively low when the system becomes
unstable.
In the aim to investigate this further, I run a test creating a bogus
blockfile of the size of about 100MB (and 3.6M lines/domains). Here are
the grep results to fetch info from it:
root@router:/mnt/USB/adblock# time grep 'mytestdomain.com'
/mnt/USB/adblock/test2
mytestdomain.com
real 0m 2.44s
user 0m 0.49s
sys 0m 0.43s
root@router:/mnt/USB/adblock# cp /mnt/USB/adblock/test2 /tmp/test2
root@router:/mnt/USB/adblock# time grep 'mytestdomain.com' /tmp/test2
mytestdomain.com
real 0m 0.65s
user 0m 0.39s
sys 0m 0.20s
What I'm trying to demonstrate here is that a USB2 device can extract a
domain via grep in 2.44sec, and if that file was to be placed in RAM
(/tmp is mapped in RAM on devices with squashfs) it's just 0.65sec.
Admittedly /tmp compresses the content so the 100MB uses about 38MB,
still the point on performance is valid and tells me we could fit
200-250MB blockfile if ever needed, looking at RAM capacity only.
As a point of discussion/improvement, I believe dnsmasq uploads the
custom config (so the blockfile in this case) into RAM, why do we
experience poor resolution performance and system instability at just 10MB?
Considering the system grep is so fast, could this be an alternative
method for dnsmasq to address locally defined domains? If given the
possibility, I would be very happy to map a file in RAM knowing that
this is handled differently from the "standard" conf-file.
I suppose the first step would be to fully understand where the
limitation we currently have comes from.
Then, I'm not suggesting we should re-invent the wheel, but perhaps
there's a margin for a new directive whose behavior is a simple grep
against a mapped file to be used as an authority for those domains?
Might be restricted to blocking only (returning NX or 0.0.0.0 or
127.0.0.1)? Not sure what the secondary implications of such an idea
would be, but I'll be glad to hear some comments/opinions on this topic.
Thanks
_______________________________________________
Dnsmasq-discuss mailing list
Dnsmasq-discuss@lists.thekelleys.org.uk
https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
