> OK, but as you rightly pointed out a couple of messages ago, all these
> operations are performed before the service dnsmasq restart.
>
> Because of this I have performed an additional test to complete the
> picture:
>
> my usual 650K list of domains is now unzipped and I have simply scripted
> the usual formatting with address=/domain/. So no compression involved
> this time. e.g.
>
> /bin/cat /mnt/USB/adblock/adblock.domains | /bin/sed -e "s:^:address=/:"
> -e "s:$:/:"
>
>
> The service dnsmasq restart went from 15.something sec of the script+zip
> version to 14.4 sec of this script only version.
> root at sparrow:/tmp/mnt/USB/adblock# time service dnsmasq restart
>
................................................................................................................................................
> Done.
> real 0m 14.43s
> user 0m 0.00s
> sys 0m 0.00s
>
> What I was trying to say is that feeding pre-formatted config (as per
> prev message posted here) takes 4.2 sec on the very same file.
>
> And you're right the aim here is to minimize storage hence RAM demand,
> but the dnsmasq restart time must also stay within acceptable levels as
> nobody wants the DNS resolution to be down for too long.
>
> So to summarize: Because the conf-script option works great on RAM
> reduction but takes a big hit on the restart time, I was suggesting if a
> different approach could/should be considered. Since we learnt from this
> last test that the zip operation add only 1-2 sec of delay on the restart
>
> root at sparrow:/mnt/USB/adblock# time gzip -d
adblock.domains.unloaded.gz
> real 0m 1.76s
> user 0m 0.67s
> sys 0m 0.18s
>
> perhaps allowing the conf-file to process zipped content (internal zcat
> or something) would suffice to achieve the desired result to minimize
> storage demand and retain restart time within acceptable levels.
> Allowing something like conf-block and import domain only is indeed
> nice-to-have but secondary in my opinion compared to support for
> compressed config:
>
> root at sparrow:/mnt/USB/adblock# ls -lh adblock.domains.unloaded
> -rwxrwxrwx 1 root root 14.3M Mar 31 18:42
> root at sparrow:/mnt/USB/adblock# gzip adblock.domains.unloaded
> root at sparrow:/mnt/USB/adblock# ls -lh adblock.domains.unloaded.gz
> -rwxrwxrwx 1 root root 4.2M Mar 31 18:42
Simon, it is not my intention to put pressure or else but I just wanted
to give a final feedback on this topic. The conf-script is not usable on
a router for large lists/config. It simply takes dnsmasq far too long to
restart. There are a series of event that restart dnsmasq from a simple
Save Config from the GUI to an automatic event e.g. a multiWAN swap due
to external factors and anything in between. on a 1M record blocklist it
takes dnsmasq almost 30 before before it can resolve domains again.
On one side I'm grateful this was considered by the dnsmasq team, on the
other side I still believe (as per message above) that allowing dnsmasq
to read natively a conf-file in .gz would actually be the simplest and
possibly most effective of the solutions to resolve the large list
issue. On plain config (conf-file) even a huge file takes dnsmasq 5-6
seconds to restart. The gzip operation on such file are about 1 sec on
top which is totally an acceptable compromise. Finally the size of a 1M
records conf-file compresses down from 40M to 8Mb.
A win win.
I'm not sure if you still have interest in tackling this topic but I
felt a feedback is expected as a minimum given all the work done so far.
It would be good to know either ways if you don't mind.
Thanks!
_______________________________________________
Dnsmasq-discuss mailing list
Dnsmasq-discuss@lists.thekelleys.org.uk
https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
