Re: Incremental transfers generate complete zone reloading
On 15. 01. 23 19:02, Jesus Cea wrote:
I have a huge zone receiving a constant flow of small dns updates. My
secondaries receive notifications and transfer the zone incrementally.
Cool, everything works as expected.
Nevertheless, I see this lines in my logs, constantly (every time a
change arrives incrementally):
"""
15-Jan-2023 17:49:47.662 general: info: rpz: rpz.local: new zone version
came too soon, deferring update for 28 seconds
15-Jan-2023 17:49:54.716 notify: info: client @11f80268 X.X.X.X#63514:
received notify for zone 'rpz.local'
15-Jan-2023 17:49:54.716 general: info: zone rpz.local/IN: notify from
X.X.X.X#63514: serial 8991
15-Jan-2023 17:50:15.662 general: info: rpz: rpz.local: reload start
15-Jan-2023 17:50:16.884 general: info: rpz: rpz.local: reload done
"""
Ok, my updates are coming too fast (first line). No problem, the
secondary will eventually retrieve the changes. What worries me is the
last couple of lines: The rpz zone (big, around 800.000 domains) is
being reloaded constantly and it takes a couple of seconds eating CPU,
when the incremental changes are actually pretty tiny.
That's correct - this is result of the current implementation of RPZ.
It is tracked as
https://gitlab.isc.org/isc-projects/bind9/-/issues/3746
I would guess the incremental changes would do an incremental change in
memory structures, not a full zone reload taking a couple of seconds and
sucking an entire CPU core.
Updates to "normal" DNS zones are indeed updated incrementally, but RPZ
is different kind of beast.
My secondary configuration is pretty trivial:
"""
[...]
response-policy {
zone "rpz.local" policy nxdomain;
};
[...]
zone "rpz.local" {
type slave;
file "../secundarios/db.rpz.local";
allow-query { 127.0.0.1; };
allow-transfer { none; };
masters {
X.X.X.X;
};
};
"""
Is this maybe related with being a "response-policy" zone?
That's correct.
If this is
the case and a malware RPZ is going to be BIG by definition, what would
be the suggested approach?
Depends on your operational needs, and if it is indeed causing
measurable trouble in your environment.
With multiple the CPU cores available you can trade "more even" latency
for efficiency by configuring workaround mentioned here:
https://gitlab.isc.org/isc-projects/bind9/-/issues/3746#note_337063
Thanks!
PS: I have not tried alternative secondary storage backends yet, like
"map". I am trying to understand what is going on first.
First, it would not help - this is caused by RPZ "frontend", not the
storage "backend".
Second, map is deprecated in 9.16 and removed from 9.18 onward. In case
you use it somewhere it's time to move on!
HTH.
--
Petr Špaček
Internet Systems Consortium
--
Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from
this list
ISC funds the development of this software with paid support subscriptions.
Contact us at https://www.isc.org/contact/ for more information.
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
Re: Incremental transfers generate complete zone reloading
On Sun, 15 Jan 2023, Jesus Cea wrote: I have a huge zone receiving a constant flow of small dns updates. My secondaries receive notifications and transfer the zone incrementally. Cool, everything works as expected. [...] Ok, my updates are coming too fast (first line). No problem, the secondary will eventually retrieve the changes. What worries me is the last couple of lines: The rpz zone (big, around 800.000 domains) is being reloaded constantly and it takes a couple of seconds eating CPU, when the incremental changes are actually pretty tiny. [...] not a full zone reload taking a couple of seconds and sucking an entire CPU core. Is that a fact or conjecture? There's a lot of "marketecture" in threat indicators generally. We can start with notifications versus polling. Secondaries can do either. Tell me why one is better, other than the vendor says so. Polling just does an SOA query, so two UDP packets; notify sends one. Is that extra packet more important than control? If this is a vendor and they're doing this why don't other customers see this as a problem? Is this just a "tax" for dealing with that vendor? What proof do you have that the CPU usage correlates, and that it's a problem? What are the vendor's recommendations (for provisioning and operational management), and are you following them? -- Fred Morris -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Incremental transfers generate complete zone reloading
I have a huge zone receiving a constant flow of small dns updates. My
secondaries receive notifications and transfer the zone incrementally.
Cool, everything works as expected.
Nevertheless, I see this lines in my logs, constantly (every time a
change arrives incrementally):
"""
15-Jan-2023 17:49:47.662 general: info: rpz: rpz.local: new zone version
came too soon, deferring update for 28 seconds
15-Jan-2023 17:49:54.716 notify: info: client @11f80268 X.X.X.X#63514:
received notify for zone 'rpz.local'
15-Jan-2023 17:49:54.716 general: info: zone rpz.local/IN: notify from
X.X.X.X#63514: serial 8991
15-Jan-2023 17:50:15.662 general: info: rpz: rpz.local: reload start
15-Jan-2023 17:50:16.884 general: info: rpz: rpz.local: reload done
"""
Ok, my updates are coming too fast (first line). No problem, the
secondary will eventually retrieve the changes. What worries me is the
last couple of lines: The rpz zone (big, around 800.000 domains) is
being reloaded constantly and it takes a couple of seconds eating CPU,
when the incremental changes are actually pretty tiny.
I would guess the incremental changes would do an incremental change in
memory structures, not a full zone reload taking a couple of seconds and
sucking an entire CPU core.
My secondary configuration is pretty trivial:
"""
[...]
response-policy {
zone "rpz.local" policy nxdomain;
};
[...]
zone "rpz.local" {
type slave;
file "../secundarios/db.rpz.local";
allow-query { 127.0.0.1; };
allow-transfer { none; };
masters {
X.X.X.X;
};
};
"""
Is this maybe related with being a "response-policy" zone? If this is
the case and a malware RPZ is going to be BIG by definition, what would
be the suggested approach?
Thanks!
PS: I have not tried alternative secondary storage backends yet, like
"map". I am trying to understand what is going on first.
--
Jesús Cea Avión _/_/ _/_/_/_/_/_/
j...@jcea.es - https://www.jcea.es/_/_/_/_/ _/_/_/_/ _/_/
Twitter: @jcea_/_/_/_/ _/_/_/_/_/
jabber / xmpp:j...@jabber.org _/_/ _/_/_/_/ _/_/ _/_/
"Things are not so easy" _/_/ _/_/_/_/ _/_/_/_/ _/_/
"My name is Dump, Core Dump" _/_/_/_/_/_/ _/_/ _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
--
Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from
this list
ISC funds the development of this software with paid support subscriptions.
Contact us at https://www.isc.org/contact/ for more information.
bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users
