Re: Thread handling
On Tue, Oct 04, 2022 at 09:55:46PM +, Hamid Maadani wrote: > So at the moment, to build each DLZ module, you cd into > the respective directory and run make to build the shared object, > correct? All I need to provide for a new DLZ, is test files (if any), > source files, and the Makefile? No package requirement checking > through configure.ac? Correct. -- Evan Hunt -- e...@isc.org Internet Systems Consortium, Inc. -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Thread handling
Understood. So at the moment, to build each DLZ module, you cd into the respective directory and run make to build the shared object, correct? All I need to provide for a new DLZ, is test files (if any), source files, and the Makefile? No package requirement checking through configure.ac? Regards Hamid Maadani October 4, 2022 2:47 PM, "Evan Hunt" wrote: > On Tue, Oct 04, 2022 at 05:28:59PM +, Hamid Maadani wrote: > >> Back to DLZs, I see that they are removed from the configure.ac Are >> they supposed to be built individually now, and not from the main >> build system? > > DLZ drivers used to be linked directly into named at compile time, > that's what was in configure.ac. > > Later we added dlopen-able DLZ modules, which are built separately > and linked in at runtime (see contrib/dlz/modules and subdirectories). > > For a long time both methods were available but in (I think?) 9.18 we > finally removed the old-style drivers. > > -- > Evan Hunt -- e...@isc.org > Internet Systems Consortium, Inc. -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Thread handling
On Tue, Oct 04, 2022 at 05:28:59PM +, Hamid Maadani wrote: > Back to DLZs, I see that they are removed from the configure.ac Are > they supposed to be built individually now, and not from the main > build system? DLZ drivers used to be linked directly into named at compile time, that's what was in configure.ac. Later we added dlopen-able DLZ modules, which are built separately and linked in at runtime (see contrib/dlz/modules and subdirectories). For a long time both methods were available but in (I think?) 9.18 we finally removed the old-style drivers. -- Evan Hunt -- e...@isc.org Internet Systems Consortium, Inc. -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Thread handling
Makes sense, thank you.
Back to DLZs, I see that they are removed from the configure.ac
Are they supposed to be built individually now, and not from the main build
system?
Regards
Hamid Maadani
September 12, 2022 5:13 AM, "Petr Špaček" wrote:
> On 24. 08. 22 17:48, hamid wrote:
>
>>> Such use case (authoritative data) is fine, I was merely speaking
>> about caching server before.
>>
>> Understood. Interesting. Was my understanding of DynDB correct? It reads
>> from a backend DB into memory, to eliminate the latency?
>
> Not necessairly. DynDB API simply allows you to hook into BIND internals
> and gives you access to all the gory details in the database API.
>
> It's up to the dyndb module implementation to decide what to do with it.
>
> An complex example is:
> https://pagure.io/bind-dyndb-ldap
>
> It establishes persistent connection to a RFC 4533-compliant LDAP server
> (that's no-SQL, see! :-)), pulls in all the data into BIND server
> memory, and keeps monitoring the server for changes. Any changes on the
> LDAP side and applied to the DNS side in real-time, and it also works
> the other way around and writes DNS updates back into LDAP.
>
> This is one possible way how it can be implemented. With all the data in
> memory it is "simple" to turn on inline-signing and thus have proper
> DNSSEC support "for free". Doing so with incomplete data would be way
> more challenging to implement in BIND.
>
>> In my proposed "workaround" (a hidden primary server with DLZ and
>> multiple secondary caching nodes) we gain speed but lose the dynamic
>> element. Meaning if a record is updated in the DB, the change will be
>> reflected on the caching servers after ttl.
>
> It sounds almost like confusing authoritative and recursive roles? Auth
> servers can be updated at any time using dynamic updates or zone
> transfers. All what's needed is to send out NOTIFY message from the
> primary ("master") to secondaries. No need to wait for TTL to expire.
>
>> With DynDB, a reload would be necessary after any update if I understand
>> correctly.
>
> No, see above.
>
>> Trying to understand how would a hidden primary with DLZ, paired with
>> multiple secondaries with dynamic zones leveraging nsupdate work. Do
>> secondaries poll the primary? Or is primary sending updates to
>> secondaries when a record is updated? I believe it's the former, just
>> trying to clarify.
>
> Depending on amount of data, source system/protocol, and other local
> conditions you might be able to use other ways to export data in DNS
> format than DLZ or DynDB. For example implementing "something" which
> pretends to do only AXFR/IXFR/NOTIFY might be another option. Yet
> another option might be _something else_ based on AXFR/nsupdate.
>
> I hope it helps.
> Petr Špaček
>
>> Regards
>> Hamid Maadani
>>
>> Original message
>> From: Ondřej Surý
>> Date: 8/24/22 02:32 (GMT-08:00)
>> To: hamid
>> Cc: ML BIND Users
>> Subject: Re: Thread handling
>>
>>> On 24. 8. 2022, at 11:01, hamid >> > wrote:
>>>
>>>> Perhaps, describing the use case first (why do you want to use
>>> MongoDB at all) might have the benefit of not wasting time on your end.
>>>
>>> Forgot to answer this, my use case would be the same as someone who
>>> uses a SQL DB backend I imagine: to be able to configure multiple BIND
>>> endpoints, using the same backend DB instead of configuration files,
>>> so there is no need to worry about change propagation and use of
>>> configuration management tools like chef, ansible etc.
>>> I just prefer to use no-sql backends like MongoDB, or Amazon's DocumentDB.
>>>
>>> If there is any specific downside to using no-sql databases, or any
>>> reason it would not make sense, I would appreciate it if you can
>>> explain it a bit. I am aware of the latency it would introduce, but
>>> was under the impression that DynDB is introduced to address that.
>>
>> Such use case (authoritative data) is fine, I was merely speaking about
>> caching server before.
>>
>> You have to calculate the benefit-cost ratio yourself compared to other
>> provisioning systems - f.e. hidden primary with multiple secondaries
>> updated with nsupdate works reasonably well in smaller deployments.
>>
>> Cheers,
>> Ondrej
>> --
>> Ondřej Surý (He/Him)
>> ond...@isc.org
>>
>> My working hours and your working hours may be different. Please do not
>> f
Re: Thread handling
On 24. 08. 22 17:48, hamid wrote:
> Such use case (authoritative data) is fine, I was merely speaking
about caching server before.
Understood. Interesting. Was my understanding of DynDB correct? It reads
from a backend DB into memory, to eliminate the latency?
Not necessairly. DynDB API simply allows you to hook into BIND internals
and gives you access to all the gory details in the database API.
It's up to the dyndb module implementation to decide what to do with it.
An complex example is:
https://pagure.io/bind-dyndb-ldap
It establishes persistent connection to a RFC 4533-compliant LDAP server
(that's no-SQL, see! :-)), pulls in all the data into BIND server
memory, and keeps monitoring the server for changes. Any changes on the
LDAP side and applied to the DNS side in real-time, and it also works
the other way around and writes DNS updates back into LDAP.
This is one possible way how it can be implemented. With all the data in
memory it is "simple" to turn on inline-signing and thus have proper
DNSSEC support "for free". Doing so with incomplete data would be way
more challenging to implement in BIND.
In my proposed "workaround" (a hidden primary server with DLZ and
multiple secondary caching nodes) we gain speed but lose the dynamic
element. Meaning if a record is updated in the DB, the change will be
reflected on the caching servers after ttl.
It sounds almost like confusing authoritative and recursive roles? Auth
servers can be updated at any time using dynamic updates or zone
transfers. All what's needed is to send out NOTIFY message from the
primary ("master") to secondaries. No need to wait for TTL to expire.
With DynDB, a reload would be necessary after any update if I understand
correctly.
No, see above.
Trying to understand how would a hidden primary with DLZ, paired with
multiple secondaries with dynamic zones leveraging nsupdate work. Do
secondaries poll the primary? Or is primary sending updates to
secondaries when a record is updated? I believe it's the former, just
trying to clarify.
Depending on amount of data, source system/protocol, and other local
conditions you might be able to use other ways to export data in DNS
format than DLZ or DynDB. For example implementing "something" which
pretends to do only AXFR/IXFR/NOTIFY might be another option. Yet
another option might be _something else_ based on AXFR/nsupdate.
I hope it helps.
Petr Špaček
Regards
Hamid Maadani
Original message
From: Ondřej Surý
Date: 8/24/22 02:32 (GMT-08:00)
To: hamid
Cc: ML BIND Users
Subject: Re: Thread handling
On 24. 8. 2022, at 11:01, hamid <mailto:ha...@dexo.tech>> wrote:
> Perhaps, describing the use case first (why do you want to use
MongoDB at all) might have the benefit of not wasting time on your end.
Forgot to answer this, my use case would be the same as someone who
uses a SQL DB backend I imagine: to be able to configure multiple BIND
endpoints, using the same backend DB instead of configuration files,
so there is no need to worry about change propagation and use of
configuration management tools like chef, ansible etc.
I just prefer to use no-sql backends like MongoDB, or Amazon's DocumentDB.
If there is any specific downside to using no-sql databases, or any
reason it would not make sense, I would appreciate it if you can
explain it a bit. I am aware of the latency it would introduce, but
was under the impression that DynDB is introduced to address that.
Such use case (authoritative data) is fine, I was merely speaking about
caching server before.
You have to calculate the benefit-cost ratio yourself compared to other
provisioning systems - f.e. hidden primary with multiple secondaries
updated with nsupdate works reasonably well in smaller deployments.
Cheers,
Ondrej
--
Ondřej Surý (He/Him)
ond...@isc.org <mailto:ond...@isc.org>
My working hours and your working hours may be different. Please do not
feel obligated to reply outside your normal working hours.
Regards
Hamid Maadani
Original message
From: Hamid Maadani mailto:ha...@dexo.tech>>
Date: 8/24/22 01:08 (GMT-08:00)
To: Ondřej Surý mailto:ond...@isc.org>>, Evan Hunt
mailto:e...@isc.org>>
Cc:bind-users@lists.isc.org <mailto:bind-users@lists.isc.org>
Subject: Re: Thread handling
> BIND does have dyndb support, since 9.11.
> As far as I know, though, the only two dyndb modules in existence are
> the bind-dyndb-ldap modiule that was written by Red Hat as part of
> FreeIPA, and a toy module used for testing. If you were interested in
> writing your MongoDB module for dyndb instead of DLZ, I'd be quite
> excited about that, I've long hoped the API would get more use.
Interesting. I looked in the contrib directory and only found the DLZ
modules there. Can you please point me in the direction of the source
Re: Thread handling
> Such use case (authoritative data) is fine, I was merely speaking about > caching server before.Understood. Interesting. Was my understanding of DynDB > correct? It reads from a backend DB into memory, to eliminate the latency?In > my proposed "workaround" (a hidden primary server with DLZ and multiple > secondary caching nodes) we gain speed but lose the dynamic element. Meaning > if a record is updated in the DB, the change will be reflected on the caching > servers after ttl.With DynDB, a reload would be necessary after any update if > I understand correctly. Trying to understand how would a hidden primary with > DLZ, paired with multiple secondaries with dynamic zones leveraging nsupdate > work. Do secondaries poll the primary? Or is primary sending updates to > secondaries when a record is updated? I believe it's the former, just trying > to clarify.RegardsHamid Maadani Original message From: Ondřej Surý Date: 8/24/22 02:32 (GMT-08:00) To: hamid Cc: ML BIND Users Subject: Re: Thread handling On 24. 8. 2022, at 11:01, hamid wrote:> Perhaps, describing the use case first (why do you want to use MongoDB at all) might have the benefit of not wasting time on your end.Forgot to answer this, my use case would be the same as someone who uses a SQL DB backend I imagine: to be able to configure multiple BIND endpoints, using the same backend DB instead of configuration files, so there is no need to worry about change propagation and use of configuration management tools like chef, ansible etc.I just prefer to use no-sql backends like MongoDB, or Amazon's DocumentDB.If there is any specific downside to using no-sql databases, or any reason it would not make sense, I would appreciate it if you can explain it a bit. I am aware of the latency it would introduce, but was under the impression that DynDB is introduced to address that.Such use case (authoritative data) is fine, I was merely speaking about caching server before.You have to calculate the benefit-cost ratio yourself compared to other provisioning systems - f.e. hidden primary with multiple secondaries updated with nsupdate works reasonably well in smaller deployments.Cheers,Ondrej--Ondřej Surý (He/Him)ondrej@isc.orgMy working hours and your working hours may be different. Please do not feel obligated to reply outside your normal working hours.RegardsHamid Maadani Original message From: Hamid Maadani Date: 8/24/22 01:08 (GMT-08:00) To: Ondřej Surý , Evan Hunt Cc: bind-us...@lists.isc.orgSubject: Re: Thread handling > BIND does have dyndb support, since 9.11.> As far as I know, though, the only two dyndb modules in existence are> the bind-dyndb-ldap modiule that was written by Red Hat as part of> FreeIPA, and a toy module used for testing. If you were interested in> writing your MongoDB module for dyndb instead of DLZ, I'd be quite> excited about that, I've long hoped the API would get more use.Interesting. I looked in the contrib directory and only found the DLZ modules there. Can you please point me in the direction of the source code for that toy module?I would definitely work on a mongo-dyndb implementation as well, when the time permits.> I am fairly confident that any advantage from shared cache will be lost because the extra latency caused by communication with the MongoDB (or any other no-sql systems).> Perhaps, describing the use case first (why do you want to use MongoDB at all) might have the benefit of not wasting time on your end.This is a bit confusing to me. As I understand it, a normal bind zone is loaded into memory, and requests are served from that memory-cache, hence super fast.DLZ modules however, make a call to the backend database per query. Which would introduce latency (in my tests, 50ms when using an Atlas cluster in the same geographical region). However, why would this be specific to no-sql databases?! Doesn't this also apply to any sql based DB?Now, an advantage of DLZ, is any change you make to the backend DB takes place immediately without the need to reload the server. Not a huge advantage in my personal opinion, but I do see the use case for it.Looking for a way to load queried records from a backend database into memory to speed up the responses, I found posts about the DynDB API. If Evan can kindly point me in the right direction there, I will be developing both DLZ and DynDB modules for MongoDB, as I do see use cases for each one.The caching question that I asked, was more around having a workaround without DynDB, because I was under the impression that DynDB API is not available at the moment. My understanding of a BIND caching server (and admittedly , I am by no means an advanced user when it comes to BIND), is that it would query records from another server, and cache them for the life (TTL) of that record, and serve it. This cache, exists i
Re: Thread handling
> On 24. 8. 2022, at 11:01, hamid wrote: > > > Perhaps, describing the use case first (why do you want to use MongoDB at > > all) might have the benefit of not wasting time on your end. > > Forgot to answer this, my use case would be the same as someone who uses a > SQL DB backend I imagine: to be able to configure multiple BIND endpoints, > using the same backend DB instead of configuration files, so there is no need > to worry about change propagation and use of configuration management tools > like chef, ansible etc. > I just prefer to use no-sql backends like MongoDB, or Amazon's DocumentDB. > > If there is any specific downside to using no-sql databases, or any reason it > would not make sense, I would appreciate it if you can explain it a bit. I am > aware of the latency it would introduce, but was under the impression that > DynDB is introduced to address that. Such use case (authoritative data) is fine, I was merely speaking about caching server before. You have to calculate the benefit-cost ratio yourself compared to other provisioning systems - f.e. hidden primary with multiple secondaries updated with nsupdate works reasonably well in smaller deployments. Cheers, Ondrej -- Ondřej Surý (He/Him) ond...@isc.org My working hours and your working hours may be different. Please do not feel obligated to reply outside your normal working hours. > Regards > Hamid Maadani > > > Original message > From: Hamid Maadani mailto:ha...@dexo.tech>> > Date: 8/24/22 01:08 (GMT-08:00) > To: Ondřej Surý mailto:ond...@isc.org>>, Evan Hunt > mailto:e...@isc.org>> > Cc: bind-users@lists.isc.org <mailto:bind-users@lists.isc.org> > Subject: Re: Thread handling > > > BIND does have dyndb support, since 9.11. > > As far as I know, though, the only two dyndb modules in existence are > > the bind-dyndb-ldap modiule that was written by Red Hat as part of > > FreeIPA, and a toy module used for testing. If you were interested in > > writing your MongoDB module for dyndb instead of DLZ, I'd be quite > > excited about that, I've long hoped the API would get more use. > > Interesting. I looked in the contrib directory and only found the DLZ modules > there. Can you please point me in the direction of the source code for that > toy module? > I would definitely work on a mongo-dyndb implementation as well, when the > time permits. > > I am fairly confident that any advantage from shared cache will be lost > > because the extra latency caused by communication with the MongoDB (or any > > other no-sql systems). > > Perhaps, describing the use case first (why do you want to use MongoDB at > > all) might have the benefit of not wasting time on your end. > > This is a bit confusing to me. As I understand it, a normal bind zone is > loaded into memory, and requests are served from that memory-cache, hence > super fast. > DLZ modules however, make a call to the backend database per query. Which > would introduce latency (in my tests, 50ms when using an Atlas cluster in the > same geographical region). However, why would this be specific to no-sql > databases?! Doesn't this also apply to any sql based DB? > > Now, an advantage of DLZ, is any change you make to the backend DB takes > place immediately without the need to reload the server. Not a huge advantage > in my personal opinion, but I do see the use case for it. > Looking for a way to load queried records from a backend database into memory > to speed up the responses, I found posts about the DynDB API. If Evan can > kindly point me in the right direction there, I will be developing both DLZ > and DynDB modules for MongoDB, as I do see use cases for each one. > > The caching question that I asked, was more around having a workaround > without DynDB, because I was under the impression that DynDB API is not > available at the moment. My understanding of a BIND caching server (and > admittedly , I am by no means an advanced user when it comes to BIND), is > that it would query records from another server, and cache them for the life > (TTL) of that record, and serve it. This cache, exists in memory, correct? > So in theory, if I was to use a DLZ to keep my records in a backend DB, I can > technically create a BIND server with the DLZ enabled (let's say a docker > image), and then put a caching server in front of it, which is "customer > facing". > That way, all queries will come to the caching server, and will be served > super fast because they are cached in memory, but the actual records live in > a backend DB somewhere. > Long story short, I was trying to see if the same can be achieved with one > s
Re: Thread handling
> Perhaps, describing the use case first (why do you want to use MongoDB at > all) might have the benefit of not wasting time on your end.Forgot to answer > this, my use case would be the same as someone who uses a SQL DB backend I > imagine: to be able to configure multiple BIND endpoints, using the same > backend DB instead of configuration files, so there is no need to worry about > change propagation and use of configuration management tools like chef, > ansible etc.I just prefer to use no-sql backends like MongoDB, or Amazon's > DocumentDB.If there is any specific downside to using no-sql databases, or > any reason it would not make sense, I would appreciate it if you can explain > it a bit. I am aware of the latency it would introduce, but was under the > impression that DynDB is introduced to address that.RegardsHamid Maadani Original message From: Hamid Maadani Date: 8/24/22 01:08 (GMT-08:00) To: Ondřej Surý , Evan Hunt Cc: bind-users@lists.isc.org Subject: Re: Thread handling > BIND does have dyndb support, since 9.11.> As far as I know, though, the only two dyndb modules in existence are> the bind-dyndb-ldap modiule that was written by Red Hat as part of> FreeIPA, and a toy module used for testing. If you were interested in> writing your MongoDB module for dyndb instead of DLZ, I'd be quite> excited about that, I've long hoped the API would get more use.Interesting. I looked in the contrib directory and only found the DLZ modules there. Can you please point me in the direction of the source code for that toy module?I would definitely work on a mongo-dyndb implementation as well, when the time permits. > I am fairly confident that any advantage from shared cache will be lost because the extra latency caused by communication with the MongoDB (or any other no-sql systems).> Perhaps, describing the use case first (why do you want to use MongoDB at all) might have the benefit of not wasting time on your end.This is a bit confusing to me. As I understand it, a normal bind zone is loaded into memory, and requests are served from that memory-cache, hence super fast.DLZ modules however, make a call to the backend database per query. Which would introduce latency (in my tests, 50ms when using an Atlas cluster in the same geographical region). However, why would this be specific to no-sql databases?! Doesn't this also apply to any sql based DB?Now, an advantage of DLZ, is any change you make to the backend DB takes place immediately without the need to reload the server. Not a huge advantage in my personal opinion, but I do see the use case for it.Looking for a way to load queried records from a backend database into memory to speed up the responses, I found posts about the DynDB API. If Evan can kindly point me in the right direction there, I will be developing both DLZ and DynDB modules for MongoDB, as I do see use cases for each one.The caching question that I asked, was more around having a workaround without DynDB, because I was under the impression that DynDB API is not available at the moment. My understanding of a BIND caching server (and admittedly , I am by no means an advanced user when it comes to BIND), is that it would query records from another server, and cache them for the life (TTL) of that record, and serve it. This cache, exists in memory, correct?So in theory, if I was to use a DLZ to keep my records in a backend DB, I can technically create a BIND server with the DLZ enabled (let's say a docker image), and then put a caching server in front of it, which is "customer facing".That way, all queries will come to the caching server, and will be served super fast because they are cached in memory, but the actual records live in a backend DB somewhere.Long story short, I was trying to see if the same can be achieved with one single instance instead of two, which sounds like it can not be done.RegardsHamid MaadaniAugust 24, 2022 12:40 AM, "Ondřej Surý" wrote: On 24. 8. 2022, at 8:48, Evan Hunt wrote: In the absence of that, is caching from DLZ a possible configurationon a single BIND server?Not DLZ, no. And I'm not sure dyndb can be used for the cache database,either; do you know something about it that I don't?It would definitely be easier to *make* dyndb work for the cache;it has all the necessary API calls, and DLZ doesn't. But I don'tknow a way to configure it to take the place of the cache currently.If you do, please educate me. I am fairly confident that any advantage from shared cache will be lost because the extra latency caused by communication with the MongoDB (or any other no-sql systems).Perhaps, describing the use case first (why do you want to use MongoDB at all) might have the benefit of not wasting time on your end.Ondrej--Ondřej Surý (He/Him)ondrej@isc.orgMy working hours and your working hours may be differ
Re: Thread handling
> BIND does have dyndb support, since 9.11. > As far as I know, though, the only two dyndb modules in existence are > the bind-dyndb-ldap modiule that was written by Red Hat as part of > FreeIPA, and a toy module used for testing. If you were interested in > writing your MongoDB module for dyndb instead of DLZ, I'd be quite > excited about that, I've long hoped the API would get more use. Interesting. I looked in the contrib directory and only found the DLZ modules there. Can you please point me in the direction of the source code for that toy module? I would definitely work on a mongo-dyndb implementation as well, when the time permits. > I am fairly confident that any advantage from shared cache will be lost > because the extra latency caused by communication with the MongoDB (or any > other no-sql systems). > Perhaps, describing the use case first (why do you want to use MongoDB at > all) might have the benefit of not wasting time on your end. This is a bit confusing to me. As I understand it, a normal bind zone is loaded into memory, and requests are served from that memory-cache, hence super fast. DLZ modules however, make a call to the backend database per query. Which would introduce latency (in my tests, 50ms when using an Atlas cluster in the same geographical region). However, why would this be specific to no-sql databases?! Doesn't this also apply to any sql based DB? Now, an advantage of DLZ, is any change you make to the backend DB takes place immediately without the need to reload the server. Not a huge advantage in my personal opinion, but I do see the use case for it. Looking for a way to load queried records from a backend database into memory to speed up the responses, I found posts about the DynDB API. If Evan can kindly point me in the right direction there, I will be developing both DLZ and DynDB modules for MongoDB, as I do see use cases for each one. The caching question that I asked, was more around having a workaround without DynDB, because I was under the impression that DynDB API is not available at the moment. My understanding of a BIND caching server (and admittedly , I am by no means an advanced user when it comes to BIND), is that it would query records from another server, and cache them for the life (TTL) of that record, and serve it. This cache, exists in memory, correct? So in theory, if I was to use a DLZ to keep my records in a backend DB, I can technically create a BIND server with the DLZ enabled (let's say a docker image), and then put a caching server in front of it, which is "customer facing". That way, all queries will come to the caching server, and will be served super fast because they are cached in memory, but the actual records live in a backend DB somewhere. Long story short, I was trying to see if the same can be achieved with one single instance instead of two, which sounds like it can not be done. Regards Hamid Maadani August 24, 2022 12:40 AM, "Ondřej Surý" mailto:ond...@isc.org?to=%22Ond%C5%99ej%20Sur%C3%BD%22%20)> wrote: On 24. 8. 2022, at 8:48, Evan Hunt mailto:e...@isc.org)> wrote: In the absence of that, is caching from DLZ a possible configuration on a single BIND server? Not DLZ, no. And I'm not sure dyndb can be used for the cache database, either; do you know something about it that I don't? It would definitely be easier to *make* dyndb work for the cache; it has all the necessary API calls, and DLZ doesn't. But I don't know a way to configure it to take the place of the cache currently. If you do, please educate me. I am fairly confident that any advantage from shared cache will be lost because the extra latency caused by communication with the MongoDB (or any other no-sql systems). Perhaps, describing the use case first (why do you want to use MongoDB at all) might have the benefit of not wasting time on your end. Ondrej -- Ondřej Surý (He/Him) ond...@isc.org (mailto:ond...@isc.org) My working hours and your working hours may be different. Please do not feel obligated to reply outside your normal working hours. -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Thread handling
> On 24. 8. 2022, at 8:48, Evan Hunt wrote: > >> In the absence of that, is caching from DLZ a possible configuration >> on a single BIND server? > > Not DLZ, no. And I'm not sure dyndb can be used for the cache database, > either; do you know something about it that I don't? > > It would definitely be easier to *make* dyndb work for the cache; > it has all the necessary API calls, and DLZ doesn't. But I don't > know a way to configure it to take the place of the cache currently. > If you do, please educate me. I am fairly confident that any advantage from shared cache will be lost because the extra latency caused by communication with the MongoDB (or any other no-sql systems). Perhaps, describing the use case first (why do you want to use MongoDB at all) might have the benefit of not wasting time on your end. Ondrej -- Ondřej Surý (He/Him) ond...@isc.org My working hours and your working hours may be different. Please do not feel obligated to reply outside your normal working hours. -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Thread handling
> Regarding the child process(es), does named create one child process, > or can it be multiple processes? I assume each process loads the > shared objects for itself, so only one call to dns_dlzcreate per > process? I'm pretty sure it's called only once on startup, after daemonizing, and again whenever you reconfigure the server. (I haven't checked, but I think that's the case.) > Question about the cache mentioned here. Can cache be configured to > pull from a DLZ? No. > As far as I know, the DynDB API is not merged to BIND's codebase yet. BIND does have dyndb support, since 9.11. As far as I know, though, the only two dyndb modules in existence are the bind-dyndb-ldap modiule that was written by Red Hat as part of FreeIPA, and a toy module used for testing. If you were interested in writing your MongoDB module for dyndb instead of DLZ, I'd be quite excited about that, I've long hoped the API would get more use. > In the absence of that, is caching from DLZ a possible configuration > on a single BIND server? Not DLZ, no. And I'm not sure dyndb can be used for the cache database, either; do you know something about it that I don't? It would definitely be easier to *make* dyndb work for the cache; it has all the necessary API calls, and DLZ doesn't. But I don't know a way to configure it to take the place of the cache currently. If you do, please educate me. -- Evan Hunt -- e...@isc.org Internet Systems Consortium, Inc. -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Thread handling
These are perfect answers, thank you very much Evan. > All supported versions of BIND are now multi-threaded. Single-threaded > support was removed in 9.14 Since BIND is now multi-threaded only, I will stick with that model. No need to cover a single-threaded model I suppose. > It does, actually, unless you run it with -g or -f. That's how it puts > itself in the background, the function is named_os_daemonize(). Regarding the child process(es), does named create one child process, or can it be multiple processes? I assume each process loads the shared objects for itself, so only one call to dns_dlzcreate per process? > In the DLZ module, there can be an arbitrary number of separate database > handles in use, and the query can pick one that isn't busy and use it to > send the query. Regarding handling the database connections, fortunately mongo-c-driver provides a thread-safe connection pool. It does the connection monitoring and handling very well. From the sounds of it, that would fit perfectly with the BIND model. I don't think I'm gonna need to be too creative on that front :) > If no local database does and recursion is enabled, then we try the cache, > and failing that, we recurse. Question about the cache mentioned here. Can cache be configured to pull from a DLZ? As far as I know, the DynDB API is not merged to BIND's codebase yet. In the absence of that, is caching from DLZ a possible configuration on a single BIND server? Regards Hamid Maadani August 23, 2022 6:49 PM, "Evan Hunt" wrote: > On Tue, Aug 23, 2022 at 05:59:49PM +, Hamid Maadani wrote: > >> I'm working on a MongoDB DLZ for bind9, and have a few questions about >> the multi-threaded version of named: 1. Is there a pre-processor flag >> I can use to determine if thread support is enabled? Can't seem to >> find it in configure.ac > > All supported versions of BIND are now multi-threaded. Single-threaded > support was removed in 9.14, and everything older is now past end of > life. > > If you're building for an older version, though, the flag is PTHREADS. > >> 2. named does not fork itself, or create child processes, correct? > > It does, actually, unless you run it with -g or -f. That's how it puts > itself in the background, the function is named_os_daemonize(). > >> 3. when multi-threading is enabled, does named still call dlz_create >> only once when loading the shared object (dlopen)? > > I believe so. That happens in dns_dlzcreate() when setting up the view > configuration, if I recall correctly, so it should only occur once in the > main thread. > >> 4. when multi-threading is enabled, how is each request handled? are >> there multiple worker threads and requests passed to them? or would >> each request spin up a new thread? Trying to figure out how dlz_lookup >> is called in a multi-threaded version. > > In named, there are multiple worker threads that handle queries. > > The first step in query processing is to identify which local database, > if any, has data responsive to the query. If no local database does and > recursion is enabled, then we try the cache, and failing that, we recurse. > But if there *is* a local database that's authoritative for the name, then > we call dns_db_findext(), which is a front-end for the 'find' function in > the database implementation (whatever it may be). If the implementation > happens to be a DLZ, then it'll be calilng findext() in lib/dns/sdlz.c, > which in turn calls getnodedata(), which calls the DLZ's 'lookup' method. > > In the DLZ module, there can be an arbitrary number of separate database > handles in use, and the query can pick one that isn't busy and use it to > send the query. > > There are existing modules in the contrib tree (for example, > contrib/dlz/modules/ldap and contrib/dlz/modules/mysql) that have > examples of this you can use as a model. > > (Should you come up with a better way to do this, I'd be happy to > see it. It's always seemed pretty clunky to me but I've never had > the necessary combination of time and brains to improve it.) > > -- > Evan Hunt -- e...@isc.org > Internet Systems Consortium, Inc. -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
Re: Thread handling
On Tue, Aug 23, 2022 at 05:59:49PM +, Hamid Maadani wrote: > I'm working on a MongoDB DLZ for bind9, and have a few questions about > the multi-threaded version of named: 1. Is there a pre-processor flag > I can use to determine if thread support is enabled? Can't seem to > find it in configure.ac All supported versions of BIND are now multi-threaded. Single-threaded support was removed in 9.14, and everything older is now past end of life. If you're building for an older version, though, the flag is PTHREADS. > 2. named does not fork itself, or create child processes, correct? It does, actually, unless you run it with -g or -f. That's how it puts itself in the background, the function is named_os_daemonize(). > 3. when multi-threading is enabled, does named still call dlz_create > only once when loading the shared object (dlopen)? I believe so. That happens in dns_dlzcreate() when setting up the view configuration, if I recall correctly, so it should only occur once in the main thread. > 4. when multi-threading is enabled, how is each request handled? are > there multiple worker threads and requests passed to them? or would > each request spin up a new thread? Trying to figure out how dlz_lookup > is called in a multi-threaded version. In named, there are multiple worker threads that handle queries. The first step in query processing is to identify which local database, if any, has data responsive to the query. If no local database does and recursion is enabled, then we try the cache, and failing that, we recurse. But if there *is* a local database that's authoritative for the name, then we call dns_db_findext(), which is a front-end for the 'find' function in the database implementation (whatever it may be). If the implementation happens to be a DLZ, then it'll be calilng findext() in lib/dns/sdlz.c, which in turn calls getnodedata(), which calls the DLZ's 'lookup' method. In the DLZ module, there can be an arbitrary number of separate database handles in use, and the query can pick one that isn't busy and use it to send the query. There are existing modules in the contrib tree (for example, contrib/dlz/modules/ldap and contrib/dlz/modules/mysql) that have examples of this you can use as a model. (Should you come up with a better way to do this, I'd be happy to see it. It's always seemed pretty clunky to me but I've never had the necessary combination of time and brains to improve it.) -- Evan Hunt -- e...@isc.org Internet Systems Consortium, Inc. -- Visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. bind-users mailing list bind-users@lists.isc.org https://lists.isc.org/mailman/listinfo/bind-users
