Re: [Pdns-users] remote backend
Finally got it fixed. Did this, isolated the backend, made a pcap of the production site of the traffic flowing from pdns -> backend and filtered that to get the json’s and use those as input/real queries was ~350 queries. A new python script, simple one with requests module using those jsons as inputs and keeping how many queries has responses, and the time for the response. With that, i made a min,max,avg calculations on the response times and started the tests with a single thread on the python script and slowly increasing the threads. I found that ~15 threads started to push the ‘max’ response time from the backend, with 20 threads i had a LOT of queries far beyond 2 seconds max time (the pdns timeout on the backend that i have configured) Long story short, i have no control over the network or even the mongodb which the backend was using, so i’ve installed a mongodb in a new vm right on the same hypervisor where the pdns+backends run, all problems gone, have no way if it was network problems or mongodb problems (no way to find it) but everything runs perfect now. Thanks for your help guys, and for pointing me lmdb, not using it for this now (i will as soon as i can change the code) but started to test and using it in a few SIP proxy/routers and it’s great regards > On 29 Nov 2023, at 11:53, Brian Candler wrote: > > On 29/11/2023 14:04, Alexis Fidalgo wrote: >> So, by now, i dont know what is making for a query to be answered and >> another not (timeout) and in a retry is answered ok. (this is why i thought >> on speed and considered the unix socket but now i know it’s not that) > > Put logging in your remote backend and show what queries it receives and how > long it takes to respond to each one. Use these logs to check that the > queries generated by PowerDNS are what you expect (it may make multiple > requests for a single received DNS query). > > You can also take PowerDNS entirely out of the problem by making a set of > suitable test HTTP calls directly to your backend, for the same set queries > that PowerDNS would generate. If you can prove that your backend is taking > too long to answer them (on the first attempt at least), then you know where > to investigate. > > For example, it might be that MongoDB is doing a lot of slow disk seeks (is > it spinning rust or SSD?) but once it has the answer, everything it needs is > cached in RAM so it's much quicker on the second attempt. Or maybe it's not > indexed properly. You really need to drill down further to prove or disprove > that idea. > > If you find that MongoDB is the bottleneck and can't be tuned, then there are > other options. For example, if this database doesn't change very often, then > you could write it out to a CDB file: > > https://cr.yp.to/cdb.html > https://en.wikipedia.org/wiki/Cdb_(software) > > This is optimised for very fast lookup with minimum seeking, and can be > indexed in a single pass - but it can't be modified, so you'd have to > regenerate the whole file periodically. Also it has a 4GB size limit which > is probably an issue here (limiting you to avg 14 bytes per key/value pair) > so you may need to split into multiple files. > > A suitably-indexed Postgres table with 300 million entries is big but not > impossible, and PowerDNS could query it directly. > ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
Yes, I’m on it right now. Changes are very often, mongo is in ssd and I’m querying by the _id that is indexed, plus I’m using a mongo replica (it’s supposed to be a benefit for reading) But yes, I’m right now doing exactly what you told, writing an http test unit in golang to heavily charge the responder. Let’s see what happens. Again guys, thanks again for your help and support on this. Enviado desde dispositivo móvil > El 29 nov 2023, a la(s) 11:53, Brian Candler escribió: > > On 29/11/2023 14:04, Alexis Fidalgo wrote: >> So, by now, i dont know what is making for a query to be answered and >> another not (timeout) and in a retry is answered ok. (this is why i thought >> on speed and considered the unix socket but now i know it’s not that) > > Put logging in your remote backend and show what queries it receives and how > long it takes to respond to each one. Use these logs to check that the > queries generated by PowerDNS are what you expect (it may make multiple > requests for a single received DNS query). > > You can also take PowerDNS entirely out of the problem by making a set of > suitable test HTTP calls directly to your backend, for the same set queries > that PowerDNS would generate. If you can prove that your backend is taking > too long to answer them (on the first attempt at least), then you know where > to investigate. > > For example, it might be that MongoDB is doing a lot of slow disk seeks (is > it spinning rust or SSD?) but once it has the answer, everything it needs is > cached in RAM so it's much quicker on the second attempt. Or maybe it's not > indexed properly. You really need to drill down further to prove or disprove > that idea. > > If you find that MongoDB is the bottleneck and can't be tuned, then there are > other options. For example, if this database doesn't change very often, then > you could write it out to a CDB file: > > https://cr.yp.to/cdb.html > https://en.wikipedia.org/wiki/Cdb_(software) > > This is optimised for very fast lookup with minimum seeking, and can be > indexed in a single pass - but it can't be modified, so you'd have to > regenerate the whole file periodically. Also it has a 4GB size limit which > is probably an issue here (limiting you to avg 14 bytes per key/value pair) > so you may need to split into multiple files. > > A suitably-indexed Postgres table with 300 million entries is big but not > impossible, and PowerDNS could query it directly. > ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
On 29/11/2023 14:04, Alexis Fidalgo wrote: So, by now, i dont know what is making for a query to be answered and another not (timeout) and in a retry is answered ok. (this is why i thought on speed and considered the unix socket but now i know it’s not that) Put logging in your remote backend and show what queries it receives and how long it takes to respond to each one. Use these logs to check that the queries generated by PowerDNS are what you expect (it may make multiple requests for a single received DNS query). You can also take PowerDNS entirely out of the problem by making a set of suitable test HTTP calls directly to your backend, for the same set queries that PowerDNS would generate. If you can prove that your backend is taking too long to answer them (on the first attempt at least), then you know where to investigate. For example, it might be that MongoDB is doing a lot of slow disk seeks (is it spinning rust or SSD?) but once it has the answer, everything it needs is cached in RAM so it's much quicker on the second attempt. Or maybe it's not indexed properly. You really need to drill down further to prove or disprove that idea. If you find that MongoDB is the bottleneck and can't be tuned, then there are other options. For example, if this database doesn't change very often, then you could write it out to a CDB file: https://cr.yp.to/cdb.html https://en.wikipedia.org/wiki/Cdb_(software) This is optimised for very fast lookup with minimum seeking, and can be indexed in a single pass - but it can't be modified, so you'd have to regenerate the whole file periodically. Also it has a 4GB size limit which is probably an issue here (limiting you to avg 14 bytes per key/value pair) so you may need to split into multiple files. A suitably-indexed Postgres table with 300 million entries is big but not impossible, and PowerDNS could query it directly. ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
On 29/11/2023 10:19, Alexis Fidalgo wrote: by the responder, what im not understanding is, why in 2 different languages (golang and python) i get the same behavior. Well, you haven't shown the code from either. It would be extremely inefficient for PowerDNS to open a new connection for every message and close it immediately afterwards. That would be like HTTP without keepalive. And it would make no sense to send an "initialize" message to setup a connection, only to drop the connection immediately afterwards. I agree on this, makes no sense at all to close the socket after each message, what i found even worse is (again, at least in golang and python), from server side on the UDS. Im seeing . bind the socket to the fd . open the socket . Accept from the socket (locks) Note that accept() returns you a *new* fd (socket) representing this connection. . Reads from the socket ...where "the socket" here means the new one returned from accept() . Answer to the socket (*) . Locks for ever if i dont close it Maybe you have a problem with buffering then. Perhaps you need to flush the buffer after sending the response? If you don't show code, I can only speculate. But I guess we're moving away from the topic of PowerDNS, and onto the topic of socket programming. Im with you on this, right after Accept + Read, if im processing/answering in a different thread, it should go immediately to accept again and lock there waiting for a new message and so on, that’s the correct way to act, but im not getting that. No, accept() is not for receiving another message, it's for receiving another connection on the same Unix domain socket, which will require another thread to answer. Whether this happens is a question of whether PowerDNS will open multiple connections to the same Unix domain socket; this doesn't seem to be documented but it certainly could. So you need a main (thread/goroutine/green thread) calling accept() on the socket, which starts a new (thread/goroutine/green thread) to process each connection, and each connection can process multiple messages in sequence. ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
> On 29 Nov 2023, at 05:00, Brian Candler wrote:
>
> On 29/11/2023 00:07, Alexis Fidalgo via Pdns-users wrote:
>> I think i found why this is not working, as you can see below, socket is
>> connected and first message is sent (the initialize message), which is
>> answered and the response is read ({“result”: true}).
>>
>> Problem is (and i’ve testing with golang and python) after the answer the
>> “initialize” message, the socket is closed,
>
> By whom - by PowerDNS, or by your remote backend application code? I think
> it's the latter, and if so, the problem is at your side, not PowerDNS.
>
by the responder, what im not understanding is, why in 2 different languages
(golang and python) i get the same behavior.
> It would be extremely inefficient for PowerDNS to open a new connection for
> every message and close it immediately afterwards. That would be like HTTP
> without keepalive. And it would make no sense to send an "initialize"
> message to setup a connection, only to drop the connection immediately
> afterwards.
I agree on this, makes no sense at all to close the socket after each message,
what i found even worse is (again, at least in golang and python), from server
side on the UDS. Im seeing
. bind the socket to the fd
. open the socket
. Accept from the socket (locks)
. Reads from the socket
. Answer to the socket (*)
. Locks for ever if i dont close it
(*) it doesnt matter if i process/answer in the same thread/coroutine or in a
different one.
Im with you on this, right after Accept + Read, if im processing/answering in a
different thread, it should go immediately to accept again and lock there
waiting for a new message and so on, that’s the correct way to act, but im not
getting that.
Still convinced that there’s no better option as to write my backend but im not
good at c++ (im not good with others eigther :) :) )
___
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
On 29/11/2023 00:07, Alexis Fidalgo via Pdns-users wrote:
I think i found why this is not working, as you can see below, socket
is connected and first message is sent (the initialize message), which
is answered and the response is read ({“result”: true}).
Problem is (and i’ve testing with golang and python) after the answer
the “initialize” message, the socket is closed,
By whom - by PowerDNS, or by your remote backend application code? I
think it's the latter, and if so, the problem is at your side, not PowerDNS.
It would be extremely inefficient for PowerDNS to open a new connection
for every message and close it immediately afterwards. That would be
like HTTP without keepalive. And it would make no sense to send an
"initialize" message to setup a connection, only to drop the connection
immediately afterwards.
___
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
I think i found why this is not working, as you can see below, socket is
connected and first message is sent (the initialize message), which is answered
and the response is read ({“result”: true}).
Problem is (and i’ve testing with golang and python) after the answer the
“initialize” message, the socket is closed, so, getAllDomains message is being
sent using a closed socket and that’s why i don’t see it on the responder side
and pdns does not receive and answer, polls 2 times and reaches timeout.
i can see there’s no test for unixsocket in the source tree.
So this arises my question to you, do you think my conclusion is ok? do the
unixconnector.cc
Re: [Pdns-users] remote backend
has to be on my side. i made a change in the way i read/write to the socket and
now i get
socket(AF_UNIX, SOCK_STREAM, 0) = 12
connect(12, {sa_family=AF_UNIX, sun_path="/tmp/pra.sock"}, 110) = 0
write(12, "{\"method\": \"initialize\", \"parame"..., 66) = 66
poll([{fd=12, events=POLLIN}], 1, 1000) = 1 ([{fd=12, revents=POLLIN}])
read(12, "{\"result\":true}", 1500) = 15
write(12, "{\"method\": \"getAllDomains\", \"par"..., 70) = 70
poll([{fd=12, events=POLLIN}], 1, 1000) = 0 (Timeout)
poll([{fd=12, events=POLLIN}], 1, 1000) = 0 (Timeout)
close(12)
pdns now dies after the two POLL (timeout), i’ll work on my read/write code to
fix this and let you know.
thanks again
> On 28 Nov 2023, at 16:21, Alexis Fidalgo wrote:
>
> the mail with the whole strace is kept because of the attach but i can see
> this
>
>
> sendto(3, "<30>Nov 28 16:17:46 pdns[167106]"..., 57, MSG_NOSIGNAL, NULL, 0) =
> 57
> socket(AF_UNIX, SOCK_STREAM, 0) = 12
> connect(12, {sa_family=AF_UNIX, sun_path="/tmp/pra.sock"}, 110) = 0
> write(12, "{\"method\": \"initialize\", \"parame"..., 66) = 66
> poll([{fd=12, events=POLLIN}], 1, 1000) = 1 ([{fd=12, revents=POLLIN}])
> read(12, "{\"result\":true}", 1500) = 15
> write(12, "{\"method\": \"getAllDomains\", \"par"..., 70) = -1 EPIPE (Broken
> pipe)
> --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=167106, si_uid=0} ---
> close(12) = 0
> futex(0x7f5d0c6181e0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
> write(2, "Nov 28 16:17:46 PDNSException wh"..., 134Nov 28 16:17:46
> PDNSException while filling the zone cache: Exception caught when sending:
> Could not send a message to remote process
> ) = 134
>
>
>
> it looks like the socket is being closed when pdns is trying to send a new
> message (the getAllDomains)?
>
>
>> On 28 Nov 2023, at 16:13, Remi Gacogne via Pdns-users
>> wrote:
>>
>> Hi!
>>
>> On 28/11/2023 19:59, Alexis Fidalgo via Pdns-users wrote:
>>> Sorry about that, yes, this will work locally, meaning the remote responder
>>> (my script) will run on the same VM than pdns-auth, so pdns-auth will
>>> connect using a unix socket with the responder using remote backend.
>>> That actually occurs, this is what is shown from the pdns
>>> ---
>>> alz@nuc /opt/pdns-auth-4.8.3/sbin ./pdns_server
>>> Nov 28 14:52:54 This is a standalone pdns
>>> Nov 28 14:52:54 Listening on controlsocket in
>>> '/var/run/pdns/pdns.controlsocket'
>>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>>> Nov 28 14:52:54 TCP server bound to 0.0.0.0:5300
>>> Nov 28 14:52:54 PowerDNS Authoritative Server 4.8.3 (C) 2001-2022
>>> PowerDNS.COM BV
>>> Nov 28 14:52:54 Using 64-bits mode. Built using gcc 10.2.1 20210110 on Nov
>>> 28 2023 11:42:16 by a...@nuc.lesi.com.
>>> Nov 28 14:52:54 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free
>>> software, and you are welcome to redistribute it according to the terms of
>>> the GPL version 2.
>>> Nov 28 14:52:54 [stub-resolver] Doing stub resolving for
>>> 'auth-4.8.3.security-status.secpoll.powerdns.com.|TXT', using resolvers:
>>> 192.168.86.1
>>> Nov 28 14:52:54 [stub-resolver] Question for
>>> 'auth-4.8.3.security-status.secpoll.powerdns.com.|TXT' got answered by
>>> 192.168.86.1
>>> Nov 28 14:52:54 Polled security status of version 4.8.3 at startup, no
>>> known issues reported: OK
>>> Nov 28 14:52:54 Reconnecting to backend
>>> Nov 28 14:52:54 PDNSException while filling the zone cache: Exception
>>> caught when sending: Could not send a message to remote process
>>> —
>>> this is what is showed on the responder when the
>>> ---
>>> 2023-11-28T14:52:54.907-0300 DEBUG handlers/handlers.go:65 pdns request
>>> received: {"method": "initialize", "parameters": {"path": "/tmp/pra.sock"}}
>>> 2023-11-28T14:52:54.907-0300 DEBUG handlers/handlers.go:50 Response
>>> {"result":true}
>>> —
>>> This same responder script, if using http returns exactly the same json,
>>> getAllDomains comes after the initialize, then the lookups, so the
>>> responder works ok.
>>> Problem is, when i switch to unix socket, throws the error on the red line
>>> after the initialize and dies
>>
>> That's very weird indeed, and unfortunately the unix connector is lacking a
>> bit of logging in this area. Any chance you would be able to strace the
>> authoritative server process?
>>
>> Cheers,f
>> --
>> Remi Gacogne
>> PowerDNS.COM
>> https://mailman.powerdns.com/mailman/listinfo/pdns-users
>
Re: [Pdns-users] remote backend
the mail with the whole strace is kept because of the attach but i can see this
sendto(3, "<30>Nov 28 16:17:46 pdns[167106]"..., 57, MSG_NOSIGNAL, NULL, 0) = 57
socket(AF_UNIX, SOCK_STREAM, 0) = 12
connect(12, {sa_family=AF_UNIX, sun_path="/tmp/pra.sock"}, 110) = 0
write(12, "{\"method\": \"initialize\", \"parame"..., 66) = 66
poll([{fd=12, events=POLLIN}], 1, 1000) = 1 ([{fd=12, revents=POLLIN}])
read(12, "{\"result\":true}", 1500) = 15
write(12, "{\"method\": \"getAllDomains\", \"par"..., 70) = -1 EPIPE (Broken
pipe)
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=167106, si_uid=0} ---
close(12) = 0
futex(0x7f5d0c6181e0, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(2, "Nov 28 16:17:46 PDNSException wh"..., 134Nov 28 16:17:46
PDNSException while filling the zone cache: Exception caught when sending:
Could not send a message to remote process
) = 134
it looks like the socket is being closed when pdns is trying to send a new
message (the getAllDomains)?
> On 28 Nov 2023, at 16:13, Remi Gacogne via Pdns-users
> wrote:
>
> Hi!
>
> On 28/11/2023 19:59, Alexis Fidalgo via Pdns-users wrote:
>> Sorry about that, yes, this will work locally, meaning the remote responder
>> (my script) will run on the same VM than pdns-auth, so pdns-auth will
>> connect using a unix socket with the responder using remote backend.
>> That actually occurs, this is what is shown from the pdns
>> ---
>> alz@nuc /opt/pdns-auth-4.8.3/sbin ./pdns_server
>> Nov 28 14:52:54 This is a standalone pdns
>> Nov 28 14:52:54 Listening on controlsocket in
>> '/var/run/pdns/pdns.controlsocket'
>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>> Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
>> Nov 28 14:52:54 TCP server bound to 0.0.0.0:5300
>> Nov 28 14:52:54 PowerDNS Authoritative Server 4.8.3 (C) 2001-2022
>> PowerDNS.COM BV
>> Nov 28 14:52:54 Using 64-bits mode. Built using gcc 10.2.1 20210110 on Nov
>> 28 2023 11:42:16 by a...@nuc.lesi.com.
>> Nov 28 14:52:54 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free
>> software, and you are welcome to redistribute it according to the terms of
>> the GPL version 2.
>> Nov 28 14:52:54 [stub-resolver] Doing stub resolving for
>> 'auth-4.8.3.security-status.secpoll.powerdns.com.|TXT', using resolvers:
>> 192.168.86.1
>> Nov 28 14:52:54 [stub-resolver] Question for
>> 'auth-4.8.3.security-status.secpoll.powerdns.com.|TXT' got answered by
>> 192.168.86.1
>> Nov 28 14:52:54 Polled security status of version 4.8.3 at startup, no known
>> issues reported: OK
>> Nov 28 14:52:54 Reconnecting to backend
>> Nov 28 14:52:54 PDNSException while filling the zone cache: Exception caught
>> when sending: Could not send a message to remote process
>> —
>> this is what is showed on the responder when the
>> ---
>> 2023-11-28T14:52:54.907-0300 DEBUG handlers/handlers.go:65 pdns request
>> received: {"method": "initialize", "parameters": {"path": "/tmp/pra.sock"}}
>> 2023-11-28T14:52:54.907-0300 DEBUG handlers/handlers.go:50 Response
>> {"result":true}
>> —
>> This same responder script, if using http returns exactly the same json,
>> getAllDomains comes after the initialize, then the lookups, so the responder
>> works ok.
>> Problem is, when i switch to unix socket, throws the error on the red line
>> after the initialize and dies
>
> That's very weird indeed, and unfortunately the unix connector is lacking a
> bit of logging in this area. Any chance you would be able to strace the
> authoritative server process?
>
> Cheers,f
> --
> Remi Gacogne
> PowerDNS.COM
> https://mailman.powerdns.com/mailman/listinfo/pdns-users
___
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
Hi!
On 28/11/2023 19:59, Alexis Fidalgo via Pdns-users wrote:
Sorry about that, yes, this will work locally, meaning the remote
responder (my script) will run on the same VM than pdns-auth, so
pdns-auth will connect using a unix socket with the responder using
remote backend.
That actually occurs, this is what is shown from the pdns
---
alz@nuc /opt/pdns-auth-4.8.3/sbin ./pdns_server
Nov 28 14:52:54 This is a standalone pdns
Nov 28 14:52:54 Listening on controlsocket in
'/var/run/pdns/pdns.controlsocket'
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 TCP server bound to 0.0.0.0:5300
Nov 28 14:52:54 PowerDNS Authoritative Server 4.8.3 (C) 2001-2022
PowerDNS.COM BV
Nov 28 14:52:54 Using 64-bits mode. Built using gcc 10.2.1 20210110 on
Nov 28 2023 11:42:16 by a...@nuc.lesi.com.
Nov 28 14:52:54 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free
software, and you are welcome to redistribute it according to the terms
of the GPL version 2.
Nov 28 14:52:54 [stub-resolver] Doing stub resolving for
'auth-4.8.3.security-status.secpoll.powerdns.com.|TXT', using resolvers:
192.168.86.1
Nov 28 14:52:54 [stub-resolver] Question for
'auth-4.8.3.security-status.secpoll.powerdns.com.|TXT' got answered by
192.168.86.1
Nov 28 14:52:54 Polled security status of version 4.8.3 at startup, no
known issues reported: OK
Nov 28 14:52:54 Reconnecting to backend
Nov 28 14:52:54 PDNSException while filling the zone cache: Exception
caught when sending: Could not send a message to remote process
—
this is what is showed on the responder when the
---
2023-11-28T14:52:54.907-0300 DEBUG handlers/handlers.go:65 pdns request
received: {"method": "initialize", "parameters": {"path": "/tmp/pra.sock"}}
2023-11-28T14:52:54.907-0300 DEBUG handlers/handlers.go:50 Response
{"result":true}
—
This same responder script, if using http returns exactly the same json,
getAllDomains comes after the initialize, then the lookups, so the
responder works ok.
Problem is, when i switch to unix socket, throws the error on the red
line after the initialize and dies
That's very weird indeed, and unfortunately the unix connector is
lacking a bit of logging in this area. Any chance you would be able to
strace the authoritative server process?
Cheers,f
--
Remi Gacogne
PowerDNS.COM BV - https://www.powerdns.com/
___
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
Sorry about that, yes, this will work locally, meaning the remote responder (my
script) will run on the same VM than pdns-auth, so pdns-auth will connect using
a unix socket with the responder using remote backend.
That actually occurs, this is what is shown from the pdns
---
alz@nuc /opt/pdns-auth-4.8.3/sbin ./pdns_server
Nov 28 14:52:54 This is a standalone pdns
Nov 28 14:52:54 Listening on controlsocket in '/var/run/pdns/pdns.controlsocket'
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 UDP server bound to 0.0.0.0:5300
Nov 28 14:52:54 TCP server bound to 0.0.0.0:5300
Nov 28 14:52:54 PowerDNS Authoritative Server 4.8.3 (C) 2001-2022 PowerDNS.COM
BV
Nov 28 14:52:54 Using 64-bits mode. Built using gcc 10.2.1 20210110 on Nov 28
2023 11:42:16 by a...@nuc.lesi.com.
Nov 28 14:52:54 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free
software, and you are welcome to redistribute it according to the terms of the
GPL version 2.
Nov 28 14:52:54 [stub-resolver] Doing stub resolving for
'auth-4.8.3.security-status.secpoll.powerdns.com.|TXT', using resolvers:
192.168.86.1
Nov 28 14:52:54 [stub-resolver] Question for
'auth-4.8.3.security-status.secpoll.powerdns.com.|TXT' got answered by
192.168.86.1
Nov 28 14:52:54 Polled security status of version 4.8.3 at startup, no known
issues reported: OK
Nov 28 14:52:54 Reconnecting to backend
Nov 28 14:52:54 PDNSException while filling the zone cache: Exception caught
when sending: Could not send a message to remote process
—
this is what is showed on the responder when the
---
2023-11-28T14:52:54.907-0300 DEBUG handlers/handlers.go:65 pdns request
received: {"method": "initialize", "parameters": {"path": "/tmp/pra.sock"}}
2023-11-28T14:52:54.907-0300 DEBUG handlers/handlers.go:50 Response
{"result":true}
—
This same responder script, if using http returns exactly the same json,
getAllDomains comes after the initialize, then the lookups, so the responder
works ok.
Problem is, when i switch to unix socket, throws the error on the red line
after the initialize and dies
> On 28 Nov 2023, at 15:10, Walter Parker via Pdns-users
> wrote:
>
> Unclear as to what you mean by “remote backend connected using Unix sockets”
>
> If you mean that you wish to have a process on another system connect using
> Unix sockets, that will not work. Unix sockets only work locally. TCP sockets
> (which is what HTTP uses) are what is required for connections between two
> servers.
>
>
> Walter
>
>
> On Tue, Nov 28, 2023 at 9:39 AM Alexis Fidalgo via Pdns-users
> mailto:pdns-users@mailman.powerdns.com>>
> wrote:
>> Hello all, i have a running remote backend using http with no problems at
>> all, everything runs smoothly.
>> Now i need to enable unix sockets, so the remote backend is not connected
>> anymore using http but unix socket.
>>
>> Nothing has changed beside to listen in the unix socket, meaning im
>> returning exactly the same JSON i return in the http but when i start pdns
>> auth i get this
>>
>> PDNSException while filling the zone cache: Exception caught when sending:
>> Could not send a message to remote process
>>
>> right after the initialize call which im answering with {"result":true} as
>> usual.
>>
>> Same result for 4.7.4 and 4.8.3 , do i need to change anything else since in
>> cannot find in the doc any reference to a different response
>>
>>
>> thanks in advance
>>
>>
>> ___
>> Pdns-users mailing list
>> Pdns-users@mailman.powerdns.com
>> https://mailman.powerdns.com/mailman/listinfo/pdns-users
> ___
> Pdns-users mailing list
> Pdns-users@mailman.powerdns.com
> https://mailman.powerdns.com/mailman/listinfo/pdns-users
___
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
In that case, one of the common problems found when switching to Unix sockets is that the reading process needs to have Read access to the Unix socket (this also means that process must be able to traverse to the directory containing the Unix socket). Please check your file system permissions. Walter On Tue, Nov 28, 2023 at 10:27 AM Brian Candler wrote: > On 28/11/2023 18:10, Walter Parker via Pdns-users wrote: > > Unclear as to what you mean by “remote backend connected using Unix > > sockets” > > See: https://doc.powerdns.com/authoritative/backends/remote.html > > "Remote backend" in this case means "out-of-process", not necessarily on > a different server. > > ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
On 28/11/2023 18:10, Walter Parker via Pdns-users wrote: Unclear as to what you mean by “remote backend connected using Unix sockets” See: https://doc.powerdns.com/authoritative/backends/remote.html "Remote backend" in this case means "out-of-process", not necessarily on a different server. ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend
Unclear as to what you mean by “remote backend connected using Unix sockets”
If you mean that you wish to have a process on another system connect using
Unix sockets, that will not work. Unix sockets only work locally. TCP
sockets (which is what HTTP uses) are what is required for connections
between two servers.
Walter
On Tue, Nov 28, 2023 at 9:39 AM Alexis Fidalgo via Pdns-users <
pdns-users@mailman.powerdns.com> wrote:
> Hello all, i have a running remote backend using http with no problems at
> all, everything runs smoothly.
> Now i need to enable unix sockets, so the remote backend is not connected
> anymore using http but unix socket.
>
> Nothing has changed beside to listen in the unix socket, meaning im
> returning exactly the same JSON i return in the http but when i start pdns
> auth i get this
>
> PDNSException while filling the zone cache: Exception caught when sending:
> Could not send a message to remote process
>
> right after the initialize call which im answering with {"result":true} as
> usual.
>
> Same result for 4.7.4 and 4.8.3 , do i need to change anything else since
> in cannot find in the doc any reference to a different response
>
>
> thanks in advance
>
>
> ___
> Pdns-users mailing list
> Pdns-users@mailman.powerdns.com
> https://mailman.powerdns.com/mailman/listinfo/pdns-users
>
___
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Remote Backend SOA Response
Hi Chris, Adding my 2cts: On Sat, 23 Feb 2019, bert hubert wrote: On Sat, Feb 23, 2019 at 08:12:40PM +1100, Chris Jones wrote: Thanks Bert, but I don’t understand how my backend is doing too much work? How does PowerDNS know what the zone is if my backend doesn’t figure it out? Chris, please carefully read the blog post. It is in there. "The PacketHandler can send many kinds of questions depending on the nature of your zone. For example, it may ask about SOA records, even for zones you do not host in your backend. This is because when a question comes in for ‘www.something.com’, PowerDNS must go hunt for a backend with relevant data." What might help your understanding of what the backend should do is to look at the kind of queries the various sql backends make to the database. Set up a zone using a mysql or postgresql backend and enable query logging on the *sql server, now do some dns queries using dig and look at the querty pattern in the query log. Basically the backend should be rather 'dumb' and just answer the exact questions that pdns asks: on a SOA query for 'sdfsdf.sdf.kjhkjh.domain.net' reply with 'not here'. Pdns will then do a soa query for 'sdf.kjhkjh.domain.net' and continue walking up the tree until it finds something or runs out of tree. ~~ Wieger Opmeer -- There's no time like Guinness time..___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Remote Backend SOA Response
On Sat, Feb 23, 2019 at 08:12:40PM +1100, Chris Jones wrote: > Thanks Bert, but I don’t understand how my backend is doing too much work? > How does PowerDNS know what the zone is if my backend doesn’t figure it out? Chris, please carefully read the blog post. It is in there. "The PacketHandler can send many kinds of questions depending on the nature of your zone. For example, it may ask about SOA records, even for zones you do not host in your backend. This is because when a question comes in for ‘www.something.com’, PowerDNS must go hunt for a backend with relevant data." Bert ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Remote Backend SOA Response
Thanks Bert, but I don’t understand how my backend is doing too much work? How does PowerDNS know what the zone is if my backend doesn’t figure it out? Regards, Chris. On Sat, 23 Feb 2019 at 7:23 pm, bert hubert wrote: > On Sat, Feb 23, 2019 at 03:49:28PM +1100, Chris Jones wrote: > > Hi there, > > > > I am in the process of writing a custom backend with PowerDNS 4.1.5 and I > > have a question on the expected response for SOA records. > > Hi Chris Jones 44, > > It looks like your backend is doing too much work. In > > https://blog.powerdns.com/2015/06/23/what-is-a-powerdns-backend-and-how-do-i-make-it-send-an-nxdomain/ > we clarify what is expected of a backend - just answer questions. PowerDNS > will find out what zones exists and what don't (by asking your backend > several questions). > > Good luck! > > Bert > ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Remote Backend SOA Response
On Sat, Feb 23, 2019 at 03:49:28PM +1100, Chris Jones wrote: > Hi there, > > I am in the process of writing a custom backend with PowerDNS 4.1.5 and I > have a question on the expected response for SOA records. Hi Chris Jones 44, It looks like your backend is doing too much work. In https://blog.powerdns.com/2015/06/23/what-is-a-powerdns-backend-and-how-do-i-make-it-send-an-nxdomain/ we clarify what is expected of a backend - just answer questions. PowerDNS will find out what zones exists and what don't (by asking your backend several questions). Good luck! Bert ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] remote backend questions
On Fri, Feb 10, 2017 at 06:07:38PM -0500, Jeff Weber wrote: > I've been working on implementing a remote backend and I've got a few > questions now that I've gotten far enough along that queries are > answered with my initial attempts. > > I'm using the http connector and I've noticed that the documentation > in a few places talks about a domain-id being passed. For example: > > GET /dnsapi/list/-1/example.com HTTP/1.1 > X-RemoteBackend-domain-id: -1 > > When I implement this query I never see the domain id passed in the > arguments list or as a header. > > Looking at the remote backend unit tests they don't seem to account > for the domain id being in the path here, is it a bug in the > documentation or have a missed something else? This is a bug, sorry about this. I'll fix this. > Separately I've noticed when I query the rest api on one of my systems > backed by sqlite that the id comes back as a string for the domain > with a trailing dot. Based on the source code for the remote backend > requires the domain id be an integer so I can't use this method. > > Is it safe to ignore the concept of domain id with respect to the > remote backend if I'll never be using the powerdns rest api to > manipulate records and only to answer queries? Domain id is optional and setting it -1 is quite safe is you don't need it for anything particular. Aki ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Remote backend docs
* on the Mon, Dec 05, 2016 at 02:25:08PM +0200, Aki Tuomi wrote:
>>> scopeMask = how many bits of requestor IP was used to produce this
>>> result. this should 0 unless you are using the client's (real) IP
>>> somehow to come up with the answer, and if you do, how many bits of the
>>> value was used. 128 (v6) or 32 (v4) means you used up the whole IP, 0
>>> means no bits were used.
>> This sounds like it will work if I want to for example return a different
>> A record depending on the source IP address. However, what if I want to
>> return a specific A record for some source IPs, and *no* A record for
>> other IPs? How do I set a scopeMask on an empty response?
>
> Set scopeMask = 0 when you are not using it. There are two kinds of
> empty responses,
>
> 1. No such domain at all (i have no idea what this domain is)
>
> You return false.
>
> 2. No such record (for requested type, or at all)
>
> You return empty array. If you are asked for ANY or SOA you can reply
> with domain SOA. I am not 100% sure what you should do in your use case,
> but I guess I would check if you can return 1 here if your known values
> are above or below x.x.x.x/1 OR something::/1 and client's IP is on this
> half, so it can cache half the internet. Otherwise you should return 128
> or 32 to be sure.
So for an IPv4 client querying an A record for www.example.com I can do
the following and it will not be cached for any other IP:
[
{
qtype: 'A',
qname: 'www.example.com.',
content: '1.2.3.4',
scopeMask: 32,
}
]
But then if a different client comes along and makes the same request and
I want to respond with nothing, I have to return an empty array:
[]
But then if the first client comes back again, it will get the "nothing
response" too, as that wasn't given a scopeMask. Because you can not apply
a scopeMask to an empty response by doing something like:
[
{
scopeMask: 32
}
]
?
--
Mike Cardwell https://grepular.com https://emailprivacytester.com
OpenPGP Key35BC AF1D 3AA2 1F84 3DC3 B0CF 70A5 F512 0018 461F
XMPP OTR Key 8924 B06A 7917 AAF3 DBB1 BF1B 295C 3C78 3EF1 46B4
signature.asc
Description: Digital signature
___
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Remote backend docs
On 05.12.2016 13:56, Mike Cardwell wrote: > * on the Mon, Dec 05, 2016 at 01:34:07PM +0200, Aki Tuomi wrote: > >>> Just got started with PowerDNS. Developing an application using the >>> Remote backend, but the docs at >>> https://doc.powerdns.com/md/authoritative/backend-remote/ seem a >>> little sparse. What do the following parameters contain and when >>> should they be used? >>> >>> zone_id >>> domain_id >>> auth >>> scopeMask >> zone_id = optional zone id, or use -1 if you do not care about such >> things. This is an opaque integer used by powerdns in various places to >> tell you which zone it means. >> >> domain_id = ditto above, but for domains > It sounds like I can ignore these two. > >> auth = 1 if your data is authoritative for the zone and 0 if not. >> usually you should set this to 1. setting it 0 will put the data into >> additional section. > Hmm. ISTR that answers were having their additional section populated > automatically even though I wasn't setting "auth" to anything. Will > double check that later. But it's good to know that's what the intention > of the variable is at least. > >> scopeMask = how many bits of requestor IP was used to produce this >> result. this should 0 unless you are using the client's (real) IP >> somehow to come up with the answer, and if you do, how many bits of the >> value was used. 128 (v6) or 32 (v4) means you used up the whole IP, 0 >> means no bits were used. > This sounds like it will work if I want to for example return a different > A record depending on the source IP address. However, what if I want to > return a specific A record for some source IPs, and *no* A record for > other IPs? How do I set a scopeMask on an empty response? Set scopeMask = 0 when you are not using it. There are two kinds of empty responses, 1. No such domain at all (i have no idea what this domain is) You return false. 2. No such record (for requested type, or at all) You return empty array. If you are asked for ANY or SOA you can reply with domain SOA. I am not 100% sure what you should do in your use case, but I guess I would check if you can return 1 here if your known values are above or below x.x.x.x/1 OR something::/1 and client's IP is on this half, so it can cache half the internet. Otherwise you should return 128 or 32 to be sure. Aki ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Remote backend docs
* on the Mon, Dec 05, 2016 at 01:34:07PM +0200, Aki Tuomi wrote: >> Just got started with PowerDNS. Developing an application using the >> Remote backend, but the docs at >> https://doc.powerdns.com/md/authoritative/backend-remote/ seem a >> little sparse. What do the following parameters contain and when >> should they be used? >> >> zone_id >> domain_id >> auth >> scopeMask > > zone_id = optional zone id, or use -1 if you do not care about such > things. This is an opaque integer used by powerdns in various places to > tell you which zone it means. > > domain_id = ditto above, but for domains It sounds like I can ignore these two. > auth = 1 if your data is authoritative for the zone and 0 if not. > usually you should set this to 1. setting it 0 will put the data into > additional section. Hmm. ISTR that answers were having their additional section populated automatically even though I wasn't setting "auth" to anything. Will double check that later. But it's good to know that's what the intention of the variable is at least. > scopeMask = how many bits of requestor IP was used to produce this > result. this should 0 unless you are using the client's (real) IP > somehow to come up with the answer, and if you do, how many bits of the > value was used. 128 (v6) or 32 (v4) means you used up the whole IP, 0 > means no bits were used. This sounds like it will work if I want to for example return a different A record depending on the source IP address. However, what if I want to return a specific A record for some source IPs, and *no* A record for other IPs? How do I set a scopeMask on an empty response? -- Mike Cardwell https://grepular.com https://emailprivacytester.com OpenPGP Key35BC AF1D 3AA2 1F84 3DC3 B0CF 70A5 F512 0018 461F XMPP OTR Key 8924 B06A 7917 AAF3 DBB1 BF1B 295C 3C78 3EF1 46B4 signature.asc Description: Digital signature ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] Remote Backend and Query / Packet Cache
Ah perfect, there goes my Sunday :) Thank You Aki AJ On 1 February 2015 at 08:14, Aki Tuomi cmo...@youzen.ext.b2.fi wrote: On Sun, Feb 01, 2015 at 02:41:07AM +, AJ McKee wrote: Being the weekend, I decided to write a HTTP backend for pdns as a fun thing to do. One thing springs to mind however are the packet and query cache. In particular, how they cache. Do they use the remote clients ip as part of the caching key, thus only serving from the cache if the client is repeatedly asking? AFAIK this is not the case. If I added simple bind style views to my backend, would this be pointless? My thinking here, if a request came from netblock A and it was cached, followed by a request from netblock C, C would get the cached answer instead of querying the backend for its corrected view. Is there a way that the remote backend can influence the cache in the response it sends back? I am aware of all the other backend, this is just my fun-time thing to play with the new features. Thanks in advance -- AJ McKee phone: +353 83 1130 545 profile: http://linkedin.com/in/ajmkee jid: aj.mc...@druid-dns.com blog:http://aj.mc-kee.com/ twitter: @ajmckee You can set scopeBits to size of netblock. Should do what you want. Aki -- AJ McKee phone: +353 83 1130 545 profile: http://linkedin.com/in/ajmkee jid: aj.mc...@druid-dns.com blog:http://aj.mc-kee.com/ twitter: @ajmckee ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
