Re: DNS SRV support for LDAP authentication
On Tue, Mar 19, 2019 at 9:01 PM Thomas Munro wrote: > I'd like to commit this soon. Done, after some more comment adjustments. Thanks Daniel and Graham for your feedback! -- Thomas Munro https://enterprisedb.com
Re: DNS SRV support for LDAP authentication
On Sat, Feb 16, 2019 at 10:57 PM Thomas Munro
wrote:
> Yeah. This coding is ugly and StringInfo would be much nicer.
> Thinking about that made me realise that the proposed SRV case should
> also handle multiple SRV records by building a multi-URL string too
> (instead of just taking the first one). I will make it so.
Done, in the attached. Reviewing your comments again, from the top:
On Sat, Feb 2, 2019 at 12:48 AM Daniel Gustafsson wrote:
> + If PostgreSQL was compiled with OpenLDAP as
>
> Should OpenLDAP be wrapped in tags as well? If so, there is
> another “bare” instance in client-auth.sgml which perhaps can be wrapped into
> this patch while at it.
Fixed.
> + ereport(LOG,
> + (errmsg("could not look up a hostlist for %s",
> + domain)));
>
> Should this be \”%s\”?
Yep, fixed.
> + new_uris = psprintf("%s%s%s://%s:%d",
>
> While this construction isn't introduced in this patch, would it not make
> sense
> to convert uris to StringInfo instead to improve readability?
Agreed, fixed.
> + /* Step over this hostname and any spaces. */
>
> Nitpicking on a moved hunk, but single-line comments shouldn’t end in a period
> I believe.
Huh. And yet they are sentences.
tmunro@dogmatix $ git grep '/\* [A-Za-z].*\. \*/' | wc -l
5607
tmunro@dogmatix $ git grep '/\* [A-Za-z].*[a-z] \*/' | wc -l
59500
Yep, you win!
I also fixed a bug where some error messages could pass a NULL pointer
for %s when we don't have a server name.
I also added a hint to the error message you get if it can't find DNS
SRV records, so that if you accidentally activate this feature by
forgetting to set the server name, it'll remind you that you could do
that:
LOG: LDAP authentication could not find DNS SRV records for "example.net"
HINT: Set an LDAP server name explicitly.
Unfortunately, no feedback from MS Active Directory users has been
forthcoming, but I guess that might take a beta release. See below
for new more complete instructions for testing this with an open
source stack (now that I know there is a lazy way to stand up an LDAP
server using the TAP test stuff, I've adjusted the instructions to
work with that).
I'd like to commit this soon.
Some random things I noticed that I am not fixing in this patch but
wanted to mention: I don't like the asymmetry initStringInfo(si),
pfree(si->data). I don't like si->data as a way to get a C string
from a StringInfo. There are a couple of references to StringBuffer
that surely mean StringInfo in comments.
=== How to test ===
1. Start up an LDAP server that has a user test1/secret1 under
dc=example,dc=net (it runs in the background and you can stop it with
SIGINT):
$ make -C src/test/ldap check
$ /usr/local/libexec/slapd -f src/test/ldap/tmp_check/slapd.conf -h
ldap://127.0.0.1:
2. Start up a BIND daemon that has multiple SRV records for LDAP at
example.com:
$ tail -4 /usr/local/etc/namedb/named.conf
zone "example.net" {
type master;
file "/usr/local/etc/namedb/master/example.net";
};
$ cat /usr/local/etc/namedb/master/example.net
$TTL10
@ IN SOA ns.example.net. admin.example.net. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
IN NS ns.example.net.
ns.example.net. IN A 127.0.0.1
example.net. IN A 127.0.0.1
ldap1.example.net. IN A 127.0.0.1
ldap2.example.net. IN A 127.0.0.1
_ldap._tcp.example.net. IN SRV 0 0 ldap1
_ldap._tcp.example.net. IN SRV 1 0 ldap2
3. Tell your OS to talk to that DNS server (and, erm, keep what you
had here so you can restore it later):
$ cat /etc/resolv.conf
nameserver 127.0.0.1
4. Check that standard DNS and LDAP tools can find their way to your
LDAP servers via these breadcrumbs:
$ host -t srv _ldap._tcp.example.net
_ldap._tcp.example.net has SRV record 0 0 ldap1.example.net.
_ldap._tcp.example.net has SRV record 1 0 ldap2.example.net.
$ ldapsearch -H 'ldap:///dc%3Dexample%2Cdc%3Dnet' -b 'dc=example,dc=net'
5. Tell PostgreSQL to use SRV records in pg_hba.conf using either of
these styles:
host all test1 127.0.0.1/32 ldap basedn="dc=example,dc=net"
host all test1 127.0.0.1/32 ldap ldapurl="ldap:///dc=example,dc=net?uid?sub;
6. Check that you now log in as test1/secret1:
$ psql -h 127.0.0.1 postgres test1
--
Thomas Munro
https://enterprisedb.com
0001-Add-DNS-SRV-support-for-LDAP-server-discovery-v3.patch
Description: Binary data
Re: DNS SRV support for LDAP authentication
On Sat, Feb 2, 2019 at 12:48 AM Daniel Gustafsson wrote:
> + new_uris = psprintf("%s%s%s://%s:%d",
>
> While this construction isn't introduced in this patch, would it not make
> sense
> to convert uris to StringInfo instead to improve readability?
Yeah. This coding is ugly and StringInfo would be much nicer.
Thinking about that made me realise that the proposed SRV case should
also handle multiple SRV records by building a multi-URL string too
(instead of just taking the first one). I will make it so.
--
Thomas Munro
http://www.enterprisedb.com
Re: DNS SRV support for LDAP authentication
On Sat, Feb 2, 2019 at 10:34 PM Graham Leggett wrote: > On 02 Feb 2019, at 01:57, Thomas Munro wrote: > > On Sat, Feb 2, 2019 at 9:25 AM Graham Leggett wrote: > >> Does this support SSL/TLS? > > I didn't try it myself but I found several claims that it works. I > > see complaints that it always looks for _ldap._tcp and not _ldaps._tcp > > as you might expect when using ldascheme=ldaps, but that doesn't seem > > to be a big problem. As for ldaptls=1, that must work because it > > doesn't even negotiate that until after the connection is made. > > If the LDAP server was bound to port 636, how would the client know to use a > direct SSL/TLS connection and not STARTTLS? SRV records don't control that, so it looks like the person configuring pg_hba.conf would simply have to know which of the following formats to use: ldapurl=ldap:///dc=example,dc=net ldapurl=ldap:///dc=example,dc=net ldaptls=1 ldapurl=ldaps:///dc=example,dc=net Only the port and host are obtained from the SRV record, not those protocol details. Nothing in RFC 2782 prevents you from setting up separate "_ldaps._tcp" SRV records (and I can find discussions of that idea on the net) and then writing custom resolver code that knows to look for that, but the OpenLDAP code we're using (for compatibility with the command line tools) is hard coded to use "_ldap._tcp" always[1]. Active Directory apparently automatically creates only "_ldap._tcp" SRV records according to its documentation and that's the user base I was aiming for with this patch, so I think it makes sense to just use the routines they provide, despite this weakness. [1] https://github.com/openldap/openldap/blob/b06f5b0493937fc28f2cc86df1d7f464aa4504d8/libraries/libldap/dnssrv.c#L276 -- Thomas Munro http://www.enterprisedb.com
Re: DNS SRV support for LDAP authentication
On 02 Feb 2019, at 01:57, Thomas Munro wrote: > On Sat, Feb 2, 2019 at 9:25 AM Graham Leggett wrote: >> On 25 Sep 2018, at 04:09, Thomas Munro wrote: >>> Some people like to use DNS SRV records to advertise LDAP servers on >>> their network. Microsoft Active Directory is usually (always?) set up >>> that way. Here is a patch to allow our LDAP auth module to support >>> that kind of discovery. >> >> Does this support SSL/TLS? > > I didn't try it myself but I found several claims that it works. I > see complaints that it always looks for _ldap._tcp and not _ldaps._tcp > as you might expect when using ldascheme=ldaps, but that doesn't seem > to be a big problem. As for ldaptls=1, that must work because it > doesn't even negotiate that until after the connection is made. If the LDAP server was bound to port 636, how would the client know to use a direct SSL/TLS connection and not STARTTLS? Regards, Graham —
Re: DNS SRV support for LDAP authentication
On Sat, Feb 2, 2019 at 9:25 AM Graham Leggett wrote: > On 25 Sep 2018, at 04:09, Thomas Munro wrote: > > Some people like to use DNS SRV records to advertise LDAP servers on > > their network. Microsoft Active Directory is usually (always?) set up > > that way. Here is a patch to allow our LDAP auth module to support > > that kind of discovery. > > Does this support SSL/TLS? I didn't try it myself but I found several claims that it works. I see complaints that it always looks for _ldap._tcp and not _ldaps._tcp as you might expect when using ldascheme=ldaps, but that doesn't seem to be a big problem. As for ldaptls=1, that must work because it doesn't even negotiate that until after the connection is made. -- Thomas Munro http://www.enterprisedb.com
Re: DNS SRV support for LDAP authentication
On 25 Sep 2018, at 04:09, Thomas Munro wrote: > Some people like to use DNS SRV records to advertise LDAP servers on > their network. Microsoft Active Directory is usually (always?) set up > that way. Here is a patch to allow our LDAP auth module to support > that kind of discovery. Does this support SSL/TLS? Regards, Graham —
Re: DNS SRV support for LDAP authentication
> On 25 Sep 2018, at 04:09, Thomas Munro wrote:
> Some people like to use DNS SRV records to advertise LDAP servers on
> their network. Microsoft Active Directory is usually (always?) set up
> that way. Here is a patch to allow our LDAP auth module to support
> that kind of discovery. It copies the convention of the OpenLDAP
> command line tools: if you give it a URL that has no hostname, it'll
> try to extract a domain name from the bind DN, and then ask your DNS
> server for a SRV record for LDAP-over-TCP at that domain. The
> OpenLDAP version of libldap.so exports the magic to do that, so the
> patch is very small (but the infrastructure set-up to test it is a bit
> of a schlep, see below). I'll add this to the next Commitfest.
Sounds like a reasonable feature.
> Testing instructions for (paths and commands given for FreeBSD, adjust
> as appropriate):
Trying this quickly on macOS while at a conference didn’t yield much success,
will do another attempt when I’m on a more reliable connection.
> This is a first draft. Not tested much yet. I wonder if
> HAVE_LDAP_INITIALIZE is a reasonable way to detact OpenLDAP. The
> documentation was written in about 7 seconds so probably needs work.
> There is probably a Windowsy way to do this too but I didn't look into
> that.
Reading through the patch, and related OpenLDAP code, this seems like a good
approach. A few small comments:
+ If PostgreSQL was compiled with OpenLDAP as
Should OpenLDAP be wrapped in tags as well? If so, there is
another “bare” instance in client-auth.sgml which perhaps can be wrapped into
this patch while at it.
+ ereport(LOG,
+ (errmsg("could not look up a hostlist for %s",
+ domain)));
Should this be \”%s\”?
+ new_uris = psprintf("%s%s%s://%s:%d",
While this construction isn't introduced in this patch, would it not make sense
to convert uris to StringInfo instead to improve readability?
+ /* Step over this hostname and any spaces. */
Nitpicking on a moved hunk, but single-line comments shouldn’t end in a period
I believe.
cheers ./daniel
Re: DNS SRV support for LDAP authentication
On Wed, Nov 7, 2018 at 4:39 PM Thomas Munro
wrote:
> On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro
> wrote:
> > Some people like to use DNS SRV records to advertise LDAP servers on
> > their network. Microsoft Active Directory is usually (always?) set up
> > that way. Here is a patch to allow our LDAP auth module to support
> > that kind of discovery.
Rebased.
I took the liberty of CCing Mark Cave-Ayland, who had some great
advice on the last round of LDAP feature tweaks[1]. Mark, if you have
any comments on the sanity of this proposal, they'd be much
appreciated, otherwise of course please feel free to ignore. Thanks!
[1]
https://www.postgresql.org/message-id/flat/CAEepm%3D0XTkYvMci0WRubZcf_1am8%3DgP%3D7oJErpsUfRYcKF2gwg%40mail.gmail.com
--
Thomas Munro
http://www.enterprisedb.com
From 4be9142998e76614146f88c825179264573bcf7c Mon Sep 17 00:00:00 2001
From: Thomas Munro
Date: Fri, 16 Nov 2018 14:32:00 +1300
Subject: [PATCH] Add DNS SRV support for LDAP server discovery.
LDAP servers can be advertised on a network by registering DNS SRV
records for _ldap._tcp.. The OpenLDAP command-line tools
know how to find servers via those records, if no server name is
provided by the user. Teach PostgreSQL to follow the same convention
using non-standard extensions provided by OpenLDAP, where available.
Author: Thomas Munro
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm=2hAnSfhdsd6vXsM6VZVN0br-FbAZ-O+Swk18S5HkCP=A@mail.gmail.com
---
doc/src/sgml/client-auth.sgml | 19 ++
src/backend/libpq/auth.c | 113 +-
src/backend/libpq/hba.c | 2 +
3 files changed, 105 insertions(+), 29 deletions(-)
diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-auth.sgml
index c2114021c3..f9e7416c79 100644
--- a/doc/src/sgml/client-auth.sgml
+++ b/doc/src/sgml/client-auth.sgml
@@ -1671,6 +1671,16 @@ ldap[s]://host[:port]/ldapsearchattribute=uid.
+
+ If PostgreSQL was compiled with OpenLDAP as
+ the LDAP client library, the ldapserver setting may be
+ omitted. In that case, the hostname and port are looked up via DNS
+ service records. The "SRV" records for the service
+ _ldap._tcp.domain are requested, where
+ domain is extracted from basedn.
+ This follows a convention used by OpenLDAP command-line tools.
+
+
Here is an example for a simple-bind LDAP configuration:
@@ -1716,6 +1726,15 @@ host ... ldap ldapserver=ldap.example.net ldapbasedn="dc=example, dc=net" ldapse
+
+Here is an example for a search+bind configuration that uses DNS SRV
+discovery to find the hostname and port for the LDAP service using the
+domain name example.net":
+
+host ... ldap ldapurl="ldap:///ou=people,dc=example,dc=net?cn;
+
+
+
Since LDAP often uses commas and spaces to separate the different
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 4f9d697d6d..cb62540c9f 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -2356,37 +2356,81 @@ InitializeLDAPConnection(Port *port, LDAP **ldap)
char *uris = NULL;
/*
- * We have a space-separated list of hostnames. Convert it
- * to a space-separated list of URIs.
+ * If the user provided no hostname, we can ask OpenLDAP to try to
+ * find one by extracting a domain name from the base DN and then
+ * using a DSN SRV record for _ldap._tcp.. If one or more
+ * such SRV records have been defined, we can get a hostname and
+ * port. The same convention is used by the OpenLDAP command line
+ * tools.
*/
- do
+ if (!hostnames || hostnames[0] == '\0')
{
- char *hostname;
- size_t hostname_size;
- char *new_uris;
-
- /* Find the leading hostname. */
- hostname_size = strcspn(hostnames, " ");
- hostname = pnstrdup(hostnames, hostname_size);
-
- /* Append a URI for this hostname. */
- new_uris = psprintf("%s%s%s://%s:%d",
-uris ? uris : "",
-uris ? " " : "",
-scheme,
-hostname,
-port->hba->ldapport);
-
- pfree(hostname);
- if (uris)
-pfree(uris);
- uris = new_uris;
-
- /* Step over this hostname and any spaces. */
- hostnames += hostname_size;
- while (*hostnames == ' ')
-++hostnames;
- } while (*hostnames);
+ char *domain;
+ char *hostlist;
+ char *end;
+
+ /* ou=blah,dc=foo,dc=bar -> foo.bar */
+ if (ldap_dn2domain(port->hba->ldapbasedn, ))
+ {
+ereport(LOG,
+ (errmsg("could not extract domain name from basedn")));
+return STATUS_ERROR;
+ }
+ /* Look up host:port using DNS SRV for _ldap._tcp.foo.bar */
+ if (ldap_domain2hostlist(domain, ))
+ {
+ereport(LOG,
+ (errmsg("could not look up a hostlist for %s",
+domain)));
+ldap_memfree(domain);
+return STATUS_ERROR;
+ }
+ ldap_memfree(domain);
+ /*
+ * OpenLDAP already ordered by weight and shuffled equal weight
+ * servers, so we'll just take the first one. The
Re: DNS SRV support for LDAP authentication
On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro wrote: > Some people like to use DNS SRV records to advertise LDAP servers on > their network. Microsoft Active Directory is usually (always?) set up > that way. Here is a patch to allow our LDAP auth module to support > that kind of discovery. It copies the convention of the OpenLDAP > command line tools: if you give it a URL that has no hostname, it'll > try to extract a domain name from the bind DN, and then ask your DNS > server for a SRV record for LDAP-over-TCP at that domain. The > OpenLDAP version of libldap.so exports the magic to do that, so the > patch is very small (but the infrastructure set-up to test it is a bit > of a schlep, see below). I'll add this to the next Commitfest. > > [long tedious explanation of how to set up a test with BIND and OpenLDAP on > Unix] Of course the point of this is not really for the Unix-based set-up I described, but for Microsoft environments with one or more AD servers and a PostgreSQL server running on (eg) Linux that wants to find AD. In such environments, from what I can tell, the following should work: Standard DNS lookup tools should be able to find SRV records advertising the host, port and weight (priority) of any AD servers on the network: $ nslookup -type=any _ldap._tcp.YOUR.DOMAIN $ dig srv _ldap._tcp.YOUR.DOMAIN $ host -t srv _ldp._tcp.YOUR.DOMAIN OpenLDAP command line tools should be able to find the AD server via those SRV records, extracting YOUR.DOMAIN from the base DN: $ ldapsearch -H 'ldap:///dc%3DYOUR%2Cdc%3DDOMAIN' ... pg_hba.conf with an explicit LDAP server name should be able to talk to Active Directory without using this patch with something like: host all all 127.0.0.1/32 ldap ldapurl="ldap://YOUR-AD-SERVER.YOUR.DOMAIN/dc=YOUR,dc=DOMAIN?cn?sub; pg_hba.conf using this patch should be able to discover the LDAP server via SRV if you take out the server name: host all all 127.0.0.1/32 ldap ldapurl="ldap:///dc=YOUR,dc=DOMAIN?cn?sub; I'm hoping someone can help test this in a real Active Directory environment. -- Thomas Munro http://www.enterprisedb.com
Re: DNS SRV support for LDAP authentication
On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro
wrote:
> 2. Define a new zone for testing, by adding the following to the end
> 3. Create that zone file in /usr/local/etc/namedb/master/my.test.domain:
Oops, I changed my testing domain name in the middle of my experiment,
but pasted the older version into the previous message. Here are the
corrected steps 2 and 3, consistent with the rest:
= end of /usr/local/etc/namedb/named.conf =
zone "my-domain.com" {
type master;
file "/usr/local/etc/namedb/master/my-domain.com";
};
=
= /usr/local/etc/namedb/master/my-domain.com =
$TTL10
@ IN SOA ns.my-domain.com. admin.my-domain.com. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
IN NS ns.my-domain.com.
ns.my-domain.com. IN A 127.0.0.1
my-domain.com. IN A 127.0.0.1
ldap-server.my-domain.com. IN A 127.0.0.1
_ldap._tcp.my-domain.com. IN SRV 0 0 389
ldap-server
=
--
Thomas Munro
http://www.enterprisedb.com
DNS SRV support for LDAP authentication
Hello hackers,
Some people like to use DNS SRV records to advertise LDAP servers on
their network. Microsoft Active Directory is usually (always?) set up
that way. Here is a patch to allow our LDAP auth module to support
that kind of discovery. It copies the convention of the OpenLDAP
command line tools: if you give it a URL that has no hostname, it'll
try to extract a domain name from the bind DN, and then ask your DNS
server for a SRV record for LDAP-over-TCP at that domain. The
OpenLDAP version of libldap.so exports the magic to do that, so the
patch is very small (but the infrastructure set-up to test it is a bit
of a schlep, see below). I'll add this to the next Commitfest.
Testing instructions for (paths and commands given for FreeBSD, adjust
as appropriate):
1. Install BIND:
$ sudo pkg install bind99
2. Define a new zone for testing, by adding the following to the end
of /usr/local/etc/namedb/named.conf:
= 8< =
zone "my.test.domain" {
type master;
file "/usr/local/etc/namedb/master/my.test.domain";
};
= 8< =
3. Create that zone file in /usr/local/etc/namedb/master/my.test.domain:
= 8< =
$TTL10
@ IN SOA ns.my.test.domain. admin.my.test.domain. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
IN NS ns.my.test.domain.
ns.my.test.domain. IN A 127.0.0.1
my.test.domain. IN A 127.0.0.1
ldap-server.my.test.domain. IN A 127.0.0.1
_ldap._tcp.my.test.domain. IN SRV 0 0 389
ldap-server
= 8< =
4. Start up bind:
# service named onestart
5. Confirm that SRV lookups find our record:
$ dig @localhost _ldap._tcp.my-domain.com SRV
...
;; ANSWER SECTION:
_ldap._tcp.my-domain.com. 10IN SRV 0 0 389
ldap-server.my-domain.com.
6. Tell your system libraries to use this DNS server by temporarily
changing /etc/resolv.conf to say:
= 8< =
nameserver 127.0.0.1
= 8< =
7. Confirm that the OpenLDAP tools can look that SRV record up:
$ ldapsearch -H 'ldap:///ou%3Dblah%2Cdc%3Dmy-domain%2Cdc%3Dcom'
(That's "ou=blah,dc=my-domain,dc=com" URL-encoded, from which
"my-domain.com" will be extracted.) You should see that it's trying
to connect to ldap-server port 389, and you can stick 'x' on the end
of it to see what it looks like when it can't find a SRV record, as a
sanity check:
$ ldapsearch -H 'ldap:///ou%3Dblah%2Cdc%3Dmy-domain%2Cdc%3Dcomx'
DNS SRV: Could not turn domain=my-domain.comx into a hostlist
8. Set up an LDAP server listening on localhost port 389, and create
a user, such that you can actually authenticate from PostgreSQL with
it. Gory details omitted. First test that you can log in with LDAP
authentication when using a pg_hba.conf line like this:
host all fred 127.0.0.1/32 ldap
ldapurl="ldap://ldap-server.my-domain.com/dc=my-domain,dc=com?cn?sub;
9. Next apply the patch and verify that you can take out the hostname
and let it be discovered via DNS SRV:
host all fred 127.0.0.1/32 ldap ldapurl="ldap:///dc=my-domain,dc=com?cn?sub;
(You can stick some elog(LOG, ...) lines into
InitializeLDAPConnection() if you want to check that
ldap_domain2hostlist() is in fact finding the hostname and port.)
This is a first draft. Not tested much yet. I wonder if
HAVE_LDAP_INITIALIZE is a reasonable way to detact OpenLDAP. The
documentation was written in about 7 seconds so probably needs work.
There is probably a Windowsy way to do this too but I didn't look into
that.
--
Thomas Munro
http://www.enterprisedb.com
0001-Add-DNS-SRV-support-for-LDAP-server-discovery.patch
Description: Binary data
