Re: DNS SRV support for LDAP authentication
On Tue, Mar 19, 2019 at 9:01 PM Thomas Munro wrote: > I'd like to commit this soon. Done, after some more comment adjustments. Thanks Daniel and Graham for your feedback! -- Thomas Munro https://enterprisedb.com
Re: DNS SRV support for LDAP authentication
On Sat, Feb 16, 2019 at 10:57 PM Thomas Munro wrote: > Yeah. This coding is ugly and StringInfo would be much nicer. > Thinking about that made me realise that the proposed SRV case should > also handle multiple SRV records by building a multi-URL string too > (instead of just taking the first one). I will make it so. Done, in the attached. Reviewing your comments again, from the top: On Sat, Feb 2, 2019 at 12:48 AM Daniel Gustafsson wrote: > + If PostgreSQL was compiled with OpenLDAP as > > Should OpenLDAP be wrapped in tags as well? If so, there is > another “bare” instance in client-auth.sgml which perhaps can be wrapped into > this patch while at it. Fixed. > + ereport(LOG, > + (errmsg("could not look up a hostlist for %s", > + domain))); > > Should this be \”%s\”? Yep, fixed. > + new_uris = psprintf("%s%s%s://%s:%d", > > While this construction isn't introduced in this patch, would it not make > sense > to convert uris to StringInfo instead to improve readability? Agreed, fixed. > + /* Step over this hostname and any spaces. */ > > Nitpicking on a moved hunk, but single-line comments shouldn’t end in a period > I believe. Huh. And yet they are sentences. tmunro@dogmatix $ git grep '/\* [A-Za-z].*\. \*/' | wc -l 5607 tmunro@dogmatix $ git grep '/\* [A-Za-z].*[a-z] \*/' | wc -l 59500 Yep, you win! I also fixed a bug where some error messages could pass a NULL pointer for %s when we don't have a server name. I also added a hint to the error message you get if it can't find DNS SRV records, so that if you accidentally activate this feature by forgetting to set the server name, it'll remind you that you could do that: LOG: LDAP authentication could not find DNS SRV records for "example.net" HINT: Set an LDAP server name explicitly. Unfortunately, no feedback from MS Active Directory users has been forthcoming, but I guess that might take a beta release. See below for new more complete instructions for testing this with an open source stack (now that I know there is a lazy way to stand up an LDAP server using the TAP test stuff, I've adjusted the instructions to work with that). I'd like to commit this soon. Some random things I noticed that I am not fixing in this patch but wanted to mention: I don't like the asymmetry initStringInfo(si), pfree(si->data). I don't like si->data as a way to get a C string from a StringInfo. There are a couple of references to StringBuffer that surely mean StringInfo in comments. === How to test === 1. Start up an LDAP server that has a user test1/secret1 under dc=example,dc=net (it runs in the background and you can stop it with SIGINT): $ make -C src/test/ldap check $ /usr/local/libexec/slapd -f src/test/ldap/tmp_check/slapd.conf -h ldap://127.0.0.1: 2. Start up a BIND daemon that has multiple SRV records for LDAP at example.com: $ tail -4 /usr/local/etc/namedb/named.conf zone "example.net" { type master; file "/usr/local/etc/namedb/master/example.net"; }; $ cat /usr/local/etc/namedb/master/example.net $TTL10 @ IN SOA ns.example.net. admin.example.net. ( 2 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 604800 ) ; Negative Cache TTL IN NS ns.example.net. ns.example.net. IN A 127.0.0.1 example.net. IN A 127.0.0.1 ldap1.example.net. IN A 127.0.0.1 ldap2.example.net. IN A 127.0.0.1 _ldap._tcp.example.net. IN SRV 0 0 ldap1 _ldap._tcp.example.net. IN SRV 1 0 ldap2 3. Tell your OS to talk to that DNS server (and, erm, keep what you had here so you can restore it later): $ cat /etc/resolv.conf nameserver 127.0.0.1 4. Check that standard DNS and LDAP tools can find their way to your LDAP servers via these breadcrumbs: $ host -t srv _ldap._tcp.example.net _ldap._tcp.example.net has SRV record 0 0 ldap1.example.net. _ldap._tcp.example.net has SRV record 1 0 ldap2.example.net. $ ldapsearch -H 'ldap:///dc%3Dexample%2Cdc%3Dnet' -b 'dc=example,dc=net' 5. Tell PostgreSQL to use SRV records in pg_hba.conf using either of these styles: host all test1 127.0.0.1/32 ldap basedn="dc=example,dc=net" host all test1 127.0.0.1/32 ldap ldapurl="ldap:///dc=example,dc=net?uid?sub"; 6. Check that you now log in as test1/secret1: $ psql -h 127.0.0.1 postgres test1 -- Thomas Munro https://enterprisedb.com 0001-Add-DNS-SRV-support-for-LDAP-server-discovery-v3.patch Description: Binary data
Re: DNS SRV support for LDAP authentication
On Sat, Feb 2, 2019 at 12:48 AM Daniel Gustafsson wrote: > + new_uris = psprintf("%s%s%s://%s:%d", > > While this construction isn't introduced in this patch, would it not make > sense > to convert uris to StringInfo instead to improve readability? Yeah. This coding is ugly and StringInfo would be much nicer. Thinking about that made me realise that the proposed SRV case should also handle multiple SRV records by building a multi-URL string too (instead of just taking the first one). I will make it so. -- Thomas Munro http://www.enterprisedb.com
Re: DNS SRV support for LDAP authentication
On Sat, Feb 2, 2019 at 10:34 PM Graham Leggett wrote: > On 02 Feb 2019, at 01:57, Thomas Munro wrote: > > On Sat, Feb 2, 2019 at 9:25 AM Graham Leggett wrote: > >> Does this support SSL/TLS? > > I didn't try it myself but I found several claims that it works. I > > see complaints that it always looks for _ldap._tcp and not _ldaps._tcp > > as you might expect when using ldascheme=ldaps, but that doesn't seem > > to be a big problem. As for ldaptls=1, that must work because it > > doesn't even negotiate that until after the connection is made. > > If the LDAP server was bound to port 636, how would the client know to use a > direct SSL/TLS connection and not STARTTLS? SRV records don't control that, so it looks like the person configuring pg_hba.conf would simply have to know which of the following formats to use: ldapurl=ldap:///dc=example,dc=net ldapurl=ldap:///dc=example,dc=net ldaptls=1 ldapurl=ldaps:///dc=example,dc=net Only the port and host are obtained from the SRV record, not those protocol details. Nothing in RFC 2782 prevents you from setting up separate "_ldaps._tcp" SRV records (and I can find discussions of that idea on the net) and then writing custom resolver code that knows to look for that, but the OpenLDAP code we're using (for compatibility with the command line tools) is hard coded to use "_ldap._tcp" always[1]. Active Directory apparently automatically creates only "_ldap._tcp" SRV records according to its documentation and that's the user base I was aiming for with this patch, so I think it makes sense to just use the routines they provide, despite this weakness. [1] https://github.com/openldap/openldap/blob/b06f5b0493937fc28f2cc86df1d7f464aa4504d8/libraries/libldap/dnssrv.c#L276 -- Thomas Munro http://www.enterprisedb.com
Re: DNS SRV support for LDAP authentication
On 02 Feb 2019, at 01:57, Thomas Munro wrote: > On Sat, Feb 2, 2019 at 9:25 AM Graham Leggett wrote: >> On 25 Sep 2018, at 04:09, Thomas Munro wrote: >>> Some people like to use DNS SRV records to advertise LDAP servers on >>> their network. Microsoft Active Directory is usually (always?) set up >>> that way. Here is a patch to allow our LDAP auth module to support >>> that kind of discovery. >> >> Does this support SSL/TLS? > > I didn't try it myself but I found several claims that it works. I > see complaints that it always looks for _ldap._tcp and not _ldaps._tcp > as you might expect when using ldascheme=ldaps, but that doesn't seem > to be a big problem. As for ldaptls=1, that must work because it > doesn't even negotiate that until after the connection is made. If the LDAP server was bound to port 636, how would the client know to use a direct SSL/TLS connection and not STARTTLS? Regards, Graham —
Re: DNS SRV support for LDAP authentication
On Sat, Feb 2, 2019 at 9:25 AM Graham Leggett wrote: > On 25 Sep 2018, at 04:09, Thomas Munro wrote: > > Some people like to use DNS SRV records to advertise LDAP servers on > > their network. Microsoft Active Directory is usually (always?) set up > > that way. Here is a patch to allow our LDAP auth module to support > > that kind of discovery. > > Does this support SSL/TLS? I didn't try it myself but I found several claims that it works. I see complaints that it always looks for _ldap._tcp and not _ldaps._tcp as you might expect when using ldascheme=ldaps, but that doesn't seem to be a big problem. As for ldaptls=1, that must work because it doesn't even negotiate that until after the connection is made. -- Thomas Munro http://www.enterprisedb.com
Re: DNS SRV support for LDAP authentication
On 25 Sep 2018, at 04:09, Thomas Munro wrote: > Some people like to use DNS SRV records to advertise LDAP servers on > their network. Microsoft Active Directory is usually (always?) set up > that way. Here is a patch to allow our LDAP auth module to support > that kind of discovery. Does this support SSL/TLS? Regards, Graham —
Re: DNS SRV support for LDAP authentication
> On 25 Sep 2018, at 04:09, Thomas Munro wrote: > Some people like to use DNS SRV records to advertise LDAP servers on > their network. Microsoft Active Directory is usually (always?) set up > that way. Here is a patch to allow our LDAP auth module to support > that kind of discovery. It copies the convention of the OpenLDAP > command line tools: if you give it a URL that has no hostname, it'll > try to extract a domain name from the bind DN, and then ask your DNS > server for a SRV record for LDAP-over-TCP at that domain. The > OpenLDAP version of libldap.so exports the magic to do that, so the > patch is very small (but the infrastructure set-up to test it is a bit > of a schlep, see below). I'll add this to the next Commitfest. Sounds like a reasonable feature. > Testing instructions for (paths and commands given for FreeBSD, adjust > as appropriate): Trying this quickly on macOS while at a conference didn’t yield much success, will do another attempt when I’m on a more reliable connection. > This is a first draft. Not tested much yet. I wonder if > HAVE_LDAP_INITIALIZE is a reasonable way to detact OpenLDAP. The > documentation was written in about 7 seconds so probably needs work. > There is probably a Windowsy way to do this too but I didn't look into > that. Reading through the patch, and related OpenLDAP code, this seems like a good approach. A few small comments: + If PostgreSQL was compiled with OpenLDAP as Should OpenLDAP be wrapped in tags as well? If so, there is another “bare” instance in client-auth.sgml which perhaps can be wrapped into this patch while at it. + ereport(LOG, + (errmsg("could not look up a hostlist for %s", + domain))); Should this be \”%s\”? + new_uris = psprintf("%s%s%s://%s:%d", While this construction isn't introduced in this patch, would it not make sense to convert uris to StringInfo instead to improve readability? + /* Step over this hostname and any spaces. */ Nitpicking on a moved hunk, but single-line comments shouldn’t end in a period I believe. cheers ./daniel
Re: DNS SRV support for LDAP authentication
On Wed, Nov 7, 2018 at 4:39 PM Thomas Munro wrote: > On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro > wrote: > > Some people like to use DNS SRV records to advertise LDAP servers on > > their network. Microsoft Active Directory is usually (always?) set up > > that way. Here is a patch to allow our LDAP auth module to support > > that kind of discovery. Rebased. I took the liberty of CCing Mark Cave-Ayland, who had some great advice on the last round of LDAP feature tweaks[1]. Mark, if you have any comments on the sanity of this proposal, they'd be much appreciated, otherwise of course please feel free to ignore. Thanks! [1] https://www.postgresql.org/message-id/flat/CAEepm%3D0XTkYvMci0WRubZcf_1am8%3DgP%3D7oJErpsUfRYcKF2gwg%40mail.gmail.com -- Thomas Munro http://www.enterprisedb.com From 4be9142998e76614146f88c825179264573bcf7c Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Fri, 16 Nov 2018 14:32:00 +1300 Subject: [PATCH] Add DNS SRV support for LDAP server discovery. LDAP servers can be advertised on a network by registering DNS SRV records for _ldap._tcp.. The OpenLDAP command-line tools know how to find servers via those records, if no server name is provided by the user. Teach PostgreSQL to follow the same convention using non-standard extensions provided by OpenLDAP, where available. Author: Thomas Munro Reviewed-by: Discussion: https://postgr.es/m/CAEepm=2hAnSfhdsd6vXsM6VZVN0br-FbAZ-O+Swk18S5HkCP=A@mail.gmail.com --- doc/src/sgml/client-auth.sgml | 19 ++ src/backend/libpq/auth.c | 113 +- src/backend/libpq/hba.c | 2 + 3 files changed, 105 insertions(+), 29 deletions(-) diff --git a/doc/src/sgml/client-auth.sgml b/doc/src/sgml/client-auth.sgml index c2114021c3..f9e7416c79 100644 --- a/doc/src/sgml/client-auth.sgml +++ b/doc/src/sgml/client-auth.sgml @@ -1671,6 +1671,16 @@ ldap[s]://host[:port]/ldapsearchattribute=uid. + + If PostgreSQL was compiled with OpenLDAP as + the LDAP client library, the ldapserver setting may be + omitted. In that case, the hostname and port are looked up via DNS + service records. The "SRV" records for the service + _ldap._tcp.domain are requested, where + domain is extracted from basedn. + This follows a convention used by OpenLDAP command-line tools. + + Here is an example for a simple-bind LDAP configuration: @@ -1716,6 +1726,15 @@ host ... ldap ldapserver=ldap.example.net ldapbasedn="dc=example, dc=net" ldapse + +Here is an example for a search+bind configuration that uses DNS SRV +discovery to find the hostname and port for the LDAP service using the +domain name example.net": + +host ... ldap ldapurl="ldap:///ou=people,dc=example,dc=net?cn"; + + + Since LDAP often uses commas and spaces to separate the different diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c index 4f9d697d6d..cb62540c9f 100644 --- a/src/backend/libpq/auth.c +++ b/src/backend/libpq/auth.c @@ -2356,37 +2356,81 @@ InitializeLDAPConnection(Port *port, LDAP **ldap) char *uris = NULL; /* - * We have a space-separated list of hostnames. Convert it - * to a space-separated list of URIs. + * If the user provided no hostname, we can ask OpenLDAP to try to + * find one by extracting a domain name from the base DN and then + * using a DSN SRV record for _ldap._tcp.. If one or more + * such SRV records have been defined, we can get a hostname and + * port. The same convention is used by the OpenLDAP command line + * tools. */ - do + if (!hostnames || hostnames[0] == '\0') { - char *hostname; - size_t hostname_size; - char *new_uris; - - /* Find the leading hostname. */ - hostname_size = strcspn(hostnames, " "); - hostname = pnstrdup(hostnames, hostname_size); - - /* Append a URI for this hostname. */ - new_uris = psprintf("%s%s%s://%s:%d", -uris ? uris : "", -uris ? " " : "", -scheme, -hostname, -port->hba->ldapport); - - pfree(hostname); - if (uris) -pfree(uris); - uris = new_uris; - - /* Step over this hostname and any spaces. */ - hostnames += hostname_size; - while (*hostnames == ' ') -++hostnames; - } while (*hostnames); + char *domain; + char *hostlist; + char *end; + + /* ou=blah,dc=foo,dc=bar -> foo.bar */ + if (ldap_dn2domain(port->hba->ldapbasedn, &domain)) + { +ereport(LOG, + (errmsg("could not extract domain name from basedn"))); +return STATUS_ERROR; + } + /* Look up host:port using DNS SRV for _ldap._tcp.foo.bar */ + if (ldap_domain2hostlist(domain, &hostlist)) + { +ereport(LOG, + (errmsg("could not look up a hostlist for %s", +domain))); +ldap_memfree(domain); +return STATUS_ERROR; + } + ldap_memfree(domain); + /* + * OpenLDAP already ordered by weight and shuffled equal weight + * servers, so we'll just take the firs
Re: DNS SRV support for LDAP authentication
On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro wrote: > Some people like to use DNS SRV records to advertise LDAP servers on > their network. Microsoft Active Directory is usually (always?) set up > that way. Here is a patch to allow our LDAP auth module to support > that kind of discovery. It copies the convention of the OpenLDAP > command line tools: if you give it a URL that has no hostname, it'll > try to extract a domain name from the bind DN, and then ask your DNS > server for a SRV record for LDAP-over-TCP at that domain. The > OpenLDAP version of libldap.so exports the magic to do that, so the > patch is very small (but the infrastructure set-up to test it is a bit > of a schlep, see below). I'll add this to the next Commitfest. > > [long tedious explanation of how to set up a test with BIND and OpenLDAP on > Unix] Of course the point of this is not really for the Unix-based set-up I described, but for Microsoft environments with one or more AD servers and a PostgreSQL server running on (eg) Linux that wants to find AD. In such environments, from what I can tell, the following should work: Standard DNS lookup tools should be able to find SRV records advertising the host, port and weight (priority) of any AD servers on the network: $ nslookup -type=any _ldap._tcp.YOUR.DOMAIN $ dig srv _ldap._tcp.YOUR.DOMAIN $ host -t srv _ldp._tcp.YOUR.DOMAIN OpenLDAP command line tools should be able to find the AD server via those SRV records, extracting YOUR.DOMAIN from the base DN: $ ldapsearch -H 'ldap:///dc%3DYOUR%2Cdc%3DDOMAIN' ... pg_hba.conf with an explicit LDAP server name should be able to talk to Active Directory without using this patch with something like: host all all 127.0.0.1/32 ldap ldapurl="ldap://YOUR-AD-SERVER.YOUR.DOMAIN/dc=YOUR,dc=DOMAIN?cn?sub"; pg_hba.conf using this patch should be able to discover the LDAP server via SRV if you take out the server name: host all all 127.0.0.1/32 ldap ldapurl="ldap:///dc=YOUR,dc=DOMAIN?cn?sub"; I'm hoping someone can help test this in a real Active Directory environment. -- Thomas Munro http://www.enterprisedb.com
Re: DNS SRV support for LDAP authentication
On Tue, Sep 25, 2018 at 2:09 PM Thomas Munro wrote: > 2. Define a new zone for testing, by adding the following to the end > 3. Create that zone file in /usr/local/etc/namedb/master/my.test.domain: Oops, I changed my testing domain name in the middle of my experiment, but pasted the older version into the previous message. Here are the corrected steps 2 and 3, consistent with the rest: = end of /usr/local/etc/namedb/named.conf = zone "my-domain.com" { type master; file "/usr/local/etc/namedb/master/my-domain.com"; }; = = /usr/local/etc/namedb/master/my-domain.com = $TTL10 @ IN SOA ns.my-domain.com. admin.my-domain.com. ( 2 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 604800 ) ; Negative Cache TTL IN NS ns.my-domain.com. ns.my-domain.com. IN A 127.0.0.1 my-domain.com. IN A 127.0.0.1 ldap-server.my-domain.com. IN A 127.0.0.1 _ldap._tcp.my-domain.com. IN SRV 0 0 389 ldap-server = -- Thomas Munro http://www.enterprisedb.com