Re: [HACKERS] PostgreSQL Service Name Enhancement - Wildcard support for LDAP/DNS lookup
I have suggested a similar feature before and met with little enthusiasm: http://www.postgresql.org/message-id/d960cb61b694cf459dcfb4b0128514c2f34...@exadv11.host.magwien.gv.at I still think it would be a nice feature and would make pg_service.conf more useful than it is now. Yours, Laurenz Albe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL Service Name Enhancement - Wildcard support for LDAP/DNS lookup
Tom, I believe there are two main concerns that you raise, addressed below: First: > It needs to be a more constrained syntax. > One possibility is to insist that the wildcard be only a part > of the name string, eg > > [myservers-%] > host=%.domain.com > port=5433 > user=admin * This counter-proposal is getting closer to the more complex matching requirements that I was attempting to de-scope, but it should certainly be considered. I can see where someone may want to have a different LDAP/DNS domain or something like that through prefix convention, though we would likely want to restrict to one "%" symbol in the service definition, yes? I am fine with including this capability in the patch, provided the general [%] case is still supported (see below for expanded reasons). Second: > since we check service names > before other possibilities such as host/database names, the entry would > then proceed to capture every possible connection request * I should have explicitly covered the case where no service name is provided at connection time - If a service name is not specified in the connection string/connection parameters at all, I would propose that this wildcard entry not match (even if service names are processed first) and normal processing proceed. As a comparable, the '%' in a like statement doesn't match a NULL after all. I don't think having a blank replacement value would make much sense either. Please inform me if I am not addressing some part of your concern with this mindset. Some additional comments: * It is in fact is desirable for us (and likely others) to capture all service names in one entry; I expect to utilize it exclusively once implemented. I would like to look up all service name entries from a single LDAP location if a previous entry in the file does not short circuit the match. To do this, I am explicitly looking to use a [%] entry in our service name file - the prefix requirement is not consistent with our environmental requirements. The service name file is a client construct that can be overridden by the caller if they desire, but keep in mind that this type of feature is targeted for controlled/managed environments. * Adding additional processing logic for 'myserver-%' would only make this more flexible for other use cases and would certainly meet the goals of this proposal, so I am fine including it in scope if the [%] is also allowed per above. When these other wildcard w/ prefix entries can match above the general [%] entry, it could be compelling (see my first email regarding entry order considerations). Those not wanting the [%] could choose to not implement it and stick to something closer to the prefix approach you have in mind. Summary of Open Questions: * (From Above) For prefix wildcards, OK to restrict to one % replacement? * Do the above points address initial concerns regarding service names being processed before host/db names? * If both prefix/non-prefix are allowed, what should be the behavior for cases where [prefix-%] matches and fails to connect/lookup and then [%] is also located further down? ** Without additional discussion, I would assume that it would attempt a connection for consistency. Again, people can choose to not use both features together. Another option would be to somehow introduce a "stop processing" flag in the service entries on connection/lookup failures, which may be generally useful even when wild cards are not in use. Thanks for the reply and your thoughts on this proposal so far. I am looking forward to the continued conversation. Bryan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] PostgreSQL Service Name Enhancement - Wildcard support for LDAP/DNS lookup
"Doyle, Bryan" writes: > Would specifying a special value for the service name, perhaps [%], be an > acceptable implementation of this enhancement/fix to my above concerns? > Example: > # comment > [%] > host=%.domain.com > port=5433 > user=admin This doesn't seem like a terribly good idea, because such an entry would capture *any* service name whatsoever. And, since we check service names before other possibilities such as host/database names, the entry would then proceed to capture every possible connection request. I follow what you're trying to do, but it needs to be a more constrained syntax. One possibility is to insist that the wildcard be only a part of the name string, eg [myservers-%] host=%.domain.com port=5433 user=admin regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] PostgreSQL Service Name Enhancement - Wildcard support for LDAP/DNS lookup
Hi, I am looking to patch the LDAP Service Name feature and would like some feedback prior to doing so. In general, while I personally view this as a bug fix, it could also easily be considered to simply be an enhancement to current functionality and therefore the below is written as a proposed enhancement. Note: This will also potentially serve as my first patch submission, so this will be a "smoke test" of my direct interaction with the mailing list as a Goldman Sachs Employee for proposal/feedback as well as following up with a potential patch/participation in commit processes, etc. As such my replies and related code submissions may be delayed for processes to happen on my end, with my associated apologies in advance. I hope to have this process completed for contribution to the 9.5 release. ## Currently: * Service names are provided as an available abstraction mechanism for a host:port (and user, dbname, ssl requirements, etc.) information for connection parameters via libpq. * If service names are used, they must each individually have an entry in pg_service.conf. * In a service name entry, host:port entries, hosts can either be directly provided or their values may be referenced via an LDAP location that provides equivalent name/value pairs for connection information. The service name feature facilitates the use of dynamic/managed host entries or allows for active connection configuration management with the goal in either case of separating such overhead from application or other process configuration. Therefore: * Dynamic DNS entries require a fixed port or active management of the file (DNS resolution can happen elsewhere) * Non-Dynamic DNS entries require active management of pg_service.conf, but may still be desirable for configuration management reasons - version control? * LDAP entries are largely static, but do require active management of pg_service.conf if new data servers are to be added (or old ones decommissioned) In all cases described above, a newly provisioned data server requires a new service name entry, which requires the file to be actively managed. This mitigates the value of using LDAP or Dynamic DNS entries (with fixed ports in the DNS case) since an active distribution mechanism must still be in place, especially for deployments of many servers and perhaps this useful mechanism is used less often as a result. ## Proposal: * Have a wildcard entry that allows string substitution of the service name into either the host field or an LDAP record. I do not want to over complicate the existing configuration file or parsing logic as it is pretty lightweight currently. As such, I am looking for feedback related to if it would be acceptable to have a "wildcard" entry for service names as well as what that wildcard should be. I do *not* propose that complex string substitution, including regular expressions, etc. be utilized for service name matching at this time. Per 31.16 in the 9.4 Documentation, a valid service entry is of the following format: # comment [mydb] host=somehost.domain.com port=5433 user=admin Would specifying a special value for the service name, perhaps [%], be an acceptable implementation of this enhancement/fix to my above concerns? Example: # comment [%] host=%.domain.com port=5433 user=admin ...or more interestingly, per 31.17: # comment [%] ldap=ldap://ldap.mycompany.com/dc=mycompany,dc=com?uniqueMember?one?(cn=%) The location of [%] and therefore order of the service names would still be honored. I believe this allows for useful functionality: * LDAP wildcard listed first - libpq can always try an LDAP lookup and proceed with further service names if one is not found in LDAP location * LDAP wildcard listed last - only look up LDAP entries if a service name doesn't match prior service names * LDAP wildcard in the middle - combination of first 2 behaviors * DNS wildcard listed last - try connecting with service name in well-formed dns-based host entry (this likely can't be first as it would always match) * Combination of the LDAP/keyword = value entries per 31.17: >From the existing documentation: "Processing of pg_service.conf is terminated >after a successful LDAP lookup, but is continued if the LDAP server cannot be >contacted. This is to provide a fallback with further LDAP URL lines that >point to different LDAP servers, classical keyword = value pairs, or default >connection options. If you would rather get an error message in this case, add >a syntactically incorrect line after the LDAP URL." The "%" in the ldap, host entry would replace with the service name in use - also possible to make it less likely to match a future valid "%" by providing it as [%] in the string, though I don't know why a % would reasonably show up in an ldap or host record. I
