Issue #13871 has been updated by Kelsey Hightower.
Andreas, Thanks for posting this. A change like this to core Puppet would require some discussion around backwards compatibility and evaluation of the proposed features. I would also recommend issuing a pull request against the master branch of Puppet following our [development process](https://github.com/puppetlabs/puppet/blob/master/CONTRIBUTING.md) ---------------------------------------- Feature #13871: LDAP - Referrals, SeachBase, multiValued-Attributes and multiParentnode Feature https://projects.puppetlabs.com/issues/13871#change-60696 Author: Andreas Schuster Status: Unreviewed Priority: Normal Assignee: Category: LDAP Target version: Affected Puppet version: 2.7.9 Keywords: Branch: Hi all, within our copmany, we are currently evaluating puppet 2.7.9 for our config-managment. Main focus is currently on Solaris Operating Systm. But Linux, AIX and HP-UX are also of interrest. First experiences are fine, but to rollout Puppet into our existing environment it was necessary to implement following features: 1. modification of ldapsearch to follow referrals and modification of LDAP-SearchBase (`ldapbase`) to look for Full-Qulified-Domain-Name of client 1. provision for multi-valued LDAP-Attributes. A Multivalued Attribute must be also collected from parentnode(s) if it already exists. 1. implementation of multi-parentnode feature to support our existing LDAP-Structure 1. introduction of LDAP-Certifcate-Directory variable `ldapcrtdir` for use of an SSL encrypted connection to the LDAP Server I hope you loke our implementation. And if it is possible, we are lucky if you will implement our features/changes into official puppet-source-code. Regards Andreas # Our LDAP-Directory and requirements To understand our environment, i will give a short introduction to our LDAP-Structure. The PuppetClients are applied for example within the LDAP directory at cn=systemX,ou=hostconfig,dc=network_L,dc=customerA, dc=ourcompany. The domainname is here dc=customerA,dc=ourcompany => customerA.ourcompmany. But it is possible that also another client with the same name (cn=systemX) exists at a different customer-Tree. If the search base (`Puppet::ldapbase`) is only dc=ourcompany, 2 or more PuppetClients with name cn=systemX will be found. To ensure that only one PuppetClient is found, the search base must be set to the domain of the PuppetClients to look for. Likewise, the parentnodes (eg. cn=HostConfig) exists across multiple LDAP branches. So the parentnode must be given as a FQDN. See the picture "LDAP Directory Tree":  The client "cn=systemZ,ou=HostConfig,dc=network_L,dc=customerA,dc=ourcompany" has following parentnodes: 1. cn=HostConfig,dc=network_L,dc=customerA,dc=ourcompany 1. cn=HostConfig,ou=buildingX,ou=cityA,ou=location,dc=ourcompany 1. cn=HostConfig,ou=cityA,ou=location,dc=ourcompany 1. cn=HostConfig,dc=customerA,dc=ourcompany 1. cn=HostConfig,dc=ourcompany As you can see, the parentnode "cn=HostConfig,dc=network_L,dc=customerA,dc=ourcompany" has 2 parentnodes. And the parentnodes must be evaluated in the given order! # Follow Referrals and use Domain of client as searchbase To follow referrals and to search only within Domain of the client or the parentnode, we have implemented following changes: <pre><code class="ruby"> Index: lib/puppet/indirector/ldap.rb --- lib/puppet/indirector/ldap.rb Base (BASE) +++ lib/puppet/indirector/ldap.rb Locally Modified (Based On LOCAL) @@ -18,10 +18,16 @@ nil end - def search_base - Puppet[:ldapbase] + # Setting LDAP SearchBase + def search_base(base=nil) + base.nil? ? Puppet[:ldapbase] : base end + # Set LDAP SearchScope + def search_scope(scope=nil) + scope.nil? ? LDAP::LDAP_SCOPE_SUBTREE : scope + end + # The ldap search filter to use. def search_filter(name) raise Puppet::DevError, "No search string set for LDAP terminus for #{self.name}" @@ -29,14 +35,14 @@ # Find the ldap node, return the class list and parent node specially, # and everything else in a parameter hash. - def ldapsearch(filter) + def ldapsearch(filter, base=nil, scope=nil) raise ArgumentError.new("You must pass a block to ldapsearch") unless block_given? found = false count = 0 begin - connection.search(search_base, 2, filter, search_attributes) do |entry| + connection.search(search_base(base), search_scope(scope), filter, search_attributes) do |entry| found = true yield entry end Index: lib/puppet/indirector/node/ldap.rb --- lib/puppet/indirector/node/ldap.rb Base (BASE) +++ lib/puppet/indirector/node/ldap.rb Locally Modified (Based On LOCAL) @@ -19,9 +19,21 @@ # LAK:NOTE Unfortunately, the ldap support is too stupid to throw anything # but LDAP::ResultError, even on bad connections, so we are rough handed # with our error handling. - def name2hash(name) + # Added 'parentmode' to use different ldapsearchscope for 'normal' clients and parentnode + def name2hash(name, parentmode=false) info = nil - ldapsearch(search_filter(name)) { |entry| info = entry2hash(entry) } + base = nil + scope = nil + + if parentmode + base = name + name = "*" + scope = LDAP::LDAP_SCOPE_BASE + else + name, base = name.gsub('.', ",dc=").split(',', 2) + scope = LDAP::LDAP_SCOPE_SUBTREE + end + ldapsearch(search_filter(name), base, scope) { |entry| info = entry2hash(entry) } info end @@ -171,7 +183,7 @@ # Find information for our parent and merge it into the current info. def find_and_merge_parent(parent, information) - parent_info = name2hash(parent) || raise(Puppet::Error.new("Could not find parent node '#{parent}'")) + parent_info = name2hash(parent, parentmode=true) || raise(Puppet::Error.new("Could not find parent node '#{parent}'")) information[:classes] += parent_info[:classes] parent_info[:parameters].each do |param, value| # Specifically test for whether it's set, so false values are handled correctly. </code></pre> **Patch File**: *02-01_puppet-2.7.9_ldap-referrals-and-searchbase.patch* The searchbase will be set in non-parentmode to the domainname ot the puppetclient. If parentmode is true, the seachbase will be set to the parentnode himself. Therefore the parentnode must be given as FQDN. LDAP-Attribute 'parentnode' looks like this: parentnode: cn=HostConfig,dc=network_L,dc=customerA,dc=ourcompany # Multi-Valued LDAP-Attributes A additinal requirement is to collect all multiValued LDAP-Atributes. And not only if it is not yet included/collected. <pre><code class="ruby"> Index: lib/puppet/indirector/ldap.rb --- lib/puppet/indirector/ldap.rb Base (BASE) +++ lib/puppet/indirector/ldap.rb Locally Modified (Based On LOCAL) @@ -1,5 +1,6 @@ require 'puppet/indirector/terminus' require 'puppet/util/ldap/connection' +require 'ldap/schema' class Puppet::Indirector::Ldap < Puppet::Indirector::Terminus # Perform our ldap search and process the result. @@ -81,4 +82,11 @@ @connection end + + # give configured LDAP-Schema + def ldapschema + @ldap_schema = @connection.schema() unless defined?(@ldap_schema) + @ldap_schema end + +end Index: lib/puppet/indirector/node/ldap.rb --- lib/puppet/indirector/node/ldap.rb Base (BASE) +++ lib/puppet/indirector/node/ldap.rb Locally Modified (Based On LOCAL) @@ -151,10 +151,36 @@ private + # create hash-table of ldap-attributes and if it is multivalued + def is_multivalued?(name) + unless defined?(@attr_lut) + @attr_lut = {} + ldapschema["attributeTypes"].each do |entry| + multivalued = entry.include?("SINGLE-VALUE") ? false : true + attributes = entry.sub(/.*NAME (\([\s'\w\-]+\)|['\w\-]+).*/, '\1').gsub(/(\( |'| \))/, '').split(' ') + + attributes.each do |attribute| + @attr_lut[attribute] = multivalued + end + end + end + @attr_lut[name] + end + # Add our hash of ldap information to the node instance. def add_to_node(node, information) node.classes = information[:classes].uniq unless information[:classes].nil? or information[:classes].empty? - node.parameters = information[:parameters] unless information[:parameters].nil? or information[:parameters].empty? + #node.parameters = information[:parameters] unless information[:parameters].nil? or information[:parameters].empty? + unless information[:parameters].nil? or information[:parameters].empty? + information[:parameters].each do |key, value| + if value.is_a?(Array) + information[:parameters][key].flatten! + information[:parameters][key].uniq! + end + end + node.parameters = information[:parameters] + end + node.environment = information[:environment] if information[:environment] end @@ -187,8 +213,20 @@ information[:classes] += parent_info[:classes] parent_info[:parameters].each do |param, value| # Specifically test for whether it's set, so false values are handled correctly. - information[:parameters][param] = value unless information[:parameters].include?(param) + #information[:parameters][param] = value unless information[:parameters].include?(param) + if information[:parameters].include?(param) + if is_multivalued?(param) + if information[:parameters][param].is_a?(Array) + information[:parameters][param] << value + else + information[:parameters][param] = [information[:parameters][param], value] end + end + else + information[:parameters][param] = value + end + end + information[:environment] ||= parent_info[:environment] parent_info[:parent] end </code></pre> Now all multiValued LDAP-Attributes are collected. Even if a Attribute already filled and the next parentnode also have additional values it will added to the Attribute. **Patch File**: *02-02_puppet-2.7.9_ldap-multivalued-attributetypes.patch* # Support of more than one ParentNode As you can see at out LDAP-Directory-Structure, we need a aolution to support more then one parentnode. Actually we implemented it this way: <pre><code class="ruby"> Index: lib/puppet/indirector/node/ldap.rb --- lib/puppet/indirector/node/ldap.rb Base (BASE) +++ lib/puppet/indirector/node/ldap.rb Locally Modified (Based On LOCAL) @@ -246,15 +246,22 @@ def merge_parent(info) parent_info = nil - parent = info[:parent] - # Preload the parent array with the node name. + # load parentnode(s) of client + parentnodes = info[:parent] + + # Pre-load the parant array with the node name. parents = [info[:name]] - while parent - raise ArgumentError, "Found loop in LDAP node parents; #{parent} appears twice" if parents.include?(parent) - parents << parent - parent = find_and_merge_parent(parent, info) + parentnodes.each do |parentnode| + if parents.include?(parentnode) + raise ArgumentError, "Found loop in LDAP node parents; #{parentnode} appears twice" end + parents << parentnode + parentnode2add = find_and_merge_parent(parentnode, info) + unless parentnode2add.nil? + parentnodes.insert(parentnodes.index(parentnode)+1 , parentnode2add).flatten! + end + end info end @@ -288,11 +295,16 @@ return nil unless values = entry.vals(pattr) - if values.length > 1 + if values.length > 2 raise Puppet::Error, - "Node entry #{entry.dn} specifies more than one parent: #{values.inspect}" + "Node entry #{entry.dn} specifies more than two parent nodes: #{values.inspect}" end - return(values.empty? ? nil : values.shift) + + return nil if values.empty? + + # sort values if necessary + values = sort_and_remove_delimiter(values, ':') + return values end def get_stacked_values_from_entry(entry) @@ -303,4 +315,28 @@ result end end + + def sort_and_remove_delimiter(array_to_sort, delimiter) + new_array = [] + + return nil if array_to_sort.empty? + + if array_to_sort.length > 1 + array_to_sort.each do |entry| + if entry.include? delimiter + new_entry_index = entry.split(delimiter)[1].to_i + new_entry_value = entry.split(delimiter)[0] + new_array.insert(new_entry_index, new_entry_value) + else + new_array << entry + Puppet.warning "More then one parentnode defined, but no sort index given: \n\t#{array_to_sort.inspect}" end + + end + else + new_array = [array_to_sort.first.split(delimiter)[0]] + end + + return new_array.compact + end +end </code></pre> Now it is possible to set 2 parentnode LDAP-Attributes to a puppetClient and add a index as additional element to this Attribute: * parentnode: cn=HostConfig,ou=buildingX,ou=cityA,ou=location,dc=ourcompany:2 * parentnode: cn=HostConfig,ou=cityA,ou=location,dc=ourcompany:1 and the parentnodes of this puppetClient will be evaluated in the given order. If you set more then one parentnode and not setting the index value, a warning will be given. **Patch File**: *02-03_puppet-2.7.9_ldap-multi-parentnode.patch* # LDAP-Certificate-Directory To use the server side certificate for LDAP connection we implemented a new variable **ldapcrtdir** to pint to the directory which containes the certificate. On Solaris it points to /var/ldap. Don't know, if this patch is of generall interest... <pre><code class="ruby"> Index: lib/puppet/defaults.rb --- lib/puppet/defaults.rb Base (BASE) +++ lib/puppet/defaults.rb Locally Modified (Based On LOCAL) @@ -855,6 +855,9 @@ "Whether SSL should be used when searching for nodes. Defaults to false because SSL usually requires certificates to be set up on the client side."], + :ldapcrtdir => [false, + "A path to cerificate directories. + Defaults to 'false'."], :ldaptls => [false, "Whether TLS should be used when searching for nodes. Defaults to false because TLS usually requires certificates Index: lib/puppet/util/ldap/connection.rb --- lib/puppet/util/ldap/connection.rb Base (BASE) +++ lib/puppet/util/ldap/connection.rb Locally Modified (Based On LOCAL) @@ -1,7 +1,7 @@ require 'puppet/util/ldap' class Puppet::Util::Ldap::Connection - attr_accessor :host, :port, :user, :password, :reset, :ssl + attr_accessor :host, :port, :user, :password, :reset, :ssl, :crtdir attr_reader :connection @@ -15,6 +15,8 @@ false end + crtdir = Puppet[:ldapcrtdir] + options = {} options[:ssl] = ssl if user = Puppet.settings[:ldapuser] and user != "" @@ -47,7 +49,7 @@ # Create a per-connection unique name. def name - [host, port, user, password, ssl].collect { |p| p.to_s }.join("/") + [host, port, user, password, ssl, crtdir].collect { |p| p.to_s }.join("/") end # Should we reset the connection? @@ -59,10 +61,18 @@ def start case ssl when :tls + if crtdir + @connection = LDAP::SSLConn.new(host, port, true, crtdir); + else @connection = LDAP::SSLConn.new(host, port, true) + end when true - @connection = LDAP::SSLConn.new(host, port) + if crtdir + @connection = LDAP::SSLConn.new(host, port, true, crtdir); else + @connection = LDAP::SSLConn.new(host, port, true) + end + else @connection = LDAP::Conn.new(host, port) end @connection.set_option(LDAP::LDAP_OPT_PROTOCOL_VERSION, 3) </code></pre> **Patch File**: *01-01_puppet-2.7.9_add-ldap-crtdir.patch* -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
