Jira (PUP-10628) Performance regression with large hashes using lookup

[Oleksandr Shtepa (Jira)]

Title: Message Title

Oleksandr Shtepa created an issue   Puppet / PUP-10628 Performance regression with large hashes using lookup Issue Type: Bug Affects Versions: PUP 6.17.0 Assignee: Unassigned Components: Hiera & Lookup Created: 2020/08/19 7:55 AM Environment: Test configuration (2000 items hash): $ cat > hiera.yaml <<EOF --- version: 5 defaults: datadir: . data_hash: yaml_data hierarchy: - name: test path: test.yaml EOF $ echo -e 'hash1:\n hash2:\n hash3:' > test.yaml $ for i in {1..2000}; do echo " key$i: value$i" >> test.yaml ; done Priority: Normal Reporter: Oleksandr Shtepa Puppet Version: starting from 5.0.0 ** After upgrading our Puppet installation from 4.10 to 6.x I found large catalog compile time increasing (over 300 seconds) for some nodes. Profiling shows problem with large hashes in Hiera yamls.   Good case:   $ sudo yum downgrade -y http://yum.puppetlabs.com/el/6x/PC1/x86_64/puppet-agent-1.10.14-1.el6.x86_64.rpm $ puppet --version 4.10.12 $ time puppet lookup --hiera_config hiera.yaml hash1.hash2.hash3.key1 --- value1 ... real 0m2.195s user 0m2.010s sys 0m0.181s Bad cases:     $ sudo yum update -y http://yum.puppetlabs.com/puppet5/el/6/x86_64/puppet-agent-5.0.0-1.el6.x86_64.rpm $ puppet --version 5.0.0 $ time puppet lookup --hiera_config hiera.yaml hash1.hash2.hash3.key1 --- value1 ... real 0m9.837s user 0m9.623s sys 0m0.207s $ sudo yum update -y puppet-agent $ puppet --version 6.17.0 $ time puppet lookup --hiera_config hiera.yaml hash1.hash2.hash3.key1 --- value1 real 0m9.788s user 0m9.494s sys 0m0.296s My research ended on follow debug points:   $ patch /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/lookup/data_provider.rb <<EOF --- data_provider.rb +++ data_provider.rb @@ -80,11 +80,14 @@ # @param data_provider [DataProvider] The data provider that produced the hash # @return [Object] The data value def validate_data_value(value, &block) +require 'time' +print "data_provider.rb::validate_data_value::1 ", ::Time.now.to_f, "\n" # The DataProvider.value_type is self recursive so further recursive check of collections is needed here unless value_is_validated? || DataProvider.value_type.instance?(value) actual_type = Types::TypeCalculator.singleton.infer(value) raise Types::TypeAssertionError.new("#{yield} has wrong type, expects Puppet::LookupValue, got #{actual_type}", DataProvider.value_type, actual_type) end +print "data_provider.rb::validate_data_value::2 ", ::Time.now.to_f, "\n" value end end EOF Run with debug timestamps on 6.17.0: $ time puppet lookup --hiera_config hiera.yaml hash1.hash2.hash3.key1 data_provider.rb::validate_data_value::1 1597847682.16178 data_provider.rb::validate_data_value::2 1597847689.6889844 --- value1 real 0m9.819s user 0m9.557s sys 0m0.261s Run with debug timestamps on 4.10.12: $ time puppet lookup --hiera_config hiera.yaml hash1.hash2.hash3.key1 data_provider.rb::validate_data_value::1 1597847876.923722 data_provider.rb::validate_data_value::2 1597847876.9429817 --- value1 ... real 0m2.206s user 0m2.050s sys 0m0.151s Slowdown from 19ms to 7.5s (390 times) on 2000 items.     For 4000 items - slowndown from 38ms to 31s (800 times)   Add Comment   This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.369684.1597848926000.19097.1597848960051%40Atlassian.JIRA.