Jira (PUP-10540) Performance of puppet grows worse as the size of /proc/mounts increases

[Michele Baldessari (Jira)]

Title: Message Title

Michele Baldessari created an issue   Puppet / PUP-10540 Performance of puppet grows worse as the size of /proc/mounts increases Issue Type: Bug Assignee: Unassigned Created: 2020/06/08 9:40 AM Priority: Normal Reporter: Michele Baldessari Puppet Version: puppet-5.5.10-5.el8ost Puppet Server Version: N/A OS Name/Version: Centos 8 A system that has many containers (~1000) will make /proc/mounts rather large (700-900 kbytes). Running any trivial puppet command will take a very long time there due to the following line: https://github.com/puppetlabs/puppet/blob/master/lib/puppet/util/selinux.rb#L210 In there we're reading /proc/mounts 1024 bytes at the time, which is clearly suboptimal in this case (but also in most default cases as /proc/mounts is rarely < 1024 bytes these days on most modern linux systems). On such an environment we will observe: [root@database-0 ~]# time puppet apply --noop --detailed-exitcodes --color=false -e 'file {"/tmp/foobar": content => "baz" } ' Notice: Compiled catalog for database-0.redhat.local in environment production in 0.16 seconds Notice: /Stage[main]/Main/File[/tmp/foobar]/ensure: current_value absent, should be file (noop) Notice: Class[Main]: Would have triggered 'refresh' from 1 events Notice: Stage[main]: Would have triggered 'refresh' from 1 events Notice: Applied catalog in 0.05 seconds real 5m43.258s user 5m7.929s sys 0m5.603s Increasing the read size to 1024 ** 2 provides immediate benefits: [root@database-0 ~]# time puppet apply --noop --detailed-exitcodes --color=false -e 'file {"/tmp/foobar": content => "baz" } ' Notice: Compiled catalog for database-0.redhat.local in environment production in 0.11 seconds Notice: /Stage[main]/Main/File[/tmp/foobar]/ensure: current_value absent, should be file (noop) Notice: Class[Main]: Would have triggered 'refresh' from 1 events Notice: Stage[main]: Would have triggered 'refresh' from 1 events Notice: Applied catalog in 0.13 seconds real 1m22.954s user 0m50.423s sys 0m5.115s In such an environment strace confirms that the number of small read() calls goes from: [root@database-0 ~]# grep -ir -e 'read(7.*1024' /tmp/strace-puppet.txt |wc -l 231381 To: [root@database-0 ~]# grep -ir -e 'read(7.*1024' /tmp/strace-puppet-after.txt |wc -l 810 Add Comment   This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.362109.1591634455000.81156.1591634460041%40Atlassian.JIRA.