Issue #15665 has been updated by Andrew Parker. Status changed from In Topic Branch Pending Review to Merged - Pending Release
---------------------------------------- Bug #15665: eix-update is run every time from the package Provider on gentoo systems https://projects.puppetlabs.com/issues/15665#change-68175 Author: Seraphim Mellos Status: Merged - Pending Release Priority: Normal Assignee: Andrew Parker Category: provider Target version: 2.7.x Affected Puppet version: 2.7.18 Keywords: gentoo Branch: https://github.com/puppetlabs/puppet/pull/983 As mentioned in the subject, when you run anything that involves the `package {}` type, the package provider for Gentoo runs the `eix-update` command multiple times for every "package {}" you have defined. A simple example is this: <pre><code class="sh"> # puppet apply --debug --execute 'package { "gentoolkit" : ensure => latest, category => "app-portage"; }' info: Loading facts in /var/lib/puppet/lib/facter/portage_flags.rb info: Loading facts in /var/lib/puppet/lib/facter/syslog_ssl.rb info: Loading facts in /var/lib/puppet/lib/facter/portage_keyword.rb info: Loading facts in /var/lib/puppet/lib/facter/environment_extra.rb info: Loading facts in /var/lib/puppet/lib/facter/ipaddress_dns.rb info: Loading facts in /var/lib/puppet/lib/facter/apachegid.rb info: Loading facts in /var/lib/puppet/lib/facter/layman_conf.rb info: Loading facts in /var/lib/puppet/lib/facter/portage_profile.rb info: Loading facts in /var/lib/puppet/lib/facter/raidtype.rb info: Loading facts in /var/lib/puppet/lib/facter/fqdn_rdns.rb debug: Puppet::Type::Package::ProviderFink: file /sw/bin/fink does not exist debug: Puppet::Type::Package::ProviderHpux: file /usr/sbin/swinstall does not exist debug: Puppet::Type::Package::ProviderYum: file yum does not exist debug: Puppet::Type::Package::ProviderAptitude: file /usr/bin/aptitude does not exist debug: Puppet::Type::Package::ProviderDpkg: file /usr/bin/dpkg does not exist debug: Puppet::Type::Package::ProviderRpm: file rpm does not exist debug: Puppet::Type::Package::ProviderPortupgrade: file /usr/local/sbin/portupgrade does not exist debug: Puppet::Type::Package::ProviderOpenbsd: file pkg_info does not exist debug: Puppet::Type::Package::ProviderSunfreeware: file pkg-get does not exist debug: Puppet::Type::Package::ProviderApt: file /usr/bin/apt-get does not exist debug: Puppet::Type::Package::ProviderPacman: file /usr/bin/pacman does not exist debug: Puppet::Type::Package::ProviderNim: file /usr/sbin/nimclient does not exist debug: Puppet::Type::Package::ProviderPkg: file /usr/bin/pkg does not exist debug: Puppet::Type::Package::ProviderZypper: file /usr/bin/zypper does not exist debug: Puppet::Type::Package::ProviderPorts: file /usr/local/sbin/portupgrade does not exist debug: Puppet::Type::Package::ProviderRug: file /usr/bin/rug does not exist debug: Puppet::Type::Package::ProviderFreebsd: file /usr/sbin/pkg_info does not exist debug: Puppet::Type::Package::ProviderUrpmi: file rpm does not exist debug: Puppet::Type::Package::ProviderSun: file /usr/bin/pkginfo does not exist debug: Puppet::Type::Package::ProviderUp2date: file /usr/sbin/up2date-nox does not exist debug: Puppet::Type::Package::ProviderAix: file /usr/bin/lslpp does not exist debug: Puppet::Type::Package::ProviderAptrpm: file apt-get does not exist debug: Creating default schedules debug: Failed to load library 'selinux' for feature 'selinux' debug: Failed to load library 'shadow' for feature 'libshadow' debug: Puppet::Type::User::ProviderUser_role_add: file roledel does not exist debug: Puppet::Type::User::ProviderDirectoryservice: file /usr/bin/dscl does not exist debug: Puppet::Type::User::ProviderPw: file pw does not exist debug: Puppet::Type::User::ProviderLdap: true value when expecting false debug: Failed to load library 'ldap' for feature 'ldap' debug: /File[/var/lib/puppet/ssl/private_keys]: Autorequiring File[/var/lib/puppet/ssl] debug: /File[/var/lib/puppet/clientbucket]: Autorequiring File[/var/lib/puppet] debug: /File[/var/lib/puppet/state/resources.txt]: Autorequiring File[/var/lib/puppet/state] debug: /File[/var/lib/puppet/client_yaml]: Autorequiring File[/var/lib/puppet] debug: /File[/var/lib/puppet/ssl/private]: Autorequiring File[/var/lib/puppet/ssl] debug: /File[/var/lib/puppet/ssl/certificate_requests]: Autorequiring File[/var/lib/puppet/ssl] debug: /File[/var/lib/puppet/state/graphs]: Autorequiring File[/var/lib/puppet/state] debug: /File[/var/lib/puppet/ssl/certs]: Autorequiring File[/var/lib/puppet/ssl] debug: /File[/var/lib/puppet/state/classes.txt]: Autorequiring File[/var/lib/puppet/state] debug: /File[/var/lib/puppet/state/last_run_summary.yaml]: Autorequiring File[/var/lib/puppet/state] debug: /File[/var/lib/puppet/client_data]: Autorequiring File[/var/lib/puppet] debug: /File[/var/lib/puppet/ssl/private_keys/rama.office.fcpl.com.pem]: Autorequiring File[/var/lib/puppet/ssl/private_keys] debug: /File[/var/lib/puppet/ssl/certs/ca.pem]: Autorequiring File[/var/lib/puppet/ssl/certs] debug: /File[/var/lib/puppet/lib]: Autorequiring File[/var/lib/puppet] debug: /File[/var/lib/puppet/ssl]: Autorequiring File[/var/lib/puppet] debug: /File[/var/lib/puppet/ssl/certs/rama.office.fcpl.com.pem]: Autorequiring File[/var/lib/puppet/ssl/certs] debug: /File[/var/lib/puppet/state]: Autorequiring File[/var/lib/puppet] debug: /File[/var/lib/puppet/state/state.yaml]: Autorequiring File[/var/lib/puppet/state] debug: /File[/var/lib/puppet/ssl/public_keys]: Autorequiring File[/var/lib/puppet/ssl] debug: /File[/var/lib/puppet/facts]: Autorequiring File[/var/lib/puppet] debug: /File[/var/lib/puppet/ssl/crl.pem]: Autorequiring File[/var/lib/puppet/ssl] debug: /File[/var/lib/puppet/state/last_run_report.yaml]: Autorequiring File[/var/lib/puppet/state] debug: /File[/var/lib/puppet/ssl/public_keys/rama.office.fcpl.com.pem]: Autorequiring File[/var/lib/puppet/ssl/public_keys] debug: Finishing transaction -615254638 debug: Loaded state in 0.00 seconds debug: Loaded state in 0.00 seconds info: Applying configuration version '1343139062' debug: /Schedule[daily]: Skipping device resources because running on a host debug: /Schedule[monthly]: Skipping device resources because running on a host debug: /Schedule[hourly]: Skipping device resources because running on a host debug: Prefetching portage resources for package debug: Puppet::Type::Package::ProviderPortage: Executing '/usr/bin/eix-update' # 1st run debug: Puppet::Type::Package::ProviderPortage: Executing '/usr/bin/eix --nocolor --pure-packages --stable --installed --format <category> <name> [<installedversions:LASTVERSION>] [<bestversion:LASTVERSION>] <homepage> <description> ' debug: Puppet::Type::Package::ProviderPortage: Executing '/usr/bin/eix-update' # 2nd run debug: Puppet::Type::Package::ProviderPortage: Executing '/usr/bin/eix --nocolor --pure-packages --stable --format <category> <name> [<installedversions:LASTVERSION>] [<bestversion:LASTVERSION>] <homepage> <description> --exact --category-name app-portage/gentoolkit' debug: /Schedule[never]: Skipping device resources because running on a host debug: /Schedule[weekly]: Skipping device resources because running on a host debug: /Schedule[puppet]: Skipping device resources because running on a host debug: Finishing transaction -614910978 debug: Storing state debug: Stored state in 0.01 seconds notice: Finished catalog run in 11.04 seconds debug: Finishing transaction -611854778 debug: Received report to process from rama.office.fcpl.com debug: Processing report from rama.office.fcpl.com with processor Puppet::Reports::Store </code></pre> I'm sure this is not the intended behavior, but if it was I'd urge you to reconsider. Rebuilding the whole eix database can be really slow depending on the machine that puppet is running on and as a result puppet runs take a long time to finish. Also, there's no point in having `eix-update` run more than once per puppet run no matter what. I've reproduced this with puppet 2.7.13 and 2.7.18 (as found on the Gentoo portage) >From a quick look at the puppet code I can see that the following changes >solve this problem: <pre><code class="diff"> --- portage.rb.orig 2012-07-24 15:35:12.686017175 +0100 +++ portage.rb 2012-07-24 15:35:25.070163322 +0100 @@ -20,7 +20,7 @@ search_format = "<category> <name> [<installedversions:LASTVERSION>] [<bestversion:LASTVERSION>] <homepage> <description>\n" begin - update_eix if !FileUtils.uptodate?("/var/cache/eix", %w{/usr/bin/eix /usr/portage/metadata/timestamp}) + update_eix if !FileUtils.uptodate?("/var/cache/eix/portage.eix", %w{/usr/bin/eix /usr/portage/metadata/timestamp}) search_output = nil Puppet::Util::Execution.withenv :LASTVERSION => version_format do @@ -81,7 +81,7 @@ search_value = package_name begin - update_eix if !FileUtils.uptodate?("/var/cache/eix", %w{/usr/bin/eix /usr/portage/metadata/timestamp}) + update_eix if !FileUtils.uptodate?("/var/cache/eix/portage.eix", %w{/usr/bin/eix /usr/portage/metadata/timestamp}) search_output = nil Puppet::Util::Execution.withenv :LASTVERSION => version_format do </code></pre>g <pre><code class="sh"> # puppet apply --debug --execute 'package { "gentoolkit" : ensure => latest, category => "app-portage"; }' [...] debug: /Schedule[daily]: Skipping device resources because running on a host debug: /Schedule[monthly]: Skipping device resources because running on a host debug: /Schedule[hourly]: Skipping device resources because running on a host debug: Prefetching portage resources for package debug: Puppet::Type::Package::ProviderPortage: Executing '/usr/bin/eix --nocolor --pure-packages --stable --installed --format <category> <name> [<installedversions:LASTVERSION>] [<bestversion:LASTVERSION>] <homepage> <description> ' debug: Puppet::Type::Package::ProviderPortage: Executing '/usr/bin/eix --nocolor --pure-packages --stable --format <category> <name> [<installedversions:LASTVERSION>] [<bestversion:LASTVERSION>] <homepage> <description> --exact --category-name app-portage/gentoolkit' debug: /Schedule[never]: Skipping device resources because running on a host debug: /Schedule[weekly]: Skipping device resources because running on a host debug: /Schedule[puppet]: Skipping device resources because running on a host debug: Finishing transaction -615026328 debug: Storing state debug: Stored state in 0.01 seconds notice: Finished catalog run in 0.85 seconds debug: Finishing transaction -611960138 debug: Received report to process from rama.office.fcpl.com debug: Processing report from rama.office.fcpl.com with processor Puppet::Reports::Store </code></pre> -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
