Ray,
Are you the author of Class[Csapiclient] ? If so, the only suggestion I
have is to try and find API calls that return the data you need in a
quicker manner. The way the catalog and the resource abstraction layer
operate is built on top of an assumption of synchronous and relatively fast
system calls. Asynchronous and relatively slow network API calls aren't
well suited for types and providers.
Hope this helps,
-Jeff
On Tue, Oct 23, 2012 at 9:02 AM, Ray <r...@renegade.zapto.org> wrote:
> root@puppet30client:~# puppet agent --evaltrace --trace --test
> --configtimeout 5m
>
> Info: Retrieving plugin
> Info: Loading facts in /var/lib/puppet/lib/facter/centos_version.rb
> Info: Loading facts in /var/lib/puppet/lib/facter/network-bonds.rb
> Info: Loading facts in /var/lib/puppet/lib/facter/powerstates.rb
> Info: Loading facts in /var/lib/puppet/lib/facter/software_raid.rb
> Info: Loading facts in /var/lib/puppet/lib/facter/centos-version.rb
> Info: Caching catalog for puppet30client
> Info: Applying configuration version '1351007512'
> Info: /Schedule[daily]: Starting to evaluate the resource
> Info: /Schedule[daily]: Evaluated in 0.00 seconds
> Info: /Schedule[monthly]: Starting to evaluate the resource
> Info: /Schedule[monthly]: Evaluated in 0.00 seconds
> Info: /Schedule[hourly]: Starting to evaluate the resource
> Info: /Schedule[hourly]: Evaluated in 0.00 seconds
> Info: Stage[main]: Starting to evaluate the resource
> Info: Stage[main]: Evaluated in 0.00 seconds
> Info: Class[Main]: Starting to evaluate the resource
> Info: Class[Main]: Evaluated in 0.00 seconds
> Info: Class[Yum]: Starting to evaluate the resource
> Info: Class[Yum]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Yum/Package[yum-plugin-priorities]: Starting to
> evaluate the resource
> Info: /Stage[main]/Yum/Package[yum-plugin-priorities]: Evaluated in 0.00
> seconds
> Info: Set_yum_repo[epel]: Starting to evaluate the resource
> Info: Set_yum_repo[epel]: Evaluated in 0.04 seconds
> Info: Class[Csapiclient]: Starting to evaluate the resource
> Info: Class[Csapiclient]: Evaluated in 0.00 seconds
> Info: Node[puppet30client]: Starting to evaluate the resource
> Info: Node[puppet30client]: Evaluated in 0.00 seconds
> Info: Class[Centos]: Starting to evaluate the resource
> Info: Class[Centos]: Evaluated in 0.00 seconds
> Info: /File[/etc/profile.d/raimunds_favorite_aliases.sh]: Starting to
> evaluate the resource
> Info: /File[/etc/profile.d/raimunds_favorite_aliases.sh]: Evaluated in
> 0.00 seconds
> Info: Replace_line[Set_default_global_HISTSIZE]: Starting to evaluate the
> resource
> Info: Replace_line[Set_default_global_HISTSIZE]: Evaluated in 0.00 seconds
> Info: Class[Settings]: Starting to evaluate the resource
> Info: Class[Settings]: Evaluated in 0.00 seconds
> Info: Class[Settings]: Starting to evaluate the resource
> Info: Class[Settings]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Yum/Package[yum-plugin-fastestmirror]: Starting to
> evaluate the resource
> Info: /Stage[main]/Yum/Package[yum-plugin-fastestmirror]: Evaluated in
> 0.00 seconds
> Info: Class[Puppet::Client]: Starting to evaluate the resource
> Info: Class[Puppet::Client]: Evaluated in 0.00 seconds
> Info: /File[yum_epel.repo]: Starting to evaluate the resource
> Info: /File[yum_epel.repo]: Evaluated in 0.00 seconds
> Info: Set_yum_repo[epel]: Starting to evaluate the resource
> Info: Set_yum_repo[epel]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[bind-utils]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[bind-utils]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[xorg-x11-fonts-Type1]: Starting to
> evaluate the resource
> Info: /Stage[main]/Centos/Package[xorg-x11-fonts-Type1]: Evaluated in 0.00
> seconds
> Info: /Stage[main]/Centos/Package[make]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[make]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[finger]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[finger]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[openssh-clients]: Starting to evaluate
> the resource
> Info: /Stage[main]/Centos/Package[openssh-clients]: Evaluated in 0.00
> seconds
> Info: /Stage[main]/Centos/Package[traceroute]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[traceroute]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[htop]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[htop]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[tcpdump]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[tcpdump]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[wget]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[wget]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[screen]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[screen]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Yum/Exec[yum-clean-all]: Starting to evaluate the
> resource
> Info: /Stage[main]/Yum/Exec[yum-clean-all]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[vim-common]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[vim-common]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[sysstat]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[sysstat]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[man]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[man]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[curl]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[curl]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Yum/Package[yum-metadata-parser]: Starting to evaluate
> the resource
> Info: /Stage[main]/Yum/Package[yum-metadata-parser]: Evaluated in 0.00
> seconds
> Info: /Stage[main]/Centos/Package[vim-enhanced]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[vim-enhanced]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Yum/Package[yum]: Starting to evaluate the resource
> Info: /Stage[main]/Yum/Package[yum]: Evaluated in 0.00 seconds
> Info: Class[Yum]: Starting to evaluate the resource
> Info: Class[Yum]: Evaluated in 0.00 seconds
> Info: Class[Common]: Starting to evaluate the resource
> Info: Class[Common]: Evaluated in 0.00 seconds
> Info: Class[Common]: Starting to evaluate the resource
> Info: Class[Common]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[telnet]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[telnet]: Evaluated in 0.00 seconds
> Info: /File[/root/.ssh]: Starting to evaluate the resource
> Info: /File[/root/.ssh]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[pbzip2]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[pbzip2]: Evaluated in 0.00 seconds
> Info: /Schedule[never]: Starting to evaluate the resource
> Info: /Schedule[never]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[lsof]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[lsof]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[sudo]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[sudo]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[bc]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[bc]: Evaluated in 0.00 seconds
> Info: /File[/root/.vimrc]: Starting to evaluate the resource
> Info: /File[/root/.vimrc]: Evaluated in 0.00 seconds
> Info: /Filebucket[puppet]: Starting to evaluate the resource
> Info: /Filebucket[puppet]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[nc]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[nc]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[pigz]: Starting to evaluate the resource
> Info: /Stage[main]/Centos/Package[pigz]: Evaluated in 0.00 seconds
> Info: /Schedule[weekly]: Starting to evaluate the resource
> Info: /Schedule[weekly]: Evaluated in 0.00 seconds
> Info: Node[basenode]: Starting to evaluate the resource
> Info: Node[basenode]: Evaluated in 0.00 seconds
> Info: Node[basenode]: Starting to evaluate the resource
> Info: Node[basenode]: Evaluated in 0.00 seconds
> Info: /Schedule[puppet]: Starting to evaluate the resource
> Info: /Schedule[puppet]: Evaluated in 0.00 seconds
> Info: Class[Csapiclient]: Starting to evaluate the resource
> Info: Class[Csapiclient]: Evaluated in 0.00 seconds
> Info: Csapiclient::Addpfrule[forwardTCP324]: Starting to evaluate the
> resource
> Info: Csapiclient::Addpfrule[forwardTCP324]: Evaluated in 0.00 seconds
> Info: Csapiclient::Addpfrule[forwardTCP324]: Starting to evaluate the
> resource
> Info: Csapiclient::Addpfrule[forwardTCP324]: Evaluated in 0.00 seconds
> Info: Node[puppet30client]: Starting to evaluate the resource
> Info: Node[puppet30client]: Evaluated in 0.00 seconds
> Info: Class[Main]: Starting to evaluate the resource
> Info: Class[Main]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Centos/Package[augeas]: Starting to evaluate the
> resource
> Info: /Stage[main]/Centos/Package[augeas]: Evaluated in 0.00 seconds
> Info: /File[/etc/profile.d/raimunds_favorite_bash_prompt.sh]: Starting to
> evaluate the resource
> Info: /File[/etc/profile.d/raimunds_favorite_bash_prompt.sh]: Evaluated in
> 0.09 seconds
> Info:
> /Stage[main]/Centos/Replace_line[Set_default_global_HISTSIZE]/Exec[/bin/sed
> -i -r -e 's/HISTSIZE=1000/HISTSIZE=1000000/' /etc/profile]: Starting to
> evaluate the resource
> Info:
> /Stage[main]/Centos/Replace_line[Set_default_global_HISTSIZE]/Exec[/bin/sed
> -i -r -e 's/HISTSIZE=1000/HISTSIZE=1000000/' /etc/profile]: Evaluated in
> 0.06 seconds
> Info: Replace_line[Set_default_global_HISTSIZE]: Starting to evaluate the
> resource
> Info: Replace_line[Set_default_global_HISTSIZE]: Evaluated in 0.00 seconds
> Info: Class[Centos]: Starting to evaluate the resource
> Info: Class[Centos]: Evaluated in 0.00 seconds
> Info: /Stage[main]/Puppet::Client/Package[puppet]: Starting to evaluate
> the resource
> Info: /Stage[main]/Puppet::Client/Package[puppet]: Evaluated in 0.00
> seconds
> Info: /Stage[main]/Puppet::Client/Service[puppet]: Starting to evaluate
> the resource
> Info: /Stage[main]/Puppet::Client/Service[puppet]: Evaluated in 0.05
> seconds
> Info: Class[Puppet::Client]: Starting to evaluate the resource
> Info: Class[Puppet::Client]: Evaluated in 0.00 seconds
> Info: Stage[main]: Starting to evaluate the resource
> Info: Stage[main]: Evaluated in 0.00 seconds
> Finished catalog run in 1.87 seconds
>
> This again took like three minutes. When I drop the CSApiClient entry it
> runs quick, then the timing appears right.
>
> I assume the Cloudstack API takes agaes to respond and that's where things
> hang. As this is evaluated on the server, I assume the cataloge compilation
> (and therefore the retrieval also) is taking ages. The stuff that happens
> on the client appears to be quick.
>
> Cheers,
> Ray
>
>
> Am Dienstag, 23. Oktober 2012 16:20:25 UTC+2 schrieb Ray:
>
>> Hi,
>>
>> I want Puppet to talk to the CloudStack API. CloudStack has many
>> asynchronous API calls and Puppet's catalog run tends to expire even during
>> simple tasks such as adding port forwarding rules:
>>
>> root@puppet30client:~# puppet agent --test
>> Info: Retrieving plugin
>> Info: Loading facts in /var/lib/puppet/lib/facter/**centos_version.rb
>> Info: Loading facts in /var/lib/puppet/lib/facter/**network-bonds.rb
>> Info: Loading facts in /var/lib/puppet/lib/facter/**powerstates.rb
>> Info: Loading facts in /var/lib/puppet/lib/facter/**software_raid.rb
>> Info: Loading facts in /var/lib/puppet/lib/facter/**centos-version.rb
>>
>> Error: Could not retrieve catalog from remote server: execution expired
>> Warning: Not using cache on failed catalog
>> Error: Could not retrieve catalog; skipping run
>> root@puppet30client:~#
>>
>>
>> When I grant the puppetrun more time than the default 2 minutes, it comes
>> back successfully:
>>
>> root@puppet30client:~# puppet agent --test --configtimeout 5m
>> Info: Retrieving plugin
>> Info: Loading facts in /var/lib/puppet/lib/facter/**centos_version.rb
>> Info: Loading facts in /var/lib/puppet/lib/facter/**network-bonds.rb
>> Info: Loading facts in /var/lib/puppet/lib/facter/**powerstates.rb
>> Info: Loading facts in /var/lib/puppet/lib/facter/**software_raid.rb
>> Info: Loading facts in /var/lib/puppet/lib/facter/**centos-version.rb
>>
>> Info: Caching catalog for puppet30client.swisstxt.ch
>> Info: Applying configuration version '1351000748'
>> Finished catalog run in 1.88 seconds
>> root@puppet30client:~#
>>
>> (Don't believe the 1.88 seconds - this run took about three minutes(!))
>>
>>
>> My question is: What is the best practise to let Puppet interact with the
>> CloudStack API?
>>
>>
>> What I did:
>>
>> 1) Created a csapiclient module and dumped a custom function:
>> /etc/puppet/modules/**csapiclient/lib/puppet/parser/**
>> functions/add_pf_rule.rb
>>
>> require File.join(File.dirname(__FILE_**_), 'cloudstack_client')
>>
>> module Puppet::Parser::Functions
>> newfunction(:add_pf_rule) do |args|
>> apiurl = args[0]
>> apikey = args[1]
>> secretkey = args[2]
>> proto = args[3]
>> pubPort = args[4]
>> privPort = args[5]
>> ipId = args[6]
>> vmId = args[7]
>>
>> cs = CloudstackClient::Connection.**new(apiurl,apikey,secretkey)
>>
>> cs.create_port_forwarding_**rule( ipId, privPort, proto, pubPort,
>> vmId)
>> end
>> end
>>
>> 2) Wrote a defined resource that calls the Ruby function:
>>
>> define addPFRule( $ipId, $vmId, $proto, $pubPort, $privPort ) {
>> # Pass arguments to Ruby function…
>> add_pf_rule( $apiUrl, $apiKey , $secretKey, $proto, $pubPort,
>> $privPort, $ipId, $vmId )
>> }
>>
>> 3) Call the define from a recipe, feeding it with parameters.
>>
>> Is this a viable way to go or are there better/simpler/more beautiful
>> solutions to talk to CloudStack?
>>
>> Cheers,
>> Raiy
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/nBxpu-11DlgJ.
>
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
