Andy,
did this on a vagrant box with eyaml and file backends.
The hierarchy has 7 levels.
Debug: PROFILE [apply] 1 Setup server facts for compiling: took 0.0000
seconds
Debug: PROFILE [apply] 2.1 Compile: Set node parameters: took 0.0000 seconds
Debug: PROFILE [apply] 2.2 Compile: Created settings scope: took 0.0100
seconds
Debug: hiera(): [eyaml_backend]: Hiera eYAML backend starting
Debug: hiera(): [eyaml_backend]: Set option: datadir = /vagrant/hiera/
Debug: hiera(): [eyaml_backend]: Set option: pkcs7_private_key =
/vagrant/keys/private_key.pkcs7.pem
Debug: hiera(): [eyaml_backend]: Set option: pkcs7_public_key =
/vagrant/keys/public_key.pkcs7.pem
Debug: hiera(): [eyaml_backend]: Looking up somekey in eYAML backend
Debug: hiera(): [eyaml_backend]: Looking for data source
it-mil1/dev/nodes/dnsns001
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/nodes/dnsns001.eyaml, skipping
Debug: hiera(): [eyaml_backend]: Looking for data source
it-mil1/dev/roles/dnsns
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/roles/dnsns.eyaml, skipping
Debug: hiera(): [eyaml_backend]: Looking for data source it-mil1/dev/global
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/global.eyaml, skipping
Debug: hiera(): [eyaml_backend]: Looking for data source nodes/dnsns001
Debug: hiera(): [eyaml_backend]: Looking for data source roles/dnsns
Debug: hiera(): Cannot find datafile /vagrant/hiera/roles/dnsns.eyaml,
skipping
Debug: hiera(): [eyaml_backend]: Looking for data source dev
Debug: hiera(): Cannot find datafile /vagrant/hiera/dev.eyaml, skipping
Debug: hiera(): [eyaml_backend]: Looking for data source global
Debug: hiera(): Hiera File backend starting
Debug: hiera(): Looking up somekey in File backend
Debug: hiera(): Hiera File_backend: looking for data source
'it-mil1/dev/nodes/dnsns001'
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/nodes/dnsns001.d, skipping
Debug: hiera(): Hiera File_backend: looking for data source
'it-mil1/dev/roles/dnsns'
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/roles/dnsns.d, skipping
Debug: hiera(): Hiera File_backend: looking for data source
'it-mil1/dev/global'
Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/global.d,
skipping
Debug: hiera(): Hiera File_backend: looking for data source 'nodes/dnsns001'
Debug: hiera(): Cannot find datafile /vagrant/hiera/nodes/dnsns001.d,
skipping
Debug: hiera(): Hiera File_backend: looking for data source 'roles/dnsns'
Debug: hiera(): Cannot find datafile /vagrant/hiera/roles/dnsns.d, skipping
Debug: hiera(): Hiera File_backend: looking for data source 'dev'
Debug: hiera(): Cannot find datafile /vagrant/hiera/dev.d, skipping
Debug: hiera(): Hiera File_backend: looking for data source 'global'
Debug: PROFILE [apply] 2.3.1 Called hiera: took 0.0336 seconds
Debug: hiera(): [eyaml_backend]: Set option: datadir = /vagrant/hiera/
Debug: hiera(): [eyaml_backend]: Set option: pkcs7_private_key =
/vagrant/keys/private_key.pkcs7.pem
Debug: hiera(): [eyaml_backend]: Set option: pkcs7_public_key =
/vagrant/keys/public_key.pkcs7.pem
Debug: hiera(): [eyaml_backend]: Looking up otherkey in eYAML backend
Debug: hiera(): [eyaml_backend]: Looking for data source
it-mil1/dev/nodes/dnsns001
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/nodes/dnsns001.eyaml, skipping
Debug: hiera(): [eyaml_backend]: Looking for data source
it-mil1/dev/roles/dnsns
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/roles/dnsns.eyaml, skipping
Debug: hiera(): [eyaml_backend]: Looking for data source it-mil1/dev/global
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/global.eyaml, skipping
Debug: hiera(): [eyaml_backend]: Looking for data source nodes/dnsns001
Debug: hiera(): [eyaml_backend]: Looking for data source roles/dnsns
Debug: hiera(): Cannot find datafile /vagrant/hiera/roles/dnsns.eyaml,
skipping
Debug: hiera(): [eyaml_backend]: Looking for data source dev
Debug: hiera(): Cannot find datafile /vagrant/hiera/dev.eyaml, skipping
Debug: hiera(): [eyaml_backend]: Looking for data source global
Debug: hiera(): Looking up otherkey in File backend
Debug: hiera(): Hiera File_backend: looking for data source
'it-mil1/dev/nodes/dnsns001'
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/nodes/dnsns001.d, skipping
Debug: hiera(): Hiera File_backend: looking for data source
'it-mil1/dev/roles/dnsns'
Debug: hiera(): Cannot find datafile
/vagrant/hiera/it-mil1/dev/roles/dnsns.d, skipping
Debug: hiera(): Hiera File_backend: looking for data source
'it-mil1/dev/global'
Debug: hiera(): Cannot find datafile /vagrant/hiera/it-mil1/dev/global.d,
skipping
Debug: hiera(): Hiera File_backend: looking for data source 'nodes/dnsns001'
Debug: hiera(): Cannot find datafile /vagrant/hiera/nodes/dnsns001.d,
skipping
Debug: hiera(): Hiera File_backend: looking for data source 'roles/dnsns'
Debug: hiera(): Cannot find datafile /vagrant/hiera/roles/dnsns.d, skipping
Debug: hiera(): Hiera File_backend: looking for data source 'dev'
Debug: hiera(): Cannot find datafile /vagrant/hiera/dev.d, skipping
Debug: hiera(): Hiera File_backend: looking for data source 'global'
Debug: PROFILE [apply] 2.3.2 Called hiera: took 0.0156 seconds
On Friday, May 16, 2014 6:29:28 PM UTC+2, Andy Parker wrote:
>
> On Thu, May 15, 2014 at 10:24 PM, David Schmitt <da...@dasz.at<javascript:>
> > wrote:
>
>> Hi,
>>
>>
>> On 16.05.2014 02:17, Alessandro Franceschi wrote:
>>
>>>
>>>
>>> On Thursday, May 15, 2014 9:55:05 PM UTC+2, John Bollinger wrote:
>>>
>>> I am concerned about the impact. It is already somewhat costly for
>>> Puppet to evaluate data bindings for class parameters, and adding
>>> bindings for resource parameters (even just for resources of defined
>>> types) will magnify that. Note that the cost scales with the
>>> aggregate number of defined parameters for all declared resources,
>>> independent whether any data are actually bound. In fact, the cases
>>> were no data are bound are the most costly, because hiera must then
>>> search the entire hierarchy.
>>>
>>>
>>> Yes, these are valid and convincing points.
>>> Anyway if we find data binding useful for classes and can bear the
>>> performance overhead, I suppose we can do the same for defined types.
>>>
>>
>> I very much like the idea for data binding on all parameters, but this
>> statement is really not universally true. I've got catalogs with only tens
>> of classes but thousands of resources. That works out to 100x more hiera
>> calls. :-/
>>
>>
> John makes a very good point about the performance cost. The impact of
> putting data bindings on everything is made even worse by the fact that it
> is an all or nothing price you have to pay, there isn't any way to use it
> for just a subset of your catalog.
>
> As a little number collection, would some of you mind doing a little
> experiment and posting the numbers?
>
> Run "puppet apply -e 'notice(hiera("somekey", 1), hiera("otherkey", 1))'
> --profile --debug"
>
> Make sure that it is using your real hiera config with your real data
> files or else it won't be an accurate reflection of the cost. You'll get
> output that should contain something like this:
>
> Debug: PROFILE [apply] 2.2 Compile: Created settings scope: took 0.0103
> seconds
> Debug: hiera(): Hiera YAML backend starting
> Debug: hiera(): Looking up somekey in YAML backend
> Debug: hiera(): Looking for data source common
> Debug: hiera(): Cannot find datafile /var/lib/hiera/common.yaml, skipping
> Debug: PROFILE [apply] 2.3.1 Called hiera: took 0.0019 seconds
> Debug: hiera(): Looking up otherkey in YAML backend
> Debug: hiera(): Looking for data source common
> Debug: hiera(): Cannot find datafile /var/lib/hiera/common.yaml, skipping
> Debug: PROFILE [apply] 2.3.2 Called hiera: took 0.0002 seconds
>
> I don't have hiera setup on this machine so it didn't do anything, but you
> can see how much overhead *per-parameter* data bindings introduces. So in a
> system with no hierarchy, and no data files (and in fact no hiera config)
> it adds about 2 usec per parameter.
>
> If some of you could post results as well as estimates of the number of
> defined types and parameters that you have in a catalog that would be great.
>
>
>>
>> Regards, David
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to puppet-dev+...@googlegroups.com <javascript:>.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/puppet-dev/5375A109.7020209%40dasz.at.
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Andrew Parker
> a...@puppetlabs.com <javascript:>
> Freenode: zaphod42
> Twitter: @aparker42
> Software Developer
>
> *Join us at PuppetConf 2014 <http://www.puppetconf.com/>, September
> 22-24 in San Francisco*
> *Register by May 30th to take advantage of the Early Adopter discount
> <http://links.puppetlabs.com/puppetconf-early-adopter> **—**save $349!*
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/63ce435d-0f8a-447d-ac3c-9999c3b2874a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
