Issue #1472 has been updated by masterzen.
luke wrote:
> Hmm, if you're right about what's happening (sorry, don't have time to
> reproduce), then I think the bug is that the defined resource is being
> evaluated even though it's virtual. Neither virtual nor exported defined
> resources should be evaluated at all, meaning that they should never produce
> contained virtual or exported resources. That is, stage 2 in your described
> process shouldn't happen if the definition is virtual.
No, that's not exactly the issue. The issue is that collecting a define should
collect underlying resources, which is not that happens (see below).
I just checked again, with the following minimal site.pp:
<pre>
define deftest ()
{
warning("DEFTEST ${fqdn} on ${name}")
file { "/tmp/${name}":
content => "test"
}
}
node "pouet.internal" {
@@deftest {
"pouet":
}
Deftest <<| |>>
}
</pre>
I instrumented (ie lots of puts) and here is what happens:
1. The node is evaluated:
<pre>
Compiler::add_resource Node[pouet.internal]
resource is now Node[pouet.internal] exp:
Resource::evaluate Node[pouet.internal]
Resource::evaluating a definedtype Node[pouet.internal]
</pre>
2. It gives the Deftest[pouet] virtual,exported resource:
<pre>
Resource::initialize for pouet
resource::initialize set option type = deftest
resource::initialize set option virtual = true
resource::initialize set option scope = Scope(Node[pouet.internal])
resource::initialize set option source = pouet.internal
resource::initialize set option title = pouet
resource::initialize set option params =
resource::initialize set option exported = true
Resource::exported= true
Resource::exported= force virtual and exported to true
resource::initialize set option line = 12
resource::initialize set option file = /tmp/test/manifests/site.pp
</pre>
3. the deftest[pouet] resource is added to the catalog
4. then, to the current collections
5. the collections are then evaluated (first pass of
compiler::evaluate_generators):
<pre>
Compiler::evaluate_generators pass: 0
Compiler::evaluate_collections (entering)
Compiler::evaluate_collections collections not empty
Compiler::evaluate_collections for #<Puppet::Parser::Collector:0xb741c7e4>
</pre>
6. Since deftest[pouet] is exported and virtual and matches the current collect
selector, it is selected for collection:
<pre>
Collector::collect_exported (entering)
Collector::collect_virtual for exported: true
Collector::collect_virtual can Deftest[pouet] be collected? true
Collector::collect_virtual can Node[pouet.internal] be collected? false
Collector::collect_virtual can Class[main] be collected? false
Collector::collect_exported resources Deftest[pouet]
debug: Scope(Node[pouet.internal]): Collected 1 Deftest resource in 0.01 seconds
</pre>
7. Then, back to evaluate_generators, evaluate_definitions is called and tries
to evaluate deftest[pouet] which is not virtual anymore (because we collected
it):
<pre>
Compiler::evaluate_collectionse #<Puppet::Parser::Collector:0xb741c7e4>
collection evaluation found sth
Compiler::evaluate_collections (leaving)
Compiler::evaluate_generators after eval_collections done? false
Compiler::evaluate_definitions for Deftest[pouet]
Compiler::evaluate_definitions for Deftest[pouet], not virtual
</pre>
8. the deftest underlying resources are then evaluated and created as it should:
<pre>
Resource::evaluate Deftest[pouet]
Resource::evaluating a definedtype Deftest[pouet]
evaluate_code for Deftest[pouet]
warning: Scope(Deftest[pouet]): DEFTEST arsenic.internal on pouet
Resource::initialize for /tmp/pouet
resource::initialize set option type = file
resource::initialize set option virtual = true
resource::initialize set option scope = Scope(Deftest[pouet])
resource::initialize set option source = deftest
resource::initialize set option title = /tmp/pouet
resource::initialize set option params = content => test
resource::initialize set option exported = true
Resource::exported= true
Resource::exported= force virtual and exported to true
resource::initialize set option line = 6
resource::initialize set option file = /tmp/test/manifests/site.pp
Compiler::add_resource File[/tmp/pouet]
</pre>
You can see that File[/tmp/pouet] is now added to the catalog, which is right.
9. Then we're going to have another pass in Compiler::evaluate_generators to
collect/evaluate resources or define that could have been created in the
previous pass:
<pre>
Compiler::evaluate_generators after evaluate_definitions done? false
Compiler::evaluate_generators pass: 1
Compiler::evaluate_collections (entering)
Compiler::evaluate_collections collections not empty
Compiler::evaluate_collections for #<Puppet::Parser::Collector:0xb741c7e4>
</pre>
10. Guess what? There are some resources to collect:
<pre>
Collector::collect_exported (entering)
Collector::collect_virtual for exported: true
Collector::collect_virtual can Deftest[pouet] be collected? true
Collector::collect_virtual can Node[pouet.internal] be collected? false
Collector::collect_virtual can File[/tmp/pouet] be collected? false
Collector::collect_virtual can Class[main] be collected? false
Collector::collect_exported resources
debug: Scope(Node[pouet.internal]): Collected 0 Deftest resources in 0.00
seconds
</pre>
Wait!! Something is wrong, it didn't collect the File[/tmp/pouet], so this
resource won't ever be evaluated.
Why is that: simply the resource type (File) doesn't match the collected type
(Deftest).
That's why my patch goes beyond that to check that there is a parent defined
resource that could match the type of the collected resource.
Hope that helps you understand the root issue.
----------------------------------------
Bug #1472: Defined exported resource are not collected properly.
http://projects.reductivelabs.com/issues/show/1472
Author: masterzen
Status: Code Insufficient
Priority: High
Assigned to: luke
Category: transactions
Target version:
Complexity: Unknown
Patch: None
Affected version: 0.24.4
Keywords: exported collect resource define
Hi,
On puppet > 0.24.4 (including 0.24.5), I can't seem to collect "defined"
resource. Native resource are collected fine.
This doesn't work:
<pre>
# Non-working site.pp
define deftest ()
{
warning("DEFTEST ${fqdn} on ${name}")
file { "${name}":
content => "test"
}
}
node "xen1.internal" {
$part = "xen1"
@@deftest {
"/tmp/${part}":
tag => "test"
}
Deftest <<| tag=='test' |>>
}
node "xen2.internal" {
$part = "xen2"
@@deftest {
"/tmp/${part}":
tag => "test"
}
Deftest <<| tag=='test' |>>
}
</pre>
Proof:
running manually on xen1.internal gives:
<pre>
info: Loading fact netmask
info: Loading fact ldap_auth_bind_user
info: Loading fact configured_ntp_servers
info: Retrieving plugins
info: Loading fact netmask
info: Loading fact ldap_auth_bind_user
info: Loading fact configured_ntp_servers
info: Caching catalog at /var/lib/puppet/state/localconfig.yaml
notice: Starting catalog run
notice: Finished catalog run in 0.11 seconds
</pre>
localconfig.yaml doesn't contain anything about /tmp/xen1.
Puppetmaster says:
<pre>
info: Expiring the node cache of xen1.internal
info: Not using expired node for xen1.internal from cache; expired at Thu Jul
31 13:26:12 +0200 2008
info: Caching node for xen1.internal
debug: Creating default schedules
debug:
/Settings[/etc/puppet/puppet.conf]/Settings[puppetmasterd]/File[/etc/puppet/manifests/site.pp]:
Autorequiring File[/etc/puppet/manifests]
debug: Finishing transaction 23539572950260 with 0 changes
debug: Scope(Node[xen1.internal]): Collected 1 Deftest resource in 0.01 seconds
warning: Scope(Deftest[/tmp/xen1]): DEFTEST xen1.internal on /tmp/xen1
debug: Scope(Node[xen1.internal]): Collected 0 Deftest resources in 0.00 seconds
info: Stored catalog for xen1.internal in 0.35 seconds
notice: Compiled catalog for xen1.internal in 0.50 seconds
</pre>
Which proved the puppetmaster has seen the collected resource.
As the storeconfig database shows too:
<pre>
mysql> select * from resources where host_id=79 and exported is not null;
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
| id | title | restype | host_id | source_file_id | exported | line |
updated_at |
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
| 51638 | /tmp/xen1 | File | 79 | 1 | 1 | 12 |
2008-07-31 13:27:12 |
| 51639 | /tmp/xen1 | Deftest | 79 | 1 | 1 | 23 |
2008-07-31 13:27:12 |
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
2 rows in set (0.01 sec)
</pre>
Let's play with xen2.internal now:
<pre>
info: Loading fact configured_ntp_servers
info: Loading fact ldap_auth_bind_user
info: Loading fact netmask
info: Retrieving plugins
info: Loading fact configured_ntp_servers
info: Loading fact ldap_auth_bind_user
info: Loading fact netmask
info: Caching catalog at /var/lib/puppet/state/localconfig.yaml
notice: Starting catalog run
warning: //Node[xen2.internal]/Deftest[/tmp/xen1]/File[/tmp/xen1]/checksum:
File /tmp/xen1 does not exist -- cannot checksum
notice: //Node[xen2.internal]/Deftest[/tmp/xen1]/File[/tmp/xen1]/checksum:
defined 'checksum' as '{md5}d9b5059238064ee18522f48e51730c21'
notice: //Node[xen2.internal]/Deftest[/tmp/xen1]/File[/tmp/xen1]/content:
created file with contents {md5}b23295cbdd57c2f30132fb656a6ddb17
notice: Finished catalog run in 0.41 seconds
</pre>
The file /tmp/xen1 has been created (ie it exported the resource), but not the
/tmp/xen2 file.
And the puppetmaster says:
<pre>
info: Expiring the node cache of xen2.internal
info: Not using expired node for xen2.internal from cache; expired at Thu Jul
31 13:29:53 +0200 2008
info: Caching node for xen2.internal
debug: Scope(Node[xen2.internal]): Collected 2 Deftest resources in 0.01 seconds
warning: Scope(Deftest[/tmp/xen1]): DEFTEST xen2.internal on /tmp/xen1
warning: Scope(Deftest[/tmp/xen2]): DEFTEST xen2.internal on /tmp/xen2
debug: Scope(Node[xen2.internal]): Collected 0 Deftest resources in 0.00 seconds
info: Stored catalog for xen2.internal in 0.36 seconds
notice: Compiled catalog for xen2.internal in 0.40 seconds
</pre>
Definitely, the puppetmaster knows both exported resources.
The database now contains:
<pre>
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
| id | title | restype | host_id | source_file_id | exported | line |
updated_at |
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
| 51638 | /tmp/xen1 | File | 79 | 1 | 1 | 12 |
2008-07-31 13:27:12 |
| 51639 | /tmp/xen1 | Deftest | 79 | 1 | 1 | 23 |
2008-07-31 13:27:12 |
| 51642 | /tmp/xen2 | File | 80 | 1 | 1 | 12 |
2008-07-31 13:30:53 |
| 51643 | /tmp/xen2 | Deftest | 80 | 1 | 1 | 33 |
2008-07-31 13:30:53 |
+-------+-----------+---------+---------+----------------+----------+------+---------------------+
</pre>
Back to xen1, now puppet can see the /tmp/xen2 file:
<pre>
info: Loading fact netmask
info: Loading fact ldap_auth_bind_user
info: Loading fact configured_ntp_servers
info: Retrieving plugins
info: Loading fact netmask
info: Loading fact ldap_auth_bind_user
info: Loading fact configured_ntp_servers
info: Caching catalog at /var/lib/puppet/state/localconfig.yaml
notice: Starting catalog run
warning: //Node[xen1.internal]/Deftest[/tmp/xen2]/File[/tmp/xen2]/checksum:
File /tmp/xen2 does not exist -- cannot checksum
notice: //Node[xen1.internal]/Deftest[/tmp/xen2]/File[/tmp/xen2]/checksum:
defined 'checksum' as '{md5}2eba568f285d93e679f9d362dcf049b7'
notice: //Node[xen1.internal]/Deftest[/tmp/xen2]/File[/tmp/xen2]/content:
created file with contents {md5}572ac7bb156d4895b43e08ccc8ef8076
notice: Finished catalog run in 0.08 seconds
</pre>
Conclusion: it seems that each node cannot collect its own "defined" resources,
but can still see the effects of the other nodes defined exported resources.
Note1: if I use only file{} instead of defines in the manifest, it works fine.
Note2: it used to work fine at least in 0.23.x, but the collection should have
been written with the underlying type of the define (ie file{} instead of
Deftest). I'm not sure at which version it started to not work, I noticed it
first on 0.24.4.
Note3: I noticed the issue on a more complex sample, which had another outcome
(basically an already defined resource error). I tried to simplify the failing
testcase but lost the hability to generate this error message, and discovered
this issue. I think solving this bug will solve my original issue.
Please see:
http://groups.google.com/group/puppet-users/browse_thread/thread/d04dccd244340581
----------------------------------------
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://reductivelabs.com/redmine/my/account
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en
-~----------~----~----~----~------~----~------~--~---
