Well FWIW, I can't replicate this when I upgrade from 1.0.1 to 1.3.2, I even attempted to name things in the same way, no luck:
https://gist.github.com/kbarber/6184213. Looking at the schema changes in code, not much has changed to do with parameters since 1.0.1 either: https://github.com/puppetlabs/puppetdb/blob/master/src/com/puppetlabs/puppetdb/scf/migrate.clj#L326-L331 So its probably not specifically a bug with the schema change. On Thu, Aug 8, 2013 at 12:48 PM, Ken Barber <[email protected]> wrote: >> We've come across a rather strange problem where the parameters of some >> resources in PuppetDB are now empty. >> >> We have a Nagios server collecting resources from PuppetDB and we've started >> to get failures like this for one resource type: >> >> Error: Could not retrieve catalog from remote server: Error 400 on SERVER: >> Must pass host_alias to Nagios::Config::Host[hostname] on node nagiosserver >> >> The Puppet manifest that defines that resources is this, it is impossible to >> not populate host_alias: >> >> ******************************* >> define nagios::host::host($host = $::fqdn, $tag = undef) { >> @@nagios::config::host { $host: >> host_alias => $host, >> address => $host, >> tag => $use_tag, >> } >> } >> ******************************* >> >> If we query PuppetDB directly (redacted), there are indeed no parameters at >> all on this resource: >> >> ******************************* >> # curl -H 'Accept: application/json' -X GET >> 'https://puppet:8081/v2/resources' --cacert >> /var/lib/puppet/ssl/ca/ca_crt.pem --cert >> /var/lib/puppet/ssl/certs/puppet.pem --key >> /var/lib/puppet/ssl/private_keys/puppet.pem --data-urlencode 'query=["=", >> "type", "Nagios::Config::Host"]' | jgrep "certname=hostname" >> [ >> { >> "resource": "8ba4379c364b9dba9d18836ef52ce5f4f82d0468", >> "parameters": { >> }, >> "title": "hostname", >> "exported": true, >> "certname": "hostname", >> "type": "Nagios::Config::Host", >> "sourceline": 27, >> "sourcefile": >> "/etc/puppet/environments/production/modules/nagios/manifests/host/host.pp", >> "tags": [ >> ] >> } >> ] >> ******************************* >> >> After a Puppet run and a new catalog, this resource now looks normal: >> >> ******************************* >> # curl -H 'Accept: application/json' -X GET >> 'https://puppet:8081/v2/resources' --cacert >> /var/lib/puppet/ssl/ca/ca_crt.pem --cert >> /var/lib/puppet/ssl/certs/puppet.pem --key >> /var/lib/puppet/ssl/private_keys/puppet.pem --data-urlencode 'query=["=", >> "type", "Nagios::Config::Host"]' | jgrep "certname=hostname" >> [ >> { >> "type": "Nagios::Config::Host", >> "sourceline": 27, >> "title": "hostname", >> "certname": "hostname", >> "resource": "8ba4379c364b9dba9d18836ef52ce5f4f82d0468", >> "parameters": { >> "address": "hostname", >> "tag": "tag", >> "host_alias": "hostname" >> }, >> "exported": true, >> "sourcefile": >> "/etc/puppet/environments/production/modules/nagios/manifests/host/host.pp", >> "tags": [ >> ] >> } >> ] >> ******************************* >> >> These nodes do not have Puppet run on them regularly. We did upgrade from >> PuppetDB 1.0.1-1.el6.noarch to 1.3.2-1.el6.noarch about 3 weeks ago. We >> don't do any automatic report or node expiry. >> >> This started happening back on 2nd August, just halfway through the day the >> Puppet runs on the Nagios server start failing with this error. Now if I >> think back, at this time I think I had a broken Nagios module and a lot of >> manifests were failing to compile, but I fixed this and re-ran the failures, >> and everything was ok. PuppetDB only stores the last catalog, so there's no >> way a broken catalog could have stayed there, right? >> >> I've fixed this by refreshing the catalog of all nodes in PuppetDB, but I've >> got no idea how it got into this state. Any ideas? > > No good idea yet, but there is something suspicious in your curl > responses - the "resource" hash, did you obfuscate this yourself on > purpose? The two hashes between the first and second requests are > identical. That hash is calculated based on the sum of the resource, > including parameters - so it seems impossible that PuppetDB arrived at > the same hash with and without parameters. > > Maybe just maybe the responses were identical, and somehow PuppetDB > was not returning parameters as a fault. This might indicate some sort > of integrity problem, whereby the link to the parameters in the RDBMS > was lost somehow, although this is the first time I've heard of it > Luke. Maybe this was an upgrade schema change failure between 1.0.1 > and 1.3.2? I'd have to consult what changed in the schema between > those two points to determine if this is likely however. Does the > timing of your upgrade, and the first time you saw this fault line up > with such a possibility? Remember a schema change will only occur > after a restart of PuppetDB ... (so perhaps consult your logs to see > when this happened after upgrade). > > Let me at least try to replicate while I await your responses. > > ken. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
