Issue #16327 has been updated by Shane Madden.
Seems like the cache is the issue, but I'm not sure why that would be the case.
The master that the client's reporting to (which is the master that's
configured to send the facts to the central-inventory master via the
inventory_service terminus) has been minimally modified from defaults in terms
of facts/inventory:
[master]
...
facts_terminus = inventory_service
inventory_server = inventory-master.example.com
inventory_port = 8140
Commenting those out yields a good cache in /var/lib/puppet/yaml/facts on the
satellite master on the client's next run, but that good cache doesn't help to
avoid the error when I switch the terminus back to inventory_service. The
local cache doesn't seem to be used for anything when inventory_service is set
as the terminus.
Is there something that changed about cache configuration in 3.0 that I've
missed and need to configure? If the save is supposed to happen before the
find, then that might be the problem - the warning that the inventory_service
terminus fires when the save method fails to send never occurs in 3.0, so the
run doesn't seem to be getting as far as the save method before failing.
It's also worth noting that this config works great with no modification when I
downgrade back to 2.7; the warning about the unavailable inventory server fires
on the attempt at the save method, the /var/lib/puppet/yaml/facts cache for the
node is updated successfully, and no further errors occur for the run.
P.S, I've found that when the satellite master is set to use_srv_records in its
[main] block, it ignores the configuration for inventory_server until after the
responses from the SRV lookup have all failed to return satisfactory results.
If one of those SRV records points to itself (and requests to /facts are
allowed), a request loop to itself occurs, hanging both the agent and master.
Is it intended for SRV records to override the inventory_server setting?
----------------------------------------
Bug #16327: Fact Terminus inventory_service not failing gracefully as intended
https://projects.puppetlabs.com/issues/16327#change-71034
Author: Shane Madden
Status: Needs More Information
Priority: Normal
Assignee:
Category:
Target version: 3.0.0
Affected Puppet version: 3.0.0-rc5
Keywords:
Branch:
The inventory_service fact terminus was created for bug #10289, to allow for
the updating of facts in a central inventory service while avoiding the single
point of failure behavior that occurs for the rest terminus when the inventory
server is down or misbehaving.
In 2.7.x, the failure is graceful, throwing a warning on the master that is
attempting attempting to send facts to the terminus:
warning: Could not upload facts for node.example.com to inventory service:
Connection refused - connect(2)
But in 3.0.0rc5, a failure occurs and the agent's run fails:
Error: Could not retrieve facts for node.example.com: Connection refused -
connect(2)
/usr/lib/ruby/1.8/net/http.rb:560
/usr/lib/ruby/1.8/net/http.rb:560
/usr/lib/ruby/1.8/net/http.rb:560
/usr/lib/ruby/1.8/timeout.rb:67
/usr/lib/ruby/1.8/timeout.rb:101
/usr/lib/ruby/1.8/net/http.rb:560
/usr/lib/ruby/1.8/net/http.rb:553
/usr/lib/ruby/1.8/net/http.rb:542
/usr/lib/ruby/1.8/net/http.rb:1035
/usr/lib/ruby/1.8/net/http.rb:772
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/connection.rb:61
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/connection.rb:61
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/connection.rb:25
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:105
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:105
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:84
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:118
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:177
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/request.rb:216
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:177
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/rest.rb:112
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/indirection.rb:191
/usr/lib/ruby/site_ruby/1.8/puppet/node.rb:87
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/node/plain.rb:17
/usr/lib/ruby/site_ruby/1.8/puppet/indirector/indirection.rb:191
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:105
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:68
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:68
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick/rest.rb:24
/usr/lib/ruby/1.8/webrick/httpserver.rb:104
/usr/lib/ruby/1.8/webrick/httpserver.rb:65
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:36
/usr/lib/ruby/1.8/webrick/server.rb:173
/usr/lib/ruby/1.8/webrick/server.rb:173
/usr/lib/ruby/1.8/webrick/server.rb:162
/usr/lib/ruby/1.8/webrick/server.rb:162
/usr/lib/ruby/1.8/webrick/server.rb:95
/usr/lib/ruby/1.8/webrick/server.rb:92
/usr/lib/ruby/1.8/webrick/server.rb:92
/usr/lib/ruby/1.8/webrick/server.rb:23
/usr/lib/ruby/1.8/webrick/server.rb:82
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:33
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:32
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:32
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:32
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:29
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:29
/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:96
/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:112
/usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:136
/usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:199
/usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:148
/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:342
/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:436
/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:342
/usr/lib/ruby/site_ruby/1.8/puppet/util.rb:513
/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:342
/usr/lib/ruby/site_ruby/1.8/puppet/util/command_line.rb:76
/usr/bin/puppet:10
Looks like the find method needs to get the same graceful handling treatment
that the save method is getting in indirector/facts/inventory_service.rb.
Thanks!
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
