Issue #2691 has been updated by Marc Fournier.
Status changed from Ready for Testing to Ready for Checkin
> You are completely right, the query is incorrect. It seems that it
> depends on the rail version, since on my test system I had the correct
> query.
My system has activerecord 2.3.2. Should we fear this bug could reappear if
people use other versions ?
> I'll send you another patch tonight.
This one works fine. Many thanks Brice !
SQL query:
<pre>
SELECT `resources`.* FROM `resources` INNER JOIN `resource_tags` ON
resource_tags.resource_id = resources.id INNER JOIN `puppet_tags` ON
`puppet_tags`.id = `resource_tags`.puppet_tag_id WHERE (host_id != 6 AND
(exported=1 AND restype='Nagios_service') AND (puppet_tags.name =
'nagios-nagiosserver.domain.tld'));
</pre>
Query plan:
<pre>
+----+-------------+---------------+--------+-------------------------------------------------------------------------+------------------------------------+---------+------------------------------------+------+-------------+
| id | select_type | table | type | possible_keys
| key |
key_len | ref | rows | Extra |
+----+-------------+---------------+--------+-------------------------------------------------------------------------+------------------------------------+---------+------------------------------------+------+-------------+
| 1 | SIMPLE | resources | ref |
PRIMARY,index_resources_on_host_id,typentitle |
typentitle | 767 | const
| 1212 | Using where |
| 1 | SIMPLE | resource_tags | ref |
index_resource_tags_on_resource_id,index_resource_tags_on_puppet_tag_id |
index_resource_tags_on_resource_id | 5 | puppet.resources.id
| 5 | Using where |
| 1 | SIMPLE | puppet_tags | eq_ref | PRIMARY,index_puppet_tags_on_id
| PRIMARY | 4
| puppet.resource_tags.puppet_tag_id | 1 | Using where |
+----+-------------+---------------+--------+-------------------------------------------------------------------------+------------------------------------+---------+------------------------------------+------+-------------+
</pre>
sample output (one row per exported resource, as in 0.24.8):
<pre>
+-------+----------------------------------+----------------+---------+----------------+----------+------+---------------------+---------------------+
| id | title | restype | host_id |
source_file_id | exported | line | updated_at | created_at |
+-------+----------------------------------+----------------+---------+----------------+----------+------+---------------------+---------------------+
| 38793 | @@check_load on cognac5-mgr | Nagios_service | 41 |
575 | 1 | 39 | 2009-10-21 01:31:26 | 2009-10-20 01:31:30 |
| 38856 | @@check_load on cognac6-mgr | Nagios_service | 42 |
575 | 1 | 39 | 2009-10-20 16:10:31 | 2009-10-20 01:34:28 |
| 38447 | @@check_load on cognac7-mgr | Nagios_service | 37 |
575 | 1 | 39 | 2009-10-21 00:39:27 | 2009-10-20 00:39:29 |
| 38975 | @@check_load on cognac8-mgr | Nagios_service | 43 |
575 | 1 | 39 | 2009-10-21 01:40:25 | 2009-10-20 01:40:34 |
| 39581 | @@check_load on cognac9-mgr | Nagios_service | 51 |
575 | 1 | 39 | 2009-10-21 02:47:24 | 2009-10-20 02:47:30 |
[...]
</pre>
----------------------------------------
Bug #2691: "Could not retrieve catalog: HTTP-Error: 500 Internal Server Error"
with tagged exported resources
http://projects.reductivelabs.com/issues/2691
Author: Marc Fournier
Status: Ready for Checkin
Priority: Normal
Assigned to: Brice Figureau
Category:
Target version: 0.25.1
Affected version: 0.25.1rc2
Keywords: rails, storeconfigs
Branch:
The problem happens with this sort of manifest (in fact, this simple example
works fine):
<pre>
node a {
@@file { "/tmp/foo": content => "fjskfjs\n", tag => "foofile", }
}
node b {
File <<| tag == 'foofile' |>>
}
</pre>
In my case, I have a about 20 nodes, each one having:
<pre>
@@nagios_host { $fqdn: tag => "nagiosserver.domain.tld", ... }
@@nagios_service { "a": tag => "nagiosserver.domain.tld", ... }
@@nagios_service { "b": tag => "nagiosserver.domain.tld", ... }
@@nagios_service { "c": tag => "nagiosserver.domain.tld", ... }
@@nagios_service { "d": tag => "nagiosserver.domain.tld", ... }
@@nagios_service { "e": tag => "nagiosserver.domain.tld", ... }
</pre>
I suspect the problem occurs when many exported resources are involved (if 20x5
can be considered many).
The problem happens when running puppet on a node containing:
<pre>
Nagios_host <<| tag == "nagiosserver.domain.tld" |>>
Nagios_service <<| tag == nagiosserver.domain.tld" |>>
</pre>
On the puppetmaster, one ruby process consumes 100% CPU during a few minutes,
then the client says:
<pre>
err: Could not call puppetmaster.getconfig: #<RuntimeError: HTTP-Error: 500
Internal Server Error>
err: Could not retrieve catalog: HTTP-Error: 500 Internal Server Error
</pre>
the ruby process continues running at 100%, and after a while we get this
backtrace:
<pre>
[ pid=3338 file=ext/apache2/Hooks.cpp:547 time=2009-10-01 11:16:46.189 ]:
Backend process 3465 did not return a valid HTTP response. It returned no
data.
*** Exception Errno::EPIPE in Passenger RequestHandler (Broken pipe) (process
3465):
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/request_handler.rb:83:in
`write'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/request_handler.rb:83:in
`process_request'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_request_handler.rb:203:in
`main_loop'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:110:in
`run'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:67:in
`spawn_application'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/utils.rb:181:in
`safe_fork'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:60:in
`spawn_application'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:45:in
`spawn_application'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/spawn_manager.rb:158:in
`spawn_application'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/spawn_manager.rb:282:in
`handle_spawn_application'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:337:in
`__send__'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:337:in
`main_loop'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:187:in
`start_synchronously'
from
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/bin/passenger-spawn-server:61
</pre>
The problem can be produced with 0.24.8 and 0.25.0 clients, and 0.25.0 as well
as 0.25.1rc1 server, with passenger and these gems installed:
activerecord (2.3.2)
activesupport (2.3.2)
fastthread (1.0.7)
passenger (2.2.2)
rack (1.0.0)
rake (0.8.7)
It should be noted that this works fine:
<pre>
Nagios_host <<| |>>
Nagios_service <<| |>>
</pre>
I've also noticed the following error from time to time on 0.25.0. But I'm
unable to reproduce it, and never seen it with 0.25.1rc1, so I'm not sure it's
related:
<pre>
/srv/puppet/lib/puppet/util/settings/file_setting.rb:19:in `group='
/srv/puppet/lib/puppet/util/settings/setting.rb:44:in `send'
/srv/puppet/lib/puppet/util/settings/setting.rb:44:in `initialize'
/srv/puppet/lib/puppet/util/settings/setting.rb:38:in `each'
/srv/puppet/lib/puppet/util/settings/setting.rb:38:in `initialize'
/srv/puppet/lib/puppet/util/settings.rb:398:in `new'
/srv/puppet/lib/puppet/util/settings.rb:398:in `newsetting'
/srv/puppet/lib/puppet/util/settings.rb:533:in `setdefaults'
/srv/puppet/lib/puppet/util/settings.rb:518:in `each'
/srv/puppet/lib/puppet/util/settings.rb:518:in `setdefaults'
/srv/puppet/lib/puppet/reports/store.rb:15:in `mkclientdir'
/srv/puppet/lib/puppet/reports/store.rb:35:in `process'
/srv/puppet/lib/puppet/network/handler/report.rb:66:in `process'
/srv/puppet/lib/puppet/network/handler/report.rb:59:in `each'
/srv/puppet/lib/puppet/network/handler/report.rb:59:in `process'
/srv/puppet/lib/puppet/network/handler/report.rb:33:in `report'
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `to_proc'
/srv/puppet/lib/puppet/network/xmlrpc/processor.rb:52:in `call'
/srv/puppet/lib/puppet/network/xmlrpc/processor.rb:52:in `protect_service'
/srv/puppet/lib/puppet/network/xmlrpc/processor.rb:85:in `setup_processor'
/usr/lib/ruby/1.8/xmlrpc/server.rb:336:in `call'
/usr/lib/ruby/1.8/xmlrpc/server.rb:336:in `dispatch'
/usr/lib/ruby/1.8/xmlrpc/server.rb:323:in `each'
/usr/lib/ruby/1.8/xmlrpc/server.rb:323:in `dispatch'
/usr/lib/ruby/1.8/xmlrpc/server.rb:366:in `call_method'
/usr/lib/ruby/1.8/xmlrpc/server.rb:378:in `handle'
/srv/puppet/lib/puppet/network/xmlrpc/processor.rb:44:in `process'
/srv/puppet/lib/puppet/network/http/rack/xmlrpc.rb:35:in `process'
/srv/puppet/lib/puppet/network/http/rack.rb:48:in `call'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/request_handler.rb:81:in
`process_request'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_request_handler.rb:203:in
`main_loop'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:110:in
`run'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:67:in
`spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/utils.rb:181:in
`safe_fork'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:60:in
`spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/rack/application_spawner.rb:45:in
`spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/spawn_manager.rb:158:in
`spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/spawn_manager.rb:282:in
`handle_spawn_application'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:337:in
`__send__'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:337:in
`main_loop'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/lib/phusion_passenger/abstract_server.rb:187:in
`start_synchronously'
/usr/lib/ruby/gems/1.8/gems/passenger-2.2.2/bin/passenger-spawn-server:61
</pre>
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://reductivelabs.com/redmine/my/account
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en
-~----------~----~----~----~------~----~------~--~---
