Agents can now have an class method called activate? that can return true, false or raise an exception. For false or exception the agent will not be added to the running mcollectived and so the node will not respond to discovery.
The SimpleRPC base agent has a default activate? method that looks in the config file and allows users to disable agents based on config. A helper activate_when was added to the SimpleRPC base agent that users can use to override the default activate? method with their own logic for activation. This should be used by agents to figure out if they're capable of working on a given node. It should make deployment simpler as given correctly written agents you should now be able to copy all agents to all machines. Test coverage was added to the M::Agents class Signed-off-by: R.I.Pienaar <[email protected]> --- Local-branch: feature/master/7583 lib/mcollective/agents.rb | 59 +++++++-- lib/mcollective/rpc/agent.rb | 48 +++++++- spec/unit/agents_spec.rb | 280 ++++++++++++++++++++++++++++++++++++++++++ website/simplerpc/agents.md | 42 +++++++ 4 files changed, 415 insertions(+), 14 deletions(-) create mode 100644 spec/unit/agents_spec.rb diff --git a/lib/mcollective/agents.rb b/lib/mcollective/agents.rb index 05167aa..594df20 100644 --- a/lib/mcollective/agents.rb +++ b/lib/mcollective/agents.rb @@ -2,26 +2,30 @@ module MCollective # A collection of agents, loads them, reloads them and dispatches messages to them. # It uses the PluginManager to store, load and manage instances of plugins. class Agents - def initialize + def initialize(agents = {}) @config = Config.instance raise ("Configuration has not been loaded, can't load agents") unless @config.configured - @@agents = {} + @@agents = agents loadagents end - # Loads all agents from disk - def loadagents - Log.debug("Reloading all agents from disk") - - # We're loading all agents so just nuke all the old agents and unsubscribe + # Deletes all agents + def clear! @@agents.each_key do |agent| PluginManager.delete "#{agent}_agent" Util.unsubscribe(Util.make_target(agent, :command)) end @@agents = {} + end + + # Loads all agents from disk + def loadagents + Log.debug("Reloading all agents from disk") + + clear! @config.libdir.each do |libdir| agentdir = "#{libdir}/mcollective/agent" @@ -38,7 +42,7 @@ module MCollective def loadagent(agentname) agentfile = findagentfile(agentname) return false unless agentfile - classname = "MCollective::Agent::#{agentname.capitalize}" + classname = class_for_agent(agentname) PluginManager.delete("#{agentname}_agent") @@ -46,22 +50,51 @@ module MCollective single_instance = ["registration", "discovery"].include?(agentname) PluginManager.loadclass(classname) - PluginManager << {:type => "#{agentname}_agent", :class => classname, :single_instance => single_instance} - Util.subscribe(Util.make_target(agentname, :command)) unless @@agents.include?(agentname) + if activate_agent?(agentname) + PluginManager << {:type => "#{agentname}_agent", :class => classname, :single_instance => single_instance} - @@agents[agentname] = {:file => agentfile} - return true + Util.subscribe(Util.make_target(agentname, :command)) unless @@agents.include?(agentname) + + @@agents[agentname] = {:file => agentfile} + return true + else + Log.debug("Not activating agent #{agentname} due to agent policy in activate? method") + return false + end rescue Exception => e Log.error("Loading agent #{agentname} failed: #{e}") PluginManager.delete("#{agentname}_agent") + return false end end + # Builds a class name string given a Agent name + def class_for_agent(agent) + "MCollective::Agent::#{agent.capitalize}" + end + + # Checks if a plugin should be activated by + # calling #activate? on it if it responds to + # that method else always activate it + def activate_agent?(agent) + klass = Kernel.const_get("MCollective").const_get("Agent").const_get(agent.capitalize) + + if klass.respond_to?("activate?") + return klass.activate? + else + Log.debug("#{klass} does not have an activate? method, activating as default") + return true + end + rescue Exception => e + Log.warn("Agent activation check for #{agent} failed: #{e.class}: #{e}") + return false + end + # searches the libdirs for agents def findagentfile(agentname) @config.libdir.each do |libdir| - agentfile = "#{libdir}/mcollective/agent/#{agentname}.rb" + agentfile = File.join([libdir, "mcollective", "agent", "#{agentname}.rb"]) if File.exist?(agentfile) Log.debug("Found #{agentname} at #{agentfile}") return agentfile diff --git a/lib/mcollective/rpc/agent.rb b/lib/mcollective/rpc/agent.rb index ebff6df..113f8e2 100644 --- a/lib/mcollective/rpc/agent.rb +++ b/lib/mcollective/rpc/agent.rb @@ -60,7 +60,8 @@ module MCollective # and help generation begin @ddl = DDL.new(@agent_name) - rescue + rescue Exception => e + Log.warn("Failed to load DDL for agent: #{e.class}: #{e}") @ddl = nil end @@ -124,6 +125,39 @@ module MCollective end end + # By default RPC Agents support a toggle in the configuration that + # can enable and disable them based on the agent name + # + # Example an agent called Foo can have: + # + # plugin.foo.activate_agent = false + # + # and this will prevent the agent from loading on this particular + # machine. + # + # Agents can use the activate_when helper to override this for example: + # + # activate_when do + # File.exist?("/usr/bin/puppet") + # end + def self.activate? + agent_name = self.to_s.split("::").last.downcase + + Log.debug("Meh? : #{respond_to?("meh")}") + Log.debug("Starting default activation checks for #{agent_name}") + + should_activate = Config.instance.pluginconf["#{agent_name}.activate_agent"] + + if should_activate + Log.debug("Found plugin config #{agent_name}.activate_agent with value #{should_activate}") + unless should_activate =~ /^1|y|true$/ + return false + end + end + + return true + end + # Generates help using the template based on the data # created with metadata and input def self.help(template) @@ -235,6 +269,18 @@ module MCollective } end + # Creates the needed activate? class in a manner similar to the other + # helpers like action, authorized_by etc + # + # activate_when do + # File.exist?("/usr/bin/puppet") + # end + def self.activate_when(&block) + (class << self; self; end).instance_eval do + define_method("activate?", &block) + end + end + # Creates a new action with the block passed and sets some defaults # # action "status" do diff --git a/spec/unit/agents_spec.rb b/spec/unit/agents_spec.rb new file mode 100644 index 0000000..50caebc --- /dev/null +++ b/spec/unit/agents_spec.rb @@ -0,0 +1,280 @@ +#!/usr/bin/env ruby + +require File.dirname(__FILE__) + '/../spec_helper' + +module MCollective + describe Agents do + before do + tmpfile = Tempfile.new("mc_agents_spec") + path = tmpfile.path + tmpfile.close! + + @tmpdir = FileUtils.mkdir_p(path) + @tmpdir = @tmpdir[0] if @tmpdir.is_a?(Array) # ruby 1.9.2 + + @agentsdir = File.join([@tmpdir, "mcollective", "agent"]) + FileUtils.mkdir_p(@agentsdir) + + logger = mock + logger.stubs(:log) + logger.stubs(:start) + Log.configure(logger) + end + + after do + FileUtils.rm_r(@tmpdir) + end + + describe "#initialize" do + it "should fail if configuration has not been loaded" do + Config.any_instance.expects(:configured).returns(false) + + expect { + Agents.new + }.to raise_error("Configuration has not been loaded, can't load agents") + end + + it "should load agents" do + Config.any_instance.expects(:configured).returns(true) + Agents.any_instance.expects(:loadagents).once + + Agents.new + end + end + + describe "#clear!" do + it "should delete and unsubscribe all loaded agents" do + Config.any_instance.expects(:configured).returns(true).at_least_once + Config.any_instance.expects(:libdir).returns([@tmpdir]) + PluginManager.expects(:delete).with("foo_agent").once + Util.expects(:make_target).with("foo", :command).returns("foo_target") + Util.expects(:unsubscribe).with("foo_target") + + a = Agents.new({"foo" => 1}) + end + end + + describe "#loadagents" do + before do + Config.any_instance.stubs(:configured).returns(true) + Config.any_instance.stubs(:libdir).returns([@tmpdir]) + Agents.any_instance.stubs("clear!").returns(true) + end + + it "should delete all existing agents" do + Agents.any_instance.expects("clear!").once + a = Agents.new + end + + it "should attempt to load agents from all libdirs" do + Config.any_instance.expects(:libdir).returns(["/nonexisting", "/nonexisting"]) + File.expects("directory?").with("/nonexisting/mcollective/agent").twice + + a = Agents.new + end + + it "should load found agents" do + Agents.any_instance.expects("loadagent").with("test").once + + FileUtils.touch(File.join([@agentsdir, "test.rb"])) + + a = Agents.new + end + + it "should load each agent unless already loaded" do + Agents.any_instance.expects("loadagent").with("test").never + + FileUtils.touch(File.join([@agentsdir, "test.rb"])) + + PluginManager << {:type => "test_agent", :class => String.new} + a = Agents.new + end + end + + describe "#loadagent" do + before do + FileUtils.touch(File.join([@agentsdir, "test.rb"])) + Config.any_instance.stubs(:configured).returns(true) + Config.any_instance.stubs(:libdir).returns([@tmpdir]) + Agents.any_instance.stubs("clear!").returns(true) + PluginManager.stubs(:loadclass).returns(true) + Util.stubs("subscribe").with("test_target").returns(true) + Util.stubs("make_target").with("test", :command).returns("test_target") + Agents.stubs(:findagentfile).returns(File.join([@agentsdir, "test.rb"])) + Agents.any_instance.stubs("activate_agent?").returns(true) + + @a = Agents.new + end + + it "should return false if the agent file is missing" do + Agents.any_instance.expects(:findagentfile).returns(false).once + @a.loadagent("test").should == false + end + + it "should delete the agent before loading again" do + PluginManager.expects(:delete).with("test_agent").twice + @a.loadagent("test") + end + + it "should load the agent class from disk" do + PluginManager.expects(:loadclass).with("MCollective::Agent::Test") + @a.loadagent("test") + end + + it "should check if the agent should be activated" do + Agents.any_instance.expects(:findagentfile).with("test").returns(File.join([@agentsdir, "test.rb"])) + Agents.any_instance.expects("activate_agent?").with("test").returns(true) + @a.loadagent("test").should == true + end + + it "should set discovery and registration to be single instance plugins" do + PluginManager.expects("<<").with({:type => "registration_agent", :class => "MCollective::Agent::Registration", :single_instance => true}).once + PluginManager.expects("<<").with({:type => "discovery_agent", :class => "MCollective::Agent::Discovery", :single_instance => true}).once + Agents.any_instance.expects("activate_agent?").with("registration").returns(true) + Agents.any_instance.expects("activate_agent?").with("discovery").returns(true) + + PluginManager.expects(:loadclass).with("MCollective::Agent::Registration").returns(true).once + PluginManager.expects(:loadclass).with("MCollective::Agent::Discovery").returns(true).once + + FileUtils.touch(File.join([@agentsdir, "registration.rb"])) + FileUtils.touch(File.join([@agentsdir, "discovery.rb"])) + + @a.loadagent("registration") + @a.loadagent("discovery") + end + + it "should add general plugins as multiple instance plugins" do + PluginManager.expects("<<").with({:type => "test_agent", :class => "MCollective::Agent::Test", :single_instance => false}).once + @a.loadagent("test") + end + + it "should add the agent to the plugin manager and subscribe" do + PluginManager.expects("<<").with({:type => "foo_agent", :class => "MCollective::Agent::Foo", :single_instance => false}) + Util.expects("make_target").with("foo", :command).returns("foo_target") + Util.expects("subscribe").with("foo_target").returns(true) + Agents.any_instance.expects(:findagentfile).with("foo").returns(File.join([@agentsdir, "foo.rb"])) + + FileUtils.touch(File.join([@agentsdir, "foo.rb"])) + + @a.loadagent("foo") + end + + it "should add the agent to the agent list" do + Agents.agentlist.should == ["test"] + end + + it "should return true on success" do + @a.loadagent("test").should == true + end + + it "should handle load exceptions" do + Agents.any_instance.expects(:findagentfile).with("foo").returns(File.join([@agentsdir, "foo.rb"])) + Log.expects(:error).with(regexp_matches(/Loading agent foo failed/)) + @a.loadagent("foo").should == false + end + + it "should delete plugins that failed to load" do + Agents.any_instance.expects(:findagentfile).with("foo").returns(File.join([@agentsdir, "foo.rb"])) + PluginManager.expects(:delete).with("foo_agent").twice + + @a.loadagent("foo").should == false + end + end + + describe "#class_for_agent" do + it "should return the correct class" do + Config.any_instance.stubs(:configured).returns(true) + Agents.any_instance.stubs(:loadagents).returns(true) + Agents.new.class_for_agent("foo").should == "MCollective::Agent::Foo" + end + end + + describe "#activate_agent?" do + before do + Config.any_instance.stubs(:configured).returns(true) + Agents.any_instance.stubs(:loadagents).returns(true) + @a = Agents.new + + module MCollective::Agent;end + class MCollective::Agent::Test; end + end + + it "should check if the correct class has an activation method" do + Agent::Test.expects("respond_to?").with("activate?").once + + @a.activate_agent?("test") + end + + it "should call the activation method" do + Agent::Test.expects("activate?").returns(true).once + @a.activate_agent?("test") + end + + it "should log a debug message and return true if the class has no activation method" do + Agent::Test.expects("respond_to?").with("activate?").returns(false).once + Log.expects(:debug).with("MCollective::Agent::Test does not have an activate? method, activating as default") + + @a.activate_agent?("test").should == true + end + + it "should handle exceptions in the activation as false" do + Agent::Test.expects("activate?").raises(RuntimeError) + @a.activate_agent?("test").should == false + end + end + + describe "#findagentfile" do + before do + Config.any_instance.stubs(:configured).returns(true) + Config.any_instance.stubs(:libdir).returns([@tmpdir]) + Agents.any_instance.stubs(:loadagents).returns(true) + @a = Agents.new + end + + it "should support multiple libdirs" do + Config.any_instance.expects(:libdir).returns([@tmpdir, @tmpdir]).once + File.expects("exist?").returns(false).twice + @a.findagentfile("test") + end + + it "should look for the correct filename in the libdir" do + File.expects("exist?").with(File.join([@tmpdir, "mcollective", "agent", "test.rb"])).returns(false).once + @a.findagentfile("test") + end + + it "should return the full path if the agent is found" do + agentfile = File.join([@tmpdir, "mcollective", "agent", "test.rb"]) + File.expects("exist?").with(agentfile).returns(true).once + @a.findagentfile("test").should == agentfile + end + + it "should return false if no agent is found" do + @a.findagentfile("foo").should == false + end + end + + describe "#include?" do + it "should correctly report the plugin state" do + Config.any_instance.stubs(:configured).returns(true) + Config.any_instance.stubs(:libdir).returns([@tmpdir]) + Agents.any_instance.stubs(:loadagents).returns(true) + PluginManager.expects("include?").with("test_agent").returns(true) + + @a = Agents.new + + @a.include?("test").should == true + end + end + + describe "#agentlist" do + it "should return the correct agent list" do + Config.any_instance.stubs(:configured).returns(true) + Config.any_instance.stubs(:libdir).returns([@tmpdir]) + Agents.any_instance.stubs(:loadagents).returns(true) + + @a = Agents.new("test" => true) + Agents.agentlist.should == ["test"] + end + end + end +end diff --git a/website/simplerpc/agents.md b/website/simplerpc/agents.md index d4cebd7..04037f9 100644 --- a/website/simplerpc/agents.md +++ b/website/simplerpc/agents.md @@ -127,6 +127,48 @@ These two code blocks have the identical outcome, the 2nd usage is recommended. Creates an action called "echo". They don't and can't take any arguments. +## Agent Activation +In the past you had to copy an agent only to machines that they should be running on as +all agents were activated regardless of dependencies. + +To make deployment simpler agents support the ability to determine if they should run +on a particular platform. By default SimpleRPC agents can be configured to activate +or not: + +{% highlight ini %} +plugin.helloworld.activate_agent = false +{% endhighlight %} + +You can also place the following in _/etc/mcollective/plugins.d/helloworld.cfg_: + +{% highlight ini %} +activate_agent = false +{% endhighlight %} + +This is a simple way to enable or disable an agent on your machine, agents can also +declare their own logic that will get called each time an agent gets loaded from disk. + +{% highlight ruby %} +module MCollective + module Agent + class Helloworld<RPC::Agent + + activate_when do + File.executable?("/usr/bin/puppet") + end + end + end +end +{% endhighlight %} + +If this block returns false or raises an exception then the agent will not be active on +this machine and it will not be discovered. + +When the agent gets loaded it will test if _/usr/bin/puppet_ exist and only if it does +will this agent be enabled. + +This feature is available since version 1.3.0 + ## Help and the Data Description Language We have a separate file that goes together with an agent and is used to describe the agent in detail, a DDL file for the above echo agent can be seen below: -- 1.7.1 -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
