Hello Cleber Dne 9.5.2016 v 17:12 Cleber Rosa napsal(a):
Note: the same content on this message is available at:https://github.com/clebergnu/avocado/blob/rfc_plugin_api/docs/rfcs/plugin-management.rst Some users may find it easier to read with a prettier formatting. Problem statement ================= This RFC is an offshoot from the "Avocado Job API" RFC. It aims to review the current extensibility architecture in Avocado, define common terminology and propose an API that will allow management and configuration of those extensible components. Again, the upcoming proposals, while hopefully useful in their own rights, are heavily influenced by the perceived needs of the "Job API". To put it simply, the Job API is also in the scope of the problem attempted to be solved here. Review ====== During its development, Avocado has adopted many different extensibility techniques. This is common in many projects and even more common in projects built around languages that facilitate the loading of modules at runtime and `duck typing`_. On the bright side, Avocado has already got rid of much of the custom code for handling those extensions, by adopting a mature external library called `stevedore`_. Also on the bright side, there's now the formal definition of the interfaces expected to be available by users of those plugins. Terminology =========== It was noticed during recent discussions that there seems to be a lack of common understanding about what a Plugin is, how they're supposed to be implemented and consumed. This section attempts to settle the proposed terminology. Plugin type ----------- A logical categorization of an extensible subsystem. In a music player application, valid plugin types may include: * Audio decoder * Lyrics fetcher In an application such as Avocado, valid plugin types may include: * Test runner * Result formatter In Avocado, a given Plugin type maps to one setuptools entrypoint namespace, and to a interface declared as Python `abstract base classes`_. Plugin patterns --------------- A "plugin" is a generic enough term. A "plugin type" doesn't adds much more than a label to an interface. So, it's also useful to define different usage patterns. Driver ~~~~~~ When a single plugin is used at a given time. This usage pattern provides its user an abstract and usually simplified view of the controlled resource. The obvious examples are, of course, device drivers. On most, if not all, Operating Systems there will only be one driver implementation active to control a given device at a given time. That is true even when there may be multiple driver implementations for the same type of device. Using the music application example, for any given audio file, a single audio decoder "driver" is used. Using Avocado as an example, for any given test to be run, a single runner "driver" would be used. Extensions ~~~~~~~~~~ When multiple plugins can be used simultaneously. This usage pattern allows multiple and independent extensions to act on on a given subsystem or resource or task. Using the music application example, there may be multiple lyrics fetcher extensions, one for each different lyrics database. Using Avocado as an example, multiple result formatter may generate reports in different file formats for a single job. Interface Declaration --------------------- The formal announcement of the interfaces that can be expected by users of a given plugin. Consequently an interface declaration defines what must be implemented by individual plugins. Plugin Management API --------------------- A set of functionalities that enable the application to load, unload, configure, activate and deactivate plugins. Current Plugin Review ===================== Avocado relies exclusively on setuptools entrypoints to locate plugins. A setuptools entrypoint ties together the following 3 pieces of information: 1. namespace, associated with a specific plugin type 2. plugin name, should be unique for a given namespace 3. reference to the plugin implementation, pointing all the way to the Python class One example of such an entrypoint declaration:: 'avocado.plugins.cli': [ 'gdb = avocado.plugins.gdb:GDB', ] Here we have: 1. ``avocado.plugins.cli``, the namespace for all plugins of the ``CLI`` type. 2. ``gdb`` as the name for this ``CLI`` plugin. 3. Implemented as the ``GDB`` class of the ``avocado.plugins.gdb`` Python module. Avocado, by means of the various dispatcher classes, loads all registered plugins. Thus, if the plugin class can be loaded, it will be loaded by the current dispatcher at its initialization type. Finally, most dispatchers, at the appropriate time, will attempt to execute the implemented methods of all loaded plugins, by means of a ``map_method()`` call. The reason for that simplistic behavior is that all the current "new style" plugin are mapped to the "extensions" pattern. For instance, simply by adding registering a new plugin implementation at the ``avocado.plugins.cli`` namespace, that plugin method ``configure()`` will be executed (which is supposed to add its arguments to the command line application). For other plugin types and usage patterns, the current activation method is not always the best choice. That is, even if a plugin can be loaded by Avocado and is ready to be used, it doesn't mean that it should be used. For instance, on plugins that map to the driver usage patterns, it does not make sense to load all plugins of a given type. The loading and activation should be deferred to the time that a choice is made on a specific implementation. For extension usage patterns, it should allow the user to add or remove any number of implementations for a given user or resource at runtime. Example ======= Now, let's give a more concrete example of some of the concepts introduced earlier. The context used here is composed by the latest Avocado developments in the extensibility area, and also the proposed upcoming developments, with a focus on the "Job API" proposal. Interface Declaration --------------------- One of the recent additions to the "new plugin" standard in Avocado is the formal declaration of interfaces. When a new type of plugin is introduced, to make a specific subsystem of Avocado more modular, a formal interface is introduced. Then, when code is written to implement a specific Avocado subsystem, it MUST implement at least a minimum set of methods as properly defined in that interface. One example would be test runners. The declaration of an interface could be similar to:: class Runner(Plugin): @abstractmethod # makes the implementation mandatory def run_test(self, test_factory): # noop to be inherited by implementations of this interface def setup(self): pass # noop to be inherited by implementations of this interface def tear_down(self): pass Please note that, while a single method (``run_test()``) is marked as mandatory by the ``@abstractmethod`` decorator, the overall interface is also composed by ``setup()`` and ``tear_down()``. Users of plugins that implement this interface should be able to safely call all interface methods, given that non-mandatory will have default implementations. These default implementations can either be "noops" or code that is sensible enough to be used by default by any plugin implementing the given interface. Interface Implementation ------------------------ The implementation of a valid test runner could be similar to:: class Container(Runner): def setup(self): prepare_container_image() def run_test(self, test_factory): reference = create_reference(test_factory) copy_to_image(reference) return run_from_image(reference) def tear_down(self): destroy_container_image() Also valid would be one that implements only the mandatory methods and inherit default implementations from the base class:: class Local(Runner): def run_test(self, test_factory): test = create_test_from_factory(test_factory) return fork_and_run(test) Using an Interface ------------------ Assuming that implementations of a given interface are correctly registered and loaded (more on that on the next section), users interested in that subsystem can use any implmentation in an agnostic way. As an example, code that implements the concept of a Job, can leverage any of the "Runner" implementations transparently. If a Job implementation wants to respect what the user somehow set as the default runner, it could implement something similar to:: class Job(object): def __init__(self): default_runner = settings.get_value('runner', 'default', default='Local') self.runner = plugin_mgmt.new('avocado.plugins.runner', default_runner) def run(self): pre_tests_hooks() self.runner.setup() for test in tests: self.runner.run_test(test) self.runner.tear_down() post_test_hooks() Both example implementations should work similarly and transparently to the user (the Job implementation in this example). One of the proposed methods for the Plugin Management API has been quietly introduced in this example. It will be thoroughly discussed later. Plugin management API ===================== On the previous section, the idea of a plugin management API was quietly and briefly planted:: 1 class Job(object): 2 def __init__(self): 3 default_runner = settings.get_value('runner', 'default', 4 default='local') 5 self.runner = plugin_mgmt.new('avocado.plugins.runner', 6 default_runner)
so far so good.
Lines 5 and 6, refer to a proposed method called ``new()`` of an also
proposed module named ``plugin_mgmt``. Names are quite controversial,
and not really the goal at this point, so please bear with the naming
choices made so far, and feel free to suggest better ones.
It should be clear that the goal of the ``new()`` method is to make
an extensible subsystem implementation ready to be used. Its
implementation, directly or indirectly, may involve locating the
Python file that contains the associated class, loading that file into
the current interpreter, creating a class instance, and finally,
returning it.
This maps well to the driver pattern, where little or no code is necessary
around the plugin class instance itself.
For usage patterns that map to the extensions definition given before, the
"dispatcher" code may have higher level and additional methods::
01 class ResultFormatterDispatcher:
02
03 NAMESPACE = "avocado.plugin.result.format"
04
05 def add(self, name):
06 "Adds a plugin to the list of active result format writers"
07 self.active_set.add(plugin_mgmt.new(self.NAMESPACE, name)
08
09 def remove(self, name):
10 "Removes a plugin from the list of active result format
writers"
11 self.active_set.remove_by_name(name)
12
13 def set_active(self, names):
14 "Adds or removes plugins so that only given plugin names
are active"
15 ...
Which could be used as::
class Job(object):
def __init__(self):
...
self.result_formats = settings.get_value('result', 'formats',
default=['json',
'xunit'])
self.result_dispatcher = plugin_mgmt.ResultFormatterDispatcher()
...
def run(self):
self.result_dispatcher.set_active(self.result_formats)
...
for test in tests:
self.runner.run_test(test)
...
This would only work for a single-plugin-instance-environment. So for `Cli` plugins that's fine. But this does not suit the current Results plugins and it does not fits the possibility to get different plugins within one execution (Job/Test API).
So we either need to modify result plugins to produce multiple outputs and modify the framework/runner code to collect and forward all the arguments to the single plugin instance, or we can allow the dispatcher to add not plugins, but instances, using `dispatcher.add(plugin, *args, **kwargs)`.
self.result_dispatcher = plugin_mgmt.new('avocado.plugins.result.'
'format')
self.result_dispatcher.add('xunit', result_dir + "/results.xml")
self.result_dispatcher.add('json', result_dir + "/results.json")
if args.get('json'):
# args.get(json) contains result dir of json plugin (--json)
self.result_dispatcher.add('json', args.get('json'))
The initialization could also report requirements, for example `--json
-` would report `["stdout"]` and if `result_dispatcher` receives
multiple requests for the same requirement, it should fail to register
the plugin.
Worth mention we should allow the dispatcher to initialize all available plugins (with black list support) using default params, which would be used by `Cli` plugins.
If I understand this properly you're suggesting that the plugin interface should allow several dispatchers of the same time to be instantiated, containing different instances of the same plugins?Activation Scope ---------------- It was mentioned during the definition of the different plugin patterns that only one driver plugin would be active at a given time. This is a simplification, one that doesn't take into account any kind of scopes. Avocado's code should implement contained scopes and add/remove plugins instances to these scopes only. For instance, on a single job, there may be multiple parallel test runners. The activation scope for a test runner driver plugin, is of course, individual to each runner.
Layered APIs
============
It may be useful to provide more focused APIs as, supposedly, thin
layers around the features provided by Plugin Management API. One
example may be the activation and deactivation of test result
formatters. Example::
class Job(object):
def add_runner(self, plugin_name):
self.runners.append(plugin_mgmt.new('avocado.plugins.runner',
plugin_name))
This simplistic but quite real example has the goal of allowing users
of the ``Job`` class to simply call::
parallel_job = Job()
for count in xrange(multiprocessing.cpu_count()):
parallel_job.add_runner('local')
...
job.run_parallel(test_list)
Yep, the introduction is awesome. I'm concerned about the multiple-plugin-instances. We need a means to pass arguments to each instance.Conclusion ========== Hopefully this text helps to pin-point the aspects of the Avocado architecture that, even though may need adjustments, can contribute to the implementation of the ultimate goal of providing a "Job API".
So IMO we have 3 types of plugins: 1. driver (allow just one instance at a time and scope) 2. extension (allow and usually uses one instance of each existing plugin)3. proxy-like (allow several instances of any supported plugin, usually mixed)
Examples would be: 1. CliCmd 2. Cli 3. TestResultWhere the 1 and 2 uses settings/args, but the 3 gets the arguments from code. So we either need to adjust the plugins to allow multiple arguments and pass the arguments from code to args, or to allow multiple instances and arguments to dispatcher.
Lukáš
.. _duck typing: https://en.wikipedia.org/wiki/Duck_typing .. _stevedore: https://pypi.python.org/pypi/stevedore .. _abstract base classes: https://docs.python.org/2/library/abc.html
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Avocado-devel mailing list [email protected] https://www.redhat.com/mailman/listinfo/avocado-devel
