Dne 27.3.2017 v 17:12 Radek Duda napsal(a):
Oh, I missed that requirement. Unfortunately that was introduced later. Honestly I'd recommend using non-lts version instead, they are also available in PIP and we are already talking about stabilizing before new LTS release.Hello Lukas,pro/post plugin does not solve this problem. We need to report to Beaker test name and result after *every* test run (and not after job - e.g. set of tests if I understand it correctly) (see Andrei's post https://www.redhat.com/archives/avocado-devel/2017-February/msg00043.html). pre/post plugin script is run only before/after job run. So is there any other way besides `ResultEvents` plugin (not available in avocado 36.0lts) to get result and name of each test?
Anyway if you still want to do this in 36lts, you can write a pre plugin, which would use `job.result_proxy.add_output_plugin()` to register your custom plugin inherited from `avocado.core.result.TestResult`. Basically this was the old way of defining plugins before Cleber turned them to the proper Stevedore plugins so it should work as long as you stay with 36lts, then you'd simply adopt the code to be the proper `ResultEvents` plugin (with similar structure).
Does this help? LukášPS: I'm tracing the master (not released, but really the master branch) in my testing virt mini-CI and never had an issue with that.
kind regards, Radek On Thu, Mar 16, 2017 at 11:56 AM, Lukáš Doktor <[email protected] <mailto:[email protected]>> wrote: Hello Radek, my proposal was to write a custom pre/post plugin, not just a pre/post script. You'd then ship your plugin to your test machines and install similarly to avocado-vt installation and if your plugin is generic enough, you can even propose it to avocado-vt upstream. There is already the `vt_joblock` pre/post plugin there (avocado-vt/avocado_vt/plugins/vt_joblock.py). I don't know of any way to obtain the information you need in `pre.d` script. In `post.d` script you could theoretically parse the `results` directory which you can get from the environment, but such solution would be quite ugly. Kind regards, Lukáš Dne 15.3.2017 v 15:41 Radek Duda napsal(a): Hi Lukas, concerning our IRC discussion (Lukas proposed to use job.test_suite[0][1].vt_param to obtain particular test parameters at the pre test phase): I still do not know how to solve accessibility of `job` variable in bash script (which should be created in dirs /etc/avocado/scripts/job/{pre.d, post.d}). The only solution seems to me is to export desired variables (test status, test name) to env (add it to `_job_to_environment_variables` method in `jobscripts.py`), but it is not good idea to mix avcado-vt variables to avocado code and I prefer to avoid editing avocado code at all. regards, Radek On Tue, Mar 14, 2017 at 6:17 PM, Lukáš Doktor <[email protected] <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>> wrote: Hello Radek, Dne 14.3.2017 v 12:14 Radek Duda napsal(a): Is there a way to implement this using Avocado 36.0lts?. I cannot find ResultEvent method in lts release nor plugin_interfaces.py file. The location is different, but the pre/post plugins were already supported in 36lts. The definition is in `avocado.plugins.base` and there is `avocado.plugins.jobscripts` which is an implementation of a pre/post plugin (which allows to execute custom scripts, which might actually be enough in your case, don't know). Anyway I don't know how are you executing the tests, but if you use Jenkins (or other automated way) I'd recommend adding those commands there. Anyway if you run them manually then shared configuration or even a plugin is the best approach. Lukáš regards, Radek Duda On Fri, Feb 10, 2017 at 9:19 PM, Lucas Meneghel Rodrigues <[email protected] <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>> <mailto:[email protected] <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>>> wrote: On Fri, Feb 10, 2017 at 11:16 AM Cleber Rosa <[email protected] <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>> <mailto:[email protected] <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>>> wrote: On 02/10/2017 05:01 AM, Andrei Stepanov wrote: > Hi. > > We need a reliable mechanism to notify Beaker server about start+result > of a avocado-vt test. As you know, avocado-vt test is one of the many > tests produced from cartesian config. We need to notify Beaker about > start+results of _each_ test. > > Currently we use: pre_command / post_command > > I discovered yesterday, that this commands are not called for FAILED > tests. Especially those have a error in syntax. > > In IRC Lukáš Doktor suggested to: > > 19:22<ldoktor>astepano: Hello Andrei, the `post_command` is part of the > `env_process` during the postprocess which means it's one of the cleanup > steps the problem is when the postprocess fails before getting to > `post_command` it will not be executed. It all depends on many factors. > Anyway I don't know what all information do you need to produce your > results but I'd strongly recommend writing either `JobPostTests` plugin, > or if you need per-test granularity t > 19:23<ldoktor>Also writing such plugin is really simple and there are > examples in our sources... > > So, I want to bring our conversation to this mail listing. > > I am not sure that such plugin will do job for avocado-vt tests. > Avacodo-vt tests generated from cartesian configs. > > Could you please suggest me the right approach to coupe with this issue? > > I need mechanism to call external program before & after _each_ > avocado-vt test. The program's environment should have variables: > TESTNAME / TESTRESULT. > Andrei, As a *very* brief answer, I'd say this looks like something that can be implemented as a `ResultEvents` plugin: * http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats>> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats> <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats <http://avocado-framework.readthedocs.io/en/45.0/ResultFormats.html#implementing-other-result-formats>>> * http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents>> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents> <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents <http://avocado-framework.readthedocs.io/en/45.0/api/core/avocado.core.html#avocado.core.plugin_interfaces.ResultEvents>>> You would write methods such as "start_test" and "end_test" to send the needed info to Beaker. This would be a feature generic to all Avocado supported tests (including Avocado-VT tests). Nice. with a ResultEvents beaker plugin, any avocado test would be able to report results to a beaker server, and plugin configuration can be done using standard config file sections. +1.
signature.asc
Description: OpenPGP digital signature
