Hello Lukas, pro/post plugin does not solve this problem. We need to report to Beaker test name and result after *every* test run (and not after job - e.g. set of tests if I understand it correctly) (see Andrei's post https://www.redhat.com/archives/avocado-devel/2017-February/msg00043.html). pre/post plugin script is run only before/after job run. So is there any other way besides `ResultEvents` plugin (not available in avocado 36.0lts) to get result and name of each test?
kind regards, Radek On Thu, Mar 16, 2017 at 11:56 AM, Lukáš Doktor <[email protected]> wrote: > Hello Radek, > > my proposal was to write a custom pre/post plugin, not just a pre/post > script. You'd then ship your plugin to your test machines and install > similarly to avocado-vt installation and if your plugin is generic enough, > you can even propose it to avocado-vt upstream. There is already the > `vt_joblock` pre/post plugin there (avocado-vt/avocado_vt/plugins > /vt_joblock.py). > > I don't know of any way to obtain the information you need in `pre.d` > script. In `post.d` script you could theoretically parse the `results` > directory which you can get from the environment, but such solution would > be quite ugly. > > Kind regards, > Lukáš > > Dne 15.3.2017 v 15:41 Radek Duda napsal(a): > >> Hi Lukas, >> concerning our IRC discussion (Lukas proposed to use >> job.test_suite[0][1].vt_param to obtain particular test parameters at >> the pre test phase): >> I still do not know how to solve accessibility of `job` variable in bash >> script (which should be created in dirs /etc/avocado/scripts/job/{pre.d, >> post.d}). The only solution seems to me is to export desired variables >> (test status, test name) to env (add it to >> `_job_to_environment_variables` method in `jobscripts.py`), but it is >> not good idea to mix avcado-vt variables to avocado code and I prefer to >> avoid editing avocado code at all. >> >> regards, >> >> Radek >> >> On Tue, Mar 14, 2017 at 6:17 PM, Lukáš Doktor <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hello Radek, >> >> Dne 14.3.2017 v 12:14 Radek Duda napsal(a): >> >> Is there a way to implement this using Avocado 36.0lts?. I >> cannot find >> ResultEvent method in lts release nor plugin_interfaces.py file. >> >> The location is different, but the pre/post plugins were already >> supported in 36lts. The definition is in `avocado.plugins.base` and >> there is `avocado.plugins.jobscripts` which is an implementation of >> a pre/post plugin (which allows to execute custom scripts, which >> might actually be enough in your case, don't know). >> >> Anyway I don't know how are you executing the tests, but if you use >> Jenkins (or other automated way) I'd recommend adding those commands >> there. Anyway if you run them manually then shared configuration or >> even a plugin is the best approach. >> >> Lukáš >> >> >> regards, >> >> Radek Duda >> >> On Fri, Feb 10, 2017 at 9:19 PM, Lucas Meneghel Rodrigues >> <[email protected] <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>>> wrote: >> >> >> >> On Fri, Feb 10, 2017 at 11:16 AM Cleber Rosa >> <[email protected] <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>>> wrote: >> >> >> >> On 02/10/2017 05:01 AM, Andrei Stepanov wrote: >> > Hi. >> > >> > We need a reliable mechanism to notify Beaker server >> about start+result >> > of a avocado-vt test. As you know, avocado-vt test is >> one of the many >> > tests produced from cartesian config. We need to >> notify Beaker about >> > start+results of _each_ test. >> > >> > Currently we use: pre_command / post_command >> > >> > I discovered yesterday, that this commands are not >> called for FAILED >> > tests. Especially those have a error in syntax. >> > >> > In IRC Lukáš Doktor suggested to: >> > >> > 19:22<ldoktor>astepano: Hello Andrei, the >> `post_command` is part of the >> > `env_process` during the postprocess which means it's >> one of the cleanup >> > steps the problem is when the postprocess fails before >> getting to >> > `post_command` it will not be executed. It all depends >> on many factors. >> > Anyway I don't know what all information do you need >> to produce your >> > results but I'd strongly recommend writing either >> `JobPostTests` plugin, >> > or if you need per-test granularity t >> > 19:23<ldoktor>Also writing such plugin is really >> simple and there are >> > examples in our sources... >> > >> > So, I want to bring our conversation to this mail >> listing. >> > >> > I am not sure that such plugin will do job for >> avocado-vt tests. >> > Avacodo-vt tests generated from cartesian configs. >> > >> > Could you please suggest me the right approach to >> coupe with this issue? >> > >> > I need mechanism to call external program before & >> after _each_ >> > avocado-vt test. The program's environment should have >> variables: >> > TESTNAME / TESTRESULT. >> > >> >> Andrei, >> >> As a *very* brief answer, I'd say this looks like >> something that >> can be >> implemented as a `ResultEvents` plugin: >> >> * >> >> http://avocado-framework.readthedocs.io/en/45.0/ResultFormat >> s.html#implementing-other-result-formats >> <http://avocado-framework.readthedocs.io/en/45.0/ResultForma >> ts.html#implementing-other-result-formats> >> >> <http://avocado-framework.readthedocs.io/en/45.0/ResultForma >> ts.html#implementing-other-result-formats >> <http://avocado-framework.readthedocs.io/en/45.0/ResultForma >> ts.html#implementing-other-result-formats>> >> * >> >> http://avocado-framework.readthedocs.io/en/45.0/api/core/ >> avocado.core.html#avocado.core.plugin_interfaces.ResultEvents >> <http://avocado-framework.readthedocs.io/en/45.0/api/core/ >> avocado.core.html#avocado.core.plugin_interfaces.ResultEvents> >> >> <http://avocado-framework.readthedocs.io/en/45.0/api/core/ >> avocado.core.html#avocado.core.plugin_interfaces.ResultEvents >> <http://avocado-framework.readthedocs.io/en/45.0/api/core/ >> avocado.core.html#avocado.core.plugin_interfaces.ResultEvents>> >> >> You would write methods such as "start_test" and >> "end_test" to >> send the >> needed info to Beaker. >> >> This would be a feature generic to all Avocado supported >> tests >> (including Avocado-VT tests). >> >> >> Nice. with a ResultEvents beaker plugin, any avocado test >> would be >> able to report results to a beaker server, and plugin >> configuration >> can be done using standard config file sections. +1. >> >> >> >> >> >
