Hello!
Finally it works! You are right - the problem was due to absence of plugin
'loader_yaml'.
I have another two questions:
1. How to use --vt-extra-params in command line and vt_extra_params section
in YAML at the same time? Option kernel is ignored when I specify it in
'avocado run config.yaml --vt-extra-params kernel=path_to_kernel'.
config.yaml:
basic: !mux
test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
timeout: 360
test_reference_resolver_args: !!python/dict
vt_machine_type: q35
vt_extra_params:
- extra_params=" -cpu host"
- nic_model=rtl8139
- drive_format=ahci
- mem=2048
shutdown:
test_reference: io-github-autotest-qemu.shutdown
test_reference_resolver_extra:
avocado_vt_extra_params:
- shutdown_method = shell
How to avoid it?
2. What is a proper way to specify guest os in YAML config?
I want to make a separate YAML config with guest OSes and mux it with test
config inserted above. As far as I understand it is need to somehow
override vt_guest option or use specific parameter from guest OS configs
(like image_name or os_variant) in YAML. But vt_guest_os is not processed
in avocado_vt/loader.py and second variant doesn't work too. Is is ever
possible without touching source code?
Sergey
вт, 7 авг. 2018 г. в 12:59, Lukáš Doktor <ldok...@redhat.com>:
> Dne 7.8.2018 v 11:16 Sergey Bronnikov napsal(a):
> > пн, 6 авг. 2018 г. в 19:03, Lukáš Doktor <ldok...@redhat.com>:
> >
> >> Dne 6.8.2018 v 09:52 Sergey Bronnikov napsal(a):
> >>> пт, 3 авг. 2018 г. в 14:55, Lukáš Doktor <ldok...@redhat.com>:
> >>>
> >>>> Well the execution is actually simpler than you think, `avocado run
> >>>> basic.yaml` should does the trick. Anyway the yaml-file is slightly
> >>>> corrupted and even then some things won't probably result in what you
> >>>> expect. Let me explain it a bit and provide better example)
> >>>>
> >>>> Dne 3.8.2018 v 10:20 Sergey Bronnikov napsal(a):
> >>>>> Lukas, thanks for explanation! But I still have a question.
> >>>>>
> >>>>> Imagine we have an YAML config with a list of tests and their
> >> parameters:
> >>>>>
> >>>>> ```yaml
> >>>>> basic: !mux
> >>>>>
> >>>>> test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
> >>>>> timeout: 360
> >>>>> test_reference_resolver_args: !!python/dict
> >>>>> vt_extra_params:
> >>>>> - nic_model=rtl8139
> >>>>> - extra_params=" -cpu host"
> >>>>> - drive_format=ahci
> >>>>> - mem=2048
> >>>>> vt-machine-type: q35
> >>>>
> >>>> this should be "vt_machine_type"
> >>>>
> >>>>>
> >>>>> shutdown:
> >>>>> test_reference: io-github-autotest-qemu.shutdown
> >>>>
> >>>> Here `test_reference_resolver_extra:` is missing.
> >>>>
> >>>>> vt_extra_params:
> >>>>
> >>>> this should be `avocado_vt_extra_params:` (see `avocado_vt/loader.py`)
> >>>>
> >>>>> - shutdown_method = shell
> >>>>>
> >>>>> reboot:
> >>>>> test_reference: io-github-autotest-qemu.reboot
> >>>>
> >>>> Here `test_reference_resolver_extra:` is missing.
> >>>>
> >>>>> vt_extra_params:
> >>>>> - param1=1
> >>>>> - param2=2
> >>>>>
> >>>>> stress:
> >>>>> test_reference: io-github-autotest-qemu.linux_stress
> >>>>>
> >>>>> iofuzz:
> >>>>> test_reference: io-github-autotest-qemu.iofuzz
> >>>>
> >>>> Here `test_reference_resolver_extra:` is missing.
> >>>>
> >>>>> vt_extra_params:
> >>>>> - param1=1
> >>>>> - param2=2
> >>>>>
> >>>>> ```
> >>>>>
> >>>>> How to execute tests from this testplan?
> >>>>
> >>>> as mentioned earlier, it's recognized as test format, therefor simply
> >>>> `avocado run basic.yaml`.
> >>>>
> >>>> Also one thing, the vt discovery is quite slow (yeah we parse
> cartesian
> >>>> config on each invocation as it might have different values). You can
> >> put
> >>>> multiple `test_reference` into the same line and separate them by
> space
> >> in
> >>>> avocado-vt loader. But beware, it works as filters, therefor the
> >> discovered
> >>>> tests will be re-oredered accordingly to cartesian config (rather than
> >> how
> >>>> you put them there) and they'll occur only once. Therefor:
> >>>>
> >>>> test_reference: shutdown boot boot
> >>>>
> >>>> will generate:
> >>>>
> >>>> avocado list basic.yaml
> >>>> VT io-github-autotest-qemu.boot
> >>>> VT io-github-autotest-qemu.shutdown
> >>>>
> >>>> Anyway the speed of that is incomparably faster, which is why I use it
> >>>> when possible.
> >>>>
> >>>
> >>>
> >>> Finally with corrections I got a following testplan:
> >>
> >> Dear Sergey,
> >>
> >> actually you unveiled a bug; fix can be found here:
> >> https://github.com/avocado-framework/avocado-vt/pull/1696 (in my CI I
> >> always use `--vt-extra-params` so I forgot it might not be set. Sorry
> about
> >> this).
> >>
> >>
> > `avocado list plan.yaml` still fails. With --verbose it awlays shows
> error
> > for any testplan:
> > "Unexpected character / on pos 7. Special chars are allowed only in
> > variable assignation statement: 'only ../avocado-vt/plans/lukas.yaml'
> > (<string>:1)"
> >
> > Common error for any YAML testplan:
> > ~$ avocado list --verbose ../avocado-vt/plans/lukas.yaml
> > Type
> > Test
> > Tag(s)
> > NOT_A_TEST ../avocado-vt/plans/lukas.yaml: Not an INSTRUMENTED
> > (avocado.Test based), PyUNITTEST (unittest.TestCase based) or SIMPLE
> > (executable) test
> > !VT ../avocado-vt/plans/lukas.yaml: Unexpected character / on pos
> > 7. Special chars are allowed only in variable assignation statement:
> 'only
> > ../avocado-vt/plans/lukas.yaml' (<string>:1)
> >
> > TEST TYPES SUMMARY
> > ...
> >
> > ~$ curl
> >
> https://github.com/avocado-framework/avocado/blob/master/examples/yaml_to_mux_loader/basic.yaml
>
> I assume you used `curl
> https://raw.githubusercontent.com/avocado-framework/avocado/master/examples/yaml_to_mux_loader/basic.yaml`
> <https://raw.githubusercontent.com/avocado-framework/avocado/master/examples/yaml_to_mux_loader/basic.yaml>
>
> >> ../avocado-vt/plans/basic.yaml
> > ~$ avocado list --verbose ../avocado-vt/plans/basic.yaml
> > Type
> > Test
> > Tag(s)
> > NOT_A_TEST ../avocado-vt/plans/basic.yaml: Not an INSTRUMENTED
> > (avocado.Test based), PyUNITTEST (unittest.TestCase based) or SIMPLE
> > (executable) test
> > !VT ../avocado-vt/plans/basic-plain.yaml: Unexpected character /
> on
> > pos 7. Special chars are allowed only in variable assignation statement:
> > 'only ../avocado-vt/plans/basic-plain.yaml' (<string>:1)
> >
>
> This is correct, it's NOT_A_TEST and !VT. Anyway there should be the
> "loader_yaml" installed which should understand that format and discover
> the tests defined by the varianter. What is the output of:
>
> $ avocado list --loaders help
> Unknown loader 'help'. Available plugins are:
> glib: glib.GLIB
> golang: golang.GOLANG
> yaml_testsuite:
> robot: robot.ROBOT
> vt: vt.VT
> file: file.SIMPLE, file.INSTRUMENTED, file.PyUNITTEST
> external: external.EXTERNAL
>
> If the "yaml_testsuite is not there you need to install it (cd
> optional_plugins/loader_yaml && python setup.py install). If you use `make
> link` it should be there but `make install` won't install the optional
> plugins.
>
> $ avocado list --verbose examples/mux-lukas.yaml
> Type Test
>
> Tag(s)
> NOT_A_TEST examples/mux-lukas.yaml: Not an INSTRUMENTED (avocado.Test
> based), PyUNITTEST (unittest.TestCase based) or SIMPLE (executable) test
> !GLIB examples/mux-lukas.yaml: No GLib-like tests found
> !GOLANG examples/mux-lukas.yaml: Go binary not found.
> INSTRUMENTED passtest.py:PassTest.test
>
> fast
> INSTRUMENTED sleeptest.py:SleepTest.test
> INSTRUMENTED failtest.py:FailTest.test
>
> failure_expected
> INSTRUMENTED failtest.py:FailTest.test
>
> failure_expected
> INSTRUMENTED failtest.py:FailTest.test
>
> failure_expected
> !ROBOT examples/mux-lukas.yaml: Unsupported file format 'yaml'.
> !VT examples/mux-lukas.yaml: Unexpected character / on pos 13.
> Special chars are allowed only in variable assignation statement: 'only
> examples/mux-lukas.yaml' (<string>:1)
>
> $ avocado list examples/mux-lukas.yaml
> INSTRUMENTED passtest.py:PassTest.test
> INSTRUMENTED sleeptest.py:SleepTest.test
> INSTRUMENTED failtest.py:FailTest.test
> INSTRUMENTED failtest.py:FailTest.test
> INSTRUMENTED failtest.py:FailTest.test
>
> Hopefully it's just the missing plugin,
> Lukáš
>
>
> > TEST TYPES SUMMARY
> > ==================
> > !VT: 1
> > ACCESS_DENIED: 0
> > BROKEN_SYMLINK: 0
> > EXTERNAL: 0
> > INSTRUMENTED: 0
> > MISSING: 0
> > NOT_A_TEST: 1
> > PYUNITTEST: 0
> > SIMPLE: 0
> > VT: 0
> >
> > Sergey
> >
> >
> >> With this there is still one little remaining issue in your
> `basic.yaml`,
> >> see below...
> >>
> >>>
> >>> basic: !mux
> >>> test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
> >>> timeout: 360
> >>> test_reference_resolver_extra: !!python/dict
> >>
> >> This is supposed to be "test_reference_resolver_args" (you're modifying
> >> the `self.args` of Avocado for the purpose of this loader. Note this
> only
> >> affects things inside the loader, so you can't for example change the
> >> job-results-dir or so...)
> >>
> >>> vt_extra_params:
> >>> - mem=2048
> >>> - nic_model=rtl8139
> >>> - drive_format=ahci
> >>> - disk_bus=ahci
> >>> - extra_params=" -cpu host"
> >>> vt_machine_type: q35
> >>>
> >>> shutdown:
> >>> test_reference: io-github-autotest-qemu.shutdown
> >>> test_reference_resolver_extra: !!python/dict
> >>> avocado_vt_extra_params:
> >>> - shutdown_method = shell
> >>>
> >>> reboot:
> >>> test_reference: io-github-autotest-qemu.reboot
> >>> test_reference_resolver_extra: !!python/dict
> >>> avocado_vt_extra_params:
> >>> - param1 = 1
> >>> - param2 = 2
> >>>
> >>> stress:
> >>> test_reference: io-github-autotest-qemu.linux_stress
> >>>
> >>> iofuzz:
> >>> test_reference: io-github-autotest-qemu.iofuzz
> >>> test_reference_resolver_extra: !!python/dict
> >>> avocado_vt_extra_params:
> >>> - param1 = 1
> >>> - param2 = 2
> >>>
> >>> ~$ avocado variants -m testplans/basic.yaml
> >>> Multiplex variants (4):
> >>> Variant shutdown-edec: /run/basic/shutdown
> >>> Variant reboot-c8ba: /run/basic/reboot
> >>> Variant stress-fd6a: /run/basic/stress
> >>> Variant iofuzz-e8bc: /run/basic/iofuzz
> >>> ~$
> >>>
> >>> But it is still not accepted by 'avocado {list, run}':
> >>> ~$ avocado list testplans/basic.yaml
> >>> Unable to resolve reference(s) 'testplans/basic.yaml' with plugins(s)
> >>> 'file', 'vt', 'external', try running 'avocado list -V
> >>> testplans/basic.yaml' to see the details.
> >>>
> >>> Do you have any idea why it may happen?
> >>
> >> With that single change it works well for me (tried changing the
> >> "test_reference_resolver_args" and first test's
> >> "test_reference_resolver_extra" and it successfully propagated to test
> >> params).
> >>
> >
> >
> >
> >> Regards,
> >> Lukáš
> >>
> >>>> autotest==0.16.4
> >>> avocado-framework==63.0
> >>> avocado-framework-plugin-varianter-yaml-to-mux==63.0
> >>> avocado-plugins-vt==63.0
> >>>
> >>> Sergey
> >>>
> >>>
> >>>>
> >>>> Regards,
> >>>> Lukáš
> >>>>
> >>>> PS: If this happens to work for you, would you please write an
> >> Avocado-vt
> >>>> documentation about this? I'd prefer someone external doing so because
> >>>> (apparently) I take many things as given... (you might as well improve
> >> the
> >>>> Avocado documentation, but looking at this I think the Avocado-vt one
> is
> >>>> missing more)
> >>>>
> >>>>>
> >>>>> $ avocado run -m basic.yaml
> >>>>> No test references provided nor any other arguments resolved into
> >> tests.
> >>>>> Please double check the executed command.
> >>>>>
> >>>>>
> >>>>> Sergey
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>>>
> >>>>> чт, 2 авг. 2018 г. в 17:08, Lukáš Doktor <ldok...@redhat.com>:
> >>>>>
> >>>>>> Dne 1.8.2018 v 13:19 Sergey Bronnikov napsal(a):
> >>>>>>> Hello!
> >>>>>>>
> >>>>>>> I'm trying to find a way to organize my tests for running without
> >>>>>>> specifying all of them in command-line. Imagine we have a set of
> >>>>>> different
> >>>>>>> tests: sleep.py, passtest.py and fail.py. To run all of them with
> >>>>>> required
> >>>>>>> parameters (aka variants) we may execute a command:
> >>>>>>> avocado run sleep.py fail.py passtest.py -m basic.yaml
> >>>>>>>
> >>>>>>> Is there a way to run the same tests without explicit specifying
> >> them?
> >>>>>>> For example: avocado run -m basic.yaml
> >>>>>>>
> >>>>>>> Sergey
> >>>>>>>
> >>>>>>
> >>>>>> Dear Sergey,
> >>>>>>
> >>>>>> as a matter of fact, there are multiple ways.
> >>>>>>
> >>>>>> 1. filter-by-tag:
> >>>>>>
> >>>>
> >>
> http://avocado-framework.readthedocs.io/en/latest/WritingTests.html#categorizing-tests
> >>>>>>
> >>>>>> you can tag your tests eg. `sanity`, `release`, ... and then run
> >>>>>> `avocado run --filter-by-tag=sanity -- TEST_REPO` to only run tests
> >>>> tagged
> >>>>>> `sanity` (note the granularity is per-class or even per-test-method.
> >>>>>>
> >>>>>> 2. yaml loader optional plugin:
> >>>>>>
> >>>>
> >>
> http://avocado-framework.readthedocs.io/en/latest/optional_plugins/yaml_loader.html
> >>>>>>
> >>>>>> which is probably more what you're looking for, but currently
> only
> >>>>>> works on python2 (but should get py3 support, hopefully, soon). It
> >> works
> >>>>>> similarly to yaml_to_mux varianter plugin, but allows special keys
> >> like
> >>>>>> `test_reference` to specify what test will be used or even
> >>>>>> `test_reference_resolver_class` when your tests are not the default
> >>>> ones.
> >>>>>> This is for example useful when you want to include multiple
> >>>>>> external-runner tests with different external-runner. There are
> >> examples
> >>>>>> in:
> >>>>>>
> >>>>
> >>
> https://github.com/avocado-framework/avocado/tree/master/examples/yaml_to_mux_loader
> >>>>>> but let me share my slighly advanced example here:
> >>>>>>
> >>>>>> ```yaml
> >>>>>> # Generated s390x mux-suite
> >>>>>> !mux
> >>>>>> qemu_make: !mux
> >>>>>> mux_suite_test_name_prefix: 'qemu_make/git '
> >>>>>> check:
> >>>>>> test_reference:
> >>>>>> /home/jenkins/s390x/qemu-master/avocado-qemu-make-check
> >>>>>> install:
> >>>>>> test_reference:
> >>>>>> /home/jenkins/s390x/qemu-master/avocado-qemu-make-install
> >>>>>>
> >>>>>> unit:
> >>>>>> variants: !mux
> >>>>>> test_reference_resolver_class:
> >>>> avocado.core.loader.ExternalLoader
> >>>>>> timeout: 360
> >>>>>> rpm-tcg:
> >>>>>> mux_suite_test_name_prefix: 'unit/rpm/tcg '
> >>>>>> test_reference_resolver_extra: !!python/dict
> >>>>>> loader_options:
> >>>>>> /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-tcg
> >>>>>> rpm-kvm:
> >>>>>> mux_suite_test_name_prefix: 'unit/rpm/kvm '
> >>>>>> test_reference_resolver_extra: !!python/dict
> >>>>>> loader_options:
> >>>>>> /home/jenkins/s390x/kvm-unit-test/avocado-runner-rpm-kvm
> >>>>>> git-tcg:
> >>>>>> mux_suite_test_name_prefix: 'unit/git/tcg '
> >>>>>> test_reference_resolver_extra: !!python/dict
> >>>>>> loader_options:
> >>>>>> /home/jenkins/s390x/kvm-unit-test/avocado-runner-git-tcg
> >>>>>> git-kvm:
> >>>>>> mux_suite_test_name_prefix: 'unit/git/kvm '
> >>>>>> test_reference_resolver_extra: !!python/dict
> >>>>>> loader_options:
> >>>>>> /home/jenkins/s390x/kvm-unit-test/avocado-runner-git-kvm
> >>>>>> tests: !mux
> >>>>>> cmm:
> >>>>>> test_reference: cmm
> >>>>>> diag10:
> >>>>>> test_reference: diag10
> >>>>>> emulator:
> >>>>>> test_reference: emulator
> >>>>>> gs:
> >>>>>> test_reference: gs
> >>>>>> iep:
> >>>>>> test_reference: iep
> >>>>>> intercept:
> >>>>>> test_reference: intercept
> >>>>>> pfmf:
> >>>>>> test_reference: pfmf
> >>>>>> selftest-setup:
> >>>>>> test_reference: selftest-setup
> >>>>>> sieve:
> >>>>>> test_reference: sieve
> >>>>>> skey:
> >>>>>> test_reference: skey
> >>>>>> sthyi:
> >>>>>> test_reference: sthyi
> >>>>>> vector:
> >>>>>> test_reference: vector
> >>>>>> functional: !mux
> >>>>>> test_reference_resolver_class: avocado_vt.loader.VirtTestLoader
> >>>>>> s390x-rpm-libvirt/dvd:
> >>>>>> mux_suite_test_name_prefix: 'functional/rpm/libvirt/dvd '
> >>>>>> test_reference:
> >>>>>> unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads
> >>>>>> io-github-autotest-qemu.boot remove_guest.without_disk
> >>>>>> test_reference_resolver_args: !!python/dict
> >>>>>> vt_qemu_bin: /usr/libexec/qemu-kvm
> >>>>>> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> >>>>>> vt_type: libvirt
> >>>>>> vt_extra_params:
> >>>>>> - automem=no
> >>>>>> - url=$URL
> >>>>>> - vga=none
> >>>>>> s390x-rpm-libvirt/url:
> >>>>>> mux_suite_test_name_prefix: 'functional/rpm/libvirt/url '
> >>>>>> test_reference:
> >>>>>> unattended_install.url.extra_cdrom_ks.default_install.aio_threads
> >>>>>> io-github-autotest-qemu.boot remove_guest.without_disk
> >>>>>> test_reference_resolver_args: !!python/dict
> >>>>>> vt_qemu_bin: /usr/libexec/qemu-kvm
> >>>>>> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> >>>>>> vt_type: libvirt
> >>>>>> vt_extra_params:
> >>>>>> - automem=no
> >>>>>> - url=$URL
> >>>>>> - vga=none
> >>>>>> s390x-rpm-dvd:
> >>>>>> mux_suite_test_name_prefix: 'functional/rpm/dvd '
> >>>>>> test_reference:
> >>>>>> unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads
> >> boot
> >>>>>> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
> >>>>>> migrate.default.tcp.with_post_copy
> >>>>>> test_reference_resolver_args: !!python/dict
> >>>>>> vt_qemu_bin: /usr/libexec/qemu-kvm
> >>>>>> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> >>>>>> vt_extra_params:
> >>>>>> - automem=no
> >>>>>> - url=$URL
> >>>>>> - ping_pong=5
> >>>>>> - vga=none
> >>>>>> s390x-rpm-url:
> >>>>>> mux_suite_test_name_prefix: 'functional/rpm/url '
> >>>>>> test_reference:
> >>>>>> unattended_install.url.extra_cdrom_ks.default_install.aio_threads
> boot
> >>>>>> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
> >>>>>> migrate.default.tcp.with_post_copy
> >>>>>> test_reference_resolver_args: !!python/dict
> >>>>>> vt_qemu_bin: /usr/libexec/qemu-kvm
> >>>>>> vt_dst_qemu_bin: /usr/libexec/qemu-kvm
> >>>>>> vt_extra_params:
> >>>>>> - automem=no
> >>>>>> - url=$URL
> >>>>>> - ping_pong=5
> >>>>>> - vga=none
> >>>>>> s390x-git-dvd:
> >>>>>> mux_suite_test_name_prefix: 'functional/git/dvd '
> >>>>>> test_reference:
> >>>>>> unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads
> >> boot
> >>>>>> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
> >>>>>> migrate.default.tcp.with_post_copy
> >>>>>> test_reference_resolver_args: !!python/dict
> >>>>>> vt_qemu_bin:
> >>>>>> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
> >>>>>> vt_dst_qemu_bin:
> >>>>>> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
> >>>>>> vt_extra_params:
> >>>>>> - automem=no
> >>>>>> - url=$URL
> >>>>>> - ping_pong=5
> >>>>>> - vga=none
> >>>>>> s390x-git-url:
> >>>>>> mux_suite_test_name_prefix: 'functional/git/url '
> >>>>>> test_reference:
> >>>>>> unattended_install.url.extra_cdrom_ks.default_install.aio_threads
> boot
> >>>>>> migrate.with_reboot.tcp migrate.with_reboot.exec.gzip_exec
> >>>>>> migrate.default.tcp.with_post_copy
> >>>>>> test_reference_resolver_args: !!python/dict
> >>>>>> vt_qemu_bin:
> >>>>>> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
> >>>>>> vt_dst_qemu_bin:
> >>>>>> /home/jenkins/s390x/qemu-master/build/bin/qemu-system-s390x
> >>>>>> vt_extra_params:
> >>>>>> - automem=no
> >>>>>> - url=$URL
> >>>>>> - ping_pong=5
> >>>>>> - vga=none
> >>>>>> ```
> >>>>>>
> >>>>>> (feel free to ask for details, but basically it uses SIMPLE tests,
> >>>>>> EXTERNAL_RUNNER tests and then Avocado-vt tests. The test always
> >>>> receives
> >>>>>> all the params so you can override the standard arguments. If you
> >>>> specify
> >>>>>> "test_reference_resolver_args" it also updates the loader
> parameters,
> >>>> which
> >>>>>> is what I use for Avocado-vt tests).
> >>>>>>
> >>>>>> Theoretically there is yet another way and that is to create
> directory
> >>>>>> "sanity" and put symlinks to actual tests there, but the granularity
> >>>> there
> >>>>>> would be only per-test-class, which usually does not suffice.
> >>>>>>
> >>>>>> Anyway, hopefully at least one method suits you, but feel free to
> >>>>>> elaborate more, we can always improve.
> >>>>>>
> >>>>>> Kind regards,
> >>>>>> Lukáš
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >
>
>
>
