I really appreciate the new effort for updating the tests and improving the testing situation for Toaster. As one of the few people that have been contributing in the last several years, it gives some new hope.
However, we need to pause long enough to write out some basic instructions for how to run the new/existing tests. From a Python UI developer perspective, using tox might make a lot of sense. So, in the bitbake environment in which Toaster must be run, how are you using tox? Is it bringing us benefit over just running the test tools themselves? Are we certain we want the opinionated results from flake8? Does anything else in bitbake pass those tests? The only real documentation we currently have for running the Selenium tests is: https://git.openembedded.org/bitbake/tree/lib/toaster/tests/browser/README Since this tells us to `pip3 install selenium==2.53.2` we already know it cannot possibly be what is being done to run the tests. So what else is wrong in those steps? To make it possible for folks to review the new contributions, we need to pause and at least write out a simple list of the history of commands run on the command line. Otherwise all the new patches are taken on faith with no logs, no test reproducibility, no evidence that they are functional. This is a precarious position. The only testing that has been run on a regular basis for some time now is from https://github.com/crops/toaster-container, and this has caught breakage in bitbake/oe-core several times (usually after the fact). It's not perfect, but anyone should be able to replicate the tests. Likewise, anyone should be able to use the current https://git.openembedded.org/bitbake/tree/lib/toaster content to replicate the new test patches. Regards, --Tim
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#5880): https://lists.yoctoproject.org/g/toaster/message/5880 Mute This Topic: https://lists.yoctoproject.org/mt/102049394/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/toaster/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
