On Thu, 2013-10-31 at 10:34 +0000, Paul Eggleton wrote: > Hi Corneliu, > > This is a well-rounded proposal, thanks. Some comments below. > > On Thursday 31 October 2013 07:53:48 Stoicescu, CorneliuX wrote: > > This e-mail was originally sent to the Yocto mailing list in the form of 2 > > e-mails. As per Paul's and Richard's request, I am re-sending and moving > > the conversation here. Feel free to add any input :) . > > > > NOTE: I also made a small syntax correction in the example at the end of the > > e-mail for oeSelfTest.var_append() . > > > > After a chat with Richard and Stefan, I came up with an outline of how the > > oe-selftest feature(https://bugzilla.yoctoproject.org/show_bug.cgi?id=4740 > > ) should look like. I made a summary of my proposal below. Please feel free > > to add your thoughts! > > > > There will be a new script introduced(similar to bitbake-selftest) that will > > use python unit test to execute tests. Name has not been decided but it can > > be "oe-selftest". Running this script will not compromise poky in any way. > > Initially, the script does not need any preparation in order to be run. If > > this changes in the future, the user will be prompted upon execution with > > the pre-required tasks. Oe-selftest can be used together with the automated > > runtime tests if necessary. > > > > The following types of tests are targeted for the initial implementation: > > - testing the functionality of scripts in poky/scripts (such as: > > bitbake-layers, yocto-bsp, yocto-kernel, yocto-layers) - testing of the > > 'bitbake' command and its output (this includes output data validation such > > as the sstate-cache/ and tmp/ directories) > > It could be considered a separate exercise, but I'd like us to test the > installation and usage of the SDK installer as well. Initially this would be > fairly straightforward - install the SDK to a non-standard location, fetch > some nominated piece of source code and try to build it using the installed > SDK. (One issue springs to mind though - unless we take steps to guard > against > it, this won't be able to pick up problems where the SDK contains references > to files within TMPDIR, since those will still be valid on the build host). > > Later we'd want to be able to test the executables produced using the SDK on > the target; however that would presumably necessitate some integration with > the runtime tests and that could be complicated.
We could test some simple binary using user mode qemu... > Actually I think we should avoid modifying existing files if at all possible > - > instead we should add an additional layer on top to make changes, using > bbappends / overlayed recipes as necessary. There are several reasons for > this: > > * Most of the time this is the approach users should be using when they make > customisations, so it's what we ought to concentrate on testing. > > * It preserves the ability to run the tests with uncommitted changes, which > would be useful during development. Agreed, this is probably something we need to support. > I know the test case mentions it explicitly, but we don't actually need meta- > intel to check this functionality, any layer will work. We probably ought to > be creating a meta-selftest layer (or similar) within OE-Core that we could > use for this purpose and the addition of bbappends / additional > configuration. > This avoids the need for something like POKYDIR as well. Yes, having some kind of specific test layer in OE-Core would seem appropriate. I read through the proposal and it seems good to me, I know I had already given some feedback as the details were filled out but it seems the pieces I believe we need are all there. Cheers, Richard _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
