2010/7/30 Graeme Gregory <[email protected]> > On 30/07/10 09:48, Frans Meulenbroeks wrote: > > 2010/7/30 Graeme Gregory <[email protected]> > > > >> On 30/07/10 09:22, Koen Kooi wrote: > >>> On 30-07-10 09:21, Esben Haabendal wrote: > >>>> On Thu, Jul 29, 2010 at 2:07 PM, Philip Balister > >>> <[email protected]> wrote: > >>>>> > >>>>> On 07/29/2010 05:45 AM, Koen Kooi wrote: > >>>>>> -----BEGIN PGP SIGNED MESSAGE----- > >>>>>> Hash: SHA1 > >>>>>> > >>>>>> On 29-07-10 10:50, Frans Meulenbroeks wrote: > >>>>>>> Dear all, > >>>>>>> > >>>>>>> Given the discussions on quality that sometimes pop up (and also > >>>>>>> triggered > >>>>>>> by Robert's message), I decided to kick off a bitbake -k world. > >>>>>> Could you first explain to me why 'bitbake world' is a good way to > >>>>>> measure quality? > >>>>>> > >>>>>> I would think that building something like console-image and > >>> looking at > >>>>>> the following would be a much better metric: > >>>>>> > >>>>>> * does it build? > >>>>>> * are all the rootfs types working? > >>>>>> * does the image do what it is supposed to do? > >>>>>> * Are all the licenses of the output packages correct? > >>>>>> * Do the output packages have any spurious deps? > >>>>>> * Is the content of the output packages correct? > >>>>>> * Are there any known CVEs in the resulting packages? > >>>>>> * Did packaged-staging do its job? > >>>>>> * What kind of QA errors and warnings were raised? > >>>>>> * Did all recipes pass recipe_sanity? > >>>>>> * Did all recipes conform to oe-stylize.py? > >>>>>> > >>>>>> etc > >>>>>> > >>>>>> I would actually advocate removing the 'world' feature from > >> bitbake/OE > >>>>>> to stop people from wasting time on looking at bitbake world and > have > >>>>>> them fix actual problems. > >>>>> bitbake world seems to be the source of pointless listserv > >>> discussions. Does > >>>>> it serve any purpose? > >>>> Pointless or not really depends on how you look at quality. > >>>> If you look at it as you, Koen and other OE long-timers, yes, it looks > >>>> rather pointless to have bitbake world. > >>>> But for those of us who have a different view on what quality is, then > >>>> bitbake world serves a purpose. > >>> As Thomas points out, as soon as you start blacklisting things (which > >>> actually increases quality), bitbake world doesn't work anymore. > >>> That alone should be enough to kill it. > >> Time to jump in the cage here. > >> > >> "Quality" is achieved by comparing a set of known specifications against > >> a known data set. In the software case this means we need a good set of > >> specifications which we are testing against. We also need to know in > >> detail what we are testing against this set of specifications. > >> > >> bitbake world meets neither of these criteria. You have no idea what > >> your testing against, you also have no idea what you are submitting for > >> test. A random selection of 8500 files in OE. It also doesn't take into > >> account known combinations which always fail. Angstrom took care of > >> these known failures by creating the concept of a blacklist. > >> > >> In fact if you tell me bitbake world fails, I would actually suggest > >> that is a test pass, it is an expected fail. > >> > >> Graeme > >> > >> Actually I expect it to fail, but I am eager to learn which recipes > fail. > > And yes, it might well be that this is not a good test. If you have > > suggestions for a better test that tests all our recipes feel free to > speak > > up. > > > > The dependency analysis of bitbake world already showed that there are > > several issues (as listed above). > > E.g. there are apparently dependencies on non existing recipes. > > Instead of shooting the messenger and killing the tool it would be better > > to resolve these (by either fixing them as I have seen JaMa did for some > > recipes in the past) or by removing the recipes. > > Keeping them around as junk does not improve things. > > > > And failure of bitbake -k world is not necessarily bad. At least it will > > give some insight in the weak spots. > > > More interesting to me would be an algorithm more like. > > while recipe in recips/*.bb; do > rm -rf tmp/ ; bitbake recipe > done > > with the test noting if the fail is due to a blacklisted dependency, a > totally missing dependency (recipe been deleted sometime in past) or an > actual compile fail due to a missing DEPENDS entry somewhere in the tree. > > But this is going to burn CPU time like never before. > > Graeme > > Graeme,
Definitely an interesting test. Actually with packaged staging this might become manageble. I would not mind running such a test, even if it takes a week or so. I guess it is hard to do this for older versions of recipes though. afaik bitbake -b will not build things one depends on and bitbake recipe will take either the pinned version or otherwise the last one a non-negative DEFAULT_PREFERENCE. Frans. > > _______________________________________________ > Openembedded-devel mailing list > [email protected] > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel > _______________________________________________ Openembedded-devel mailing list [email protected] http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
