On 30/07/10 10:07, Frans Meulenbroeks wrote: > 2010/7/30 Graeme Gregory <[email protected]> > >> On 30/07/10 09:48, Frans Meulenbroeks wrote: >>> 2010/7/30 Graeme Gregory <[email protected]> >>> >>>> On 30/07/10 09:22, Koen Kooi wrote: >>>>> On 30-07-10 09:21, Esben Haabendal wrote: >>>>>> On Thu, Jul 29, 2010 at 2:07 PM, Philip Balister >>>>> <[email protected]> wrote: >>>>>>> On 07/29/2010 05:45 AM, Koen Kooi wrote: >>>>>>>> -----BEGIN PGP SIGNED MESSAGE----- >>>>>>>> Hash: SHA1 >>>>>>>> >>>>>>>> On 29-07-10 10:50, Frans Meulenbroeks wrote: >>>>>>>>> Dear all, >>>>>>>>> >>>>>>>>> Given the discussions on quality that sometimes pop up (and also >>>>>>>>> triggered >>>>>>>>> by Robert's message), I decided to kick off a bitbake -k world. >>>>>>>> Could you first explain to me why 'bitbake world' is a good way to >>>>>>>> measure quality? >>>>>>>> >>>>>>>> I would think that building something like console-image and >>>>> looking at >>>>>>>> the following would be a much better metric: >>>>>>>> >>>>>>>> * does it build? >>>>>>>> * are all the rootfs types working? >>>>>>>> * does the image do what it is supposed to do? >>>>>>>> * Are all the licenses of the output packages correct? >>>>>>>> * Do the output packages have any spurious deps? >>>>>>>> * Is the content of the output packages correct? >>>>>>>> * Are there any known CVEs in the resulting packages? >>>>>>>> * Did packaged-staging do its job? >>>>>>>> * What kind of QA errors and warnings were raised? >>>>>>>> * Did all recipes pass recipe_sanity? >>>>>>>> * Did all recipes conform to oe-stylize.py? >>>>>>>> >>>>>>>> etc >>>>>>>> >>>>>>>> I would actually advocate removing the 'world' feature from >>>> bitbake/OE >>>>>>>> to stop people from wasting time on looking at bitbake world and >> have >>>>>>>> them fix actual problems. >>>>>>> bitbake world seems to be the source of pointless listserv >>>>> discussions. Does >>>>>>> it serve any purpose? >>>>>> Pointless or not really depends on how you look at quality. >>>>>> If you look at it as you, Koen and other OE long-timers, yes, it looks >>>>>> rather pointless to have bitbake world. >>>>>> But for those of us who have a different view on what quality is, then >>>>>> bitbake world serves a purpose. >>>>> As Thomas points out, as soon as you start blacklisting things (which >>>>> actually increases quality), bitbake world doesn't work anymore. >>>>> That alone should be enough to kill it. >>>> Time to jump in the cage here. >>>> >>>> "Quality" is achieved by comparing a set of known specifications against >>>> a known data set. In the software case this means we need a good set of >>>> specifications which we are testing against. We also need to know in >>>> detail what we are testing against this set of specifications. >>>> >>>> bitbake world meets neither of these criteria. You have no idea what >>>> your testing against, you also have no idea what you are submitting for >>>> test. A random selection of 8500 files in OE. It also doesn't take into >>>> account known combinations which always fail. Angstrom took care of >>>> these known failures by creating the concept of a blacklist. >>>> >>>> In fact if you tell me bitbake world fails, I would actually suggest >>>> that is a test pass, it is an expected fail. >>>> >>>> Graeme >>>> >>>> Actually I expect it to fail, but I am eager to learn which recipes >> fail. >>> And yes, it might well be that this is not a good test. If you have >>> suggestions for a better test that tests all our recipes feel free to >> speak >>> up. >>> >>> The dependency analysis of bitbake world already showed that there are >>> several issues (as listed above). >>> E.g. there are apparently dependencies on non existing recipes. >>> Instead of shooting the messenger and killing the tool it would be better >>> to resolve these (by either fixing them as I have seen JaMa did for some >>> recipes in the past) or by removing the recipes. >>> Keeping them around as junk does not improve things. >>> >>> And failure of bitbake -k world is not necessarily bad. At least it will >>> give some insight in the weak spots. >>> >> More interesting to me would be an algorithm more like. >> >> while recipe in recips/*.bb; do >> rm -rf tmp/ ; bitbake recipe >> done >> >> with the test noting if the fail is due to a blacklisted dependency, a >> totally missing dependency (recipe been deleted sometime in past) or an >> actual compile fail due to a missing DEPENDS entry somewhere in the tree. >> >> But this is going to burn CPU time like never before. >> >> Graeme >> >> > Graeme, > > Definitely an interesting test. Actually with packaged staging this might > become manageble. > I would not mind running such a test, even if it takes a week or so. > > I guess it is hard to do this for older versions of recipes though. > afaik bitbake -b will not build things one depends on and bitbake recipe > will take either the pinned version or otherwise the last one a non-negative > DEFAULT_PREFERENCE. > > Frans. > bitbake blah-x.x.x should work to build exact versions with dependencies though.
G _______________________________________________ Openembedded-devel mailing list [email protected] http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
