Re: [Xen-devel] [XenSummit 2017] Build tools follow up
ping On 07/17/2017 03:47 PM, Oleksandr Andrushchenko wrote: Hi, all! This is a follow-up on Xen distribution build systems we saw at the summit and invitation for sharing thoughts and ways we build our images and distros. I would like to specifically ask OpenXT project to reply with the description of their build system so we have clear picture of what is being developed and used around. And of course if there are other approaches to do the same you are welcome to share those as well. On our side at EPAM we have developed a distribution which is called xt-distro (xt stays for Xen troops [0]) with the following goals in mind: 1. Make it possible to easily build different images as a single distribution - Separate images for different domains should be combined into a distribution, e.g. set of images/artifacts required to run them as a system - Xt-distro allows doing so as easy as running bitbake with the predefined target, e.g. “bitbake xt-image” 2. Easily deal with different BSPs for different platforms using a unified way with little or no EPAM specific code/scripts - We use bitbake and stripped version of Poky/meta-openembedded, so there are little in-house extensions we added to deal with Yocto based BSPs [1], [2] - We use bitbake’s well defined scripting language [3], so no yet another language to learn - We use Google repo tool to maintain sets of meta layers as manifest files [4] 3. Ability to easily collect component revisions, so any build can be reproduced if need be - We save commit IDs of every component of the build including versions of the meta layers 4. Ability to easily customize and tune builds, e.g. URIs, versions/commitIDs, branches used - This is purely done in bitbake’s recipe language 5. Make patching process easy - With our extensions you can use bblayers.conf, local.conf, patches which are part of a meta layer which is usually a manual step 6. Code reuse is a must, e.g. the same set of software must be easily built for different platforms without copying build scripts of the existing components - With bitbake’s meta layers we define generic recipes, e.g. suitable for all platforms [5] (think of it as a library) and tuning meta layers [6] (we use product concept here) which define specific versions/revisions of the components, apply specific patches and add/remove software 7. Cross compilation must be an easy task to do - This is easily achieved with Yocto builds and usually not a problem for other build systems as well (with SDKs) - Allows building for x86/ARM and other architectures as well 8. There must be a simple way to share build artifacts of different domains between each other, for example, DomU’s kernel should be a part of Dom0’s rootfs, so xl/libxl can access it - This is achieved with bitbake’s recipes, so no EPAM extensions 9. Possibility to easily use different build systems for different components of the system. - Bitbake allows you building makefile based projects, CMake etc. out of the box, so adding firmware, stubdoms made easy 10. Development support - We use SDKs built by bitbake so during everyday development you are packed with all that is needed to re-create build environment (with both host and target environments, e.g. x86 host tools and ARM cross-compiler) - Speed up builds for developers within organization, e.g. we reuse downloads and build cache between all of the developer’s machines on our network 11. Make the build system suitable for build automation - We have created a python script which can easily be used with Jenkins or even cron [7] For example, with xt-distro our development product delivers: Xen image and policy file, Dom0 kernel and dtb images (Dom0 + DomU), Dom0 rootfs image (with embedded kernels and configuration for DomU), DomU rootfs image. This effectively means, that the build system can provide you with the complete set of images to run your product/distribution. We are looking forward for any kind of feedback and will be glad to collaborate on the above. Thank you, Xen-troops at EPAM [0] https://github.com/xen-troops/xt-distro [1] https://git.yoctoproject.org/cgit/cgit.cgi/poky [2] http://cgit.openembedded.org/meta-openembedded [3] http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual.html [4] https://gerrit.googlesource.com/git-repo [5] https://github.com/xen-troops/meta-xt-images [6] https://github.com/xen-troops/meta-xt-prod-devel [7] https://github.com/xen-troops/build-scripts ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
[Xen-devel] [XenSummit 2017] Build tools follow up
Hi, all! This is a follow-up on Xen distribution build systems we saw at the summit and invitation for sharing thoughts and ways we build our images and distros. I would like to specifically ask OpenXT project to reply with the description of their build system so we have clear picture of what is being developed and used around. And of course if there are other approaches to do the same you are welcome to share those as well. On our side at EPAM we have developed a distribution which is called xt-distro (xt stays for Xen troops [0]) with the following goals in mind: 1. Make it possible to easily build different images as a single distribution - Separate images for different domains should be combined into a distribution, e.g. set of images/artifacts required to run them as a system - Xt-distro allows doing so as easy as running bitbake with the predefined target, e.g. “bitbake xt-image” 2. Easily deal with different BSPs for different platforms using a unified way with little or no EPAM specific code/scripts - We use bitbake and stripped version of Poky/meta-openembedded, so there are little in-house extensions we added to deal with Yocto based BSPs [1], [2] - We use bitbake’s well defined scripting language [3], so no yet another language to learn - We use Google repo tool to maintain sets of meta layers as manifest files [4] 3. Ability to easily collect component revisions, so any build can be reproduced if need be - We save commit IDs of every component of the build including versions of the meta layers 4. Ability to easily customize and tune builds, e.g. URIs, versions/commitIDs, branches used - This is purely done in bitbake’s recipe language 5. Make patching process easy - With our extensions you can use bblayers.conf, local.conf, patches which are part of a meta layer which is usually a manual step 6. Code reuse is a must, e.g. the same set of software must be easily built for different platforms without copying build scripts of the existing components - With bitbake’s meta layers we define generic recipes, e.g. suitable for all platforms [5] (think of it as a library) and tuning meta layers [6] (we use product concept here) which define specific versions/revisions of the components, apply specific patches and add/remove software 7. Cross compilation must be an easy task to do - This is easily achieved with Yocto builds and usually not a problem for other build systems as well (with SDKs) - Allows building for x86/ARM and other architectures as well 8. There must be a simple way to share build artifacts of different domains between each other, for example, DomU’s kernel should be a part of Dom0’s rootfs, so xl/libxl can access it - This is achieved with bitbake’s recipes, so no EPAM extensions 9. Possibility to easily use different build systems for different components of the system. - Bitbake allows you building makefile based projects, CMake etc. out of the box, so adding firmware, stubdoms made easy 10. Development support - We use SDKs built by bitbake so during everyday development you are packed with all that is needed to re-create build environment (with both host and target environments, e.g. x86 host tools and ARM cross-compiler) - Speed up builds for developers within organization, e.g. we reuse downloads and build cache between all of the developer’s machines on our network 11. Make the build system suitable for build automation - We have created a python script which can easily be used with Jenkins or even cron [7] For example, with xt-distro our development product delivers: Xen image and policy file, Dom0 kernel and dtb images (Dom0 + DomU), Dom0 rootfs image (with embedded kernels and configuration for DomU), DomU rootfs image. This effectively means, that the build system can provide you with the complete set of images to run your product/distribution. We are looking forward for any kind of feedback and will be glad to collaborate on the above. Thank you, Xen-troops at EPAM [0] https://github.com/xen-troops/xt-distro [1] https://git.yoctoproject.org/cgit/cgit.cgi/poky [2] http://cgit.openembedded.org/meta-openembedded [3] http://www.yoctoproject.org/docs/2.3/bitbake-user-manual/bitbake-user-manual.html [4] https://gerrit.googlesource.com/git-repo [5] https://github.com/xen-troops/meta-xt-images [6] https://github.com/xen-troops/meta-xt-prod-devel [7] https://github.com/xen-troops/build-scripts ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
