Hi Richard > I've been asked about my thoughts on the eSDK. I'm going to try and > write down some of my ideas and thinking about it. I don't have a fully > thought out plan to pull off the shelf but can at least share the > ideas. > > In summary, I think many of the concepts in the eSDK are good but the > implementation isn't great, it is effectively a beta where we needed to > go back and learn from the implementation to improve things. > > To understand why, keep in mind it was put together as an experiment. > The idea was to have an environment which rather than being an SDK or a > full build system, could be configured to be different steps in a > spectrum between those two positions. The goals included: > > * using prebuild binary objects rather than building from source > * a prebuilt toolchain > * being able to generate customised images quickly > * could be updated to new versions of the underlying environment > * making commands like devtool available to the "SDK" user > * hiding the build system complexity from the "SDK" user where possible > * allowing usage as per the current "standard SDK".
I think the goals are still valid. > The design decisions this resulted in were: > > * being able to lock the sstate hashes to a known version > * using sstate as the binary object mechanism (so not package > management based) > * needing to capture a locked down version of the metadata (i.e. the > layers in use) > Also these principles in implementation continue to seem sensible to me. > The eSDK as implemented did do many of the things it was designed to > do. As with most engineering, there were challenges that were > encountered, things that didn't work as expected and also unanticipated > issues. It really needed someone to go back over it armed with that > knowledge for a v2 and that is still where we stand today as > development stopped. > > The things which bother me about our current situation: > > * there is no way to unlock the sstate hash of a specific component > like a bblock/bbunlock command. > https://bugzilla.yoctoproject.org/show_bug.cgi?id=13425 > * we need to work out the layer setup and capture it into the eSDK. If > we had a standard mechanism for layer setup and config this would be > much easier. The current approach is very sensitive to different > layouts and lacks any tests for anything that isn't poky > * where sstate checksum mismatches occur, they are fiendishly > difficult to debug (a wider problem with bitbake but they affect > eSDK particularly badly) > * no mechanism to turn an existing build into an eSDK > * no mechanism to turn an eSDK install into a full build > * can't build eSDK within eSDK > > You can see the list of open bugs against eSDK: > > https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=esdk&list_id=649733 > > In particular there are the following issues: > > * it only works targeting a system of the same as it is being built > on (an aarch64 eSDK needs to be built on an aarch64 system) > * multilib may or may not work (and has no tests) > https://bugzilla.yoctoproject.org/show_bug.cgi?id=10693 > * devtool commands may need to be disabled depending on context > https://bugzilla.yoctoproject.org/show_bug.cgi?id=8789 > * sdk-update has issues, e.g. > https://bugzilla.yoctoproject.org/show_bug.cgi?id=10798 > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14462 > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14462 > * several things don't build within the eSDK, e.g. ovmf, multilib, > lttng, gstreamer-plugins-good > > > I've put off fixing some elements as I've hoped that if we fixed the > layer setup problem, it would then help elements of the eSDK. I think > we probably do need to go ahead and try and fix other elements without > that piece for now though (layer setup is a different problem in it's > own right). > > I'm sharing this in the hopes it gets people thinking a little more and > can at least start the discussion. We have also spent some time trying different approaches to use the eSDK. There are many things which work pretty well. But there are also use cases where we were not successfully with using the eSDK so far. There are probably two main technical reasons why some application developers do not use it: * Speed: - devtool re-parses the recipes, which takes some time - devtool sometimes forces a clean build which would not be necessary. * IDE integration: VSCode, for example, allows devtool to be integrated via the tasks.json configuration file, as well as performing remote debugging, configured through a launch.json file. However, it is complicated and not on the level of e.g. a cmake plugin. I would imagine that the following improvements would greatly enhance the user experience of the eSDK and devtool: * devtool/bitbake runs in memory-resident mode. * devtool should offer an option to skip a "make clean" (On dunfell this gets called by the configure task e.g. for cmake but also other build frameworks). * A plugin for VSCode could significantly simplify the integration of devtool. Ideally, the plugin connects to the bitbake server via the Tinfoil API instead of manually including devtool via the tasks.json file. However, this is likely to be quite time-consuming to implement. Would there perhaps be synergies with Toaster here? The second big challenge we face is the amount of data we have to deal with. We are currently creating full installers per machine (various similar products) as the update implementation has not really worked for us (stability, security, download speed). If an SDK could support more than one MACHINE and more than one image, this would also be beneficial. We would really like to contribute to improve and fix the eSDK. I am pleased to see that this topic made it to the top of the agenda of https://www.openembedded.org/wiki/OEDVM_2022_05. I see different approaches how the eSDK could be improved: * Just fix the existing code and make it work as described in the documentation. * Discard the concept with the creation of an installer. I guess it is feasible to provide the functionality of the eSDK directly in the bitbake environment, which has access to a sstate-cache mirror. Not sure if that's the idea behind "if we fixed the layer setup problem, it would then help elements of the eSDK". * Providing the SDK as an OCI container might be beneficial as well. Ideally the container would be based on the poky distro and be generated by bitbake only. (Not like crops where the installer is installed into a Ubuntu container with additional tools processing a Dockerfile). This could simplify the deployment, make it more robust and decoupled from any host distribution. It could also be a basis for the integration with e.g. VSCode. Thank you very much for initiating this discussion and sharing your thoughts. Adrian > > Cheers, > > Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1547): https://lists.openembedded.org/g/openembedded-architecture/message/1547 Mute This Topic: https://lists.openembedded.org/mt/90990557/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
