Hi Richard, On Mon, 2022-05-09 at 15:30 +0100, Richard Purdie via lists.openembedded.org wrote: > I've been asked about my thoughts on the eSDK. I'm going to try and > write down some of my ideas and thinking about it. I don't have a > fully > thought out plan to pull off the shelf but can at least share the > ideas. > > In summary, I think many of the concepts in the eSDK are good but the > implementation isn't great, it is effectively a beta where we needed > to > go back and learn from the implementation to improve things. > > To understand why, keep in mind it was put together as an experiment. > The idea was to have an environment which rather than being an SDK or > a > full build system, could be configured to be different steps in a > spectrum between those two positions. The goals included: > > * using prebuild binary objects rather than building from source > * a prebuilt toolchain > * being able to generate customised images quickly > * could be updated to new versions of the underlying environment > * making commands like devtool available to the "SDK" user > * hiding the build system complexity from the "SDK" user where > possible > * allowing usage as per the current "standard SDK".
I would like to give some input from an application developer perspective here. By this I mean using the eSDK like a "standard SDK", developing a single application working directly with CMake, in an IDE like VSCode. For this use case it works well. What's missing from my perspective is the following: 1. A way to update any dependency included in the eSDK and make it available in the sysroots, so it can be consumed by CMake. This would make it really easy for developers to test their change against a new or patched version of a dependency. 2. A reliable way to update the eSDK to avoid re-downloading again. > The design decisions this resulted in were: > > * being able to lock the sstate hashes to a known version > * using sstate as the binary object mechanism (so not package > management based) > * needing to capture a locked down version of the metadata (i.e. the > layers in use) > > The eSDK as implemented did do many of the things it was designed to > do. As with most engineering, there were challenges that were > encountered, things that didn't work as expected and also > unanticipated > issues. It really needed someone to go back over it armed with that > knowledge for a v2 and that is still where we stand today as > development stopped. > > The things which bother me about our current situation: > > * there is no way to unlock the sstate hash of a specific component > like a bblock/bbunlock command. > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.yoctoproject.org%2Fshow_bug.cgi%3Fid%3D13425&data=05%7C01%7Cpascal.bach%40siemens.com%7C238052c120044493583408da31c88936%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637877034611563685%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2Xu03DUqi7UCCVe40bg5QL1yigJUM84Z7pWiDAU60gQ%3D&reserved=0 > * we need to work out the layer setup and capture it into the eSDK. > If > we had a standard mechanism for layer setup and config this would > be > much easier. The current approach is very sensitive to different > layouts and lacks any tests for anything that isn't poky > * where sstate checksum mismatches occur, they are fiendishly > difficult to debug (a wider problem with bitbake but they affect > eSDK particularly badly) > * no mechanism to turn an existing build into an eSDK > * no mechanism to turn an eSDK install into a full build > * can't build eSDK within eSDK > At this I would also throw a completley different idea into the ring for discussion. It is to get rid of the eSDK in it's curent form. A bit of context: I have worked a lot with https://nixos.org/ recently and I think it has some similarities to bitbake. The important similarity here is that both are source based systems that can make use of a cache to accelerate the build. In nix the way you develop an application (e.g via CMake) is that you open the .nix file (similar to a .bb recipe) via nix-shell. This will drop you in a shell where all your dependencies, build tools and envioronment variables are available, ready to build your software. I think a similar approach could also be possible with bitbake/devtool as it is pretty close to what we already have with the devshell. So a potential workflow for an "sdkenv" would look as follows: 1. Download a preconfigured bitbake environment with a bundeled sstate. Could be a tarball, installer or container => This gives you a starting point as an application. Alternative to bundeling sstate, it could also be provided from a mirror on demand 2. Run "devtool sdkenv <recipe>" => This assembles an environment based on <recipes>s sysroot and drop you into a devshell like environment where you can build your the <recipe> aplication, e.g via CMake. 3. To make use of a new version of the dependency, just update the <dependency-recipe> and then run "devtool sdkenv <recipe>" again. This will build <dependency-recipe> install it in <recipe> sysroot and drop you in a new devshell. Now you can continue to build your application but with the new dependency. The advantages of this approach is: 1. The bitbake environment and the "sdkenv" are the same. The only difference is how you use it (devtool or bitbake) and how much you reley on the sstate cache. 2. Developers always have the full power of bitbake available. So it's easy to update dependencies, crate new sdk and even do completely new builds. 3. There is no need to crate a new eSDK installer as you can always test it directly from within bitbake by calling devtool sdkenv <recipe> 4. Updates are simple as it just means updating the recipes and providing an sstate-mirror. What do you think? Is this feasible or even desired? Pascal
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1548): https://lists.openembedded.org/g/openembedded-architecture/message/1548 Mute This Topic: https://lists.openembedded.org/mt/90990557/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
