So let me summarize where I think things should be heading: 1. Any new sdk work should be aiming to adjust the 'core yocto workflow', e.g. a standard layer checkout, and not create or extend any separate workflows with their own input artifacts, tests, and code paths.
2. bblock/bbunlock as core, supported, testable and tested functionality, and not an ad hoc addition to esdk. 3. devtool sdk [recipe] drops you into an environment identical to that of the standard sdk, with source code for a component optionally set up for builds (using component-level toolchain as in the sdk). 4. Layer and environment config: until we have that, kas + TEMPLATECONF could be an interim way to set things up. The tool in the core project should be bi-directional, and be able to both capture the state of the configuration, and restore it from the captured state. Sstate can be captured/restored together with layers and configs, or it can be skipped with the assumption that it will be provided from a server. Thoughts? Anything about this that you think is unworkable, or something obvious I completely missed? Alex On Wed, 18 May 2022 at 09:50, Pascal Bach <[email protected]> wrote: > > Hi Richard, > > On Mon, 2022-05-09 at 15:30 +0100, Richard Purdie via > lists.openembedded.org wrote: > > I've been asked about my thoughts on the eSDK. I'm going to try and > > write down some of my ideas and thinking about it. I don't have a > > fully > > thought out plan to pull off the shelf but can at least share the > > ideas. > > > > In summary, I think many of the concepts in the eSDK are good but the > > implementation isn't great, it is effectively a beta where we needed > > to > > go back and learn from the implementation to improve things. > > > > To understand why, keep in mind it was put together as an experiment. > > The idea was to have an environment which rather than being an SDK or > > a > > full build system, could be configured to be different steps in a > > spectrum between those two positions. The goals included: > > > > * using prebuild binary objects rather than building from source > > * a prebuilt toolchain > > * being able to generate customised images quickly > > * could be updated to new versions of the underlying environment > > * making commands like devtool available to the "SDK" user > > * hiding the build system complexity from the "SDK" user where > > possible > > * allowing usage as per the current "standard SDK". > > I would like to give some input from an application developer > perspective here. By this I mean using the eSDK like a "standard SDK", > developing a single application working directly with CMake, in an IDE > like VSCode. > > For this use case it works well. What's missing from my perspective is > the following: > > 1. A way to update any dependency included in the eSDK and make it > available in the sysroots, so it can be consumed by CMake. This would > make it really easy for developers to test their change against a new > or patched version of a dependency. > 2. A reliable way to update the eSDK to avoid re-downloading again. > > > > The design decisions this resulted in were: > > > > * being able to lock the sstate hashes to a known version > > * using sstate as the binary object mechanism (so not package > > management based) > > * needing to capture a locked down version of the metadata (i.e. the > > layers in use) > > > > The eSDK as implemented did do many of the things it was designed to > > do. As with most engineering, there were challenges that were > > encountered, things that didn't work as expected and also > > unanticipated > > issues. It really needed someone to go back over it armed with that > > knowledge for a v2 and that is still where we stand today as > > development stopped. > > > > The things which bother me about our current situation: > > > > * there is no way to unlock the sstate hash of a specific component > > like a bblock/bbunlock command. > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.yoctoproject.org%2Fshow_bug.cgi%3Fid%3D13425&data=05%7C01%7Cpascal.bach%40siemens.com%7C238052c120044493583408da31c88936%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637877034611563685%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2Xu03DUqi7UCCVe40bg5QL1yigJUM84Z7pWiDAU60gQ%3D&reserved=0 > > * we need to work out the layer setup and capture it into the eSDK. > > If > > we had a standard mechanism for layer setup and config this would > > be > > much easier. The current approach is very sensitive to different > > layouts and lacks any tests for anything that isn't poky > > * where sstate checksum mismatches occur, they are fiendishly > > difficult to debug (a wider problem with bitbake but they affect > > eSDK particularly badly) > > * no mechanism to turn an existing build into an eSDK > > * no mechanism to turn an eSDK install into a full build > > * can't build eSDK within eSDK > > > > At this I would also throw a completley different idea into the ring > for discussion. It is to get rid of the eSDK in it's curent form. > > A bit of context: I have worked a lot with https://nixos.org/ recently > and I think it has some similarities to bitbake. The important > similarity here is that both are source based systems that can make use > of a cache to accelerate the build. > > In nix the way you develop an application (e.g via CMake) is that you > open the .nix file (similar to a .bb recipe) via nix-shell. This will > drop you in a shell where all your dependencies, build tools and > envioronment variables are available, ready to build your software. > > I think a similar approach could also be possible with bitbake/devtool > as it is pretty close to what we already have with the devshell. > > So a potential workflow for an "sdkenv" would look as follows: > > 1. Download a preconfigured bitbake environment with a bundeled sstate. > Could be a tarball, installer or container => This gives you a starting > point as an application. Alternative to bundeling sstate, it could also > be provided from a mirror on demand > 2. Run "devtool sdkenv <recipe>" => This assembles an environment based > on <recipes>s sysroot and drop you into a devshell like environment > where you can build your the <recipe> aplication, e.g via CMake. > 3. To make use of a new version of the dependency, just update the > <dependency-recipe> and then run "devtool sdkenv <recipe>" again. This > will build <dependency-recipe> install it in <recipe> sysroot and drop > you in a new devshell. Now you can continue to build your application > but with the new dependency. > > > The advantages of this approach is: > 1. The bitbake environment and the "sdkenv" are the same. The only > difference is how you use it (devtool or bitbake) and how much you > reley on the sstate cache. > 2. Developers always have the full power of bitbake available. So it's > easy to update dependencies, crate new sdk and even do completely new > builds. > 3. There is no need to crate a new eSDK installer as you can always > test it directly from within bitbake by calling devtool sdkenv <recipe> > 4. Updates are simple as it just means updating the recipes and > providing an sstate-mirror. > > What do you think? Is this feasible or even desired? > > > Pascal > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1551): https://lists.openembedded.org/g/openembedded-architecture/message/1551 Mute This Topic: https://lists.openembedded.org/mt/90990557/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
