On Sat, Jul 2, 2022 at 4:13 AM Richard Purdie <richard.pur...@linuxfoundation.org> wrote: > > I admit I've been putting off this topic for a long time. I actually > wrote up something about it about 5 years ago. I sent it to four people > to get some opinions. The six opinions I got back made me despair. i > have had a while to think about it though. > > The challenge is that everyone has a workflow they like today and they > will tend to dislike anything that looks different. For that reason I > think we need a high level tool which can work with the different > approaches. Once we have that, I suspect we'll see some grow stronger > and some will wither off, which is probably as it should be through > natural evolution. The approach Alex has taken here does head in that > direction but I'm not sure it goes quite far enough to get the fraction > of users we need on board. > > I'm not going to comment directly on Alex's proposal at this point, I'd > like to put some ideas out there and see what people think of them. I > think you'll see that I'm in agreement with the idea/direction but I > have a slightly bigger idea of how to do it, which will both be harder > to implement but also have a better chance of getting more people on > board. > > So, let me go into a proposal of what I think the tool we need looks > like. I propose we create a new tool, lets call it "oe-setup". It is a > standalone project in it's own git repo and it's first command would be > "init" with a command line looking roughly like: > > oe-setup init <project-name> > oe-setup init <project-name> <target-dir> > oe-setup init <project-name> <config-name> <target-dir> > oe-setup init <project-name> <config-url> <target-dir> > oe-setup init <config-url> <target-dir> > > The idea being that the repository has some "pre-canned" idea of > certain project names, e.g. poky, yoe and whatever else we decide to > support "out the box" (criteria tbd). Those project names have a > default pre-canned config, or set of configs (e.g. taking branch/series > names) so I'd want to see these all work:
My proposal would be to try to reuse Linux kconfig. Users know how to run "make oldconfig", "make menuconfig", etc and I believe all key distro / machine configuration along with which meta layers should be enabled, which branch to use, etc can all be captured within a kconfig .config file. The task at hand would then become writing a tool to translate from a .config file to a set of distro, machine, local.conf config files (and an image recipe?) + drive whichever tool will be used to fetch meta layer git repos. The "pre-canned" knowledge of poky, yoe, etc then just becomes a set of alternative defconfig files. Although if we can generate distro configs on the fly from a defconfig file then separate meta layers to hold distro configs starts to become redundant... As a reminder, here's the Buildroot README: https://github.com/buildroot/buildroot/blob/master/README With those first 2 steps a Buildroot user can effectively create their own custom distro and machine config... all from within the user-friendly menuconfig environment. The OE equivalent (Yocto Quick Start) is not only more complex and only tells a user how to build someone else's preset configurations. Creating a custom distro and machine config in OE is beyond the quick start and requires a full knowledge of bitbake syntax etc, etc. > oe-setup init poky . > oe-setup init poky dunfell . > oe-setup init poky ./my-local-config.json . > oe-setup init poky http://someserver/my-remote-config.json . > > We'd also allow something not in the default like of project names to > be used directly with a url, or maybe added with an "add-project" > command. This could work against a user local config file, a bit like > git does with global config and adding remotes to a repo. > > You'll note I haven't made any mention of the tooling these use. The > reason is that we don't actually care. I'd propose we teach the tool > about a few common standards (kas, submodules, repo) in the form of > plugins and then hand off to those tools to do the setup. I'd also > propose we develop a "native" form where they perhaps aren't needed. > the nice thing is we can have several "native" forms if needed so if > one approach isn't working or we need to change it, we can. > > We may also want to consider an optional "sub-config" parameter which > gets passed along to the underlying tool. > > One of my conclusions after thinking about this for a long time is we > have a bootstrap problem. If everyone used git and git worked > everywhere, things would be easier. They don't and it doesn't. My > evidence? The bitbake fetcher. Much of the ugliness the bitbake fetcher > deals with applies to layers as well. Some people need proxy support, > some need mirror tarballs, some need floating revisions, some need > complete lock down and so on. > > We could simplify the problem and just say those users "can manage". If > we do that, we're giving up at the first hurdle on the idea what we can > have a (mostly) universal tool. I'm not sure I'm quite ready to do > that. > > Like most more seasoned developers, I dislike code duplication. We do > have the fetcher code, which knows something about these issues and it > even has a test suite with decent coverage. Most users already know how > to configure it too. > > Whilst I don't like it particularly, I'd propose we include a chunk of > bitbake inside oe-setup (maybe just lib/bb?). How needs discussion. I > nearly wrote combo-layer here but people might think I'm serious :). > More seriously, git subtree might be a potential option. We could then > have bitbake master, or a chunk of it, available to the tool. This > would then allow us to potentially get out of a variety of different > firewall/mirror situations, if the appropriate backends were right. > This gives us the option for fix bugs in the fetcher and update oe- > setup and still use specific versions of bitbake for the different > releases as needed. > > I'd propose oe-setup be a different kind of project to bitbake, > hopefully something we could make available via pip for example, again > to try and help the bootstrap issue for people behind firewalls etc. > although I would want it usable standalone too. It would be designed to > be a separate standalone tool installed somewhere once by a user rather > than associated with a particular build/metadata set like bitbake is. > > Beyond init, the question is probably updating. There are probably two > options, one is an update command to oe-setup and the other is > deferring to the underlying tool and it should be possible to use > either depending on the underlying tool and the circumstances. > > As for configuration of the build, I've wondered if we add a new level > of config file, basically dedicating a config file "slot" to the setup > tooling where config from the setup tool would land (i.e. like > auto.conf but dedicated to this). I need to think about this piece a > bit more. > > The other side of this would be the generation of the config(s). I'm > less worried about this piece, it would be done by the people who can > deal with more complexity and I'm sure we could figure it out. I used > "json" in the config urls above but I'm aware that yaml is going to be > a discussion there too. From the oe-setup perspective, it may support > multiple options since once it knows the tool to pass it off to, it > doesn't really need to know more. > > I'm sure there is more I could write on this topic but I'll stop there > and see what people think. > > Cheers, > > Richard > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#167633): https://lists.openembedded.org/g/openembedded-core/message/167633 Mute This Topic: https://lists.openembedded.org/mt/92117681/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
