Re: [yocto] How do you release distros produced with Yocto? How should I?
On Tue, Oct 02, 2012 at 12:43:27PM -0400, Jerrod Peach wrote:
I was thinking about doing something very close to that in actual Yocto,
except I'd store only the revisions/branches that were different from what
the bb file prescribed in local.conf. I ran that idea by a couple of
colleagues separately and they both immediately pointed out a concern: they
didn't want to have two different places that could manage the revision of
the package, i.e. local.conf and the package bb file. They don't like how
that works in the system we have today. Instead, they wanted to branch all
the metadata and have people update individual bb files with different
revisions as necessary. That leaves only one place to manage the revision:
the package bb file. Doing all that branching concerns me (it seems
heavier-weight than it needs to be, at the very least), but I can't say
specifically why branching the bb files is harmful. If you guys can think
of a reason why, let me know. If you think it's reasonable, let me know
that too.
We have solved this issue by adding OS_VERSION variable to local.conf
Lets say your product (OS) should be released in versions, Ex: 1.0 , 1.1 or
1.1.1 etc.
Which is built from packages of different versions. So each package contains
pkgname.inc and package_version.bb file.
revision number and patches are maintained in package_version.bb and all
common part under package.inc.
Now the problem comes to mapping the package versions to your product(OS)
version.
This is done having a file included into disto file listing out all the
PREFERRED_VERSIONS and PREFERRED_PROVIDERS for all packages.
ex: conf/disto/include/osversion_1.0.inc
And this is included to disto file like
require conf/disto/include/osversion_${OS_VERSION}.inc
The file osversion_1.0.inc is freezed after your one version release.
Next version will add a new file osversion_1.1.inc overiding to pick all the
required package version changes to make your product version.
This will allow you to release multiple product(OS) versions and clearly maps
the package version, creating individual distributions.
You can even build older version product by just giving OS_VERSION=1.0 in
local.conf
This eliminates branching all together.
Regards,
Ramana
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] How do you release distros produced with Yocto? How should I?
On Tue, Oct 2, 2012 at 11:45 AM, Jerrod Peach pea...@lexmark.com wrote: Tomas, Sounds to me like your situation implies a single distro + multiple machines, one for each distinct printer model; you can then specify revisions on per-machine basis. I don't think that's actually what we want. The architecture of each machine will be the same. That is, one ASIC will generally be on all the printers in a family of products. I think I actually have one machine + multiple distros. Or, should I really be counting each different printer as its own machine? well if machine is same then it should be so. But you can have different images meant for same image. e.g. image-printer1 image-printer2 etc. however this may not be best if you want two different versions of same package X in those two images. You could also look into package groups and IMAGE_FEATURES. I think if you need to put two different revs of same package then branching may be better approach or you could have multiple recipes X-printerA.bb X-printerB.bb and so on and chose the right one into image-printerA and so on. There are many ways you can use. You have to look into what fits best to your needs Whether you specify the machine specific revisions in the bb files, or whether you pull it together into an include file is a matter of taste more than anything else I suspect, as long as everyone knows what the deal is. But I'd advise not to specify package revisions local.conf, that's really for the developer/user to tweak, and it should not be stored in vcs, doings so just causes pain. That's something that's come up in our side conversations here: local.conf is really for developers to tweak. I will try to take care and not use local.conf for such a thing, and I will not store it in a VCS unless I can think of a really compelling reason to do so. Thanks for the advice there. I use the unified include file in Guacamayo for the packages that we maintain; this is for convenience, as during the development cycle I use AUTOREV for these packages, but for an actual release specify the revisions explicitly and having them all in one place makes this easier to do and not forget anything. See, https://github.com/Guacamayo/meta-guacamayo/tree/master/meta-guacamayo/conf/ for how we got it set up. So, to me it looks like you're using conf/distro/guacamayo.conf to set those revisions. That seems like that might work when there's only a handful of revisions to set. However, we'll likely have at least a hundred packages for which we need to set/manipulate revisions. I would think that would get cumbersome, and in an organization the size of ours (500 or so developers across two sites), there's not really going to be one person or team who knows what should go in that file for a release. Moreover, that still leaves the problem of all those developers needing to know there are at least two places in which their package revisions may be controlled. Is your organization significantly smaller than that or are we doing something wrong? Kind regards, Jerrod ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] How do you release distros produced with Yocto? How should I?
Hi, On 02/10/12 19:45, Jerrod Peach wrote: I don't think that's actually what we want. The architecture of each machine will be the same. That is, one ASIC will generally be on all the printers in a family of products. I think I actually have one machine + multiple distros. Or, should I really be counting each different printer as its own machine? The fact that the underlying HW is the same can be simply reflected by having the actual HW-related definitions in a common include file. But if printer X requires different version of package A from printer Y then they are really best thought of as different machines -- you can make packages specific to a machine architecture and then reliably pin the version, you can't make packages distro-specific (the whole OE/Yocto set up is built around an implicit premise that any given OE/Yocto instance is made up of a single 'distro' and many 'machines', so the distro leaves no mark on the packages). If all that is different are strictly the VCS revisions, you can use branching but if someone introduces some other subtle difference, like printer-X requires --enable-feature-x for package A and printer-Y requires --disable-feature-x for the same package, then using different machines is the only option to handle this. If you use branching instead of distinct machines in this scenario, you will have to maintain distinct build locations *and* package repositories for each branch, as there is no way to tell package-A with --enable-feature-x from branch printer-x from package-A with --disable-feature-x from branch printer-y. Another reason not to use branches is that this increases the maintenance burden when making changes to packages that are common across the branches; from experience even with just a couple of extra branches folk invariably forget to merge the changes everywhere they should (or merge them into places they should not). Using different bb files per printer seems like a good idea until you try it, but you quickly discover it does not really work. The main problem here is that now packages further down the line can no longer depend on package-A, and have to depend on package-A-printer-X instead, so you will need to have multiple versions of the dependent packages too, and then of those depending on them, and so on ... You might think that PROVIDES_virtual/package-A would be an answer here, but that really only works for distinct machines, so you can specify PREFERRED_PROVIDER or use COMPATIBLE_MACHINE. However, we'll likely have at least a hundred packages for which we need to set/manipulate revisions. I would think that would get cumbersome, and in an organization the size of ours (500 or so developers across two sites), there's not really going to be one person or team who knows what should go in that file for a release. Moreover, that still leaves the problem of all those developers needing to know there are at least two places in which their package revisions may be controlled. Is your organization significantly smaller than that or are we doing something wrong? I would agree that in this scenario it would best to maintain the source revisions in the bb recipes themselves, it keeps people rom stepping on each others toes and accidentally changing revisions for other packages (which is easily done in automatic merge-conflict resolution gone wrong). Tomas ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] How do you release distros produced with Yocto? How should I?
Tomas, On Wed, Oct 3, 2012 at 3:59 AM, Tomas Frydrych tf+lists.yo...@r-finger.comwrote: However, we'll likely have at least a hundred packages for which we need to set/manipulate revisions. I would think that would get cumbersome, and in an organization the size of ours (500 or so developers across two sites), there's not really going to be one person or team who knows what should go in that file for a release. Moreover, that still leaves the problem of all those developers needing to know there are at least two places in which their package revisions may be controlled. Is your organization significantly smaller than that or are we doing something wrong? I would agree that in this scenario it would best to maintain the source revisions in the bb recipes themselves, it keeps people rom stepping on each others toes and accidentally changing revisions for other packages (which is easily done in automatic merge-conflict resolution gone wrong). Okay, so it sounds like my game plan should be to tag my metadata and branch it only if necessary, i.e. if someone needs to change a BB file. (That makes sense, of course, but we have had a long, storied history of branching when it wasn't necessary in our build and library system, so that is a change for us.) Even if we branch our metadata, we'll be creating a new tag once whatever modifications we need have been made. Can anyone see a better way to handle releases than that? Kind regards, Jerrod ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] How do you release distros produced with Yocto? How should I?
Hi, On 02/10/12 17:43, Jerrod Peach wrote: I'm also starting to think there might be a better way to handle this with Yocto's concept of distros (perhaps have a distro for printer X, and a different one for printer Y, each pointing at versions of code that are good for the respective printer), but my research so far hasn't given me enough information on distros to know if this is a reasonable approach. (I've poked through some of the documentation and the mailing list archives.) So, what do you all do for releasing code? Does anyone have a situation similar to mine? (I can't imagine I'm unique, but maybe I'm more special than I thought.) Even if you don't have a situation like mine, what would you suggest I do for releasing code for our printers? Sounds to me like your situation implies a single distro + multiple machines, one for each distinct printer model; you can then specify revisions on per-machine basis. Whether you specify the machine specific revisions in the bb files, or whether you pull it together into an include file is a matter of taste more than anything else I suspect, as long as everyone knows what the deal is. But I'd advise not to specify package revisions local.conf, that's really for the developer/user to tweak, and it should not be stored in vcs, doings so just causes pain. I use the unified include file in Guacamayo for the packages that we maintain; this is for convenience, as during the development cycle I use AUTOREV for these packages, but for an actual release specify the revisions explicitly and having them all in one place makes this easier to do and not forget anything. See, https://github.com/Guacamayo/meta-guacamayo/tree/master/meta-guacamayo/conf/ for how we got it set up. Tomas ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] How do you release distros produced with Yocto? How should I?
Tomas, Sounds to me like your situation implies a single distro + multiple machines, one for each distinct printer model; you can then specify revisions on per-machine basis. I don't think that's actually what we want. The architecture of each machine will be the same. That is, one ASIC will generally be on all the printers in a family of products. I think I actually have one machine + multiple distros. Or, should I really be counting each different printer as its own machine? Whether you specify the machine specific revisions in the bb files, or whether you pull it together into an include file is a matter of taste more than anything else I suspect, as long as everyone knows what the deal is. But I'd advise not to specify package revisions local.conf, that's really for the developer/user to tweak, and it should not be stored in vcs, doings so just causes pain. That's something that's come up in our side conversations here: local.conf is really for developers to tweak. I will try to take care and not use local.conf for such a thing, and I will not store it in a VCS unless I can think of a really compelling reason to do so. Thanks for the advice there. I use the unified include file in Guacamayo for the packages that we maintain; this is for convenience, as during the development cycle I use AUTOREV for these packages, but for an actual release specify the revisions explicitly and having them all in one place makes this easier to do and not forget anything. See, https://github.com/Guacamayo/meta-guacamayo/tree/master/meta-guacamayo/conf/ for how we got it set up. So, to me it looks like you're using conf/distro/guacamayo.conf to set those revisions. That seems like that might work when there's only a handful of revisions to set. However, we'll likely have at least a hundred packages for which we need to set/manipulate revisions. I would think that would get cumbersome, and in an organization the size of ours (500 or so developers across two sites), there's not really going to be one person or team who knows what should go in that file for a release. Moreover, that still leaves the problem of all those developers needing to know there are at least two places in which their package revisions may be controlled. Is your organization significantly smaller than that or are we doing something wrong? Kind regards, Jerrod ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] How do you release distros produced with Yocto? How should I?
On Tue, Oct 02, 2012 at 12:43:27PM -0400, Jerrod Peach wrote: All, After spending a few years working with a several-years old forked and heavily-modified version of BitBake, my company is looking at switching to using Yocto to build embedded Linux for its printers. I've been playing with/writing tools and process around Yocto for a couple of months now, and I'm getting to the point where I can produce integration builds without a problem. Now I'm starting to think about how our release process should work with pure Yocto and I'm running into a conundrum. Before I get into that, though, let me very briefly explain how we release code currently in terms of Yocto as it is today. This is not exactly what we do, but if I had to explain the real process, it would take a while because of all the customization we bolted on top of our forked BitBake. Basically, we have a branch in a repository containing the equivalent of local.conf, and we put fixed revisions of every package for that release into the local.conf file within that branch. As patches for a release come in, the local.conf file is updated to point to the new patches. I was thinking about doing something very close to that in actual Yocto, except I'd store only the revisions/branches that were different from what the bb file prescribed in local.conf. I ran that idea by a couple of colleagues separately and they both immediately pointed out a concern: they didn't want to have two different places that could manage the revision of the package, i.e. local.conf and the package bb file. They don't like how that works in the system we have today. Instead, they wanted to branch all the metadata and have people update individual bb files with different revisions as necessary. That leaves only one place to manage the revision: the package bb file. Doing all that branching concerns me (it seems heavier-weight than it needs to be, at the very least), but I can't say specifically why branching the bb files is harmful. If you guys can think of a reason why, let me know. If you think it's reasonable, let me know that too. I think that branching (or tagging) whole metadata is best option, use AUTOREV from some .inc file (included in local.conf) for development and then just before release update in recipe SRCREVs with small script from bitbake persistent cache (tmp-eglibc/cache/bb_persist_data.sqlite3). This way you will get correct SRCREVs in branched metadata and won't need so many recipe updates during development to move SRCREV too often. Cheers, I'm also starting to think there might be a better way to handle this with Yocto's concept of distros (perhaps have a distro for printer X, and a different one for printer Y, each pointing at versions of code that are good for the respective printer), but my research so far hasn't given me enough information on distros to know if this is a reasonable approach. (I've poked through some of the documentation and the mailing list archives.) So, what do you all do for releasing code? Does anyone have a situation similar to mine? (I can't imagine I'm unique, but maybe I'm more special than I thought.) Even if you don't have a situation like mine, what would you suggest I do for releasing code for our printers? Kind regards, Jerrod ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto -- Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com signature.asc Description: Digital signature ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
