Re: [Openembedded-architecture] Backporting python module move to OE-Core in dunfell
I understand that this is an exception from regular stable process, but I believe the possible bad user experience was limited to only the users who update the layers incorrectly and the benefits and simplicity for everybody else overweight this. Using slightly older pyelftools and pycryptodome when you somehow forgot to update your meta-python checkout when updating oe-core also isn't the end of the world, you're missing all these updates when you don't update your layers, AFAIK they don't have different API/ABI or known terrible security issues in the older version which was in meta-python at the time of dunfell release, having newer version of them in oe-core dunfell doesn't make these 2 recipes so special IMHO. I don't remember who initially proposed this, nor have any personal or LGE preference for these 2 python modules, because I've already backported pyelftools into our thud based builds last year: https://github.com/webosose/meta-webosose/commit/51853342a6a16b45725759508ede1e098e80d181 and came across pycryptodome as well recently in: https://github.com/shr-project/meta-ros/commit/1149934a4d49f4921290554a4f8aa91451b5fa2c but moving them from meta-python to oe-core doesn't affect this at all (as we're always using meta-python anyway) and with more people starting to use these I believe it's right place to have them updated in oe-core (instead of people adding them in their own backports layers - or worse). I support c). I also hope there won't be many controversial proposals like this for LTS dunfell, but on the other hand I believe we need to be more flexible on these case-by-case basis, just because LTS will be there much longer and we should try to make it more useful and relevant for majority of its users, if we start asking oe-core users to work around this locally or by using some proposed mixin layer, then how messy it will look in 2 years time? Cheers, On Wed, Jun 3, 2020 at 11:27 AM Richard Purdie < richard.pur...@linuxfoundation.org> wrote: > Hi All, > > We'd like to get the first 3.1 point release Dunfell LTS out soon. > There is one issue we need to resolve first. > > meta-arm has recently been established and the hope is to avoid > duplication of a set of recipes in multiple BSP layers, a goal which I > believe is in everyone's interests. As with any new layer there are a > few growing pains like the removal of python2 dependencies and its made > good progress but there is an issue we didn't foresee. > > Two of the key components that BSPs need from there (tf-a and optee) > which are needed for handling boot crypto have dependencies on > pyelftools and pycryptodome which were in meta-python in meta- > openembedded. In master we moved them to core: > > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d52929ee35d043211cb012e1b86409599a8b53c6 > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1e327abcc92d6575df5c1871e6e1284987b74b87 > > and upgraded them to the latest versions: > > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=cb94d9019b4d1ce32ad9196cb9c22236631e8781 > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=7ebe299c00ec309cd46905ad40036d937fe609d7 > > For master that is easy, problem solved. > > For Dunfell, I've already seen complaints that using any of the BSPs > means adding meta-arm which means adding meta-python which pulls in > other chunks of meta-oe. Using a BSP shouldn't require pulling in large > numbers of other layers and this defeats what we're trying to achieve > with a granular layer model. > > Had we realised sooner, we likely would have taken this change into > dunfell before release but sadly we didn't. > > The LTS policy is quite clear we now shouldn't make a change like this. > Equally, we're early in Dunfell's development, we have an unforeseen > issue which is causing users real world pain. > > Our options are: > > a) Do nothing > b) Move the recipes > c) Move the recipes and take the upgrades > > Given the impact on end users and the desire to have good user > experience with our first LTS, as the layer maintainer and having > discussed this with others and on the tech call yesterday, I am > strongly leaning to b or c. I think this issue isn't one Steve as LTS > maintainer or I as OE-Core maintainer can make alone and I'm therefore > referring to the TSC. Which TSC is a good question. LTS is owned by YP, > the layers by OE. As such I've cc'd both sets of TSC members. I've put > it to the architecture list so everyone can see why we're considering > this and any discussion on it. > > Considering the potential regressions, if a user upgrades both layers > with these changes things are fine. If the user upgrades meta-python > but not OE-Core, they get a build error which is also fine and > straightforward to debug. If the user upgrades OE-Core but not meta- > python, there is the possibility they could see old versions being used > due to layer priority (assuming we upgrade the versions) but this isn't > known to cause real
Re: [Openembedded-architecture] Backporting python module move to OE-Core in dunfell
FWIW whilst this does fall outside normal policies, I think this is a reasonable course of action under the (fairly exceptional) circumstances. We would of course have to review future situations on a case-by-case basis. Paul On Wednesday, 3 June 2020 21:27:00 NZST Richard Purdie wrote: > Hi All, > > We'd like to get the first 3.1 point release Dunfell LTS out soon. > There is one issue we need to resolve first. > > meta-arm has recently been established and the hope is to avoid > duplication of a set of recipes in multiple BSP layers, a goal which I > believe is in everyone's interests. As with any new layer there are a > few growing pains like the removal of python2 dependencies and its made > good progress but there is an issue we didn't foresee. > > Two of the key components that BSPs need from there (tf-a and optee) > which are needed for handling boot crypto have dependencies on > pyelftools and pycryptodome which were in meta-python in meta- > openembedded. In master we moved them to core: > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d52929ee35d043211cb012e > 1b86409599a8b53c6 > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=1e327abcc92d6575df5c18 > 71e6e1284987b74b87 > > and upgraded them to the latest versions: > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=cb94d9019b4d1ce32ad9196 > cb9c22236631e8781 > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=7ebe299c00ec309cd46905 > ad40036d937fe609d7 > > For master that is easy, problem solved. > > For Dunfell, I've already seen complaints that using any of the BSPs > means adding meta-arm which means adding meta-python which pulls in > other chunks of meta-oe. Using a BSP shouldn't require pulling in large > numbers of other layers and this defeats what we're trying to achieve > with a granular layer model. > > Had we realised sooner, we likely would have taken this change into > dunfell before release but sadly we didn't. > > The LTS policy is quite clear we now shouldn't make a change like this. > Equally, we're early in Dunfell's development, we have an unforeseen > issue which is causing users real world pain. > > Our options are: > > a) Do nothing > b) Move the recipes > c) Move the recipes and take the upgrades > > Given the impact on end users and the desire to have good user > experience with our first LTS, as the layer maintainer and having > discussed this with others and on the tech call yesterday, I am > strongly leaning to b or c. I think this issue isn't one Steve as LTS > maintainer or I as OE-Core maintainer can make alone and I'm therefore > referring to the TSC. Which TSC is a good question. LTS is owned by YP, > the layers by OE. As such I've cc'd both sets of TSC members. I've put > it to the architecture list so everyone can see why we're considering > this and any discussion on it. > > Considering the potential regressions, if a user upgrades both layers > with these changes things are fine. If the user upgrades meta-python > but not OE-Core, they get a build error which is also fine and > straightforward to debug. If the user upgrades OE-Core but not meta- > python, there is the possibility they could see old versions being used > due to layer priority (assuming we upgrade the versions) but this isn't > known to cause real world issues. > > We have the option of the patch to increase the OE-Core version and > then bumping the version requirement in meta-python to further improve > the user experience. > > I'm conscious there has already been talk of "favouritism". I would > stress this issue has actually been raised by end users and that is the > driving factor here. > > The fact there are interested member companies of YP does actually help > in some ways too since it means there are people available to help > handle any issues should there be unexpected fallout from this change. > > If we do this, I would say as a TSC member that we'd do it without > setting any precedent, this is being discussed based on the situation > at hand, in the future other situations would have to be looked at on a > case by case basis. > > I did quickly sound out the YP TSC at a meeting yesterday and I believe > on balance they're in favour of b/c, two of the OE TSC members on the > tech call yesterday were also in favour in principle. Whilst that is a > majority, that does leave a couple of TSC members who haven't seen the > discussion and everyone probably benefits from this having been written > down. > > Does anyone object to doing this? If we do it, do we upgrade the > recipes or not? I'm probably leaning towards upgrading too as up to > date crypto components are probably better than not updated and some > kind of parity with master now will probably catch more potential > issues. > > Cheers, > > Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1088): https://lists.openembedded.org/g/openembedded-architecture/message/1088 Mute
Re: [Openembedded-architecture] Backporting python module move to OE-Core in dunfell
On Wed, 2020-06-03 at 13:24 +0300, Adrian Bunk wrote: > On Wed, Jun 03, 2020 at 10:27:00AM +0100, Richard Purdie wrote: > > ... > > If we do this, I would say as a TSC member that we'd do it without > > setting any precedent, this is being discussed based on the > > situation > > at hand, in the future other situations would have to be looked at > > on a > > case by case basis. > > ... > > Please document this as the official process for getting such changes > into released stable series. > > Similar situations might arise in the future with e.g. adding zstd > or core hardware enablement for other vendors to OE-core in dunfell. I do not want or expect to see a lot of these requests. The TSC is there as a final decision making authority in general so I'm not sure we need to explicitly document that. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1087): https://lists.openembedded.org/g/openembedded-architecture/message/1087 Mute This Topic: https://lists.openembedded.org/mt/74646262/21656 Group Owner: openembedded-architecture+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [Openembedded-architecture] Backporting python module move to OE-Core in dunfell
On Wed, Jun 03, 2020 at 10:27:00AM +0100, Richard Purdie wrote: >... > If we do this, I would say as a TSC member that we'd do it without > setting any precedent, this is being discussed based on the situation > at hand, in the future other situations would have to be looked at on a > case by case basis. >... Please document this as the official process for getting such changes into released stable series. Similar situations might arise in the future with e.g. adding zstd or core hardware enablement for other vendors to OE-core in dunfell. > Cheers, > > Richard cu Adrian -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1086): https://lists.openembedded.org/g/openembedded-architecture/message/1086 Mute This Topic: https://lists.openembedded.org/mt/74646262/21656 Group Owner: openembedded-architecture+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
