Can we change bitbake fetcher to default to https instead git anonymous protocol as fallback? this will be good security measure too.
On Tue, Nov 2, 2021 at 5:46 AM Richard Purdie <[email protected]> wrote: > > On Tue, 2021-11-02 at 12:32 +0000, Richard Purdie via lists.openembedded.org > wrote: > > On Tue, 2021-11-02 at 11:56 +0000, Andrei Gherzan wrote: > > > On Tue, 2 Nov 2021, at 11:52, Martin Jansa wrote: > > > > On Tue, Nov 2, 2021 at 12:46 PM Richard Purdie > > > > <[email protected]> wrote: > > > > > On Tue, 2021-11-02 at 11:32 +0100, Martin Jansa wrote: > > > > > > There is even bigger issue with git repos from github.com now: > > > > > > > > > > > > > > > > > https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git > > > > > > > > > > > > bitbake git fetcher uses git:// protocol by default and as of > > > > > today you > > > > > can > > > > > > experience "short brownouts" and on January 11 it will all fail to > > > > > fetch (and > > > > > > only fully populated PREMIRRORS can save you for a while, until > > > > > SRCREV > > > > > is > > > > > > updated). > > > > > > > > > > > > Short statistics from current oe-core/master: > > > > > > martin@jama:/OE/openembedded-core$ git grep git://github.* | grep > > > > > -v > > > > > protocol= > > > > > > | wc -l > > > > > > 52 > > > > > > martin@jama:/OE/openembedded-core$ git grep > > > > > git://github.*protocol=https | wc > > > > > > -l > > > > > > 20 > > > > > > martin@jama:/OE/openembedded-core$ git grep > > > > > git://github.*protocol=git > > > > > > wc -l > > > > > > 2 > > > > > > > > > > > > 54 from 74 recipes will fail to fetch in oe-core only. > > > > > > > > > > Thanks for reporting this, it helps to know this is happening as > > > > > we'll > > > > > probably > > > > > start seeing odd error reports for the brownouts. > > > > > > > > > > > > The brownouts are already happening, got 20+ failed jenkins jobs over > > > > night, > > > > because they failed to fetch various metadata layers over git:// from > > > > github. And hopefully my understanding of the announcement is correct > > > > and > > > > git:// brownouts are planned only for today. > > > > > > > > > I've updated the conversion script I mentioned earlier in this thread > > > > > to > > > > > handle > > > > > remapping the github.com urls too and also fixed the few corner > > > > > cases I > > > > > found > > > > > after the first conversion. I've sent those patches to OE-Core. > > > > > > > > > > > > Thanks!, looks good to me. > > > > > > > > > For the older releases, rather than trying to rewrite all the urls, I > > > > > think we > > > > > may want to patch bitbake to correctly handle the github urls > > > > > specifically. > > > > > > > > > > > > Considering how many people I've seen complaining about new overrides > > > > syntax > > > > breaking their just updated oe-core/dunfell build, just because they > > > > don't > > > > update bitbake revision it might be safer to do both (so that at least > > > > the > > > > maintained layers get the explicit protocol=https in SRC_URIs and the > > > > not- > > > > so-well-maintained layers could be saved by git fetcher changing the > > > > protocol automagically). > > > > > > I totally agree with that. I still think we should also warn out so we > > > don't > > > have to maintain this magic quirk forever. > > > > I think we put a warning on master and forwards but not older bitbakes. > > I've sent out a couple of patches for bitbake, one which does the remapping > and > a second which adds the warning. Testing would be appreciated before I merge > them (I need to focus on master first). > > Cheers, > > Richard > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#1337): https://lists.openembedded.org/g/openembedded-architecture/message/1337 Mute This Topic: https://lists.openembedded.org/mt/86675927/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
