Status update: - all recent releases of sssd and adcli have been pulled from -updates and -security, and placed back into -proposed.
- I made a debdiff to revert the problematic patches for adcli in Bionic, Lukasz has built it in https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4336/+packages - Currently waiting for adcli - 0.8.2-1ubuntu2 to be bin-synced from the above ppa to bionic-proposed for testing. - We need to release adcli - 0.8.2-1ubuntu2 to -updates and -security after. - I have written to customers and confirmed the regression to be limited to adcli on Bionic, and given them instructions to dowongrade to the version in the -release pocket. Again, I am sorry for causing the regression. On Monday I will begin fixing up cyrus-sasl2 on Bionic to have a working GSS-SPNEGO implementation. Thanks, Matthew On Sat, Dec 5, 2020 at 12:33 PM Matthew Ruffell <[email protected]> wrote: > > Hi everyone, > > Firstly, I deeply apologise for causing the regression. > > Even with three separate people testing the test packages and the packages in > -proposed, the failure still went unnoticed. I should have considered > the impacts > of changing the default behaviour of adcli a little more deeply than treating > it > like a normal SRU. > > Here are the facts: > > The failure is limited to adcli, version 0.8.2-1ubuntu1 on Bionic. At the time > of writing, it is still in the archive. To archive admins, this needs > to be pulled. > > adcli versions 0.9.0-1ubuntu0.20.04.1 in Focal, 0.9.0-1ubuntu1.2 in Groovy and > 0.9.0-1ubuntu2 in Hirsute are not affected. > > sssd 1.16.1-1ubuntu1.7 in Bionic, and 2.2.3-3ubuntu0.1 in Focal are > not affected. > > Bug Reports: > > There are two launchpad bugs open: > > LP #1906627 "adcli fails, can't contact LDAP server" > https://bugs.launchpad.net/ubuntu/+source/adcli/+bug/1906627 > > LP #1906673 "Realm join hangs" > https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1906673 > > Customer Cases: > > SF 00298839 "Ubuntu Client Not Joining the Nasdaq AD Domain" > https://canonical.my.salesforce.com/5004K000003u9EW > > SF 00299039 "Regression Issue due to > https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1906673"; > https://canonical.my.salesforce.com/5004K000003uAkL > > Root Cause: > > The recent SRU in LP #1868703 "Support "ad_use_ldaps" flag for new AD > requirements (ADV190023)" > https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1868703 > > introduced two changes for adcli on Bionic. The first, was to change from > GSS-API to GSS-SPNEGO, and the second was to implement support for the flag > --use-ldaps. > > I built a upstream master of adcli, and it still fails on Ubuntu. This > indicates > that the failure is not actually in the adcli package. adcli does not > implement > GSS-SPNEGO, it is linked in from the libsasl2-modules-gssapi-mit package, > which is a part of cyrus-sasl2. > > I built the source of cyrus-sasl2 2.1.27+dfsg-2 from Focal on Bionic, and it > works with the problematic adcli package. > > The root cause is that the implementation of GSS-SPNEGO in cyrus-sasl2 on > Bionic is broken, and has never worked. > > There is more details about commits which the cyrus-sasl2 package in Bionic is > missing in comment #5 in LP #1906627. > > https://bugs.launchpad.net/ubuntu/+source/adcli/+bug/1906627/comments/5 > > Steps taken yesterday: > > I added regression-update to LP #1906627, and I pinged ubuntu-archive in > #ubuntu-release with these details, but they seem to have been lost in the > noise. > > Located root cause to cryus-sasl2 on Bionic. > > Next steps: > > We don't need to revert any changes for adcli or sssd on Focal onward. > > We don't need to revert any changes on sssd on Bionic. > > We need to push a new adcli into Bionic with the recent patches reverted. > > We need to fix the GSS-SPNEGO implementation in cyrus-sasl2 in Bionic. > > We need to re-release all the SRUs from LP #1868703 after some very thorough > testing and validation. > > Again, I am deeply sorry for causing this regression. I will fix it, starting > with getting adcli removed from the Bionic archive. > > Thanks, > Matthew > > On Fri, Dec 4, 2020 at 10:40 PM Lukasz Zemczak > <[email protected]> wrote: > > > > Hey! > > > > I prefer broken upgrades to get pulled anyway. Besides, packages are > > updated by unattended-upgrades in up-to 24 hours, so some users might > > have not gotten it yet. And there's also those not using > > undattended-upgrades. Let me demote it back to -proposed from -updates > > as well. > > > > On Fri, 4 Dec 2020 at 10:00, Christian Ehrhardt > > <[email protected]> wrote: > > > > > > On Fri, Dec 4, 2020 at 9:49 AM Lukasz Zemczak > > > <[email protected]> wrote: > > > > > > > > Hey Christian! > > > > > > > > This sounds bad indeed, let's see what Matthew has to say. In the > > > > meantime I have backed it out from both bionic-security and > > > > focal-security. > > > > > > Thank you > > > > > > > Should we also consider dropping it from -updates? > > > > > > Well, compared to other cases in this case we don't even yet have a > > > "ok this is a mess, but this is how you can resolve it afterwards to > > > work again". > > > Therefore I think pulling it from -updates as well makes sense until > > > Matthew had time to look at it in detail and give all-clear (or not). > > > > > > P.S.: you slightly raced vorlon who had a different assessment > > > [09:30] <vorlon> cpaelzer: well, by this point almost everyone will > > > have picked it up from security via unattended-upgrades so there's not > > > much point > > > But having it pulled for now is on the safe-side and we can re-instate > > > it at any time once we know more. > > > > > > > Cheers, > > > > > > > > On Fri, 4 Dec 2020 at 09:01, Christian Ehrhardt > > > > <[email protected]> wrote: > > > > > > > > > > I was looking at 16 recently touched bugs. Of these a few needed a > > > > > comment or > > > > > task update but not a lot of work. Worth to mention are two of them. > > > > > > > > > > First we've had "one more" kind of conflicting mysql packages from > > > > > third party breaking install/upgrade of the one provided by Ubuntu. I > > > > > dupped it onto bug 1771630 which is our single place to unite all > > > > > those. > > > > > > > > > > > > > > > A recent sssd update (driven by SEG) seems to have regressed users > > > > > that now end in a hang. > > > > > I've pinged on [1], subscribed Matthew (and our Team) on [2]. I've > > > > > marked it regression-update and also pinged Matthew him via Chat. > > > > > Furthermore I've set him on CC on this mail. > > > > > @Matthew - once you've done your initial assessment would you mind > > > > > replying here with the next steps on this case please? > > > > > I've marked it prio high, if other triagers see more such reports > > > > > please mark it even critical then (in that case it is less likely to > > > > > be just one odd special setup) > > > > > The release is 21h ago, I'll ping ubuntu-archive (also on CC) if we > > > > > should - for now until clarified by Matthew - remove it from > > > > > -security. > > > > > > > > > > > > > > > [1]: > > > > > https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1868703/comments/86 > > > > > [2]: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/1906673 > > > > > > > > > > > > > > > -- > > > > > Christian Ehrhardt > > > > > Staff Engineer, Ubuntu Server > > > > > Canonical Ltd > > > > > > > > > > -- > > > > > ubuntu-archive mailing list > > > > > [email protected] > > > > > https://lists.ubuntu.com/mailman/listinfo/ubuntu-archive > > > > > > > > > > > > > > > > -- > > > > Łukasz 'sil2100' Zemczak > > > > Foundations Team > > > > [email protected] > > > > www.canonical.com > > > > > > > > > > > > -- > > > Christian Ehrhardt > > > Staff Engineer, Ubuntu Server > > > Canonical Ltd > > > > > > > > -- > > Łukasz 'sil2100' Zemczak > > Foundations Team > > [email protected] > > www.canonical.com -- ubuntu-server mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-server More info: https://wiki.ubuntu.com/ServerTeam
