I wanted to point out that not only did Microsoft add Debian Bullseye support for MSSQL today, to do so they did a major version upgrade, from msodbcsql17 to msodbcsql18. So in addition to updating the URL used to populate mssql-release.list, you also need to change to msodbcsql18. I see msodbcsql17 is in multiple places in the Airflow code base.
Instructions for installing msodbcsql18 in Debian are here: https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver15#debian18 Alex On Tue, Feb 15, 2022 at 1:32 AM Jarek Potiuk <[email protected]> wrote: > Yeah. That would definitely be a simpler approach for us. > > What we can also do is we can still make sure that you can actually > "build" Buster images if you want and we could run the simple tests we > already have for those images (those tests just check very basic stuff > - like if expected packages are importable, if Airflow commands work, > if you can run airflow as root user etc.. > This would be very little to no complication of the CI and we could > provide a recipe for the users how to build their own buster image - > but without guarantees that everything works there. > > This way might provide the users with a "workable solution" while we > could keep it simple for us and put a strong emphasis on "If you want > to get full support - also upgrade to the latest stable OS". > > J. > > > On Tue, Feb 15, 2022 at 9:53 AM Ash Berlin-Taylor <[email protected]> wrote: > > > > My vote is to just move forward and only release Bullseye images for 2.3 > onwards > > > > On 15 February 2022 08:43:50 GMT, Jarek Potiuk <[email protected]> wrote: > >> > >> Good news from today - I got the response from Microsoft and it seems > >> that MSSQL Bullseye support is going to be released today :) . So at > >> least that problem is going to be solved. Still the question of > >> whether we should build/release both Bullseye and Buster ? > >> > >> J. > >> > >> On Mon, Feb 14, 2022 at 2:14 PM Jarek Potiuk <[email protected]> wrote: > >>> > >>> > >>> And Python default images are Bullseye now. > >>> > >>> On Mon, Feb 14, 2022 at 2:13 PM Jarek Potiuk <[email protected]> > wrote: > >>>> > >>>> > >>>> Buster end of life is August 2022 of course :) > >>>> > >>>> On Mon, Feb 14, 2022 at 2:12 PM Jarek Potiuk <[email protected]> > wrote: > >>>>> > >>>>> > >>>>> Hello everyone, > >>>>> > >>>>> I am just about to complete preparation to make our images (and CI) > work on Debian 11 images (Bullseye) vs Debian 10 (Buster). All looks > (almost) good, but I would like to tap into the collective wisdom of the > group here to decide on next steps (and maybe propose a policy we might be > following in the future). The last change that allows to easily switch > between the two is in reviee: https://github.com/apache/airflow/pull/21546 > >>>>> > >>>>> Context/where we are: > >>>>> > >>>>> For quite a while we used Buster (Debian 10) as our base image. > Buster is a stable release and it is being replaced with the next stable > release Bullseye (Debian 11). > >>>>> > >>>>> The Release schedule is here: > https://wiki.debian.org/DebianReleases > >>>>> > >>>>> The most important facts from those: > >>>>> * Bullseye was introduced in August 2021 (6 months ago) and has no > end-of-life yet > >>>>> * Buster was introduced in July 2019 and its end of life is ~ > August 2020 (approx. 6 months from now) > >>>>> > >>>>> Python default images are (as of a few months), However they > provide (as usual -buster and -bullseye) specific images. > >>>>> > >>>>> The Bullseye switch is really needed to support ARM (M1) images > because of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=989604 > (still unresolved on Buster). > >>>>> > >>>>> It seems appropriate to switch to Bullseye now, however there is > one problem with it - MSSQL drivers do not yet support Bullseye. I've > reached out to the maintainers and await their answer when it might be > available: > https://github.com/MicrosoftDocs/sql-docs/issues/7255#issuecomment-1037097131 > . This is the last blocker that prevents full switch (failing PR for a full > switch is https://github.com/apache/airflow/pull/21378). All tests pass > except MSSQL. > >>>>> > >>>>> But I think we need to proceed, regardless of the timeline and > switch to Bullseye. > >>>>> > >>>>> Question: > >>>>> > >>>>> We need to decide on what support we give our users in the images > now, in the 2.3.0 release and in the future. We have no policies for that > yet, so it might be a good time to decide now. Some of the users might > "rely" on the Buster being used by the images as they might have some > incompatible libraries (similarly to MSSQL). And I believe we need to > support Buster till at least the end of life of it. > >>>>> > >>>>> This support might be twofold: > >>>>> > >>>>> * we can publish in https://hub.docker.com/r/apache/airflow our > binary images with Buster and/or Bullseye > >>>>> * we can only publish one of those (and change from Buster to > Bullseye for 2.3.0) but users will be able to build their own custom images > using our Dockerfile for either of those. > >>>>> > >>>>> Proposal: > >>>>> > >>>>> I think there are multiple reasons why we should support both > Bullseye and Buster for the coming 6 months. We might build both images in > CI and add a new "matrix" dimension and run most tests on Bullseye but some > tests on Buster (specifically MSSQL until Bullseye is supported). > >>>>> > >>>>> We can publish both types of images (with -buster, -bullseye > suffixes same as Python) but the default image should be changed to > Bullseye. > >>>>> > >>>>> This will add some complexity (and build overhead) to our CI, but I > think it's worth it. > >>>>> > >>>>> Then we could drop support and release the -buster images after it > reaches end of life (but we will leave the users possibility of building > their own buster images without guarantee that it will work). > >>>>> > >>>>> I think this might become our Policy also for the future (6 months > before end-of-life we switch by default to the new stable and provide built > images until end-of-life). > >>>>> > >>>>> WDYT? Does it look like a good proposal? > >>>>> > >>>>> J. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> >
