+1 (non binding) I also agree that we should proceed in the direction of slowly purging mssql backend.
But I also strongly feel that we should have a proper guide which is fail proof, either in form of docs or scripts. Also, as per Jarek, postgres would be the natural solution here. But we also need to consider having MYSQL as an option and should design the scripts/docs accordingly. There are some good wikis out there like: https://wiki.postgresql.org/wiki/Microsoft_SQL_Server_to_PostgreSQL_Migration_by_Ian_Harding. Hence, having a script shouldn't be too hard either. Thanks, Amogh Desai On Fri, Aug 18, 2023, 20:09 Damian Shaw <ds...@striketechnologies.com> wrote: > My previous company was split between being pro-MSSQL and pro-Oracle, if > you went to the DBA team these were the only supported DB options you had. > > As a developer it's very confusing to come across a product that is > already using multiple DB backends and won't "just" add your companies > preferred flavor. > > I think an explanation in the "Choosing database backend"[1] in addition > to why MariaDB isn't supported, also why MSSQL isn't supported, and some > explanation that each additional DB backend requires significant resources > to add in both expertise and CI cost. > > [1]: > https://airflow.apache.org/docs/apache-airflow/stable/howto/set-up-database.html#choosing-database-backend > > -----Original Message----- > From: Pierre Jeambrun <pierrejb...@gmail.com> > Sent: Friday, August 18, 2023 8:37 AM > To: dev@airflow.apache.org > Subject: Re: [DISCUSS] Drop MsSQL as supported backend > > I am also in favor of dropping MSSQL. We have seen a lot of trouble with > this particular backend, and with the data points you mentioned it > definitely seems that it's not worth all the efforts to maintain it. > > Le ven. 18 août 2023 à 16:28, Wei Lee <weilee...@gmail.com> a écrit : > > > +1 (non binding) > > > > I agree with this direction. It would be helpful to guide users on > > their next steps as Jarek suggested. > > > > Best, > > Wei > > > > > On Aug 18, 2023, at 9:29 PM, Michał Modras > > > <michalmod...@google.com.INVALID> > > wrote: > > > > > > +1 - I strongly agree with the direction too. > > > > > > On Fri, Aug 18, 2023 at 2:31 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > >> I am strongly for it. I wanted to raise the same proposal after we > > >> see > > the > > >> results of the survey (which I hope we will run after the Summit) - > > >> in order to get more data points on how many users use it. But I > > >> agree the datapoints we have support the "low usage, high trouble" > status of it. > > >> MSSQL creates a lot of troubles, it has bitten us more than once. > > >> > > >> One comment though: I think an important point of this removal > > >> should be some viable way for our users to move out to (likely) > > >> Postgres. We > > should > > >> provide at least a description, ideally some scripts that will be > > >> able > > to > > >> move data from MSSQL to Postgres as part of this "support dropping". > > >> > > >> I hate to say to anyone "sorry, no more support, deal with it > yourself". > > >> > > >> But saying: "sorry, no more support, here is a way how you can > > >> migrate > > to > > >> what we support" is a much better message . > > >> > > >> J. > > >> > > >> > > >> J. > > >> > > >> > > >> On Fri, Aug 18, 2023 at 2:06 PM Elad Kalif <elad...@apache.org> > wrote: > > >> > > >>> Hello everyone, > > >>> > > >>> In May 2021 we added MsSQL as an experimental backend. PR > > >>> https://github.com/apache/airflow/pull/9973 > > >>> Mailing list announcement: > > >>> https://lists.apache.org/thread/byx4czfyowwm3qy8c8p2o65vd53fyvsd > > >>> > > >>> During this time we hoped it would become stable and widely adopted. > > >>> To my taste MsSQL a backend has left a niche and is *not* worth > > >>> the maintenance of it in our CI. > > >>> > > >>> Some data points: > > >>> > > >>> 1. Only 5 issues > > >>> < > > >>> > > >> > > https://github.com/apache/airflow/issues?q=is%3Aissue+label%3Abackend- > > mssql-experimintal+is%3Aopen > > >>>> > > >>> were > > >>> reported on MsSQL as a backend which suggests it's not widely > > adopted. > > >>> These issues also don't have much traffic. > > >>> 2. None of the Airflow as service cloud vendors use it as a > > >>> backend > > or > > >>> expressed interest in it (to the best of my knowledge) > > >>> 3. We had many PRs trying to fix issues related to MsSQL. To > > >>> name a > > >> few: > > >>> https://github.com/apache/airflow/pull/33265 > > >>> https://github.com/apache/airflow/pull/32433 > > >>> https://github.com/apache/airflow/pull/31815 > > >>> each one required diagnostic and effort to address the problems > > >>> (in > > >> some > > >>> cases required also rebasing of all other PRs after fix was > > >>> merged) > > >>> > > >>> > > >>> I think that after 2+ years of experiment we can terminate it. > > >>> Since this is experimental we can drop support for this feature as > > >>> this > > >> is > > >>> not considered a breaking change. > > >>> > > >>> *My Proposal:* > > >>> Drop support for MsSQL as official/supported backend. > > >>> > > >>> WDYT? > > >>> > > >>> Elad Kalif > > >>> > > >> > > > > > ________________________________ > Strike Technologies, LLC (“Strike”) is part of the GTS family of > companies. Strike is a technology solutions provider, and is not a broker > or dealer and does not transact any securities related business directly > whatsoever. This communication is the property of Strike and its > affiliates, and does not constitute an offer to sell or the solicitation of > an offer to buy any security in any jurisdiction. It is intended only for > the person to whom it is addressed and may contain information that is > privileged, confidential, or otherwise protected from disclosure. > Distribution or copying of this communication, or the information contained > herein, by anyone other than the intended recipient is prohibited. If you > have received this communication in error, please immediately notify Strike > at i...@striketechnologies.com, and delete and destroy any copies hereof. > ________________________________ > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments > are intended solely for the addressee. This transmission is covered by the > Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The > information contained in this transmission is confidential in nature and > protected from further use or disclosure under U.S. Pub. L. 106-102, 113 > U.S. Stat. 1338 (1999), and may be subject to attorney-client or other > legal privilege. Your use or disclosure of this information for any purpose > other than that intended by its transmittal is strictly prohibited, and may > subject you to fines and/or penalties under federal and state law. If you > are not the intended recipient of this transmission, please DESTROY ALL > COPIES RECEIVED and confirm destruction to the sender via return > transmittal. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org >
