We explored a similar idea with Django many years ago, and the conclusion back then, which I would also put forward here, is that having a project scale down to an easy developer install is of crucial importance, and so I think SQLite has to stay in that role (as there is no reasonable alternative, at least not yet).
I do think it should be heavily discouraged in production installs, though. If there's a way you think we can pull that off while not making development annoying, I'd be all for it. Andrew On Tue, Jul 20, 2021 at 12:24 PM Shaw, Damian P. < [email protected]> wrote: > Some thought as a user of Airflow, > > > > I wouldn’t of adopted Airflow in the first place if I couldn’t test it > with sqlite. And would be the same today, accessing docker isn’t always an > easy in some companies. > > > > But having a warning that sqlite is development only and much slower than > other solutions when it’s enabled seems fair. Also forcing new time users > to edit the config on first run I think is acceptable as they will need to > get used to do that frequently anyway if they’re rolling their own install. > > > > Damian > > > > *From:* Jarek Potiuk <[email protected]> > *Sent:* Tuesday, July 20, 2021 12:23 > *To:* [email protected] > *Subject:* [DISCUSSION] Should we be more explicit about SQLite using for > dev only (or kill it for non-dev entirely????) > > > > Hello Community, > > > Recently we had several people who complained (on slack) that airflow 2.1 > is slow in scheduling tasks. After some discussion it usually turned out > that those people were using SQLite + Sequential executor. I think it gives > very bad impression to users. We even had one user who almost gave up > Airflow seeing how slow it is in scheduling tasks (!). > > I think while in Airflow 1.10 the difference was not as noticeable, > Airflow 2 with Postgres/MySQL is lightning fast comparing to sqlite. It's > like a different world. > > > > First time users might get a very bad impression when their first contact > with Airflow is via sqlite + Sequential executor. > > > > Many people choose sqlite as their first choice when they try Airflow > (Sqlite is generally seen as solid choice in many cases and people are > afraid that setting up MySQL/Postgres might take them a lot of time to > setup). > > > > However with current Docker-Compose quilck-start by Kamil it is already > rather quick to set-up a working setup with Postgres. > > > > My idea is - why don't we make SQLITE "development-only" choice. That > would require editable, development version of airflow to run and fail hard > when it is installed as regular package (with appropriate "Use proper > database - MySQL/Postgres" - and MSSQL when we release MSSQL-support in 2.2 > ). > > > > I think that would be possible, it would not violate backwards > compatibility (sqlite was anyhow for development-only) and it would help > Airflow with being seen as more "snappy". > > > > Any other ideas? > > > > WDYT? > > > > J. > > > > > > > > -- > > +48 660 796 129 > > > ============================================================================== > Please access the attached hyperlink for an important electronic > communications disclaimer: > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > > ============================================================================== >
