Hello everyone, TL;DR; I have a proposal ready on how we can refactor the installation page of Airflow (based on listening and answering to a big number of user issues/discussions/questions recently). The PR is https://github.com/apache/airflow/pull/18282 (ready for comments/discussions/critique), but I thought this is an important "entrypoint" for new users and I wanted to make sure to send it to devlist and provide more context and get feedback.
I have recently answered a number of questions and tried to help a number of Airflow users (especially new users) who tried to get their experience with installing Airflow. There were numerous issues, discussions, stack overflow questions I looked at and there was an important observation I made - I think the "installation" with Airflow was not really geared towards new users. Yeah, we have a great "quick -start" spear-headed by Kamil, and it's great for someone who just wants to try Airflow, but I think if someone wanted to decide how to approach their first "real" installation of Airflow, the "installation" page of Airflow was pretty confusing and lacked answer to many questions the users had. One very recent example from a discussion I had - nowhere in the installation part it is mentioned that you should run "airflow db upgrade" at installation and each upgrade (there was a mention in Production Deployment but not in Installation). This sounds obvious to us, but it might be a surprise for a new user that the upgrade should be run after even a minor upgrade of Airflow. Those are small things, but there is one - more important (and I think this is the most important observation I got from listening to the users). I had to explain over and over, what each installation means, what the user should expect, that things will not "magically work" in case of local installation and explain that some parts like maintenance, upgrades, cleanup and linking the components together are in the hands of users when they choose "simpler" installation methods. The installation page did not explain to new users the different installation methods the user can take (local installation, docker, helm, managed services). Also it did not explain what expectations the users should have when using each of those methods. Some users lack understanding on what they are responsible for when installing Airflow, they did not know that when they choose docker-compose path, they must know how to build their own image etc. and more importantly they did not know what they can expect from Airflow Community in terms of resources and help. I think having such an overview would be extremely helpful for our users, so that they could make a decision which route they are going to take. I attempted to turn the "installation" page into separate, interlinked pages with exactly this kind overview of installation methods: * when this option works best * intended users * what you are expected to handle * what Apache Airflow Community provides for that method I am open to change wording, or make a different split, or restructure it, but I think a change to the installation page is needed - especially that we are past the time when we focused on migrating users from 1.10 but now we are addressing new users who did not install Airflow before. The PR contains some screenshots, and you can also build the docs for yourselves from my branch. Looking forward to improving the installation docs. J.
