I think we could also consider repurposing "airflow standalone" to be "production ready," as was discussed in the past. Currently we print many warnings, and there likely few things missing - some way of backing up the sqllite database, better way to do logging so output isn't interleaved in stdout, and the ability to monitor and restart individual sub-processes when they hang. Repurpose SimpleAuthManager to also be "production ready" (or add a new Auth Manager)—with an easier way of managing users (storing them in the SQLite database and adding CLI commands, rather than adding them to configuration). And clearly state the limitations of such a setup.Think
I think resource usage will be similar to what Kaxil described. The real difference is that instead of users managing their processes, the "airflow standalone" process "manager" would handle it - so this is more about convenience of running a "small" installation. I am not sure if we "want" to do it, but I think this has quite an advantage even in different scenario - people who want to run simple agent-native workflows on a schedule and triggers. J. On Wed, Nov 4, 2026 at 4:34 PM Kaxil Naik <[email protected]> wrote: > Hi Mihir, > > Thanks for raising this. Before we look at a separate "lite" Airflow, it'd > be good to check whether what we ship today is small enough for you. It's a > plain pip install with no Docker needed, and SQLite + LocalExecutor are > already the defaults. > > The smallest setup that still schedules Dags is three processes: > > pip install "apache-airflow==3.3.2" --constraint > > https://raw.githubusercontent.com/apache/airflow/constraints-3.3.2/constraints-3.12.txt > export AIRFLOW__CORE__PARALLELISM=2 > airflow db migrate > airflow api-server & > airflow scheduler --skip-serve-logs & > airflow dag-processor & > > Swap 3.12 in the constraints URL for your Python version. The triggerer is > only needed for deferrable operators, so you can leave it out; "airflow > standalone" always starts it along with its own supervisor process, which > in my test came to about 350 MB more than running the three processes > yourself. LocalExecutor starts "parallelism" workers up front (default 32), > so set it to what the box actually runs at once. Skipping serve-logs is > fine on one machine, since finished-task logs are read from local disk. > > I tried this in an arm64 Linux container (not a real Pi) with a two-task > Dag running every minute. It stayed around 630 MB and ran fine under a 1 GB > memory limit. For comparison, 2.10 with the scheduler and a one-worker > webserver sat around 410 MB and didn't fit in 512 MB either, so 3.x is > roughly 200 MB heavier with the UI running. A Raspberry PI with 2 GB should > be comfortable. > > Kaxil > > On Wed, 4 Nov 2026 at 23:34, Vikram Koka via dev <[email protected]> > wrote: > > > Mihir, > > > > Thank you for taking the time to bring this up! > > > > A lot of the time and energy we spend on the Airflow dev list focuses on > > large enterprise use cases for running Airflow at scale. > > Your motivation as expressed to me earlier today was to use Airflow for > > small use cases at NGOs that need orchestration for their data solutions. > > Their data needs are small yet real, and computing affordability is a key > > requirement. > > > > As an open source project, we absolutely want to meet the needs of this > > audience as well, where feasible. > > The smaller, lightweight execution environment is something that Ephraim > > and Ash are already looking at, but with a different lens, so I will make > > sure to flag this to them as well. > > > > Best regards, > > Vikram > > > > > > On Wed, Nov 4, 2026 at 3:24 PM Mihir Samant <[email protected]> wrote: > > > > > Hi Team, > > > > > > During the recent Apache Airflow Reconnect, I had a brief chat with > > Vikram > > > about an idea to make Airflow more accessible for smaller > organizations. > > > > > > *The Core Idea:* > > > > > > Develop a lightweight, streamlined version of Apache Airflow tailored > for > > > teams and non-profits with modest data orchestration needs. Many small > > > organizations don't require heavy AI/ML workflows or the infrastructure > > > complexity of Airflow 3.x. > > > > > > *Key Focus Areas:* > > > > > > - > > > > > > *Simplified Scope:* Focus purely on core, standard data processing > and > > > pipeline orchestration. > > > - > > > > > > *Easier Deployment:* Offer a fast, non-Docker installation pathway > for > > > environments with limited infrastructure or system resources. > > > > > > I believe a lean Airflow profile could expand our adoption across small > > > businesses and community organizations that currently find the full > > > footprint overwhelming. > > > > > > Huge thanks to Vikram for taking the time to listen and discuss this! > I’d > > > love to hear the team's thoughts on whether this aligns with any > upcoming > > > community or roadmap discussions. > > > > > > Best, > > > > > > > > > *Mihir Samant**519-982-8632 <(519)%20982-8632> <(519)%20982-8632> > > <(519)%20982-8632>* > > > > > >
