Hi Jarek,
I like the idea very much to have a better usability for standalone.
That might make it easier for standalone first-user-experience. Being
able to restart individual processes and seeing components individual
logs is for sure a great thing.
Nevertheless though three things to consider in my view:
* Today when running standalone in most cases the intro URL and
dynamically created admin password is printed in stdout, if mprocs
is launched later this information would be hidden in the console.
So in standaonle mode we would need to mound one "dummy process"
panel giving the key "getting started" information with UI URL and
admin user/password at least. Maybe also adding a link to "first
usage help".
* We should not make it harder for users for first-time user
experience. Today after a `pip install apache-airflow` you are
potentially good to go, adding mprocs as an additional (even small)
manual dependency makes it harder to get started. Would be cool to
have a pymprocs :-D
* There might be usages where you run standaonle non-interactive in a
backend (single container for PoC) - whereas mprocs would not be a
good fit. Not sure whether this should be "default" or the
interactive case should be the default. We therefore might consider
not naming it `--mprocs` but rather `--interactive` to describe the
use case?
On the neutral side though I am not sure if a first time user is
"easier" with merged logs to see errors of if a first time user might be
overwhelmed with too many panels to scroll in to find an error not
knowing which component maybe raising a stack trace at all.
Jens
On 12/23/25 12:25, Jarek Potiuk wrote:
Hello here,
After trying mprocs for "start-airflow" I think mprocs would be a great
candidate to use for standalone airflow. I think currently one of the
problematic things for standalone airflow is the log multiplexing at the
output and lack of control over the individual processes run as part of
"standalone" airflow execution.
I think `mprocs` solves many of those problems that we have with "airflow
standalone" nicely. It provides a very nice, terminal interface to monitor
and control multiple processes - very user friendly, supporting all modern
terminals, copy&pasting, mouse control for monitoring of running processes,
ability to restart individual processes and it has also a number of
interesting features in the roadmap.
But even today it offers a lot more than we have today in standalone
airflow:
* separate logs for each process with nice navigation
* easy copy&paste
* keyboard shortcuts and navigation to switch and move between panels
* ability to individually restart processes when needed
* ability to autorestart processes when they fail
* ability to run scripts defined in package.json (it's an npm tool)
I'd say it sounds like a good idea to have it as part of standalone
airflow. One drawback it has that it has a dependency on npm, but also it
has a standalone binary, that Airflow Standalone **could** download from
https://github.com/pvolok/mprocs/releases and run internally. Or maybe
search for or even implement simple version of it as alternative in Python.
We could have an `--mprocs/-no-mprocs` flag when starting the standalone
airflow (and decide on default setting, also we could give the user a
choice to run mprocs by default when they run standalone and download the
binary, I think in general it delivers a very good and very user-friendly
(and not even expert-friendly like tmux) interface and "airflow standalone"
would get an immediate usability boost with it. I think the TUI interfaces
of those sort have shown it's power over last years (k9s being the flagship
tool).
I would love to hear your thoughts about it.
J.
