If there's no one already taking it up, I can take a crack at it. -- Regards, Aritra Basu
On Fri, 23 Jan 2026, 4:48 pm Jarek Potiuk, <[email protected]> wrote: > I have created https://github.com/apache/airflow/issues/60971 to add > `--interactive` flag to airflow standalone using mprocs. It's not a "first" > issue - more of a small mini project, but not a difficult one for > someone who knows process handling and such. > > On Mon, Dec 29, 2025 at 8:53 AM Rahul Vats <[email protected]> wrote: > > > +1 on this. The separate log panels would be a big help for debugging > > issues in standalone mode. > > > > Regards, > > Rahul Vats > > > > On Mon, 29 Dec 2025 at 11:25, Amogh Desai <[email protected]> wrote: > > > > > > mprocs also publishes statically linked binaries > > > > > > Nice! > > > > > > I think I would best favour building it ourselves or publishing it and > > > using it > > > for the `--interactive` mode because these are the two best options > long > > > term. > > > > > > Thanks & Regards, > > > Amogh Desai > > > > > > > > > On Wed, Dec 24, 2025 at 1:36 PM Jarek Potiuk <[email protected]> wrote: > > > > > > > > inherent dependency or we launch it using subprocess or we even > > > maintain > > > > a > > > > > minimalistic wrapper > > > > > around mprocs without the npm dependency. > > > > > My thought immediately goes to golang's embedded binaries > > > > > > > > Actually - (🤦) it's not an NPM app. I looked at the code. It's a > > > > (surprise, surprise) RUST app that has an NPM wrapper for easier > > > > installation only :D - that's what made me think it's npm.. That > would > > > > explain why it's so fast and nice :) > > > > > > > > So that is super easy. We don't need to "declare" mprocs as > dependency > > - > > > > when you look here: https://github.com/pvolok/mprocs/releases - > mprocs > > > > also > > > > publishes statically linked binaries which does not **require** npm > to > > be > > > > installed: > > > > > > > > So all we need to do when `--interactive` flag is used is to download > > the > > > > binary for the right architecture, decompress it to temporary > location > > > and > > > > run it from there (and we can cache it for later execution)., We > could > > > even > > > > copy and publish those binaries some place we control - somewhere in > > > > airflow.apache.org is good for that, similarly as we do with our > > chart - > > > > so > > > > that we do not have problems with GitHub rate limiting (and our > website > > > is > > > > already behind fastly cache, so it will be very fast to download it > by > > > > anyone). We could even - for maximum security, build those binaries > > > > ourselves from sources published by mprocs - > > > > https://github.com/pvolok/mprocs/tree/master/scripts has all the > > scripts > > > > needed to build those. > > > > > > > > J. > > > > > > > > > > > > > > > > On Wed, Dec 24, 2025 at 6:54 AM Amogh Desai <[email protected]> > > > wrote: > > > > > > > > > Hi Jarek, > > > > > > > > > > I think it is a good idea to try and integrate mprocs with > standalone > > > > > airflow. > > > > > > > > > > While we are at it, we should try to see if we can somehow wrap > > around > > > > the > > > > > dependency mprocs > > > > > needed in a way that users working with standalone airflow do not > > > suffer > > > > at > > > > > all. It can maybe be an > > > > > inherent dependency or we launch it using subprocess or we even > > > maintain > > > > a > > > > > minimalistic wrapper > > > > > around mprocs without the npm dependency. > > > > > > > > > > My thought immediately goes to golang's embedded binaries which we > > > could > > > > > probably leverage to > > > > > design our own using wrapper or similar, maybe check out: > > > > > > > > > > > > > > > > > > > > https://medium.com/dtoebe/embed-other-binaries-in-golang-binary-3f613314884c > > > > > > > > > > All in all, I think it's a good idea if it doesn't make user lives > > > harder > > > > > than it already is! > > > > > > > > > > Thanks & Regards, > > > > > Amogh Desai > > > > > > > > > > > > > > > On Wed, Dec 24, 2025 at 3:11 AM Natanel <[email protected]> > > > wrote: > > > > > > > > > > > Hello Jarek, I think that it is a great idea, it does look better > > and > > > > > seems > > > > > > to be more intuitive to use, when you have 1 screen at a time and > > you > > > > may > > > > > > change between them quickly and with ease, from experience, it > > would > > > > > > significantly improve the ease of use of a standalone airflow > > > instance. > > > > > > > > > > > > I think it will work well, and make it simpler to run airflow, > > > though, > > > > I > > > > > > think that Tmux is flexible enough to be able to reach a similar > > > > looking > > > > > UI > > > > > > with simple and intuitive controls, though it might require a lot > > of > > > > > trial > > > > > > and error, with configuration and community input in order to > > reach a > > > > > > simple and intuitive experience for a standalone airflow > instance. > > > > > > > > > > > > We might want to explore more solutions such as a more > > comprehensive > > > > Tmux > > > > > > configuration or maybe even Zellij, which is pretty similar to > > Tmux, > > > > and > > > > > > are both lightweight, and simple to install (though, as Jen said, > > it > > > > will > > > > > > make running airflow standalone a little harder). > > > > > > > > > > > > > I'd say it sounds like a good idea to have it as part of > > standalone > > > > > > airflow. One drawback it has that it has a dependency on npm, but > > > also > > > > it > > > > > > has a standalone binary, that Airflow Standalone **could** > download > > > > from > > > > > > https://github.com/pvolok/mprocs/releases and run internally. Or > > > maybe > > > > > > search for or even implement a simple version of it as an > > alternative > > > > in > > > > > > Python. > > > > > > > > > > > > I agree that using the npm version is not ideal, as it will add a > > > > > > dependency to airflow. > > > > > > > > > > > > A minimal version in python could also suffice, though it would > > take > > > > more > > > > > > time and add more code to maintain, though it would be the > lightest > > > > > weight > > > > > > option, and using the binary install is probably better, in order > > to > > > > not > > > > > > rely on npm, especially due to the last seen events on the npm > > > > repository > > > > > > where a lot of malware was pushed to relatively big packages. > > > > > > > > > > > > There are a few considerations here, of whether we should bundle > a > > > > > > multiplexer or something similar to ariflow, just like is done > with > > > > > airflow > > > > > > breeze, another option is always to open multiple new terminal > > > windows > > > > or > > > > > > tabs in the terminal, however, it might be too complex to do so > > with > > > > > > different working environments and people using all kinds of > > > different > > > > > > terminals (and fonts) (i.e having to support windows, macos, > xterm, > > > > > kitty, > > > > > > alacritty, Konsole, st and more), mprocs seems like a very good > > > > > candidate, > > > > > > but having to bundle it with airflow, could make the package > > heavier > > > > (or > > > > > > the installation more complex), and having an alternative python > > > > package > > > > > > will keep the airflow installation as simple and lightweight for > > > > > standalone > > > > > > as possible, the next step should the community decide to > implement > > > it > > > > in > > > > > > python, will be research on the api's of different terminals to > > check > > > > if > > > > > it > > > > > > is possible to have a simple implementation to cover most of the > > > > terminal > > > > > > emulators people are going to use. > > > > > > > > > > > > > > > > > > Over all, adding such a tool will improve the airflow standalone > > > > > experience > > > > > > and benefit a good chunk of airflow users (and contributors) (who > > may > > > > > want > > > > > > to check their dags quickly and locally), and so maybe it is > worth > > it > > > > to > > > > > > make the installation a little harder or require one additional > > > > > > *optional* dependency > > > > > > to improve the standalone experience. > > > > > > > > > > > > Such an addition would be awesome. > > > > > > > > > > > > On Tue, 23 Dec 2025 at 21:20, Dheeraj Turaga < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi Jarek! > > > > > > > > > > > > > > I like this idea. We use airflow standalone a lot and more > often > > > its > > > > > hard > > > > > > > to identify issues when the log is multiplexed onto the screen. > > > Also > > > > > new > > > > > > > users who are unaware that airflow drops a .json with a temp > > > account > > > > in > > > > > > > airflow home often find themselves scrambling to quickly copy > the > > > > auto > > > > > > > generated credentials before it vanishes into history. > > > > > > > > > > > > > > I can see instances where you may want to restart individual > > > services > > > > > > (dag > > > > > > > processor, etc) without having to kill the whole standalone run > > and > > > > > > launch > > > > > > > a new one. > > > > > > > > > > > > > > Having separate logs for each service and a way to manage each > > one > > > > > > > separately would be amazing > > > > > > > > > > > > > > I also agree with Jens here as to making the first time > > standalone > > > > run > > > > > > > working with the most minimum number of commands as possible > (pip > > > > > install > > > > > > > apache-airflow && airflow standalone) > > > > > > > > > > > > > > Adding these via additional options would be nice! > > > > > > > > > > > > > > Thanks > > > > > > > Dheeraj > > > > > > > > > > > > > > On Tue, Dec 23, 2025 at 2:42 PM Jarek Potiuk <[email protected] > > > > > > wrote: > > > > > > > > > > > > > > > All good points: > > > > > > > > > So in standaonle mode we would need to mound one "dummy > > > process" > > > > > > > > panel giving the key "getting started" information with > UI > > > URL > > > > > and > > > > > > > > admin user/password at least. > > > > > > > > > > > > > > > > Great idea > > > > > > > > > > > > > > > > > We should not make it harder for users for first-time user > > > > > > > > experience. Today after a `pip install apache-airflow` > you > > > are > > > > > > > > potentially good to go > > > > > > > > > > > > > > > > Yes. Likely we can leave standalone behaviour working as of > > today > > > > by > > > > > > > > default > > > > > > > > > > > > > > > > > We therefore might consider > > > > > > > > not naming it `--mprocs` but rather `--interactive` to > > > describe > > > > > the > > > > > > > > use case? > > > > > > > > > > > > > > > > I really like this flag idea. > > > > > > > > > > > > > > > > > On the neutral side though I am not sure if a first time > user > > > is > > > > > > > > "easier" with merged logs to see errors of if a first time > user > > > > might > > > > > > be > > > > > > > > overwhelmed with too many panels to scroll in to find an > error > > > not > > > > > > > > knowing which component maybe raising a stack trace at all. > > > > > > > > > > > > > > > > Yep. With `--interactive` flag as non-default this issue > would > > > also > > > > > be > > > > > > > > handled. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
