1fanwang commented on PR #70354: URL: https://github.com/apache/airflow/pull/70354#issuecomment-5073082310
Closing this — on reflection it doesn't earn a core extension point. Airflow already forks a fresh process per file to parse (the "prevent COW in forked process parsing dag file" path in the manager), and it already supports running a DAG processor per bundle via `--bundle-name`, each in its own environment. That gives true isolation, including bundles that need *conflicting* versions of the same package — which a shared-interpreter `sys.path` prepend fundamentally cannot (you can't import two versions of one top-level package in one interpreter). So `parse_pythonpath` only covers the additive "module missing from the base env" subset that the per-processor model already handles, at the cost of nudging users toward a shared interpreter over separate ones. I don't have an end-to-end case showing it solves something the existing mechanism can't, so it's not the right shape to add to core. If a concrete need surfaces that per-processor isolation genuinely can't meet, I'll reopen with a reproducer. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
