jason810496 commented on PR #71213: URL: https://github.com/apache/airflow/pull/71213#issuecomment-5313486776
> I think there are two issues. First of all, why does a language sdk dag need to define this at dag-level? From Airflow Core’s prespective, such a dag is essentially equivalent to a Python-defined dag that only contains stub tasks. Why can’t the dag processor just emit task-level markers? > Second, even if we accept a dag-level flag for this use case, `is_mixed_language_dag` is completely backwards. A Java dag that defines only Java tasks is entirely the opposite of “mixed” since there’s only Java. The mixed Lang artifact is **parseable** once both https://github.com/apache/airflow/pull/70805 (support optional DagBundle based deployment) and the DagImporter interface (JarImporter, able to get Dags from Jars) get merged. Without the flag, consider the case where we place the Python Dag with stub operator in one DagBundle and the Jar artifact in another DagBundle, the DagModel will jump between mixed Lang Dag and native Dag. Since the Jar artifact for mixed Lang itself is also a valid native Dag Jar. From my perspective, it's necessary to introduce the flag at the Dag-level to make the purpose explicit, so that the DagImporters are able to distinguish between the mixed Lang Jar and the native Dag Jar. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
