m8719-github commented on issue #60064: URL: https://github.com/apache/airflow/issues/60064#issuecomment-3707442393
My approach for #60056 was to have lowest footprint solution, reusing as much as possible of what's already there. However, I do agree that it's likely not the best solution due to the fact that these template var callables can be arbitrarily large and shoving all of their source code into a serialized DAG structure is probably not the best idea; for example: our production DAGs with KPO tasks require a large list of K8s pod annotations to be injected for integration with our Kubernetes observability platform, we inject these annotations via a callable and its source code representation is not small, unnecessarily bloating serialized DAG structure. I do like what's been mentioned in this [comment](https://github.com/apache/airflow/discussions/59595#discussioncomment-15373896) where the approach would be to hash the source code and have a compact representation in the serialized DAG, something like `callable <MD5_hash_value_of_source_code>` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
