jason810496 opened a new pull request, #69520: URL: https://github.com/apache/airflow/pull/69520
- related: #53821, #59027 (stale predecessor of this PR), [[DISCUSS] Drop LogTemplate DB Model for ElasticsearchTaskHandler in Airflow 3](https://lists.apache.org/thread/nlmhs1plo77qnlp7rqk27mkb2hs41f1p) ## Why The `LogTemplate` model pinned historical values of `[logging] log_filename_template` / `[elasticsearch] log_id_template` per `DagRun`, and task handlers read it via `dag_run.get_log_template(session=...)` — a direct metadata-DB dependency that conflicts with the Airflow 3 execution model and couples log handlers to an Airflow-core model just to read two strings already available from config. Airflow 2.11.1 already disabled per-run template retrieval by default for security reasons (#61880); this completes that direction on main by removing the model entirely. ## What - Remove the `LogTemplate` model (`airflow/models/tasklog.py`), the `DagRun.log_template_id` column/FK, and `DagRun.get_log_template()` / `_get_log_template()`. - Remove `synchronize_log_template()` from `airflow.utils.db` and its call sites (`initdb`, `upgradedb`, CLI `check_db` path). - `FileTaskHandler._render_filename()` now reads `[logging] log_filename_template` from live config at render time instead of the per-run pinned value. - Add migration `0124_3_4_0_drop_log_template.py`: drops the `dag_run.log_template_id` FK/column and the `log_template` table (downgrade recreates them). Both directions are wrapped in `disable_sqlite_fkeys(op)` since the `dag_run` rebuild on SQLite would otherwise cascade-delete `task_instance` rows. - Elasticsearch handler: remove the `USE_PER_RUN_LOG_ID` flag — it was defined but never used (ES stopped doing per-run log ID pinning when `ElasticsearchRemoteLogIO` was introduced), so this is dead-code cleanup with no behavior change on any core. - OpenSearch handler: keep the `USE_PER_RUN_LOG_ID = hasattr(DagRun, "get_log_template")` compatibility shim — on supported cores 3.0-3.3 per-run pinning still exists and must keep working on a provider-only upgrade; on 3.4.0+ the attribute is gone so the shim self-disables and the handler uses `[opensearch] log_id_template` from config. - Make the shared `create_log_template` test fixture version-adaptive: it always sets the filename-template config env var and additionally seeds the `log_template` table when the model exists, so the same provider tests pass against any supported core (compat CI runs them against 3.0/3.1/3.2). - Update ES provider docs (drop the `log_template`-table recovery instructions) and add a significant-change newsfragment. **Behaviour change (core 3.4.0+):** changing `log_filename_template` / `log_id_template` now applies immediately and retroactively to all DagRuns — there is no more per-run historical pinning, so logs written under a previous template are no longer reachable through the new one. --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes, with help of Claude Code (Fable 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
