shahar1 opened a new pull request, #74923:
URL: https://github.com/apache/airflow/pull/74923
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
related: #68240, #68293
<!-- HUMAN SUMMARY -->
On Airflow 3, the Cloud Logging reader now filters by `run_id` and
`map_index` to match the labels written from the task process context, and
supports runs without a logical date. Airflow 2 keeps using `execution_date`
and is unaffected. For map index 0, the filter also matches older entries
without a `map_index` label. This PR fixes three reader bugs:
- Task logs not appearing on Airflow 3.2.2–3.3.x
- An `AttributeError` for runs without a logical date
- Mapped tasks sharing one filter before Airflow 3.4
<!-- END OF HUMAN SUMMARY -->
<details><summary>AI Summary</summary>
Since Airflow 3.2.2, `CloudLoggingRemoteLogIO` labels each entry with the
values the task process binds into its log context: `ti_id`, `dag_id`,
`task_id`, `run_id`, `try_number` and `map_index`. The reader still built its
filter from a `logical_date` label, which no Airflow 3 writer sets. As a result:
- **Airflow 3.2.2 to 3.3.x:** the reader matches nothing. Task logs written
to Cloud Logging never show in the UI.
- **Any Airflow 3 version:** reading logs for a Dag run without a logical
date fails with `AttributeError: 'NoneType' object has no attribute
'isoformat'`.
- **Mapped tasks before 3.4:** the filter has no `map_index`, so all map
indexes would share one filter.
From 3.4 on, #74204 matches entries by `ti_id`. The label fallback in this
PR covers older Airflow versions, and runs without a logical date on every
version.
The reader now filters on `run_id` and `map_index` on Airflow 3. Airflow 2
still uses `execution_date`, which is what its handler writes. The writer used
to skip `map_index` 0 because the check was truthy, so the first mapped task's
existing entries have no `map_index` label. The filter for map index 0 also
accepts entries that have no `map_index` label.
On Airflow 3 nothing calls `set_context`, so `CloudLoggingTaskHandler` never
wrote `logical_date` labels for task logs there. Dropping `logical_date` from
the Airflow 3 filter makes no existing entry unreadable.
Known limitations, none of them regressions:
- **Airflow 3.0.0 to 3.2.1:** the task process binds no task context, so
entries from those versions carry no task labels and stay unreadable.
- **Supervisor's own lines:** lines the supervisor logs itself carry only
`ti_id`, so before Airflow 3.4 they are still not matched.
- **Downloading all tries on 3.4:** this path drops the `ti_id` condition.
It relies on `map_index`, which can differ from the stored index for region
tasks.
This picks up the part of #68293 that #74204 did not cover, and builds on
that PR's investigation.
</details>
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Opus 5.5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
