shahar1 opened a new pull request, #74926:
URL: https://github.com/apache/airflow/pull/74926
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
related: #68240
<!-- HUMAN SUMMARY - AGENTS DO NOT MODIFY -->
When the Dag processor log directory is outside `base_log_folder`, Google
Cloud Logging raises `ValueError` and crashes before parsing the DAG.
This catches the error and keeps those lines local; verified on Airflow
3.3.2, where the Dag parsed and its triggered run succeeded.
AWS CloudWatch has the same issue, but it isn’t fixed here.
<!-- END OF HUMAN SUMMARY -->
<details><summary>AI Summary</summary>
With Google Cloud Logging remote logging, the Dag processor crashes on its
first log line when `[logging] dag_processor_child_process_log_directory`
points outside `[logging] base_log_folder`:
```
dag-processor | [error] Exception when executing DagProcessorJob
File ".../cloud_logging_task_handler.py", line 180, in proc
if not logger or not relative_path_from_logger(logger):
ValueError: '/tmp/dagproc-logs/2026-10-17/dags-folder/repro_68293.py.log' is
not in the subpath of '/opt/airflow/logs' ...
```
`CloudLoggingRemoteLogIO.processors` calls `relative_path_from_logger`,
which uses `Path.relative_to` and raises `ValueError` for files outside
`base_log_folder`. Nothing on the way up catches it, so the exception ends the
Dag processor job and no Dag is parsed. `upload_to_remote` already skips such
files. The processor now does the same: it uses a wrapper that returns `None`
on `ValueError`, so these lines stay in the local log file and are not sent to
Cloud Logging.
The guard sits where `relative_path_from_logger` is imported, not on the
processor's early-return line, so this merges cleanly with #74923, which
changes that line.
Verified on the official Airflow 3.3.2 image writing to Cloud Logging, with
the Dag processor log directory outside `base_log_folder`:
- **Provider built from main:** the Dag processor job fails with the
`ValueError` above and no Dag is parsed.
- **This PR:** the Dag processor keeps running, the Dag is parsed, and a
triggered run succeeds with its task logs in Cloud Logging.
This may also be what #68240 describes as a Dag processor in
`CrashLoopBackOff` "on every log emit". That report attributed it to an IAM
error in `transport.send()`, which the default transport cannot raise.
Not covered here: the Amazon CloudWatch remote log IO
(`cloudwatch_task_handler.py`) calls `relative_path_from_logger` the same way
and has the same crash.
</details>
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Opus 5.5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
