jason810496 opened a new issue, #70267:
URL: https://github.com/apache/airflow/issues/70267
Part of #70265 (related: #67056).
## Why
#67056 decoupled remote logging from the hardcoded branches in
`airflow_local_settings.py`: core and
the Task SDK now resolve the handler via `ProvidersManager` dispatch on the
`[logging] remote_base_log_folder` URL scheme, instantiating the provider
class through a no-arg
`from_config()` classmethod. This issue migrates the `stackdriver` scheme.
## What
- [ ] Add `StackdriverRemoteLogIO.from_config()` in
`providers/google/src/airflow/providers/google/cloud/log/stackdriver_task_handler.py`,
mirroring the legacy branch:
https://github.com/apache/airflow/blob/104ad12e190db7197fa303d98a3ab68879eafd33/airflow-core/src/airflow/config_templates/airflow_local_settings.py#L250-L268
— including the `[logging] remote_task_handler_kwargs` IO-kwargs merge
and `expanduser` on
`base_log_folder`, so behavior is unchanged for existing configs.
- [ ] Register the `stackdriver` scheme under a `remote-logging:` section in
`providers/google/provider.yaml` and mirror it in
`providers/google/src/airflow/providers/google/get_provider_info.py`.
- [ ] Add tests mirroring `TestS3RemoteLogIOFromConfig` in
`providers/amazon/tests/unit/amazon/aws/log/test_s3_task_handler.py`.
- [ ] Verify end to end with a real system test: set up the actual backend
yourself (a real
service or a local equivalent), run a task with remote logging
enabled, and confirm logs are
uploaded and read back through the new dispatch path. Include the
setup and verification
results in the PR description.
## Notes
- The legacy branch parses the log name from the URL path:
`stackdriver:///airflow-tasks` →
`gcp_log_name="airflow-tasks"` (`urlsplit(...).path[1:]`). Mirror this in
`from_config`, with a
clear `ValueError` when no log name can be derived (see how #69816 handles
`log_group_arn`
parsing for `cloudwatch`).
- It also reads `[logging] google_key_path` (fallback `None`) into
`gcp_key_path`, and passes no
`remote_base` — unlike the other object-storage backends.
## Reference
Merged examples to follow: #69817 (s3), #69816 (cloudwatch). If
`from_config` raises on a bad
config, the shared factory falls back to the legacy path, so this is not a
breaking change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
