github-actions[bot] opened a new pull request, #74955:
URL: https://github.com/apache/airflow/pull/74955
* Show deadline callback logs in a Callbacks tab on the Dag run view
Add a Callbacks tab to the Dag run view, shown only when the run has
deadline callbacks. It lists them (path, type, state, deadline alert,
deadline time, created at) from the existing deadlines endpoint, which
now also returns each deadline's callback id, type, state and path, and
links each one to its logs, rendered with the task instance log viewer.
The new GET
/ui/dags/{dag_id}/dagRuns/{dag_run_id}/callbacks/{callback_id}/logs
reads a callback's logs from remote storage or the local log folder. The
triggerer now writes async callback output to triggerer_callbacks/... so
those logs can be read too.
* Show deadline and alert details above callback logs
Above a callback's logs, show the deadline alert and its rule, the
deadline time, how late the run was, and the callback type, state and
path, all from the existing deadlines and deadline alerts endpoints.
In the Callbacks tab, replace the Created At column with Missed by and
show callback types as Sync/Async.
* Show the callback's row from the Callbacks table above its logs
* Even out spacing above the callback log header
* Trim callback log UI and tests
* Explain why the callback log reader prefers stream() over read()
* Make the callback log reader generic
Replace read_callback_log with read_logs_at_paths, which reads logs not
tied to a TaskInstance from the given paths. The callback logs route now
builds the executor/triggerer callback paths itself.
* Type callback columns for react-table v9
* Give each triggerer callback its own log directory
With [logging] delete_local_logs=True, the S3, GCS, OSS, CloudWatch and
Elasticsearch handlers delete the uploaded log file's parent directory.
Triggerer callback logs sat directly in the per-Dag-run directory, so the
first callback to finish deleted the local logs of every other callback
still running in the same Dag run, and those were never uploaded.
* Expand callback logs in place in the Callbacks tab
Clicking Logs on a callback now opens its logs in a row under it,
instead of going to a separate page that repeated the callback's row.
That page also looked the callback up in the first page of the list,
so it lost the summary for callbacks past that page; it is removed.
DataTable gains optional getRowId and renderSubComponent props, using
TanStack's row expanding feature.
* Drop the isFetching prop from the Callbacks table
main removed isFetching from DataTable in #74426 ("Drop isFetching from
tables"), which this branch's new Callbacks tab still passed, so tsc
failed after the merge with TS2322 on Callbacks.tsx:154.
* Read callback logs only at the path for the callback's type
CloudWatch returns a lazy stream for any log stream name and reports a
missing
stream as a log line, so trying executor_callbacks/ first always won and
async
(triggerer) callback logs showed "No log stream found" instead of their log.
* Let remote log handlers read logs without a task instance
Widen RemoteLogIO.read and RemoteLogStreamIO.stream to accept ti=None,
matching
upload, so callback logs no longer need a type: ignore. Elasticsearch,
OpenSearch
and Cloud Logging key logs by task instance, so they now return no logs for
ti=None instead of raising AttributeError.
* Fix callback log tests on SQLite by passing a UUID to the callback id
filter
The Uuid column's SQLite bind processor rejects a str, so set_callback_type
failed with "'str' object has no attribute 'hex'".
(cherry picked from commit 71e74bc9554bd8783050c662af2881e70267ad22)
Co-authored-by: Sean Ghaeli <[email protected]>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
