ryanahamilton commented on code in PR #76932:
URL: https://github.com/apache/airflow/pull/76932#discussion_r4618257317
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/event_logs.py:
##########
@@ -228,40 +259,86 @@ def get_event_logs(
if region_index is not None:
attempt_ids = attempt_ids.where(attempts.c.region_index ==
region_index)
query = query.where(Log.task_instance_id.in_(attempt_ids))
+ filters: list[OrmClause] = [
+ # Exact match filters
+ dag_id,
+ task_id,
+ run_id,
+ map_index,
+ try_number,
+ task_instance_id,
+ owner,
+ event,
+ excluded_events,
+ included_events,
+ before,
+ after,
+ # Pattern search filters
+ dag_id_pattern,
+ dag_id_prefix_pattern,
+ task_id_pattern,
+ task_id_prefix_pattern,
+ run_id_pattern,
+ run_id_prefix_pattern,
+ owner_pattern,
+ owner_prefix_pattern,
+ owner_display_name_pattern,
+ owner_display_name_prefix_pattern,
+ event_pattern,
+ event_prefix_pattern,
+ teams,
+ # Permission
+ readable_event_logs_filter,
+ ]
+
+ if cursor is not None:
Review Comment:
Not for this PR, but this block is now the third near-identical copy of the
cursor flow (alongside `dag_run.py` and `task_instances.py`): limit + 1 fetch,
backward reorder, `has_prev`/`has_next`, cursor encoding, bounded count. Might
be worth a follow-up that pulls it into a shared helper in `common/cursors.py`
returning `(rows, next_cursor, previous_cursor, total_entries,
total_entries_limit)`, so fixes like the `owner_display_name` encoding order
only need to land once.
---
Drafted-by: Claude Code (Opus 5.5); reviewed by @ryanahamilton before posting
##########
airflow-core/src/airflow/ui/src/pages/Events/Events.test.tsx:
##########
@@ -16,7 +16,8 @@
* specific language governing permissions and limitations
* under the License.
*/
-import { render, waitFor } from "@testing-library/react";
+import "@testing-library/jest-dom/vitest";
Review Comment:
Nit: `testsSetup.ts` already imports `@testing-library/jest-dom/vitest` for
every test file, so this import can go (same in `SearchMapIndexes.test.tsx`).
---
Drafted-by: Claude Code (Opus 5.5); reviewed by @ryanahamilton before posting
##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/event_logs.py:
##########
@@ -228,40 +259,86 @@ def get_event_logs(
if region_index is not None:
attempt_ids = attempt_ids.where(attempts.c.region_index ==
region_index)
query = query.where(Log.task_instance_id.in_(attempt_ids))
+ filters: list[OrmClause] = [
+ # Exact match filters
+ dag_id,
+ task_id,
+ run_id,
+ map_index,
+ try_number,
+ task_instance_id,
+ owner,
+ event,
+ excluded_events,
+ included_events,
+ before,
+ after,
+ # Pattern search filters
+ dag_id_pattern,
+ dag_id_prefix_pattern,
+ task_id_pattern,
+ task_id_prefix_pattern,
+ run_id_pattern,
+ run_id_prefix_pattern,
+ owner_pattern,
+ owner_prefix_pattern,
+ owner_display_name_pattern,
+ owner_display_name_prefix_pattern,
+ event_pattern,
+ event_prefix_pattern,
+ teams,
+ # Permission
+ readable_event_logs_filter,
+ ]
+
+ if cursor is not None:
+ page_limit = cast("int", limit.value)
+ cursor_limit = LimitFilter().set_value(page_limit + 1)
+ event_logs_select = apply_filters_to_select(statement=query,
filters=[*filters, cursor_limit])
+ event_logs_select = order_by.to_orm(event_logs_select)
+
+ is_backward = False
+ if cursor:
+ token, is_backward = parse_cursor(cursor)
+ if is_backward:
+ event_logs_select = order_by.to_orm(event_logs_select,
reversed=True)
+ event_logs_select = apply_cursor_filter(
+ event_logs_select, token, order_by,
session.get_bind().dialect.name, is_backward=is_backward
+ )
+
+ fetched = list(session.scalars(event_logs_select))
+ has_more = len(fetched) > page_limit
+ page = fetched[:page_limit]
+
+ if is_backward:
+ page.reverse()
+ has_prev = has_more
+ has_next = True
+ else:
+ has_prev = bool(cursor)
+ has_next = has_more
+
+ # Encode before event_log_to_response, which overwrites
owner_display_name on the row
+ # with a fallback, so a cursor built afterwards would not match the
stored sort value.
+ next_cursor = encode_cursor(page[-1], order_by) if has_next and page
else None
Review Comment:
`extra` is still a valid `order_by` value, and in cursor mode the token
encodes every sort value, so `order_by=extra&cursor=` returns a `next_cursor`
that carries the row's full `extra` payload. That column is unbounded `Text`,
so the cursor can outgrow request-line limits in proxies and uvicorn when it's
sent back, and the audit payload lands in access logs.
On MySQL there may also be an ordering mismatch: `ORDER BY` on TEXT only
compares the first `max_sort_length` bytes, while the keyset `WHERE` compares
the full value, so rows sharing a long prefix could be skipped or repeated
across pages (I haven't reproduced this one).
The UI doesn't sort by `extra`, so this only affects API clients. Could we
reject `extra` with a 400 when `cursor` is set, or drop it from the sortable
fields for cursor mode?
---
Drafted-by: Claude Code (Opus 5.5); reviewed by @ryanahamilton before posting
##########
airflow-core/src/airflow/ui/src/layouts/Details/SearchMapIndexes.test.tsx:
##########
@@ -16,6 +16,7 @@
* specific language governing permissions and limitations
* under the License.
*/
+import "@testing-library/jest-dom/vitest";
Review Comment:
Nit: same here, `jest-dom/vitest` is already loaded by `testsSetup.ts`.
---
Drafted-by: Claude Code (Opus 5.5); reviewed by @ryanahamilton before posting
##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_event_logs.py:
##########
@@ -829,3 +829,76 @@ def test_non_dag_rows_are_gated_on_audit_logs_all(
mock_is_authorized_view.assert_called_once_with(
access_view=AccessView.AUDIT_LOGS_ALL, user=mock.ANY,
team_name=None
)
+
+ @pytest.mark.parametrize("order_by", ["-when", "event", "owner",
"-event_log_id"])
+ def test_cursor_pagination_forward_then_previous_page(self, test_client,
order_by):
+ """Every row is reached once going forward, and previous_cursor
retraces the same pages."""
+ forward_pages = []
+ cursor = ""
+ while cursor is not None:
+ response = test_client.get(
+ "/eventLogs", params={"limit": 3, "order_by": order_by,
"cursor": cursor}
+ )
+ assert response.status_code == 200, response.json()
+ body = response.json()
+ forward_pages.append(body)
+ cursor = body["next_cursor"]
+
+ forward_ids = [log["event_log_id"] for page in forward_pages for log
in page["event_logs"]]
+ assert len(forward_pages) == 2
Review Comment:
With 4 rows and `limit=3` the walk only produces two pages, so the backward
cursor is only exercised from the last page back to the first. Could we add a
fifth row (or use `limit=2`) so there's a middle page, and check that going
back from it returns the right rows with both `previous_cursor` and
`next_cursor` set?
---
Drafted-by: Claude Code (Opus 5.5); reviewed by @ryanahamilton before posting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]