steveahnahn opened a new pull request, #69675:
URL: https://github.com/apache/airflow/pull/69675
Asset event `extra` filtering (public API `GET /assets/events` `extra`
param, and the Execution API asset-event lookups) uses a dialect-aware JSON
containment filter. The SQLite fallback built an **unquoted** `json_extract`
path from the user-supplied key, so a key containing JSON-path metacharacters —
e.g. `spark.executor.memory` — was interpreted as **nested-object navigation**
instead of a literal key. Two silent wrong results follow, diverging from
PostgreSQL (`@>`) and MySQL (`JSON_CONTAINS`), which both match keys literally:
- an event with the literal key `{"spark.executor.memory": "4g"}` is
**missed** (false negative), and
- an event nesting the same path `{"spark": {"executor": {"memory": "4g"}}}`
is **wrongly matched** (false positive).
Same filter, different events returned depending on the database backend —
with no error. Both `key=value` parsers pass such keys through verbatim, so
this is reachable from both API surfaces.
The fix quotes the key in the JSON path (`$."spark.executor.memory"`), which
SQLite treats literally; `json.dumps` produces exactly the quoting/escaping the
JSON path syntax expects (also covering keys containing quotes or backslashes).
PostgreSQL and MySQL code paths are untouched.
**Evidence**
- Fail-first on SQLite: the new bracket-key case fails unfixed (returns 0
events instead of 1), and the identity test fails unfixed by returning the
*nested* event instead of the literal-key event. Notably, the dotted-key
*count* assertion alone passes unfixed because the wrong event coincidentally
keeps the count at 1 — hence the added identity assertion on `source_run_id`.
- Backend differential: the same identity test on unfixed code passes on
PostgreSQL and fails on SQLite; with the fix both agree. Verified on a real
PostgreSQL backend via breeze: the unfixed code passes all four new tests on
PostgreSQL while two of them fail on SQLite; with the fix, all four pass on
both backends.
- Regression: full `test_assets.py` (151), execution API
`test_asset_events.py` (35), `test_sqlalchemy.py` (23) all pass.
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes (please specify the tool below)
- Claude Code (Fable 5)
Generated-by: Claude Code (Fable 5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
