ankitdhandharia opened a new pull request, #41427: URL: https://github.com/apache/beam/pull/41427
Fixes #41347 The Feast online feature store enrichment handler had three independent correctness bugs, all in `sdks/python/apache_beam/transforms/enrichment_handlers/feast_feature_store.py`: 1. **`get_cache_key` crashed with `entity_row_fn`.** It took the entity-key *name* returned by `entity_row_fn` and then re-indexed the input row with it (`request._asdict()[entity_id]`). When `entity_row_fn` maps a row column to a differently-named entity key — its whole purpose — that name isn't a column on the row, so caching raised `KeyError`. It now builds the key from the key/value pairs the function returns (sorted, so compound multi-entity mappings are stable and collision-free), matching how `bigtable.py`'s handler keys off its row-key function. 2. **`exception_level` was ignored.** The constructor stored `exception_level` but `__call__` never consulted it, so missing/`None` feature values were silently returned. `__call__` now warns or raises according to `ExceptionLevel` (entity-key columns echoed back by Feast are excluded from the "missing" check), following the same pattern as the Bigtable handler. 3. **Downloaded config YAML files leaked.** `download_fs_yaml_file` creates a `NamedTemporaryFile(delete=False)` but the handler never deleted it. The path is now tracked on the instance and removed in `__exit__` and on feature-store construction failure. ## Testing Added focused unit tests in `feast_feature_store_test.py` that mock out GCS and Feast (CPU-only, no network): a cache-key test with a mapping `entity_row_fn` and a compound mapping, tests that `exception_level=RAISE`/`WARN` fire when feature values are missing (and do not fire when present), and a test that the downloaded YAML is removed on teardown. Run with `apache-beam` installed editable (`pip install -e .`) plus `feast`, in `sdks/python`: Before the fix (the five new regression tests fail for the reasons in the issue — `KeyError` from `get_cache_key`, no raise/warn on missing features, and `_local_repo_path` never recorded/removed): ``` $ python -m pytest apache_beam/transforms/enrichment_handlers/feast_feature_store_test.py::TestFeastFeatureStoreHandlerUnit -q 5 failed, 1 passed ``` After the fix: ``` $ python -m pytest apache_beam/transforms/enrichment_handlers/feast_feature_store_test.py -v -k "mapped_entity_row_fn or features_missing or removes_downloaded or level_raise or level_warn or compound" 5 passed, 5 deselected $ python -m pytest apache_beam/transforms/enrichment_handlers/feast_feature_store_test.py -q 10 passed ``` `yapf==0.43.0 --diff` and `pylint` (10.00/10) are clean on both changed Python files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
