kevinjqliu opened a new pull request, #3547:
URL: https://github.com/apache/iceberg-python/pull/3547
## Summary
Fix strict metrics evaluation for `NotEqualTo` and `NotIn` so files are only
proven to match when a column contains only nulls or only NaNs. Mixed null/NaN
files now continue through the existing bounds checks instead of being treated
as `ROWS_MUST_MATCH`.
## Root Cause
The strict evaluator used `_can_contain_nulls` / `_can_contain_nans` for
negative predicates. That is too broad: a file with values like `[null, 5]` and
bounds `5..5` cannot be proven to match `x != 5` or `x not in {5}` because the
non-null row may still fail the predicate.
## Validation
- `UV_CACHE_DIR=.cache/uv PYTHON_GIL=1 PYTHONPATH=. uv run pytest
tests/expressions/test_evaluator.py -k "mixed_nulls_and_matching_bounds or
mixed_nans_and_matching_bounds or all_nulls or all_nans or
strict_integer_not_in"`
- `UV_CACHE_DIR=.cache/uv PYTHON_GIL=1 PYTHONPATH=. uv run pytest
tests/expressions/test_evaluator.py`
- `UV_CACHE_DIR=.cache/uv PYTHON_GIL=1 PYTHONPATH=. uv run ruff check
pyiceberg/expressions/visitors.py tests/expressions/test_evaluator.py`
- `git diff --check`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
