Rodrigo-Palma opened a new pull request, #4001:
URL: https://github.com/apache/iceberg-python/pull/4001
# Rationale for this change
`parse()` hands the expression straight to pyparsing, which expands tabs in
the input before matching. A tab inside a string literal is therefore rewritten
into spaces before the literal is ever read, and because the expansion runs
over the whole expression, the number of spaces depends on where the literal
happens to sit:
```python
>>> from pyiceberg.expressions.parser import parse
>>> parse("a = 'x\ty'").literal.value
'x y'
>>> parse("ab = 'x\ty'").literal.value
'x y'
```
Two filters written against the same stored value produce two different
literals, and neither is the value the user asked for. A row whose column
really contains a tab is never matched, and nothing reports an error:
`row_filter="col = 'a\tb'"` simply returns no rows.
pyparsing exposes `parse_with_tabs()` precisely for this, and it is set once
on the top-level element rather than per call.
## Are these changes tested?
Yes, two tests in `tests/expressions/test_parser.py`: one asserting that a
tab survives the round trip, and one asserting that the same literal parses
identically from two positions, which is the part that makes the corruption
silent. Both fail on `main` at `0d58407` and pass with the change.
Ran locally: `make lint` (ruff, ruff-format, mypy, pydocstyle, codespell,
uv-lock) all green; `tests/expressions` 584 passed, `tests/table` 512 passed,
`tests/utils` 203 passed, `tests/io/test_io.py` 30 passed. Not run here:
`tests/io/test_fsspec.py`, which needs a moto server on `localhost:9000`, and
`tests/avro/test_decoder.py`, whose `CythonBinaryDecoder` cases fail on a clean
`main` in this checkout because the Cython extension is stale.
## Are there any user-facing changes?
Yes, in the sense that a filter over a string containing a tab now matches
the rows it names. No public signature changes, and expressions without tabs
parse exactly as before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
