rahulsmahadev opened a new pull request, #4371: URL: https://github.com/apache/iceberg-python/pull/4371
# Rationale for this change The row filter parser rejects two kinds of valid numeric literals: ```python tbl.scan(row_filter="price > 1e-3") # ParseException: Expected end of text, found 'e' (at char 9), (line:1, col:10) tbl.scan(row_filter="quantity in (1, 2.5)") # ParseException: Expected ')', found '.' (at char 17), (line:1, col:18) ``` Decimal literals used pyparsing's `common.real()`, which has no exponent, so `1e-3` parses as `1` and the rest is left over. A literal set had to be all decimals or all integers, so a list that mixes `1` and `2.5` did not parse either. This change uses `common.sci_real()`, which accepts everything `common.real()` does plus an exponent (`1e-3`, `-1.5E3`, `2.5e1`), and lets a literal set mix integers and decimals. Integers become `LongLiteral`s and decimal or exponent forms become `DecimalLiteral`s; binding uses the existing conversion rules (a decimal column needs a literal with its scale, and an integer column rejects fractional values). The decimal parse action already builds the `Decimal` from the matched text, so exponents are exact. A list that mixes numbers with strings still raises a `ParseException` (`test_in_different_types`). ## Are these changes tested? Yes. `test_scientific_notation` and `test_in_mixed_numbers` in `tests/expressions/test_parser.py` cover exponents on either side of a comparison and in `BETWEEN`, and mixed integer/decimal `IN` lists (in both orders) and `NOT IN` lists. Both fail on main with the `ParseException`s above and pass with this change. ## Are there any user-facing changes? Bug fix: row filter strings accept scientific notation and `IN` lists that mix integers and decimals. Expressions that parsed before parse to the same result. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
