rahulsmahadev opened a new pull request, #4371:
URL: https://github.com/apache/iceberg-python/pull/4371

   # Rationale for this change
   
   The row filter parser rejects two kinds of valid numeric literals:
   
   ```python
   tbl.scan(row_filter="price > 1e-3")
   # ParseException: Expected end of text, found 'e'  (at char 9), (line:1, 
col:10)
   tbl.scan(row_filter="quantity in (1, 2.5)")
   # ParseException: Expected ')', found '.'  (at char 17), (line:1, col:18)
   ```
   
   Decimal literals used pyparsing's `common.real()`, which has no exponent, so 
`1e-3` parses as `1` and the rest is left over. A literal set had to be all 
decimals or all integers, so a list that mixes `1` and `2.5` did not parse 
either.
   
   This change uses `common.sci_real()`, which accepts everything 
`common.real()` does plus an exponent (`1e-3`, `-1.5E3`, `2.5e1`), and lets a 
literal set mix integers and decimals. Integers become `LongLiteral`s and 
decimal or exponent forms become `DecimalLiteral`s; binding uses the existing 
conversion rules (a decimal column needs a literal with its scale, and an 
integer column rejects fractional values). The decimal parse action already 
builds the `Decimal` from the matched text, so exponents are exact. A list that 
mixes numbers with strings still raises a `ParseException` 
(`test_in_different_types`).
   
   ## Are these changes tested?
   
   Yes. `test_scientific_notation` and `test_in_mixed_numbers` in 
`tests/expressions/test_parser.py` cover exponents on either side of a 
comparison and in `BETWEEN`, and mixed integer/decimal `IN` lists (in both 
orders) and `NOT IN` lists. Both fail on main with the `ParseException`s above 
and pass with this change.
   
   ## Are there any user-facing changes?
   
   Bug fix: row filter strings accept scientific notation and `IN` lists that 
mix integers and decimals. Expressions that parsed before parse to the same 
result.
   
   This pull request and its description were written by Isaac.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to