rahulsmahadev opened a new pull request, #4372:
URL: https://github.com/apache/iceberg-python/pull/4372

   # Rationale for this change
   
   A float filter on a wide decimal column raises `decimal.InvalidOperation`:
   
   ```python
   # amt is decimal(38, 2)
   tbl.scan(row_filter=GreaterThan("amt", 1e26)).to_arrow()
   # decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
   
   # amt is decimal(38, 30)
   tbl.scan(row_filter=LessThan("amt", 0.5)).to_arrow()
   # decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
   ```
   
   `FloatLiteral.to(DecimalType)` and `DoubleLiteral.to(DecimalType)` quantize 
the value to the column's scale under the default decimal context, which keeps 
28 digits. When the quantized value has more digits than that (`1e26` with 2 
decimals, or `0.5` with 30 decimals), `quantize` raises. A decimal column can 
hold up to 38 digits, so these are valid filters.
   
   This change quantizes under a local context with enough precision for the 
result, in one helper used by both conversions. Rounding is unchanged 
(`ROUND_HALF_UP`), and positive and negative infinity still raise the 
`ValueError` added in #4287 for #4286. NaN remains rejected when a literal is 
constructed.
   
   ## Are these changes tested?
   
   Yes. `test_float_to_decimal_conversion_beyond_default_context_precision` in 
`tests/expressions/test_literals.py` converts `2.0**87` to `decimal(38, 2)` and 
`0.5` to `decimal(38, 30)` from both a `FloatLiteral` and a `DoubleLiteral`. It 
fails on main with `decimal.InvalidOperation` and passes with this change.
   
   ## Are there any user-facing changes?
   
   Bug fix: float filters on wide decimal columns no longer raise 
`decimal.InvalidOperation`. No API changes.
   
   This pull request and its description were written by Isaac.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to