Shahzeb-DataEngineer opened a new issue, #4441:
URL: https://github.com/apache/iceberg-python/issues/4441

   ### What happens?
   `FloatLiteral.to(DecimalType)` and `DoubleLiteral.to(DecimalType)` convert 
the float with `Decimal(self.value)`, which uses the exact binary expansion of 
the double. Values whose decimal spelling ends in 5 then round the wrong way 
under `ROUND_HALF_UP`:
   
   ```python
   from pyiceberg.expressions.literals import literal
   from pyiceberg.types import DecimalType
   
   literal(2.675).to(DecimalType(10, 2))   # 2.67, expected 2.68
   literal(1.005).to(DecimalType(10, 2))   # 1.00, expected 1.01
   literal(-2.675).to(DecimalType(10, 2))  # -2.67, expected -2.68
   ```
   
   The wrong value leaks into predicate binding, so a filter on a decimal 
column uses the wrong literal:
   
   ```python
   from pyiceberg.expressions import EqualTo
   from pyiceberg.schema import Schema
   from pyiceberg.types import NestedField, DecimalType
   
   schema = Schema(NestedField(1, "price", DecimalType(10, 2), required=True))
   EqualTo("price", 2.675).bind(schema, case_sensitive=True).literal  # 2.67, 
expected 2.68
   ```
   
   ### Expected behavior
   Match Apache Iceberg Java, which converts with 
`BigDecimal.valueOf(value).setScale(scale, RoundingMode.HALF_UP)` in 
`Literals.java` (all four numeric literal types). `BigDecimal.valueOf(double)` 
uses `Double.toString`, i.e. the shortest decimal that round-trips to the 
double, so `2.675` quantizes to `2.68`. Spark and Postgres agree (`2.675` casts 
to `2.68`).
   
   ### Root cause
   `pyiceberg/expressions/literals.py`, `FloatLiteral.to` and 
`DoubleLiteral.to` for `DecimalType`:
   
   ```python
   Decimal(self.value).quantize(Decimal((0, (1,), -type_var.scale)), 
rounding=ROUND_HALF_UP)
   ```
   
   `Decimal(2.675)` is `2.67499999999999982...`, which quantizes down to 
`2.67`. Using `Decimal(str(self.value))` instead gives `Decimal("2.675")`, 
which quantizes to `2.68`, matching `BigDecimal.valueOf`.
   
   ### Environment
   pyiceberg 0.12.0, Python 3.12, Linux.
   
   Happy to open the fix PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to