rahulsmahadev opened a new pull request, #4435:
URL: https://github.com/apache/iceberg-python/pull/4435

   # Rationale for this change
   
   Bug: when a data file lacks a date, time, timestamp or timestamptz column, 
`=` and `IN` miss its non-null `initial-default`, while `!=` matches it. 
`_ColumnNameTranslator` compares a Python temporal value with a bound literal 
in days or microseconds:
   
   ```python
   # v3 table: rows 1-3 were written before `d` was added with initial-default 
2020-01-01
   tbl.scan(row_filter=EqualTo("d", date(2020, 1, 1))).to_arrow()  # 0 rows, 
expected 3
   ```
   
   Fix: add a private `NestedField` accessor that returns a temporal 
initial-default in the literal's form (the conversion `add_column` uses), and 
use it in the translator, the Avro default reader and `Record` defaults. 
Partition-projected values are already in that form and keep precedence.
   
   Java: `SchemaParser` and `SchemaUpdate` supply typed defaults to row-group 
filtering; `GenericReader` normalizes reader output through 
`InternalRecordWrapper` before `Evaluator`. This PR normalizes defaults used in 
internal records.
   
   - `NestedField.initial_default` stays a Python object; internal default 
consumers share the conversion.
   
   ## Are these changes tested?
   
   Yes, filter tests in `tests/expressions/test_visitors.py` and 
`tests/io/test_pyarrow.py`, plus Avro and `Record` default tests in 
`tests/avro/test_resolver.py` and `tests/test_typedef.py`. All fail on main 
(wrong filter result, or a Python date where an int is expected) and pass with 
this change.
   
   ## Are there any user-facing changes?
   
   Yes, this fixes incorrect temporal-default filter results and range errors. 
Avro now returns defaulted temporal fields as integer days or microseconds, 
matching stored fields.
   
   Please add the `changelog` label.
   
   This pull request and its description were written by Isaac.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to