thswlsqls opened a new pull request, #10379:
URL: https://github.com/apache/paimon/pull/10379

   ### Purpose
   
   fix #10378
   
   - Java table reads lose matching rows when `withReadType` prunes a nested 
field that a filter references (`s.b = 7` with `s ROW<a>`), without 
`executeFilter()`.
   - #9858 drops filters on unprojected columns but checks only top-level 
names; after #9423 enabled nested pushdown, `s.b` passes and Parquet reads it 
as all-null.
   - `ParquetReaderFactory` now also requires the whole nested path in the read 
type. AND keeps projected conjuncts; an OR touching a pruned path is dropped.
   - Only the Java `ReadBuilder` path is affected. Flink/Spark SQL keep filter 
fields in the read type; `executeFilter()` is already safe.
   
   ### Tests
   
   - Added 
`AppendOnlySimpleTableTest#testParquetFilterOnUnprojectedNestedField`; fails on 
master.
   - Added `ParquetReadWriteTest#testReadWithUnprojectedNestedFilter` (AND, 
OR); both fail on master.
   - `ParquetReadWriteTest` 54/54 and `AppendOnlySimpleTableTest` 114/114 pass; 
checkstyle and spotless pass on `paimon-format` and `paimon-core`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to