vigneshc-dd commented on code in PR #16280:
URL: https://github.com/apache/iceberg/pull/16280#discussion_r3295587981
##########
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/BaseReader.java:
##########
@@ -204,7 +204,14 @@ protected class SparkDeleteFilter extends
DeleteFilter<InternalRow> {
SparkDeleteFilter(
String filePath, List<DeleteFile> deletes, DeleteCounter counter,
boolean needRowPosCol) {
- super(filePath, deletes, new FieldLookup(table), expectedSchema,
counter, needRowPosCol);
+ super(
+ filePath,
+ deletes,
+ table().schema(),
+ new FieldLookup(table),
Review Comment:
To fix it in `FieldLookup`,its signature should be changed to
`Function<Set<Integer>, Schema>`, to handle the nesting case where we should
return the path to the nested field as opposed to just the id. I could make
that change, it would change the protected constructor in `DeleteFilter`.
Previous approach avoided that. It did not fix the historical schema case.
I have updated `FieldLookup` now, and have removed the TODO. Added a test
for historical schema case as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
