sfiend edited a comment on issue #2627: URL: https://github.com/apache/iceberg/issues/2627#issuecomment-849458968
I had met the same problem before. When multiple pieces of data with the same primary key value are inserted in the same batch, besides equality delete files, iceberg will also write position delete files. During your query after that, when FlinkInputFormat initialize the RowDataIterator and read next, the iterator will initialize the FlinkDeleteFilter, in this initialization, the FlinkDeleteFilter's parent will add a column named '_pos' if the current split has position delete file. I think the purpose of adding column is to apply the position delete file depending on the position of each row, but iceberg did not delete it before sending the result rows to flink. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
