zhongyujiang commented on PR #5007: URL: https://github.com/apache/iceberg/pull/5007#issuecomment-1635746582
> Looks like Postgres supports this by immediately checking the table to see if there are any null values. We should be able to use a metadata check to do the same, assuming that the column's metadata collection is not set to none. This is cool and I'd like to give a try on this. I did some investigation, I think we may need to check the metadata of all files (including deleted data file), because CDC reads loads may read deleted files, we also need to check the fields in the equality delete file, because the equality fields are derived from the table schema. What do you think ? @szehon-ho @rdblue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
