vvellanki commented on pull request #2182: URL: https://github.com/apache/iceberg/pull/2182#issuecomment-773063209
@rdblue I did some experimentation with the approach you outlined by invoking planFiles(). I found that the time taken to read the manifest list file and the manifest files was quite high. Also, the time taken increases when tables have a large number of columns (> 1000) and there are per-column stats for each of the data files (string columns increase the size of the per-column stats). This increases the number of manifest files that need to be read as part of this exercise and also increases the amount of time taken. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
