szehon-ho opened a new pull request #2220: URL: https://github.com/apache/iceberg/pull/2220
Hello, we are thinking to change for default "write.target-file-size-bytes" to 512 MB -- The current default (unbound file sizes) will never take advantage of any predicate push down -- This number corresponds well with the Parquet default row-group size (4 row groups/file) -- This will have no impact on ORC file (BaseTaskWriter#shouldRollToNewFile() makes an exception for ORC files) This will result in different behaviors, so would be good to see what the community thinks, cc @aokolnychyi Thank you ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
