singhpk234 commented on PR #7430: URL: https://github.com/apache/iceberg/pull/7430#issuecomment-1526918436
> We first find matching files and then plan splits so the split size can be dynamic, we just need a good way to estimate it correctly. +1 on @aokolnychyi's suggestion on dynamic split size generation, this would help in solving the problem from both sides small table with a large split size, and large table with a small split size, since iceberg is responsible for the scan planning it's best if it's takes the call itself on how to make these splits. plz. ref [ticket](https://github.com/apache/iceberg/issues/7071) similar discussion above -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
