RussellSpitzer commented on issue #8161: URL: https://github.com/apache/iceberg/issues/8161#issuecomment-1672480765
Rusty is right here, that’s only 7.5 mb a partition. I would aim for at least 512mb maybe more for such a large table On Wed, Aug 9, 2023 at 10:00 PM Rusty Conover ***@***.***> wrote: > Have less partitions. Each partition is more File/S3 I/O. > > You have 27,500 rows per partition which is really small. Try to target at > least a few million rows per partition depending on row size. > > Rusty > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/iceberg/issues/8161#issuecomment-1672478848>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AADE2YMETHMHQYNHBEIMFP3XURFDDANCNFSM6AAAAAA2ZOK7FA> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
