RussellSpitzer commented on issue #3582: URL: https://github.com/apache/iceberg/issues/3582#issuecomment-974191227
There is a parameter for doing concurrent deletes from the driver, I was working with someone who had this issue and by increasing parallelism they resolved it. RemoveOrphanFiles is missing the parallelism parameter but I believe I made a ticket for it and it is being worked on. I think we should test this out to see if it’s sufficient before distributed deletes since it’s would be much easier to tune. Sent from my iPhone > On Nov 19, 2021, at 8:34 AM, Szehon Ho ***@***.***> wrote: > > > @rdblue @RussellSpitzer @aokolnychyi @jackye1995 > > Would either or both of these ideas will be good contributions, or any other ideas? Appreciate any thoughts > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or unsubscribe. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
