pvary commented on PR #4512: URL: https://github.com/apache/iceberg/pull/4512#issuecomment-1094658955
Thanks @rdblue for the review! Will have to find another solution then. We need to serialize the tasks and send them to the executors. When we have this big residual, then we have to serialize it for every task which is a serious unnecessary overhead, especially when the resulting expression is small. @szlta suggested that - in case the residuals are the same for every task in a table scan - we could serialize the residuals independently and send them by a different / table scan level method, and before executing the task we can restore the residuals. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
