Dandandan commented on PR #4867: URL: https://github.com/apache/arrow-datafusion/pull/4867#issuecomment-1377168697
> In practice (= for any reasonable repartition config) this will NOT lead to unbounded memory usage since virtually all partitions should eventually receive some data. I'm sure this doesn't hold in certain cases. Mainly for hash repartition, when cardinality is low, for example lower than the number of target partitions, part of them will not receive any data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
