alamb commented on code in PR #10748:
URL: https://github.com/apache/datafusion/pull/10748#discussion_r1623280130
##########
datafusion/physical-plan/src/joins/hash_join.rs:
##########
@@ -875,23 +875,12 @@ async fn collect_left_input(
// Estimation of memory size, required for hashtable, prior to allocation.
// Final result can be verified using `RawTable.allocation_info()`
- //
- // For majority of cases hashbrown overestimates buckets qty to keep ~1/8
of them empty.
- // This formula leads to overallocation for small tables (< 8 elements)
but fine overall.
- let estimated_buckets = (num_rows.checked_mul(8).ok_or_else(|| {
Review Comment:
Sorry -- i missed the
[comment](https://github.com/apache/datafusion/issues/8764#issuecomment-2143467627)
on the ticket
I think we should change the code to panic on overflow and we can sort out
what to do if that is seen in practice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
