korowa commented on code in PR #13751:
URL: https://github.com/apache/datafusion/pull/13751#discussion_r1884994212
##########
datafusion/physical-plan/src/joins/hash_join.rs:
##########
@@ -90,9 +90,6 @@ struct JoinLeftData {
/// Counter of running probe-threads, potentially
/// able to update `visited_indices_bitmap`
probe_threads_counter: AtomicUsize,
- /// Memory reservation that tracks memory used by `hash_map` hash table
- /// `batch`. Cleared on drop.
- _reservation: MemoryReservation,
Review Comment:
> Although, the memory is clean while the hash map is not freed
That actually is the issue, because there is single memory pool for a
RuntimeEnv, and it's intended to be used by multiple sessions or at least by
the multiple operators within the same query. Even in case of single query like
`select * from a join b order by a.field` untracked build side may cause OOM
due to SortExec will consider MemoryPool as being empty, while there is a build
side data in memory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
