kosiew commented on code in PR #17021:
URL: https://github.com/apache/datafusion/pull/17021#discussion_r2290686071
##########
datafusion-cli/src/main.rs:
##########
@@ -174,27 +176,13 @@ async fn main_inner() -> Result<()> {
let session_config = get_session_config(&args)?;
let mut rt_builder = RuntimeEnvBuilder::new();
- // set memory pool size
if let Some(memory_limit) = args.memory_limit {
// set memory pool type
let pool: Arc<dyn MemoryPool> = match args.mem_pool_type {
- PoolType::Fair if args.top_memory_consumers == 0 => {
- Arc::new(FairSpillPool::new(memory_limit))
- }
- PoolType::Fair => Arc::new(TrackConsumersPool::new(
- FairSpillPool::new(memory_limit),
- NonZeroUsize::new(args.top_memory_consumers).unwrap(),
- )),
- PoolType::Greedy if args.top_memory_consumers == 0 => {
- Arc::new(GreedyMemoryPool::new(memory_limit))
- }
- PoolType::Greedy => Arc::new(TrackConsumersPool::new(
- GreedyMemoryPool::new(memory_limit),
- NonZeroUsize::new(args.top_memory_consumers).unwrap(),
- )),
+ PoolType::Fair => Arc::new(FairSpillPool::new(memory_limit)),
Review Comment:
No.
✅ Previous Behavior (Before Change)
If --top-memory-consumers > 0, a TrackConsumersPool wraps the actual memory
pool.
Otherwise, just the base pool (FairSpillPool or GreedyMemoryPool) is used.
No tracking if --top-memory-consumers == 0.
✅ New Behavior (After Change)
The base_memory_pool is always constructed first, regardless of tracking.
If --top-memory-consumers > 0, a TrackConsumersPool is wrapped around
base_memory_pool and passed to the runtime.
It is also assigned to a tracked_pool: Option<Arc<dyn TrackedPool>> and
passed to ReplSessionContext.
Additionally, disable_tracking() is explicitly called to ensure it's
initially not profiling until enabled.
If --top-memory-consumers == 0, tracking is skipped, and the base memory
pool is used as-is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
