alamb commented on code in PR #20672:
URL: https://github.com/apache/datafusion/pull/20672#discussion_r2890342906
##########
datafusion/physical-plan/src/spill/spill_pool.rs:
##########
@@ -443,6 +480,7 @@ pub fn channel(
let writer = SpillPoolWriter {
max_file_size_bytes,
shared: Arc::clone(&shared),
+ writer_count: WriterCount::new(),
Review Comment:
I am confused about how this works -- doesn't it mean that each writer gets
its own (new) WriterCount? I would have expected the writer count was using
something on shared 🤔
Maybe we should go back to the original pattern of updating counts under a
lock 🤔
##########
datafusion/physical-plan/src/spill/spill_pool.rs:
##########
@@ -88,6 +89,33 @@ impl SpillPoolShared {
}
}
+/// Tracks the number of live [`SpillPoolWriter`] clones.
+///
+/// Cloning increments the count. [`WriterCount::decrement`] atomically
+/// decrements the count and reports whether the caller was the last clone.
+struct WriterCount(Arc<AtomicUsize>);
+
+impl WriterCount {
+ fn new() -> Self {
+ Self(Arc::new(AtomicUsize::new(1)))
+ }
+
+ /// Decrements the count and returns `true` if this was the last clone.
Review Comment:
👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
