andygrove commented on code in PR #1388:
URL:
https://github.com/apache/datafusion-ballista/pull/1388#discussion_r2701693445
##########
ballista/core/src/client.rs:
##########
@@ -287,6 +289,122 @@ impl BallistaClient {
}
}
+/// A connection pool for reusing `BallistaClient` connections to executors.
+///
+/// This pool caches connections by (host, port) to avoid the overhead of
+/// establishing new gRPC connections for each partition fetch during shuffle
reads.
+/// Connections are stored indefinitely and reused across multiple fetch
operations.
+///
+/// # Thread Safety
+///
+/// The pool uses a `RwLock` to allow concurrent reads while ensuring exclusive
+/// access during connection creation. The `BallistaClient` itself is `Clone`
+/// (wrapping an `Arc`), so cloned clients share the underlying connection.
+#[derive(Default)]
+pub struct BallistaClientPool {
Review Comment:
I pushed a TTL improvement, but moved this to drfraft for now
##########
ballista/core/src/client.rs:
##########
@@ -287,6 +289,122 @@ impl BallistaClient {
}
}
+/// A connection pool for reusing `BallistaClient` connections to executors.
+///
+/// This pool caches connections by (host, port) to avoid the overhead of
+/// establishing new gRPC connections for each partition fetch during shuffle
reads.
+/// Connections are stored indefinitely and reused across multiple fetch
operations.
+///
+/// # Thread Safety
+///
+/// The pool uses a `RwLock` to allow concurrent reads while ensuring exclusive
+/// access during connection creation. The `BallistaClient` itself is `Clone`
+/// (wrapping an `Arc`), so cloned clients share the underlying connection.
+#[derive(Default)]
+pub struct BallistaClientPool {
Review Comment:
I pushed a TTL improvement, but moved this to draft for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
