paleolimbot commented on code in PR #687:
URL: https://github.com/apache/sedona-db/pull/687#discussion_r2885618763
##########
rust/sedona/src/context_builder.rs:
##########
@@ -33,26 +33,54 @@ use crate::{
size_parser,
};
+/// The fraction of total physical memory to use as the default memory limit.
+const DEFAULT_MEMORY_FRACTION: f64 = 0.75;
+
+/// Compute the default memory limit as 75% of total physical memory.
+fn default_memory_limit() -> usize {
+ let mut sys = sysinfo::System::new();
+ sys.refresh_memory();
+ // `System::total_memory()` returns bytes since sysinfo 0.23+.
+ let total = sys.total_memory() as f64;
+ (total * DEFAULT_MEMORY_FRACTION) as usize
+}
+
/// Builder for constructing a [`SedonaContext`] with configurable runtime
/// environment settings.
///
/// This builder centralizes the construction of memory pools, disk managers,
/// and runtime environments so that the same logic can be reused across the
/// CLI, Python bindings, ADBC driver, and any future entry points.
///
+/// By default, the builder uses 75% of the system's physical memory as the
+/// memory limit and a fair memory pool. Use
[`without_memory_limit`](Self::without_memory_limit)
+/// or pass `"unlimited"` as the `memory_limit` option to disable the limit.
+///
/// # Examples
///
/// ```rust,no_run
/// # async fn example() -> datafusion::error::Result<()> {
Review Comment:
I'll open a follow up for this, but when we have examples we shouldn't use
`no_run` so that we don't have to keep track of whether examples work or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
