This is an automated email from the ASF dual-hosted git repository.
avantgardnerio pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-ballista.git
The following commit(s) were added to refs/heads/main by this push:
new 968086f6c docs(config): rewrite max_partitions_per_task description to
be timeless (#2093)
968086f6c is described below
commit 968086f6cfc4e54af7cab0271f86046343ca5243
Author: Brent Gardner <[email protected]>
AuthorDate: Sun Jul 19 17:53:06 2026 -0600
docs(config): rewrite max_partitions_per_task description to be timeless
(#2093)
Drop PR-relative phrasing ("pre-multi-partition-tasks execution model",
"preserving master's behaviour on merge") and the "Spark user persona"
jargon. Describe what the setting does in terms a config-doc reader can
act on.
Addresses andygrove's review comment on #2038.
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
---
ballista/core/src/config.rs | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/ballista/core/src/config.rs b/ballista/core/src/config.rs
index c3f841adf..bef5556fa 100644
--- a/ballista/core/src/config.rs
+++ b/ballista/core/src/config.rs
@@ -359,13 +359,12 @@ static CONFIG_ENTRIES: LazyLock<HashMap<String,
ConfigEntry>> = LazyLock::new(||
ConfigEntry::new(
BALLISTA_SCHEDULER_MAX_PARTITIONS_PER_TASK.to_string(),
"Upper bound on the number of input partitions packed into a
single \
- task's `partition_slice`. `1` (default) is the
pre-multi-partition-tasks \
- execution model — one task per partition, matching the Spark user
\
- persona and preserving master's behaviour on merge. Raise to
enable \
- multi-partition tasks (fewer tasks, parallel-sort / parallel-join
wins); \
- `0` means unbounded — the scheduler fills each task up to the
executor's \
- free vcore count. Does not apply to collapse stages, which must
pack \
- their full pending queue into a single task for correctness."
+ task's `partition_slice`. `1` (default) means one task per input \
+ partition. Raise to enable multi-partition tasks (fewer tasks, \
+ parallel-sort / parallel-join wins); `0` means unbounded — the \
+ scheduler fills each task up to the executor's free vcore count. \
+ Does not apply to collapse stages, which must pack their full \
+ pending queue into a single task for correctness."
.to_string(),
DataType::UInt64,
Some(1.to_string()),
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
