milenkovicm commented on code in PR #2038:
URL:
https://github.com/apache/datafusion-ballista/pull/2038#discussion_r3610965513
##########
docs/developer/architecture.md:
##########
@@ -55,8 +55,9 @@ virtual cores (`vcores`), and the scheduler packs up to that
many of a stage's o
task. All partitions in the slice execute concurrently under one DataFusion
plan invocation: scans and shuffle
readers are rewritten to see only the assigned partition ids, and DataFusion's
per-partition `execute(N)`
contract fans the work across the executor's threads. Slice size is bounded by
the executor's free vcore count
-and by `ballista.scheduler.max_partitions_per_task` (`0` = unbounded, `1` =
one task per partition — the
-pre-multi-partition-tasks model).
+and by `ballista.scheduler.max_partitions_per_task` (`1` = one task per
partition — the default and the
Review Comment:
would it make more sense to have this property
`ballista.scheduler.max_partitions_per_task_unbounded` ?
##########
ballista/core/src/config.rs:
##########
@@ -342,16 +342,16 @@ static CONFIG_ENTRIES: LazyLock<HashMap<String,
ConfigEntry>> = LazyLock::new(||
ConfigEntry::new(
Review Comment:
is there going to be any other value than 0 and 1, if not, would it make
sense to have it as boolean
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
