This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-ballista.git
The following commit(s) were added to refs/heads/asf-site by this push:
new d5c87aff6 Publish built docs triggered by
2c1357f0d61770e007eebd608391b10969ab36de
d5c87aff6 is described below
commit d5c87aff6c989e9a33bfbd74723b24330c3e74c9
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Sat Sep 12 21:08:45 2026 +0000
Publish built docs triggered by 2c1357f0d61770e007eebd608391b10969ab36de
---
_sources/user-guide/configs.md.txt | 61 ++++++++++++++++++++------------------
searchindex.js | 2 +-
user-guide/configs.html | 35 +++++++++++++++-------
3 files changed, 58 insertions(+), 40 deletions(-)
diff --git a/_sources/user-guide/configs.md.txt
b/_sources/user-guide/configs.md.txt
index 91b068356..72dafc54e 100644
--- a/_sources/user-guide/configs.md.txt
+++ b/_sources/user-guide/configs.md.txt
@@ -104,35 +104,38 @@ standard DataFusion settings.
<!-- BEGIN GENERATED CONFIG REFERENCE
prefix=ballista.job.,ballista.standalone.,ballista.cache.,ballista.client.,ballista.optimizer.,ballista.planner.,ballista.scheduler.
-->
<!-- prettier-ignore -->
-| key | type | default
| description
[...]
-| ------------------------------------------------------ | ------- |
----------------------------- |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
-| ballista.cache.noop | Boolean | true
| Disable default cache node extension
[...]
-| ballista.client.grpc_connect_timeout_seconds | UInt64 | 20
| Connection timeout for gRPC client in seconds
[...]
-| ballista.client.grpc_http2_keepalive_interval_seconds | UInt64 | 300
| HTTP/2 keep-alive interval for gRPC client in seconds
[...]
-| ballista.client.grpc_max_message_size | UInt64 | 16777216
| Configuration for max message size in gRPC clients
[...]
-| ballista.client.grpc_tcp_keepalive_seconds | UInt64 | 3600
| TCP keep-alive interval for gRPC client in seconds
[...]
-| ballista.client.grpc_timeout_seconds | UInt64 | 20
| Request timeout for gRPC client in seconds
[...]
-| ballista.client.initial_connection_window_size | UInt64 | 67108864
| HTTP/2 initial connection-level flow-control window for
gRPC data-plane clients, in bytes. Should be >= the shuffle governor byte
budget so the governor, not the transport window, is the binding backpressure.
0 leaves the tonic default.
[...]
-| ballista.client.initial_stream_window_size | UInt64 | 16777216
| HTTP/2 initial stream-level flow-control window for gRPC
data-plane clients, in bytes. 0 leaves the tonic default.
[...]
-| ballista.client.io_retries_times | UInt16 | 3
| Number of retries for IO operations in the Ballista
client.
[...]
-| ballista.client.io_retry_wait_time_ms | UInt64 | 3000
| Wait time in milliseconds between IO retries in the
Ballista client.
[...]
-| ballista.client.pull | Boolean | false
| Should client employ pull or push job tracking. In pull
mode client will make a request to server in the loop, until job finishes. Pull
mode is kept for legacy clients.
[...]
-| ballista.client.use_tls | Boolean | false
| Should connection between client, scheduler, and
executors use TLS.
[...]
-| ballista.job.name | Utf8 | (none)
| Sets the job name that will appear in the web user
interface for any submitted jobs
[...]
-| ballista.optimizer.broadcast_join_threshold_bytes | UInt64 | 134217728
| Byte-size threshold below which a hash join's smaller
side is promoted to CollectLeft and lowered via the broadcast pattern. Governs
broadcast selection under both the static distributed planner and adaptive
query planning (AQE). It also caps null-aware anti joins with a known build
size because they require single-task CollectLeft execution. Set to 0 to
disable promotion and reject null- [...]
-| ballista.optimizer.broadcast_join_threshold_rows | UInt64 | 1000000
| Row-count threshold below which a hash join's smaller
side is promoted to CollectLeft and lowered via the broadcast pattern, used as
a fallback when byte-size statistics are unavailable. Applies to adaptive query
planning (AQE). Set to 0 to disable promotion via the row-count path.
[...]
-| ballista.optimizer.hash_join_max_build_partition_bytes | UInt64 | 67108864
| Maximum per-partition hash-join build-side bytes for a
Partitioned hash join under AQE. A build partition larger than this falls back
to SortMergeJoin (spillable). Defaults to 64 MiB; 0 disables the check, which
makes AQE use a hash join regardless of build size.
[...]
-| ballista.optimizer.not_in_subquery_rewrite | Boolean | true
| Rewrites uncorrelated NOT IN (subquery) filter predicates
into a plain anti join plus a one-row count aggregate during logical
optimization. The rewrite avoids DataFusion's null-aware hash join, which
Ballista must otherwise execute in a single task. Set to false to keep the
null-aware join and its single-task lowering.
[...]
-| ballista.planner.adaptive.enabled | Boolean | true
| Enables Adaptive Query Planning: joins and partition
counts are chosen from measured runtime statistics instead of planning-time
estimates. Set to false to use the static distributed planner.
[...]
-| ballista.planner.adaptive_join.enabled | Boolean | true
| Enables the AQE dynamic join-selection rule
(SelectJoinRule). When true (default), DynamicJoinSelectionExec nodes are
resolved to concrete HashJoin or CollectLeft join implementations at runtime.
Disable only for debugging.
[...]
-| ballista.planner.coalesce.enabled | Boolean | false
| Enables the AQE coalesce-shuffle-partitions rule.
Disabled by default — opt in when fewer/larger downstream tasks matter more
than parallelism.
[...]
-| ballista.planner.coalesce.merged_partition_factor | Float64 | 1.2
| Two adjacent partitions are merged when their combined
size is below target_partition_bytes times this factor. Mirrors Spark's legacy
coalesce semantics.
[...]
-| ballista.planner.coalesce.small_partition_factor | Float64 | 0.2
| A coalesced partition smaller than target_partition_bytes
times this factor counts as small and is merged into its neighbour. Mirrors
Spark's legacy coalesce semantics.
[...]
-| ballista.planner.coalesce.target_partition_bytes | UInt64 | 67108864
| Target post-coalesce partition size in bytes. Mirrors
Spark's advisoryPartitionSizeInBytes.
[...]
-| ballista.planner.parallel_window.enabled | Boolean | false
| Enables the AQE parallel-window rule
(ParallelWindowRule), which rewrites bounded-RANGE-frame windows into a
distributed range-shuffle so BoundedWindowAggExec's single-partition constraint
is not a serial bottleneck. Disabled by default — opt in when the workload
contains matching window shapes.
[...]
-| ballista.planner.propagate_empty.enabled | Boolean | true
| Enables the AQE propagate-empty-relation rule. Injects
EmptyExec into the plan where an input is known to be empty, such as one side
of a join, allowing downstream work to be skipped.
[...]
-| ballista.scheduler.max_partitions_per_task | UInt64 | 0
| Upper bound on the number of input partitions packed into
a single task's `partition_slice`. `0` (default) means unbounded — the
scheduler fills each task up to the executor's free vcore count. `1` means one
task per input partition. Does not apply to collapse stages, which must pack
their full pending queue into a single task for correctness.
[...]
-| ballista.standalone.parallelism | UInt16 | number of
available CPU cores | Number of concurrent tasks a standalone in-process
executor will run.
[...]
+| key | type |
default | description
[...]
+| --------------------------------------------------------- | ------- |
----------------------------- |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
+| ballista.cache.noop | Boolean | true
| Disable default cache node extension
[...]
+| ballista.client.grpc_connect_timeout_seconds | UInt64 | 20
| Connection timeout for gRPC client in seconds
[...]
+| ballista.client.grpc_http2_keepalive_interval_seconds | UInt64 | 300
| HTTP/2 keep-alive interval for gRPC client in seconds
[...]
+| ballista.client.grpc_max_message_size | UInt64 |
16777216 | Configuration for max message size in gRPC
clients
[...]
+| ballista.client.grpc_tcp_keepalive_seconds | UInt64 | 3600
| TCP keep-alive interval for gRPC client in seconds
[...]
+| ballista.client.grpc_timeout_seconds | UInt64 | 20
| Request timeout for gRPC client in seconds
[...]
+| ballista.client.initial_connection_window_size | UInt64 |
67108864 | HTTP/2 initial connection-level flow-control
window for gRPC data-plane clients, in bytes. Should be >= the shuffle governor
byte budget so the governor, not the transport window, is the binding
backpressure. 0 leaves the tonic default.
[...]
+| ballista.client.initial_stream_window_size | UInt64 |
16777216 | HTTP/2 initial stream-level flow-control window
for gRPC data-plane clients, in bytes. 0 leaves the tonic default.
[...]
+| ballista.client.io_retries_times | UInt16 | 3
| Number of retries for IO operations in the Ballista
client.
[...]
+| ballista.client.io_retry_wait_time_ms | UInt64 | 3000
| Wait time in milliseconds between IO retries in the
Ballista client.
[...]
+| ballista.client.pull | Boolean | false
| Should client employ pull or push job tracking. In
pull mode client will make a request to server in the loop, until job finishes.
Pull mode is kept for legacy clients.
[...]
+| ballista.client.use_tls | Boolean | false
| Should connection between client, scheduler, and
executors use TLS.
[...]
+| ballista.job.name | Utf8 | (none)
| Sets the job name that will appear in the web user
interface for any submitted jobs
[...]
+| ballista.optimizer.broadcast_join_threshold_bytes | UInt64 |
134217728 | Byte-size threshold below which a hash join's
smaller side is promoted to CollectLeft and lowered via the broadcast pattern.
Governs broadcast selection under both the static distributed planner and
adaptive query planning (AQE). It also caps null-aware anti joins with a known
build size because they require single-task CollectLeft execution. Set to 0 to
disable promotion and reject nu [...]
+| ballista.optimizer.broadcast_join_threshold_rows | UInt64 |
1000000 | Row-count threshold below which a hash join's
smaller side is promoted to CollectLeft and lowered via the broadcast pattern,
used as a fallback when byte-size statistics are unavailable. Applies to
adaptive query planning (AQE). Set to 0 to disable promotion via the row-count
path.
[...]
+| ballista.optimizer.hash_join_max_build_partition_bytes | UInt64 |
67108864 | Maximum per-partition hash-join build-side
bytes for a Partitioned hash join under AQE. A build partition larger than this
falls back to SortMergeJoin (spillable). Defaults to 64 MiB; 0 disables the
check, which makes AQE use a hash join regardless of build size.
[...]
+| ballista.optimizer.not_in_subquery_rewrite | Boolean | true
| Rewrites uncorrelated NOT IN (subquery) filter
predicates into a plain anti join plus a one-row count aggregate during logical
optimization. The rewrite avoids DataFusion's null-aware hash join, which
Ballista must otherwise execute in a single task. Set to false to keep the
null-aware join and its single-task lowering.
[...]
+| ballista.optimizer.stage_build_side | Boolean | true
| Stages a join's prospective build side as its own
shuffle before choosing the join strategy. Applies only when that side's size
is an estimate rather than a measurement, and the probe side is far larger: an
exact size over the broadcast budget is a fact that no further measurement can
change. Lets AQE measure the build side and pick a broadcast join instead of
committing to a full shuf [...]
+| ballista.optimizer.stage_build_side_max_estimate_multiple | UInt64 | 32
| How far over broadcast_join_threshold_bytes an
estimated build side may sit and still be worth measuring. Past this multiple
the side is large on any reading and no measurement brings it back under
budget, so it is shuffled without the extra round trip. Only consulted when
stage_build_side is enabled, and only for build sides whose size is an estimate
rather than a measurement. [...]
+| ballista.optimizer.stage_build_side_min_probe_ratio | UInt64 | 10
| How many times larger the probe side must be before
AQE stages a join's build side on its own. Staging serialises two shuffles that
would otherwise run concurrently, so the deferral costs at most the smaller
side's runtime; this ratio bounds the cost of being wrong. Only consulted when
stage_build_side is enabled, and only for build sides whose size is an estimate
rather than a measure [...]
+| ballista.planner.adaptive.enabled | Boolean | true
| Enables Adaptive Query Planning: joins and partition
counts are chosen from measured runtime statistics instead of planning-time
estimates. Set to false to use the static distributed planner.
[...]
+| ballista.planner.adaptive_join.enabled | Boolean | true
| Enables the AQE dynamic join-selection rule
(SelectJoinRule). When true (default), DynamicJoinSelectionExec nodes are
resolved to concrete HashJoin or CollectLeft join implementations at runtime.
Disable only for debugging.
[...]
+| ballista.planner.coalesce.enabled | Boolean | false
| Enables the AQE coalesce-shuffle-partitions rule.
Disabled by default — opt in when fewer/larger downstream tasks matter more
than parallelism.
[...]
+| ballista.planner.coalesce.merged_partition_factor | Float64 | 1.2
| Two adjacent partitions are merged when their combined
size is below target_partition_bytes times this factor. Mirrors Spark's legacy
coalesce semantics.
[...]
+| ballista.planner.coalesce.small_partition_factor | Float64 | 0.2
| A coalesced partition smaller than
target_partition_bytes times this factor counts as small and is merged into its
neighbour. Mirrors Spark's legacy coalesce semantics.
[...]
+| ballista.planner.coalesce.target_partition_bytes | UInt64 |
67108864 | Target post-coalesce partition size in bytes.
Mirrors Spark's advisoryPartitionSizeInBytes.
[...]
+| ballista.planner.parallel_window.enabled | Boolean | false
| Enables the AQE parallel-window rule
(ParallelWindowRule), which rewrites bounded-RANGE-frame windows into a
distributed range-shuffle so BoundedWindowAggExec's single-partition constraint
is not a serial bottleneck. Disabled by default — opt in when the workload
contains matching window shapes.
[...]
+| ballista.planner.propagate_empty.enabled | Boolean | true
| Enables the AQE propagate-empty-relation rule. Injects
EmptyExec into the plan where an input is known to be empty, such as one side
of a join, allowing downstream work to be skipped.
[...]
+| ballista.scheduler.max_partitions_per_task | UInt64 | 0
| Upper bound on the number of input partitions packed
into a single task's `partition_slice`. `0` (default) means unbounded — the
scheduler fills each task up to the executor's free vcore count. `1` means one
task per input partition. Does not apply to collapse stages, which must pack
their full pending queue into a single task for correctness.
[...]
+| ballista.standalone.parallelism | UInt16 | number
of available CPU cores | Number of concurrent tasks a standalone in-process
executor will run.
[...]
<!-- END GENERATED CONFIG REFERENCE -->
diff --git a/searchindex.js b/searchindex.js
index 2f87f70c3..63fbe4dcd 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"API changes":[[31,"api-changes"]],"AQE join
behavior (opt-in)":[[30,"aqe-join-behavior-opt-in"]],"Adaptive Query Execution
(AQE)":[[23,"adaptive-query-execution-aqe"]],"Adaptive Query Execution
(Experimental)":[[54,"adaptive-query-execution-experimental"]],"Adding a
persona":[[28,"adding-a-persona"]],"Aggregate
Functions":[[53,"aggregate-functions"]],"Apache DataFusion
Ballista":[[29,null]],"Apache DataFusion Ballista 0.10.0
Changelog":[[0,null]],"Apache Da [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"API changes":[[31,"api-changes"]],"AQE join
behavior (opt-in)":[[30,"aqe-join-behavior-opt-in"]],"Adaptive Query Execution
(AQE)":[[23,"adaptive-query-execution-aqe"]],"Adaptive Query Execution
(Experimental)":[[54,"adaptive-query-execution-experimental"]],"Adding a
persona":[[28,"adding-a-persona"]],"Aggregate
Functions":[[53,"aggregate-functions"]],"Apache DataFusion
Ballista":[[29,null]],"Apache DataFusion Ballista 0.10.0
Changelog":[[0,null]],"Apache Da [...]
\ No newline at end of file
diff --git a/user-guide/configs.html b/user-guide/configs.html
index 1535fcf9a..af6a97680 100644
--- a/user-guide/configs.html
+++ b/user-guide/configs.html
@@ -560,52 +560,67 @@ standard DataFusion settings.</p>
<td><p>true</p></td>
<td><p>Rewrites uncorrelated NOT IN (subquery) filter predicates into a plain
anti join plus a one-row count aggregate during logical optimization. The
rewrite avoids DataFusion’s null-aware hash join, which Ballista must otherwise
execute in a single task. Set to false to keep the null-aware join and its
single-task lowering.</p></td>
</tr>
-<tr class="row-odd"><td><p>ballista.planner.adaptive.enabled</p></td>
+<tr class="row-odd"><td><p>ballista.optimizer.stage_build_side</p></td>
+<td><p>Boolean</p></td>
+<td><p>true</p></td>
+<td><p>Stages a join’s prospective build side as its own shuffle before
choosing the join strategy. Applies only when that side’s size is an estimate
rather than a measurement, and the probe side is far larger: an exact size over
the broadcast budget is a fact that no further measurement can change. Lets AQE
measure the build side and pick a broadcast join instead of committing to a
full shuffle of the probe side up front. Set to false to always shuffle both
sides at once.</p></td>
+</tr>
+<tr
class="row-even"><td><p>ballista.optimizer.stage_build_side_max_estimate_multiple</p></td>
+<td><p>UInt64</p></td>
+<td><p>32</p></td>
+<td><p>How far over broadcast_join_threshold_bytes an estimated build side may
sit and still be worth measuring. Past this multiple the side is large on any
reading and no measurement brings it back under budget, so it is shuffled
without the extra round trip. Only consulted when stage_build_side is enabled,
and only for build sides whose size is an estimate rather than a
measurement.</p></td>
+</tr>
+<tr
class="row-odd"><td><p>ballista.optimizer.stage_build_side_min_probe_ratio</p></td>
+<td><p>UInt64</p></td>
+<td><p>10</p></td>
+<td><p>How many times larger the probe side must be before AQE stages a join’s
build side on its own. Staging serialises two shuffles that would otherwise run
concurrently, so the deferral costs at most the smaller side’s runtime; this
ratio bounds the cost of being wrong. Only consulted when stage_build_side is
enabled, and only for build sides whose size is an estimate rather than a
measurement.</p></td>
+</tr>
+<tr class="row-even"><td><p>ballista.planner.adaptive.enabled</p></td>
<td><p>Boolean</p></td>
<td><p>true</p></td>
<td><p>Enables Adaptive Query Planning: joins and partition counts are chosen
from measured runtime statistics instead of planning-time estimates. Set to
false to use the static distributed planner.</p></td>
</tr>
-<tr class="row-even"><td><p>ballista.planner.adaptive_join.enabled</p></td>
+<tr class="row-odd"><td><p>ballista.planner.adaptive_join.enabled</p></td>
<td><p>Boolean</p></td>
<td><p>true</p></td>
<td><p>Enables the AQE dynamic join-selection rule (SelectJoinRule). When true
(default), DynamicJoinSelectionExec nodes are resolved to concrete HashJoin or
CollectLeft join implementations at runtime. Disable only for
debugging.</p></td>
</tr>
-<tr class="row-odd"><td><p>ballista.planner.coalesce.enabled</p></td>
+<tr class="row-even"><td><p>ballista.planner.coalesce.enabled</p></td>
<td><p>Boolean</p></td>
<td><p>false</p></td>
<td><p>Enables the AQE coalesce-shuffle-partitions rule. Disabled by default —
opt in when fewer/larger downstream tasks matter more than parallelism.</p></td>
</tr>
-<tr
class="row-even"><td><p>ballista.planner.coalesce.merged_partition_factor</p></td>
+<tr
class="row-odd"><td><p>ballista.planner.coalesce.merged_partition_factor</p></td>
<td><p>Float64</p></td>
<td><p>1.2</p></td>
<td><p>Two adjacent partitions are merged when their combined size is below
target_partition_bytes times this factor. Mirrors Spark’s legacy coalesce
semantics.</p></td>
</tr>
-<tr
class="row-odd"><td><p>ballista.planner.coalesce.small_partition_factor</p></td>
+<tr
class="row-even"><td><p>ballista.planner.coalesce.small_partition_factor</p></td>
<td><p>Float64</p></td>
<td><p>0.2</p></td>
<td><p>A coalesced partition smaller than target_partition_bytes times this
factor counts as small and is merged into its neighbour. Mirrors Spark’s legacy
coalesce semantics.</p></td>
</tr>
-<tr
class="row-even"><td><p>ballista.planner.coalesce.target_partition_bytes</p></td>
+<tr
class="row-odd"><td><p>ballista.planner.coalesce.target_partition_bytes</p></td>
<td><p>UInt64</p></td>
<td><p>67108864</p></td>
<td><p>Target post-coalesce partition size in bytes. Mirrors Spark’s
advisoryPartitionSizeInBytes.</p></td>
</tr>
-<tr class="row-odd"><td><p>ballista.planner.parallel_window.enabled</p></td>
+<tr class="row-even"><td><p>ballista.planner.parallel_window.enabled</p></td>
<td><p>Boolean</p></td>
<td><p>false</p></td>
<td><p>Enables the AQE parallel-window rule (ParallelWindowRule), which
rewrites bounded-RANGE-frame windows into a distributed range-shuffle so
BoundedWindowAggExec’s single-partition constraint is not a serial bottleneck.
Disabled by default — opt in when the workload contains matching window
shapes.</p></td>
</tr>
-<tr class="row-even"><td><p>ballista.planner.propagate_empty.enabled</p></td>
+<tr class="row-odd"><td><p>ballista.planner.propagate_empty.enabled</p></td>
<td><p>Boolean</p></td>
<td><p>true</p></td>
<td><p>Enables the AQE propagate-empty-relation rule. Injects EmptyExec into
the plan where an input is known to be empty, such as one side of a join,
allowing downstream work to be skipped.</p></td>
</tr>
-<tr class="row-odd"><td><p>ballista.scheduler.max_partitions_per_task</p></td>
+<tr class="row-even"><td><p>ballista.scheduler.max_partitions_per_task</p></td>
<td><p>UInt64</p></td>
<td><p>0</p></td>
<td><p>Upper bound on the number of input partitions packed into a single
task’s <code class="docutils literal notranslate"><span
class="pre">partition_slice</span></code>. <code class="docutils literal
notranslate"><span class="pre">0</span></code> (default) means unbounded — the
scheduler fills each task up to the executor’s free vcore count. <code
class="docutils literal notranslate"><span class="pre">1</span></code> means
one task per input partition. Does not apply to collapse stage [...]
</tr>
-<tr class="row-even"><td><p>ballista.standalone.parallelism</p></td>
+<tr class="row-odd"><td><p>ballista.standalone.parallelism</p></td>
<td><p>UInt16</p></td>
<td><p>number of available CPU cores</p></td>
<td><p>Number of concurrent tasks a standalone in-process executor will
run.</p></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
