This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new a785b7be4e Publish built docs triggered by
e17ca27d65d537e5ec6dcdb8cd42a6ba8f520c6c
a785b7be4e is described below
commit a785b7be4eaed2a6082abccb19894438af6e2b21
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri Oct 20 17:44:35 2023 +0000
Publish built docs triggered by e17ca27d65d537e5ec6dcdb8cd42a6ba8f520c6c
---
_sources/user-guide/configs.md.txt | 1 -
searchindex.js | 2 +-
user-guide/configs.html | 50 ++++++++++++++++++--------------------
3 files changed, 24 insertions(+), 29 deletions(-)
diff --git a/_sources/user-guide/configs.md.txt
b/_sources/user-guide/configs.md.txt
index a0451eed08..446931e544 100644
--- a/_sources/user-guide/configs.md.txt
+++ b/_sources/user-guide/configs.md.txt
@@ -78,7 +78,6 @@ Environment variables are read during `SessionConfig`
initialisation so they mus
| datafusion.execution.sort_in_place_threshold_bytes | 1048576
| When sorting, below what size should data be concatenated and
sorted in a single RecordBatch rather than sorted in batches and merged.
[...]
| datafusion.execution.meta_fetch_concurrency | 32
| Number of files to read in parallel when inferring schema and
statistics
[...]
| datafusion.execution.soft_max_rows_per_output_file | 50000000
| Target number of rows in output files when writing multiple. This
is a soft max, so it can be exceeded slightly. There also will be one file
smaller than the limit if the total number of rows written is not roughly
divisible by the soft max
[...]
-| datafusion.execution.max_parallel_ouput_files | 8
| This is the maximum number of output files being written in
parallel. Higher values can potentially give faster write performance at the
cost of higher peak memory consumption.
[...]
| datafusion.execution.max_buffered_batches_per_output_file | 2
| This is the maximum number of RecordBatches buffered for each
output file being worked. Higher values can potentially give faster write
performance at the cost of higher peak memory consumption
[...]
| datafusion.optimizer.enable_round_robin_repartition | true
| When set to true, the physical plan optimizer will try to add round
robin repartitioning to increase parallelism to leverage more CPU cores
[...]
| datafusion.optimizer.enable_topk_aggregation | true
| When set to true, the optimizer will attempt to perform limit
operations during aggregations, if possible
[...]
diff --git a/searchindex.js b/searchindex.js
index 714bd695b2..e6836eebdb 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["contributor-guide/architecture",
"contributor-guide/communication", "contributor-guide/index",
"contributor-guide/quarterly_roadmap", "contributor-guide/roadmap",
"contributor-guide/specification/index",
"contributor-guide/specification/invariants",
"contributor-guide/specification/output-field-name-semantic", "index",
"library-user-guide/adding-udfs", "library-user-guide/building-logical-plans",
"library-user-guide/catalogs", "library-user-guide/custom-tab [...]
\ No newline at end of file
+Search.setIndex({"docnames": ["contributor-guide/architecture",
"contributor-guide/communication", "contributor-guide/index",
"contributor-guide/quarterly_roadmap", "contributor-guide/roadmap",
"contributor-guide/specification/index",
"contributor-guide/specification/invariants",
"contributor-guide/specification/output-field-name-semantic", "index",
"library-user-guide/adding-udfs", "library-user-guide/building-logical-plans",
"library-user-guide/catalogs", "library-user-guide/custom-tab [...]
\ No newline at end of file
diff --git a/user-guide/configs.html b/user-guide/configs.html
index dd543a1b35..ddd23d2bf1 100644
--- a/user-guide/configs.html
+++ b/user-guide/configs.html
@@ -574,99 +574,95 @@ Environment variables are read during <code
class="docutils literal notranslate"
<td><p>50000000</p></td>
<td><p>Target number of rows in output files when writing multiple. This is a
soft max, so it can be exceeded slightly. There also will be one file smaller
than the limit if the total number of rows written is not roughly divisible by
the soft max</p></td>
</tr>
-<tr
class="row-odd"><td><p>datafusion.execution.max_parallel_ouput_files</p></td>
-<td><p>8</p></td>
-<td><p>This is the maximum number of output files being written in parallel.
Higher values can potentially give faster write performance at the cost of
higher peak memory consumption.</p></td>
-</tr>
-<tr
class="row-even"><td><p>datafusion.execution.max_buffered_batches_per_output_file</p></td>
+<tr
class="row-odd"><td><p>datafusion.execution.max_buffered_batches_per_output_file</p></td>
<td><p>2</p></td>
<td><p>This is the maximum number of RecordBatches buffered for each output
file being worked. Higher values can potentially give faster write performance
at the cost of higher peak memory consumption</p></td>
</tr>
-<tr
class="row-odd"><td><p>datafusion.optimizer.enable_round_robin_repartition</p></td>
+<tr
class="row-even"><td><p>datafusion.optimizer.enable_round_robin_repartition</p></td>
<td><p>true</p></td>
<td><p>When set to true, the physical plan optimizer will try to add round
robin repartitioning to increase parallelism to leverage more CPU cores</p></td>
</tr>
-<tr
class="row-even"><td><p>datafusion.optimizer.enable_topk_aggregation</p></td>
+<tr
class="row-odd"><td><p>datafusion.optimizer.enable_topk_aggregation</p></td>
<td><p>true</p></td>
<td><p>When set to true, the optimizer will attempt to perform limit
operations during aggregations, if possible</p></td>
</tr>
-<tr class="row-odd"><td><p>datafusion.optimizer.filter_null_join_keys</p></td>
+<tr class="row-even"><td><p>datafusion.optimizer.filter_null_join_keys</p></td>
<td><p>false</p></td>
<td><p>When set to true, the optimizer will insert filters before a join
between a nullable and non-nullable column to filter out nulls on the nullable
side. This filter can add additional overhead when the file format does not
fully support predicate push down.</p></td>
</tr>
-<tr
class="row-even"><td><p>datafusion.optimizer.repartition_aggregations</p></td>
+<tr
class="row-odd"><td><p>datafusion.optimizer.repartition_aggregations</p></td>
<td><p>true</p></td>
<td><p>Should DataFusion repartition data using the aggregate keys to execute
aggregates in parallel using the provided <code class="docutils literal
notranslate"><span class="pre">target_partitions</span></code> level</p></td>
</tr>
-<tr
class="row-odd"><td><p>datafusion.optimizer.repartition_file_min_size</p></td>
+<tr
class="row-even"><td><p>datafusion.optimizer.repartition_file_min_size</p></td>
<td><p>10485760</p></td>
<td><p>Minimum total files size in bytes to perform file scan
repartitioning.</p></td>
</tr>
-<tr class="row-even"><td><p>datafusion.optimizer.repartition_joins</p></td>
+<tr class="row-odd"><td><p>datafusion.optimizer.repartition_joins</p></td>
<td><p>true</p></td>
<td><p>Should DataFusion repartition data using the join keys to execute joins
in parallel using the provided <code class="docutils literal notranslate"><span
class="pre">target_partitions</span></code> level</p></td>
</tr>
-<tr
class="row-odd"><td><p>datafusion.optimizer.allow_symmetric_joins_without_pruning</p></td>
+<tr
class="row-even"><td><p>datafusion.optimizer.allow_symmetric_joins_without_pruning</p></td>
<td><p>true</p></td>
<td><p>Should DataFusion allow symmetric hash joins for unbounded data sources
even when its inputs do not have any ordering or filtering If the flag is not
enabled, the SymmetricHashJoin operator will be unable to prune its internal
buffers, resulting in certain join types - such as Full, Left, LeftAnti,
LeftSemi, Right, RightAnti, and RightSemi - being produced only at the end of
the execution. This is not typical in stream processing. Additionally, without
proper design for long runne [...]
</tr>
-<tr
class="row-even"><td><p>datafusion.optimizer.repartition_file_scans</p></td>
+<tr class="row-odd"><td><p>datafusion.optimizer.repartition_file_scans</p></td>
<td><p>true</p></td>
<td><p>When set to <code class="docutils literal notranslate"><span
class="pre">true</span></code>, file groups will be repartitioned to achieve
maximum parallelism. Currently Parquet and CSV formats are supported. If set to
<code class="docutils literal notranslate"><span
class="pre">true</span></code>, all files will be repartitioned evenly (i.e., a
single large file might be partitioned into smaller chunks) for parallel
scanning. If set to <code class="docutils literal notranslate"><s [...]
</tr>
-<tr class="row-odd"><td><p>datafusion.optimizer.repartition_windows</p></td>
+<tr class="row-even"><td><p>datafusion.optimizer.repartition_windows</p></td>
<td><p>true</p></td>
<td><p>Should DataFusion repartition data using the partitions keys to execute
window functions in parallel using the provided <code class="docutils literal
notranslate"><span class="pre">target_partitions</span></code> level</p></td>
</tr>
-<tr class="row-even"><td><p>datafusion.optimizer.repartition_sorts</p></td>
+<tr class="row-odd"><td><p>datafusion.optimizer.repartition_sorts</p></td>
<td><p>true</p></td>
<td><p>Should DataFusion execute sorts in a per-partition fashion and merge
afterwards instead of coalescing first and sorting globally. With this flag is
enabled, plans in the form below <code class="docutils literal
notranslate"><span class="pre">text</span> <span
class="pre">"SortExec:</span> <span class="pre">[a@0</span> <span
class="pre">ASC]",</span> <span class="pre">"</span> <span
class="pre">CoalescePartitionsExec",</span> <span
class="pre">"</span> [...]
</tr>
-<tr class="row-odd"><td><p>datafusion.optimizer.prefer_existing_sort</p></td>
+<tr class="row-even"><td><p>datafusion.optimizer.prefer_existing_sort</p></td>
<td><p>false</p></td>
<td><p>When true, DataFusion will opportunistically remove sorts when the data
is already sorted, (i.e. setting <code class="docutils literal
notranslate"><span class="pre">preserve_order</span></code> to true on <code
class="docutils literal notranslate"><span
class="pre">RepartitionExec</span></code> and using <code class="docutils
literal notranslate"><span class="pre">SortPreservingMergeExec</span></code>)
When false, DataFusion will maximize plan parallelism using <code class="docut
[...]
</tr>
-<tr class="row-even"><td><p>datafusion.optimizer.skip_failed_rules</p></td>
+<tr class="row-odd"><td><p>datafusion.optimizer.skip_failed_rules</p></td>
<td><p>false</p></td>
<td><p>When set to true, the logical plan optimizer will produce warning
messages if any optimization rules produce errors and then proceed to the next
rule. When set to false, any rules that produce errors will cause the query to
fail</p></td>
</tr>
-<tr class="row-odd"><td><p>datafusion.optimizer.max_passes</p></td>
+<tr class="row-even"><td><p>datafusion.optimizer.max_passes</p></td>
<td><p>3</p></td>
<td><p>Number of times that the optimizer will attempt to optimize the
plan</p></td>
</tr>
-<tr
class="row-even"><td><p>datafusion.optimizer.top_down_join_key_reordering</p></td>
+<tr
class="row-odd"><td><p>datafusion.optimizer.top_down_join_key_reordering</p></td>
<td><p>true</p></td>
<td><p>When set to true, the physical plan optimizer will run a top down
process to reorder the join keys</p></td>
</tr>
-<tr class="row-odd"><td><p>datafusion.optimizer.prefer_hash_join</p></td>
+<tr class="row-even"><td><p>datafusion.optimizer.prefer_hash_join</p></td>
<td><p>true</p></td>
<td><p>When set to true, the physical plan optimizer will prefer HashJoin over
SortMergeJoin. HashJoin can work more efficiently than SortMergeJoin but
consumes more memory</p></td>
</tr>
-<tr
class="row-even"><td><p>datafusion.optimizer.hash_join_single_partition_threshold</p></td>
+<tr
class="row-odd"><td><p>datafusion.optimizer.hash_join_single_partition_threshold</p></td>
<td><p>1048576</p></td>
<td><p>The maximum estimated size in bytes for one input side of a HashJoin
will be collected into a single partition</p></td>
</tr>
-<tr class="row-odd"><td><p>datafusion.explain.logical_plan_only</p></td>
+<tr class="row-even"><td><p>datafusion.explain.logical_plan_only</p></td>
<td><p>false</p></td>
<td><p>When set to true, the explain statement will only print logical
plans</p></td>
</tr>
-<tr class="row-even"><td><p>datafusion.explain.physical_plan_only</p></td>
+<tr class="row-odd"><td><p>datafusion.explain.physical_plan_only</p></td>
<td><p>false</p></td>
<td><p>When set to true, the explain statement will only print physical
plans</p></td>
</tr>
-<tr class="row-odd"><td><p>datafusion.explain.show_statistics</p></td>
+<tr class="row-even"><td><p>datafusion.explain.show_statistics</p></td>
<td><p>false</p></td>
<td><p>When set to true, the explain statement will print operator statistics
for physical plans</p></td>
</tr>
-<tr
class="row-even"><td><p>datafusion.sql_parser.parse_float_as_decimal</p></td>
+<tr
class="row-odd"><td><p>datafusion.sql_parser.parse_float_as_decimal</p></td>
<td><p>false</p></td>
<td><p>When set to true, SQL parser will parse float as decimal type</p></td>
</tr>
-<tr
class="row-odd"><td><p>datafusion.sql_parser.enable_ident_normalization</p></td>
+<tr
class="row-even"><td><p>datafusion.sql_parser.enable_ident_normalization</p></td>
<td><p>true</p></td>
<td><p>When set to true, SQL parser will normalize ident (convert ident to
lowercase when not quoted)</p></td>
</tr>
-<tr class="row-even"><td><p>datafusion.sql_parser.dialect</p></td>
+<tr class="row-odd"><td><p>datafusion.sql_parser.dialect</p></td>
<td><p>generic</p></td>
<td><p>Configure the SQL dialect used by DataFusion’s parser; supported values
include: Generic, MySQL, PostgreSQL, Hive, SQLite, Snowflake, Redshift, MsSQL,
ClickHouse, BigQuery, and Ansi.</p></td>
</tr>
