thinkharderdev commented on code in PR #4427:
URL: https://github.com/apache/arrow-datafusion/pull/4427#discussion_r1036980062
##########
docs/source/user-guide/configs.md:
##########
@@ -43,8 +43,11 @@ Environment variables are read during `SessionConfig`
initialisation so they mus
| datafusion.execution.coalesce_batches | Boolean | true
| When set to true, record batches will be examined between each operator and
small batches will be coalesced into larger batches. This is helpful when there
are highly selective filters or joins that could produce tiny output batches.
The target batch size is determined by the configuration setting
'datafusion.execution.coalesce_target_batch_size'. |
| datafusion.execution.coalesce_target_batch_size | UInt64 | 4096
| Target batch size when coalescing batches. Uses in conjunction with the
configuration setting 'datafusion.execution.coalesce_batches'.
|
| datafusion.execution.parquet.enable_page_index | Boolean | false
| If true, uses parquet data page level metadata (Page Index) statistics to
reduce the number of rows decoded.
|
+| datafusion.execution.parquet.metadata_size_hint | UInt64 | NULL
| If specified, the parquet reader will try and fetch the last `size_hint`
bytes of the parquet file optimistically. If not specified, two read are
required: One read to fetch the 8-byte parquet footer and another to fetch the
metadata length encoded in the footer.
|
Review Comment:
```suggestion
| datafusion.execution.parquet.metadata_size_hint | UInt64 | NULL
| If specified, the parquet reader will try and fetch the last `size_hint`
bytes of the parquet file optimistically. If not specified, two reads are
required: One read to fetch the 8-byte parquet footer and another to fetch the
metadata length encoded in the footer.
|
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
