This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 592f35a6c9 Publish built docs triggered by
300a39b89308ae949a99058660cb69e112e65e3c
592f35a6c9 is described below
commit 592f35a6c9e8b58d0722e6b7e6a74598a9721e06
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Sun Sep 22 11:15:23 2024 +0000
Publish built docs triggered by 300a39b89308ae949a99058660cb69e112e65e3c
---
_sources/user-guide/explain-usage.md.txt | 15 +++++++++++++++
searchindex.js | 2 +-
user-guide/explain-usage.html | 15 +++++++++++++++
3 files changed, 31 insertions(+), 1 deletion(-)
diff --git a/_sources/user-guide/explain-usage.md.txt
b/_sources/user-guide/explain-usage.md.txt
index a65fad92d1..b376c2eedb 100644
--- a/_sources/user-guide/explain-usage.md.txt
+++ b/_sources/user-guide/explain-usage.md.txt
@@ -223,6 +223,21 @@ Again, reading from bottom up:
- `SortPreservingMergeExec`
- `output_rows=5`, `elapsed_compute=2.375µs`: Produced the final 5 rows in
2.375µs (microseconds)
+When predicate pushdown is enabled, `ParquetExec` gains the following metrics:
+
+- `page_index_rows_matched`: number of rows in pages that were tested by a
page index filter, and passed
+- `page_index_rows_pruned`: number of rows in pages that were tested by a page
index filter, and did not pass
+- `row_groups_matched_bloom_filter`: number of rows in row groups that were
tested by a Bloom Filter, and passed
+- `row_groups_pruned_bloom_filter`: number of rows in row groups that were
tested by a Bloom Filter, and did not pass
+- `row_groups_matched_statistics`: number of rows in row groups that were
tested by row group statistics (min and max value), and passed
+- `row_groups_pruned_statistics`: number of rows in row groups that were
tested by row group statistics (min and max value), and did not pass
+- `pushdown_rows_matched`: rows that were tested by any of the above filtered,
and passed all of them (this should be minimum of `page_index_rows_matched`,
`row_groups_pruned_bloom_filter`, and `row_groups_pruned_statistics`)
+- `pushdown_rows_pruned`: rows that were tested by any of the above filtered,
and did not pass one of them (this should be sum of `page_index_rows_matched`,
`row_groups_pruned_bloom_filter`, and `row_groups_pruned_statistics`)
+- `predicate_evaluation_errors`: number of times evaluating the filter
expression failed (expected to be zero in normal operation)
+- `num_predicate_creation_errors`: number of errors creating predicates
(expected to be zero in normal operation)
+- `pushdown_eval_time`: time spent evaluating these filters
+- `page_index_eval_time`: time required to evaluate the page index filters
+
## Partitions and Execution
DataFusion determines the optimal number of cores to use as part of query
diff --git a/searchindex.js b/searchindex.js
index f5e2c9588f..4479351a23 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"!=": [[46, "op-neq"]], "!~": [[46,
"op-re-not-match"]], "!~*": [[46, "op-re-not-match-i"]], "!~~": [[46, "id18"]],
"!~~*": [[46, "id19"]], "#": [[46, "op-bit-xor"]], "%": [[46, "op-modulo"]],
"&": [[46, "op-bit-and"]], "(relation, name) tuples in logical fields and
logical columns are unique": [[9,
"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]], "*":
[[46, "op-multiply"]], "+": [[46, "op-plus"]], "-": [[46, "op-minus"]], "/":
[[4 [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"!=": [[46, "op-neq"]], "!~": [[46,
"op-re-not-match"]], "!~*": [[46, "op-re-not-match-i"]], "!~~": [[46, "id18"]],
"!~~*": [[46, "id19"]], "#": [[46, "op-bit-xor"]], "%": [[46, "op-modulo"]],
"&": [[46, "op-bit-and"]], "(relation, name) tuples in logical fields and
logical columns are unique": [[9,
"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]], "*":
[[46, "op-multiply"]], "+": [[46, "op-plus"]], "-": [[46, "op-minus"]], "/":
[[4 [...]
\ No newline at end of file
diff --git a/user-guide/explain-usage.html b/user-guide/explain-usage.html
index 56a554cd51..a0345c3d60 100644
--- a/user-guide/explain-usage.html
+++ b/user-guide/explain-usage.html
@@ -805,6 +805,21 @@ by all 16 cores.</p>
</ul>
</li>
</ul>
+<p>When predicate pushdown is enabled, <code class="docutils literal
notranslate"><span class="pre">ParquetExec</span></code> gains the following
metrics:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span
class="pre">page_index_rows_matched</span></code>: number of rows in pages that
were tested by a page index filter, and passed</p></li>
+<li><p><code class="docutils literal notranslate"><span
class="pre">page_index_rows_pruned</span></code>: number of rows in pages that
were tested by a page index filter, and did not pass</p></li>
+<li><p><code class="docutils literal notranslate"><span
class="pre">row_groups_matched_bloom_filter</span></code>: number of rows in
row groups that were tested by a Bloom Filter, and passed</p></li>
+<li><p><code class="docutils literal notranslate"><span
class="pre">row_groups_pruned_bloom_filter</span></code>: number of rows in row
groups that were tested by a Bloom Filter, and did not pass</p></li>
+<li><p><code class="docutils literal notranslate"><span
class="pre">row_groups_matched_statistics</span></code>: number of rows in row
groups that were tested by row group statistics (min and max value), and
passed</p></li>
+<li><p><code class="docutils literal notranslate"><span
class="pre">row_groups_pruned_statistics</span></code>: number of rows in row
groups that were tested by row group statistics (min and max value), and did
not pass</p></li>
+<li><p><code class="docutils literal notranslate"><span
class="pre">pushdown_rows_matched</span></code>: rows that were tested by any
of the above filtered, and passed all of them (this should be minimum of <code
class="docutils literal notranslate"><span
class="pre">page_index_rows_matched</span></code>, <code class="docutils
literal notranslate"><span
class="pre">row_groups_pruned_bloom_filter</span></code>, and <code
class="docutils literal notranslate"><span class="pre">row_groups_pr [...]
+<li><p><code class="docutils literal notranslate"><span
class="pre">pushdown_rows_pruned</span></code>: rows that were tested by any of
the above filtered, and did not pass one of them (this should be sum of <code
class="docutils literal notranslate"><span
class="pre">page_index_rows_matched</span></code>, <code class="docutils
literal notranslate"><span
class="pre">row_groups_pruned_bloom_filter</span></code>, and <code
class="docutils literal notranslate"><span class="pre">row_groups_p [...]
+<li><p><code class="docutils literal notranslate"><span
class="pre">predicate_evaluation_errors</span></code>: number of times
evaluating the filter expression failed (expected to be zero in normal
operation)</p></li>
+<li><p><code class="docutils literal notranslate"><span
class="pre">num_predicate_creation_errors</span></code>: number of errors
creating predicates (expected to be zero in normal operation)</p></li>
+<li><p><code class="docutils literal notranslate"><span
class="pre">pushdown_eval_time</span></code>: time spent evaluating these
filters</p></li>
+<li><p><code class="docutils literal notranslate"><span
class="pre">page_index_eval_time</span></code>: time required to evaluate the
page index filters</p></li>
+</ul>
</section>
<section id="partitions-and-execution">
<h2>Partitions and Execution<a class="headerlink"
href="#partitions-and-execution" title="Link to this heading">¶</a></h2>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
