This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 9ce0c7f5 Publish built docs triggered by
fc006aa39f113ddc57ba1bbc416241565dd86e8d
9ce0c7f5 is described below
commit 9ce0c7f5971e45f6685e974b964741e3cf73b7ab
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Thu Sep 5 18:09:37 2024 +0000
Publish built docs triggered by fc006aa39f113ddc57ba1bbc416241565dd86e8d
---
_sources/user-guide/tuning.md.txt | 8 ++++++++
searchindex.js | 2 +-
user-guide/tuning.html | 12 ++++++++++++
3 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/_sources/user-guide/tuning.md.txt
b/_sources/user-guide/tuning.md.txt
index 9704bade..f1a928ff 100644
--- a/_sources/user-guide/tuning.md.txt
+++ b/_sources/user-guide/tuning.md.txt
@@ -21,6 +21,14 @@ under the License.
Comet provides some tuning options to help you get the best performance from
your queries.
+## Metrics
+
+Comet metrics are not directly comparable to Spark metrics in some cases.
+
+`CometScanExec` uses nanoseconds for total scan time. Spark also measures scan
time in nanoseconds but converts to
+milliseconds *per batch* which can result in a large loss of precision. In one
case we saw total scan time
+of 41 seconds reported as 23 seconds for example.
+
## Memory Tuning
Comet currently doesn't share the memory allocation from Spark but owns its
own memory allocation.
diff --git a/searchindex.js b/searchindex.js
index 5745515d..3b1fb6e9 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles": {"1. Install Comet": [[9, "install-comet"]], "2.
Clone Spark and Apply Diff": [[9, "clone-spark-and-apply-diff"]], "3. Run Spark
SQL Tests": [[9, "run-spark-sql-tests"]], "ANSI mode": [[11, "ansi-mode"]],
"API Differences Between Spark Versions": [[0,
"api-differences-between-spark-versions"]], "ASF Links": [[10, null]], "Adding
Spark-side Tests for the New Expression": [[0,
"adding-spark-side-tests-for-the-new-expression"]], "Adding a New Expression":
[[0, [...]
\ No newline at end of file
+Search.setIndex({"alltitles": {"1. Install Comet": [[9, "install-comet"]], "2.
Clone Spark and Apply Diff": [[9, "clone-spark-and-apply-diff"]], "3. Run Spark
SQL Tests": [[9, "run-spark-sql-tests"]], "ANSI mode": [[11, "ansi-mode"]],
"API Differences Between Spark Versions": [[0,
"api-differences-between-spark-versions"]], "ASF Links": [[10, null]], "Adding
Spark-side Tests for the New Expression": [[0,
"adding-spark-side-tests-for-the-new-expression"]], "Adding a New Expression":
[[0, [...]
\ No newline at end of file
diff --git a/user-guide/tuning.html b/user-guide/tuning.html
index 0c1d4f65..fd0aaec6 100644
--- a/user-guide/tuning.html
+++ b/user-guide/tuning.html
@@ -271,6 +271,11 @@ under the License.
<nav id="bd-toc-nav">
<ul class="visible nav section-nav flex-column">
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#metrics">
+ Metrics
+ </a>
+ </li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#memory-tuning">
Memory Tuning
@@ -359,6 +364,13 @@ under the License.
<section id="tuning-guide">
<h1>Tuning Guide<a class="headerlink" href="#tuning-guide" title="Link to this
heading">¶</a></h1>
<p>Comet provides some tuning options to help you get the best performance
from your queries.</p>
+<section id="metrics">
+<h2>Metrics<a class="headerlink" href="#metrics" title="Link to this
heading">¶</a></h2>
+<p>Comet metrics are not directly comparable to Spark metrics in some
cases.</p>
+<p><code class="docutils literal notranslate"><span
class="pre">CometScanExec</span></code> uses nanoseconds for total scan time.
Spark also measures scan time in nanoseconds but converts to
+milliseconds <em>per batch</em> which can result in a large loss of precision.
In one case we saw total scan time
+of 41 seconds reported as 23 seconds for example.</p>
+</section>
<section id="memory-tuning">
<h2>Memory Tuning<a class="headerlink" href="#memory-tuning" title="Link to
this heading">¶</a></h2>
<p>Comet currently doesn’t share the memory allocation from Spark but owns its
own memory allocation.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
