This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 74a70b1d17 Publish built docs triggered by
496e4b67a05bb49af8d7aa1ca5035312fd4e54f9
74a70b1d17 is described below
commit 74a70b1d175f8eae6ccbc435b7da51f106376b10
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Wed Mar 20 20:32:47 2024 +0000
Publish built docs triggered by 496e4b67a05bb49af8d7aa1ca5035312fd4e54f9
---
_sources/index.rst.txt | 3 +-
_sources/library-user-guide/profiling.md.txt | 63 +++++++++++++
contributor-guide/architecture.html | 5 ++
contributor-guide/communication.html | 5 ++
contributor-guide/index.html | 11 ++-
contributor-guide/quarterly_roadmap.html | 5 ++
contributor-guide/roadmap.html | 5 ++
contributor-guide/specification/index.html | 5 ++
contributor-guide/specification/invariants.html | 5 ++
.../specification/output-field-name-semantic.html | 5 ++
genindex.html | 5 ++
index.html | 6 ++
library-user-guide/adding-udfs.html | 5 ++
library-user-guide/building-logical-plans.html | 5 ++
library-user-guide/catalogs.html | 5 ++
library-user-guide/custom-table-providers.html | 5 ++
library-user-guide/extending-operators.html | 11 ++-
library-user-guide/index.html | 5 ++
.../{extending-operators.html => profiling.html} | 97 ++++++++++++++++++---
library-user-guide/using-the-dataframe-api.html | 5 ++
library-user-guide/using-the-sql-api.html | 5 ++
library-user-guide/working-with-exprs.html | 5 ++
objects.inv | Bin 1380 -> 1404 bytes
search.html | 5 ++
searchindex.js | 2 +-
user-guide/cli.html | 5 ++
user-guide/configs.html | 5 ++
user-guide/dataframe.html | 5 ++
user-guide/example-usage.html | 5 ++
user-guide/expressions.html | 5 ++
user-guide/faq.html | 5 ++
user-guide/introduction.html | 5 ++
user-guide/sql/aggregate_functions.html | 5 ++
user-guide/sql/data_types.html | 5 ++
user-guide/sql/ddl.html | 5 ++
user-guide/sql/dml.html | 5 ++
user-guide/sql/explain.html | 5 ++
user-guide/sql/index.html | 5 ++
user-guide/sql/information_schema.html | 5 ++
user-guide/sql/operators.html | 5 ++
user-guide/sql/scalar_functions.html | 5 ++
user-guide/sql/select.html | 5 ++
user-guide/sql/sql_status.html | 5 ++
user-guide/sql/subqueries.html | 5 ++
user-guide/sql/window_functions.html | 5 ++
user-guide/sql/write_options.html | 5 ++
46 files changed, 365 insertions(+), 18 deletions(-)
diff --git a/_sources/index.rst.txt b/_sources/index.rst.txt
index f7c0873f3a..919a7ad703 100644
--- a/_sources/index.rst.txt
+++ b/_sources/index.rst.txt
@@ -79,7 +79,7 @@ Please see the `developer’s guide`_ for contributing and
`communication`_ for
.. toctree::
:maxdepth: 1
:caption: Library User Guide
-
+
library-user-guide/index
library-user-guide/using-the-sql-api
library-user-guide/working-with-exprs
@@ -89,6 +89,7 @@ Please see the `developer’s guide`_ for contributing and
`communication`_ for
library-user-guide/adding-udfs
library-user-guide/custom-table-providers
library-user-guide/extending-operators
+ library-user-guide/profiling
.. _toc.contributor-guide:
diff --git a/_sources/library-user-guide/profiling.md.txt
b/_sources/library-user-guide/profiling.md.txt
new file mode 100644
index 0000000000..a20489496f
--- /dev/null
+++ b/_sources/library-user-guide/profiling.md.txt
@@ -0,0 +1,63 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Profiling Cookbook
+
+The section contains examples how to perform CPU profiling for Apache Arrow
DataFusion on different operating systems.
+
+## MacOS
+
+### Building a flamegraph
+
+- [cargo-flamegraph](https://github.com/flamegraph-rs/flamegraph)
+
+Test:
+
+```bash
+CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --unit-test
datafusion -- dataframe::tests::test_array_agg
+```
+
+Benchmark:
+
+```bash
+CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --bench sql_planner
-- --bench
+```
+
+Open `flamegraph.svg` file with the browser
+
+- dtrace with DataFusion CLI
+
+```bash
+git clone https://github.com/brendangregg/FlameGraph.git /tmp/fg
+cd datafusion-cli
+CARGO_PROFILE_RELEASE_DEBUG=true cargo build --release
+echo "select * from table;" >> test.sql
+sudo dtrace -c './target/debug/datafusion-cli -f test.sql' -o out.stacks -n
'profile-997 /execname == "datafusion-cli"/ { @[ustack(100)] = count(); }'
+/tmp/fg/FlameGraph/stackcollapse.pl out.stacks |
/tmp/fg/FlameGraph/flamegraph.pl > flamegraph.svg
+```
+
+Open `flamegraph.svg` file with the browser
+
+### CPU profiling with XCode Instruments
+
+[Video: how to CPU profile DataFusion with XCode
Instruments](https://youtu.be/P3dXH61Kr5U)
+
+## Linux
+
+## Windows
diff --git a/contributor-guide/architecture.html
b/contributor-guide/architecture.html
index 8eb7a66d14..3a84b84072 100644
--- a/contributor-guide/architecture.html
+++ b/contributor-guide/architecture.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/contributor-guide/communication.html
b/contributor-guide/communication.html
index 8d07f22803..b8a8e36a7b 100644
--- a/contributor-guide/communication.html
+++ b/contributor-guide/communication.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/contributor-guide/index.html b/contributor-guide/index.html
index 58b34092e1..f085863d40 100644
--- a/contributor-guide/index.html
+++ b/contributor-guide/index.html
@@ -35,7 +35,7 @@
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Communication" href="communication.html" />
- <link rel="prev" title="Extending DataFusion’s operators: custom
LogicalPlan and Execution Plans"
href="../library-user-guide/extending-operators.html" />
+ <link rel="prev" title="Profiling Cookbook"
href="../library-user-guide/profiling.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
@@ -895,11 +900,11 @@ taplo<span class="w"> </span><span class="m">0</span>.9.0
<!-- Previous / next buttons -->
<div class='prev-next-area'>
- <a class='left-prev' id="prev-link"
href="../library-user-guide/extending-operators.html" title="previous page">
+ <a class='left-prev' id="prev-link"
href="../library-user-guide/profiling.html" title="previous page">
<i class="fas fa-angle-left"></i>
<div class="prev-next-info">
<p class="prev-next-subtitle">previous</p>
- <p class="prev-next-title">Extending DataFusion’s operators:
custom LogicalPlan and Execution Plans</p>
+ <p class="prev-next-title">Profiling Cookbook</p>
</div>
</a>
<a class='right-next' id="next-link" href="communication.html" title="next
page">
diff --git a/contributor-guide/quarterly_roadmap.html
b/contributor-guide/quarterly_roadmap.html
index dec5db71f0..46d840e0eb 100644
--- a/contributor-guide/quarterly_roadmap.html
+++ b/contributor-guide/quarterly_roadmap.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/contributor-guide/roadmap.html b/contributor-guide/roadmap.html
index 48cfd6ca33..a404f6004b 100644
--- a/contributor-guide/roadmap.html
+++ b/contributor-guide/roadmap.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/contributor-guide/specification/index.html
b/contributor-guide/specification/index.html
index bc07788b72..b8a506fc4f 100644
--- a/contributor-guide/specification/index.html
+++ b/contributor-guide/specification/index.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/contributor-guide/specification/invariants.html
b/contributor-guide/specification/invariants.html
index 3b6472874a..a0378a5a45 100644
--- a/contributor-guide/specification/invariants.html
+++ b/contributor-guide/specification/invariants.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/contributor-guide/specification/output-field-name-semantic.html
b/contributor-guide/specification/output-field-name-semantic.html
index 9ca021b1b4..d60a47ff46 100644
--- a/contributor-guide/specification/output-field-name-semantic.html
+++ b/contributor-guide/specification/output-field-name-semantic.html
@@ -268,6 +268,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/genindex.html b/genindex.html
index d2af76fa83..dcfec41ba3 100644
--- a/genindex.html
+++ b/genindex.html
@@ -266,6 +266,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/index.html b/index.html
index 1bfc871c55..7b9ed580e9 100644
--- a/index.html
+++ b/index.html
@@ -268,6 +268,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
@@ -427,6 +432,7 @@ community.</p>
<li class="toctree-l1"><a class="reference internal"
href="library-user-guide/adding-udfs.html">Adding User Defined Functions:
Scalar/Window/Aggregate/Table Functions</a></li>
<li class="toctree-l1"><a class="reference internal"
href="library-user-guide/custom-table-providers.html">Custom Table
Provider</a></li>
<li class="toctree-l1"><a class="reference internal"
href="library-user-guide/extending-operators.html">Extending DataFusion’s
operators: custom LogicalPlan and Execution Plans</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="library-user-guide/profiling.html">Profiling Cookbook</a></li>
</ul>
</div>
<div class="toctree-wrapper compound" id="toc-contributor-guide">
diff --git a/library-user-guide/adding-udfs.html
b/library-user-guide/adding-udfs.html
index 2c6fc107ec..19dd672a72 100644
--- a/library-user-guide/adding-udfs.html
+++ b/library-user-guide/adding-udfs.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/library-user-guide/building-logical-plans.html
b/library-user-guide/building-logical-plans.html
index b4728bf9b6..2134efbd20 100644
--- a/library-user-guide/building-logical-plans.html
+++ b/library-user-guide/building-logical-plans.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/library-user-guide/catalogs.html b/library-user-guide/catalogs.html
index b3156a2c7e..a6419ece75 100644
--- a/library-user-guide/catalogs.html
+++ b/library-user-guide/catalogs.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/library-user-guide/custom-table-providers.html
b/library-user-guide/custom-table-providers.html
index abdf06bbed..5e27b11b5c 100644
--- a/library-user-guide/custom-table-providers.html
+++ b/library-user-guide/custom-table-providers.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/library-user-guide/extending-operators.html
b/library-user-guide/extending-operators.html
index b1ba55796d..b68b08b6e5 100644
--- a/library-user-guide/extending-operators.html
+++ b/library-user-guide/extending-operators.html
@@ -34,7 +34,7 @@
<script async="true" defer="true"
src="https://buttons.github.io/buttons.js";></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Introduction"
href="../contributor-guide/index.html" />
+ <link rel="next" title="Profiling Cookbook" href="profiling.html" />
<link rel="prev" title="Custom Table Provider"
href="custom-table-providers.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
@@ -411,10 +416,10 @@
<p class="prev-next-title">Custom Table Provider</p>
</div>
</a>
- <a class='right-next' id="next-link"
href="../contributor-guide/index.html" title="next page">
+ <a class='right-next' id="next-link" href="profiling.html" title="next
page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
- <p class="prev-next-title">Introduction</p>
+ <p class="prev-next-title">Profiling Cookbook</p>
</div>
<i class="fas fa-angle-right"></i>
</a>
diff --git a/library-user-guide/index.html b/library-user-guide/index.html
index 9de05f6c5e..73267a9312 100644
--- a/library-user-guide/index.html
+++ b/library-user-guide/index.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/library-user-guide/extending-operators.html
b/library-user-guide/profiling.html
similarity index 70%
copy from library-user-guide/extending-operators.html
copy to library-user-guide/profiling.html
index b1ba55796d..986378bb93 100644
--- a/library-user-guide/extending-operators.html
+++ b/library-user-guide/profiling.html
@@ -5,7 +5,7 @@
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0"
/><meta name="viewport" content="width=device-width, initial-scale=1" />
- <title>Extending DataFusion’s operators: custom LogicalPlan and Execution
Plans — Apache Arrow DataFusion documentation</title>
+ <title>Profiling Cookbook — Apache Arrow DataFusion
documentation</title>
<link href="../_static/styles/theme.css?digest=1999514e3f237ded88cf"
rel="stylesheet">
<link
href="../_static/styles/pydata-sphinx-theme.css?digest=1999514e3f237ded88cf"
rel="stylesheet">
@@ -35,7 +35,7 @@
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Introduction"
href="../contributor-guide/index.html" />
- <link rel="prev" title="Custom Table Provider"
href="custom-table-providers.html" />
+ <link rel="prev" title="Extending DataFusion’s operators: custom
LogicalPlan and Execution Plans" href="extending-operators.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
@@ -264,9 +264,14 @@
Custom Table Provider
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="extending-operators.html">
+ Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
+ </a>
+ </li>
<li class="toctree-l1 current active">
<a class="current reference internal" href="#">
- Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
+ Profiling Cookbook
</a>
</li>
</ul>
@@ -346,9 +351,41 @@
<div class="toc-item">
+<div class="tocsection onthispage pt-5 pb-3">
+ <i class="fas fa-list"></i> On this page
+</div>
<nav id="bd-toc-nav">
-
+ <ul class="visible nav section-nav flex-column">
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#macos">
+ MacOS
+ </a>
+ <ul class="nav section-nav flex-column">
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#building-a-flamegraph">
+ Building a flamegraph
+ </a>
+ </li>
+ <li class="toc-h3 nav-item toc-entry">
+ <a class="reference internal nav-link"
href="#cpu-profiling-with-xcode-instruments">
+ CPU profiling with XCode Instruments
+ </a>
+ </li>
+ </ul>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#linux">
+ Linux
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#windows">
+ Windows
+ </a>
+ </li>
+</ul>
+
</nav>
</div>
@@ -356,7 +393,7 @@
<div class="tocsection editthispage">
- <a
href="https://github.com/apache/arrow-datafusion/edit/main/docs/source/library-user-guide/extending-operators.md";>
+ <a
href="https://github.com/apache/arrow-datafusion/edit/main/docs/source/library-user-guide/profiling.md";>
<i class="fas fa-pencil-alt"></i> Edit this page
</a>
</div>
@@ -393,9 +430,49 @@
specific language governing permissions and limitations
under the License.
-->
-<section
id="extending-datafusion-s-operators-custom-logicalplan-and-execution-plans">
-<h1>Extending DataFusion’s operators: custom LogicalPlan and Execution Plans<a
class="headerlink"
href="#extending-datafusion-s-operators-custom-logicalplan-and-execution-plans"
title="Link to this heading">¶</a></h1>
-<p>Coming soon</p>
+<section id="profiling-cookbook">
+<h1>Profiling Cookbook<a class="headerlink" href="#profiling-cookbook"
title="Link to this heading">¶</a></h1>
+<p>The section contains examples how to perform CPU profiling for Apache Arrow
DataFusion on different operating systems.</p>
+<section id="macos">
+<h2>MacOS<a class="headerlink" href="#macos" title="Link to this
heading">¶</a></h2>
+<section id="building-a-flamegraph">
+<h3>Building a flamegraph<a class="headerlink" href="#building-a-flamegraph"
title="Link to this heading">¶</a></h3>
+<ul class="simple">
+<li><p><a class="reference external"
href="https://github.com/flamegraph-rs/flamegraph";>cargo-flamegraph</a></p></li>
+</ul>
+<p>Test:</p>
+<div class="highlight-bash notranslate"><div
class="highlight"><pre><span></span><span
class="nv">CARGO_PROFILE_RELEASE_DEBUG</span><span class="o">=</span><span
class="nb">true</span><span class="w"> </span>cargo<span class="w">
</span>flamegraph<span class="w"> </span>--root<span class="w">
</span>--unit-test<span class="w"> </span>datafusion<span class="w">
</span>--<span class="w"> </span>dataframe::tests::test_array_agg
+</pre></div>
+</div>
+<p>Benchmark:</p>
+<div class="highlight-bash notranslate"><div
class="highlight"><pre><span></span><span
class="nv">CARGO_PROFILE_RELEASE_DEBUG</span><span class="o">=</span><span
class="nb">true</span><span class="w"> </span>cargo<span class="w">
</span>flamegraph<span class="w"> </span>--root<span class="w">
</span>--bench<span class="w"> </span>sql_planner<span class="w">
</span>--<span class="w"> </span>--bench
+</pre></div>
+</div>
+<p>Open <code class="docutils literal notranslate"><span
class="pre">flamegraph.svg</span></code> file with the browser</p>
+<ul class="simple">
+<li><p>dtrace with DataFusion CLI</p></li>
+</ul>
+<div class="highlight-bash notranslate"><div
class="highlight"><pre><span></span>git<span class="w"> </span>clone<span
class="w"> </span>https://github.com/brendangregg/FlameGraph.git<span
class="w"> </span>/tmp/fg
+<span class="nb">cd</span><span class="w"> </span>datafusion-cli
+<span class="nv">CARGO_PROFILE_RELEASE_DEBUG</span><span
class="o">=</span><span class="nb">true</span><span class="w">
</span>cargo<span class="w"> </span>build<span class="w"> </span>--release
+<span class="nb">echo</span><span class="w"> </span><span
class="s2">"select * from table;"</span><span class="w">
</span>>><span class="w"> </span>test.sql
+sudo<span class="w"> </span>dtrace<span class="w"> </span>-c<span class="w">
</span><span class="s1">'./target/debug/datafusion-cli -f
test.sql'</span><span class="w"> </span>-o<span class="w">
</span>out.stacks<span class="w"> </span>-n<span class="w"> </span><span
class="s1">'profile-997 /execname == "datafusion-cli"/ {
@[ustack(100)] = count(); }'</span>
+/tmp/fg/FlameGraph/stackcollapse.pl<span class="w"> </span>out.stacks<span
class="w"> </span><span class="p">|</span><span class="w">
</span>/tmp/fg/FlameGraph/flamegraph.pl<span class="w"> </span>><span
class="w"> </span>flamegraph.svg
+</pre></div>
+</div>
+<p>Open <code class="docutils literal notranslate"><span
class="pre">flamegraph.svg</span></code> file with the browser</p>
+</section>
+<section id="cpu-profiling-with-xcode-instruments">
+<h3>CPU profiling with XCode Instruments<a class="headerlink"
href="#cpu-profiling-with-xcode-instruments" title="Link to this
heading">¶</a></h3>
+<p><a class="reference external" href="https://youtu.be/P3dXH61Kr5U";>Video:
how to CPU profile DataFusion with XCode Instruments</a></p>
+</section>
+</section>
+<section id="linux">
+<h2>Linux<a class="headerlink" href="#linux" title="Link to this
heading">¶</a></h2>
+</section>
+<section id="windows">
+<h2>Windows<a class="headerlink" href="#windows" title="Link to this
heading">¶</a></h2>
+</section>
</section>
@@ -404,11 +481,11 @@
<!-- Previous / next buttons -->
<div class='prev-next-area'>
- <a class='left-prev' id="prev-link" href="custom-table-providers.html"
title="previous page">
+ <a class='left-prev' id="prev-link" href="extending-operators.html"
title="previous page">
<i class="fas fa-angle-left"></i>
<div class="prev-next-info">
<p class="prev-next-subtitle">previous</p>
- <p class="prev-next-title">Custom Table Provider</p>
+ <p class="prev-next-title">Extending DataFusion’s operators:
custom LogicalPlan and Execution Plans</p>
</div>
</a>
<a class='right-next' id="next-link"
href="../contributor-guide/index.html" title="next page">
diff --git a/library-user-guide/using-the-dataframe-api.html
b/library-user-guide/using-the-dataframe-api.html
index 35323badd3..ebf235948a 100644
--- a/library-user-guide/using-the-dataframe-api.html
+++ b/library-user-guide/using-the-dataframe-api.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/library-user-guide/using-the-sql-api.html
b/library-user-guide/using-the-sql-api.html
index f7b131b87a..19953f27bb 100644
--- a/library-user-guide/using-the-sql-api.html
+++ b/library-user-guide/using-the-sql-api.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/library-user-guide/working-with-exprs.html
b/library-user-guide/working-with-exprs.html
index 47c2314906..b359ac5aab 100644
--- a/library-user-guide/working-with-exprs.html
+++ b/library-user-guide/working-with-exprs.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/objects.inv b/objects.inv
index f85bda1dfd..f170dd9192 100644
Binary files a/objects.inv and b/objects.inv differ
diff --git a/search.html b/search.html
index a833ae6006..e8e1a40768 100644
--- a/search.html
+++ b/search.html
@@ -272,6 +272,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/searchindex.js b/searchindex.js
index 4e9370c3bc..e9182b2979 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["contributor-guide/architecture",
"contributor-guide/communication", "contributor-guide/index",
"contributor-guide/quarterly_roadmap", "contributor-guide/roadmap",
"contributor-guide/specification/index",
"contributor-guide/specification/invariants",
"contributor-guide/specification/output-field-name-semantic", "index",
"library-user-guide/adding-udfs", "library-user-guide/building-logical-plans",
"library-user-guide/catalogs", "library-user-guide/custom-tab [...]
\ No newline at end of file
+Search.setIndex({"docnames": ["contributor-guide/architecture",
"contributor-guide/communication", "contributor-guide/index",
"contributor-guide/quarterly_roadmap", "contributor-guide/roadmap",
"contributor-guide/specification/index",
"contributor-guide/specification/invariants",
"contributor-guide/specification/output-field-name-semantic", "index",
"library-user-guide/adding-udfs", "library-user-guide/building-logical-plans",
"library-user-guide/catalogs", "library-user-guide/custom-tab [...]
\ No newline at end of file
diff --git a/user-guide/cli.html b/user-guide/cli.html
index 3fd8f13cd3..289544ccd4 100644
--- a/user-guide/cli.html
+++ b/user-guide/cli.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/configs.html b/user-guide/configs.html
index ac6663939e..3f57d6d02f 100644
--- a/user-guide/configs.html
+++ b/user-guide/configs.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/dataframe.html b/user-guide/dataframe.html
index 1b9f9b920f..a44d1d6fe7 100644
--- a/user-guide/dataframe.html
+++ b/user-guide/dataframe.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/example-usage.html b/user-guide/example-usage.html
index 6dae936bd9..21a88da395 100644
--- a/user-guide/example-usage.html
+++ b/user-guide/example-usage.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/expressions.html b/user-guide/expressions.html
index 093cfb29c2..2f3bc9d484 100644
--- a/user-guide/expressions.html
+++ b/user-guide/expressions.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/faq.html b/user-guide/faq.html
index 8baef125af..4c6b82f1d8 100644
--- a/user-guide/faq.html
+++ b/user-guide/faq.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/introduction.html b/user-guide/introduction.html
index 864fc98b8d..33518f7818 100644
--- a/user-guide/introduction.html
+++ b/user-guide/introduction.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/aggregate_functions.html
b/user-guide/sql/aggregate_functions.html
index e2d99dbdd6..4fd889618b 100644
--- a/user-guide/sql/aggregate_functions.html
+++ b/user-guide/sql/aggregate_functions.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/data_types.html b/user-guide/sql/data_types.html
index bac6aef40c..e2d4782048 100644
--- a/user-guide/sql/data_types.html
+++ b/user-guide/sql/data_types.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/ddl.html b/user-guide/sql/ddl.html
index 163fb1c92e..634e06d4f0 100644
--- a/user-guide/sql/ddl.html
+++ b/user-guide/sql/ddl.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/dml.html b/user-guide/sql/dml.html
index ed95837186..937cd8a279 100644
--- a/user-guide/sql/dml.html
+++ b/user-guide/sql/dml.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/explain.html b/user-guide/sql/explain.html
index bd5a132316..bc7bd17468 100644
--- a/user-guide/sql/explain.html
+++ b/user-guide/sql/explain.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/index.html b/user-guide/sql/index.html
index 5bad086863..3938bce887 100644
--- a/user-guide/sql/index.html
+++ b/user-guide/sql/index.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/information_schema.html
b/user-guide/sql/information_schema.html
index 94a2236159..fa539ffa9a 100644
--- a/user-guide/sql/information_schema.html
+++ b/user-guide/sql/information_schema.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/operators.html b/user-guide/sql/operators.html
index 35056edc7b..7ec7fa0576 100644
--- a/user-guide/sql/operators.html
+++ b/user-guide/sql/operators.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/scalar_functions.html
b/user-guide/sql/scalar_functions.html
index 468b5aac45..007621f779 100644
--- a/user-guide/sql/scalar_functions.html
+++ b/user-guide/sql/scalar_functions.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/select.html b/user-guide/sql/select.html
index 8973f90e1b..2a2e21e507 100644
--- a/user-guide/sql/select.html
+++ b/user-guide/sql/select.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/sql_status.html b/user-guide/sql/sql_status.html
index f734ebdff4..3354a03ab0 100644
--- a/user-guide/sql/sql_status.html
+++ b/user-guide/sql/sql_status.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/subqueries.html b/user-guide/sql/subqueries.html
index 302fe89844..0aafde6c37 100644
--- a/user-guide/sql/subqueries.html
+++ b/user-guide/sql/subqueries.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/window_functions.html
b/user-guide/sql/window_functions.html
index f501a705c6..a3940b438f 100644
--- a/user-guide/sql/window_functions.html
+++ b/user-guide/sql/window_functions.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
diff --git a/user-guide/sql/write_options.html
b/user-guide/sql/write_options.html
index a16f6e51d7..df59ffd1e3 100644
--- a/user-guide/sql/write_options.html
+++ b/user-guide/sql/write_options.html
@@ -269,6 +269,11 @@
Extending DataFusion’s operators: custom LogicalPlan and Execution Plans
</a>
</li>
+ <li class="toctree-l1">
+ <a class="reference internal" href="../../library-user-guide/profiling.html">
+ Profiling Cookbook
+ </a>
+ </li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
