This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new a8d30dd066 Publish built docs triggered by
c91dcfdb6a5814e87c796caa3bc7606f01425f00
a8d30dd066 is described below
commit a8d30dd066b8ce27de804c7e64f5a90d57c10e79
Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri Jan 16 12:27:41 2026 +0000
Publish built docs triggered by c91dcfdb6a5814e87c796caa3bc7606f01425f00
---
_sources/library-user-guide/extending-sql.md.txt | 5 ++++
.../functions/adding-udfs.md.txt | 4 ++++
_sources/library-user-guide/query-optimizer.md.txt | 9 +++++++
.../user-guide/concepts-readings-events.md.txt | 28 ++++++++++++++++++++++
_sources/user-guide/sql/data_types.md.txt | 5 ++++
library-user-guide/extending-sql.html | 2 ++
library-user-guide/functions/adding-udfs.html | 1 +
library-user-guide/query-optimizer.html | 4 ++++
searchindex.js | 2 +-
user-guide/concepts-readings-events.html | 14 +++++++++++
user-guide/sql/data_types.html | 2 ++
11 files changed, 75 insertions(+), 1 deletion(-)
diff --git a/_sources/library-user-guide/extending-sql.md.txt
b/_sources/library-user-guide/extending-sql.md.txt
index 409a0fb89a..b4d2519dd6 100644
--- a/_sources/library-user-guide/extending-sql.md.txt
+++ b/_sources/library-user-guide/extending-sql.md.txt
@@ -27,6 +27,11 @@ need to:
- Add custom data types not natively supported
- Implement SQL constructs like `TABLESAMPLE`, `PIVOT`/`UNPIVOT`, or
`MATCH_RECOGNIZE`
+You can read more about this topic in the [Extending SQL in DataFusion: from
->>
+to TABLESAMPLE] blog.
+
+[extending sql in datafusion: from ->> to tablesample]:
https://datafusion.apache.org/blog/2026/01/12/extending-sql
+
## Architecture Overview
When DataFusion processes a SQL query, it goes through these stages:
diff --git a/_sources/library-user-guide/functions/adding-udfs.md.txt
b/_sources/library-user-guide/functions/adding-udfs.md.txt
index 5d033ae3f9..c707495737 100644
--- a/_sources/library-user-guide/functions/adding-udfs.md.txt
+++ b/_sources/library-user-guide/functions/adding-udfs.md.txt
@@ -684,6 +684,10 @@ No function matches the given name and argument types
substr(Utf8).
Scalar UDFs are functions that take a row of data and return a single value.
Window UDFs are similar, but they also have
access to the rows around them. Access to the proximal rows is helpful, but
adds some complexity to the implementation.
+For background and other considerations, see the [User defined Window
Functions in DataFusion] blog.
+
+[user defined window functions in datafusion]:
https://datafusion.apache.org/blog/2025/04/19/user-defined-window-functions
+
For example, we will declare a user defined window function that computes a
moving average.
```rust
diff --git a/_sources/library-user-guide/query-optimizer.md.txt
b/_sources/library-user-guide/query-optimizer.md.txt
index b70ff38575..d85ddff3c9 100644
--- a/_sources/library-user-guide/query-optimizer.md.txt
+++ b/_sources/library-user-guide/query-optimizer.md.txt
@@ -28,8 +28,17 @@ This crate is a submodule of DataFusion that provides a
query optimizer for logi
contains an extensive set of [`OptimizerRule`]s and [`PhysicalOptimizerRule`]s
that may rewrite the plan and/or its expressions so
they execute more quickly while still computing the same result.
+For a deeper background on optimizer architecture and rule types and
predicates, see
+[Optimizing SQL (and DataFrames) in DataFusion, Part 1], [Part 2],
+[Using Ordering for Better Plans in Apache DataFusion], and
+[Dynamic Filters: Passing Information Between Operators During Execution for
25x Faster Queries].
+
[`optimizerrule`]:
https://docs.rs/datafusion/latest/datafusion/optimizer/trait.OptimizerRule.html
[`physicaloptimizerrule`]:
https://docs.rs/datafusion/latest/datafusion/physical_optimizer/trait.PhysicalOptimizerRule.html
+[optimizing sql (and dataframes) in datafusion, part 1]:
https://datafusion.apache.org/blog/2025/06/15/optimizing-sql-dataframes-part-one
+[part 2]:
https://datafusion.apache.org/blog/2025/06/15/optimizing-sql-dataframes-part-two
+[using ordering for better plans in apache datafusion]:
https://datafusion.apache.org/blog/2025/03/11/ordering-analysis
+[dynamic filters: passing information between operators during execution for
25x faster queries]:
https://datafusion.apache.org/blog/2025/09/10/dynamic-filters
[`logicalplan`]:
https://docs.rs/datafusion/latest/datafusion/logical_expr/enum.LogicalPlan.html
## Running the Optimizer
diff --git a/_sources/user-guide/concepts-readings-events.md.txt
b/_sources/user-guide/concepts-readings-events.md.txt
index ad444ef91c..5f6f30c1e7 100644
--- a/_sources/user-guide/concepts-readings-events.md.txt
+++ b/_sources/user-guide/concepts-readings-events.md.txt
@@ -37,6 +37,34 @@
This is a list of DataFusion related blog posts, articles, and other
resources. Please open a PR to add any new resources you create or find
+- **2026-01-12** [Blog: Extending SQL in DataFusion: from ->> to
TABLESAMPLE](https://datafusion.apache.org/blog/2026/01/12/extending-sql)
+
+- **2025-12-15** [Blog: Optimizing Repartitions in DataFusion: How I Went From
Database Noob to Core
Contribution](https://datafusion.apache.org/blog/2025/12/15/avoid-consecutive-repartitions)
+
+- **2025-09-21** [Blog: Implementing User Defined Types and Custom Metadata in
DataFusion](https://datafusion.apache.org/blog/2025/09/21/custom-types-using-metadata)
+
+- **2025-09-10** [Blog: Dynamic Filters: Passing Information Between Operators
During Execution for 25x Faster
Queries](https://datafusion.apache.org/blog/2025/09/10/dynamic-filters)
+
+- **2025-08-15** [Blog: Using External Indexes, Metadata Stores, Catalogs and
Caches to Accelerate Queries on Apache
Parquet](https://datafusion.apache.org/blog/2025/08/15/external-parquet-indexes)
+
+- **2025-07-14** [Blog: Embedding User-Defined Indexes in Apache Parquet
Files](https://datafusion.apache.org/blog/2025/07/14/user-defined-parquet-indexes)
+
+- **2025-06-30** [Blog: Using Rust async for Query Execution and Cancelling
Long-Running
Queries](https://datafusion.apache.org/blog/2025/06/30/cancellation)
+
+- **2025-06-15** [Blog: Optimizing SQL (and DataFrames) in DataFusion, Part 1:
Query Optimization
Overview](https://datafusion.apache.org/blog/2025/06/15/optimizing-sql-dataframes-part-one)
+
+- **2025-06-15** [Blog: Optimizing SQL (and DataFrames) in DataFusion, Part 2:
Optimizers in Apache
DataFusion](https://datafusion.apache.org/blog/2025/06/15/optimizing-sql-dataframes-part-two)
+
+- **2025-04-19** [Blog: User defined Window Functions in
DataFusion](https://datafusion.apache.org/blog/2025/04/19/user-defined-window-functions)
+
+- **2025-04-10** [Blog: tpchgen-rs World's fastest open source TPC-H data
generator, written in
Rust](https://datafusion.apache.org/blog/2025/04/10/fastest-tpch-generator)
+
+- **2025-03-11** [Blog: Using Ordering for Better Plans in Apache
DataFusion](https://datafusion.apache.org/blog/2025/03/11/ordering-analysis)
+
+- **2024-05-07** [Blog: Announcing Apache Arrow DataFusion is now Apache
DataFusion](https://datafusion.apache.org/blog/2024/05/07/datafusion-tlp)
+
+- **2024-03-06** [Blog: Announcing Apache Arrow DataFusion
Comet](https://datafusion.apache.org/blog/2024/03/06/comet-donation)
+
- **2025-03-21** [Blog: Efficient Filter Pushdown in
Parquet](https://datafusion.apache.org/blog/2025/03/21/parquet-pushdown/)
- **2025-03-20** [Blog: Parquet Pruning in DataFusion: Read Only What
Matters](https://datafusion.apache.org/blog/2025/03/20/parquet-pruning/)
diff --git a/_sources/user-guide/sql/data_types.md.txt
b/_sources/user-guide/sql/data_types.md.txt
index 02edb6371c..4841098dd5 100644
--- a/_sources/user-guide/sql/data_types.md.txt
+++ b/_sources/user-guide/sql/data_types.md.txt
@@ -25,6 +25,11 @@ execution. The SQL types from
are mapped to [Arrow data
types](https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html)
according to the following table.
This mapping occurs when defining the schema in a `CREATE EXTERNAL TABLE`
command or when performing a SQL `CAST` operation.
+For background on extension types and custom metadata, see the
+[Implementing User Defined Types and Custom Metadata in DataFusion] blog.
+
+[implementing user defined types and custom metadata in datafusion]:
https://datafusion.apache.org/blog/2025/09/21/custom-types-using-metadata
+
You can see the corresponding Arrow type for any SQL expression using
the `arrow_typeof` function. For example:
diff --git a/library-user-guide/extending-sql.html
b/library-user-guide/extending-sql.html
index 9e4d112d4a..e21a749a8d 100644
--- a/library-user-guide/extending-sql.html
+++ b/library-user-guide/extending-sql.html
@@ -413,6 +413,8 @@ need to:</p>
<li><p>Add custom data types not natively supported</p></li>
<li><p>Implement SQL constructs like <code class="docutils literal
notranslate"><span class="pre">TABLESAMPLE</span></code>, <code class="docutils
literal notranslate"><span class="pre">PIVOT</span></code>/<code
class="docutils literal notranslate"><span class="pre">UNPIVOT</span></code>,
or <code class="docutils literal notranslate"><span
class="pre">MATCH_RECOGNIZE</span></code></p></li>
</ul>
+<p>You can read more about this topic in the <a class="reference external"
href="https://datafusion.apache.org/blog/2026/01/12/extending-sql";>Extending
SQL in DataFusion: from ->>
+to TABLESAMPLE</a> blog.</p>
<section id="architecture-overview">
<h2>Architecture Overview<a class="headerlink" href="#architecture-overview"
title="Link to this heading">#</a></h2>
<p>When DataFusion processes a SQL query, it goes through these stages:</p>
diff --git a/library-user-guide/functions/adding-udfs.html
b/library-user-guide/functions/adding-udfs.html
index aa2840952b..d0f454a3f9 100644
--- a/library-user-guide/functions/adding-udfs.html
+++ b/library-user-guide/functions/adding-udfs.html
@@ -826,6 +826,7 @@ UDF.</p>
<h2>Adding a Window UDF<a class="headerlink" href="#adding-a-window-udf"
title="Link to this heading">#</a></h2>
<p>Scalar UDFs are functions that take a row of data and return a single
value. Window UDFs are similar, but they also have
access to the rows around them. Access to the proximal rows is helpful, but
adds some complexity to the implementation.</p>
+<p>For background and other considerations, see the <a class="reference
external"
href="https://datafusion.apache.org/blog/2025/04/19/user-defined-window-functions";>User
defined Window Functions in DataFusion</a> blog.</p>
<p>For example, we will declare a user defined window function that computes a
moving average.</p>
<div class="highlight-rust notranslate"><div
class="highlight"><pre><span></span><span class="k">use</span><span class="w">
</span><span class="n">datafusion</span><span class="p">::</span><span
class="n">arrow</span><span class="p">::{</span><span
class="n">array</span><span class="p">::{</span><span
class="n">ArrayRef</span><span class="p">,</span><span class="w"> </span><span
class="n">Float64Array</span><span class="p">,</span><span class="w">
</span><span class="n">AsArray</span><sp [...]
<span class="k">use</span><span class="w"> </span><span
class="n">datafusion</span><span class="p">::</span><span
class="n">logical_expr</span><span class="p">::{</span><span
class="n">PartitionEvaluator</span><span class="p">};</span>
diff --git a/library-user-guide/query-optimizer.html
b/library-user-guide/query-optimizer.html
index 4148a5c3dc..a3b65cb295 100644
--- a/library-user-guide/query-optimizer.html
+++ b/library-user-guide/query-optimizer.html
@@ -411,6 +411,10 @@ format.</p>
<p>This crate is a submodule of DataFusion that provides a query optimizer for
logical plans, and
contains an extensive set of <a class="reference external"
href="https://docs.rs/datafusion/latest/datafusion/optimizer/trait.OptimizerRule.html";><code
class="docutils literal notranslate"><span
class="pre">OptimizerRule</span></code></a>s and <a class="reference external"
href="https://docs.rs/datafusion/latest/datafusion/physical_optimizer/trait.PhysicalOptimizerRule.html";><code
class="docutils literal notranslate"><span
class="pre">PhysicalOptimizerRule</span></code></a>s that may rew [...]
they execute more quickly while still computing the same result.</p>
+<p>For a deeper background on optimizer architecture and rule types and
predicates, see
+<a class="reference external"
href="https://datafusion.apache.org/blog/2025/06/15/optimizing-sql-dataframes-part-one";>Optimizing
SQL (and DataFrames) in DataFusion, Part 1</a>, <a class="reference external"
href="https://datafusion.apache.org/blog/2025/06/15/optimizing-sql-dataframes-part-two";>Part
2</a>,
+<a class="reference external"
href="https://datafusion.apache.org/blog/2025/03/11/ordering-analysis";>Using
Ordering for Better Plans in Apache DataFusion</a>, and
+<a class="reference external"
href="https://datafusion.apache.org/blog/2025/09/10/dynamic-filters";>Dynamic
Filters: Passing Information Between Operators During Execution for 25x Faster
Queries</a>.</p>
<section id="running-the-optimizer">
<h2>Running the Optimizer<a class="headerlink" href="#running-the-optimizer"
title="Link to this heading">#</a></h2>
<p>The following code demonstrates the basic flow of creating the optimizer
with a default set of optimization rules
diff --git a/searchindex.js b/searchindex.js
index db6580bde1..1d534802d4 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[62,"op-neq"]],"!~":[[62,"op-re-not-match"]],"!~*":[[62,"op-re-not-match-i"]],"!~~":[[62,"id19"]],"!~~*":[[62,"id20"]],"#":[[62,"op-bit-xor"]],"%":[[62,"op-modulo"]],"&":[[62,"op-bit-and"]],"(relation,
name) tuples in logical fields and logical columns are
unique":[[14,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[62,"op-multiply"]],"+":[[62,"op-plus"]],"-":[[62,"op-minus"]],"/":[[62,"op-divide"]],"<":[[62,"op-lt"]],"<
[...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[62,"op-neq"]],"!~":[[62,"op-re-not-match"]],"!~*":[[62,"op-re-not-match-i"]],"!~~":[[62,"id19"]],"!~~*":[[62,"id20"]],"#":[[62,"op-bit-xor"]],"%":[[62,"op-modulo"]],"&":[[62,"op-bit-and"]],"(relation,
name) tuples in logical fields and logical columns are
unique":[[14,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[62,"op-multiply"]],"+":[[62,"op-plus"]],"-":[[62,"op-minus"]],"/":[[62,"op-divide"]],"<":[[62,"op-lt"]],"<
[...]
\ No newline at end of file
diff --git a/user-guide/concepts-readings-events.html
b/user-guide/concepts-readings-events.html
index bdf7321efd..a110b5fcbf 100644
--- a/user-guide/concepts-readings-events.html
+++ b/user-guide/concepts-readings-events.html
@@ -420,6 +420,20 @@
<h2>✨ Good Reads<a class="headerlink" href="#good-reads" title="Link to this
heading">#</a></h2>
<p>This is a list of DataFusion related blog posts, articles, and other
resources. Please open a PR to add any new resources you create or find</p>
<ul class="simple">
+<li><p><strong>2026-01-12</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2026/01/12/extending-sql";>Blog:
Extending SQL in DataFusion: from ->> to TABLESAMPLE</a></p></li>
+<li><p><strong>2025-12-15</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/12/15/avoid-consecutive-repartitions";>Blog:
Optimizing Repartitions in DataFusion: How I Went From Database Noob to Core
Contribution</a></p></li>
+<li><p><strong>2025-09-21</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/09/21/custom-types-using-metadata";>Blog:
Implementing User Defined Types and Custom Metadata in DataFusion</a></p></li>
+<li><p><strong>2025-09-10</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/09/10/dynamic-filters";>Blog:
Dynamic Filters: Passing Information Between Operators During Execution for 25x
Faster Queries</a></p></li>
+<li><p><strong>2025-08-15</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/08/15/external-parquet-indexes";>Blog:
Using External Indexes, Metadata Stores, Catalogs and Caches to Accelerate
Queries on Apache Parquet</a></p></li>
+<li><p><strong>2025-07-14</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/07/14/user-defined-parquet-indexes";>Blog:
Embedding User-Defined Indexes in Apache Parquet Files</a></p></li>
+<li><p><strong>2025-06-30</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/06/30/cancellation";>Blog: Using
Rust async for Query Execution and Cancelling Long-Running Queries</a></p></li>
+<li><p><strong>2025-06-15</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/06/15/optimizing-sql-dataframes-part-one";>Blog:
Optimizing SQL (and DataFrames) in DataFusion, Part 1: Query Optimization
Overview</a></p></li>
+<li><p><strong>2025-06-15</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/06/15/optimizing-sql-dataframes-part-two";>Blog:
Optimizing SQL (and DataFrames) in DataFusion, Part 2: Optimizers in Apache
DataFusion</a></p></li>
+<li><p><strong>2025-04-19</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/04/19/user-defined-window-functions";>Blog:
User defined Window Functions in DataFusion</a></p></li>
+<li><p><strong>2025-04-10</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/04/10/fastest-tpch-generator";>Blog:
tpchgen-rs World’s fastest open source TPC-H data generator, written in
Rust</a></p></li>
+<li><p><strong>2025-03-11</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/03/11/ordering-analysis";>Blog:
Using Ordering for Better Plans in Apache DataFusion</a></p></li>
+<li><p><strong>2024-05-07</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2024/05/07/datafusion-tlp";>Blog:
Announcing Apache Arrow DataFusion is now Apache DataFusion</a></p></li>
+<li><p><strong>2024-03-06</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2024/03/06/comet-donation";>Blog:
Announcing Apache Arrow DataFusion Comet</a></p></li>
<li><p><strong>2025-03-21</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/03/21/parquet-pushdown/";>Blog:
Efficient Filter Pushdown in Parquet</a></p></li>
<li><p><strong>2025-03-20</strong> <a class="reference external"
href="https://datafusion.apache.org/blog/2025/03/20/parquet-pruning/";>Blog:
Parquet Pruning in DataFusion: Read Only What Matters</a></p></li>
<li><p><strong>2025-02-12</strong> <a class="reference external"
href="https://www.youtube.com/watch?v=6A4vFRpSq3k";>Video: Alex Kesling on
Apache Arrow DataFusion - Papers We Love NYC </a></p></li>
diff --git a/user-guide/sql/data_types.html b/user-guide/sql/data_types.html
index 1692ebb8e6..1272bcbd6d 100644
--- a/user-guide/sql/data_types.html
+++ b/user-guide/sql/data_types.html
@@ -413,6 +413,8 @@ execution. The SQL types from
<a class="reference external"
href="https://github.com/sqlparser-rs/sqlparser-rs/blob/main/src/ast/data_type.rs#L27";>sqlparser-rs</a>
are mapped to <a class="reference external"
href="https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html";>Arrow
data types</a> according to the following table.
This mapping occurs when defining the schema in a <code class="docutils
literal notranslate"><span class="pre">CREATE</span> <span
class="pre">EXTERNAL</span> <span class="pre">TABLE</span></code> command or
when performing a SQL <code class="docutils literal notranslate"><span
class="pre">CAST</span></code> operation.</p>
+<p>For background on extension types and custom metadata, see the
+<a class="reference external"
href="https://datafusion.apache.org/blog/2025/09/21/custom-types-using-metadata";>Implementing
User Defined Types and Custom Metadata in DataFusion</a> blog.</p>
<p>You can see the corresponding Arrow type for any SQL expression using
the <code class="docutils literal notranslate"><span
class="pre">arrow_typeof</span></code> function. For example:</p>
<div class="highlight-sql notranslate"><div
class="highlight"><pre><span></span><span class="k">select</span><span
class="w"> </span><span class="n">arrow_typeof</span><span
class="p">(</span><span class="nb">interval</span><span class="w"> </span><span
class="s1">'1 month'</span><span class="p">);</span>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
