This is an automated email from the ASF dual-hosted git repository.

github-actions[bot] pushed a commit to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/datafusion-site.git


The following commit(s) were added to refs/heads/asf-staging by this push:
     new ddc6b82  Commit build products
ddc6b82 is described below

commit ddc6b822f1348483d680f4469caab2b54578d0a7
Author: Build Pelican (action) <[email protected]>
AuthorDate: Sun Aug 9 14:43:43 2026 +0000

    Commit build products
---
 blog/2026/08/07/datafusion-comet-1.0.0/index.html  | 318 ---------------------
 .../08/09/datafusion-ballista-54.1.0/index.html    | 201 +++++++++++++
 blog/author/pmc.html                               |  12 +-
 blog/category/blog.html                            |  12 +-
 blog/feed.xml                                      |  10 +-
 blog/feeds/all-en.atom.xml                         | 193 ++++---------
 blog/feeds/blog.atom.xml                           | 193 ++++---------
 blog/feeds/pmc.atom.xml                            | 203 ++++---------
 blog/feeds/pmc.rss.xml                             |  10 +-
 blog/index.html                                    |  14 +-
 10 files changed, 388 insertions(+), 778 deletions(-)

diff --git a/blog/2026/08/07/datafusion-comet-1.0.0/index.html 
b/blog/2026/08/07/datafusion-comet-1.0.0/index.html
deleted file mode 100644
index a909d02..0000000
--- a/blog/2026/08/07/datafusion-comet-1.0.0/index.html
+++ /dev/null
@@ -1,318 +0,0 @@
-<!doctype html>
-<html class="no-js" lang="en" dir="ltr">
-  <head>
-    <meta charset="utf-8">
-    <meta http-equiv="x-ua-compatible" content="ie=edge">
-    <meta name="viewport" content="width=device-width, initial-scale=1.0">
-    <title>Apache DataFusion Comet 1.0.0 Release - Apache DataFusion 
Blog</title>
-<link href="/blog/favicon.ico" rel="icon" type="image/x-icon">
-<link href="/blog/favicon.ico" rel="shortcut icon" type="image/x-icon">
-<link href="/blog/css/bootstrap.min.css" rel="stylesheet">
-<link href="/blog/css/fontawesome.all.min.css" rel="stylesheet">
-<link href="/blog/css/headerlink.css" rel="stylesheet">
-<link href="/blog/highlight/default.min.css" rel="stylesheet">
-<link href="/blog/css/app.css" rel="stylesheet">
-<link href="/blog/css/dark-mode.css" rel="stylesheet">
-<script src="/blog/js/dark-mode.js"></script>
-<script src="/blog/highlight/highlight.js"></script>
-<script>hljs.highlightAll();</script>  </head>
-  <body class="d-flex flex-column h-100">
-  <main class="flex-shrink-0">
-<!-- nav bar -->
-<nav class="navbar navbar-expand-lg navbar-dark bg-dark" aria-label="Fifth 
navbar example">
-    <div class="container-fluid">
-        <a class="navbar-brand" href="/blog/"><img 
src="/blog/images/logo_original4x.png" style="height: 32px;"/> Apache 
DataFusion Blog</a>
-        <button class="navbar-toggler" type="button" data-bs-toggle="collapse" 
data-bs-target="#navbarADP" aria-controls="navbarADP" aria-expanded="false" 
aria-label="Toggle navigation">
-            <span class="navbar-toggler-icon"></span>
-        </button>
-
-        <div class="collapse navbar-collapse" id="navbarADP">
-            <ul class="navbar-nav me-auto mb-2 mb-lg-0">
-                <li class="nav-item">
-                    <a class="nav-link" href="/blog/about.html">About</a>
-                </li>
-                <li class="nav-item">
-                    <a class="nav-link" href="/blog/feed.xml">RSS</a>
-                </li>
-            </ul>
-            <button id="dark-mode-toggle" type="button" 
class="dark-mode-toggle" aria-label="Toggle dark mode" aria-pressed="false" 
title="Toggle dark mode">
-                <span class="sun-icon" aria-hidden="true">☀</span>
-                <span class="moon-icon" aria-hidden="true">☾</span>
-            </button>
-        </div>
-    </div>
-</nav>    
-<!-- article contents -->
-<div id="contents">
-  <div class="bg-white p-4 p-md-5 rounded">
-    <div class="row justify-content-center">
-      <div class="col-12 col-md-8 main-content">
-        <h1>
-          Apache DataFusion Comet 1.0.0 Release
-        </h1>
-        <p>Posted on: Fri 07 August 2026 by pmc</p>
-
-        <aside class="toc-container d-md-none mb-2">
-          <div class="toc"><span class="toctitle">Contents</span><ul>
-<li><a href="#the-road-to-10">The Road to 1.0</a><ul>
-<li><a href="#support-for-spark-40-with-ansi-mode">Support for Spark 4.0+ with 
ANSI mode</a></li>
-<li><a href="#correctness-testing">Correctness Testing</a></li>
-<li><a href="#performance">Performance</a></li>
-<li><a href="#codegen-dispatch">Codegen Dispatch</a></li>
-</ul>
-</li>
-<li><a href="#improvements-since-0170">Improvements since 0.17.0</a><ul>
-<li><a href="#experimental-pyarrow-udf-support">Experimental PyArrow UDF 
Support</a></li>
-<li><a href="#new-expression-and-aggregate-support">New Expression and 
Aggregate Support</a></li>
-<li><a href="#faster-parquet-scans">Faster Parquet Scans</a></li>
-<li><a href="#iceberg-table-format-v3">Iceberg Table Format V3</a></li>
-<li><a href="#native-expression-performance">Native Expression 
Performance</a></li>
-</ul>
-</li>
-<li><a href="#deprecation-notice">Deprecation Notice</a></li>
-<li><a href="#compatibility">Compatibility</a></li>
-<li><a href="#get-started-with-comet-100">Get Started with Comet 1.0.0</a></li>
-</ul>
-</div>
-        </aside>
-
-        <!--
-{% comment %}
-Licensed to the Apache Software Foundation (ASF) under one or more
-contributor license agreements.  See the NOTICE file distributed with
-this work for additional information regarding copyright ownership.
-The ASF licenses this file to you under the Apache License, Version 2.0
-(the "License"); you may not use this file except in compliance with
-the License.  You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-{% endcomment %}
--->
-
-<p>The Apache DataFusion PMC is pleased to announce version 1.0.0 of the <a 
href="https://datafusion.apache.org/comet/";>Comet</a> subproject.</p>
-<p>Comet is an accelerator for Apache Spark that translates Spark physical 
plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code changes.</p>
-<p>This release covers roughly six weeks of development since 0.17.0 and 
consists of 244 commits from 23
-contributors. See the <a 
href="https://github.com/apache/datafusion-comet/blob/main/docs/source/changelog/1.0.0.md";>change
 log</a> for the full list of changes.</p>
-<h2 id="the-road-to-10">The Road to 1.0<a class="headerlink" 
href="#the-road-to-10" title="Permanent link">¶</a></h2>
-<p>Comet was <a 
href="https://datafusion.apache.org/blog/2024/03/06/comet-donation/";>donated</a>
 to the Apache DataFusion project in March 2024 and cut its first release, 
0.1.0, five
-months later with support for 13 operators and 106 expressions. Since then, 
the project has shipped 20
-releases and drawn contributions from more than 120 developers, and the 
codebase now recognizes over 400
-Spark expressions. Operator coverage has grown alongside it: 1.0 accelerates 
each of Spark's four join
-operators, window functions, generators (<code>explode</code>, 
<code>explode_outer</code>, <code>posexplode</code>, and 
<code>posexplode_outer</code>
-over arrays), sampling, in-memory table scans, and a fully native shuffle.</p>
-<p>The 1.0 release marks the point at which Comet begins following <a 
href="https://datafusion.apache.org/comet/about/versioning_policy.html";>semantic
 versioning</a>. Users upgrading
-within the 1.x line can expect backward-compatible changes only; features 
slated for removal will be
-deprecated in a minor release before being dropped in the next major version. 
This is why the deprecations
-of JDK 11 and Spark 3.4 announced below are scheduled for 1.1 rather than 
landing in 1.0 itself.</p>
-<h3 id="support-for-spark-40-with-ansi-mode">Support for Spark 4.0+ with ANSI 
mode<a class="headerlink" href="#support-for-spark-40-with-ansi-mode" 
title="Permanent link">¶</a></h3>
-<p>Comet 1.0.0 supports Spark versions 3.4 through 4.1, with experimental 
support for 4.2. Comet fully supports Spark's ANSI mode, which is enabled by 
default starting with Spark 4.0.</p>
-<h3 id="correctness-testing">Correctness Testing<a class="headerlink" 
href="#correctness-testing" title="Permanent link">¶</a></h3>
-<p>It is important that queries accelerated by Comet produce the same results 
as Spark. Correctness checking has always been a large effort in Comet 
development, but the approach has evolved over time.</p>
-<ul>
-<li><strong>Upstream Spark tests</strong>: Comet runs Spark's own test suite 
with Comet enabled, providing more than 24,000 unit tests effectively for free. 
These tests run in Comet's CI for all supported Spark versions.</li>
-<li><strong>Scala tests</strong>: end-to-end queries that run with Comet 
enabled versus disabled, checking that results match.</li>
-<li><strong>Fuzz testing</strong>: many of the Scala tests generate randomized 
data to catch regressions around edge cases such as nulls, NaN, Infinity, and 
timezone issues.</li>
-<li><strong>Comet SQL tests</strong>: a sqllogictest-inspired approach that 
makes end-to-end tests easier to write.</li>
-<li><strong>Generative AI audits</strong>: agentic skills sweep every 
expression, comparing Comet's implementation to Spark's source and ensuring 
tests cover important edge cases.</li>
-</ul>
-<h3 id="performance">Performance<a class="headerlink" href="#performance" 
title="Permanent link">¶</a></h3>
-<p>The early Comet releases provided a very modest speedup and the published 
benchmark results were based on running TPC workloads at small scale factors on 
a single node. There are now independent benchmark results published by AWS 
Labs that show significant speedups for <a 
href="https://awslabs.github.io/data-on-eks/docs/benchmarks/spark-datafusion-comet-benchmark";>TPC-DS
 @ 3TB running in EKS</a>.</p>
-<h3 id="codegen-dispatch">Codegen Dispatch<a class="headerlink" 
href="#codegen-dispatch" title="Permanent link">¶</a></h3>
-<p>Comet 0.17.0 introduced a new approach to filling gaps in expression 
coverage. In earlier releases,
-whenever Comet's planner encountered an expression that lacked a native Rust 
implementation, it fell back
-to executing an entire subtree of the plan in Spark. That required converting 
Arrow columns back to Spark
-rows before the expression ran and back to Arrow after, and the cost was often 
enough to erase the speedup
-Comet had bought elsewhere in the plan.</p>
-<p>Codegen dispatch narrows that fallback to the expression itself: the batch 
stays in the Comet pipeline
-and Comet invokes Spark's own generated code for just the missing expression, 
leaving the rest of the query
-running natively. Four consequences are worth calling out.</p>
-<ul>
-<li><strong>Coverage.</strong> Expressions that would previously have blocked 
native execution of a whole subtree are
-  now supported immediately, without a Rust port.</li>
-<li><strong>Compatibility.</strong> For categories where a native 
reimplementation would inevitably diverge from Spark's
-  semantics — regular expressions being the canonical case, given the gap 
between Java's regex engine and
-  any Rust or C++ equivalent — codegen dispatch delivers bit-for-bit Spark 
parity because it <em>is</em> Spark's
-  implementation.</li>
-<li><strong>Expression fusion.</strong> A dispatched expression tree 
(<em>i.e.</em>, nested expressions) is compiled into a single method, so the 
Arrow
-  input reads, the expression evaluation, and the Arrow output writes are 
fused together. The compiler
-  is free to optimize across the whole tree, and no intermediate Arrow 
<code>RecordBatch</code> is materialized
-  between one expression and the next.</li>
-<li><strong>Scala and Java UDFs.</strong> User-defined functions are compiled 
to the same codegen surface as built-in
-  expressions, so they can flow through codegen dispatch without any change 
from the user. Queries that
-  were previously disqualified from acceleration only because they contained a 
UDF can now benefit as
-  long as the surrounding operators are supported. See the <a 
href="https://datafusion.apache.org/comet/user-guide/latest/scala_java_udfs.html";>Scala
 and Java UDF guide</a> for details.</li>
-</ul>
-<p>Comet 1.0 widens the mechanism in three ways.</p>
-<p>The first is the biggest. An expression that opts into codegen dispatch 
previously reached the dispatcher
-only when Comet reported it as <em>incompatible</em> for the given input; an 
<em>unsupported</em> report still sent the
-whole subtree back to Spark. In 1.0 both support levels route through the 
dispatcher, so an input that
-Spark handles and Comet's native code does not now stays inside the Comet 
pipeline.</p>
-<p>Second, casts join the same path. Cast expressions that Comet declines to 
run natively — including legacy
-configuration variants such as 
<code>spark.sql.legacy.castComplexTypesToString.enabled</code> — are now 
dispatched
-rather than falling back, and more string, array, and interval expressions 
were opted in as well.</p>
-<p>Third, the path is now visible. Comet's extended explain output reports 
native versus codegen-dispatch
-coverage for a plan, so you can see which path each expression actually took 
rather than inferring it from
-the absence of a fallback reason.</p>
-<h2 id="improvements-since-0170">Improvements since 0.17.0<a 
class="headerlink" href="#improvements-since-0170" title="Permanent 
link">¶</a></h2>
-<p>The rest of this post covers what is new since the 0.17.0 release.</p>
-<h3 id="experimental-pyarrow-udf-support">Experimental PyArrow UDF Support<a 
class="headerlink" href="#experimental-pyarrow-udf-support" title="Permanent 
link">¶</a></h3>
-<p>This release adds experimental support for accelerated PyArrow UDFs, 
allowing PyArrow-based user-defined
-functions to participate in native execution instead of forcing a fallback to 
Spark. When the feature is
-disabled, Comet now hints at the native PyArrow UDF path in its fallback 
reasons so users know the option
-exists. This is an early-stage feature and we welcome feedback from users 
experimenting with it.</p>
-<h3 id="new-expression-and-aggregate-support">New Expression and Aggregate 
Support<a class="headerlink" href="#new-expression-and-aggregate-support" 
title="Permanent link">¶</a></h3>
-<p>This release expands the set of Spark expressions and aggregates that are 
accelerated by Comet:</p>
-<ul>
-<li><strong>Aggregates</strong>: <code>approx_percentile</code> / 
<code>percentile_approx</code>, exact <code>percentile</code> / 
<code>median</code>,
-  <code>approx_count_distinct</code>, and native <code>collect_list</code> / 
<code>array_agg</code>.</li>
-<li><strong>Cast</strong>: Cast expressions where the native implementation is 
marked as incompatible or unsupported
-  are now routed through codegen dispatch.</li>
-<li><strong>Grouping</strong>: <code>grouping()</code> and 
<code>grouping_id()</code>.</li>
-<li><strong>Intervals</strong>: interval types via 
<code>make_ym_interval</code> and <code>make_dt_interval</code>, 
<code>CalendarIntervalType</code>,
-  <code>multiply_dt_interval</code>, and interval codegen dispatch for nested 
values and native shuffle.</li>
-<li><strong>String</strong>: <code>base64</code>, <code>split_part</code> via 
<code>StringSplitSQL</code>, native <code>levenshtein</code>, and native 
<code>randstr</code> and
-  <code>uuid</code> — both bit-for-bit compatible with Spark for a given 
seed.</li>
-<li><strong>Array / map</strong>: <code>array_prepend</code>, the 
<code>shuffle()</code> array function, <code>size()</code> for 
<code>MapType</code>, and <code>ElementAt</code>
-  over <code>MapType</code>.</li>
-<li><strong>Date/time</strong>: native <code>TimestampNTZ</code> inputs for 
<code>hour</code> / <code>minute</code> / <code>second</code> and
-  <code>PreciseTimestampConversion</code> for native time-window grouping.</li>
-<li><strong>Windows</strong>: extended native window function support and 
Spark 4 decimal window average.</li>
-</ul>
-<h3 id="faster-parquet-scans">Faster Parquet Scans<a class="headerlink" 
href="#faster-parquet-scans" title="Permanent link">¶</a></h3>
-<p>Parquet reads pick up several improvements as well. Full Parquet metadata, 
including the page index, is now
-cached via DataFusion's <code>CachedParquetFileReaderFactory</code>; identity 
casts are unwrapped in the schema adapter
-so Parquet statistics pruning can engage; filter pushdown configuration has 
been revised; the native scan
-passes a metadata size hint so a single read usually captures the footer; and 
the native Parquet scan seeds
-its reader options from the session config so Parquet settings you already set 
take effect.</p>
-<h3 id="iceberg-table-format-v3">Iceberg Table Format V3<a class="headerlink" 
href="#iceberg-table-format-v3" title="Permanent link">¶</a></h3>
-<p>Comet now supports Iceberg 1.11 and its first Iceberg table format V3 
feature: full table encryption. Other V3 features like deletion vectors and new 
data types (<em>e.g.</em>, <code>VARIANT</code>) fall back gracefully. The 
native Iceberg scan supports the <code>_pos</code>, <code>_spec</code>, 
<code>_file</code>, and
-<code>_partition</code> metadata columns, sizes delete files correctly to 
avoid dropped deletes, disambiguates scans that
-share a <code>metadata_location</code>, and dedupes residuals and delete files 
in the native scan serde. A prior case
-where Iceberg native scan exchange reuse with different pushed filters could 
produce wrong results is also
-fixed.</p>
-<h3 id="native-expression-performance">Native Expression Performance<a 
class="headerlink" href="#native-expression-performance" title="Permanent 
link">¶</a></h3>
-<p>Many native expression implementations have been optimized to more 
efficiently leverage Arrow kernels or to avoid per-row builders.</p>
-<ul>
-<li><strong>Casts</strong> between numeric, string, decimal, and date types, 
including a faster float-to-decimal cast, an
-  optimized integer-to-integer cast, shared no-overflow fast paths in 
<code>CheckOverflow</code> and
-  <code>DecimalRescaleCheckOverflow</code>, and a 
<code>cast_binary_to_string</code> that is up to 27x faster on binary-format
-  styles.</li>
-<li><strong>JSON, regex, and URL parsing</strong>: 
<code>get_json_object</code>, <code>regexp_extract</code>, and 
<code>parse_url</code>.</li>
-<li><strong>Date/time and decimal kernels</strong>: <code>date_trunc</code>, 
<code>spark_ceil</code>, and a vectorized 
<code>spark_unscaled_value</code>.</li>
-<li><strong>String and array kernels</strong>: <code>lpad</code>, 
<code>unhex</code>, <code>size</code>, <code>arrays_overlap</code>, 
<code>escape_string</code>, and the <code>try_*</code>
-  arithmetic kernel.</li>
-</ul>
-<p>To make this kind of work repeatable, the release also adds a <a 
href="https://datafusion.apache.org/comet/contributor-guide/optimizing_expressions.html";>scalar
 expression optimization guide</a>
-documenting how to benchmark a kernel, keep its output bit-identical to Spark, 
and gate changes on a
-no-regression check.</p>
-<h2 id="deprecation-notice">Deprecation Notice<a class="headerlink" 
href="#deprecation-notice" title="Permanent link">¶</a></h2>
-<p>With the move to a stable 1.0 release line, Comet begins deprecating older 
platforms under its
-<a 
href="https://datafusion.apache.org/comet/about/versioning_policy.html";>versioning
 policy</a>:</p>
-<ul>
-<li><strong>JDK 11</strong> is deprecated and scheduled for removal in Comet 
1.1.0.</li>
-<li><strong>Apache Spark 3.4</strong> is deprecated and scheduled for removal 
in Comet 1.1.0.</li>
-</ul>
-<p>Comet aligns its Spark support window with upstream Apache Spark 
maintenance. Spark 3.4 is no longer
-maintained upstream, so under the versioning policy it is deprecated in the 
first Comet minor release
-after that point and removed in the following one. Comet 1.0.0 still builds 
and publishes Spark 3.4
-binaries.</p>
-<p>Users on these platforms should plan to move to JDK 17+ and Spark 3.5 or 
later before upgrading to 1.1.0.</p>
-<h2 id="compatibility">Compatibility<a class="headerlink" 
href="#compatibility" title="Permanent link">¶</a></h2>
-<p>Supported platforms include:</p>
-<ul>
-<li><strong>Spark 3.4.3</strong> with Java 11/17 and Scala 2.12/2.13 
(deprecated, removal in 1.1.0)</li>
-<li><strong>Spark 3.5.9</strong> with Java 11/17 and Scala 2.12/2.13</li>
-<li><strong>Spark 4.0.4</strong> with Java 17 and Scala 2.13</li>
-<li><strong>Spark 4.1.3</strong> with Java 17/21 and Scala 2.13</li>
-<li><strong>Spark 4.2</strong> with Java 17 and Scala 2.13 (experimental, for 
early evaluation only)</li>
-</ul>
-<p>See the <a 
href="https://datafusion.apache.org/comet/user-guide/latest/compatibility/spark-versions.html";>Spark
 Version Compatibility</a> page for known limitations specific to each 
version.</p>
-<p>This release upgrades to <strong>DataFusion 54.1</strong> and <strong>Arrow 
58.4</strong>.</p>
-<h2 id="get-started-with-comet-100">Get Started with Comet 1.0.0<a 
class="headerlink" href="#get-started-with-comet-100" title="Permanent 
link">¶</a></h2>
-<p>Ready to try it out? Follow the <a 
href="https://datafusion.apache.org/comet/user-guide/1.0/installation.html";>Comet
 1.0.0 Installation Guide</a>
-to get up and running, then point Comet at your existing Spark workloads and 
see the speedup for yourself.</p>
-
-<!--
-  Comments Section
-  Loaded only after explicit visitor consent to comply with ASF policy.
--->
-
-<div>
-  <hr>
-  <h3 id="comments">Comments<a class="headerlink" href="#comments" 
title="Permanent link">&para;</a></h3>
-
-  <!-- Local loader script -->
-  <script src="/blog/js/giscus-consent.js" defer></script>
-
-  <!-- Consent UI -->
-  <div id="giscus-consent">
-    <p>
-        We use <a href="https://giscus.app/";>Giscus</a> for comments, powered 
by GitHub Discussions.
-        To respect your privacy, Giscus and comments will load only if you 
click "Show Comments"
-    </p>
-
-    <div class="consent-actions">
-      <button id="giscus-load" type="button">Show Comments</button>
-      <button id="giscus-revoke" type="button" hidden>Hide Comments</button>
-    </div>
-
-    <noscript>JavaScript is required to load comments from Giscus.</noscript>
-  </div>
-
-  <!-- Container where Giscus will render -->
-  <div id="comment-thread"></div>
-</div>      </div>
-      <aside class="toc-container d-none d-md-block col-md-4 col-xl-3 ms-xl-2">
-        <div class="toc"><span class="toctitle">Contents</span><ul>
-<li><a href="#the-road-to-10">The Road to 1.0</a><ul>
-<li><a href="#support-for-spark-40-with-ansi-mode">Support for Spark 4.0+ with 
ANSI mode</a></li>
-<li><a href="#correctness-testing">Correctness Testing</a></li>
-<li><a href="#performance">Performance</a></li>
-<li><a href="#codegen-dispatch">Codegen Dispatch</a></li>
-</ul>
-</li>
-<li><a href="#improvements-since-0170">Improvements since 0.17.0</a><ul>
-<li><a href="#experimental-pyarrow-udf-support">Experimental PyArrow UDF 
Support</a></li>
-<li><a href="#new-expression-and-aggregate-support">New Expression and 
Aggregate Support</a></li>
-<li><a href="#faster-parquet-scans">Faster Parquet Scans</a></li>
-<li><a href="#iceberg-table-format-v3">Iceberg Table Format V3</a></li>
-<li><a href="#native-expression-performance">Native Expression 
Performance</a></li>
-</ul>
-</li>
-<li><a href="#deprecation-notice">Deprecation Notice</a></li>
-<li><a href="#compatibility">Compatibility</a></li>
-<li><a href="#get-started-with-comet-100">Get Started with Comet 1.0.0</a></li>
-</ul>
-</div>
-      </aside>
-    </div>
-  </div>
-</div>    
-    <!-- footer -->
-    <div class="row g-0">
-      <div class="col-12">
-        <p style="font-style: italic; font-size: 0.8rem; text-align: center;">
-          Copyright 2026, <a href="https://www.apache.org/";>The Apache 
Software Foundation</a>, Licensed under the <a 
href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 
2.0</a>.<br/>
-          Apache&reg; and the Apache logo are trademarks of The Apache 
Software Foundation.
-        </p>
-        <p style="font-size: 0.8rem; text-align: center;">
-          <a href="https://www.apache.org/";>Foundation</a> |
-          <a 
href="https://www.apache.org/events/current-event.html";>Events</a> |
-          <a href="https://www.apache.org/licenses/";>License</a> |
-          <a href="https://www.apache.org/security/";>Security</a> |
-          <a 
href="https://www.apache.org/foundation/sponsorship.html";>Sponsorship</a> |
-          <a href="https://www.apache.org/foundation/thanks.html";>Thanks</a> |
-          <a 
href="https://privacy.apache.org/policies/privacy-policy-public.html";>Privacy</a>
-        </p>
-      </div>
-    </div>
-    <script src="/blog/js/bootstrap.bundle.min.js"></script>  </main>
-  </body>
-</html>
diff --git a/blog/2026/08/09/datafusion-ballista-54.1.0/index.html 
b/blog/2026/08/09/datafusion-ballista-54.1.0/index.html
new file mode 100644
index 0000000..ce0fe81
--- /dev/null
+++ b/blog/2026/08/09/datafusion-ballista-54.1.0/index.html
@@ -0,0 +1,201 @@
+<!doctype html>
+<html class="no-js" lang="en" dir="ltr">
+  <head>
+    <meta charset="utf-8">
+    <meta http-equiv="x-ua-compatible" content="ie=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Apache DataFusion Ballista 54.1.0 Released - Apache DataFusion 
Blog</title>
+<link href="/blog/favicon.ico" rel="icon" type="image/x-icon">
+<link href="/blog/favicon.ico" rel="shortcut icon" type="image/x-icon">
+<link href="/blog/css/bootstrap.min.css" rel="stylesheet">
+<link href="/blog/css/fontawesome.all.min.css" rel="stylesheet">
+<link href="/blog/css/headerlink.css" rel="stylesheet">
+<link href="/blog/highlight/default.min.css" rel="stylesheet">
+<link href="/blog/css/app.css" rel="stylesheet">
+<link href="/blog/css/dark-mode.css" rel="stylesheet">
+<script src="/blog/js/dark-mode.js"></script>
+<script src="/blog/highlight/highlight.js"></script>
+<script>hljs.highlightAll();</script>  </head>
+  <body class="d-flex flex-column h-100">
+  <main class="flex-shrink-0">
+<!-- nav bar -->
+<nav class="navbar navbar-expand-lg navbar-dark bg-dark" aria-label="Fifth 
navbar example">
+    <div class="container-fluid">
+        <a class="navbar-brand" href="/blog/"><img 
src="/blog/images/logo_original4x.png" style="height: 32px;"/> Apache 
DataFusion Blog</a>
+        <button class="navbar-toggler" type="button" data-bs-toggle="collapse" 
data-bs-target="#navbarADP" aria-controls="navbarADP" aria-expanded="false" 
aria-label="Toggle navigation">
+            <span class="navbar-toggler-icon"></span>
+        </button>
+
+        <div class="collapse navbar-collapse" id="navbarADP">
+            <ul class="navbar-nav me-auto mb-2 mb-lg-0">
+                <li class="nav-item">
+                    <a class="nav-link" href="/blog/about.html">About</a>
+                </li>
+                <li class="nav-item">
+                    <a class="nav-link" href="/blog/feed.xml">RSS</a>
+                </li>
+            </ul>
+            <button id="dark-mode-toggle" type="button" 
class="dark-mode-toggle" aria-label="Toggle dark mode" aria-pressed="false" 
title="Toggle dark mode">
+                <span class="sun-icon" aria-hidden="true">☀</span>
+                <span class="moon-icon" aria-hidden="true">☾</span>
+            </button>
+        </div>
+    </div>
+</nav>    
+<!-- article contents -->
+<div id="contents">
+  <div class="bg-white p-4 p-md-5 rounded">
+    <div class="row justify-content-center">
+      <div class="col-12 col-md-8 main-content">
+        <h1>
+          Apache DataFusion Ballista 54.1.0 Released
+        </h1>
+        <p>Posted on: Sun 09 August 2026 by pmc</p>
+
+        <aside class="toc-container d-md-none mb-2">
+          <div class="toc"><span class="toctitle">Contents</span><ul>
+<li><a href="#upgrade-to-datafusion-5410">Upgrade to DataFusion 54.1.0</a></li>
+<li><a href="#cluster-stability-fixes">Cluster stability fixes</a></li>
+<li><a href="#task-retry-and-shuffle-memory">Task retry and shuffle 
memory</a></li>
+<li><a href="#planning-correctness">Planning correctness</a></li>
+<li><a href="#thank-you">Thank You</a></li>
+</ul>
+</div>
+        </aside>
+
+        <!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements.  See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License.  You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+<p>We are pleased to announce version <a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md";>54.1.0</a>
 of <a href="https://datafusion.apache.org/ballista/";>Apache DataFusion 
Ballista</a>. Ballista is a distributed query
+execution engine that enhances <a href="https://datafusion.apache.org";>Apache 
DataFusion</a> by enabling parallel execution of workloads across multiple
+nodes.</p>
+<p>This is a patch release, consisting of 10 commits from 3 contributors. It 
follows <a href="/blog/2026/07/12/datafusion-ballista-54.0.0/">54.0.0</a> and 
contains no
+new features and no breaking changes — only an upgrade to DataFusion 54.1.0 
and a set of bug fixes
+backported from <code>main</code>. Users running 54.0.0 are encouraged to 
upgrade.</p>
+<h2 id="upgrade-to-datafusion-5410">Upgrade to DataFusion 54.1.0<a 
class="headerlink" href="#upgrade-to-datafusion-5410" title="Permanent 
link">¶</a></h2>
+<p>Ballista 54.1.0 is built on DataFusion 54.1.0, picking up the fixes in that 
patch release. No API changes
+were required, so the <a 
href="https://datafusion.apache.org/ballista/upgrading/54.0.0.html";>54.0.0 
upgrade guide</a> still applies for anyone moving from an earlier release.</p>
+<h2 id="cluster-stability-fixes">Cluster stability fixes<a class="headerlink" 
href="#cluster-stability-fixes" title="Permanent link">¶</a></h2>
+<p>Three fixes address problems that only appear in a real multi-node 
deployment:</p>
+<ul>
+<li><strong>Executor client pool enabled by default.</strong> The executor's 
<code>client_ttl</code> defaulted to <code>0</code>, which disabled
+  the gRPC client pool entirely. Without a pool, the shuffle-read path opened 
and discarded a TCP connection
+  per fetch, so a single shuffle-heavy query could exhaust the host's 
ephemeral ports and fail with
+  <code>AddrNotAvailable</code> — as would every query after it, until those 
sockets left <code>TIME_WAIT</code>. A single-executor
+  cluster never showed this, because shuffle reads are served from local 
files. <code>client_ttl</code> now defaults to
+  30 seconds, so the pool is on unless explicitly disabled.</li>
+<li><strong>Executor startup race.</strong> The executor spawned its gRPC 
server and registered with the scheduler
+  immediately afterwards, but the socket was not bound yet when the scheduler 
dialed back. Losing that race
+  produced <code>ECONNREFUSED</code>, a failed registration, and an executor 
that died at startup. The listener is now
+  bound before registration can run.</li>
+<li><strong>Executor shutdown.</strong> The shutdown notification is now sent 
before the notifier is dropped, so a graceful
+  shutdown is actually observed by the components waiting on it.</li>
+</ul>
+<h2 id="task-retry-and-shuffle-memory">Task retry and shuffle memory<a 
class="headerlink" href="#task-retry-and-shuffle-memory" title="Permanent 
link">¶</a></h2>
+<ul>
+<li><strong>Retryable IO errors on a join's build side.</strong> A retryable 
IO error raised on a join's shared build side
+  arrives at the failed-task classifier wrapped in 
<code>DataFusionError::Shared</code>, so a match on the outermost
+  variant never saw the underlying <code>IoError</code> and the task was 
marked non-retryable. Classification now
+  operates on the root error, regardless of wrapping.</li>
+<li><strong>Sort shuffle writer respects memory pool pressure.</strong> The 
sort shuffle writer discarded the result of
+  <code>try_grow</code> on its memory reservation, so a rejected grow produced 
no spill: the batch stayed buffered while
+  the shared <code>MemoryPool</code> had not accounted for it, and other 
consumers saw memory as free that was not. A
+  rejected grow is now treated as a spill trigger alongside the existing 
per-task budget.</li>
+</ul>
+<h2 id="planning-correctness">Planning correctness<a class="headerlink" 
href="#planning-correctness" title="Permanent link">¶</a></h2>
+<ul>
+<li><strong><code>GROUP BY</code>-less aggregates are preserved</strong> when 
the scheduler propagates empty stages, so a query whose
+  input produces no rows still returns the correct single-row aggregate result 
rather than nothing.</li>
+<li><strong><code>EmptyExec</code> partition counts survive stage 
serialization</strong>, fixing plans whose partitioning changed as a
+  stage was sent to an executor.</li>
+<li><strong>Task partitions are mapped through <code>UnionExec</code></strong> 
when restricting file scans, so each task scans the
+  files it was assigned.</li>
+<li><strong><code>SortMergeJoinExec</code> broadcast conversion is now 
disabled by default.</strong> The static-planner optimization
+  that broadcasts the small build side of a sort-merge join, added in 54.0.0, 
is opt-in in this release while
+  outstanding issues are resolved.</li>
+</ul>
+<h2 id="thank-you">Thank You<a class="headerlink" href="#thank-you" 
title="Permanent link">¶</a></h2>
+<p>Thanks to Andy Grove, Alexander Domenti, and goingforstudying-ctrl for the 
commits in this release, and to
+everyone who contributed by filing issues, reviewing PRs, and providing 
feedback. Full details are in the
+<a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md";>changelog</a>.</p>
+
+<!--
+  Comments Section
+  Loaded only after explicit visitor consent to comply with ASF policy.
+-->
+
+<div>
+  <hr>
+  <h3 id="comments">Comments<a class="headerlink" href="#comments" 
title="Permanent link">&para;</a></h3>
+
+  <!-- Local loader script -->
+  <script src="/blog/js/giscus-consent.js" defer></script>
+
+  <!-- Consent UI -->
+  <div id="giscus-consent">
+    <p>
+        We use <a href="https://giscus.app/";>Giscus</a> for comments, powered 
by GitHub Discussions.
+        To respect your privacy, Giscus and comments will load only if you 
click "Show Comments"
+    </p>
+
+    <div class="consent-actions">
+      <button id="giscus-load" type="button">Show Comments</button>
+      <button id="giscus-revoke" type="button" hidden>Hide Comments</button>
+    </div>
+
+    <noscript>JavaScript is required to load comments from Giscus.</noscript>
+  </div>
+
+  <!-- Container where Giscus will render -->
+  <div id="comment-thread"></div>
+</div>      </div>
+      <aside class="toc-container d-none d-md-block col-md-4 col-xl-3 ms-xl-2">
+        <div class="toc"><span class="toctitle">Contents</span><ul>
+<li><a href="#upgrade-to-datafusion-5410">Upgrade to DataFusion 54.1.0</a></li>
+<li><a href="#cluster-stability-fixes">Cluster stability fixes</a></li>
+<li><a href="#task-retry-and-shuffle-memory">Task retry and shuffle 
memory</a></li>
+<li><a href="#planning-correctness">Planning correctness</a></li>
+<li><a href="#thank-you">Thank You</a></li>
+</ul>
+</div>
+      </aside>
+    </div>
+  </div>
+</div>    
+    <!-- footer -->
+    <div class="row g-0">
+      <div class="col-12">
+        <p style="font-style: italic; font-size: 0.8rem; text-align: center;">
+          Copyright 2026, <a href="https://www.apache.org/";>The Apache 
Software Foundation</a>, Licensed under the <a 
href="https://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 
2.0</a>.<br/>
+          Apache&reg; and the Apache logo are trademarks of The Apache 
Software Foundation.
+        </p>
+        <p style="font-size: 0.8rem; text-align: center;">
+          <a href="https://www.apache.org/";>Foundation</a> |
+          <a 
href="https://www.apache.org/events/current-event.html";>Events</a> |
+          <a href="https://www.apache.org/licenses/";>License</a> |
+          <a href="https://www.apache.org/security/";>Security</a> |
+          <a 
href="https://www.apache.org/foundation/sponsorship.html";>Sponsorship</a> |
+          <a href="https://www.apache.org/foundation/thanks.html";>Thanks</a> |
+          <a 
href="https://privacy.apache.org/policies/privacy-policy-public.html";>Privacy</a>
+        </p>
+      </div>
+    </div>
+    <script src="/blog/js/bootstrap.bundle.min.js"></script>  </main>
+  </body>
+</html>
diff --git a/blog/author/pmc.html b/blog/author/pmc.html
index fdd6050..9c4c717 100644
--- a/blog/author/pmc.html
+++ b/blog/author/pmc.html
@@ -22,7 +22,7 @@
 
 
         <article>
-            <header> <h2><a 
href="https://datafusion.apache.org/blog/2026/08/07/datafusion-comet-1.0.0"; 
rel="bookmark" title="Permalink to Apache DataFusion Comet 1.0.0 
Release">Apache DataFusion Comet 1.0.0 Release</a></h2> </header>
+            <header> <h2><a 
href="https://datafusion.apache.org/blog/2026/08/09/datafusion-ballista-54.1.0"; 
rel="bookmark" title="Permalink to Apache DataFusion Ballista 54.1.0 
Released">Apache DataFusion Ballista 54.1.0 Released</a></h2> </header>
             <section><!--
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
@@ -42,12 +42,12 @@ limitations under the License.
 {% endcomment %}
 -->
 
-<p>The Apache DataFusion PMC is pleased to announce version 1.0.0 of the <a 
href="https://datafusion.apache.org/comet/";>Comet</a> subproject.</p>
-<p>Comet is an accelerator for Apache Spark that translates Spark physical 
plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code changes.</p>
-<p>This release covers roughly six weeks of development …</p></section>
+<p>We are pleased to announce version <a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md";>54.1.0</a>
 of <a href="https://datafusion.apache.org/ballista/";>Apache DataFusion 
Ballista</a>. Ballista is a distributed query
+execution engine that enhances <a href="https://datafusion.apache.org";>Apache 
DataFusion</a> by enabling parallel execution of workloads across multiple
+nodes.</p>
+<p>This is a patch release, consisting of 10 commits from 3 contributors. It 
follows <a href="/blog/2026/07/12/datafusion-ballista-54.0.0/">54.0.0 
…</a></p></section>
             <footer>
-                <p>Published: <time datetime="2026-08-07T00:00:00+00:00"> Fri 
07 August 2026 </time></p>
+                <p>Published: <time datetime="2026-08-09T00:00:00+00:00"> Sun 
09 August 2026 </time></p>
                 <address>By
                         <a 
href="https://datafusion.apache.org/blog/author/pmc.html";>pmc</a>
                 </address>
diff --git a/blog/category/blog.html b/blog/category/blog.html
index f86bab5..dd438c3 100644
--- a/blog/category/blog.html
+++ b/blog/category/blog.html
@@ -23,7 +23,7 @@
 
 
         <article>
-            <header> <h2><a 
href="https://datafusion.apache.org/blog/2026/08/07/datafusion-comet-1.0.0"; 
rel="bookmark" title="Permalink to Apache DataFusion Comet 1.0.0 
Release">Apache DataFusion Comet 1.0.0 Release</a></h2> </header>
+            <header> <h2><a 
href="https://datafusion.apache.org/blog/2026/08/09/datafusion-ballista-54.1.0"; 
rel="bookmark" title="Permalink to Apache DataFusion Ballista 54.1.0 
Released">Apache DataFusion Ballista 54.1.0 Released</a></h2> </header>
             <section><!--
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
@@ -43,12 +43,12 @@ limitations under the License.
 {% endcomment %}
 -->
 
-<p>The Apache DataFusion PMC is pleased to announce version 1.0.0 of the <a 
href="https://datafusion.apache.org/comet/";>Comet</a> subproject.</p>
-<p>Comet is an accelerator for Apache Spark that translates Spark physical 
plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code changes.</p>
-<p>This release covers roughly six weeks of development …</p></section>
+<p>We are pleased to announce version <a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md";>54.1.0</a>
 of <a href="https://datafusion.apache.org/ballista/";>Apache DataFusion 
Ballista</a>. Ballista is a distributed query
+execution engine that enhances <a href="https://datafusion.apache.org";>Apache 
DataFusion</a> by enabling parallel execution of workloads across multiple
+nodes.</p>
+<p>This is a patch release, consisting of 10 commits from 3 contributors. It 
follows <a href="/blog/2026/07/12/datafusion-ballista-54.0.0/">54.0.0 
…</a></p></section>
             <footer>
-                <p>Published: <time datetime="2026-08-07T00:00:00+00:00"> Fri 
07 August 2026 </time></p>
+                <p>Published: <time datetime="2026-08-09T00:00:00+00:00"> Sun 
09 August 2026 </time></p>
                 <address>By
                         <a 
href="https://datafusion.apache.org/blog/author/pmc.html";>pmc</a>
                 </address>
diff --git a/blog/feed.xml b/blog/feed.xml
index 9d4ef18..e9bb105 100644
--- a/blog/feed.xml
+++ b/blog/feed.xml
@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="utf-8"?>
-<rss version="2.0" 
xmlns:atom="http://www.w3.org/2005/Atom";><channel><title>Apache DataFusion 
Blog</title><link>https://datafusion.apache.org/blog/</link><description/><atom:link
 href="https://datafusion.apache.org/blog/feed.xml"; 
rel="self"/><lastBuildDate>Fri, 07 Aug 2026 00:00:00 
+0000</lastBuildDate><item><title>Apache DataFusion Comet 1.0.0 
Release</title><link>https://datafusion.apache.org/blog/2026/08/07/datafusion-comet-1.0.0</link><description>&lt;!--
+<rss version="2.0" 
xmlns:atom="http://www.w3.org/2005/Atom";><channel><title>Apache DataFusion 
Blog</title><link>https://datafusion.apache.org/blog/</link><description/><atom:link
 href="https://datafusion.apache.org/blog/feed.xml"; 
rel="self"/><lastBuildDate>Sun, 09 Aug 2026 00:00:00 
+0000</lastBuildDate><item><title>Apache DataFusion Ballista 54.1.0 
Released</title><link>https://datafusion.apache.org/blog/2026/08/09/datafusion-ballista-54.1.0</link><description>&lt;!--
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
@@ -18,10 +18,10 @@ limitations under the License.
 {% endcomment %}
 --&gt;
 
-&lt;p&gt;The Apache DataFusion PMC is pleased to announce version 1.0.0 of the 
&lt;a href="https://datafusion.apache.org/comet/"&gt;Comet&lt;/a&gt; 
subproject.&lt;/p&gt;
-&lt;p&gt;Comet is an accelerator for Apache Spark that translates Spark 
physical plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code 
changes.&lt;/p&gt;
-&lt;p&gt;This release covers roughly six weeks of development 
…&lt;/p&gt;</description><dc:creator 
xmlns:dc="http://purl.org/dc/elements/1.1/";>pmc</dc:creator><pubDate>Fri, 07 
Aug 2026 00:00:00 
+0000</pubDate><guid>tag:datafusion.apache.org,2026-08-07:/blog/2026/08/07/datafusion-comet-1.0.0</guid><category>blog</category></item><item><title>Optimizing
 for Almost Sorted Data: Sort Pushdown in Apache 
DataFusion</title><link>https://datafusion.apache.org/blog/2026/07/20/sort-pushdown</link>
 [...]
+&lt;p&gt;We are pleased to announce version &lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;54.1.0&lt;/a&gt;
 of &lt;a href="https://datafusion.apache.org/ballista/"&gt;Apache DataFusion 
Ballista&lt;/a&gt;. Ballista is a distributed query
+execution engine that enhances &lt;a 
href="https://datafusion.apache.org"&gt;Apache DataFusion&lt;/a&gt; by enabling 
parallel execution of workloads across multiple
+nodes.&lt;/p&gt;
+&lt;p&gt;This is a patch release, consisting of 10 commits from 3 
contributors. It follows &lt;a 
href="/blog/2026/07/12/datafusion-ballista-54.0.0/"&gt;54.0.0 
…&lt;/a&gt;&lt;/p&gt;</description><dc:creator 
xmlns:dc="http://purl.org/dc/elements/1.1/";>pmc</dc:creator><pubDate>Sun, 09 
Aug 2026 00:00:00 
+0000</pubDate><guid>tag:datafusion.apache.org,2026-08-09:/blog/2026/08/09/datafusion-ballista-54.1.0</guid><category>blog</category></item><item><title>Optimizing
 for Almost Sorted Data: Sor [...]
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
diff --git a/blog/feeds/all-en.atom.xml b/blog/feeds/all-en.atom.xml
index 0a7a7dc..4c3e65e 100644
--- a/blog/feeds/all-en.atom.xml
+++ b/blog/feeds/all-en.atom.xml
@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="utf-8"?>
-<feed xmlns="http://www.w3.org/2005/Atom";><title>Apache DataFusion 
Blog</title><link href="https://datafusion.apache.org/blog/"; 
rel="alternate"/><link 
href="https://datafusion.apache.org/blog/feeds/all-en.atom.xml"; 
rel="self"/><id>https://datafusion.apache.org/blog/</id><updated>2026-08-07T00:00:00+00:00</updated><entry><title>Apache
 DataFusion Comet 1.0.0 Release</title><link 
href="https://datafusion.apache.org/blog/2026/08/07/datafusion-comet-1.0.0"; 
rel="alternate"/><published>2026-08- [...]
+<feed xmlns="http://www.w3.org/2005/Atom";><title>Apache DataFusion 
Blog</title><link href="https://datafusion.apache.org/blog/"; 
rel="alternate"/><link 
href="https://datafusion.apache.org/blog/feeds/all-en.atom.xml"; 
rel="self"/><id>https://datafusion.apache.org/blog/</id><updated>2026-08-09T00:00:00+00:00</updated><entry><title>Apache
 DataFusion Ballista 54.1.0 Released</title><link 
href="https://datafusion.apache.org/blog/2026/08/09/datafusion-ballista-54.1.0"; 
rel="alternate"/><published [...]
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
@@ -18,10 +18,10 @@ limitations under the License.
 {% endcomment %}
 --&gt;
 
-&lt;p&gt;The Apache DataFusion PMC is pleased to announce version 1.0.0 of the 
&lt;a href="https://datafusion.apache.org/comet/"&gt;Comet&lt;/a&gt; 
subproject.&lt;/p&gt;
-&lt;p&gt;Comet is an accelerator for Apache Spark that translates Spark 
physical plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code 
changes.&lt;/p&gt;
-&lt;p&gt;This release covers roughly six weeks of development 
…&lt;/p&gt;</summary><content type="html">&lt;!--
+&lt;p&gt;We are pleased to announce version &lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;54.1.0&lt;/a&gt;
 of &lt;a href="https://datafusion.apache.org/ballista/"&gt;Apache DataFusion 
Ballista&lt;/a&gt;. Ballista is a distributed query
+execution engine that enhances &lt;a 
href="https://datafusion.apache.org"&gt;Apache DataFusion&lt;/a&gt; by enabling 
parallel execution of workloads across multiple
+nodes.&lt;/p&gt;
+&lt;p&gt;This is a patch release, consisting of 10 commits from 3 
contributors. It follows &lt;a 
href="/blog/2026/07/12/datafusion-ballista-54.0.0/"&gt;54.0.0 
…&lt;/a&gt;&lt;/p&gt;</summary><content type="html">&lt;!--
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
@@ -40,149 +40,58 @@ limitations under the License.
 {% endcomment %}
 --&gt;
 
-&lt;p&gt;The Apache DataFusion PMC is pleased to announce version 1.0.0 of the 
&lt;a href="https://datafusion.apache.org/comet/"&gt;Comet&lt;/a&gt; 
subproject.&lt;/p&gt;
-&lt;p&gt;Comet is an accelerator for Apache Spark that translates Spark 
physical plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code 
changes.&lt;/p&gt;
-&lt;p&gt;This release covers roughly six weeks of development since 0.17.0 and 
consists of 244 commits from 23
-contributors. See the &lt;a 
href="https://github.com/apache/datafusion-comet/blob/main/docs/source/changelog/1.0.0.md"&gt;change
 log&lt;/a&gt; for the full list of changes.&lt;/p&gt;
-&lt;h2 id="the-road-to-10"&gt;The Road to 1.0&lt;a class="headerlink" 
href="#the-road-to-10" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;Comet was &lt;a 
href="https://datafusion.apache.org/blog/2024/03/06/comet-donation/"&gt;donated&lt;/a&gt;
 to the Apache DataFusion project in March 2024 and cut its first release, 
0.1.0, five
-months later with support for 13 operators and 106 expressions. Since then, 
the project has shipped 20
-releases and drawn contributions from more than 120 developers, and the 
codebase now recognizes over 400
-Spark expressions. Operator coverage has grown alongside it: 1.0 accelerates 
each of Spark's four join
-operators, window functions, generators (&lt;code&gt;explode&lt;/code&gt;, 
&lt;code&gt;explode_outer&lt;/code&gt;, &lt;code&gt;posexplode&lt;/code&gt;, 
and &lt;code&gt;posexplode_outer&lt;/code&gt;
-over arrays), sampling, in-memory table scans, and a fully native 
shuffle.&lt;/p&gt;
-&lt;p&gt;The 1.0 release marks the point at which Comet begins following &lt;a 
href="https://datafusion.apache.org/comet/about/versioning_policy.html"&gt;semantic
 versioning&lt;/a&gt;. Users upgrading
-within the 1.x line can expect backward-compatible changes only; features 
slated for removal will be
-deprecated in a minor release before being dropped in the next major version. 
This is why the deprecations
-of JDK 11 and Spark 3.4 announced below are scheduled for 1.1 rather than 
landing in 1.0 itself.&lt;/p&gt;
-&lt;h3 id="support-for-spark-40-with-ansi-mode"&gt;Support for Spark 4.0+ with 
ANSI mode&lt;a class="headerlink" href="#support-for-spark-40-with-ansi-mode" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Comet 1.0.0 supports Spark versions 3.4 through 4.1, with 
experimental support for 4.2. Comet fully supports Spark's ANSI mode, which is 
enabled by default starting with Spark 4.0.&lt;/p&gt;
-&lt;h3 id="correctness-testing"&gt;Correctness Testing&lt;a class="headerlink" 
href="#correctness-testing" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;It is important that queries accelerated by Comet produce the same 
results as Spark. Correctness checking has always been a large effort in Comet 
development, but the approach has evolved over time.&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Upstream Spark tests&lt;/strong&gt;: Comet runs 
Spark's own test suite with Comet enabled, providing more than 24,000 unit 
tests effectively for free. These tests run in Comet's CI for all supported 
Spark versions.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Scala tests&lt;/strong&gt;: end-to-end queries that 
run with Comet enabled versus disabled, checking that results match.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Fuzz testing&lt;/strong&gt;: many of the Scala tests 
generate randomized data to catch regressions around edge cases such as nulls, 
NaN, Infinity, and timezone issues.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Comet SQL tests&lt;/strong&gt;: a 
sqllogictest-inspired approach that makes end-to-end tests easier to 
write.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Generative AI audits&lt;/strong&gt;: agentic skills 
sweep every expression, comparing Comet's implementation to Spark's source and 
ensuring tests cover important edge cases.&lt;/li&gt;
-&lt;/ul&gt;
-&lt;h3 id="performance"&gt;Performance&lt;a class="headerlink" 
href="#performance" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;The early Comet releases provided a very modest speedup and the 
published benchmark results were based on running TPC workloads at small scale 
factors on a single node. There are now independent benchmark results published 
by AWS Labs that show significant speedups for &lt;a 
href="https://awslabs.github.io/data-on-eks/docs/benchmarks/spark-datafusion-comet-benchmark"&gt;TPC-DS
 @ 3TB running in EKS&lt;/a&gt;.&lt;/p&gt;
-&lt;h3 id="codegen-dispatch"&gt;Codegen Dispatch&lt;a class="headerlink" 
href="#codegen-dispatch" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Comet 0.17.0 introduced a new approach to filling gaps in expression 
coverage. In earlier releases,
-whenever Comet's planner encountered an expression that lacked a native Rust 
implementation, it fell back
-to executing an entire subtree of the plan in Spark. That required converting 
Arrow columns back to Spark
-rows before the expression ran and back to Arrow after, and the cost was often 
enough to erase the speedup
-Comet had bought elsewhere in the plan.&lt;/p&gt;
-&lt;p&gt;Codegen dispatch narrows that fallback to the expression itself: the 
batch stays in the Comet pipeline
-and Comet invokes Spark's own generated code for just the missing expression, 
leaving the rest of the query
-running natively. Four consequences are worth calling out.&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Coverage.&lt;/strong&gt; Expressions that would 
previously have blocked native execution of a whole subtree are
-  now supported immediately, without a Rust port.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Compatibility.&lt;/strong&gt; For categories where a 
native reimplementation would inevitably diverge from Spark's
-  semantics — regular expressions being the canonical case, given the gap 
between Java's regex engine and
-  any Rust or C++ equivalent — codegen dispatch delivers bit-for-bit Spark 
parity because it &lt;em&gt;is&lt;/em&gt; Spark's
-  implementation.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Expression fusion.&lt;/strong&gt; A dispatched 
expression tree (&lt;em&gt;i.e.&lt;/em&gt;, nested expressions) is compiled 
into a single method, so the Arrow
-  input reads, the expression evaluation, and the Arrow output writes are 
fused together. The compiler
-  is free to optimize across the whole tree, and no intermediate Arrow 
&lt;code&gt;RecordBatch&lt;/code&gt; is materialized
-  between one expression and the next.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Scala and Java UDFs.&lt;/strong&gt; User-defined 
functions are compiled to the same codegen surface as built-in
-  expressions, so they can flow through codegen dispatch without any change 
from the user. Queries that
-  were previously disqualified from acceleration only because they contained a 
UDF can now benefit as
-  long as the surrounding operators are supported. See the &lt;a 
href="https://datafusion.apache.org/comet/user-guide/latest/scala_java_udfs.html"&gt;Scala
 and Java UDF guide&lt;/a&gt; for details.&lt;/li&gt;
-&lt;/ul&gt;
-&lt;p&gt;Comet 1.0 widens the mechanism in three ways.&lt;/p&gt;
-&lt;p&gt;The first is the biggest. An expression that opts into codegen 
dispatch previously reached the dispatcher
-only when Comet reported it as &lt;em&gt;incompatible&lt;/em&gt; for the given 
input; an &lt;em&gt;unsupported&lt;/em&gt; report still sent the
-whole subtree back to Spark. In 1.0 both support levels route through the 
dispatcher, so an input that
-Spark handles and Comet's native code does not now stays inside the Comet 
pipeline.&lt;/p&gt;
-&lt;p&gt;Second, casts join the same path. Cast expressions that Comet 
declines to run natively — including legacy
-configuration variants such as 
&lt;code&gt;spark.sql.legacy.castComplexTypesToString.enabled&lt;/code&gt; — 
are now dispatched
-rather than falling back, and more string, array, and interval expressions 
were opted in as well.&lt;/p&gt;
-&lt;p&gt;Third, the path is now visible. Comet's extended explain output 
reports native versus codegen-dispatch
-coverage for a plan, so you can see which path each expression actually took 
rather than inferring it from
-the absence of a fallback reason.&lt;/p&gt;
-&lt;h2 id="improvements-since-0170"&gt;Improvements since 0.17.0&lt;a 
class="headerlink" href="#improvements-since-0170" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;The rest of this post covers what is new since the 0.17.0 
release.&lt;/p&gt;
-&lt;h3 id="experimental-pyarrow-udf-support"&gt;Experimental PyArrow UDF 
Support&lt;a class="headerlink" href="#experimental-pyarrow-udf-support" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;This release adds experimental support for accelerated PyArrow UDFs, 
allowing PyArrow-based user-defined
-functions to participate in native execution instead of forcing a fallback to 
Spark. When the feature is
-disabled, Comet now hints at the native PyArrow UDF path in its fallback 
reasons so users know the option
-exists. This is an early-stage feature and we welcome feedback from users 
experimenting with it.&lt;/p&gt;
-&lt;h3 id="new-expression-and-aggregate-support"&gt;New Expression and 
Aggregate Support&lt;a class="headerlink" 
href="#new-expression-and-aggregate-support" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;This release expands the set of Spark expressions and aggregates that 
are accelerated by Comet:&lt;/p&gt;
+&lt;p&gt;We are pleased to announce version &lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;54.1.0&lt;/a&gt;
 of &lt;a href="https://datafusion.apache.org/ballista/"&gt;Apache DataFusion 
Ballista&lt;/a&gt;. Ballista is a distributed query
+execution engine that enhances &lt;a 
href="https://datafusion.apache.org"&gt;Apache DataFusion&lt;/a&gt; by enabling 
parallel execution of workloads across multiple
+nodes.&lt;/p&gt;
+&lt;p&gt;This is a patch release, consisting of 10 commits from 3 
contributors. It follows &lt;a 
href="/blog/2026/07/12/datafusion-ballista-54.0.0/"&gt;54.0.0&lt;/a&gt; and 
contains no
+new features and no breaking changes — only an upgrade to DataFusion 54.1.0 
and a set of bug fixes
+backported from &lt;code&gt;main&lt;/code&gt;. Users running 54.0.0 are 
encouraged to upgrade.&lt;/p&gt;
+&lt;h2 id="upgrade-to-datafusion-5410"&gt;Upgrade to DataFusion 54.1.0&lt;a 
class="headerlink" href="#upgrade-to-datafusion-5410" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;p&gt;Ballista 54.1.0 is built on DataFusion 54.1.0, picking up the fixes 
in that patch release. No API changes
+were required, so the &lt;a 
href="https://datafusion.apache.org/ballista/upgrading/54.0.0.html"&gt;54.0.0 
upgrade guide&lt;/a&gt; still applies for anyone moving from an earlier 
release.&lt;/p&gt;
+&lt;h2 id="cluster-stability-fixes"&gt;Cluster stability fixes&lt;a 
class="headerlink" href="#cluster-stability-fixes" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;p&gt;Three fixes address problems that only appear in a real multi-node 
deployment:&lt;/p&gt;
 &lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Aggregates&lt;/strong&gt;: 
&lt;code&gt;approx_percentile&lt;/code&gt; / 
&lt;code&gt;percentile_approx&lt;/code&gt;, exact 
&lt;code&gt;percentile&lt;/code&gt; / &lt;code&gt;median&lt;/code&gt;,
-  &lt;code&gt;approx_count_distinct&lt;/code&gt;, and native 
&lt;code&gt;collect_list&lt;/code&gt; / 
&lt;code&gt;array_agg&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Cast&lt;/strong&gt;: Cast expressions where the native 
implementation is marked as incompatible or unsupported
-  are now routed through codegen dispatch.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Grouping&lt;/strong&gt;: 
&lt;code&gt;grouping()&lt;/code&gt; and 
&lt;code&gt;grouping_id()&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Intervals&lt;/strong&gt;: interval types via 
&lt;code&gt;make_ym_interval&lt;/code&gt; and 
&lt;code&gt;make_dt_interval&lt;/code&gt;, 
&lt;code&gt;CalendarIntervalType&lt;/code&gt;,
-  &lt;code&gt;multiply_dt_interval&lt;/code&gt;, and interval codegen dispatch 
for nested values and native shuffle.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;String&lt;/strong&gt;: 
&lt;code&gt;base64&lt;/code&gt;, &lt;code&gt;split_part&lt;/code&gt; via 
&lt;code&gt;StringSplitSQL&lt;/code&gt;, native 
&lt;code&gt;levenshtein&lt;/code&gt;, and native 
&lt;code&gt;randstr&lt;/code&gt; and
-  &lt;code&gt;uuid&lt;/code&gt; — both bit-for-bit compatible with Spark for a 
given seed.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Array / map&lt;/strong&gt;: 
&lt;code&gt;array_prepend&lt;/code&gt;, the &lt;code&gt;shuffle()&lt;/code&gt; 
array function, &lt;code&gt;size()&lt;/code&gt; for 
&lt;code&gt;MapType&lt;/code&gt;, and &lt;code&gt;ElementAt&lt;/code&gt;
-  over &lt;code&gt;MapType&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Date/time&lt;/strong&gt;: native 
&lt;code&gt;TimestampNTZ&lt;/code&gt; inputs for &lt;code&gt;hour&lt;/code&gt; 
/ &lt;code&gt;minute&lt;/code&gt; / &lt;code&gt;second&lt;/code&gt; and
-  &lt;code&gt;PreciseTimestampConversion&lt;/code&gt; for native time-window 
grouping.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Windows&lt;/strong&gt;: extended native window 
function support and Spark 4 decimal window average.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Executor client pool enabled by 
default.&lt;/strong&gt; The executor's &lt;code&gt;client_ttl&lt;/code&gt; 
defaulted to &lt;code&gt;0&lt;/code&gt;, which disabled
+  the gRPC client pool entirely. Without a pool, the shuffle-read path opened 
and discarded a TCP connection
+  per fetch, so a single shuffle-heavy query could exhaust the host's 
ephemeral ports and fail with
+  &lt;code&gt;AddrNotAvailable&lt;/code&gt; — as would every query after it, 
until those sockets left &lt;code&gt;TIME_WAIT&lt;/code&gt;. A single-executor
+  cluster never showed this, because shuffle reads are served from local 
files. &lt;code&gt;client_ttl&lt;/code&gt; now defaults to
+  30 seconds, so the pool is on unless explicitly disabled.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Executor startup race.&lt;/strong&gt; The executor 
spawned its gRPC server and registered with the scheduler
+  immediately afterwards, but the socket was not bound yet when the scheduler 
dialed back. Losing that race
+  produced &lt;code&gt;ECONNREFUSED&lt;/code&gt;, a failed registration, and 
an executor that died at startup. The listener is now
+  bound before registration can run.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Executor shutdown.&lt;/strong&gt; The shutdown 
notification is now sent before the notifier is dropped, so a graceful
+  shutdown is actually observed by the components waiting on it.&lt;/li&gt;
 &lt;/ul&gt;
-&lt;h3 id="faster-parquet-scans"&gt;Faster Parquet Scans&lt;a 
class="headerlink" href="#faster-parquet-scans" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Parquet reads pick up several improvements as well. Full Parquet 
metadata, including the page index, is now
-cached via DataFusion's 
&lt;code&gt;CachedParquetFileReaderFactory&lt;/code&gt;; identity casts are 
unwrapped in the schema adapter
-so Parquet statistics pruning can engage; filter pushdown configuration has 
been revised; the native scan
-passes a metadata size hint so a single read usually captures the footer; and 
the native Parquet scan seeds
-its reader options from the session config so Parquet settings you already set 
take effect.&lt;/p&gt;
-&lt;h3 id="iceberg-table-format-v3"&gt;Iceberg Table Format V3&lt;a 
class="headerlink" href="#iceberg-table-format-v3" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Comet now supports Iceberg 1.11 and its first Iceberg table format V3 
feature: full table encryption. Other V3 features like deletion vectors and new 
data types (&lt;em&gt;e.g.&lt;/em&gt;, &lt;code&gt;VARIANT&lt;/code&gt;) fall 
back gracefully. The native Iceberg scan supports the 
&lt;code&gt;_pos&lt;/code&gt;, &lt;code&gt;_spec&lt;/code&gt;, 
&lt;code&gt;_file&lt;/code&gt;, and
-&lt;code&gt;_partition&lt;/code&gt; metadata columns, sizes delete files 
correctly to avoid dropped deletes, disambiguates scans that
-share a &lt;code&gt;metadata_location&lt;/code&gt;, and dedupes residuals and 
delete files in the native scan serde. A prior case
-where Iceberg native scan exchange reuse with different pushed filters could 
produce wrong results is also
-fixed.&lt;/p&gt;
-&lt;h3 id="native-expression-performance"&gt;Native Expression 
Performance&lt;a class="headerlink" href="#native-expression-performance" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Many native expression implementations have been optimized to more 
efficiently leverage Arrow kernels or to avoid per-row builders.&lt;/p&gt;
+&lt;h2 id="task-retry-and-shuffle-memory"&gt;Task retry and shuffle 
memory&lt;a class="headerlink" href="#task-retry-and-shuffle-memory" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
 &lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Casts&lt;/strong&gt; between numeric, string, decimal, 
and date types, including a faster float-to-decimal cast, an
-  optimized integer-to-integer cast, shared no-overflow fast paths in 
&lt;code&gt;CheckOverflow&lt;/code&gt; and
-  &lt;code&gt;DecimalRescaleCheckOverflow&lt;/code&gt;, and a 
&lt;code&gt;cast_binary_to_string&lt;/code&gt; that is up to 27x faster on 
binary-format
-  styles.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;JSON, regex, and URL parsing&lt;/strong&gt;: 
&lt;code&gt;get_json_object&lt;/code&gt;, 
&lt;code&gt;regexp_extract&lt;/code&gt;, and 
&lt;code&gt;parse_url&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Date/time and decimal kernels&lt;/strong&gt;: 
&lt;code&gt;date_trunc&lt;/code&gt;, &lt;code&gt;spark_ceil&lt;/code&gt;, and a 
vectorized &lt;code&gt;spark_unscaled_value&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;String and array kernels&lt;/strong&gt;: 
&lt;code&gt;lpad&lt;/code&gt;, &lt;code&gt;unhex&lt;/code&gt;, 
&lt;code&gt;size&lt;/code&gt;, &lt;code&gt;arrays_overlap&lt;/code&gt;, 
&lt;code&gt;escape_string&lt;/code&gt;, and the &lt;code&gt;try_*&lt;/code&gt;
-  arithmetic kernel.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Retryable IO errors on a join's build 
side.&lt;/strong&gt; A retryable IO error raised on a join's shared build side
+  arrives at the failed-task classifier wrapped in 
&lt;code&gt;DataFusionError::Shared&lt;/code&gt;, so a match on the outermost
+  variant never saw the underlying &lt;code&gt;IoError&lt;/code&gt; and the 
task was marked non-retryable. Classification now
+  operates on the root error, regardless of wrapping.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Sort shuffle writer respects memory pool 
pressure.&lt;/strong&gt; The sort shuffle writer discarded the result of
+  &lt;code&gt;try_grow&lt;/code&gt; on its memory reservation, so a rejected 
grow produced no spill: the batch stayed buffered while
+  the shared &lt;code&gt;MemoryPool&lt;/code&gt; had not accounted for it, and 
other consumers saw memory as free that was not. A
+  rejected grow is now treated as a spill trigger alongside the existing 
per-task budget.&lt;/li&gt;
 &lt;/ul&gt;
-&lt;p&gt;To make this kind of work repeatable, the release also adds a &lt;a 
href="https://datafusion.apache.org/comet/contributor-guide/optimizing_expressions.html"&gt;scalar
 expression optimization guide&lt;/a&gt;
-documenting how to benchmark a kernel, keep its output bit-identical to Spark, 
and gate changes on a
-no-regression check.&lt;/p&gt;
-&lt;h2 id="deprecation-notice"&gt;Deprecation Notice&lt;a class="headerlink" 
href="#deprecation-notice" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;With the move to a stable 1.0 release line, Comet begins deprecating 
older platforms under its
-&lt;a 
href="https://datafusion.apache.org/comet/about/versioning_policy.html"&gt;versioning
 policy&lt;/a&gt;:&lt;/p&gt;
+&lt;h2 id="planning-correctness"&gt;Planning correctness&lt;a 
class="headerlink" href="#planning-correctness" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
 &lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;JDK 11&lt;/strong&gt; is deprecated and scheduled for 
removal in Comet 1.1.0.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Apache Spark 3.4&lt;/strong&gt; is deprecated and 
scheduled for removal in Comet 1.1.0.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;&lt;code&gt;GROUP BY&lt;/code&gt;-less aggregates are 
preserved&lt;/strong&gt; when the scheduler propagates empty stages, so a query 
whose
+  input produces no rows still returns the correct single-row aggregate result 
rather than nothing.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;&lt;code&gt;EmptyExec&lt;/code&gt; partition counts 
survive stage serialization&lt;/strong&gt;, fixing plans whose partitioning 
changed as a
+  stage was sent to an executor.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Task partitions are mapped through 
&lt;code&gt;UnionExec&lt;/code&gt;&lt;/strong&gt; when restricting file scans, 
so each task scans the
+  files it was assigned.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;&lt;code&gt;SortMergeJoinExec&lt;/code&gt; broadcast 
conversion is now disabled by default.&lt;/strong&gt; The static-planner 
optimization
+  that broadcasts the small build side of a sort-merge join, added in 54.0.0, 
is opt-in in this release while
+  outstanding issues are resolved.&lt;/li&gt;
 &lt;/ul&gt;
-&lt;p&gt;Comet aligns its Spark support window with upstream Apache Spark 
maintenance. Spark 3.4 is no longer
-maintained upstream, so under the versioning policy it is deprecated in the 
first Comet minor release
-after that point and removed in the following one. Comet 1.0.0 still builds 
and publishes Spark 3.4
-binaries.&lt;/p&gt;
-&lt;p&gt;Users on these platforms should plan to move to JDK 17+ and Spark 3.5 
or later before upgrading to 1.1.0.&lt;/p&gt;
-&lt;h2 id="compatibility"&gt;Compatibility&lt;a class="headerlink" 
href="#compatibility" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;Supported platforms include:&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Spark 3.4.3&lt;/strong&gt; with Java 11/17 and Scala 
2.12/2.13 (deprecated, removal in 1.1.0)&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 3.5.9&lt;/strong&gt; with Java 11/17 and Scala 
2.12/2.13&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 4.0.4&lt;/strong&gt; with Java 17 and Scala 
2.13&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 4.1.3&lt;/strong&gt; with Java 17/21 and Scala 
2.13&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 4.2&lt;/strong&gt; with Java 17 and Scala 2.13 
(experimental, for early evaluation only)&lt;/li&gt;
-&lt;/ul&gt;
-&lt;p&gt;See the &lt;a 
href="https://datafusion.apache.org/comet/user-guide/latest/compatibility/spark-versions.html"&gt;Spark
 Version Compatibility&lt;/a&gt; page for known limitations specific to each 
version.&lt;/p&gt;
-&lt;p&gt;This release upgrades to &lt;strong&gt;DataFusion 54.1&lt;/strong&gt; 
and &lt;strong&gt;Arrow 58.4&lt;/strong&gt;.&lt;/p&gt;
-&lt;h2 id="get-started-with-comet-100"&gt;Get Started with Comet 1.0.0&lt;a 
class="headerlink" href="#get-started-with-comet-100" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;Ready to try it out? Follow the &lt;a 
href="https://datafusion.apache.org/comet/user-guide/1.0/installation.html"&gt;Comet
 1.0.0 Installation Guide&lt;/a&gt;
-to get up and running, then point Comet at your existing Spark workloads and 
see the speedup for yourself.&lt;/p&gt;</content><category 
term="blog"/></entry><entry><title>Optimizing for Almost Sorted Data: Sort 
Pushdown in Apache DataFusion</title><link 
href="https://datafusion.apache.org/blog/2026/07/20/sort-pushdown"; 
rel="alternate"/><published>2026-07-20T00:00:00+00:00</published><updated>2026-07-20T00:00:00+00:00</updated><author><name>Qi
 Zhu and Andrew Lamb</name></author><id>tag:da [...]
+&lt;h2 id="thank-you"&gt;Thank You&lt;a class="headerlink" href="#thank-you" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;p&gt;Thanks to Andy Grove, Alexander Domenti, and goingforstudying-ctrl 
for the commits in this release, and to
+everyone who contributed by filing issues, reviewing PRs, and providing 
feedback. Full details are in the
+&lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;changelog&lt;/a&gt;.&lt;/p&gt;</content><category
 term="blog"/></entry><entry><title>Optimizing for Almost Sorted Data: Sort 
Pushdown in Apache DataFusion</title><link 
href="https://datafusion.apache.org/blog/2026/07/20/sort-pushdown"; 
rel="alternate"/><published>2026-07-20T00:00:00+00:00</published><updated>2026-07-20T00:00:00+00:00</updated><author><name>Qi
 Zhu and Andrew Lamb</name [...]
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
diff --git a/blog/feeds/blog.atom.xml b/blog/feeds/blog.atom.xml
index 87629b6..e6e7c24 100644
--- a/blog/feeds/blog.atom.xml
+++ b/blog/feeds/blog.atom.xml
@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="utf-8"?>
-<feed xmlns="http://www.w3.org/2005/Atom";><title>Apache DataFusion Blog - 
blog</title><link href="https://datafusion.apache.org/blog/"; 
rel="alternate"/><link 
href="https://datafusion.apache.org/blog/feeds/blog.atom.xml"; 
rel="self"/><id>https://datafusion.apache.org/blog/</id><updated>2026-08-07T00:00:00+00:00</updated><entry><title>Apache
 DataFusion Comet 1.0.0 Release</title><link 
href="https://datafusion.apache.org/blog/2026/08/07/datafusion-comet-1.0.0"; 
rel="alternate"/><published>202 [...]
+<feed xmlns="http://www.w3.org/2005/Atom";><title>Apache DataFusion Blog - 
blog</title><link href="https://datafusion.apache.org/blog/"; 
rel="alternate"/><link 
href="https://datafusion.apache.org/blog/feeds/blog.atom.xml"; 
rel="self"/><id>https://datafusion.apache.org/blog/</id><updated>2026-08-09T00:00:00+00:00</updated><entry><title>Apache
 DataFusion Ballista 54.1.0 Released</title><link 
href="https://datafusion.apache.org/blog/2026/08/09/datafusion-ballista-54.1.0"; 
rel="alternate"/><publ [...]
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
@@ -18,10 +18,10 @@ limitations under the License.
 {% endcomment %}
 --&gt;
 
-&lt;p&gt;The Apache DataFusion PMC is pleased to announce version 1.0.0 of the 
&lt;a href="https://datafusion.apache.org/comet/"&gt;Comet&lt;/a&gt; 
subproject.&lt;/p&gt;
-&lt;p&gt;Comet is an accelerator for Apache Spark that translates Spark 
physical plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code 
changes.&lt;/p&gt;
-&lt;p&gt;This release covers roughly six weeks of development 
…&lt;/p&gt;</summary><content type="html">&lt;!--
+&lt;p&gt;We are pleased to announce version &lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;54.1.0&lt;/a&gt;
 of &lt;a href="https://datafusion.apache.org/ballista/"&gt;Apache DataFusion 
Ballista&lt;/a&gt;. Ballista is a distributed query
+execution engine that enhances &lt;a 
href="https://datafusion.apache.org"&gt;Apache DataFusion&lt;/a&gt; by enabling 
parallel execution of workloads across multiple
+nodes.&lt;/p&gt;
+&lt;p&gt;This is a patch release, consisting of 10 commits from 3 
contributors. It follows &lt;a 
href="/blog/2026/07/12/datafusion-ballista-54.0.0/"&gt;54.0.0 
…&lt;/a&gt;&lt;/p&gt;</summary><content type="html">&lt;!--
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
@@ -40,149 +40,58 @@ limitations under the License.
 {% endcomment %}
 --&gt;
 
-&lt;p&gt;The Apache DataFusion PMC is pleased to announce version 1.0.0 of the 
&lt;a href="https://datafusion.apache.org/comet/"&gt;Comet&lt;/a&gt; 
subproject.&lt;/p&gt;
-&lt;p&gt;Comet is an accelerator for Apache Spark that translates Spark 
physical plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code 
changes.&lt;/p&gt;
-&lt;p&gt;This release covers roughly six weeks of development since 0.17.0 and 
consists of 244 commits from 23
-contributors. See the &lt;a 
href="https://github.com/apache/datafusion-comet/blob/main/docs/source/changelog/1.0.0.md"&gt;change
 log&lt;/a&gt; for the full list of changes.&lt;/p&gt;
-&lt;h2 id="the-road-to-10"&gt;The Road to 1.0&lt;a class="headerlink" 
href="#the-road-to-10" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;Comet was &lt;a 
href="https://datafusion.apache.org/blog/2024/03/06/comet-donation/"&gt;donated&lt;/a&gt;
 to the Apache DataFusion project in March 2024 and cut its first release, 
0.1.0, five
-months later with support for 13 operators and 106 expressions. Since then, 
the project has shipped 20
-releases and drawn contributions from more than 120 developers, and the 
codebase now recognizes over 400
-Spark expressions. Operator coverage has grown alongside it: 1.0 accelerates 
each of Spark's four join
-operators, window functions, generators (&lt;code&gt;explode&lt;/code&gt;, 
&lt;code&gt;explode_outer&lt;/code&gt;, &lt;code&gt;posexplode&lt;/code&gt;, 
and &lt;code&gt;posexplode_outer&lt;/code&gt;
-over arrays), sampling, in-memory table scans, and a fully native 
shuffle.&lt;/p&gt;
-&lt;p&gt;The 1.0 release marks the point at which Comet begins following &lt;a 
href="https://datafusion.apache.org/comet/about/versioning_policy.html"&gt;semantic
 versioning&lt;/a&gt;. Users upgrading
-within the 1.x line can expect backward-compatible changes only; features 
slated for removal will be
-deprecated in a minor release before being dropped in the next major version. 
This is why the deprecations
-of JDK 11 and Spark 3.4 announced below are scheduled for 1.1 rather than 
landing in 1.0 itself.&lt;/p&gt;
-&lt;h3 id="support-for-spark-40-with-ansi-mode"&gt;Support for Spark 4.0+ with 
ANSI mode&lt;a class="headerlink" href="#support-for-spark-40-with-ansi-mode" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Comet 1.0.0 supports Spark versions 3.4 through 4.1, with 
experimental support for 4.2. Comet fully supports Spark's ANSI mode, which is 
enabled by default starting with Spark 4.0.&lt;/p&gt;
-&lt;h3 id="correctness-testing"&gt;Correctness Testing&lt;a class="headerlink" 
href="#correctness-testing" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;It is important that queries accelerated by Comet produce the same 
results as Spark. Correctness checking has always been a large effort in Comet 
development, but the approach has evolved over time.&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Upstream Spark tests&lt;/strong&gt;: Comet runs 
Spark's own test suite with Comet enabled, providing more than 24,000 unit 
tests effectively for free. These tests run in Comet's CI for all supported 
Spark versions.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Scala tests&lt;/strong&gt;: end-to-end queries that 
run with Comet enabled versus disabled, checking that results match.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Fuzz testing&lt;/strong&gt;: many of the Scala tests 
generate randomized data to catch regressions around edge cases such as nulls, 
NaN, Infinity, and timezone issues.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Comet SQL tests&lt;/strong&gt;: a 
sqllogictest-inspired approach that makes end-to-end tests easier to 
write.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Generative AI audits&lt;/strong&gt;: agentic skills 
sweep every expression, comparing Comet's implementation to Spark's source and 
ensuring tests cover important edge cases.&lt;/li&gt;
-&lt;/ul&gt;
-&lt;h3 id="performance"&gt;Performance&lt;a class="headerlink" 
href="#performance" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;The early Comet releases provided a very modest speedup and the 
published benchmark results were based on running TPC workloads at small scale 
factors on a single node. There are now independent benchmark results published 
by AWS Labs that show significant speedups for &lt;a 
href="https://awslabs.github.io/data-on-eks/docs/benchmarks/spark-datafusion-comet-benchmark"&gt;TPC-DS
 @ 3TB running in EKS&lt;/a&gt;.&lt;/p&gt;
-&lt;h3 id="codegen-dispatch"&gt;Codegen Dispatch&lt;a class="headerlink" 
href="#codegen-dispatch" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Comet 0.17.0 introduced a new approach to filling gaps in expression 
coverage. In earlier releases,
-whenever Comet's planner encountered an expression that lacked a native Rust 
implementation, it fell back
-to executing an entire subtree of the plan in Spark. That required converting 
Arrow columns back to Spark
-rows before the expression ran and back to Arrow after, and the cost was often 
enough to erase the speedup
-Comet had bought elsewhere in the plan.&lt;/p&gt;
-&lt;p&gt;Codegen dispatch narrows that fallback to the expression itself: the 
batch stays in the Comet pipeline
-and Comet invokes Spark's own generated code for just the missing expression, 
leaving the rest of the query
-running natively. Four consequences are worth calling out.&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Coverage.&lt;/strong&gt; Expressions that would 
previously have blocked native execution of a whole subtree are
-  now supported immediately, without a Rust port.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Compatibility.&lt;/strong&gt; For categories where a 
native reimplementation would inevitably diverge from Spark's
-  semantics — regular expressions being the canonical case, given the gap 
between Java's regex engine and
-  any Rust or C++ equivalent — codegen dispatch delivers bit-for-bit Spark 
parity because it &lt;em&gt;is&lt;/em&gt; Spark's
-  implementation.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Expression fusion.&lt;/strong&gt; A dispatched 
expression tree (&lt;em&gt;i.e.&lt;/em&gt;, nested expressions) is compiled 
into a single method, so the Arrow
-  input reads, the expression evaluation, and the Arrow output writes are 
fused together. The compiler
-  is free to optimize across the whole tree, and no intermediate Arrow 
&lt;code&gt;RecordBatch&lt;/code&gt; is materialized
-  between one expression and the next.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Scala and Java UDFs.&lt;/strong&gt; User-defined 
functions are compiled to the same codegen surface as built-in
-  expressions, so they can flow through codegen dispatch without any change 
from the user. Queries that
-  were previously disqualified from acceleration only because they contained a 
UDF can now benefit as
-  long as the surrounding operators are supported. See the &lt;a 
href="https://datafusion.apache.org/comet/user-guide/latest/scala_java_udfs.html"&gt;Scala
 and Java UDF guide&lt;/a&gt; for details.&lt;/li&gt;
-&lt;/ul&gt;
-&lt;p&gt;Comet 1.0 widens the mechanism in three ways.&lt;/p&gt;
-&lt;p&gt;The first is the biggest. An expression that opts into codegen 
dispatch previously reached the dispatcher
-only when Comet reported it as &lt;em&gt;incompatible&lt;/em&gt; for the given 
input; an &lt;em&gt;unsupported&lt;/em&gt; report still sent the
-whole subtree back to Spark. In 1.0 both support levels route through the 
dispatcher, so an input that
-Spark handles and Comet's native code does not now stays inside the Comet 
pipeline.&lt;/p&gt;
-&lt;p&gt;Second, casts join the same path. Cast expressions that Comet 
declines to run natively — including legacy
-configuration variants such as 
&lt;code&gt;spark.sql.legacy.castComplexTypesToString.enabled&lt;/code&gt; — 
are now dispatched
-rather than falling back, and more string, array, and interval expressions 
were opted in as well.&lt;/p&gt;
-&lt;p&gt;Third, the path is now visible. Comet's extended explain output 
reports native versus codegen-dispatch
-coverage for a plan, so you can see which path each expression actually took 
rather than inferring it from
-the absence of a fallback reason.&lt;/p&gt;
-&lt;h2 id="improvements-since-0170"&gt;Improvements since 0.17.0&lt;a 
class="headerlink" href="#improvements-since-0170" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;The rest of this post covers what is new since the 0.17.0 
release.&lt;/p&gt;
-&lt;h3 id="experimental-pyarrow-udf-support"&gt;Experimental PyArrow UDF 
Support&lt;a class="headerlink" href="#experimental-pyarrow-udf-support" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;This release adds experimental support for accelerated PyArrow UDFs, 
allowing PyArrow-based user-defined
-functions to participate in native execution instead of forcing a fallback to 
Spark. When the feature is
-disabled, Comet now hints at the native PyArrow UDF path in its fallback 
reasons so users know the option
-exists. This is an early-stage feature and we welcome feedback from users 
experimenting with it.&lt;/p&gt;
-&lt;h3 id="new-expression-and-aggregate-support"&gt;New Expression and 
Aggregate Support&lt;a class="headerlink" 
href="#new-expression-and-aggregate-support" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;This release expands the set of Spark expressions and aggregates that 
are accelerated by Comet:&lt;/p&gt;
+&lt;p&gt;We are pleased to announce version &lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;54.1.0&lt;/a&gt;
 of &lt;a href="https://datafusion.apache.org/ballista/"&gt;Apache DataFusion 
Ballista&lt;/a&gt;. Ballista is a distributed query
+execution engine that enhances &lt;a 
href="https://datafusion.apache.org"&gt;Apache DataFusion&lt;/a&gt; by enabling 
parallel execution of workloads across multiple
+nodes.&lt;/p&gt;
+&lt;p&gt;This is a patch release, consisting of 10 commits from 3 
contributors. It follows &lt;a 
href="/blog/2026/07/12/datafusion-ballista-54.0.0/"&gt;54.0.0&lt;/a&gt; and 
contains no
+new features and no breaking changes — only an upgrade to DataFusion 54.1.0 
and a set of bug fixes
+backported from &lt;code&gt;main&lt;/code&gt;. Users running 54.0.0 are 
encouraged to upgrade.&lt;/p&gt;
+&lt;h2 id="upgrade-to-datafusion-5410"&gt;Upgrade to DataFusion 54.1.0&lt;a 
class="headerlink" href="#upgrade-to-datafusion-5410" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;p&gt;Ballista 54.1.0 is built on DataFusion 54.1.0, picking up the fixes 
in that patch release. No API changes
+were required, so the &lt;a 
href="https://datafusion.apache.org/ballista/upgrading/54.0.0.html"&gt;54.0.0 
upgrade guide&lt;/a&gt; still applies for anyone moving from an earlier 
release.&lt;/p&gt;
+&lt;h2 id="cluster-stability-fixes"&gt;Cluster stability fixes&lt;a 
class="headerlink" href="#cluster-stability-fixes" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;p&gt;Three fixes address problems that only appear in a real multi-node 
deployment:&lt;/p&gt;
 &lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Aggregates&lt;/strong&gt;: 
&lt;code&gt;approx_percentile&lt;/code&gt; / 
&lt;code&gt;percentile_approx&lt;/code&gt;, exact 
&lt;code&gt;percentile&lt;/code&gt; / &lt;code&gt;median&lt;/code&gt;,
-  &lt;code&gt;approx_count_distinct&lt;/code&gt;, and native 
&lt;code&gt;collect_list&lt;/code&gt; / 
&lt;code&gt;array_agg&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Cast&lt;/strong&gt;: Cast expressions where the native 
implementation is marked as incompatible or unsupported
-  are now routed through codegen dispatch.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Grouping&lt;/strong&gt;: 
&lt;code&gt;grouping()&lt;/code&gt; and 
&lt;code&gt;grouping_id()&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Intervals&lt;/strong&gt;: interval types via 
&lt;code&gt;make_ym_interval&lt;/code&gt; and 
&lt;code&gt;make_dt_interval&lt;/code&gt;, 
&lt;code&gt;CalendarIntervalType&lt;/code&gt;,
-  &lt;code&gt;multiply_dt_interval&lt;/code&gt;, and interval codegen dispatch 
for nested values and native shuffle.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;String&lt;/strong&gt;: 
&lt;code&gt;base64&lt;/code&gt;, &lt;code&gt;split_part&lt;/code&gt; via 
&lt;code&gt;StringSplitSQL&lt;/code&gt;, native 
&lt;code&gt;levenshtein&lt;/code&gt;, and native 
&lt;code&gt;randstr&lt;/code&gt; and
-  &lt;code&gt;uuid&lt;/code&gt; — both bit-for-bit compatible with Spark for a 
given seed.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Array / map&lt;/strong&gt;: 
&lt;code&gt;array_prepend&lt;/code&gt;, the &lt;code&gt;shuffle()&lt;/code&gt; 
array function, &lt;code&gt;size()&lt;/code&gt; for 
&lt;code&gt;MapType&lt;/code&gt;, and &lt;code&gt;ElementAt&lt;/code&gt;
-  over &lt;code&gt;MapType&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Date/time&lt;/strong&gt;: native 
&lt;code&gt;TimestampNTZ&lt;/code&gt; inputs for &lt;code&gt;hour&lt;/code&gt; 
/ &lt;code&gt;minute&lt;/code&gt; / &lt;code&gt;second&lt;/code&gt; and
-  &lt;code&gt;PreciseTimestampConversion&lt;/code&gt; for native time-window 
grouping.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Windows&lt;/strong&gt;: extended native window 
function support and Spark 4 decimal window average.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Executor client pool enabled by 
default.&lt;/strong&gt; The executor's &lt;code&gt;client_ttl&lt;/code&gt; 
defaulted to &lt;code&gt;0&lt;/code&gt;, which disabled
+  the gRPC client pool entirely. Without a pool, the shuffle-read path opened 
and discarded a TCP connection
+  per fetch, so a single shuffle-heavy query could exhaust the host's 
ephemeral ports and fail with
+  &lt;code&gt;AddrNotAvailable&lt;/code&gt; — as would every query after it, 
until those sockets left &lt;code&gt;TIME_WAIT&lt;/code&gt;. A single-executor
+  cluster never showed this, because shuffle reads are served from local 
files. &lt;code&gt;client_ttl&lt;/code&gt; now defaults to
+  30 seconds, so the pool is on unless explicitly disabled.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Executor startup race.&lt;/strong&gt; The executor 
spawned its gRPC server and registered with the scheduler
+  immediately afterwards, but the socket was not bound yet when the scheduler 
dialed back. Losing that race
+  produced &lt;code&gt;ECONNREFUSED&lt;/code&gt;, a failed registration, and 
an executor that died at startup. The listener is now
+  bound before registration can run.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Executor shutdown.&lt;/strong&gt; The shutdown 
notification is now sent before the notifier is dropped, so a graceful
+  shutdown is actually observed by the components waiting on it.&lt;/li&gt;
 &lt;/ul&gt;
-&lt;h3 id="faster-parquet-scans"&gt;Faster Parquet Scans&lt;a 
class="headerlink" href="#faster-parquet-scans" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Parquet reads pick up several improvements as well. Full Parquet 
metadata, including the page index, is now
-cached via DataFusion's 
&lt;code&gt;CachedParquetFileReaderFactory&lt;/code&gt;; identity casts are 
unwrapped in the schema adapter
-so Parquet statistics pruning can engage; filter pushdown configuration has 
been revised; the native scan
-passes a metadata size hint so a single read usually captures the footer; and 
the native Parquet scan seeds
-its reader options from the session config so Parquet settings you already set 
take effect.&lt;/p&gt;
-&lt;h3 id="iceberg-table-format-v3"&gt;Iceberg Table Format V3&lt;a 
class="headerlink" href="#iceberg-table-format-v3" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Comet now supports Iceberg 1.11 and its first Iceberg table format V3 
feature: full table encryption. Other V3 features like deletion vectors and new 
data types (&lt;em&gt;e.g.&lt;/em&gt;, &lt;code&gt;VARIANT&lt;/code&gt;) fall 
back gracefully. The native Iceberg scan supports the 
&lt;code&gt;_pos&lt;/code&gt;, &lt;code&gt;_spec&lt;/code&gt;, 
&lt;code&gt;_file&lt;/code&gt;, and
-&lt;code&gt;_partition&lt;/code&gt; metadata columns, sizes delete files 
correctly to avoid dropped deletes, disambiguates scans that
-share a &lt;code&gt;metadata_location&lt;/code&gt;, and dedupes residuals and 
delete files in the native scan serde. A prior case
-where Iceberg native scan exchange reuse with different pushed filters could 
produce wrong results is also
-fixed.&lt;/p&gt;
-&lt;h3 id="native-expression-performance"&gt;Native Expression 
Performance&lt;a class="headerlink" href="#native-expression-performance" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Many native expression implementations have been optimized to more 
efficiently leverage Arrow kernels or to avoid per-row builders.&lt;/p&gt;
+&lt;h2 id="task-retry-and-shuffle-memory"&gt;Task retry and shuffle 
memory&lt;a class="headerlink" href="#task-retry-and-shuffle-memory" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
 &lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Casts&lt;/strong&gt; between numeric, string, decimal, 
and date types, including a faster float-to-decimal cast, an
-  optimized integer-to-integer cast, shared no-overflow fast paths in 
&lt;code&gt;CheckOverflow&lt;/code&gt; and
-  &lt;code&gt;DecimalRescaleCheckOverflow&lt;/code&gt;, and a 
&lt;code&gt;cast_binary_to_string&lt;/code&gt; that is up to 27x faster on 
binary-format
-  styles.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;JSON, regex, and URL parsing&lt;/strong&gt;: 
&lt;code&gt;get_json_object&lt;/code&gt;, 
&lt;code&gt;regexp_extract&lt;/code&gt;, and 
&lt;code&gt;parse_url&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Date/time and decimal kernels&lt;/strong&gt;: 
&lt;code&gt;date_trunc&lt;/code&gt;, &lt;code&gt;spark_ceil&lt;/code&gt;, and a 
vectorized &lt;code&gt;spark_unscaled_value&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;String and array kernels&lt;/strong&gt;: 
&lt;code&gt;lpad&lt;/code&gt;, &lt;code&gt;unhex&lt;/code&gt;, 
&lt;code&gt;size&lt;/code&gt;, &lt;code&gt;arrays_overlap&lt;/code&gt;, 
&lt;code&gt;escape_string&lt;/code&gt;, and the &lt;code&gt;try_*&lt;/code&gt;
-  arithmetic kernel.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Retryable IO errors on a join's build 
side.&lt;/strong&gt; A retryable IO error raised on a join's shared build side
+  arrives at the failed-task classifier wrapped in 
&lt;code&gt;DataFusionError::Shared&lt;/code&gt;, so a match on the outermost
+  variant never saw the underlying &lt;code&gt;IoError&lt;/code&gt; and the 
task was marked non-retryable. Classification now
+  operates on the root error, regardless of wrapping.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Sort shuffle writer respects memory pool 
pressure.&lt;/strong&gt; The sort shuffle writer discarded the result of
+  &lt;code&gt;try_grow&lt;/code&gt; on its memory reservation, so a rejected 
grow produced no spill: the batch stayed buffered while
+  the shared &lt;code&gt;MemoryPool&lt;/code&gt; had not accounted for it, and 
other consumers saw memory as free that was not. A
+  rejected grow is now treated as a spill trigger alongside the existing 
per-task budget.&lt;/li&gt;
 &lt;/ul&gt;
-&lt;p&gt;To make this kind of work repeatable, the release also adds a &lt;a 
href="https://datafusion.apache.org/comet/contributor-guide/optimizing_expressions.html"&gt;scalar
 expression optimization guide&lt;/a&gt;
-documenting how to benchmark a kernel, keep its output bit-identical to Spark, 
and gate changes on a
-no-regression check.&lt;/p&gt;
-&lt;h2 id="deprecation-notice"&gt;Deprecation Notice&lt;a class="headerlink" 
href="#deprecation-notice" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;With the move to a stable 1.0 release line, Comet begins deprecating 
older platforms under its
-&lt;a 
href="https://datafusion.apache.org/comet/about/versioning_policy.html"&gt;versioning
 policy&lt;/a&gt;:&lt;/p&gt;
+&lt;h2 id="planning-correctness"&gt;Planning correctness&lt;a 
class="headerlink" href="#planning-correctness" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
 &lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;JDK 11&lt;/strong&gt; is deprecated and scheduled for 
removal in Comet 1.1.0.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Apache Spark 3.4&lt;/strong&gt; is deprecated and 
scheduled for removal in Comet 1.1.0.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;&lt;code&gt;GROUP BY&lt;/code&gt;-less aggregates are 
preserved&lt;/strong&gt; when the scheduler propagates empty stages, so a query 
whose
+  input produces no rows still returns the correct single-row aggregate result 
rather than nothing.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;&lt;code&gt;EmptyExec&lt;/code&gt; partition counts 
survive stage serialization&lt;/strong&gt;, fixing plans whose partitioning 
changed as a
+  stage was sent to an executor.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Task partitions are mapped through 
&lt;code&gt;UnionExec&lt;/code&gt;&lt;/strong&gt; when restricting file scans, 
so each task scans the
+  files it was assigned.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;&lt;code&gt;SortMergeJoinExec&lt;/code&gt; broadcast 
conversion is now disabled by default.&lt;/strong&gt; The static-planner 
optimization
+  that broadcasts the small build side of a sort-merge join, added in 54.0.0, 
is opt-in in this release while
+  outstanding issues are resolved.&lt;/li&gt;
 &lt;/ul&gt;
-&lt;p&gt;Comet aligns its Spark support window with upstream Apache Spark 
maintenance. Spark 3.4 is no longer
-maintained upstream, so under the versioning policy it is deprecated in the 
first Comet minor release
-after that point and removed in the following one. Comet 1.0.0 still builds 
and publishes Spark 3.4
-binaries.&lt;/p&gt;
-&lt;p&gt;Users on these platforms should plan to move to JDK 17+ and Spark 3.5 
or later before upgrading to 1.1.0.&lt;/p&gt;
-&lt;h2 id="compatibility"&gt;Compatibility&lt;a class="headerlink" 
href="#compatibility" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;Supported platforms include:&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Spark 3.4.3&lt;/strong&gt; with Java 11/17 and Scala 
2.12/2.13 (deprecated, removal in 1.1.0)&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 3.5.9&lt;/strong&gt; with Java 11/17 and Scala 
2.12/2.13&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 4.0.4&lt;/strong&gt; with Java 17 and Scala 
2.13&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 4.1.3&lt;/strong&gt; with Java 17/21 and Scala 
2.13&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 4.2&lt;/strong&gt; with Java 17 and Scala 2.13 
(experimental, for early evaluation only)&lt;/li&gt;
-&lt;/ul&gt;
-&lt;p&gt;See the &lt;a 
href="https://datafusion.apache.org/comet/user-guide/latest/compatibility/spark-versions.html"&gt;Spark
 Version Compatibility&lt;/a&gt; page for known limitations specific to each 
version.&lt;/p&gt;
-&lt;p&gt;This release upgrades to &lt;strong&gt;DataFusion 54.1&lt;/strong&gt; 
and &lt;strong&gt;Arrow 58.4&lt;/strong&gt;.&lt;/p&gt;
-&lt;h2 id="get-started-with-comet-100"&gt;Get Started with Comet 1.0.0&lt;a 
class="headerlink" href="#get-started-with-comet-100" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;Ready to try it out? Follow the &lt;a 
href="https://datafusion.apache.org/comet/user-guide/1.0/installation.html"&gt;Comet
 1.0.0 Installation Guide&lt;/a&gt;
-to get up and running, then point Comet at your existing Spark workloads and 
see the speedup for yourself.&lt;/p&gt;</content><category 
term="blog"/></entry><entry><title>Optimizing for Almost Sorted Data: Sort 
Pushdown in Apache DataFusion</title><link 
href="https://datafusion.apache.org/blog/2026/07/20/sort-pushdown"; 
rel="alternate"/><published>2026-07-20T00:00:00+00:00</published><updated>2026-07-20T00:00:00+00:00</updated><author><name>Qi
 Zhu and Andrew Lamb</name></author><id>tag:da [...]
+&lt;h2 id="thank-you"&gt;Thank You&lt;a class="headerlink" href="#thank-you" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;p&gt;Thanks to Andy Grove, Alexander Domenti, and goingforstudying-ctrl 
for the commits in this release, and to
+everyone who contributed by filing issues, reviewing PRs, and providing 
feedback. Full details are in the
+&lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;changelog&lt;/a&gt;.&lt;/p&gt;</content><category
 term="blog"/></entry><entry><title>Optimizing for Almost Sorted Data: Sort 
Pushdown in Apache DataFusion</title><link 
href="https://datafusion.apache.org/blog/2026/07/20/sort-pushdown"; 
rel="alternate"/><published>2026-07-20T00:00:00+00:00</published><updated>2026-07-20T00:00:00+00:00</updated><author><name>Qi
 Zhu and Andrew Lamb</name [...]
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
diff --git a/blog/feeds/pmc.atom.xml b/blog/feeds/pmc.atom.xml
index 8509a6b..209358e 100644
--- a/blog/feeds/pmc.atom.xml
+++ b/blog/feeds/pmc.atom.xml
@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="utf-8"?>
-<feed xmlns="http://www.w3.org/2005/Atom";><title>Apache DataFusion Blog - 
pmc</title><link href="https://datafusion.apache.org/blog/"; 
rel="alternate"/><link 
href="https://datafusion.apache.org/blog/feeds/pmc.atom.xml"; 
rel="self"/><id>https://datafusion.apache.org/blog/</id><updated>2026-08-07T00:00:00+00:00</updated><entry><title>Apache
 DataFusion Comet 1.0.0 Release</title><link 
href="https://datafusion.apache.org/blog/2026/08/07/datafusion-comet-1.0.0"; 
rel="alternate"/><published>2026- [...]
+<feed xmlns="http://www.w3.org/2005/Atom";><title>Apache DataFusion Blog - 
pmc</title><link href="https://datafusion.apache.org/blog/"; 
rel="alternate"/><link 
href="https://datafusion.apache.org/blog/feeds/pmc.atom.xml"; 
rel="self"/><id>https://datafusion.apache.org/blog/</id><updated>2026-08-09T00:00:00+00:00</updated><entry><title>Apache
 DataFusion Ballista 54.1.0 Released</title><link 
href="https://datafusion.apache.org/blog/2026/08/09/datafusion-ballista-54.1.0"; 
rel="alternate"/><publis [...]
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
@@ -18,10 +18,10 @@ limitations under the License.
 {% endcomment %}
 --&gt;
 
-&lt;p&gt;The Apache DataFusion PMC is pleased to announce version 1.0.0 of the 
&lt;a href="https://datafusion.apache.org/comet/"&gt;Comet&lt;/a&gt; 
subproject.&lt;/p&gt;
-&lt;p&gt;Comet is an accelerator for Apache Spark that translates Spark 
physical plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code 
changes.&lt;/p&gt;
-&lt;p&gt;This release covers roughly six weeks of development 
…&lt;/p&gt;</summary><content type="html">&lt;!--
+&lt;p&gt;We are pleased to announce version &lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;54.1.0&lt;/a&gt;
 of &lt;a href="https://datafusion.apache.org/ballista/"&gt;Apache DataFusion 
Ballista&lt;/a&gt;. Ballista is a distributed query
+execution engine that enhances &lt;a 
href="https://datafusion.apache.org"&gt;Apache DataFusion&lt;/a&gt; by enabling 
parallel execution of workloads across multiple
+nodes.&lt;/p&gt;
+&lt;p&gt;This is a patch release, consisting of 10 commits from 3 
contributors. It follows &lt;a 
href="/blog/2026/07/12/datafusion-ballista-54.0.0/"&gt;54.0.0 
…&lt;/a&gt;&lt;/p&gt;</summary><content type="html">&lt;!--
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
@@ -40,149 +40,58 @@ limitations under the License.
 {% endcomment %}
 --&gt;
 
-&lt;p&gt;The Apache DataFusion PMC is pleased to announce version 1.0.0 of the 
&lt;a href="https://datafusion.apache.org/comet/"&gt;Comet&lt;/a&gt; 
subproject.&lt;/p&gt;
-&lt;p&gt;Comet is an accelerator for Apache Spark that translates Spark 
physical plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code 
changes.&lt;/p&gt;
-&lt;p&gt;This release covers roughly six weeks of development since 0.17.0 and 
consists of 244 commits from 23
-contributors. See the &lt;a 
href="https://github.com/apache/datafusion-comet/blob/main/docs/source/changelog/1.0.0.md"&gt;change
 log&lt;/a&gt; for the full list of changes.&lt;/p&gt;
-&lt;h2 id="the-road-to-10"&gt;The Road to 1.0&lt;a class="headerlink" 
href="#the-road-to-10" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;Comet was &lt;a 
href="https://datafusion.apache.org/blog/2024/03/06/comet-donation/"&gt;donated&lt;/a&gt;
 to the Apache DataFusion project in March 2024 and cut its first release, 
0.1.0, five
-months later with support for 13 operators and 106 expressions. Since then, 
the project has shipped 20
-releases and drawn contributions from more than 120 developers, and the 
codebase now recognizes over 400
-Spark expressions. Operator coverage has grown alongside it: 1.0 accelerates 
each of Spark's four join
-operators, window functions, generators (&lt;code&gt;explode&lt;/code&gt;, 
&lt;code&gt;explode_outer&lt;/code&gt;, &lt;code&gt;posexplode&lt;/code&gt;, 
and &lt;code&gt;posexplode_outer&lt;/code&gt;
-over arrays), sampling, in-memory table scans, and a fully native 
shuffle.&lt;/p&gt;
-&lt;p&gt;The 1.0 release marks the point at which Comet begins following &lt;a 
href="https://datafusion.apache.org/comet/about/versioning_policy.html"&gt;semantic
 versioning&lt;/a&gt;. Users upgrading
-within the 1.x line can expect backward-compatible changes only; features 
slated for removal will be
-deprecated in a minor release before being dropped in the next major version. 
This is why the deprecations
-of JDK 11 and Spark 3.4 announced below are scheduled for 1.1 rather than 
landing in 1.0 itself.&lt;/p&gt;
-&lt;h3 id="support-for-spark-40-with-ansi-mode"&gt;Support for Spark 4.0+ with 
ANSI mode&lt;a class="headerlink" href="#support-for-spark-40-with-ansi-mode" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Comet 1.0.0 supports Spark versions 3.4 through 4.1, with 
experimental support for 4.2. Comet fully supports Spark's ANSI mode, which is 
enabled by default starting with Spark 4.0.&lt;/p&gt;
-&lt;h3 id="correctness-testing"&gt;Correctness Testing&lt;a class="headerlink" 
href="#correctness-testing" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;It is important that queries accelerated by Comet produce the same 
results as Spark. Correctness checking has always been a large effort in Comet 
development, but the approach has evolved over time.&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Upstream Spark tests&lt;/strong&gt;: Comet runs 
Spark's own test suite with Comet enabled, providing more than 24,000 unit 
tests effectively for free. These tests run in Comet's CI for all supported 
Spark versions.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Scala tests&lt;/strong&gt;: end-to-end queries that 
run with Comet enabled versus disabled, checking that results match.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Fuzz testing&lt;/strong&gt;: many of the Scala tests 
generate randomized data to catch regressions around edge cases such as nulls, 
NaN, Infinity, and timezone issues.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Comet SQL tests&lt;/strong&gt;: a 
sqllogictest-inspired approach that makes end-to-end tests easier to 
write.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Generative AI audits&lt;/strong&gt;: agentic skills 
sweep every expression, comparing Comet's implementation to Spark's source and 
ensuring tests cover important edge cases.&lt;/li&gt;
-&lt;/ul&gt;
-&lt;h3 id="performance"&gt;Performance&lt;a class="headerlink" 
href="#performance" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;The early Comet releases provided a very modest speedup and the 
published benchmark results were based on running TPC workloads at small scale 
factors on a single node. There are now independent benchmark results published 
by AWS Labs that show significant speedups for &lt;a 
href="https://awslabs.github.io/data-on-eks/docs/benchmarks/spark-datafusion-comet-benchmark"&gt;TPC-DS
 @ 3TB running in EKS&lt;/a&gt;.&lt;/p&gt;
-&lt;h3 id="codegen-dispatch"&gt;Codegen Dispatch&lt;a class="headerlink" 
href="#codegen-dispatch" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Comet 0.17.0 introduced a new approach to filling gaps in expression 
coverage. In earlier releases,
-whenever Comet's planner encountered an expression that lacked a native Rust 
implementation, it fell back
-to executing an entire subtree of the plan in Spark. That required converting 
Arrow columns back to Spark
-rows before the expression ran and back to Arrow after, and the cost was often 
enough to erase the speedup
-Comet had bought elsewhere in the plan.&lt;/p&gt;
-&lt;p&gt;Codegen dispatch narrows that fallback to the expression itself: the 
batch stays in the Comet pipeline
-and Comet invokes Spark's own generated code for just the missing expression, 
leaving the rest of the query
-running natively. Four consequences are worth calling out.&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Coverage.&lt;/strong&gt; Expressions that would 
previously have blocked native execution of a whole subtree are
-  now supported immediately, without a Rust port.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Compatibility.&lt;/strong&gt; For categories where a 
native reimplementation would inevitably diverge from Spark's
-  semantics — regular expressions being the canonical case, given the gap 
between Java's regex engine and
-  any Rust or C++ equivalent — codegen dispatch delivers bit-for-bit Spark 
parity because it &lt;em&gt;is&lt;/em&gt; Spark's
-  implementation.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Expression fusion.&lt;/strong&gt; A dispatched 
expression tree (&lt;em&gt;i.e.&lt;/em&gt;, nested expressions) is compiled 
into a single method, so the Arrow
-  input reads, the expression evaluation, and the Arrow output writes are 
fused together. The compiler
-  is free to optimize across the whole tree, and no intermediate Arrow 
&lt;code&gt;RecordBatch&lt;/code&gt; is materialized
-  between one expression and the next.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Scala and Java UDFs.&lt;/strong&gt; User-defined 
functions are compiled to the same codegen surface as built-in
-  expressions, so they can flow through codegen dispatch without any change 
from the user. Queries that
-  were previously disqualified from acceleration only because they contained a 
UDF can now benefit as
-  long as the surrounding operators are supported. See the &lt;a 
href="https://datafusion.apache.org/comet/user-guide/latest/scala_java_udfs.html"&gt;Scala
 and Java UDF guide&lt;/a&gt; for details.&lt;/li&gt;
-&lt;/ul&gt;
-&lt;p&gt;Comet 1.0 widens the mechanism in three ways.&lt;/p&gt;
-&lt;p&gt;The first is the biggest. An expression that opts into codegen 
dispatch previously reached the dispatcher
-only when Comet reported it as &lt;em&gt;incompatible&lt;/em&gt; for the given 
input; an &lt;em&gt;unsupported&lt;/em&gt; report still sent the
-whole subtree back to Spark. In 1.0 both support levels route through the 
dispatcher, so an input that
-Spark handles and Comet's native code does not now stays inside the Comet 
pipeline.&lt;/p&gt;
-&lt;p&gt;Second, casts join the same path. Cast expressions that Comet 
declines to run natively — including legacy
-configuration variants such as 
&lt;code&gt;spark.sql.legacy.castComplexTypesToString.enabled&lt;/code&gt; — 
are now dispatched
-rather than falling back, and more string, array, and interval expressions 
were opted in as well.&lt;/p&gt;
-&lt;p&gt;Third, the path is now visible. Comet's extended explain output 
reports native versus codegen-dispatch
-coverage for a plan, so you can see which path each expression actually took 
rather than inferring it from
-the absence of a fallback reason.&lt;/p&gt;
-&lt;h2 id="improvements-since-0170"&gt;Improvements since 0.17.0&lt;a 
class="headerlink" href="#improvements-since-0170" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;The rest of this post covers what is new since the 0.17.0 
release.&lt;/p&gt;
-&lt;h3 id="experimental-pyarrow-udf-support"&gt;Experimental PyArrow UDF 
Support&lt;a class="headerlink" href="#experimental-pyarrow-udf-support" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;This release adds experimental support for accelerated PyArrow UDFs, 
allowing PyArrow-based user-defined
-functions to participate in native execution instead of forcing a fallback to 
Spark. When the feature is
-disabled, Comet now hints at the native PyArrow UDF path in its fallback 
reasons so users know the option
-exists. This is an early-stage feature and we welcome feedback from users 
experimenting with it.&lt;/p&gt;
-&lt;h3 id="new-expression-and-aggregate-support"&gt;New Expression and 
Aggregate Support&lt;a class="headerlink" 
href="#new-expression-and-aggregate-support" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;This release expands the set of Spark expressions and aggregates that 
are accelerated by Comet:&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Aggregates&lt;/strong&gt;: 
&lt;code&gt;approx_percentile&lt;/code&gt; / 
&lt;code&gt;percentile_approx&lt;/code&gt;, exact 
&lt;code&gt;percentile&lt;/code&gt; / &lt;code&gt;median&lt;/code&gt;,
-  &lt;code&gt;approx_count_distinct&lt;/code&gt;, and native 
&lt;code&gt;collect_list&lt;/code&gt; / 
&lt;code&gt;array_agg&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Cast&lt;/strong&gt;: Cast expressions where the native 
implementation is marked as incompatible or unsupported
-  are now routed through codegen dispatch.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Grouping&lt;/strong&gt;: 
&lt;code&gt;grouping()&lt;/code&gt; and 
&lt;code&gt;grouping_id()&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Intervals&lt;/strong&gt;: interval types via 
&lt;code&gt;make_ym_interval&lt;/code&gt; and 
&lt;code&gt;make_dt_interval&lt;/code&gt;, 
&lt;code&gt;CalendarIntervalType&lt;/code&gt;,
-  &lt;code&gt;multiply_dt_interval&lt;/code&gt;, and interval codegen dispatch 
for nested values and native shuffle.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;String&lt;/strong&gt;: 
&lt;code&gt;base64&lt;/code&gt;, &lt;code&gt;split_part&lt;/code&gt; via 
&lt;code&gt;StringSplitSQL&lt;/code&gt;, native 
&lt;code&gt;levenshtein&lt;/code&gt;, and native 
&lt;code&gt;randstr&lt;/code&gt; and
-  &lt;code&gt;uuid&lt;/code&gt; — both bit-for-bit compatible with Spark for a 
given seed.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Array / map&lt;/strong&gt;: 
&lt;code&gt;array_prepend&lt;/code&gt;, the &lt;code&gt;shuffle()&lt;/code&gt; 
array function, &lt;code&gt;size()&lt;/code&gt; for 
&lt;code&gt;MapType&lt;/code&gt;, and &lt;code&gt;ElementAt&lt;/code&gt;
-  over &lt;code&gt;MapType&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Date/time&lt;/strong&gt;: native 
&lt;code&gt;TimestampNTZ&lt;/code&gt; inputs for &lt;code&gt;hour&lt;/code&gt; 
/ &lt;code&gt;minute&lt;/code&gt; / &lt;code&gt;second&lt;/code&gt; and
-  &lt;code&gt;PreciseTimestampConversion&lt;/code&gt; for native time-window 
grouping.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Windows&lt;/strong&gt;: extended native window 
function support and Spark 4 decimal window average.&lt;/li&gt;
-&lt;/ul&gt;
-&lt;h3 id="faster-parquet-scans"&gt;Faster Parquet Scans&lt;a 
class="headerlink" href="#faster-parquet-scans" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Parquet reads pick up several improvements as well. Full Parquet 
metadata, including the page index, is now
-cached via DataFusion's 
&lt;code&gt;CachedParquetFileReaderFactory&lt;/code&gt;; identity casts are 
unwrapped in the schema adapter
-so Parquet statistics pruning can engage; filter pushdown configuration has 
been revised; the native scan
-passes a metadata size hint so a single read usually captures the footer; and 
the native Parquet scan seeds
-its reader options from the session config so Parquet settings you already set 
take effect.&lt;/p&gt;
-&lt;h3 id="iceberg-table-format-v3"&gt;Iceberg Table Format V3&lt;a 
class="headerlink" href="#iceberg-table-format-v3" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Comet now supports Iceberg 1.11 and its first Iceberg table format V3 
feature: full table encryption. Other V3 features like deletion vectors and new 
data types (&lt;em&gt;e.g.&lt;/em&gt;, &lt;code&gt;VARIANT&lt;/code&gt;) fall 
back gracefully. The native Iceberg scan supports the 
&lt;code&gt;_pos&lt;/code&gt;, &lt;code&gt;_spec&lt;/code&gt;, 
&lt;code&gt;_file&lt;/code&gt;, and
-&lt;code&gt;_partition&lt;/code&gt; metadata columns, sizes delete files 
correctly to avoid dropped deletes, disambiguates scans that
-share a &lt;code&gt;metadata_location&lt;/code&gt;, and dedupes residuals and 
delete files in the native scan serde. A prior case
-where Iceberg native scan exchange reuse with different pushed filters could 
produce wrong results is also
-fixed.&lt;/p&gt;
-&lt;h3 id="native-expression-performance"&gt;Native Expression 
Performance&lt;a class="headerlink" href="#native-expression-performance" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h3&gt;
-&lt;p&gt;Many native expression implementations have been optimized to more 
efficiently leverage Arrow kernels or to avoid per-row builders.&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Casts&lt;/strong&gt; between numeric, string, decimal, 
and date types, including a faster float-to-decimal cast, an
-  optimized integer-to-integer cast, shared no-overflow fast paths in 
&lt;code&gt;CheckOverflow&lt;/code&gt; and
-  &lt;code&gt;DecimalRescaleCheckOverflow&lt;/code&gt;, and a 
&lt;code&gt;cast_binary_to_string&lt;/code&gt; that is up to 27x faster on 
binary-format
-  styles.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;JSON, regex, and URL parsing&lt;/strong&gt;: 
&lt;code&gt;get_json_object&lt;/code&gt;, 
&lt;code&gt;regexp_extract&lt;/code&gt;, and 
&lt;code&gt;parse_url&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Date/time and decimal kernels&lt;/strong&gt;: 
&lt;code&gt;date_trunc&lt;/code&gt;, &lt;code&gt;spark_ceil&lt;/code&gt;, and a 
vectorized &lt;code&gt;spark_unscaled_value&lt;/code&gt;.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;String and array kernels&lt;/strong&gt;: 
&lt;code&gt;lpad&lt;/code&gt;, &lt;code&gt;unhex&lt;/code&gt;, 
&lt;code&gt;size&lt;/code&gt;, &lt;code&gt;arrays_overlap&lt;/code&gt;, 
&lt;code&gt;escape_string&lt;/code&gt;, and the &lt;code&gt;try_*&lt;/code&gt;
-  arithmetic kernel.&lt;/li&gt;
-&lt;/ul&gt;
-&lt;p&gt;To make this kind of work repeatable, the release also adds a &lt;a 
href="https://datafusion.apache.org/comet/contributor-guide/optimizing_expressions.html"&gt;scalar
 expression optimization guide&lt;/a&gt;
-documenting how to benchmark a kernel, keep its output bit-identical to Spark, 
and gate changes on a
-no-regression check.&lt;/p&gt;
-&lt;h2 id="deprecation-notice"&gt;Deprecation Notice&lt;a class="headerlink" 
href="#deprecation-notice" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;With the move to a stable 1.0 release line, Comet begins deprecating 
older platforms under its
-&lt;a 
href="https://datafusion.apache.org/comet/about/versioning_policy.html"&gt;versioning
 policy&lt;/a&gt;:&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;JDK 11&lt;/strong&gt; is deprecated and scheduled for 
removal in Comet 1.1.0.&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Apache Spark 3.4&lt;/strong&gt; is deprecated and 
scheduled for removal in Comet 1.1.0.&lt;/li&gt;
-&lt;/ul&gt;
-&lt;p&gt;Comet aligns its Spark support window with upstream Apache Spark 
maintenance. Spark 3.4 is no longer
-maintained upstream, so under the versioning policy it is deprecated in the 
first Comet minor release
-after that point and removed in the following one. Comet 1.0.0 still builds 
and publishes Spark 3.4
-binaries.&lt;/p&gt;
-&lt;p&gt;Users on these platforms should plan to move to JDK 17+ and Spark 3.5 
or later before upgrading to 1.1.0.&lt;/p&gt;
-&lt;h2 id="compatibility"&gt;Compatibility&lt;a class="headerlink" 
href="#compatibility" title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;Supported platforms include:&lt;/p&gt;
-&lt;ul&gt;
-&lt;li&gt;&lt;strong&gt;Spark 3.4.3&lt;/strong&gt; with Java 11/17 and Scala 
2.12/2.13 (deprecated, removal in 1.1.0)&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 3.5.9&lt;/strong&gt; with Java 11/17 and Scala 
2.12/2.13&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 4.0.4&lt;/strong&gt; with Java 17 and Scala 
2.13&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 4.1.3&lt;/strong&gt; with Java 17/21 and Scala 
2.13&lt;/li&gt;
-&lt;li&gt;&lt;strong&gt;Spark 4.2&lt;/strong&gt; with Java 17 and Scala 2.13 
(experimental, for early evaluation only)&lt;/li&gt;
+&lt;p&gt;We are pleased to announce version &lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;54.1.0&lt;/a&gt;
 of &lt;a href="https://datafusion.apache.org/ballista/"&gt;Apache DataFusion 
Ballista&lt;/a&gt;. Ballista is a distributed query
+execution engine that enhances &lt;a 
href="https://datafusion.apache.org"&gt;Apache DataFusion&lt;/a&gt; by enabling 
parallel execution of workloads across multiple
+nodes.&lt;/p&gt;
+&lt;p&gt;This is a patch release, consisting of 10 commits from 3 
contributors. It follows &lt;a 
href="/blog/2026/07/12/datafusion-ballista-54.0.0/"&gt;54.0.0&lt;/a&gt; and 
contains no
+new features and no breaking changes — only an upgrade to DataFusion 54.1.0 
and a set of bug fixes
+backported from &lt;code&gt;main&lt;/code&gt;. Users running 54.0.0 are 
encouraged to upgrade.&lt;/p&gt;
+&lt;h2 id="upgrade-to-datafusion-5410"&gt;Upgrade to DataFusion 54.1.0&lt;a 
class="headerlink" href="#upgrade-to-datafusion-5410" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;p&gt;Ballista 54.1.0 is built on DataFusion 54.1.0, picking up the fixes 
in that patch release. No API changes
+were required, so the &lt;a 
href="https://datafusion.apache.org/ballista/upgrading/54.0.0.html"&gt;54.0.0 
upgrade guide&lt;/a&gt; still applies for anyone moving from an earlier 
release.&lt;/p&gt;
+&lt;h2 id="cluster-stability-fixes"&gt;Cluster stability fixes&lt;a 
class="headerlink" href="#cluster-stability-fixes" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;p&gt;Three fixes address problems that only appear in a real multi-node 
deployment:&lt;/p&gt;
+&lt;ul&gt;
+&lt;li&gt;&lt;strong&gt;Executor client pool enabled by 
default.&lt;/strong&gt; The executor's &lt;code&gt;client_ttl&lt;/code&gt; 
defaulted to &lt;code&gt;0&lt;/code&gt;, which disabled
+  the gRPC client pool entirely. Without a pool, the shuffle-read path opened 
and discarded a TCP connection
+  per fetch, so a single shuffle-heavy query could exhaust the host's 
ephemeral ports and fail with
+  &lt;code&gt;AddrNotAvailable&lt;/code&gt; — as would every query after it, 
until those sockets left &lt;code&gt;TIME_WAIT&lt;/code&gt;. A single-executor
+  cluster never showed this, because shuffle reads are served from local 
files. &lt;code&gt;client_ttl&lt;/code&gt; now defaults to
+  30 seconds, so the pool is on unless explicitly disabled.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Executor startup race.&lt;/strong&gt; The executor 
spawned its gRPC server and registered with the scheduler
+  immediately afterwards, but the socket was not bound yet when the scheduler 
dialed back. Losing that race
+  produced &lt;code&gt;ECONNREFUSED&lt;/code&gt;, a failed registration, and 
an executor that died at startup. The listener is now
+  bound before registration can run.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Executor shutdown.&lt;/strong&gt; The shutdown 
notification is now sent before the notifier is dropped, so a graceful
+  shutdown is actually observed by the components waiting on it.&lt;/li&gt;
+&lt;/ul&gt;
+&lt;h2 id="task-retry-and-shuffle-memory"&gt;Task retry and shuffle 
memory&lt;a class="headerlink" href="#task-retry-and-shuffle-memory" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;ul&gt;
+&lt;li&gt;&lt;strong&gt;Retryable IO errors on a join's build 
side.&lt;/strong&gt; A retryable IO error raised on a join's shared build side
+  arrives at the failed-task classifier wrapped in 
&lt;code&gt;DataFusionError::Shared&lt;/code&gt;, so a match on the outermost
+  variant never saw the underlying &lt;code&gt;IoError&lt;/code&gt; and the 
task was marked non-retryable. Classification now
+  operates on the root error, regardless of wrapping.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Sort shuffle writer respects memory pool 
pressure.&lt;/strong&gt; The sort shuffle writer discarded the result of
+  &lt;code&gt;try_grow&lt;/code&gt; on its memory reservation, so a rejected 
grow produced no spill: the batch stayed buffered while
+  the shared &lt;code&gt;MemoryPool&lt;/code&gt; had not accounted for it, and 
other consumers saw memory as free that was not. A
+  rejected grow is now treated as a spill trigger alongside the existing 
per-task budget.&lt;/li&gt;
+&lt;/ul&gt;
+&lt;h2 id="planning-correctness"&gt;Planning correctness&lt;a 
class="headerlink" href="#planning-correctness" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;ul&gt;
+&lt;li&gt;&lt;strong&gt;&lt;code&gt;GROUP BY&lt;/code&gt;-less aggregates are 
preserved&lt;/strong&gt; when the scheduler propagates empty stages, so a query 
whose
+  input produces no rows still returns the correct single-row aggregate result 
rather than nothing.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;&lt;code&gt;EmptyExec&lt;/code&gt; partition counts 
survive stage serialization&lt;/strong&gt;, fixing plans whose partitioning 
changed as a
+  stage was sent to an executor.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;Task partitions are mapped through 
&lt;code&gt;UnionExec&lt;/code&gt;&lt;/strong&gt; when restricting file scans, 
so each task scans the
+  files it was assigned.&lt;/li&gt;
+&lt;li&gt;&lt;strong&gt;&lt;code&gt;SortMergeJoinExec&lt;/code&gt; broadcast 
conversion is now disabled by default.&lt;/strong&gt; The static-planner 
optimization
+  that broadcasts the small build side of a sort-merge join, added in 54.0.0, 
is opt-in in this release while
+  outstanding issues are resolved.&lt;/li&gt;
 &lt;/ul&gt;
-&lt;p&gt;See the &lt;a 
href="https://datafusion.apache.org/comet/user-guide/latest/compatibility/spark-versions.html"&gt;Spark
 Version Compatibility&lt;/a&gt; page for known limitations specific to each 
version.&lt;/p&gt;
-&lt;p&gt;This release upgrades to &lt;strong&gt;DataFusion 54.1&lt;/strong&gt; 
and &lt;strong&gt;Arrow 58.4&lt;/strong&gt;.&lt;/p&gt;
-&lt;h2 id="get-started-with-comet-100"&gt;Get Started with Comet 1.0.0&lt;a 
class="headerlink" href="#get-started-with-comet-100" title="Permanent 
link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
-&lt;p&gt;Ready to try it out? Follow the &lt;a 
href="https://datafusion.apache.org/comet/user-guide/1.0/installation.html"&gt;Comet
 1.0.0 Installation Guide&lt;/a&gt;
-to get up and running, then point Comet at your existing Spark workloads and 
see the speedup for yourself.&lt;/p&gt;</content><category 
term="blog"/></entry><entry><title>Apache DataFusion Ballista 54.0.0 
Released</title><link 
href="https://datafusion.apache.org/blog/2026/07/12/datafusion-ballista-54.0.0"; 
rel="alternate"/><published>2026-07-12T00:00:00+00:00</published><updated>2026-07-12T00:00:00+00:00</updated><author><name>pmc</name></author><id>tag:datafusion.apache.org,2026-07-12:/b
 [...]
+&lt;h2 id="thank-you"&gt;Thank You&lt;a class="headerlink" href="#thank-you" 
title="Permanent link"&gt;¶&lt;/a&gt;&lt;/h2&gt;
+&lt;p&gt;Thanks to Andy Grove, Alexander Domenti, and goingforstudying-ctrl 
for the commits in this release, and to
+everyone who contributed by filing issues, reviewing PRs, and providing 
feedback. Full details are in the
+&lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;changelog&lt;/a&gt;.&lt;/p&gt;</content><category
 term="blog"/></entry><entry><title>Apache DataFusion Ballista 54.0.0 
Released</title><link 
href="https://datafusion.apache.org/blog/2026/07/12/datafusion-ballista-54.0.0"; 
rel="alternate"/><published>2026-07-12T00:00:00+00:00</published><updated>2026-07-12T00:00:00+00:00</updated><author><name>pmc</name></author><id>tag:datafusion.apac
 [...]
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
diff --git a/blog/feeds/pmc.rss.xml b/blog/feeds/pmc.rss.xml
index 9633359..ff84c71 100644
--- a/blog/feeds/pmc.rss.xml
+++ b/blog/feeds/pmc.rss.xml
@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="utf-8"?>
-<rss version="2.0" 
xmlns:atom="http://www.w3.org/2005/Atom";><channel><title>Apache DataFusion Blog 
- 
pmc</title><link>https://datafusion.apache.org/blog/</link><description/><atom:link
 href="https://datafusion.apache.org/blog/feeds/pmc.rss.xml"; 
rel="self"/><lastBuildDate>Fri, 07 Aug 2026 00:00:00 
+0000</lastBuildDate><item><title>Apache DataFusion Comet 1.0.0 
Release</title><link>https://datafusion.apache.org/blog/2026/08/07/datafusion-comet-1.0.0</link><description>&lt;!--
+<rss version="2.0" 
xmlns:atom="http://www.w3.org/2005/Atom";><channel><title>Apache DataFusion Blog 
- 
pmc</title><link>https://datafusion.apache.org/blog/</link><description/><atom:link
 href="https://datafusion.apache.org/blog/feeds/pmc.rss.xml"; 
rel="self"/><lastBuildDate>Sun, 09 Aug 2026 00:00:00 
+0000</lastBuildDate><item><title>Apache DataFusion Ballista 54.1.0 
Released</title><link>https://datafusion.apache.org/blog/2026/08/09/datafusion-ballista-54.1.0</link><description>&lt;!--
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
@@ -18,10 +18,10 @@ limitations under the License.
 {% endcomment %}
 --&gt;
 
-&lt;p&gt;The Apache DataFusion PMC is pleased to announce version 1.0.0 of the 
&lt;a href="https://datafusion.apache.org/comet/"&gt;Comet&lt;/a&gt; 
subproject.&lt;/p&gt;
-&lt;p&gt;Comet is an accelerator for Apache Spark that translates Spark 
physical plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code 
changes.&lt;/p&gt;
-&lt;p&gt;This release covers roughly six weeks of development 
…&lt;/p&gt;</description><dc:creator 
xmlns:dc="http://purl.org/dc/elements/1.1/";>pmc</dc:creator><pubDate>Fri, 07 
Aug 2026 00:00:00 
+0000</pubDate><guid>tag:datafusion.apache.org,2026-08-07:/blog/2026/08/07/datafusion-comet-1.0.0</guid><category>blog</category></item><item><title>Apache
 DataFusion Ballista 54.0.0 
Released</title><link>https://datafusion.apache.org/blog/2026/07/12/datafusion-ballista-54.0.0</link><description>&lt;!--
+&lt;p&gt;We are pleased to announce version &lt;a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md"&gt;54.1.0&lt;/a&gt;
 of &lt;a href="https://datafusion.apache.org/ballista/"&gt;Apache DataFusion 
Ballista&lt;/a&gt;. Ballista is a distributed query
+execution engine that enhances &lt;a 
href="https://datafusion.apache.org"&gt;Apache DataFusion&lt;/a&gt; by enabling 
parallel execution of workloads across multiple
+nodes.&lt;/p&gt;
+&lt;p&gt;This is a patch release, consisting of 10 commits from 3 
contributors. It follows &lt;a 
href="/blog/2026/07/12/datafusion-ballista-54.0.0/"&gt;54.0.0 
…&lt;/a&gt;&lt;/p&gt;</description><dc:creator 
xmlns:dc="http://purl.org/dc/elements/1.1/";>pmc</dc:creator><pubDate>Sun, 09 
Aug 2026 00:00:00 
+0000</pubDate><guid>tag:datafusion.apache.org,2026-08-09:/blog/2026/08/09/datafusion-ballista-54.1.0</guid><category>blog</category></item><item><title>Apache
 DataFusion Ballista 54.0.0 Rele [...]
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
diff --git a/blog/index.html b/blog/index.html
index a1c79ae..49ee0ba 100644
--- a/blog/index.html
+++ b/blog/index.html
@@ -59,8 +59,8 @@
             <article class="post">
                 <header>
                     <div class="title">
-                        <h1><a 
href="/blog/2026/08/07/datafusion-comet-1.0.0">Apache DataFusion Comet 1.0.0 
Release</a></h1>
-                        <p>Posted on: Fri 07 August 2026 by pmc</p>
+                        <h1><a 
href="/blog/2026/08/09/datafusion-ballista-54.1.0">Apache DataFusion Ballista 
54.1.0 Released</a></h1>
+                        <p>Posted on: Sun 09 August 2026 by pmc</p>
                         <p><!--
 {% comment %}
 Licensed to the Apache Software Foundation (ASF) under one or more
@@ -80,13 +80,13 @@ limitations under the License.
 {% endcomment %}
 -->
 
-<p>The Apache DataFusion PMC is pleased to announce version 1.0.0 of the <a 
href="https://datafusion.apache.org/comet/";>Comet</a> subproject.</p>
-<p>Comet is an accelerator for Apache Spark that translates Spark physical 
plans to DataFusion physical plans for
-improved performance and efficiency without requiring any code changes.</p>
-<p>This release covers roughly six weeks of development …</p></p>
+<p>We are pleased to announce version <a 
href="https://github.com/apache/datafusion-ballista/blob/54.1.0/docs/source/changelog/54.1.0.md";>54.1.0</a>
 of <a href="https://datafusion.apache.org/ballista/";>Apache DataFusion 
Ballista</a>. Ballista is a distributed query
+execution engine that enhances <a href="https://datafusion.apache.org";>Apache 
DataFusion</a> by enabling parallel execution of workloads across multiple
+nodes.</p>
+<p>This is a patch release, consisting of 10 commits from 3 contributors. It 
follows <a href="/blog/2026/07/12/datafusion-ballista-54.0.0/">54.0.0 
…</a></p></p>
                         <footer>
                             <ul class="actions">
-                                <div style="text-align: right"><a 
href="/blog/2026/08/07/datafusion-comet-1.0.0" class="button medium">Continue 
Reading</a></div>
+                                <div style="text-align: right"><a 
href="/blog/2026/08/09/datafusion-ballista-54.1.0" class="button 
medium">Continue Reading</a></div>
                             </ul>
                             <ul class="stats">
                             </ul>


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to