This is an automated email from the ASF dual-hosted git repository.
andygrove pushed a commit to branch branch-1.0
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/branch-1.0 by this push:
new 98b01810bb docs: add 1.0.0 changelog (#5243)
98b01810bb is described below
commit 98b01810bb720bf5cf09a5d3c6ae1ced46c16bb8
Author: Andy Grove <[email protected]>
AuthorDate: Mon Aug 3 17:11:17 2026 -0600
docs: add 1.0.0 changelog (#5243)
---
docs/source/changelog/1.0.0.md | 313 +++++++++++++++++++++++++++++++++++++++++
docs/source/changelog/index.md | 1 +
2 files changed, 314 insertions(+)
diff --git a/docs/source/changelog/1.0.0.md b/docs/source/changelog/1.0.0.md
new file mode 100644
index 0000000000..fbd8ca3ba7
--- /dev/null
+++ b/docs/source/changelog/1.0.0.md
@@ -0,0 +1,313 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# DataFusion Comet 1.0.0 Changelog
+
+This release consists of 244 commits from 23 contributors. See credits at the
end of this changelog for more information.
+
+**Fixed bugs:**
+
+- fix: decline native V1 scans on object_store-unsupported filesystem schemes
[#4525](https://github.com/apache/datafusion-comet/pull/4525) (schenksj)
+- fix: exclude release scratch dirs from RAT and license skill docs
[#4685](https://github.com/apache/datafusion-comet/pull/4685) (andygrove)
+- fix(shuffle): tolerate non-UTF-8 bytes in get_string (lossy decode)
[#4524](https://github.com/apache/datafusion-comet/pull/4524) (schenksj)
+- fix: decline CreateArray with struct-nullability-divergent children
[#4533](https://github.com/apache/datafusion-comet/pull/4533) (schenksj)
+- fix: propagate nested cast errors
[#4675](https://github.com/apache/datafusion-comet/pull/4675) (manuzhang)
+- fix: reject Parquet INT96 as TimestampNTZ on Spark 3.x
[#4357](https://github.com/apache/datafusion-comet/pull/4357) (andygrove)
+- fix: gate str_to_map collations
[#4701](https://github.com/apache/datafusion-comet/pull/4701) (manuzhang)
+- fix: repair broken rust-test build on main (init_datasource_exec arg
mismatch) [#4712](https://github.com/apache/datafusion-comet/pull/4712)
(andygrove)
+- fix: correct stale unix_timestamp NTZ and date_format codegen-default doc
text (#4502) [#4645](https://github.com/apache/datafusion-comet/pull/4645)
(andygrove)
+- fix: fall back for decimal SUM/AVG over sliding window frames (window audit)
[#4732](https://github.com/apache/datafusion-comet/pull/4732) (andygrove)
+- fix: date_trunc schema mismatch and DST handling in non-UTC timezones
[#4761](https://github.com/apache/datafusion-comet/pull/4761) (andygrove)
+- fix: size Iceberg delete files in the native scan to avoid dropping deletes
[#4760](https://github.com/apache/datafusion-comet/pull/4760) (mbutrovich)
+- fix: gate non-default collations for Spark 4 datetime expressions
[#4693](https://github.com/apache/datafusion-comet/pull/4693) (0lai0)
+- fix: ArrayInsert evaluation for null source arrays
[#4726](https://github.com/apache/datafusion-comet/pull/4726) (peterxcli)
+- fix: support Spark 4 decimal window avg
[#4749](https://github.com/apache/datafusion-comet/pull/4749) (manuzhang)
+- fix: array expression audit follow-ups (#4503)
[#4713](https://github.com/apache/datafusion-comet/pull/4713) (andygrove)
+- fix: prevent wrong results from Iceberg native scan exchange reuse with
different pushed filters
[#4812](https://github.com/apache/datafusion-comet/pull/4812) (mbutrovich)
+- fix: count returning zero when scan is disabled and going through
CometSparkColumnarToColumnar
[#4795](https://github.com/apache/datafusion-comet/pull/4795) (Dummk0pf)
+- fix: handle null sub-arrays in flatten
[#4822](https://github.com/apache/datafusion-comet/pull/4822)
(michaelmitchell-bit)
+- fix: correct user guide URL in compatibility fallback messages
[#4854](https://github.com/apache/datafusion-comet/pull/4854) (andygrove)
+- fix: report FromUnixTime non-default format as Unsupported
[#4847](https://github.com/apache/datafusion-comet/pull/4847) (andygrove)
+- fix: emit JDK libjvm search path from core build script to fix -ljvm CI link
failures [#4868](https://github.com/apache/datafusion-comet/pull/4868)
(andygrove)
+- fix: register partitioning scalar subqueries for native shuffle to avoid
"Subquery N not found"
[#4869](https://github.com/apache/datafusion-comet/pull/4869) (mbutrovich)
+- fix: match Spark percentile interpolation precision
[#4792](https://github.com/apache/datafusion-comet/pull/4792) (manuzhang)
+- fix: restrict array_filter array_compact fast path to the lambda variable
[#4848](https://github.com/apache/datafusion-comet/pull/4848) (andygrove)
+- fix: suppress spurious WriteFilesExec fallback reason on native writes
[#4928](https://github.com/apache/datafusion-comet/pull/4928) (andygrove)
+- fix: decode CAST(binary AS string) JVM-compatibly instead of unsafe
reinterpret [#4763](https://github.com/apache/datafusion-comet/pull/4763)
(andygrove)
+- fix: widen local table scan child nullability to match native kernels
[#4843](https://github.com/apache/datafusion-comet/pull/4843) (andygrove)
+- fix: apply 1.11 Iceberg diff changes from #4991 to other versions
[#5020](https://github.com/apache/datafusion-comet/pull/5020) (mbutrovich)
+- fix: raise CAST_INVALID_INPUT for invalid calendar dates in ANSI cast to
date [#5014](https://github.com/apache/datafusion-comet/pull/5014) (andygrove)
+- fix: materialize ConstantColumnVector on Comet's serialize/export paths
[#4532](https://github.com/apache/datafusion-comet/pull/4532) (schenksj)
+- fix: make native pow implementation compatible with Spark
[#5033](https://github.com/apache/datafusion-comet/pull/5033) (andygrove)
+- fix: resolve Comet jar without hardcoding the SNAPSHOT qualifier
[#5108](https://github.com/apache/datafusion-comet/pull/5108) (andygrove)
+- fix: honour Spark's legacy `null IN ()` behavior
[#5127](https://github.com/apache/datafusion-comet/pull/5127) (andygrove)
+- fix: honor fail_on_error in native make_decimal
[#5080](https://github.com/apache/datafusion-comet/pull/5080) (andygrove)
+- fix: work around DataFusion 54.1.0 Parquet page-index regression
[#5132](https://github.com/apache/datafusion-comet/pull/5132) (mbutrovich)
+- fix: throw ARITHMETIC_OVERFLOW for Long.MinValue div -1 under ANSI mode
[#5084](https://github.com/apache/datafusion-comet/pull/5084) (andygrove)
+- fix: seed native Parquet scan reader options from session config
[#5107](https://github.com/apache/datafusion-comet/pull/5107) (mbutrovich)
+- fix: raise REMAINDER_BY_ZERO for Float/Double under ANSI mode
[#5081](https://github.com/apache/datafusion-comet/pull/5081) (andygrove)
+- fix: reduce log verbosity at per-task logging callsites
[#5155](https://github.com/apache/datafusion-comet/pull/5155) (mbutrovich)
+- fix: round on Int64 with scale <= -19 now overflows correctly
[#5082](https://github.com/apache/datafusion-comet/pull/5082) (andygrove)
+- fix: match Spark's whitespace trim semantics for casts from string to
boolean, integral, float/double and decimal
[#5150](https://github.com/apache/datafusion-comet/pull/5150) (andygrove)
+- fix: disambiguate Iceberg scans that share a metadata_location
[#5180](https://github.com/apache/datafusion-comet/pull/5180) (mbutrovich)
+- fix: use per-expression eval mode for decimal promotion
[#5171](https://github.com/apache/datafusion-comet/pull/5171) (peterxcli)
+- fix: codegen dispatcher null short-circuit swallowed ANSI errors, plus two
latent TIME-type gaps
[#5219](https://github.com/apache/datafusion-comet/pull/5219) (andygrove)
+- fix: count ReusedSubquery and CometSubqueryBroadcast correctly in extended
explain [#5206](https://github.com/apache/datafusion-comet/pull/5206)
(andygrove)
+
+**Performance related:**
+
+- perf: O(1) PlanDataInjector lookup by op kind
[#4535](https://github.com/apache/datafusion-comet/pull/4535) (schenksj)
+- perf: cache full Parquet metadata (incl. page index) via DataFusion's
CachedParquetFileReaderFactory
[#4707](https://github.com/apache/datafusion-comet/pull/4707) (mbutrovich)
+- perf: add metadata size hint to Parquet reader to match Iceberg path
[#4717](https://github.com/apache/datafusion-comet/pull/4717) (mbutrovich)
+- perf(parquet): revise filter pushdown configuration
[#4722](https://github.com/apache/datafusion-comet/pull/4722) (mbutrovich)
+- perf: unwrap identity casts in schema adapter to enable Parquet stats
pruning [#4730](https://github.com/apache/datafusion-comet/pull/4730)
(mbutrovich)
+- perf: avoid excessive timer calls
[#4739](https://github.com/apache/datafusion-comet/pull/4739) (wForget)
+- perf: optimize spark_size in spark-expr
[#4877](https://github.com/apache/datafusion-comet/pull/4877) (andygrove)
+- perf: optimize spark_unhex in spark-expr
[#4876](https://github.com/apache/datafusion-comet/pull/4876) (andygrove)
+- perf: optimize `parse_url` (50x speedup)
[#4893](https://github.com/apache/datafusion-comet/pull/4893) (andygrove)
+- perf: optimize `to_json` (2x faster)
[#4902](https://github.com/apache/datafusion-comet/pull/4902) (andygrove)
+- perf: optimize `spark_cast_int_to_int` (100x faster)
[#4920](https://github.com/apache/datafusion-comet/pull/4920) (andygrove)
+- perf: optimize `try_arithmetic_kernel` (up to 4x faster)
[#4910](https://github.com/apache/datafusion-comet/pull/4910) (andygrove)
+- perf: optimize `spark_arrays_overlap` (up to 18x faster)
[#4906](https://github.com/apache/datafusion-comet/pull/4906) (andygrove)
+- perf: optimize `spark_lpad` (up to 2x faster)
[#4919](https://github.com/apache/datafusion-comet/pull/4919) (andygrove)
+- perf: optimize `CheckOverflow` with a shared no-overflow fast path (ANSI and
non-ANSI) [#4937](https://github.com/apache/datafusion-comet/pull/4937)
(andygrove)
+- perf: skip the null-masking pass in `DecimalRescaleCheckOverflow` when
nothing overflows [#4938](https://github.com/apache/datafusion-comet/pull/4938)
(andygrove)
+- perf: vectorize floating-point-to-decimal cast
[#4940](https://github.com/apache/datafusion-comet/pull/4940) (andygrove)
+- perf: optimize `spark_get_json_object` (4x faster)
[#4907](https://github.com/apache/datafusion-comet/pull/4907) (andygrove)
+- perf: optimize `spark_regexp_extract`
[#4894](https://github.com/apache/datafusion-comet/pull/4894) (andygrove)
+- perf: vectorize `spark_unscaled_value` (9x faster)
[#4972](https://github.com/apache/datafusion-comet/pull/4972) (u70b3)
+- perf: optimize `cast_binary_to_string` binary-format styles (up to 27x
faster) [#4912](https://github.com/apache/datafusion-comet/pull/4912)
(andygrove)
+- perf: optimize `parse_string_to_decimal` (30-40% faster)
[#4916](https://github.com/apache/datafusion-comet/pull/4916) (andygrove)
+- perf: optimize `date_parser` / cast string to date (up to 2x faster)
[#4917](https://github.com/apache/datafusion-comet/pull/4917) (andygrove)
+- perf: optimize cast_decimal128_to_utf8 in datafusion-comet-spark-expr
[#4924](https://github.com/apache/datafusion-comet/pull/4924) (andygrove)
+- perf: optimize `date_trunc` (>2x faster)
[#4915](https://github.com/apache/datafusion-comet/pull/4915) (andygrove)
+- perf: optimize `spark_cast_float64_to_utf8` (~40% faster)
[#4918](https://github.com/apache/datafusion-comet/pull/4918) (andygrove)
+- perf: dedupe Iceberg residuals and delete files in native scan serde
[#4982](https://github.com/apache/datafusion-comet/pull/4982) (mbutrovich)
+- perf: optimize `spark_ceil` (3x faster)
[#4926](https://github.com/apache/datafusion-comet/pull/4926) (andygrove)
+- perf: bypass shuffle BatchCoalescer for already-sized batches
[#5003](https://github.com/apache/datafusion-comet/pull/5003) (andygrove)
+- perf: encode shuffle IPC schema once per writer instead of per block
[#5006](https://github.com/apache/datafusion-comet/pull/5006) (andygrove)
+- refactor: drop redundant concat layer in single-partition shuffle
[#5004](https://github.com/apache/datafusion-comet/pull/5004) (andygrove)
+- fix: make native cast from float/double to decimal compatible with Spark
[#5136](https://github.com/apache/datafusion-comet/pull/5136) (andygrove)
+- perf: intern QueryContext SQL text into a per-plan pool (up to 20x smaller
serialized plans for TPC-DS)
[#5204](https://github.com/apache/datafusion-comet/pull/5204) (andygrove)
+- perf: avoid rebuilding untouched operators in
PlanDataInjector.injectPlanData
[#5220](https://github.com/apache/datafusion-comet/pull/5220) (andygrove)
+
+**Implemented enhancements:**
+
+- feat: surface native parquet read failures as FAILED_READ_FILE
[#4536](https://github.com/apache/datafusion-comet/pull/4536) (schenksj)
+- feat: opt concat into codegen dispatch for non-UTF8_BINARY collations
[#4640](https://github.com/apache/datafusion-comet/pull/4640) (andygrove)
+- feat: opt sort_array into codegen dispatch under strict floating-point mode
[#4637](https://github.com/apache/datafusion-comet/pull/4637) (andygrove)
+- feat: support MapType input for ElementAt
[#4697](https://github.com/apache/datafusion-comet/pull/4697) (0lai0)
+- feat: extend native windows support
[#4209](https://github.com/apache/datafusion-comet/pull/4209) (comphead)
+- feat: add array_prepend support
[#4716](https://github.com/apache/datafusion-comet/pull/4716) (andygrove)
+- feat: surface native opt-in expressions as compatible-by-default with a
COMET-INFO plan hint
[#4721](https://github.com/apache/datafusion-comet/pull/4721) (andygrove)
+- feat: support StringSplitSQL for split_part
[#4592](https://github.com/apache/datafusion-comet/pull/4592)
(michaelmitchell-bit)
+- feat: support exact percentile and median aggregates natively
[#4542](https://github.com/apache/datafusion-comet/pull/4542) (andygrove)
+- feat: support TimestampNTZ inputs natively for hour/minute/second
[#4753](https://github.com/apache/datafusion-comet/pull/4753) (andygrove)
+- feat: add unsupported metadata column names to fallback reasons
[#4758](https://github.com/apache/datafusion-comet/pull/4758) (hsiang-c)
+- feat: Add support for `base64`
[#4778](https://github.com/apache/datafusion-comet/pull/4778) (andygrove)
+- feat: support interval types and make_ym_interval / make_dt_interval
[#4541](https://github.com/apache/datafusion-comet/pull/4541) (andygrove)
+- feat: support PreciseTimestampConversion for native batch time-window
grouping [#4784](https://github.com/apache/datafusion-comet/pull/4784)
(andygrove)
+- feat: support Azure authentication for native Parquet scan
[#4783](https://github.com/apache/datafusion-comet/pull/4783) (Dummk0pf)
+- feat: support shuffle array expression
[#4797](https://github.com/apache/datafusion-comet/pull/4797) (andygrove)
+- feat: rebalance associative bitwise/Add/Multiply chains to avoid protobuf
recursion limit [#4588](https://github.com/apache/datafusion-comet/pull/4588)
(schenksj)
+- feat: support grouping() and grouping_id() indicator functions
[#4815](https://github.com/apache/datafusion-comet/pull/4815) (andygrove)
+- feat: Stage based fallback
[#4519](https://github.com/apache/datafusion-comet/pull/4519) (karuppayya)
+- feat: release tokio runtime on driver/executor exit
[#4734](https://github.com/apache/datafusion-comet/pull/4734) (wForget)
+- feat: route Unsupported through codegen dispatch for opt-in serdes
[#4728](https://github.com/apache/datafusion-comet/pull/4728) (andygrove)
+- docs: comet docs design overhaul- phase 1
[#4353](https://github.com/apache/datafusion-comet/pull/4353) (pranamya123)
+- feat: name incompatible aggregate functions in mixed-execution fallback
reason [#4750](https://github.com/apache/datafusion-comet/pull/4750) (andygrove)
+- feat: implement native empty2null spark inner function
[#4683](https://github.com/apache/datafusion-comet/pull/4683) (kazantsev-maksim)
+- feat: enable mixed partial/final execution for sum and non-decimal avg
[#4861](https://github.com/apache/datafusion-comet/pull/4861) (andygrove)
+- feat: Add experimental support for accelerated PyArrow UDFs
[#4234](https://github.com/apache/datafusion-comet/pull/4234) (andygrove)
+- feat: support Iceberg 1.11, audit existing Iceberg diffs, bump iceberg-rust
dep, add run-iceberg-tests CI trigger
[#4840](https://github.com/apache/datafusion-comet/pull/4840) (mbutrovich)
+- feat: support approx_percentile / percentile_approx aggregate
[#4801](https://github.com/apache/datafusion-comet/pull/4801) (andygrove)
+- feat: remove constraint on array of nested elements
[#4714](https://github.com/apache/datafusion-comet/pull/4714) (hsiang-c)
+- feat: core SPI for contrib leaf scans (CometScanWithPlanData) [Delta contrib
split, part 1] [#4700](https://github.com/apache/datafusion-comet/pull/4700)
(schenksj)
+- feat: support gzip compression in native Parquet writes
[#4930](https://github.com/apache/datafusion-comet/pull/4930) (andygrove)
+- feat: support size() for MapType input
[#4580](https://github.com/apache/datafusion-comet/pull/4580) (marvelshan)
+- feat: hint at native pyarrow UDF path when the feature is disabled
[#4892](https://github.com/apache/datafusion-comet/pull/4892) (andygrove)
+- feat: Implement TimeType support - Infrastructure - shuffle
[#4398](https://github.com/apache/datafusion-comet/pull/4398) (YutaLin)
+- feat: fall back to Spark for collated predicate operands
[#4948](https://github.com/apache/datafusion-comet/pull/4948) (comphead)
+- feat: add withAlternative alias mechanism for CometConf
[#4979](https://github.com/apache/datafusion-comet/pull/4979) (andygrove)
+- feat: add spark.comet.shuffle.maxBufferBytes to cap native shuffle writer
memory [#4989](https://github.com/apache/datafusion-comet/pull/4989) (andygrove)
+- feat: support approx_count_distinct aggregate expression
[#4819](https://github.com/apache/datafusion-comet/pull/4819) (andygrove)
+- feat: Iceberg table format V3: native table decryption, fall back for other
V3 features [#4991](https://github.com/apache/datafusion-comet/pull/4991)
(mbutrovich)
+- feat: support multiply_dt_interval with codegen dispatch
[#4900](https://github.com/apache/datafusion-comet/pull/4900) (peterxcli)
+- feat: Support Spark levenshtein expression in native execution
[#4105](https://github.com/apache/datafusion-comet/pull/4105) (Myx778)
+- feat: native collect_list / array_agg aggregate
[#4720](https://github.com/apache/datafusion-comet/pull/4720) (andygrove)
+- feat: add CalendarIntervalType support
[#4898](https://github.com/apache/datafusion-comet/pull/4898) (peterxcli)
+- feat: native randstr implementation compatible with Spark
[#5035](https://github.com/apache/datafusion-comet/pull/5035) (andygrove)
+- feat: support interval codegen dispatch for nested values and native shuffle
[#4976](https://github.com/apache/datafusion-comet/pull/4976) (peterxcli)
+- feat: add codegen dispatch fallback for CometCast incompatible/unsupported
cases (including legacy config paths)
[#5079](https://github.com/apache/datafusion-comet/pull/5079) (comphead)
+- feat: support SampleExec natively for sampling without replacement
[#5110](https://github.com/apache/datafusion-comet/pull/5110) (andygrove)
+- feat: disable native columnar-to-row conversion by default
[#5114](https://github.com/apache/datafusion-comet/pull/5114) (andygrove)
+- feat: expose Comet version as spark.comet.version runtime config
[#5049](https://github.com/apache/datafusion-comet/pull/5049) (andygrove)
+- feat: native uuid() implementation compatible with Spark
[#5034](https://github.com/apache/datafusion-comet/pull/5034) (andygrove)
+- feat: support Iceberg metadata columns \_pos, \_spec, \_file, and
\_partition [#4752](https://github.com/apache/datafusion-comet/pull/4752)
(parthchandra)
+- feat: report native vs codegen-dispatch expression coverage in extended
explain [#5201](https://github.com/apache/datafusion-comet/pull/5201)
(andygrove)
+
+**Documentation updates:**
+
+- docs: update release_process for changelog
[#4668](https://github.com/apache/datafusion-comet/pull/4668) (mbutrovich)
+- docs: restore 0.13 user guide dropped in 0.17 release
[#4704](https://github.com/apache/datafusion-comet/pull/4704) (andygrove)
+- docs: fix prettier check error
[#4748](https://github.com/apache/datafusion-comet/pull/4748) (wForget)
+- docs: refresh expression audit notes for resolved correctness issues
[#4762](https://github.com/apache/datafusion-comet/pull/4762) (andygrove)
+- docs: mark try_avg and try_sum as natively supported
[#4776](https://github.com/apache/datafusion-comet/pull/4776) (andygrove)
+- docs: update Iceberg docs to clarify S3 storage credentials
[#4767](https://github.com/apache/datafusion-comet/pull/4767) (mbutrovich)
+- docs: render captioned sidebar sections for versioned user guides
[#4699](https://github.com/apache/datafusion-comet/pull/4699) (andygrove)
+- docs: triage bug vs enhancement and apply type labels consistently
[#4768](https://github.com/apache/datafusion-comet/pull/4768) (andygrove)
+- docs: mark make_dt_interval and make_ym_interval as supported
[#4790](https://github.com/apache/datafusion-comet/pull/4790) (andygrove)
+- docs: correct window function support status and limitations
[#4833](https://github.com/apache/datafusion-comet/pull/4833) (andygrove)
+- docs: minor docs site improvements
[#4855](https://github.com/apache/datafusion-comet/pull/4855) (andygrove)
+- docs: remove references to closed and fixed issues from compatibility guide
[#4856](https://github.com/apache/datafusion-comet/pull/4856) (andygrove)
+- docs: fix sidebar drawer mis-tap, missing aria-expanded, homepage permalink,
and ASF links page
[#4858](https://github.com/apache/datafusion-comet/pull/4858) (pranamya123)
+- docs: announce JDK 11 and Spark 3.4 deprecation for 1.1.0 removal
[#4857](https://github.com/apache/datafusion-comet/pull/4857) (andygrove)
+- docs: note invalid UTF-8 scan limitation and FFI import risk
[#4846](https://github.com/apache/datafusion-comet/pull/4846) (andygrove)
+- docs: update Gluten comparison with AWS Labs benchmark and Comet strengths
[#4873](https://github.com/apache/datafusion-comet/pull/4873) (andygrove)
+- docs: add scalar expression optimization guide, skill, and performance
audits [#4933](https://github.com/apache/datafusion-comet/pull/4933) (andygrove)
+- doc: Document scan tuning for `spark.sql.files.maxPartitionBytes`
[#4931](https://github.com/apache/datafusion-comet/pull/4931) (comphead)
+- docs: add 0.17.1 changelog
[#4961](https://github.com/apache/datafusion-comet/pull/4961) (mbutrovich)
+- docs: document Spark version adoption and support-lifetime policy
[#4977](https://github.com/apache/datafusion-comet/pull/4977) (andygrove)
+- docs: show implementation kind for each expression
[#5028](https://github.com/apache/datafusion-comet/pull/5028) (andygrove)
+- docs: add blog posts and talks page
[#5043](https://github.com/apache/datafusion-comet/pull/5043) (andygrove)
+- docs: avoid "not a blocker" comments in reviews for review skill
[#5057](https://github.com/apache/datafusion-comet/pull/5057) (mbutrovich)
+- docs: refresh stale issue references and normalize issue link format
[#5062](https://github.com/apache/datafusion-comet/pull/5062) (andygrove)
+- docs: fix references to configuration keys that do not exist
[#5063](https://github.com/apache/datafusion-comet/pull/5063) (andygrove)
+- docs: define post-1.0 versioning policy and legacy config process
[#5056](https://github.com/apache/datafusion-comet/pull/5056) (andygrove)
+- docs: expand tuning guide with performance and memory configs
[#4908](https://github.com/apache/datafusion-comet/pull/4908) (andygrove)
+- docs: document known correctness issues in the compatibility guide
[#5085](https://github.com/apache/datafusion-comet/pull/5085) (andygrove)
+- docs: update post 1.0.0 roadmap
[#5064](https://github.com/apache/datafusion-comet/pull/5064) (mbutrovich)
+- docs: drop compatibility notes for bugs that are now fixed
[#5154](https://github.com/apache/datafusion-comet/pull/5154) (andygrove)
+- docs: document ReusedExchange caveat and both CometSparkToColumnar names in
operator-count exclusions
[#5240](https://github.com/apache/datafusion-comet/pull/5240) (andygrove)
+
+**Other:**
+
+- chore: fix generate-release-docs.sh for per-Spark-version doc layout
[#4662](https://github.com/apache/datafusion-comet/pull/4662) (mbutrovich)
+- chore: add branch protection to release branches, update release_process.md
[#4665](https://github.com/apache/datafusion-comet/pull/4665) (mbutrovich)
+- chore: start 0.18.0 development
[#4664](https://github.com/apache/datafusion-comet/pull/4664) (mbutrovich)
+- test: cover nested complex casts
[#4608](https://github.com/apache/datafusion-comet/pull/4608) (manuzhang)
+- refactor: move string expression support checks to getSupportLevel
[#4676](https://github.com/apache/datafusion-comet/pull/4676) (andygrove)
+- refactor: move arithmetic and math support checks to getSupportLevel
[#4674](https://github.com/apache/datafusion-comet/pull/4674) (andygrove)
+- refactor: move aggregate expression support checks to getSupportLevel
[#4678](https://github.com/apache/datafusion-comet/pull/4678) (andygrove)
+- refactor: move array expression support checks to getSupportLevel
[#4677](https://github.com/apache/datafusion-comet/pull/4677) (andygrove)
+- chore: add array tests with NaN handling
[#4686](https://github.com/apache/datafusion-comet/pull/4686) (comphead)
+- chore: tweak CI execution memory params
[#4687](https://github.com/apache/datafusion-comet/pull/4687) (comphead)
+- chore: add ordering tests for `array_union`
[#4688](https://github.com/apache/datafusion-comet/pull/4688) (comphead)
+- chore: fix `ConstantFolding` rule exclusion for benchmarks
[#4689](https://github.com/apache/datafusion-comet/pull/4689) (comphead)
+- chore(deps): bump actions/checkout from 6 to 7
[#4690](https://github.com/apache/datafusion-comet/pull/4690) (dependabot[bot])
+- chore(deps): bump the all-other-cargo-deps group in /native with 2 updates
[#4691](https://github.com/apache/datafusion-comet/pull/4691) (dependabot[bot])
+- chore(deps): bump itertools from 0.14.0 to 0.15.0 in /native
[#4692](https://github.com/apache/datafusion-comet/pull/4692) (dependabot[bot])
+- chore: update documentation links for 0.17.0 release
[#4698](https://github.com/apache/datafusion-comet/pull/4698) (andygrove)
+- chore: add optional CI flow for parquet writes
[#4696](https://github.com/apache/datafusion-comet/pull/4696) (comphead)
+- deps: Upgrade to DataFusion 54.0.0
[#4062](https://github.com/apache/datafusion-comet/pull/4062) (andygrove)
+- chore: apply same SBT profile for Spark 4.0
[#4709](https://github.com/apache/datafusion-comet/pull/4709) (comphead)
+- refactor: remove map_contains_key serde
[#4703](https://github.com/apache/datafusion-comet/pull/4703) (manuzhang)
+- chore: Spark 4.0 restore `DEDICATED_JVM_SBT_TESTS`
[#4711](https://github.com/apache/datafusion-comet/pull/4711) (comphead)
+- refactor: share Spark 4.1+ shim sources to remove duplication
[#4710](https://github.com/apache/datafusion-comet/pull/4710) (andygrove)
+- chore(deps): bump object_store from 0.13.2 to 0.14.0 in /native
[#4737](https://github.com/apache/datafusion-comet/pull/4737) (dependabot[bot])
+- chore(deps): bump the all-other-cargo-deps group in /native with 3 updates
[#4736](https://github.com/apache/datafusion-comet/pull/4736) (dependabot[bot])
+- chore(deps): bump actions/cache from 5 to 6
[#4735](https://github.com/apache/datafusion-comet/pull/4735) (dependabot[bot])
+- deps: revert object_store to 0.13.2
[#4740](https://github.com/apache/datafusion-comet/pull/4740) (mbutrovich)
+- chore: fix `4.0.2` diff to handle `CometWindowsExec` properly
[#4743](https://github.com/apache/datafusion-comet/pull/4743) (comphead)
+- chore: surface DataFusion 54 PruningMetrics and Ratio in CometNativeScan
metrics [#4733](https://github.com/apache/datafusion-comet/pull/4733)
(mbutrovich)
+- refactor: gate CometUnaryMinus input types in getSupportLevel
[#4759](https://github.com/apache/datafusion-comet/pull/4759) (andygrove)
+- test: enable literal sha2 now that native engine handles scalar args
[#4773](https://github.com/apache/datafusion-comet/pull/4773) (andygrove)
+- ci: republish docs site when generated-doc sources change
[#4777](https://github.com/apache/datafusion-comet/pull/4777) (andygrove)
+- test: enable SparkSessionExtensionSuite with Comet
[#4772](https://github.com/apache/datafusion-comet/pull/4772) (andygrove)
+- chore: start 1.0.0 development
[#4794](https://github.com/apache/datafusion-comet/pull/4794) (andygrove)
+- chore(deps): bump rand from 0.10.1 to 0.10.2 in /native in the
all-other-cargo-deps group
[#4805](https://github.com/apache/datafusion-comet/pull/4805) (dependabot[bot])
+- deps: bump iceberg-rust version to a rev with `ArrowReader` fix
[#4826](https://github.com/apache/datafusion-comet/pull/4826) (sandugood)
+- chore: Use native impl of `soundex` function
[#4824](https://github.com/apache/datafusion-comet/pull/4824) (kazantsev-maksim)
+- refactor(shuffle): Introduce PartitionWriter interface to decouple shuffle
partitioning from storage
[#4779](https://github.com/apache/datafusion-comet/pull/4779) (wForget)
+- chore: fallback for `spark.sql.mapKeyDedupPolicy` == `LAST_WIN`
[#4863](https://github.com/apache/datafusion-comet/pull/4863) (comphead)
+- chore: remove `hdfs` Comet crate
[#4904](https://github.com/apache/datafusion-comet/pull/4904) (comphead)
+- test: cover invalid unhex inputs in SQL file tests
[#4890](https://github.com/apache/datafusion-comet/pull/4890) (andygrove)
+- chore: Codegen fallback explain for plan
[#4891](https://github.com/apache/datafusion-comet/pull/4891) (comphead)
+- test: add version check on an Iceberg test for Variant type
[#4935](https://github.com/apache/datafusion-comet/pull/4935) (mbutrovich)
+- chore: use Datafusion `substring`
[#4161](https://github.com/apache/datafusion-comet/pull/4161) (comphead)
+- chore: Support native writer `Overwrite` and `ErrorIfExists` modes
[#4946](https://github.com/apache/datafusion-comet/pull/4946) (comphead)
+- refactor: move AttributeReference and KnownFloatingPointNormalized
expression support checks to getSupportLevel
[#4745](https://github.com/apache/datafusion-comet/pull/4745) (peterxcli)
+- chore(deps): bump actions/setup-node from 6 to 7
[#4955](https://github.com/apache/datafusion-comet/pull/4955) (dependabot[bot])
+- chore: bump spark-4.2 profile to the released 4.2.0
[#4960](https://github.com/apache/datafusion-comet/pull/4960) (andygrove)
+- chore(deps): bump the all-other-cargo-deps group across 1 directory with 6
updates [#4956](https://github.com/apache/datafusion-comet/pull/4956)
(dependabot[bot])
+- chore(deps): bump lz4_flex from 0.13.1 to 0.14.0 in /native
[#4958](https://github.com/apache/datafusion-comet/pull/4958) (dependabot[bot])
+- chore: codeql dependabot fix
[#4962](https://github.com/apache/datafusion-comet/pull/4962) (comphead)
+- chore(deps): bump the codeql-actions group with 2 updates
[#4970](https://github.com/apache/datafusion-comet/pull/4970) (dependabot[bot])
+- test: promote `try_to_date`/`try_to_timestamp` SQL tests to native coverage
[#4973](https://github.com/apache/datafusion-comet/pull/4973) (u70b3)
+- chore: remove dead Parquet parallel-IO configs
[#4981](https://github.com/apache/datafusion-comet/pull/4981) (andygrove)
+- refactor: remove untested async columnar shuffle
[#4985](https://github.com/apache/datafusion-comet/pull/4985) (andygrove)
+- deps: bump to datafusion 54.1.0 and latest iceberg-rust
[#4996](https://github.com/apache/datafusion-comet/pull/4996) (mbutrovich)
+- chore: use DF `array_repeat` and `array_compact`
[#4741](https://github.com/apache/datafusion-comet/pull/4741) (comphead)
+- chore: remove legacy use.lazyMaterialization
[#4998](https://github.com/apache/datafusion-comet/pull/4998) (kazuyukitanimura)
+- chore(deps): bump actions/stale from 10.3.0 to 10.4.0
[#4879](https://github.com/apache/datafusion-comet/pull/4879) (dependabot[bot])
+- chore(deps): bump base64 from 0.22.1 to 0.23.0 in /native
[#5018](https://github.com/apache/datafusion-comet/pull/5018) (dependabot[bot])
+- chore(deps): bump actions/setup-python from 6 to 7
[#5017](https://github.com/apache/datafusion-comet/pull/5017) (dependabot[bot])
+- chore(deps): bump the codeql-actions group with 2 updates
[#5016](https://github.com/apache/datafusion-comet/pull/5016) (dependabot[bot])
+- chore(deps): bump actions/cache from 5 to 6
[#4882](https://github.com/apache/datafusion-comet/pull/4882) (dependabot[bot])
+- chore(deps): bump actions/checkout from 6 to 7
[#4881](https://github.com/apache/datafusion-comet/pull/4881) (dependabot[bot])
+- refactor: group orphan `spark.comet.explain.*` configs under one prefix
[#5026](https://github.com/apache/datafusion-comet/pull/5026) (andygrove)
+- refactor: unify shuffle configs under `spark.comet.shuffle.*` prefix
[#4986](https://github.com/apache/datafusion-comet/pull/4986) (andygrove)
+- test: add isolated and small-batch benchmarks for columnar-to-row conversion
[#5113](https://github.com/apache/datafusion-comet/pull/5113) (andygrove)
+- chore: stop enabling incompatible casts in plan stability suite
[#5139](https://github.com/apache/datafusion-comet/pull/5139) (andygrove)
+- chore: replace deprecated symbol literals with $"col" in tests
[#5168](https://github.com/apache/datafusion-comet/pull/5168) (andygrove)
+- test: cover ArrayExists three-valued logic config
[#5000](https://github.com/apache/datafusion-comet/pull/5000) (manuzhang)
+- chore(deps): bump actions/stale from 10.4.0 to 11.0.0
[#5163](https://github.com/apache/datafusion-comet/pull/5163) (dependabot[bot])
+- chore: remove Array -> Seq round trips flagged by the 2.13 copy deprecation
[#5170](https://github.com/apache/datafusion-comet/pull/5170) (andygrove)
+- chore: supply explicit empty argument lists for 2.13
[#5175](https://github.com/apache/datafusion-comet/pull/5175) (andygrove)
+- chore: clean up Scala compiler warnings
[#5141](https://github.com/apache/datafusion-comet/pull/5141) (andygrove)
+- chore: drop deprecated .toIterable from protobuf builder calls
[#5173](https://github.com/apache/datafusion-comet/pull/5173) (andygrove)
+- chore: bump Spark 3.5 to 3.5.9
[#5181](https://github.com/apache/datafusion-comet/pull/5181) (andygrove)
+- chore: bump Spark 4.1 to 4.1.3
[#5183](https://github.com/apache/datafusion-comet/pull/5183) (andygrove)
+- refactor: use arrow make_comparator for nested structural equality in
arrays_overlap and array_position
[#5176](https://github.com/apache/datafusion-comet/pull/5176) (peterxcli)
+- chore: bump Spark 4.0 to 4.0.4
[#5182](https://github.com/apache/datafusion-comet/pull/5182) (andygrove)
+- chore: remove unused spark.comet.exceptionOnDatetimeRebase config
[#5221](https://github.com/apache/datafusion-comet/pull/5221) (andygrove)
+- refactor: rename pyarrowUdf config to pyarrowUDF and fix stale config docs
[#5197](https://github.com/apache/datafusion-comet/pull/5197) (andygrove)
+- chore: cargo update for 1.0 release
[#5226](https://github.com/apache/datafusion-comet/pull/5226) (mbutrovich)
+
+## Credits
+
+Thank you to everyone who contributed to this release. Here is a breakdown of
commits (PRs merged) per contributor.
+
+```
+ 132 Andy Grove
+ 28 Matt Butrovich
+ 19 Oleks V
+ 18 dependabot[bot]
+ 8 Scott Schenkein
+ 7 Manu Zhang
+ 7 Peter Lee
+ 4 Zhen Wang
+ 2 ChenChen Lai
+ 2 Kazantsev Maksim
+ 2 Prames Maanikam
+ 2 Pranamya Vadlamani
+ 2 hsiang-c
+ 2 kid
+ 1 Bolin Lin
+ 1 KAZUYUKI TANIMURA
+ 1 Karuppayya
+ 1 Mitchell
+ 1 Myx778
+ 1 Parth Chandra
+ 1 Zaki
+ 1 alexander domenti
+ 1 michaelmitchell-bit
+```
+
+Thank you also to everyone who contributed in other ways such as filing
issues, reviewing PRs, and providing feedback on this release.
diff --git a/docs/source/changelog/index.md b/docs/source/changelog/index.md
index 609c76ed36..e79878f7e7 100644
--- a/docs/source/changelog/index.md
+++ b/docs/source/changelog/index.md
@@ -24,6 +24,7 @@ Per-release change logs for Apache DataFusion Comet.
```{toctree}
:maxdepth: 1
+1.0.0 <1.0.0>
0.17.1 <0.17.1>
0.17.0 <0.17.0>
0.16.0 <0.16.0>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
