This is an automated email from the ASF dual-hosted git repository.
gabotechs pushed a change to branch add-statistics-tests
in repository https://gitbox.apache.org/repos/asf/datafusion.git
omit fe5591032c use assert! instead of insta
omit 2ee489c5cb Add statistics integration tests
add 0f74dbfda1 FilterExec should remap indices of parent dynamic filters
(#20286)
add f9d37cf769 Clean up expression placement UDF usage in tests (#20272)
add 6b6f64d7fb chore(deps): bump the arrow-parquet group with 7 updates
(#20256)
add 17416bf92f Cleanup example metadata parsing utilities(#20251) (#20252)
add 4e2c0f1fea perf: improve performance of
`array_union`/`array_intersect` with batched row conversion (#20243)
add ecf3b502cf Add `StructArray` and `RunArray` benchmark tests to
`with_hashes` (#20182)
add a544b8fc08 fix: disable dynamic filter pushdown for non min/max
aggregates (#20279)
add aa9520e6e2 feat: Add `partition_stats()` for `EmptyExec` (#20203)
add 5fccac12cc Add protoc support for ArrowScanExecNode (#20280) (#20284)
add 85cdf53fab perf: various optimizations to eliminate branch
misprediction in hash_utils (#20168)
add 0401a4732f fix: Avoid integer overflow in substr() (#20199)
add 73bce15f29 fix: Fix scalar broadcast for to_timestamp() (#20224)
add 4d06c40d8d Improve ExternalSorter ResourcesExhausted Error Message
(#20226)
add 9fd84e796a Introduce ProjectionExprs::unproject_exprs/project_exprs
and improve docs (#20193)
add e4e8598be4 fix: Add integer check for bitwise coercion (#20241)
add 2cca3fad33 chore: Remove "extern crate criterion" in benches (#20299)
add af5f470465 Support pushing down empty projections into joins (#20191)
add 2e202fd6fe docs: fix typos and improve wording in README (#20301)
add a3d46510f1 chore: change width_bucket buckets parameter from i32 to
i64 (#20330)
add 51e5c98cf5 fix null handling for `nanvl` & implement fast path (#20205)
add f48dc72220 Remove the statistics() api in execution plan (#20319)
add 9c6a35ffea feat: add ExtractLeafExpressions optimizer rule for
get_field pushdown (#20117)
add 682da846b5 feat: Push limit into hash join (#20228)
add 4ad5c3dab7 perf: Optimize strpos() for ASCII-only inputs (#20295)
add f5a2ac3ec9 fix: percentile_cont interpolation causes NaN for f16 input
(#20208)
add e9bcfb449c perf: Optimize compare_element_to_list (#20323)
add 80485d7516 unify the prettier version adv(#20024) (#20311)
add bdad206a3d chore: Make memchr a workspace dependency (#20345)
add 7d217b1ec3 Reduce ExtractLeafExpressions optimizer overhead with fast
pre-scan (#20341)
add f471aaf275 feat(datafusion-cli): enhance CLI helper with default hint
(#20310)
add 3f38609277 perf: Optimize replace() fastpath by avoiding alloc (#20344)
add 98cc75361f Adds support for ANSI mode in negative function (#20189)
add 53b0ffb93d fix: validate inter-file ordering in eq_properties()
(#20329)
add d47bd599ee Support parent dynamic filters for more join types (#20192)
add b0566c57cc fix: update filter predicates for min/max aggregates only
if bounds change (#20380)
add 132b04369d Fix incorrect `SortExec` removal before `AggregateExec`
(option 2) (#20247)
add 6798dff0d7 Fix `try_shrink` not freeing back to pool (#20382)
add bd31db9d16 chore(deps): bump sysinfo from 0.38.1 to 0.38.2 (#20411)
add 5f0408c55d chore(deps): bump indicatif from 0.18.3 to 0.18.4 (#20410)
add a829b216da chore(deps): bump liblzma from 0.4.5 to 0.4.6 (#20409)
add 9acf387a21 chore(deps): bump aws-config from 1.8.13 to 1.8.14 (#20407)
add abe5d9862d chore(deps): bump tonic from 0.14.3 to 0.14.4 (#20406)
add 833635557b chore(deps): bump clap from 4.5.57 to 4.5.59 (#20404)
add 939621f790 chore(deps): bump sqllogictest from 0.29.0 to 0.29.1
(#20405)
add 2325aaad25 chore(deps): bump env_logger from 0.11.8 to 0.11.9 (#20402)
add 35588bc183 chore(deps): bump actions/stale from 10.1.1 to 10.2.0
(#20397)
add 4e43ac5a06 chore(deps): bump maturin from 1.11.5 to 1.12.2 in /docs
(#20400)
add 8172526134 chore(deps): bump uuid from 1.20.0 to 1.21.0 (#20401)
add ea51d9049a [Minor] Update object_store to 0.12.5 (#20378)
add 468b690d71 perf: optimize `array_distinct` with batched row conversion
(#20364)
add b6e4f95a5c chore(deps): bump syn from 2.0.114 to 2.0.116 (#20399)
add d692df0358 feat: Optimize hash util for `MapArray` (#20179)
add c0e80edc30 perf: Optimize scalar fast path of atan2 (#20336)
add 726d73067f chore(deps): bump taiki-e/install-action from 2.67.27 to
2.68.0 (#20398)
add 5bfcf9500c feat: Implement Spark `bitmap_bit_position` function
(#20275)
add 08c09db5e5 feat: support sqllogictest output coloring (#20368)
add c699361cb4 fix: Handle Utf8View and LargeUtf8 separators in concat_ws
(#20361)
add 4f4e814ce3 perf: Optimize concat()/concat_ws() UDFs (#20317)
add c3f080774c perf: Optimize translate() UDF for scalar inputs (#20305)
add 0022d8e503 chore: Cleanup returning null arrays (#20423)
add 0294a22cf9 perf: Optimize `array_has()` for scalar needle (#20374)
add 3157a2e131 chore: fix labeler for `datafusion-functions-nested`
(#20442)
add b7f6090874 perf: Optimize lpad, rpad for ASCII strings (#20278)
add ba267acf8f build: update Rust toolchain version from 1.92.0 to 1.93.0
in `rust-toolchain.toml` (#20309)
add ace9cd44b7 perf: Optimize trim UDFs for single-character trims (#20328)
add 1ee782f783 Migrate Python usage to uv workspace (#20414)
add 0f7a405b8c feat: support Spark-compatible `json_tuple` function
(#20412)
add a936d0de95 test: Extend Spark Array functions: `array_repeat `,
`shuffle` and `slice` test coverage (#20420)
add 7f99947390 chore: Cleanup "!is_valid(i)" -> "is_null(i)" (#20453)
add fc98d5c282 feat: Implement Spark `bitmap_bucket_number` function
(#20288)
add 1736fd2a40 refactor: Extract sort-merge join filter logic into
separate module (#19614)
add 0d63ced04a Implement FFI table provider factory (#20326)
add 42dd4279de bench: Add criterion benchmark for sort merge join (#20464)
add d03601547a chore: group minor dependencies into single PR (#20457)
add d2c5666f5a chore(deps): bump taiki-e/install-action from 2.68.0 to
2.68.6 (#20467)
add 626bc01b04 chore(deps): bump astral-sh/setup-uv from 6.1.0 to 7.3.0
(#20468)
add 043f908b60 chore(deps): bump the all-other-cargo-deps group with 6
updates (#20470)
add cfdd7c180c chore(deps): bump testcontainers-modules from 0.14.0 to
0.15.0 (#20471)
add f488a9071b perf: Optimize scalar fast path for `regexp_like` and
rejects g inside combined flags like ig (#20354)
add c1ad8636a0 [Minor] Use buffer_unordered (#20462)
add bfc012e638 bench: Add IN list benchmarks for non-constant list
expressions (#20444)
add 9660c98743 perf: Use zero-copy slice instead of take kernel in sort
merge join (#20463)
add 7815732f0f feat(memory-tracking): implement arrow_buffer::MemoryPool
for MemoryPool (#18928)
add 60457d0b0a Runs-on for more actions (#20274)
add 89a8576171 docs: Document that adding new optimizer rules are
expensive (#20348)
add ed0323a2bb feat: support `arrays_zip` function (#20440)
add df8f818b29 chore: Avoid build fails on MinIO rate limits (#20472)
add d303f5817f chore: Add end-to-end benchmark for array_agg, code cleanup
(#20496)
add b9328b9734 Upgrade to sqlparser 0.61.0 (#20177)
add 7602913b0f Switch to the latest Mac OS (#20510)
add b6d46a6382 perf: Optimize `initcap()` (#20352)
add d59cdfe999 Fix name tracker (#19856)
add 11ef486e6c Runs-on for extended CI checks (#20511)
add 3aa34b33f5 chore(deps): bump strum from 0.27.2 to 0.28.0 (#20520)
add 4c0a6531ca chore(deps): bump taiki-e/install-action from 2.68.6 to
2.68.8 (#20518)
add 6c793694e9 chore(deps): bump the all-other-cargo-deps group with 2
updates (#20519)
add 0dfa542201 fix: HashJoin panic with dictionary-encoded columns in
multi-key joins (#20441)
add 4a41587bdf Make `custom_file_casts` example schema nullable to allow
null `id` values during casting (#20486)
add a9c090141d Add support for FFI config extensions (#19469)
add 9c85ac608f perf: Fix quadratic behavior of `to_array_of_size` (#20459)
add 17d770d6e5 fix: handle out of range errors in DATE_BIN instead of
panicking (#20221)
add 670dbf481c fix: prevent duplicate alias collision with user-provided
__datafusion_extracted names (#20432)
add e71e7a39bf chore: Cleanup code to use `repeat_n` in a few places
(#20527)
add 932418b20c chore(deps): bump strum_macros from 0.27.2 to 0.28.0
(#20521)
add db5197b742 chore: Replace `matches!` on fieldless enums with `==`
(#20525)
add b16ad9badc fix: SortMergeJoin don't wait for all input before emitting
(#20482)
add fdd36d0d21 Update comments on OptimizerRule about function name
matching (#20346)
add e80694e369 Remove recursive const check in `simplify_const_expr`
(#20234)
add b8cebdde2a Fix incorrect regex pattern in regex_replace_posix_groups
(#19827)
add 34dad2ccee Cache `PlanProperties`, add fast-path for
`with_new_children` (#19792)
add 585bbf35d3 perf: Optimize `array_has_any()` with scalar arg (#20385)
add 387e20cc58 Improve `HashJoinExecBuilder` to save state from previous
fields (#20276)
add 2347306943 [Minor] Fix error messages for `shrink` and `try_shrink`
(#20422)
add d75fcb83e3 Fix physical expr adapter to resolve physical fields by
name, not column index (#20485)
add e937cadbcc [fix] Add type coercion from NULL to Interval to make
date_bin more postgres compatible (#20499)
add d7d646164d feat: Implement Spark `bin` function (#20479)
add e6849945bf fix: `cardinality()` of an empty array should be zero
(#20533)
add e894a03bea perf: Use Hashbrown for array_distinct (#20538)
add 3a970c58ff Clamp early aggregation emit to the sort boundary when
using partial group ordering (#20446)
add 33b86fe02e perf: Cache num_output_rows in sort merge join to avoid
O(n) recount (#20478)
add bcd42b090a fix: Unaccounted spill sort in row_hash (#20314)
add a026e7da2f perf: Optimize heap handling in TopK operator (#20556)
add d6fb3608b0 perf: Optimize `array_position` for scalar needle (#20532)
add e76f0eebe3 fix: IS NULL panic with invalid function without input
arguments (#20306)
add 3ab1301c53 fix: handle empty delimiter in split_part (closes #20503)
(#20542)
add a257c29c26 add redirect for old upgrading.html URL to fix broken
changelog links (#20582)
add a79e6e6b39 fix(substrait): Correctly parse field references in
subqueries (#20439)
add bc600b3090 Split `push_down_filter.slt` into standalone sqllogictest
files to reduce long-tail runtime (#20566)
add e583fe9d22 Add deterministic per-file timing summary to sqllogictest
runner (#20569)
add c63ca330d7 fix: increase ROUND decimal precision to prevent overflow
truncation (#19926)
add 7ef62b988d chore: Enable workspace lint for all workspace members
(#20577)
add 451c79fc00 fix: Fix `array_to_string` with columnar third arg (#20536)
add e567cb91f4 Fix serde of window lead/lag defaults (#20608)
add 5d8249ff16 fix: Fix and Refactor Spark `shuffle` function (#20484)
add acec058cb5 perf: Use Arrow vectorized eq kernel for IN list with
column references (#20528)
add 73fbd48070 Upgrade DataFusion to arrow-rs/parquet 58.0.0 /
`object_store` 0.13.0 (#19728)
add 3a23bb2531 perf: Optimize `array_agg()` using `GroupsAccumulator`
(#20504)
add 8df75c3f04 Document guidance on how to evaluate breaking API changes
(#20584)
add 8482e2e448 feat: support extension planner for `TableScan` (#20548)
add 6713439497 perf: Optimize `array_to_string()`, support more types
(#20553)
add 95de1bf410 Add metrics for parquet sink (#20307)
add 1f37a33ce5 Update DataFusion meetups page on docs (#20629)
add 93d177db4e Extend dynamic filter to joins that preserve probe side ON
(#20447)
add 0af9ff5998 Improve sqllogicteset speed by creating only a single large
file rather than 2 (#20586)
add 43584caf11 cli: Fix datafusion-cli hint edge cases (#20609)
add 02dae77420 Speedup sqllogictests by running long running tests first
(#20576)
add a5f490e4c3 Add `ExecutionPlan::apply_expressions()` (#20337)
add 12314c5564 perf: Optimize `array_to_string` to avoid a copy (#20639)
add 0bf376735e fix: make the `sql` feature truly optional (#20625)
add 657887dca9 Fix custom metric display (#20643)
add 476c200f00 refactor: Set expected runtime config in error message when
the used disk space during the spilling process has exceeded the allocation
limit (#20375)
add 3238a7e45b more families for the CI (#20663)
add 0ca9d6586a CI: Add CodeQL workflow for GitHub Actions security
scanning (#20636)
add d1a305821f perf: Apply logical regexp optimizations to Utf8View and
LargeUtf8 inputs (#20581)
add 88fa0dfce6 Add `Field` to `Expr::Cast` -- allow logical expressions to
express a cast to an extension type (#18136)
add d2df7a57c4 perf: Optimize `array_concat` using `MutableArrayData`
(#20620)
add 7f42c8c40e chore(deps): bump astral-sh/setup-uv from 7.3.0 to 7.3.1
(#20660)
add bcc52cd4c2 chore(deps): bump taiki-e/install-action from 2.68.8 to
2.68.16 (#20661)
add 2cbee474d7 fix: use try_shrink instead of shrink in try_resize (#20424)
add 79b5b24099 fix: Provide more generic API for the capacity limit
parsing (#20372)
add 3b028fe677 Improve formatting of datatypes (#20605)
add a1148409bf docs: Update `datafusion-cli` doc for
`top-memory-consumers` config (#20390)
add 92d0a5c527 Add explain plans for ClickBench queries (#20666)
add 10b5f22411 [main] Update version to 52.2.0 (#20573)
add 909608aa5f fix: Fix bug in `array_has` scalar path with sliced arrays
(#20677)
add 028e351595 Add files_processed and files_scanned metrics to
FileStreamMetrics (#20592)
add d025869526 fix: `HashJoin` panic with String dictionary keys (don't
flatten keys) (#20505)
add b092bd4117 Update releases links with releases in 2025-2026 (#20630)
add d412ba5d18 Speedup push_down_filter_regression.slt by using
uncompressed parquet (#20652)
add b0349ffefc feat: support nanosecond date_part (#20674)
add 0f093f405c Implement cardinality_effect for window execs and UnionExec
(#20321)
add 5d27860438 feat: parse `JsonAccess` as a binary operator, add
`Operator::Colon` (#20628)
add cbe5cb36ce ci: Harden labeler workflow, remove unnecessary checkout
from pull_request_target job (#20637)
add ddfc282008 Add tests for sqllogictest prioritization (#20656)
add 340ef606cf perf: Optimize `to_char` to allocate less, fix NULL
handling (#20635)
add 27c9cdaecc correct parquet leaf index mapping when schema contains
struct cols (#20698)
add 1f0232c995 Reattach parquet metadata cache after deserializing in
datafusion-proto (#20574)
add 848cd6387a Eliminate deterministic group by keys with deterministic
transformations (#20706)
add 46ac990878 Wire up with_new_state with DataSource (#20718)
add 29e849519e chore: Enable `assigning_clones` clippy lint (#20670)
add c919054241 perf: short-circuit and collect_bool for IN list with
column references (#20694)
add 953bdf4100 feat: Support Spark `array_contains` builtin function
(#20685)
add 00e36e8b72 fix: Return `probe_side.len()` for RightMark/Anti count(*)
queries (#20710)
add 13cebf8432 FFI_TableOptions are using default values only (#20721)
add 631c918039 perf: sort replace free()->try_grow() pattern with
try_resize() to reduce memory pool interactions (#20729)
add 03d17e8473 Improve documentation for `AggregateUdfImpl::simplify` and
`WindowUDFImpl::simplify` (#20712)
add a95da7028b doc: Add more context to `Precision` (#20713)
add dd988f6efc Fix test that's broken on Windows due to naive path
handling (#20692)
add b3976d6320 Fix DELETE/UPDATE filter extraction when predicates are
pushed down into TableScan (#19884)
add 678d1ad7f4 Minor: Add comment explaining rationale to avoid
dependencies on functions (#20667)
add 33c922f254 use linker optimization for extended sqllogictests (#20740)
add d72b0b8061 fix: preserve None projection semantics across FFI boundary
in ForeignTableProvider::scan (#20393)
add 02ce571910 Push even local limits past windows (#20752)
add 0ac434d0e5 Add case-heavy LEFT JOIN benchmark and debug timing/logging
for PushDownFilter hot paths (#20664)
add 5211a8b7c5 Fix repartition from dropping data when spilling (#20672)
add bfa0ea8056 Hash join buffering on probe side (#19761)
add 8fe926d387 test: Add `datafusion-cli` `fair` and `unbounded`
memory-pool test coverage (#20565)
add 92078d932d Copy limits before repartitions (#20736)
add 4dbb4498fc ser/de fetch in FilterExec (#20738)
add 1eb520675f feat: Integrate CastColumnExpr into PhysicalExprAdapter
(#20269)
add 37b9a466c4 feat: `partition_statistics()` for HashJoinExec (#20711)
add 15bc6bd5c8 feat: make DefaultLogicalExtensionCodec support
serialisation of buil… (#20638)
add bb421db72a Add tests for simplifying multiple aggregate expressions
(#20723)
add 33b9afa911 Allow SQL `TypePlanner` to plan SQL types as extension
types (#20676)
add b51edffc44 Update reverse UDF to emit utf8view when input is utf8view
(#20604)
add 9b3d6a4909 Make lower and upper emit Utf8View for Utf8View input
(#20616)
add 097f04c2ec fix(spark): handle divide-by-zero in Spark `mod`/`pmod`
with ANSI mode support (#20461)
add aca8c14478 Fix FilterExec converting Absent column stats to
Exact(NULL) (#20391)
add 44dfa7b8a5 Clean up date_part preimage implementation (#20350)
add 84a22eaa32 Wrap Arc to Statistics for `partition_statistics` API
(#20570)
add fd97799ddc Make Physical CastExpr Field-aware and unify cast semantics
across physical expressions (#20814)
add 75c7da5da3 Pass ConfigOptions to scalar UDFs via FFI (#20454)
add 39226c3f30 [datafusion-cli] Replace mutex with AtomicU64 for stream
duration tracking in instrumentedObjectStore (#20802)
add af79d14e7d Make translate emit Utf8View for Utf8View input (#20624)
add 23b88fbed7 Allow filters on struct fields to be pushed down into
Parquet scan (#20822)
add 6f86c8d878 Used constant with mapping instead of write! to display
scalar value bytes (#20719)
add 1f87930940 fix: sqllogictest cannot convert <subquery> to Substrait
(#19739)
add daa8f52c16 fix: interval analysis error when have two filterexec that
inner filter proves zero selectivity (#20743)
add fc514c2506 perf: Optimize set operations to avoid RowConverter
deserialization overhead (#20623)
add 31a4037da5 chore(deps): bump taiki-e/install-action from 2.68.16 to
2.68.25 (#20842)
add 64b5228790 chore(deps): bump github/codeql-action from 4.32.5 to
4.32.6 (#20843)
add 8e02b8ede5 chore: Ignore RUSTSEC-2024-0421 (#20850)
add 5af7361a98 fix: SanityCheckPlan error with window functions and NVL
filter (#20231)
add 9b7cddaa04 chore(deps): bump quinn-proto from 0.11.13 to 0.11.14
(#20859)
add 48199b9da3 Use `ParquetPushDecoder` in `ParquetOpener` (#20839)
add 86cb815e03 [Minor] Remove redundant ProjectionExec nodes in sort-based
plans (#20780)
add 2589fa8ac0 doc: Add documentation for pushing limit into plan (#20271)
add 4bac1cf878 impl ser/de for preserve_order in RepartitionExec (#20798)
add da05287c0f Fix FileStream scanning_total to include sync next-file
open time (#20627)
add 95a3dfd153 chore: Ignore RUSTSEC-2024-0014 (#20862)
add ed793f0de0 chore: clean up dependencies (#20861)
add 1efcbf57dd Add benchmark for struct field filter pushdown in Parquet
(#20829)
add 21cf60a9d3 Add Null Type Coercions for Placeholders (#20543)
add d68b8009e4 Minor: Deprecate unused `PartitionedFileStream` (#20869)
add f8fb5bd505 fix: Avoid unnecessary type casts in `concat_ws` (#20436)
add 981b5c37bf chore(deps): bump substrait from 0.62 to 0.63.0 (#20876)
add 129c58fa43 fix: Remove `!=0` check from
`supports_collect_by_thresholds` (#20730)
add 8d9b080882 [Minor] propagate distinct_count as inexact through unions
(#20846)
add 4b022c0e33 fix: do not recompute hash join exec properties if not
required (#20900)
add 6b7152387c [main] Bump to 52.3.0 and changelog (#20790) (#20849)
add 385d9dbe71 try to remove redundant alias in expression rewriter and
select (#20867)
add 8b412deaca fix: Optimize `!~ '.*'` case to `col IS NULL AND
Boolean(NULL)` instead of `Eq ""` (#20702)
add 57b275a8d3 feat: correct struct column names for `arrays_zip` return
type (#20886)
add fcb1c935c7 Fix duplicate group keys after hash aggregation spill
(#20724) (#20858)
add 422b54579f fix: Track metrics in hash joins with empty build sides
(#20810)
add 3c56e5d77f perf: Use batched row conversion for `array_has_any`,
`array_has_all` (#20588)
add b7e421325c Include .proto files in datafusion-proto-common
distribution (#20921)
add d2278a90b4 Check sqllogictests for any dangling config settings
(#17914) (#20838)
add 10d8bcba63 Add support for ListView in unnest (#20760)
add d09ff92def feat: Reduce allocations for aggregating `Statistics`
(#20768)
add 2c871b2f30 Project only accessed struct leaves in Parquet row filter
pushdown (#20854)
add 9c3c01aa54 refactor: Improve `SessionContext::parse_duration` API
(#20816)
add c74976f8be minor: Move PreparedAccessPlan to same module as
ParquetAccessPlan (#20929)
add 5db04b8503 chore(deps): bump pyjwt from 2.11.0 to 2.12.0 (#20938)
add 6d3a846483 Rewrite `SUM(expr + scalar)` --> `SUM(expr) +
scalar*COUNT(expr)` (#20749)
add 9b7d0924bc Add AGENTS.md / CLAUDE.md (#20939)
add 538a2019f8 perf: Optimize array set ops on sliced arrays (#20693)
add 1f59d327ba fix: dfbench respects DATAFUSION_RUNTIME_MEMORY_LIMIT env
var (#20631)
add ab2823475d Support `columns_sorted` in row_filters (#20497)
add 8609288ae0 Add --simulate-latency / SIMULATE_LATENCY option to
dfbench / ./bench.sh (#20954)
add b61aee77c8 Minor: make signatures of `SessionContext::register_*`
methods consistent (#20873)
add 3ece9ec2e4 test: add reproducer for Dictionary InList pushdown type
mismatch (#2… (#20960)
add c6f7145e31 Extract shared `ParquetReadPlan` for leaf column resolution
(#20913)
add 5d37babf36 chore: Remove usage of `paste` crate (#20946)
add 4166a6d60f perf: Optimize comparison on nested types (#20716)
add bd071be1dc feat: add `custom_string_literal_override` to unparser
Dialect trait (#20590)
add 26251bb37a Use exact distinct_count from statistics if exists for
`COUNT(DISTINCT column))` calculations (#20845)
add a7e09412a2 fix(spark): return input string for PATH/FILE on schemeless
URLs in `parse_url` (#20506)
add 972b890027 thin-ci (#20972)
add fa6706a8cd chore(deps): bump lz4_flex from 0.12.0 to 0.12.1 (#20973)
add 0dfcd97a37 Replace ahash with foldhash for faster hashing in
datafusion-common (#20958)
add 4c961256da Fix decimal log precision for non-power values (#20433)
add 9756146583 chore(deps): bump Swatinem/rust-cache from 2.8.2 to 2.9.1
(#20979)
add 1e99ed5cbd chore(deps): bump taiki-e/install-action from 2.68.25 to
2.68.34 (#20983)
add 13a39d7cd2 chore(deps): bump github/codeql-action from 4.32.6 to
4.33.0 (#20982)
add 8a95c4ccb1 chore(deps): bump astral-sh/setup-uv from 7.3.1 to 7.6.0
(#20981)
add 11b9693952 chore(deps): bump runs-on/action from 2.0.3 to 2.1.0
(#20980)
add 84a79e1be5 fix: InList Dictionary filter pushdown type mismatch
(#20962)
add 50b6bf866d fix: Run release verification with `--profile=ci` (#20987)
add fd145c4be6 [Minor] Update Cargo.lock, Fix Tokio minor breaking change
(#20978)
add 81423085b4 chore(deps): Revert "chore(deps): bump runs-on/action from
2.0.3 to 2.1.0 (#20980)" (#21002)
add e74e58f109 fix: move overflow guard before dense ratio in hash join to
prevent overflows (#20998)
add 6ab16cc7fd bug: fix `array_remove_*` with NULLS (#21013)
add cf0a182604 Simplify logic for memory pressure partial emit from
ordered group by (#20559)
add b7a3f53eb9 docs: in release email, be specific about changelog
location (#20975)
add a6a4df9996 Fix memory reservation starvation in sort-merge (#20642)
add b6b542e87b perf: Optimize `array_positions()` for scalar needle
(#20770)
add 22ca7201cd Add statistics integration tests
add 97efdb96b9 use assert! instead of insta
add 090157c887 Improve naming and doc comments
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (fe5591032c)
\
N -- N -- N refs/heads/add-statistics-tests (090157c887)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/dependabot.yml | 17 +
.github/workflows/audit.yml | 4 +-
.github/workflows/{stale.yml => codeql.yml} | 45 +-
.github/workflows/docs.yaml | 15 +-
.github/workflows/docs_pr.yaml | 15 +-
.github/workflows/extended.yml | 29 +-
.github/workflows/labeler.yml | 2 -
.github/workflows/labeler/labeler-config.yml | 2 +-
.github/workflows/rust.yml | 61 +-
.github/workflows/stale.yml | 2 +-
AGENTS.md | 34 +
CLAUDE.md | 1 +
Cargo.lock | 1212 +++++++-----
Cargo.toml | 119 +-
README.md | 16 +-
benchmarks/Cargo.toml | 4 +-
benchmarks/README.md | 6 +-
benchmarks/bench.sh | 186 +-
benchmarks/pyproject.toml | 6 +
benchmarks/requirements.txt | 18 -
benchmarks/src/clickbench.rs | 4 +-
benchmarks/src/h2o.rs | 4 +-
benchmarks/src/hj.rs | 4 +-
benchmarks/src/imdb/run.rs | 14 +-
benchmarks/src/nlj.rs | 4 +-
benchmarks/src/smj.rs | 4 +-
benchmarks/src/sort_tpch.rs | 4 +-
benchmarks/src/tpcds/run.rs | 10 +-
benchmarks/src/tpch/run.rs | 14 +-
benchmarks/src/util/latency_object_store.rs | 157 ++
benchmarks/src/util/mod.rs | 1 +
benchmarks/src/util/options.rs | 112 +-
ci/scripts/check_examples_docs.sh | 5 +-
ci/scripts/doc_prettier_check.sh | 10 +-
.../{rust_docs.sh => utils/tool_versions.sh} | 8 +-
datafusion-cli/Cargo.toml | 9 +-
datafusion-cli/src/exec.rs | 11 +-
datafusion-cli/src/functions.rs | 8 +-
datafusion-cli/src/helper.rs | 37 +-
datafusion-cli/src/highlighter.rs | 13 +-
datafusion-cli/src/main.rs | 12 +-
datafusion-cli/src/object_storage.rs | 25 +-
datafusion-cli/src/object_storage/instrumented.rs | 335 ++--
datafusion-cli/src/print_format.rs | 20 +-
datafusion-cli/src/print_options.rs | 16 +-
datafusion-cli/tests/cli_integration.rs | 106 +-
.../cli_top_memory_consumers@no_track.snap | 2 +-
.../snapshots/[email protected] | 2 +-
.../cli_top_memory_consumers@top3_default.snap | 2 +-
...ory_consumers_with_mem_pool_type@no_track.snap} | 4 +-
[email protected]} | 4 +-
.../[email protected] | 36 +
.../custom_data_source/adapter_serialization.rs | 2 +-
.../examples/custom_data_source/csv_json_opener.rs | 2 +-
.../custom_data_source/custom_datasource.rs | 23 +-
.../custom_data_source/custom_file_casts.rs | 6 +-
.../custom_data_source/default_column_values.rs | 4 +-
.../examples/data_io/json_shredding.rs | 4 +-
.../examples/data_io/parquet_advanced_index.rs | 8 +-
.../memory_pool_execution_plan.rs | 23 +-
.../execution_monitoring/memory_pool_tracking.rs | 3 +-
.../ffi/ffi_example_table_provider/Cargo.toml | 3 +
.../examples/ffi/ffi_module_interface/Cargo.toml | 3 +
.../examples/ffi/ffi_module_loader/Cargo.toml | 3 +
.../examples/proto/composed_extension_codec.rs | 23 +-
.../examples/relation_planner/table_sample.rs | 30 +-
datafusion-examples/src/utils/csv_to_parquet.rs | 3 +-
datafusion-examples/src/utils/datasets/mod.rs | 8 +-
.../src/utils/example_metadata/discover.rs | 32 +-
.../src/utils/example_metadata/model.rs | 11 +-
.../src/utils/example_metadata/parser.rs | 29 +-
.../src/utils/example_metadata/render.rs | 9 -
datafusion/catalog/src/memory/table.rs | 21 +-
datafusion/common/Cargo.toml | 11 +-
datafusion/common/benches/scalar_to_array.rs | 107 +
datafusion/common/benches/stats_merge.rs | 85 +
datafusion/common/benches/with_hashes.rs | 420 +++-
datafusion/common/src/alias.rs | 10 +
datafusion/common/src/config.rs | 71 +-
datafusion/common/src/error.rs | 141 +-
datafusion/common/src/file_options/mod.rs | 2 +-
.../common/src/file_options/parquet_writer.rs | 8 +-
datafusion/common/src/hash_utils.rs | 481 +++--
datafusion/common/src/join_type.rs | 29 +
datafusion/common/src/scalar/mod.rs | 121 +-
datafusion/common/src/stats.rs | 714 +++++--
datafusion/common/src/types/logical.rs | 120 +-
datafusion/common/src/types/native.rs | 124 +-
datafusion/common/src/utils/aggregate.rs | 149 ++
datafusion/common/src/utils/mod.rs | 1 +
datafusion/core/Cargo.toml | 15 +-
datafusion/core/benches/aggregate_query_sql.rs | 51 +-
datafusion/core/benches/csv_load.rs | 7 +-
datafusion/core/benches/data_utils/mod.rs | 119 +-
datafusion/core/benches/dataframe.rs | 7 +-
datafusion/core/benches/distinct_query_sql.rs | 8 +-
datafusion/core/benches/math_query_sql.rs | 7 +-
datafusion/core/benches/parquet_query_sql.rs | 4 +-
datafusion/core/benches/parquet_struct_query.rs | 4 +-
datafusion/core/benches/physical_plan.rs | 6 +-
datafusion/core/benches/push_down_filter.rs | 2 +-
.../core/benches/range_and_generate_series.rs | 6 +-
datafusion/core/benches/reset_plan_states.rs | 2 +
datafusion/core/benches/sort_limit_query_sql.rs | 7 +-
datafusion/core/benches/spm.rs | 5 +-
datafusion/core/benches/sql_planner.rs | 7 +-
datafusion/core/benches/sql_planner_extended.rs | 239 ++-
datafusion/core/benches/sql_query_with_io.rs | 2 +-
datafusion/core/benches/topk_repartition.rs | 90 +
datafusion/core/benches/window_query_sql.rs | 7 +-
datafusion/core/src/dataframe/parquet.rs | 133 ++
datafusion/core/src/datasource/file_format/csv.rs | 29 +-
.../core/src/datasource/file_format/parquet.rs | 38 +-
datafusion/core/src/datasource/listing/mod.rs | 7 +-
datafusion/core/src/datasource/mod.rs | 4 +-
.../core/src/datasource/physical_plan/csv.rs | 2 +-
datafusion/core/src/execution/context/mod.rs | 281 ++-
datafusion/core/src/physical_planner.rs | 469 ++++-
datafusion/core/src/test/object_store.rs | 65 +-
.../tests/catalog_listing/pruned_partition_list.rs | 2 +-
.../tests/custom_sources_cases/dml_planning.rs | 426 +++-
datafusion/core/tests/custom_sources_cases/mod.rs | 38 +-
.../provider_filter_pushdown.rs | 32 +-
.../core/tests/custom_sources_cases/statistics.rs | 41 +-
datafusion/core/tests/dataframe/mod.rs | 45 +-
.../core/tests/datasource/object_store_access.rs | 113 +-
datafusion/core/tests/execution/coop.rs | 6 +-
datafusion/core/tests/fuzz_cases/aggregate_fuzz.rs | 2 +-
datafusion/core/tests/fuzz_cases/once_exec.rs | 23 +-
datafusion/core/tests/fuzz_cases/pruning.rs | 4 +-
.../spilling_fuzz_in_memory_constrained_env.rs | 16 +-
.../core/tests/fuzz_cases/topk_filter_pushdown.rs | 2 +-
datafusion/core/tests/fuzz_cases/window_fuzz.rs | 2 +-
datafusion/core/tests/memory_limit/mod.rs | 15 +-
datafusion/core/tests/parquet/custom_reader.rs | 2 +-
datafusion/core/tests/parquet/expr_adapter.rs | 148 +-
.../core/tests/parquet/external_access_plan.rs | 2 +-
datafusion/core/tests/parquet/filter_pushdown.rs | 4 +-
datafusion/core/tests/parquet/mod.rs | 2 +-
datafusion/core/tests/parquet/page_pruning.rs | 620 +++---
datafusion/core/tests/parquet/row_group_pruning.rs | 642 +++---
.../physical_optimizer/aggregate_statistics.rs | 153 +-
.../physical_optimizer/enforce_distribution.rs | 31 +-
.../tests/physical_optimizer/filter_pushdown.rs | 617 +++++-
.../tests/physical_optimizer/join_selection.rs | 51 +-
.../tests/physical_optimizer/limit_pushdown.rs | 167 +-
.../physical_optimizer/partition_statistics.rs | 427 +++-
.../physical_optimizer/projection_pushdown.rs | 44 +
.../tests/physical_optimizer/pushdown_utils.rs | 31 +-
.../replace_with_order_preserving_variants.rs | 2 +-
.../core/tests/physical_optimizer/test_utils.rs | 52 +-
datafusion/core/tests/sql/path_partition.rs | 75 +-
datafusion/core/tests/sql/runtime_config.rs | 23 +-
datafusion/core/tests/sql/unparser.rs | 16 +-
datafusion/core/tests/statistics/mod.rs | 41 +-
datafusion/core/tests/statistics/tpcds.rs | 4 +-
.../core/tests/tracing/traceable_object_store.rs | 34 +-
.../core/tests/user_defined/insert_operation.rs | 37 +-
.../core/tests/user_defined/user_defined_plan.rs | 34 +-
.../user_defined/user_defined_scalar_functions.rs | 5 +-
datafusion/datasource-arrow/src/file_format.rs | 3 +-
datafusion/datasource-arrow/src/source.rs | 17 +-
datafusion/datasource-avro/src/file_format.rs | 2 +-
datafusion/datasource-avro/src/source.rs | 17 +-
datafusion/datasource-csv/src/file_format.rs | 4 +-
datafusion/datasource-csv/src/source.rs | 15 +
datafusion/datasource-json/Cargo.toml | 4 +-
datafusion/datasource-json/src/file_format.rs | 2 +-
datafusion/datasource-json/src/source.rs | 17 +-
datafusion/datasource-parquet/Cargo.toml | 6 +
.../benches/parquet_nested_filter_pushdown.rs | 12 +-
.../benches/parquet_struct_filter_pushdown.rs | 353 ++++
datafusion/datasource-parquet/src/access_plan.rs | 61 +-
datafusion/datasource-parquet/src/file_format.rs | 55 +-
datafusion/datasource-parquet/src/opener.rs | 366 ++--
datafusion/datasource-parquet/src/row_filter.rs | 978 ++++++++--
.../datasource-parquet/src/row_group_filter.rs | 3 +-
datafusion/datasource-parquet/src/sort.rs | 79 +-
datafusion/datasource-parquet/src/source.rs | 21 +
datafusion/datasource/Cargo.toml | 2 +-
datafusion/datasource/src/file.rs | 22 +
datafusion/datasource/src/file_scan_config.rs | 231 ++-
datafusion/datasource/src/file_stream.rs | 80 +-
datafusion/datasource/src/memory.rs | 29 +-
datafusion/datasource/src/mod.rs | 6 +-
datafusion/datasource/src/sink.rs | 22 +-
datafusion/datasource/src/source.rs | 91 +-
datafusion/datasource/src/statistics.rs | 3 +-
datafusion/datasource/src/test_util.rs | 9 +-
datafusion/datasource/src/url.rs | 42 +-
datafusion/execution/Cargo.toml | 6 +-
datafusion/execution/src/disk_manager.rs | 3 +-
datafusion/execution/src/memory_pool/arrow.rs | 142 ++
datafusion/execution/src/memory_pool/mod.rs | 85 +-
datafusion/expr-common/Cargo.toml | 4 +-
datafusion/expr-common/src/interval_arithmetic.rs | 38 +-
datafusion/expr-common/src/operator.rs | 17 +-
datafusion/expr-common/src/signature.rs | 197 +-
.../expr-common/src/type_coercion/aggregates.rs | 11 +-
datafusion/expr-common/src/type_coercion/binary.rs | 15 +-
.../src/type_coercion/binary/tests/arithmetic.rs | 47 +
datafusion/expr/Cargo.toml | 1 -
datafusion/expr/src/expr.rs | 66 +-
datafusion/expr/src/expr_rewriter/order_by.rs | 258 ++-
datafusion/expr/src/expr_schema.rs | 29 +-
datafusion/expr/src/function.rs | 40 +-
datafusion/expr/src/logical_plan/plan.rs | 56 +-
datafusion/expr/src/planner.rs | 18 +
datafusion/expr/src/test/function_stub.rs | 2 -
datafusion/expr/src/tree_node.rs | 8 +-
datafusion/expr/src/type_coercion/functions.rs | 59 +-
datafusion/expr/src/udaf.rs | 132 +-
datafusion/expr/src/udf.rs | 4 +-
datafusion/expr/src/udwf.rs | 27 +-
datafusion/expr/src/utils.rs | 100 +
datafusion/ffi/src/config/extension_options.rs | 288 +++
datafusion/ffi/src/config/mod.rs | 169 ++
datafusion/ffi/src/execution_plan.rs | 53 +-
datafusion/ffi/src/lib.rs | 2 +
datafusion/ffi/src/session/config.rs | 39 +-
datafusion/ffi/src/session/mod.rs | 107 +-
datafusion/ffi/src/table_provider.rs | 77 +-
datafusion/ffi/src/table_provider_factory.rs | 429 +++++
datafusion/ffi/src/tests/async_provider.rs | 25 +-
datafusion/ffi/src/tests/config.rs | 51 +
datafusion/ffi/src/tests/mod.rs | 28 +-
datafusion/ffi/src/tests/table_provider_factory.rs | 58 +
datafusion/ffi/src/tests/udf_udaf_udwf.rs | 44 +-
datafusion/ffi/src/udf/mod.rs | 39 +-
datafusion/ffi/tests/ffi_config.rs | 124 ++
datafusion/ffi/tests/ffi_integration.rs | 46 +-
datafusion/ffi/tests/ffi_udf.rs | 48 +-
datafusion/functions-aggregate-common/Cargo.toml | 1 -
.../benches/accumulate.rs | 2 -
.../src/aggregate/count_distinct/native.rs | 2 +-
.../src/aggregate/groups_accumulator/nulls.rs | 4 +-
datafusion/functions-aggregate-common/src/utils.rs | 2 +-
datafusion/functions-aggregate/Cargo.toml | 4 +-
datafusion/functions-aggregate/src/array_agg.rs | 718 ++++++-
.../functions-aggregate/src/bit_and_or_xor.rs | 2 +-
datafusion/functions-aggregate/src/count.rs | 51 +-
datafusion/functions-aggregate/src/hyperloglog.rs | 11 +-
datafusion/functions-aggregate/src/macros.rs | 2 -
.../functions-aggregate/src/percentile_cont.rs | 93 +-
datafusion/functions-aggregate/src/regr.rs | 2 +-
datafusion/functions-aggregate/src/sum.rs | 52 +-
datafusion/functions-nested/Cargo.toml | 19 +-
.../functions-nested/benches/array_concat.rs | 94 +
.../functions-nested/benches/array_expression.rs | 6 +-
datafusion/functions-nested/benches/array_has.rs | 856 +++++---
.../functions-nested/benches/array_position.rs | 344 ++++
.../functions-nested/benches/array_remove.rs | 7 +-
.../functions-nested/benches/array_repeat.rs | 7 +-
.../functions-nested/benches/array_reverse.rs | 6 +-
.../functions-nested/benches/array_set_ops.rs | 389 ++++
datafusion/functions-nested/benches/array_slice.rs | 2 -
.../functions-nested/benches/array_to_string.rs | 188 ++
datafusion/functions-nested/benches/map.rs | 2 -
datafusion/functions-nested/src/array_has.rs | 612 +++++-
datafusion/functions-nested/src/arrays_zip.rs | 336 ++++
datafusion/functions-nested/src/cardinality.rs | 13 +-
datafusion/functions-nested/src/concat.rs | 101 +-
datafusion/functions-nested/src/except.rs | 119 +-
datafusion/functions-nested/src/lib.rs | 3 +
datafusion/functions-nested/src/macros.rs | 6 -
datafusion/functions-nested/src/position.rs | 560 +++++-
datafusion/functions-nested/src/range.rs | 25 +-
datafusion/functions-nested/src/remove.rs | 2 +-
datafusion/functions-nested/src/set_ops.rs | 418 +++-
datafusion/functions-nested/src/string.rs | 450 ++---
datafusion/functions-nested/src/utils.rs | 5 +-
datafusion/functions-table/Cargo.toml | 1 -
datafusion/functions-table/src/lib.rs | 34 +-
datafusion/functions-window/Cargo.toml | 1 -
datafusion/functions-window/src/cume_dist.rs | 1 +
datafusion/functions-window/src/lead_lag.rs | 2 +
datafusion/functions-window/src/macros.rs | 60 +-
datafusion/functions-window/src/nth_value.rs | 5 +-
datafusion/functions-window/src/ntile.rs | 1 +
datafusion/functions-window/src/rank.rs | 3 +
datafusion/functions-window/src/row_number.rs | 1 +
datafusion/functions/Cargo.toml | 11 +
datafusion/functions/benches/ascii.rs | 1 -
datafusion/functions/benches/{cot.rs => atan2.rs} | 79 +-
datafusion/functions/benches/character_length.rs | 2 -
datafusion/functions/benches/chr.rs | 2 -
datafusion/functions/benches/contains.rs | 2 -
datafusion/functions/benches/cot.rs | 2 -
datafusion/functions/benches/crypto.rs | 2 -
datafusion/functions/benches/date_bin.rs | 2 -
datafusion/functions/benches/date_trunc.rs | 2 -
datafusion/functions/benches/encoding.rs | 2 -
datafusion/functions/benches/ends_with.rs | 2 -
datafusion/functions/benches/factorial.rs | 2 -
datafusion/functions/benches/find_in_set.rs | 2 -
datafusion/functions/benches/floor_ceil.rs | 2 -
datafusion/functions/benches/gcd.rs | 2 -
datafusion/functions/benches/initcap.rs | 110 +-
datafusion/functions/benches/isnan.rs | 2 -
datafusion/functions/benches/iszero.rs | 2 -
datafusion/functions/benches/left_right.rs | 2 -
datafusion/functions/benches/levenshtein.rs | 2 -
datafusion/functions/benches/lower.rs | 2 -
datafusion/functions/benches/make_date.rs | 2 -
datafusion/functions/benches/nanvl.rs | 114 ++
datafusion/functions/benches/nullif.rs | 2 -
datafusion/functions/benches/pad.rs | 184 +-
datafusion/functions/benches/random.rs | 2 -
datafusion/functions/benches/regexp_count.rs | 2 -
datafusion/functions/benches/regx.rs | 42 +-
datafusion/functions/benches/repeat.rs | 2 -
datafusion/functions/benches/replace.rs | 2 -
datafusion/functions/benches/reverse.rs | 1 -
datafusion/functions/benches/round.rs | 2 -
datafusion/functions/benches/signum.rs | 2 -
datafusion/functions/benches/split_part.rs | 2 -
datafusion/functions/benches/starts_with.rs | 2 -
datafusion/functions/benches/strpos.rs | 11 +-
datafusion/functions/benches/substr.rs | 2 -
datafusion/functions/benches/substr_index.rs | 2 -
datafusion/functions/benches/to_char.rs | 63 +-
datafusion/functions/benches/to_hex.rs | 2 -
datafusion/functions/benches/to_timestamp.rs | 2 -
datafusion/functions/benches/translate.rs | 50 +-
datafusion/functions/benches/trim.rs | 134 +-
datafusion/functions/benches/trunc.rs | 2 -
datafusion/functions/benches/upper.rs | 2 -
datafusion/functions/benches/uuid.rs | 2 -
datafusion/functions/src/core/arrow_cast.rs | 8 +-
datafusion/functions/src/core/getfield.rs | 2 +-
datafusion/functions/src/datetime/date_bin.rs | 291 ++-
datafusion/functions/src/datetime/date_part.rs | 95 +-
datafusion/functions/src/datetime/to_char.rs | 214 +-
datafusion/functions/src/datetime/to_timestamp.rs | 71 +-
datafusion/functions/src/macros.rs | 102 +-
datafusion/functions/src/math/gcd.rs | 7 +-
datafusion/functions/src/math/log.rs | 107 +-
datafusion/functions/src/math/nanvl.rs | 96 +-
datafusion/functions/src/math/round.rs | 472 ++++-
datafusion/functions/src/regex/regexplike.rs | 239 ++-
datafusion/functions/src/regex/regexpreplace.rs | 48 +-
datafusion/functions/src/string/btrim.rs | 6 +-
datafusion/functions/src/string/common.rs | 92 +-
datafusion/functions/src/string/concat.rs | 57 +-
datafusion/functions/src/string/concat_ws.rs | 536 +++++-
datafusion/functions/src/string/lower.rs | 25 +-
datafusion/functions/src/string/ltrim.rs | 6 +-
datafusion/functions/src/string/replace.rs | 20 +-
datafusion/functions/src/string/rtrim.rs | 6 +-
datafusion/functions/src/string/split_part.rs | 130 +-
datafusion/functions/src/string/upper.rs | 25 +-
datafusion/functions/src/strings.rs | 35 +-
datafusion/functions/src/unicode/find_in_set.rs | 26 +-
datafusion/functions/src/unicode/initcap.rs | 197 +-
datafusion/functions/src/unicode/lpad.rs | 104 +-
datafusion/functions/src/unicode/reverse.rs | 64 +-
datafusion/functions/src/unicode/rpad.rs | 112 +-
datafusion/functions/src/unicode/strpos.rs | 68 +-
datafusion/functions/src/unicode/substr.rs | 29 +-
datafusion/functions/src/unicode/translate.rs | 282 ++-
datafusion/macros/Cargo.toml | 2 +-
datafusion/optimizer/Cargo.toml | 8 +
datafusion/optimizer/src/analyzer/type_coercion.rs | 6 +-
.../src/decorrelate_predicate_subquery.rs | 2 +-
.../optimizer/src/eliminate_group_by_constant.rs | 111 +-
datafusion/optimizer/src/eliminate_outer_join.rs | 4 +-
.../optimizer/src/extract_leaf_expressions.rs | 2034 +++++++++++++++++---
.../optimizer/src/optimize_projections/mod.rs | 14 +-
datafusion/optimizer/src/optimizer.rs | 58 +-
datafusion/optimizer/src/push_down_filter.rs | 67 +-
.../src/simplify_expressions/expr_simplifier.rs | 81 +-
.../src/simplify_expressions/linear_aggregates.rs | 229 +++
.../optimizer/src/simplify_expressions/mod.rs | 1 +
.../optimizer/src/simplify_expressions/regex.rs | 132 +-
.../src/simplify_expressions/simplify_exprs.rs | 210 +-
datafusion/optimizer/src/test/udfs.rs | 20 +-
datafusion/physical-expr-adapter/Cargo.toml | 3 +
.../physical-expr-adapter/src/schema_rewriter.rs | 266 ++-
datafusion/physical-expr-common/Cargo.toml | 9 +-
.../physical-expr-common/benches/compare_nested.rs | 74 +
datafusion/physical-expr-common/src/binary_map.rs | 4 +-
.../physical-expr-common/src/binary_view_map.rs | 6 +-
datafusion/physical-expr-common/src/datum.rs | 12 +-
.../physical-expr-common/src/metrics/value.rs | 10 +-
datafusion/physical-expr/Cargo.toml | 2 -
datafusion/physical-expr/benches/in_list.rs | 166 ++
.../src/equivalence/properties/dependency.rs | 40 +-
.../src/equivalence/properties/mod.rs | 62 +-
datafusion/physical-expr/src/expressions/binary.rs | 2 +-
datafusion/physical-expr/src/expressions/case.rs | 2 +-
datafusion/physical-expr/src/expressions/cast.rs | 264 ++-
.../physical-expr/src/expressions/cast_column.rs | 8 +
.../physical-expr/src/expressions/in_list.rs | 674 ++++++-
.../physical-expr/src/expressions/negative.rs | 19 +-
.../physical-expr/src/expressions/try_cast.rs | 4 +-
datafusion/physical-expr/src/partitioning.rs | 2 +-
datafusion/physical-expr/src/planner.rs | 89 +-
datafusion/physical-expr/src/projection.rs | 115 +-
.../src/simplifier/const_evaluator.rs | 102 +-
datafusion/physical-expr/src/simplifier/mod.rs | 9 +-
datafusion/physical-expr/src/simplifier/not.rs | 4 +
datafusion/physical-optimizer/Cargo.toml | 1 +
.../src/combine_partial_final_agg.rs | 2 +-
.../physical-optimizer/src/enforce_distribution.rs | 85 +-
.../src/enforce_sorting/sort_pushdown.rs | 74 +
datafusion/physical-optimizer/src/ensure_coop.rs | 16 +-
.../physical-optimizer/src/hash_join_buffering.rs | 103 +
.../physical-optimizer/src/join_selection.rs | 21 +-
datafusion/physical-optimizer/src/lib.rs | 2 +
.../physical-optimizer/src/limit_pushdown.rs | 41 +
.../src/limit_pushdown_past_window.rs | 119 +-
datafusion/physical-optimizer/src/optimizer.rs | 17 +
.../physical-optimizer/src/output_requirements.rs | 46 +-
.../physical-optimizer/src/topk_repartition.rs | 368 ++++
datafusion/physical-plan/Cargo.toml | 6 +-
.../physical-plan/benches/sort_merge_join.rs | 204 ++
.../aggregates/group_values/multi_group_by/mod.rs | 2 +-
.../src/aggregates/group_values/row.rs | 2 +-
.../group_values/single_group_by/primitive.rs | 3 +-
datafusion/physical-plan/src/aggregates/mod.rs | 249 ++-
.../physical-plan/src/aggregates/no_grouping.rs | 21 +-
.../physical-plan/src/aggregates/order/mod.rs | 85 +-
.../physical-plan/src/aggregates/row_hash.rs | 182 +-
.../src/aggregates/topk/hash_table.rs | 3 +-
.../physical-plan/src/aggregates/topk/heap.rs | 4 +-
datafusion/physical-plan/src/analyze.rs | 15 +-
datafusion/physical-plan/src/async_func.rs | 37 +-
datafusion/physical-plan/src/buffer.rs | 33 +-
datafusion/physical-plan/src/coalesce_batches.rs | 44 +-
.../physical-plan/src/coalesce_partitions.rs | 45 +-
datafusion/physical-plan/src/coop.rs | 34 +-
datafusion/physical-plan/src/display.rs | 40 +-
datafusion/physical-plan/src/empty.rs | 56 +-
datafusion/physical-plan/src/execution_plan.rs | 307 ++-
datafusion/physical-plan/src/explain.rs | 16 +-
datafusion/physical-plan/src/filter.rs | 381 +++-
datafusion/physical-plan/src/filter_pushdown.rs | 174 +-
datafusion/physical-plan/src/joins/cross_join.rs | 51 +-
.../physical-plan/src/joins/hash_join/exec.rs | 747 ++++---
.../src/joins/hash_join/inlist_builder.rs | 75 +-
.../src/joins/hash_join/partitioned_hash_eval.rs | 84 +-
.../physical-plan/src/joins/hash_join/stream.rs | 52 +-
.../physical-plan/src/joins/nested_loop_join.rs | 62 +-
.../src/joins/piecewise_merge_join/classic_join.rs | 2 +-
.../src/joins/piecewise_merge_join/exec.rs | 99 +-
.../src/joins/sort_merge_join/exec.rs | 62 +-
.../src/joins/sort_merge_join/filter.rs | 595 ++++++
.../physical-plan/src/joins/sort_merge_join/mod.rs | 1 +
.../src/joins/sort_merge_join/stream.rs | 660 ++-----
.../src/joins/sort_merge_join/tests.rs | 519 ++++-
.../physical-plan/src/joins/symmetric_hash_join.rs | 54 +-
datafusion/physical-plan/src/joins/utils.rs | 72 +-
datafusion/physical-plan/src/limit.rs | 100 +-
datafusion/physical-plan/src/memory.rs | 35 +-
datafusion/physical-plan/src/placeholder_row.rs | 27 +-
datafusion/physical-plan/src/projection.rs | 91 +-
datafusion/physical-plan/src/recursive_query.rs | 21 +-
datafusion/physical-plan/src/repartition/mod.rs | 63 +-
datafusion/physical-plan/src/sorts/builder.rs | 54 +-
datafusion/physical-plan/src/sorts/mod.rs | 2 +
.../physical-plan/src/sorts/multi_level_merge.rs | 48 +-
datafusion/physical-plan/src/sorts/partial_sort.rs | 47 +-
datafusion/physical-plan/src/sorts/sort.rs | 380 ++--
.../src/sorts/sort_preserving_merge.rs | 61 +-
datafusion/physical-plan/src/sorts/stream.rs | 164 +-
datafusion/physical-plan/src/spill/mod.rs | 14 +-
.../physical-plan/src/spill/spill_manager.rs | 51 +-
datafusion/physical-plan/src/spill/spill_pool.rs | 102 +-
datafusion/physical-plan/src/streaming.rs | 17 +-
datafusion/physical-plan/src/test.rs | 43 +-
datafusion/physical-plan/src/test/exec.rs | 228 ++-
datafusion/physical-plan/src/topk/mod.rs | 44 +-
datafusion/physical-plan/src/union.rs | 445 ++++-
datafusion/physical-plan/src/unnest.rs | 69 +-
.../src/windows/bounded_window_agg_exec.rs | 67 +-
datafusion/physical-plan/src/windows/mod.rs | 12 +
.../physical-plan/src/windows/window_agg_exec.rs | 101 +-
datafusion/physical-plan/src/work_table.rs | 26 +-
datafusion/proto-common/Cargo.toml | 3 -
datafusion/proto/proto/datafusion.proto | 33 +-
datafusion/proto/src/generated/pbjson.rs | 476 ++++-
datafusion/proto/src/generated/prost.rs | 83 +-
datafusion/proto/src/logical_plan/from_proto.rs | 20 +-
datafusion/proto/src/logical_plan/mod.rs | 103 +-
datafusion/proto/src/logical_plan/to_proto.rs | 12 +-
datafusion/proto/src/physical_plan/from_proto.rs | 7 +-
datafusion/proto/src/physical_plan/mod.rs | 72 +-
datafusion/proto/src/physical_plan/to_proto.rs | 9 +-
.../proto/tests/cases/roundtrip_logical_plan.rs | 186 ++
.../proto/tests/cases/roundtrip_physical_plan.rs | 175 +-
datafusion/pruning/Cargo.toml | 2 +-
datafusion/pruning/src/pruning_predicate.rs | 2 +-
datafusion/spark/Cargo.toml | 1 +
datafusion/spark/benches/char.rs | 2 -
datafusion/spark/benches/hex.rs | 2 -
datafusion/spark/benches/sha2.rs | 2 -
datafusion/spark/benches/slice.rs | 2 -
datafusion/spark/benches/space.rs | 2 -
datafusion/spark/benches/substring.rs | 2 -
datafusion/spark/benches/unhex.rs | 2 -
.../spark/src/function/array/array_contains.rs | 168 ++
datafusion/spark/src/function/array/mod.rs | 15 +-
datafusion/spark/src/function/array/shuffle.rs | 16 +-
.../src/function/bitmap/bitmap_bit_position.rs | 141 ++
.../src/function/bitmap/bitmap_bucket_number.rs | 141 ++
datafusion/spark/src/function/bitmap/mod.rs | 23 +-
datafusion/spark/src/function/datetime/last_day.rs | 6 +-
datafusion/spark/src/function/datetime/next_day.rs | 9 +-
datafusion/spark/src/function/hash/sha2.rs | 9 +-
datafusion/spark/src/function/json/json_tuple.rs | 244 +++
datafusion/spark/src/function/json/mod.rs | 17 +-
.../spark/src/function/map/map_from_arrays.rs | 4 +-
datafusion/spark/src/function/map/utils.rs | 4 +-
datafusion/spark/src/function/math/bin.rs | 109 ++
datafusion/spark/src/function/math/mod.rs | 8 +
datafusion/spark/src/function/math/modulus.rs | 144 +-
datafusion/spark/src/function/math/negative.rs | 430 +++--
datafusion/spark/src/function/math/width_bucket.rs | 104 +-
.../spark/src/function/string/format_string.rs | 2 +-
datafusion/spark/src/function/url/parse_url.rs | 126 +-
datafusion/sql/Cargo.toml | 1 -
datafusion/sql/src/expr/binary_op.rs | 3 +-
datafusion/sql/src/expr/function.rs | 4 +-
datafusion/sql/src/expr/mod.rs | 123 +-
datafusion/sql/src/planner.rs | 12 +-
datafusion/sql/src/select.rs | 18 +-
datafusion/sql/src/statement.rs | 128 +-
datafusion/sql/src/unparser/ast.rs | 7 +-
datafusion/sql/src/unparser/dialect.rs | 12 +
datafusion/sql/src/unparser/expr.rs | 252 ++-
datafusion/sql/src/utils.rs | 6 +-
datafusion/sql/tests/cases/plan_to_sql.rs | 38 +-
datafusion/sql/tests/common/mod.rs | 16 +-
datafusion/sql/tests/sql_integration.rs | 51 +-
datafusion/sqllogictest/Cargo.toml | 4 +-
datafusion/sqllogictest/README.md | 30 +
datafusion/sqllogictest/bin/sqllogictests.rs | 359 +++-
.../src/engines/datafusion_engine/normalize.rs | 2 +-
.../src/engines/datafusion_engine/runner.rs | 63 +-
datafusion/sqllogictest/src/lib.rs | 2 +
datafusion/sqllogictest/src/test_file.rs | 186 ++
datafusion/sqllogictest/test_files/aggregate.slt | 26 +-
.../test_files/aggregate_skip_partial.slt | 41 +-
.../test_files/aggregates_simplify.slt | 358 ++++
datafusion/sqllogictest/test_files/array.slt | 610 +++++-
.../sqllogictest/test_files/arrow_typeof.slt | 2 +-
datafusion/sqllogictest/test_files/binary.slt | 2 +-
datafusion/sqllogictest/test_files/clickbench.slt | 932 ++++++++-
datafusion/sqllogictest/test_files/copy.slt | 11 +
datafusion/sqllogictest/test_files/cte.slt | 2 +-
.../sqllogictest/test_files/date_bin_errors.slt | 60 +
.../test_files/datetime/arith_date_time.slt | 1 -
.../datetime/arith_timestamp_duration.slt | 2 +-
.../sqllogictest/test_files/datetime/date_part.slt | 92 +-
.../test_files/datetime/timestamps.slt | 73 +-
datafusion/sqllogictest/test_files/decimal.slt | 22 +-
.../test_files/dynamic_filter_pushdown_config.slt | 370 +++-
datafusion/sqllogictest/test_files/encoding.slt | 6 +-
datafusion/sqllogictest/test_files/explain.slt | 26 +-
.../sqllogictest/test_files/explain_tree.slt | 14 +-
datafusion/sqllogictest/test_files/expr.slt | 43 +-
.../test_files/filter_without_sort_exec.slt | 44 +
datafusion/sqllogictest/test_files/functions.slt | 68 +-
.../sqllogictest/test_files/information_schema.slt | 6 +
datafusion/sqllogictest/test_files/insert.slt | 2 +-
.../test_files/join_disable_repartition_joins.slt | 4 +-
.../test_files/join_limit_pushdown.slt | 269 +++
datafusion/sqllogictest/test_files/joins.slt | 226 ++-
datafusion/sqllogictest/test_files/limit.slt | 10 +-
.../test_files/limit_single_row_batches.slt | 2 +-
datafusion/sqllogictest/test_files/math.slt | 6 +-
datafusion/sqllogictest/test_files/nvl.slt | 2 +-
datafusion/sqllogictest/test_files/order.slt | 48 +
.../test_files/parquet_filter_pushdown.slt | 203 ++
.../test_files/parquet_sorted_statistics.slt | 2 +-
datafusion/sqllogictest/test_files/projection.slt | 13 +-
.../test_files/projection_pushdown.slt | 580 ++++--
.../sqllogictest/test_files/push_down_filter.slt | 744 -------
.../test_files/push_down_filter_outer_joins.slt | 264 +++
.../test_files/push_down_filter_parquet.slt | 188 ++
.../test_files/push_down_filter_regression.slt | 195 ++
.../test_files/push_down_filter_unnest.slt | 148 ++
datafusion/sqllogictest/test_files/scalar.slt | 82 +-
datafusion/sqllogictest/test_files/select.slt | 27 +-
.../sqllogictest/test_files/set_variable.slt | 33 +
.../sqllogictest/test_files/simplify_expr.slt | 23 +-
.../sqllogictest/test_files/sort_pushdown.slt | 748 +++++++
.../test_files/spark/array/array_contains.slt | 140 ++
.../test_files/spark/array/array_repeat.slt | 20 +
.../test_files/spark/array/shuffle.slt | 30 +
.../sqllogictest/test_files/spark/array/slice.slt | 15 +
.../spark/bitmap/bitmap_bit_position.slt | 112 ++
.../spark/bitmap/bitmap_bucket_number.slt | 122 ++
.../test_files/spark/collection/size.slt | 1 -
.../test_files/spark/datetime/make_interval.slt | 10 +-
.../test_files/spark/datetime/time_trunc.slt | 1 -
.../test_files/spark/datetime/trunc.slt | 1 -
.../test_files/spark/datetime/unix.slt | 8 +-
.../test_files/spark/json/json_tuple.slt | 154 ++
.../sqllogictest/test_files/spark/math/bin.slt | 79 +-
.../sqllogictest/test_files/spark/math/mod.slt | 29 +
.../test_files/spark/math/negative.slt | 52 +
.../sqllogictest/test_files/spark/math/pmod.slt | 22 +-
.../test_files/spark/string/base64.slt | 4 +-
.../test_files/spark/url/parse_url.slt | 133 ++
.../test_files/spark/url/try_parse_url.slt | 133 ++
.../test_files/string/string_literal.slt | 37 +
.../sqllogictest/test_files/string/string_view.slt | 14 +-
datafusion/sqllogictest/test_files/struct.slt | 2 +-
datafusion/sqllogictest/test_files/truncate.slt | 2 +-
.../sqllogictest/test_files/type_coercion.slt | 21 +
datafusion/sqllogictest/test_files/unnest.slt | 98 +-
datafusion/sqllogictest/test_files/update.slt | 63 +-
datafusion/sqllogictest/test_files/window.slt | 58 +-
.../sqllogictest/test_files/window_limits.slt | 7 +-
.../test_files/window_topk_pushdown.slt | 141 ++
datafusion/substrait/Cargo.toml | 3 +-
datafusion/substrait/src/extensions.rs | 8 +-
.../src/logical_plan/consumer/expr/cast.rs | 11 +-
.../logical_plan/consumer/expr/field_reference.rs | 56 +-
.../src/logical_plan/consumer/expr/literal.rs | 24 +-
.../src/logical_plan/consumer/expr/mod.rs | 8 +-
.../src/logical_plan/consumer/expr/subquery.rs | 44 +-
.../substrait/src/logical_plan/consumer/plan.rs | 5 +-
.../src/logical_plan/consumer/rel/exchange_rel.rs | 3 +-
.../src/logical_plan/consumer/rel/project_rel.rs | 15 +-
.../logical_plan/consumer/substrait_consumer.rs | 153 +-
.../substrait/src/logical_plan/consumer/types.rs | 20 +-
.../substrait/src/logical_plan/consumer/utils.rs | 224 ++-
.../src/logical_plan/producer/expr/cast.rs | 18 +-
.../logical_plan/producer/expr/field_reference.rs | 16 +
.../src/logical_plan/producer/expr/literal.rs | 2 +-
.../src/logical_plan/producer/expr/mod.rs | 12 +-
.../logical_plan/producer/expr/scalar_function.rs | 1 +
.../logical_plan/producer/expr/singular_or_list.rs | 22 +-
.../src/logical_plan/producer/expr/subquery.rs | 81 +-
.../substrait/src/logical_plan/producer/plan.rs | 4 -
.../logical_plan/producer/substrait_producer.rs | 30 +-
.../substrait/src/logical_plan/producer/types.rs | 44 +-
.../substrait/src/logical_plan/producer/utils.rs | 27 +-
.../substrait/tests/cases/consumer_integration.rs | 118 +-
datafusion/substrait/tests/cases/logical_plans.rs | 35 +-
.../tests/cases/roundtrip_logical_plan.rs | 230 ++-
.../nested_correlated_subquery.substrait.json | 265 +++
datafusion/substrait/tests/utils.rs | 1 +
datafusion/wasmtest/Cargo.toml | 2 +-
datafusion/wasmtest/src/lib.rs | 2 +-
dev/changelog/52.2.0.md | 47 +
dev/changelog/{49.0.2.md => 52.3.0.md} | 21 +-
dev/pyproject.toml | 5 +
dev/release/README.md | 6 +-
dev/release/create-tarball.sh | 2 +-
dev/release/verify-release-candidate.sh | 8 +-
dev/requirements.txt | 2 -
dev/update_arrow_deps.py | 2 +-
dev/update_config_docs.sh | 10 +-
dev/update_datafusion_versions.py | 2 +-
dev/update_function_docs.sh | 9 +-
docs/README.md | 15 +-
docs/pyproject.toml | 13 +
docs/requirements.txt | 24 -
docs/source/conf.py | 1 +
docs/source/contributor-guide/api-health.md | 71 +-
docs/source/contributor-guide/index.md | 28 +-
docs/source/contributor-guide/testing.md | 4 +-
docs/source/download.md | 2 +-
.../library-user-guide/custom-table-providers.md | 36 +-
docs/source/library-user-guide/extending-sql.md | 51 +-
docs/source/library-user-guide/upgrading/52.0.0.md | 2 +-
docs/source/library-user-guide/upgrading/53.0.0.md | 74 +
docs/source/library-user-guide/upgrading/54.0.0.md | 124 ++
docs/source/library-user-guide/upgrading/index.rst | 1 +
docs/source/user-guide/cli/usage.md | 4 +-
docs/source/user-guide/concepts-readings-events.md | 51 +-
docs/source/user-guide/configs.md | 5 +-
docs/source/user-guide/crate-configuration.md | 2 +-
docs/source/user-guide/example-usage.md | 2 +-
docs/source/user-guide/sql/scalar_functions.md | 81 +-
pyproject.toml | 2 +
rust-toolchain.toml | 2 +-
uv.lock | 1149 +++++++++++
681 files changed, 45407 insertions(+), 10421 deletions(-)
copy .github/workflows/{stale.yml => codeql.yml} (53%)
create mode 100644 AGENTS.md
create mode 120000 CLAUDE.md
create mode 100644 benchmarks/pyproject.toml
delete mode 100644 benchmarks/requirements.txt
create mode 100644 benchmarks/src/util/latency_object_store.rs
copy ci/scripts/{rust_docs.sh => utils/tool_versions.sh} (80%)
mode change 100755 => 100644
copy datafusion-cli/tests/snapshots/{cli_top_memory_consumers@no_track.snap =>
cli_top_memory_consumers_with_mem_pool_type@no_track.snap} (71%)
copy datafusion-cli/tests/snapshots/{[email protected] =>
[email protected]} (78%)
create mode 100644
datafusion-cli/tests/snapshots/[email protected]
create mode 100644 datafusion/common/benches/scalar_to_array.rs
create mode 100644 datafusion/common/benches/stats_merge.rs
create mode 100644 datafusion/common/src/utils/aggregate.rs
create mode 100644 datafusion/core/benches/topk_repartition.rs
create mode 100644
datafusion/datasource-parquet/benches/parquet_struct_filter_pushdown.rs
create mode 100644 datafusion/execution/src/memory_pool/arrow.rs
create mode 100644 datafusion/ffi/src/config/extension_options.rs
create mode 100644 datafusion/ffi/src/config/mod.rs
create mode 100644 datafusion/ffi/src/table_provider_factory.rs
create mode 100644 datafusion/ffi/src/tests/config.rs
create mode 100644 datafusion/ffi/src/tests/table_provider_factory.rs
create mode 100644 datafusion/ffi/tests/ffi_config.rs
create mode 100644 datafusion/functions-nested/benches/array_concat.rs
create mode 100644 datafusion/functions-nested/benches/array_position.rs
create mode 100644 datafusion/functions-nested/benches/array_set_ops.rs
create mode 100644 datafusion/functions-nested/benches/array_to_string.rs
create mode 100644 datafusion/functions-nested/src/arrays_zip.rs
copy datafusion/functions/benches/{cot.rs => atan2.rs} (63%)
create mode 100644 datafusion/functions/benches/nanvl.rs
create mode 100644
datafusion/optimizer/src/simplify_expressions/linear_aggregates.rs
create mode 100644 datafusion/physical-expr-common/benches/compare_nested.rs
create mode 100644 datafusion/physical-optimizer/src/hash_join_buffering.rs
create mode 100644 datafusion/physical-optimizer/src/topk_repartition.rs
create mode 100644 datafusion/physical-plan/benches/sort_merge_join.rs
create mode 100644 datafusion/physical-plan/src/joins/sort_merge_join/filter.rs
create mode 100644 datafusion/spark/src/function/array/array_contains.rs
create mode 100644 datafusion/spark/src/function/bitmap/bitmap_bit_position.rs
create mode 100644 datafusion/spark/src/function/bitmap/bitmap_bucket_number.rs
create mode 100644 datafusion/spark/src/function/json/json_tuple.rs
create mode 100644 datafusion/spark/src/function/math/bin.rs
create mode 100644 datafusion/sqllogictest/src/test_file.rs
create mode 100644 datafusion/sqllogictest/test_files/aggregates_simplify.slt
create mode 100644 datafusion/sqllogictest/test_files/date_bin_errors.slt
create mode 100644 datafusion/sqllogictest/test_files/join_limit_pushdown.slt
delete mode 100644 datafusion/sqllogictest/test_files/push_down_filter.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_outer_joins.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_parquet.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_regression.slt
create mode 100644
datafusion/sqllogictest/test_files/push_down_filter_unnest.slt
create mode 100644
datafusion/sqllogictest/test_files/spark/array/array_contains.slt
create mode 100644
datafusion/sqllogictest/test_files/spark/bitmap/bitmap_bit_position.slt
create mode 100644
datafusion/sqllogictest/test_files/spark/bitmap/bitmap_bucket_number.slt
create mode 100644 datafusion/sqllogictest/test_files/spark/json/json_tuple.slt
create mode 100644 datafusion/sqllogictest/test_files/window_topk_pushdown.slt
create mode 100644
datafusion/substrait/tests/testdata/test_plans/nested_correlated_subquery.substrait.json
create mode 100644 dev/changelog/52.2.0.md
copy dev/changelog/{49.0.2.md => 52.3.0.md} (51%)
create mode 100644 dev/pyproject.toml
delete mode 100644 dev/requirements.txt
create mode 100644 docs/pyproject.toml
delete mode 100644 docs/requirements.txt
create mode 100644 docs/source/library-user-guide/upgrading/54.0.0.md
create mode 100644 pyproject.toml
create mode 100644 uv.lock
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
