This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/pip/docs/sphinx-9.1.0
in repository https://gitbox.apache.org/repos/asf/datafusion-sandbox.git
discard 19d1f7b7d chore(deps): bump sphinx from 8.2.3 to 9.1.0 in /docs
add c2747eb10 feat: Support log for Decimal32 and Decimal64 (#18999)
add cbf33d1ee Fix regression for negative-scale decimal128 in log (#19315)
add d493f3d44 Add Decimal support to Ceil and Floor (#18979)
add 8550010bd Fix input handling for encoding functions & various
refactors (#18754)
add 75d2473ba Remove SchemaAdapter (#19345)
add 887aa9f8c fix: preserve ListFilesCache TTL when not set in config
(#19401)
add 91cfb6990 feat(proto): Add protobuf serialization for HashExpr (#19379)
add 2e3707e38 fix: projection for `CooperativeExec` and
`CoalesceBatchesExec` (#19400)
add b3d2cb622 Fix ORDER BY positional reference regression with aliased
aggregates (#19412)
add 5419ff590 feat: hash partitioning satisfies subset (#19304)
add 8cc8c11de Optimize muti-column grouping with StringView/ByteView
(option 2) - 25% faster (#19413)
add 6fa9c1ad1 Optimize hashing for StringView and ByteView (15-70% faster)
(#19374)
add eb30c19b3 Implement disk spilling for all grouping ordering modes in
GroupedHashAggregateStream (#19287)
add 32e6fe887 feat: update FFI TableProvider and ExecutionPlan to use FFI
Session and TaskContext (#19281)
add d8e68a404 refactor: add ParquetOpenerBuilder to reduce test code
duplication (#19405)
add 4249e4ecd bench: add `range_and_generate_series` (#19428)
add 65a6bc423 chore: use extend instead of manual loop in multi group by
(#19429)
add 2c3566ce8 doc: add example for cache factory (#19139)
add 1acaf7a9a chore(deps): bump taiki-e/install-action from 2.64.0 to
2.64.2 (#19399)
add 9fe9ec744 fix: spark crc32 custom nullability (#19271)
add 9a9e4dd5c Add recursive protection on planner's `create_physical_expr`
(#19299)
add d9d55cfc6 chore(deps): bump aws-config from 1.8.11 to 1.8.12 (#19453)
add edc693f99 chore(deps): bump log from 0.4.28 to 0.4.29 (#19452)
add c7f9fdf90 chore(deps): bump taiki-e/install-action from 2.64.2 to
2.65.1 (#19451)
add 8e95627d3 chore(deps): bump sphinx-reredirects from 1.0.0 to 1.1.0 in
/docs (#19455)
add 5fedb8423 chore(deps): bump insta from 1.44.3 to 1.45.0 (#19454)
add a886b9eb4 added support for negative scale for log decimal32/64 and
power (#19409)
add 1e591640d Remove core dependency from ffi (#19422)
add bb9a4a7ea bench: increase in_list benchmark coverage (#19443)
add 48f5d0b72 fix: Fix skip aggregate test to cover regression (#19461)
add d0d93117b fix: [19450]Added flush for tokio file(substrait) write
(#19456)
add 258e18cf8 Use SortMergeJoinExec name consistently in physical plan
outputs (#19246)
add d844f8687 Add:arrow_metadata() UDF (#19435)
add 4a1f69f9b Update date_bin to support Time32 and Time64 data types
(#19341)
add 72f174616 feat: Add decimal support for round (#19384)
add e6faacbf5 Fix panic during spill to disk in clickbench query (#19421)
add 677c543ae Optimize memory footprint of view arrays from
`ScalarValue::to_array_of_size` (#19441)
add 33ac70dd6 minor: refactoring of some `ScalarValue` code (#19439)
add 0bd880931 fix: csv schema_infer_max_records set to 0 return null
datatype (#19432)
add 902d3b32b fix: Add custom nullability for Spark LIKE function (#19218)
add 67b526a62 Refactor Spark crc32 & sha1 to remove unnecessary scalar
argument check (#19466)
add 47ddd5035 Add link to arrow-rs ticket in comments (#19479)
add a405d3fe4 Support nested field access in `get_field` with multiple
path arguments (#19389)
add d2830b6be chore(deps): bump taiki-e/install-action from 2.65.1 to
2.65.2 (#19474)
add 6ce237492 Improve plan_to_sql handling of empty projections with
dialect-specific SELECT list support (#19221)
add ef2c1a30e examples: replace sql_dialect with custom_sql_parser example
(#19383)
add 03904e1b2 Replace custom merge operator with arrow-rs implementation
(#19424)
add ea2e22c74 Implement nested recursive CTEs (#18956)
add e586ff532 fix: implement custom nullability for spark abs function
(#19395)
add 058bcb001 fix: custom nullability for format_string (#19173) (#19190)
add 62740802f Update `to_unixtime` udf function to support a consistent
set of argument types (#19442)
add ed7af0b12 Add: PI upper/lower bound f16 constants to ScalarValue
(#19497)
add 853273157 chore: enforce clippy::allow_attributes for datafusion-ffi
crate (#19480)
add ae35177df Add CI check to ensure examples are documented in README
(#19371)
add e5ca510dc perf: Improve performance of `to_hex` (> 2x) (#19503)
add d20c5d68f fix : snapshot to the modern multiline format (#19517)
add 5b90ceef1 perf: improve performance of string repeat (#19502)
add 134be4ce5 chore(deps): bump taiki-e/install-action from 2.65.2 to
2.65.3 (#19499)
add d825e5f39 docs : clarify unused test utility (#19508)
add bb4e0eca2 perf: Optimize `starts_with` and `ends_with` for scalar
arguments (#19516)
add 85c696df4 Date / time / interval arithmetic improvements (#19460)
add 8246631bf fix: Implement `reset_state` for `LazyMemoryExec` (#19362)
add 6ac7b898e Preserve ORDER BY in Unparser for projection -> order by
pattern (#19483)
add 9eddf473f fix: CteWorkTable: properly apply TableProvider::scan
projection argument (#18993)
add 496028454 fix: Median() integer overflow (#19509)
add 10db6b371 Redesign the try_reverse_output to support more cases
(#19446)
add a95c7fc2c feat: fix matching for named parameters with non-lowercase
signatures (#19378)
add 83ed19235 refactor: Spark `ascii` signature away from `user_defined`
(#19513)
add 36df145e9 feat: Add per-expression evaluation timing metrics to
ProjectionExec (#19447)
add 3aa0ab78b Fix: SparkAscii nullability to depend on input nullability
(#19531)
add 1dbf9a6df chore(deps): bump tracing from 0.1.41 to 0.1.43 (#19543)
add c37db42ae chore(deps): bump substrait from 0.62.0 to 0.62.2 (#19542)
add d7e5190f7 chore(deps): bump taiki-e/install-action from 2.65.3 to
2.65.6 (#19541)
add bd10f2744 minor: run all examples by default (#19506)
add 94709dc02 perf: improve performance of string replace (#19530)
add 43567b468 perf: improve performance of levenshtein by reusing cache
buffer (#19532)
add 3f0b3425c feat: Improve sort memory resilience (#19494)
add 7c50448f5 perf: improve performance of translate by reusing buffers
(#19533)
add 673d7c93a Refactor TopKHashTable to use HashTable API (#19464)
add f9cdfea7f docs: Improve config tables' readability (#19522)
add 8ac500bf0 Revert Spark Elt nullability change (#19510)
add 8469aa1dc minor: implement more arms for `get_data_types()` for
`NativeType` (#19449)
add 1d2b38959 perf: Optimize `contains` for scalar search arg (#19529)
add 13f38435a Introduce `TypeSignatureClass::Any` (#19485)
add a6fd5cc84 Upgrade hashbrown to 0.16 (#19554)
add a51e3a079 minor : add crypto function benchmark (#19539)
add 3420a2d4a chore(deps): bump taiki-e/install-action from 2.65.6 to
2.65.8 (#19559)
add 34addca6b bugfix: preserve schema metadata for record batch in FFI
(#19293)
add d13d89129 feat: Add DELETE/UPDATE hooks to TableProvider trait and to
MemTable implementation (#19142)
add 1704d1e74 refactor: extract the data generate out of aggregate_topk
benchmark (#19523)
add 9690f958e perf: improve performance of lpad/rpad by reusing buffers
(#19558)
add 56a2be17d perf: optimize regexp_count to avoid String allocation when
start position is provided (#19553)
add 4e45c19d1 Enables DefaultListFilesCache by default (#19366)
add f1e5c94f3 Compute Dynamic Filters only when a consumer supports them
(#19546)
add 1ce4b51a4 Various refactors to string functions (#19402)
add 27de50d05 fix: Reverse row selection should respect the row group
index (#19557)
add 79f67b8ef feat: implement partition_statistics for WindowAggExec
(#18534)
add b818f9341 perf: Improve performance of `md5` (#19568)
add fd263216c feat: integrate batch coalescer with async fn exec (#19342)
add 8959b3d11 feat: output statistics for constant columns in projections
(#19419)
add db7b8cc4d Implement `partition_statistics` API for
`NestedLoopJoinExec` (#19468)
add cd12d5103 Replace deprecated structopt with clap in
datafusion-benchmarks (#19492)
add 818706ab7 feat: `to_time` function (#19540)
add 0db668bc9 Refactor duplicate code in `type_coercion/functions.rs`
(#19518)
add 90f5bfe30 feat: Implement Spark functions hour, minute, second (#19512)
add bc753c201 chore(deps): bump taiki-e/install-action from 2.65.8 to
2.65.10 (#19578)
add 195d3d64b perf: optimize strpos by eliminating double iteration for
UTF-8 (#19572)
add 9a9ff8d61 perf: Improve performance of hex encoding in spark
functions (#19586)
add 56fec71c7 Add left function benchmark (#19600)
add 987b94ca4 chore: Add TPCDS benchmark comparison for PR (#19552)
add 132006924 Fix typo in contributor guide architecture section (#19613)
add a29569859 chore(deps): bump taiki-e/install-action from 2.65.10 to
2.65.11 (#19601)
add 715962c80 perf: optimize factorial function performance (#19575)
add 8809dae28 perf: Improve performance of ltrim, rtrim, btrim (#19551)
add 2ac032b40 fix: emit empty RecordBatch for empty file writes (#19370)
add 70daf8825 feat: plan-time SQL expression simplifying (#19311)
add 09455f181 chore: bump testcontainers-modules to 0.14 and remove
testcontainers dep (#19620)
add 7fde30a8a fix: handle invalid byte ranges in calculate_range for
single-line files (#19607)
add 955fd41d8 docs: fix typos in PartitionEvaluator trait documentation
(#19631)
add 7e049749e feat: Implement Spark function `space` (#19610)
add e0b4e8d82 feat: Implement `partition_statistics` API for
`SortMergeJoinExec` (#19567)
add 45d4948b3 Validate parquet writer version (#19515)
add 418f62ae3 fix: NULL handling in arrow_intersect and arrow_union
(#19415)
add 52bbc8afc chore(deps): bump insta from 1.45.0 to 1.46.0 (#19643)
add 47df535d2 chore(deps): bump taiki-e/install-action from 2.65.11 to
2.65.13 (#19646)
add c8620129f chore(deps): bump tracing from 0.1.43 to 0.1.44 (#19644)
add fd7924163 chore(deps): bump syn from 2.0.111 to 2.0.113 (#19645)
add ada0923a3 Respect execution timezone in to_timestamp and related
functions (#19078)
add 9b2505ce6 fix(doc): close #19393, make upgrading guide match v51 api
(#19648)
add 2d5625389 fix(spark): Use wrapping addition/subtraction in
`SparkDateAdd` and `SparkDateSub` (#19377)
add ff38480f2 Refactor `percentile_cont` to clarify support input types
(#19611)
add aee5cd9f3 fix(functions): Make translate function postgres compatible
(#19630)
add adf00a649 Add a protection to release candidate branch 52 (#19660)
add 5c2ee3650 perf: optimize `HashTableLookupExpr::evaluate` (#19602)
add c3e1c3644 Downgrade aws-smithy-runtime, update `rust_decimal`, ignore
RUSTSEC-2026-0001 to get clean CI (#19657)
add 1037f0aa2 feat: add list_files_cache table function for
`datafusion-cli` (#19388)
add 924037ea0 perf: Improve performance of `split_part` (#19570)
add a2f02f069 fix: Return Int for Date - Date instead of duration (#19563)
add 7942e751c Update dependencies (#19667)
add ed01b67f2 Refactor PartitionedFile: add ordering field and
new_from_meta constructor (#19596)
add e8196f462 Remove coalesce batches rule and deprecate
CoalesceBatchesExec (#19622)
add 166ef8112 Perf: Optimize `substring_index` via single-byte fast path
and direct indexing (#19590)
add 1f654bbe6 feat: implement metrics for AsyncFuncExec (#19626)
add ce08307a4 refactor: Use `Signature::coercible` for isnan/iszero
(#19604)
add 680ddcc6c feat: split BatchPartitioner::try_new into hash and
round-robin constructors (#19668)
add 566bcde9e Parquet: Push down supported list predicates
(array_has/any/all) during decoding (#19545)
add 3a0ca4ef7 Remove dependency on `rust_decimal`, remove ignore of
`RUSTSEC-2026-0001` (#19666)
add 142f5972d Store example data directly inside the datafusion-examples
(#19141) (#19319)
add 35ff4ab0a Allow logical optimizer to be run without evaluating now() &
refactor SimplifyInfo (#19505)
add 646213ec7 feat: add Time type support to date_trunc function (#19640)
add 102caeb22 minor: More comments to `ParquetOpener::open()` (#19677)
add d18e670e7 feat: Allow log with non-integer base on decimals (#19372)
add 1d5d63c41 Feat: Allow pow with negative & non-integer exponent on
decimals (#19369)
add e6049de5a Make default ListingFilesCache table scoped (#19616)
add 5194fd5eb chore(deps): bump taiki-e/install-action from 2.65.13 to
2.65.15 (#19676)
add 0cf45cae9 Refactor cache APIs to support ordering information (#19597)
add b9a3b9f94 Record sort order when writing Parquet with WITH ORDER
(#19595)
add a55b77e7d fix: DynamicFilterPhysicalExpr violates Hash/Eq contract
(#19659)
add 62658cd62 implement var distinct (#19706)
add c98fa5616 perfect hash join (#19411)
add b7091c0d2 Optimize `Nullstate` / accumulators (#19625)
add 07e63edfa Fix TopK aggregation for UTF-8/Utf8View group keys and add
safe fallback for unsupported string aggregates (#19285)
add 8ba46466d docs: Fix two small issues in introduction.md (#19712)
add 209a0a2e8 fix: unnest struct field with an alias failed with internal
error (#19698)
add 20870da20 infer parquet file order from metadata and use it to
optimize scans (#19433)
add 5c2b1236b feat(spark): implement array_repeat function (#19702)
add 821d410fc feat(spark): Implement collect_list/collect_set aggregate
functions (#19699)
add 3087ca8a9 perf: optimize `NthValue` when `ignore_nulls` is true
(#19496)
add afc912106 Optimize `concat/concat_ws` scalar path by pre-allocating
memory (#19547)
add 45fb0b4b9 fix(accumulators): preserve state in evaluate() for window
frame queries (#19618)
add 458b49109 perf: optimize left function by eliminating double chars()
iteration (#19571)
add 4e0161d99 fix: Don't treat quoted column names as placeholder
variables in SQL (#19339)
add 013efb4fe docs: Refine Communication documentation to highlight
Discord (#19714)
add 41a0b85af Add support for additional numeric types in to_timestamp
functions (#19663)
add 9fa7500bb Fix internal error "Physical input schema should be the same
as the one converted from logical input schema." (#18412)
add 0c5c97b22 fix(functions-aggregate): drain CORR state vectors for
streaming aggregation (#19669)
add 30c6ff198 chore: bump dependabot PR limit for cargo from 5 to 15
(#19730)
add 84ea07029 chore(deps): bump maturin from 1.10.2 to 1.11.5 in /docs
(#19740)
add 2067324f1 chore(deps): bump taiki-e/install-action from 2.65.15 to
2.66.1 (#19741)
add c03606547 chore(deps): bump sqllogictest from 0.28.4 to 0.29.0 (#19744)
add 4ca82cfdb chore(deps): bump blake3 from 1.8.2 to 1.8.3 (#19746)
add 1ddc639fa chore(deps): bump libc from 0.2.179 to 0.2.180 (#19748)
add 1c3763811 chore(deps): bump async-compression from 0.4.36 to 0.4.37
(#19742)
add f9697c14e chore(deps): bump indexmap from 2.12.1 to 2.13.0 (#19747)
add d103d8886 chore: remove LZO Parquet compression (#19726)
add bd2f34805 Improve comment for predicate_cache_inner_records (#19762)
add 383e673ac Update 52.0.0 release version number and changelog (#19767)
add 2fb9fb3fb Update the upgrading.md (#19769)
add 278950a76 Fix dynamic filter is_used function (#19734)
add 7716cae50 chore: update copyright notice year (#19758)
add cb9ec127e slt: Add test for REE arrays in group by (#19763)
add d484c09ba perf: Optimize floor and ceil scalar performance (#19752)
add e8efd5920 chore(deps): Update sqlparser to 0.60 (#19672)
add 803cce881 feat: implement Spark size function for arrays and maps
(#19592)
add 36880d89f Fix run_tpcds data dir (#19771)
add ec974ee5f chore(deps): bump taiki-e/install-action from 2.66.1 to
2.66.2 (#19778)
add f60b68a28 Include .proto files in datafusion-proto distribution
(#19490)
add e076e59b2 Simplify `expr = L1 AND expr != L2` to `expr = L1` when `L1
!= L2` (#19731)
add 4e1bc79e0 fix: enhance CTE resolution with identifier normalization
(#19519)
add 4c67d0208 feat: Add null-aware anti join support (#19635)
add 6267feef8 chore(deps): bump flate2 from 1.1.5 to 1.1.8 (#19780)
add 617700d1b Upgrade DataFusion to arrow-rs/parquet 57.2.0 (#19355)
add c4011e61f Merge branch 'main' into sandbox-main
add 6782849cb chore(deps): bump sphinx from 8.2.3 to 9.1.0 in /docs
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (19d1f7b7d)
\
N -- N -- N refs/heads/dependabot/pip/docs/sphinx-9.1.0
(6782849cb)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/dependabot.yml | 1 +
.github/workflows/audit.yml | 4 +-
.github/workflows/rust.yml | 21 +-
Cargo.lock | 1659 ++++++++------------
Cargo.toml | 103 +-
NOTICE.txt | 2 +-
benchmarks/Cargo.toml | 2 +-
benchmarks/README.md | 19 +-
benchmarks/bench.sh | 6 +-
benchmarks/compare.py | 20 +-
benchmarks/src/bin/dfbench.rs | 15 +-
benchmarks/src/bin/external_aggr.rs | 27 +-
benchmarks/src/bin/imdb.rs | 24 +-
benchmarks/src/bin/mem_profile.rs | 22 +-
benchmarks/src/cancellation.rs | 18 +-
benchmarks/src/clickbench.rs | 30 +-
benchmarks/src/h2o.rs | 26 +-
benchmarks/src/hj.rs | 422 +++--
benchmarks/src/imdb/convert.rs | 12 +-
benchmarks/src/imdb/run.rs | 22 +-
benchmarks/src/nlj.rs | 12 +-
benchmarks/src/smj.rs | 12 +-
benchmarks/src/sort_tpch.rs | 18 +-
benchmarks/src/tpcds/run.rs | 28 +-
benchmarks/src/tpch/run.rs | 28 +-
benchmarks/src/util/options.rs | 18 +-
ci/scripts/check_examples_docs.sh | 64 +
datafusion-cli/Cargo.toml | 3 +-
datafusion-cli/src/functions.rs | 185 ++-
datafusion-cli/src/main.rs | 113 +-
datafusion-cli/tests/cli_integration.rs | 8 +-
datafusion-examples/Cargo.toml | 15 +-
datafusion-examples/README.md | 21 +-
datafusion-examples/data/README.md | 25 +
.../data => datafusion-examples/data/csv}/cars.csv | 0
.../data/csv}/regex.csv | 0
.../examples/builtin_functions/function_factory.rs | 4 +-
.../examples/builtin_functions/main.rs | 2 +-
.../examples/builtin_functions/regexp.rs | 31 +-
.../examples/custom_data_source/csv_json_opener.rs | 32 +-
.../custom_data_source/csv_sql_streaming.rs | 19 +-
.../custom_data_source/default_column_values.rs | 7 +-
.../examples/custom_data_source/main.rs | 2 +-
datafusion-examples/examples/data_io/main.rs | 2 +-
.../examples/data_io/parquet_encrypted.rs | 32 +-
.../examples/data_io/parquet_exec_visitor.rs | 24 +-
.../examples/data_io/parquet_index.rs | 2 +-
.../examples/dataframe/cache_factory.rs | 229 +++
.../examples/dataframe/dataframe.rs | 84 +-
.../examples/dataframe/deserialize_to_struct.rs | 321 +++-
datafusion-examples/examples/dataframe/main.rs | 9 +-
.../examples/execution_monitoring/main.rs | 2 +-
.../examples/execution_monitoring/tracing.rs | 34 +-
.../examples/external_dependency/main.rs | 2 +-
.../ffi/ffi_example_table_provider/src/lib.rs | 7 +-
.../examples/ffi/ffi_module_interface/src/lib.rs | 4 +-
.../examples/ffi/ffi_module_loader/Cargo.toml | 1 +
.../examples/ffi/ffi_module_loader/src/main.rs | 11 +-
datafusion-examples/examples/flight/client.rs | 17 +-
datafusion-examples/examples/flight/main.rs | 2 +-
datafusion-examples/examples/flight/server.rs | 38 +-
datafusion-examples/examples/flight/sql_server.rs | 25 +-
datafusion-examples/examples/proto/main.rs | 2 +-
.../examples/query_planning/expr_api.rs | 13 +-
.../examples/query_planning/main.rs | 2 +-
.../examples/query_planning/parse_sql_expr.rs | 70 +-
.../examples/query_planning/plan_to_sql.rs | 80 +-
.../examples/query_planning/planner_api.rs | 23 +-
.../examples/query_planning/thread_pools.rs | 14 +-
.../examples/relation_planner/main.rs | 2 +-
.../examples/relation_planner/match_recognize.rs | 6 +-
.../examples/relation_planner/pivot_unpivot.rs | 10 +-
.../examples/relation_planner/table_sample.rs | 63 +-
.../examples/sql_ops/custom_sql_parser.rs | 420 +++++
datafusion-examples/examples/sql_ops/dialect.rs | 135 --
datafusion-examples/examples/sql_ops/main.rs | 14 +-
datafusion-examples/examples/sql_ops/query.rs | 64 +-
datafusion-examples/examples/udf/advanced_udaf.rs | 15 +-
datafusion-examples/examples/udf/advanced_udwf.rs | 49 +-
datafusion-examples/examples/udf/async_udf.rs | 3 +-
datafusion-examples/examples/udf/main.rs | 2 +-
datafusion-examples/examples/udf/simple_udtf.rs | 27 +-
.../mod.rs => datafusion-examples/src/lib.rs | 4 +-
datafusion-examples/src/utils/csv_to_parquet.rs | 245 +++
.../src/utils/datasets/cars.rs | 16 +-
datafusion-examples/src/utils/datasets/mod.rs | 139 ++
.../src/utils/datasets/regex.rs | 22 +-
.../src/utils}/mod.rs | 5 +-
datafusion/catalog-listing/src/config.rs | 86 +-
datafusion/catalog-listing/src/table.rs | 410 ++++-
datafusion/catalog/src/cte_worktable.rs | 38 +-
datafusion/catalog/src/memory/table.rs | 349 +++-
datafusion/catalog/src/table.rs | 25 +
datafusion/common/Cargo.toml | 2 +-
datafusion/common/src/config.rs | 87 +-
datafusion/common/src/dfschema.rs | 6 +
.../common/src/file_options/parquet_writer.rs | 34 +-
datafusion/common/src/hash_utils.rs | 136 +-
datafusion/common/src/lib.rs | 4 +-
datafusion/common/src/parquet_config.rs | 108 ++
datafusion/common/src/scalar/consts.rs | 12 +
datafusion/common/src/scalar/mod.rs | 238 ++-
datafusion/common/src/utils/mod.rs | 36 +-
datafusion/core/Cargo.toml | 5 +
.../core/benches/range_and_generate_series.rs | 90 ++
datafusion/core/benches/topk_aggregate.rs | 211 ++-
datafusion/core/src/dataframe/mod.rs | 2 +
.../core/src/datasource/file_format/arrow.rs | 93 ++
datafusion/core/src/datasource/file_format/csv.rs | 90 ++
datafusion/core/src/datasource/file_format/json.rs | 42 +
datafusion/core/src/datasource/file_format/mod.rs | 12 +-
.../core/src/datasource/file_format/parquet.rs | 22 +
datafusion/core/src/datasource/listing/table.rs | 22 +-
.../core/src/datasource/listing_table_factory.rs | 7 +-
datafusion/core/src/datasource/mod.rs | 66 +-
.../core/src/datasource/physical_plan/mod.rs | 135 --
.../core/src/datasource/physical_plan/parquet.rs | 94 +-
datafusion/core/src/execution/context/mod.rs | 35 +-
datafusion/core/src/execution/session_state.rs | 57 +-
datafusion/core/src/physical_planner.rs | 167 +-
datafusion/core/src/test_util/parquet.rs | 15 +-
datafusion/core/tests/core_integration.rs | 3 -
.../tests/custom_sources_cases/dml_planning.rs | 297 ++++
datafusion/core/tests/custom_sources_cases/mod.rs | 1 +
datafusion/core/tests/dataframe/mod.rs | 2 +-
.../core/tests/datasource/object_store_access.rs | 76 +-
datafusion/core/tests/execution/coop.rs | 59 +-
datafusion/core/tests/expr_api/mod.rs | 5 +-
datafusion/core/tests/expr_api/simplification.rs | 156 +-
datafusion/core/tests/fuzz_cases/join_fuzz.rs | 1 +
datafusion/core/tests/fuzz_cases/window_fuzz.rs | 4 +-
datafusion/core/tests/parquet/custom_reader.rs | 10 +-
.../parquet/{schema_adapter.rs => expr_adapter.rs} | 152 +-
datafusion/core/tests/parquet/mod.rs | 3 +-
datafusion/core/tests/parquet/ordering.rs | 103 ++
datafusion/core/tests/parquet/page_pruning.rs | 9 +-
.../physical_optimizer/aggregate_statistics.rs | 86 +
.../physical_optimizer/enforce_distribution.rs | 59 +-
.../tests/physical_optimizer/enforce_sorting.rs | 57 +-
.../physical_optimizer/filter_pushdown/mod.rs | 409 +++--
.../physical_optimizer/filter_pushdown/util.rs | 28 +-
.../tests/physical_optimizer/join_selection.rs | 10 +
.../tests/physical_optimizer/limit_pushdown.rs | 84 +-
.../physical_optimizer/partition_statistics.rs | 162 +-
.../physical_optimizer/projection_pushdown.rs | 46 +
.../core/tests/physical_optimizer/pushdown_sort.rs | 434 ++++-
.../replace_with_order_preserving_variants.rs | 229 ++-
.../tests/physical_optimizer/sanity_checker.rs | 6 +-
.../core/tests/physical_optimizer/test_utils.rs | 216 ++-
datafusion/core/tests/schema_adapter/mod.rs | 18 -
.../schema_adapter_integration_tests.rs | 752 ---------
datafusion/core/tests/sql/explain_analyze.rs | 1 -
datafusion/core/tests/sql/mod.rs | 1 +
datafusion/core/tests/sql/unparser.rs | 462 ++++++
.../core/tests/user_defined/relation_planner.rs | 16 +-
.../user_defined_async_scalar_functions.rs | 40 +-
.../user_defined/user_defined_scalar_functions.rs | 19 +-
datafusion/datasource-arrow/NOTICE.txt | 2 +-
datafusion/datasource-arrow/src/source.rs | 118 +-
.../datasource-avro/src/avro_to_arrow/schema.rs | 4 +-
datafusion/datasource-avro/src/source.rs | 17 -
datafusion/datasource-csv/src/file_format.rs | 50 +-
datafusion/datasource-csv/src/source.rs | 17 -
datafusion/datasource-json/src/source.rs | 17 -
datafusion/datasource-parquet/Cargo.toml | 7 +
.../benches/parquet_nested_filter_pushdown.rs | 238 +++
datafusion/datasource-parquet/src/file_format.rs | 92 +-
datafusion/datasource-parquet/src/metadata.rs | 142 +-
datafusion/datasource-parquet/src/metrics.rs | 10 +-
datafusion/datasource-parquet/src/mod.rs | 1 +
datafusion/datasource-parquet/src/opener.rs | 617 +++++---
datafusion/datasource-parquet/src/row_filter.rs | 487 +++++-
.../datasource-parquet/src/row_group_filter.rs | 9 +-
datafusion/datasource-parquet/src/sort.rs | 865 ++++++++--
datafusion/datasource-parquet/src/source.rs | 88 +-
.../datasource-parquet/src/supported_predicates.rs | 144 ++
datafusion/datasource/Cargo.toml | 2 +-
datafusion/datasource/src/display.rs | 9 +-
datafusion/datasource/src/file.rs | 87 +-
datafusion/datasource/src/file_format.rs | 76 +
datafusion/datasource/src/file_scan_config.rs | 96 +-
datafusion/datasource/src/mod.rs | 77 +-
datafusion/datasource/src/schema_adapter.rs | 1065 ++-----------
datafusion/datasource/src/test_util.rs | 18 -
datafusion/datasource/src/url.rs | 84 +-
datafusion/datasource/src/write/demux.rs | 17 +
datafusion/execution/Cargo.toml | 1 +
datafusion/execution/src/cache/cache_manager.rs | 284 +++-
datafusion/execution/src/cache/cache_unit.rs | 454 ++++--
.../execution/src/cache/file_metadata_cache.rs | 429 ++---
datafusion/execution/src/cache/list_files_cache.rs | 917 ++++++-----
datafusion/execution/src/cache/mod.rs | 48 +-
datafusion/execution/src/lib.rs | 1 -
datafusion/expr-common/src/accumulator.rs | 23 +-
datafusion/expr-common/src/signature.rs | 74 +-
datafusion/expr-common/src/type_coercion/binary.rs | 121 +-
.../src/type_coercion/binary/tests/arithmetic.rs | 8 +-
.../src/type_coercion/binary/tests/comparison.rs | 58 +-
datafusion/expr/src/arguments.rs | 433 ++++-
datafusion/expr/src/execution_props.rs | 16 +-
datafusion/expr/src/expr.rs | 3 +-
datafusion/expr/src/expr_schema.rs | 55 +-
datafusion/expr/src/function.rs | 8 +-
datafusion/expr/src/lib.rs | 4 +
datafusion/expr/src/logical_plan/builder.rs | 23 +
datafusion/expr/src/logical_plan/dml.rs | 8 +-
datafusion/expr/src/logical_plan/plan.rs | 24 +
datafusion/expr/src/logical_plan/tree_node.rs | 4 +
datafusion/expr/src/partition_evaluator.rs | 4 +-
datafusion/expr/src/planner.rs | 4 +-
datafusion/expr/src/simplify.rs | 120 +-
datafusion/expr/src/type_coercion/functions.rs | 304 ++--
datafusion/expr/src/udaf.rs | 2 +-
datafusion/expr/src/udf.rs | 14 +-
datafusion/expr/src/udwf.rs | 2 +-
datafusion/ffi/Cargo.toml | 15 +-
datafusion/ffi/src/arrow_wrappers.rs | 10 +-
datafusion/ffi/src/catalog_provider.rs | 95 +-
datafusion/ffi/src/catalog_provider_list.rs | 81 +-
datafusion/ffi/src/execution/task_ctx.rs | 1 -
datafusion/ffi/src/execution/task_ctx_provider.rs | 1 -
datafusion/ffi/src/execution_plan.rs | 98 +-
datafusion/ffi/src/expr/columnar_value.rs | 1 -
datafusion/ffi/src/expr/distribution.rs | 11 +-
datafusion/ffi/src/expr/expr_properties.rs | 3 -
datafusion/ffi/src/expr/interval.rs | 4 +-
datafusion/ffi/src/insert_op.rs | 3 +-
datafusion/ffi/src/lib.rs | 1 +
datafusion/ffi/src/physical_expr/mod.rs | 69 +-
datafusion/ffi/src/physical_expr/partitioning.rs | 7 +-
datafusion/ffi/src/physical_expr/sort.rs | 7 +-
datafusion/ffi/src/plan_properties.rs | 143 +-
.../ffi/src/proto/logical_extension_codec.rs | 39 +-
.../ffi/src/proto/physical_extension_codec.rs | 11 +-
datafusion/ffi/src/record_batch_stream.rs | 85 +-
datafusion/ffi/src/schema_provider.rs | 103 +-
datafusion/ffi/src/session/config.rs | 1 -
datafusion/ffi/src/session/mod.rs | 10 +-
datafusion/ffi/src/table_provider.rs | 259 +--
datafusion/ffi/src/table_source.rs | 7 +-
datafusion/ffi/src/tests/async_provider.rs | 64 +-
datafusion/ffi/src/tests/catalog.rs | 41 +-
datafusion/ffi/src/tests/mod.rs | 60 +-
datafusion/ffi/src/tests/sync_provider.rs | 11 +-
datafusion/ffi/src/tests/udf_udaf_udwf.rs | 70 +-
datafusion/ffi/src/tests/utils.rs | 8 +-
datafusion/ffi/src/udaf/accumulator.rs | 40 +-
datafusion/ffi/src/udaf/accumulator_args.rs | 122 +-
datafusion/ffi/src/udaf/groups_accumulator.rs | 47 +-
datafusion/ffi/src/udaf/mod.rs | 67 +-
datafusion/ffi/src/udf/mod.rs | 63 +-
datafusion/ffi/src/udf/return_type_args.rs | 16 +-
datafusion/ffi/src/udtf.rs | 158 +-
datafusion/ffi/src/udwf/mod.rs | 17 +-
datafusion/ffi/src/udwf/partition_evaluator.rs | 31 +-
.../ffi/src/udwf/partition_evaluator_args.rs | 108 +-
datafusion/ffi/src/udwf/range.rs | 1 -
datafusion/ffi/src/volatility.rs | 3 +-
datafusion/ffi/tests/ffi_catalog.rs | 14 +-
datafusion/ffi/tests/ffi_integration.rs | 11 +-
datafusion/ffi/tests/ffi_udaf.rs | 71 +-
datafusion/ffi/tests/ffi_udf.rs | 4 +-
datafusion/ffi/tests/ffi_udtf.rs | 8 +-
datafusion/ffi/tests/ffi_udwf.rs | 3 +-
datafusion/ffi/tests/utils/mod.rs | 43 +
.../src/aggregate/groups_accumulator/accumulate.rs | 214 ++-
.../src/aggregate/groups_accumulator/bool_op.rs | 2 +-
.../src/aggregate/groups_accumulator/prim_op.rs | 5 +-
datafusion/functions-aggregate/src/array_agg.rs | 4 +-
datafusion/functions-aggregate/src/average.rs | 21 +-
datafusion/functions-aggregate/src/correlation.rs | 62 +-
datafusion/functions-aggregate/src/count.rs | 4 +-
datafusion/functions-aggregate/src/median.rs | 22 +-
.../functions-aggregate/src/percentile_cont.rs | 505 +++---
datafusion/functions-aggregate/src/string_agg.rs | 13 +-
datafusion/functions-aggregate/src/variance.rs | 144 +-
datafusion/functions-nested/src/array_has.rs | 21 +-
datafusion/functions-nested/src/planner.rs | 3 +
datafusion/functions-nested/src/set_ops.rs | 19 +-
datafusion/functions-table/src/generate_series.rs | 47 +
datafusion/functions-window/Cargo.toml | 8 +
datafusion/functions-window/benches/nth_value.rs | 263 ++++
datafusion/functions-window/src/nth_value.rs | 182 ++-
datafusion/functions/Cargo.toml | 70 +-
datafusion/functions/benches/concat.rs | 100 +-
.../functions/benches/{concat.rs => concat_ws.rs} | 64 +-
datafusion/functions/benches/contains.rs | 185 +++
.../functions/benches/{upper.rs => crypto.rs} | 49 +-
datafusion/functions/benches/ends_with.rs | 185 +++
.../functions/benches/{uuid.rs => factorial.rs} | 35 +-
datafusion/functions/benches/floor_ceil.rs | 135 ++
datafusion/functions/benches/left.rs | 111 ++
datafusion/functions/benches/levenshtein.rs | 87 +
datafusion/functions/benches/pad.rs | 314 +++-
datafusion/functions/benches/regexp_count.rs | 118 ++
.../functions/benches/{repeat.rs => replace.rs} | 108 +-
datafusion/functions/benches/split_part.rs | 382 +++++
datafusion/functions/benches/starts_with.rs | 185 +++
datafusion/functions/benches/substr_index.rs | 124 +-
datafusion/functions/benches/to_hex.rs | 120 +-
datafusion/functions/benches/to_timestamp.rs | 24 +-
datafusion/functions/benches/translate.rs | 90 ++
datafusion/functions/benches/{ltrim.rs => trim.rs} | 200 ++-
datafusion/functions/src/core/arrow_cast.rs | 17 +-
datafusion/functions/src/core/arrow_metadata.rs | 160 ++
datafusion/functions/src/core/coalesce.rs | 4 +-
datafusion/functions/src/core/getfield.rs | 645 +++++---
datafusion/functions/src/core/mod.rs | 14 +
datafusion/functions/src/core/nvl.rs | 4 +-
datafusion/functions/src/core/nvl2.rs | 4 +-
datafusion/functions/src/core/union_extract.rs | 5 +-
datafusion/functions/src/crypto/basic.rs | 34 +-
datafusion/functions/src/datetime/common.rs | 153 +-
datafusion/functions/src/datetime/current_date.rs | 23 +-
datafusion/functions/src/datetime/current_time.rs | 65 +-
datafusion/functions/src/datetime/date_bin.rs | 314 +++-
datafusion/functions/src/datetime/date_trunc.rs | 284 +++-
datafusion/functions/src/datetime/mod.rs | 55 +-
datafusion/functions/src/datetime/now.rs | 18 +-
datafusion/functions/src/datetime/to_date.rs | 6 +-
datafusion/functions/src/datetime/to_time.rs | 252 +++
datafusion/functions/src/datetime/to_timestamp.rs | 1007 +++++++++---
datafusion/functions/src/datetime/to_unixtime.rs | 38 +-
datafusion/functions/src/encoding/inner.rs | 745 ++++-----
datafusion/functions/src/macros.rs | 29 +
datafusion/functions/src/math/ceil.rs | 206 +++
datafusion/functions/src/math/decimal.rs | 111 ++
datafusion/functions/src/math/factorial.rs | 78 +-
datafusion/functions/src/math/floor.rs | 206 +++
datafusion/functions/src/math/iszero.rs | 32 +-
datafusion/functions/src/math/log.rs | 230 ++-
datafusion/functions/src/math/mod.rs | 21 +-
datafusion/functions/src/math/monotonicity.rs | 48 -
datafusion/functions/src/math/nans.rs | 32 +-
datafusion/functions/src/math/nanvl.rs | 27 +-
datafusion/functions/src/math/power.rs | 368 ++++-
datafusion/functions/src/math/round.rs | 367 +++--
datafusion/functions/src/regex/regexpcount.rs | 12 +-
datafusion/functions/src/regex/regexplike.rs | 4 +-
datafusion/functions/src/string/btrim.rs | 2 +-
datafusion/functions/src/string/common.rs | 234 ++-
datafusion/functions/src/string/concat.rs | 11 +-
datafusion/functions/src/string/concat_ws.rs | 35 +-
datafusion/functions/src/string/contains.rs | 89 +-
datafusion/functions/src/string/ends_with.rs | 294 +++-
datafusion/functions/src/string/levenshtein.rs | 24 +-
datafusion/functions/src/string/ltrim.rs | 10 +-
datafusion/functions/src/string/repeat.rs | 33 +-
datafusion/functions/src/string/replace.rs | 85 +-
datafusion/functions/src/string/rtrim.rs | 10 +-
datafusion/functions/src/string/split_part.rs | 62 +-
datafusion/functions/src/string/starts_with.rs | 272 +++-
datafusion/functions/src/string/to_hex.rs | 217 ++-
datafusion/functions/src/string/uuid.rs | 2 +-
datafusion/functions/src/unicode/left.rs | 15 +-
datafusion/functions/src/unicode/lpad.rs | 33 +-
datafusion/functions/src/unicode/rpad.rs | 38 +-
datafusion/functions/src/unicode/strpos.rs | 39 +-
datafusion/functions/src/unicode/substrindex.rs | 70 +-
datafusion/functions/src/unicode/translate.rs | 69 +-
datafusion/functions/src/utils.rs | 153 +-
datafusion/macros/Cargo.toml | 2 +-
datafusion/optimizer/src/analyzer/type_coercion.rs | 185 ++-
datafusion/optimizer/src/decorrelate.rs | 10 +-
.../src/decorrelate_predicate_subquery.rs | 72 +-
datafusion/optimizer/src/eliminate_cross_join.rs | 3 +
datafusion/optimizer/src/eliminate_outer_join.rs | 1 +
.../optimizer/src/extract_equijoin_predicate.rs | 4 +
datafusion/optimizer/src/optimizer.rs | 28 +-
.../src/simplify_expressions/expr_simplifier.rs | 213 +--
.../optimizer/src/simplify_expressions/mod.rs | 3 +-
.../src/simplify_expressions/simplify_exprs.rs | 13 +-
.../src/simplify_expressions/simplify_literal.rs | 148 ++
.../src/simplify_expressions/unwrap_cast.rs | 22 +-
.../optimizer/src/simplify_expressions/utils.rs | 48 +
.../physical-expr-adapter/src/schema_rewriter.rs | 4 +-
datafusion/physical-expr-common/Cargo.toml | 3 +
datafusion/physical-expr-common/src/lib.rs | 1 +
.../src/metrics/baseline.rs | 4 +-
.../src/metrics/builder.rs | 2 +-
.../src/metrics/custom.rs | 2 +-
.../physical-expr-common/src/metrics/expression.rs | 88 ++
.../src/metrics/mod.rs | 4 +-
.../src/metrics/value.rs | 0
datafusion/physical-expr-common/src/sort_expr.rs | 21 +-
datafusion/physical-expr-common/src/utils.rs | 17 +-
datafusion/physical-expr/Cargo.toml | 4 +
datafusion/physical-expr/benches/in_list.rs | 77 +
datafusion/physical-expr/src/equivalence/class.rs | 8 +-
datafusion/physical-expr/src/expressions/binary.rs | 94 ++
datafusion/physical-expr/src/expressions/case.rs | 279 +---
.../src/expressions/dynamic_filters.rs | 189 ++-
.../physical-expr/src/expressions/in_list.rs | 302 +++-
datafusion/physical-expr/src/partitioning.rs | 586 ++++++-
datafusion/physical-expr/src/planner.rs | 33 +-
datafusion/physical-expr/src/projection.rs | 317 +++-
datafusion/physical-expr/src/scalar_function.rs | 8 +-
datafusion/physical-expr/src/utils/mod.rs | 2 +-
.../physical-optimizer/src/coalesce_batches.rs | 87 -
.../physical-optimizer/src/enforce_distribution.rs | 86 +-
.../physical-optimizer/src/join_selection.rs | 28 +-
datafusion/physical-optimizer/src/lib.rs | 1 -
datafusion/physical-optimizer/src/optimizer.rs | 4 -
.../physical-optimizer/src/sanity_checker.rs | 3 +-
.../physical-optimizer/src/topk_aggregation.rs | 21 +-
datafusion/physical-plan/Cargo.toml | 1 +
.../src/aggregates/group_values/mod.rs | 4 +-
.../group_values/multi_group_by/bytes_view.rs | 159 +-
.../aggregates/group_values/multi_group_by/mod.rs | 24 +-
.../src/aggregates/group_values/row.rs | 9 +-
.../group_values/single_group_by/boolean.rs | 3 +-
.../group_values/single_group_by/bytes.rs | 4 +-
.../group_values/single_group_by/bytes_view.rs | 4 +-
.../group_values/single_group_by/primitive.rs | 8 +-
datafusion/physical-plan/src/aggregates/mod.rs | 284 +++-
.../physical-plan/src/aggregates/row_hash.rs | 353 +++--
.../src/aggregates/topk/hash_table.rs | 416 +++--
.../physical-plan/src/aggregates/topk/heap.rs | 230 ++-
.../src/aggregates/topk/priority_map.rs | 129 +-
.../physical-plan/src/aggregates/topk_stream.rs | 14 +
datafusion/physical-plan/src/async_func.rs | 118 +-
datafusion/physical-plan/src/coalesce_batches.rs | 20 +
datafusion/physical-plan/src/coop.rs | 39 +-
datafusion/physical-plan/src/filter.rs | 5 +-
datafusion/physical-plan/src/joins/array_map.rs | 547 +++++++
datafusion/physical-plan/src/joins/chain.rs | 69 +
.../physical-plan/src/joins/hash_join/exec.rs | 1338 +++++++++++++---
.../physical-plan/src/joins/hash_join/mod.rs | 2 +-
.../src/joins/hash_join/partitioned_hash_eval.rs | 458 +++++-
.../src/joins/hash_join/shared_bounds.rs | 41 +-
.../physical-plan/src/joins/hash_join/stream.rs | 163 +-
.../physical-plan/src/joins/join_hash_map.rs | 117 +-
datafusion/physical-plan/src/joins/mod.rs | 29 +-
.../physical-plan/src/joins/nested_loop_join.rs | 33 +-
.../src/joins/sort_merge_join/exec.rs | 18 +-
.../src/joins/sort_merge_join/tests.rs | 74 +
.../physical-plan/src/joins/stream_join_utils.rs | 14 +-
datafusion/physical-plan/src/joins/test_utils.rs | 1 +
datafusion/physical-plan/src/joins/utils.rs | 30 +-
datafusion/physical-plan/src/memory.rs | 54 +
datafusion/physical-plan/src/metrics.rs | 6 +-
datafusion/physical-plan/src/projection.rs | 77 +-
datafusion/physical-plan/src/recursive_query.rs | 4 +-
datafusion/physical-plan/src/repartition/mod.rs | 128 +-
.../physical-plan/src/sorts/multi_level_merge.rs | 12 +-
datafusion/physical-plan/src/sorts/sort.rs | 465 +++++-
.../src/sorts/sort_preserving_merge.rs | 18 +-
.../physical-plan/src/spill/spill_manager.rs | 5 +-
datafusion/physical-plan/src/stream.rs | 187 ++-
datafusion/physical-plan/src/test.rs | 1 +
.../physical-plan/src/windows/window_agg_exec.rs | 38 +-
datafusion/physical-plan/src/work_table.rs | 103 +-
datafusion/proto-common/src/from_proto/mod.rs | 25 +-
datafusion/proto-common/src/to_proto/mod.rs | 2 +-
datafusion/proto/Cargo.toml | 3 -
datafusion/proto/proto/datafusion.proto | 12 +
datafusion/proto/src/generated/pbjson.rs | 225 +++
datafusion/proto/src/generated/prost.rs | 21 +-
datafusion/proto/src/logical_plan/file_formats.rs | 7 +-
datafusion/proto/src/physical_plan/from_proto.rs | 76 +-
datafusion/proto/src/physical_plan/mod.rs | 6 +
datafusion/proto/src/physical_plan/to_proto.rs | 16 +-
.../proto/tests/cases/roundtrip_logical_plan.rs | 15 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 53 +-
datafusion/spark/Cargo.toml | 5 +
datafusion/spark/benches/{char.rs => space.rs} | 25 +-
datafusion/spark/src/function/aggregate/collect.rs | 200 +++
datafusion/spark/src/function/aggregate/mod.rs | 19 +-
datafusion/spark/src/function/array/mod.rs | 9 +-
datafusion/spark/src/function/array/repeat.rs | 128 ++
datafusion/spark/src/function/collection/mod.rs | 13 +-
datafusion/spark/src/function/collection/size.rs | 162 ++
datafusion/spark/src/function/conditional/if.rs | 2 +-
datafusion/spark/src/function/datetime/date_add.rs | 25 +-
datafusion/spark/src/function/datetime/date_sub.rs | 25 +-
datafusion/spark/src/function/datetime/extract.rs | 268 ++++
datafusion/spark/src/function/datetime/mod.rs | 18 +
datafusion/spark/src/function/hash/crc32.rs | 43 +-
datafusion/spark/src/function/hash/sha1.rs | 27 +-
datafusion/spark/src/function/math/abs.rs | 78 +-
datafusion/spark/src/function/math/hex.rs | 27 +-
datafusion/spark/src/function/mod.rs | 1 +
datafusion/spark/src/function/null_utils.rs | 122 ++
datafusion/spark/src/function/string/ascii.rs | 77 +-
datafusion/spark/src/function/string/concat.rs | 110 +-
datafusion/spark/src/function/string/elt.rs | 67 +-
.../spark/src/function/string/format_string.rs | 63 +-
datafusion/spark/src/function/string/like.rs | 92 +-
datafusion/spark/src/function/string/mod.rs | 4 +
datafusion/spark/src/function/string/space.rs | 232 +++
datafusion/sql/src/expr/function.rs | 37 +-
datafusion/sql/src/expr/identifier.rs | 4 +-
datafusion/sql/src/expr/value.rs | 130 +-
datafusion/sql/src/planner.rs | 2 +-
datafusion/sql/src/query.rs | 1 +
datafusion/sql/src/relation/mod.rs | 8 +-
datafusion/sql/src/resolve.rs | 162 +-
datafusion/sql/src/select.rs | 8 +-
datafusion/sql/src/statement.rs | 192 ++-
datafusion/sql/src/unparser/ast.rs | 46 +-
datafusion/sql/src/unparser/dialect.rs | 45 +-
datafusion/sql/src/unparser/expr.rs | 44 +-
datafusion/sql/src/unparser/plan.rs | 36 +-
datafusion/sql/src/utils.rs | 25 +-
datafusion/sql/src/values.rs | 8 +-
datafusion/sql/tests/cases/plan_to_sql.rs | 80 +-
datafusion/sql/tests/common/mod.rs | 12 +-
datafusion/sql/tests/sql_integration.rs | 42 +-
datafusion/sqllogictest/Cargo.toml | 7 +-
datafusion/sqllogictest/bin/postgres_container.rs | 6 +-
datafusion/sqllogictest/src/engines/conversion.rs | 6 +-
.../src/engines/postgres_engine/mod.rs | 131 +-
.../src/engines/postgres_engine/types.rs | 45 -
datafusion/sqllogictest/src/test_context.rs | 165 +-
datafusion/sqllogictest/test_files/aggregate.slt | 501 +++++-
.../test_files/aggregate_skip_partial.slt | 17 +-
.../sqllogictest/test_files/aggregates_topk.slt | 88 ++
datafusion/sqllogictest/test_files/array.slt | 32 +
.../sqllogictest/test_files/arrow_typeof.slt | 5 +-
datafusion/sqllogictest/test_files/async_udf.slt | 12 +-
datafusion/sqllogictest/test_files/case.slt | 5 +-
datafusion/sqllogictest/test_files/cte.slt | 69 +-
.../test_files/cte_quoted_reference.slt | 70 +
.../test_files/datetime/arith_date_date.slt | 15 +
.../test_files/datetime/arith_date_integer.slt | 89 ++
.../test_files/datetime/arith_date_interval.slt | 37 +
.../test_files/datetime/arith_date_time.slt | 116 ++
.../test_files/datetime/arith_interval_double.slt | 41 +
.../datetime/arith_interval_interval.slt | 27 +
.../test_files/datetime/arith_negate_interval.slt | 13 +
.../test_files/datetime/arith_time_interval.slt | 70 +
.../test_files/datetime/arith_time_time.slt | 47 +
.../datetime/arith_timestamp_duration.slt | 147 ++
.../datetime/arith_timestamp_interval.slt | 36 +
.../datetime/arith_timestamp_timestamp.slt | 13 +
.../{ => datetime}/current_date_timezone.slt | 0
.../{ => datetime}/current_time_timezone.slt | 0
.../test_files/{expr => datetime}/date_part.slt | 0
.../test_files/{ => datetime}/dates.slt | 14 +-
.../test_files/{ => datetime}/interval.slt | 0
.../test_files/{ => datetime}/interval_mysql.slt | 0
.../test_files/{ => datetime}/timestamps.slt | 1245 ++++++++++++++-
datafusion/sqllogictest/test_files/decimal.slt | 173 +-
datafusion/sqllogictest/test_files/delete.slt | 16 +-
datafusion/sqllogictest/test_files/dml_delete.slt | 202 +++
datafusion/sqllogictest/test_files/dml_update.slt | 286 ++++
.../test_files/dynamic_filter_pushdown_config.slt | 326 ----
datafusion/sqllogictest/test_files/encoding.slt | 143 +-
datafusion/sqllogictest/test_files/errors.slt | 4 +-
datafusion/sqllogictest/test_files/explain.slt | 4 -
.../sqllogictest/test_files/explain_analyze.slt | 41 +
.../sqllogictest/test_files/information_schema.slt | 63 +-
datafusion/sqllogictest/test_files/join.slt.part | 8 +-
datafusion/sqllogictest/test_files/joins.slt | 13 +-
datafusion/sqllogictest/test_files/math.slt | 2 +-
datafusion/sqllogictest/test_files/metadata.slt | 54 +-
.../sqllogictest/test_files/named_arguments.slt | 3 +-
.../test_files/null_aware_anti_join.slt | 453 ++++++
datafusion/sqllogictest/test_files/order.slt | 11 +
datafusion/sqllogictest/test_files/parquet.slt | 4 +
.../test_files/parquet_filter_pushdown.slt | 111 ++
.../sqllogictest/test_files/repartition_scan.slt | 4 +
.../test_files/repartition_subset_satisfaction.slt | 526 +++++++
.../sqllogictest/test_files/run_end_encoded.slt | 57 +
datafusion/sqllogictest/test_files/scalar.slt | 130 +-
.../sqllogictest/test_files/schema_evolution.slt | 144 ++
.../sqllogictest/test_files/set_variable.slt | 18 +
.../sqllogictest/test_files/simplify_expr.slt | 18 +
.../sqllogictest/test_files/sort_merge_join.slt | 2 +-
.../sqllogictest/test_files/sort_pushdown.slt | 886 +++++++++++
.../test_files/spark/aggregate/collect.slt | 93 ++
.../test_files/spark/array/array_repeat.slt | 77 +-
.../test_files/spark/collection/size.slt | 132 ++
.../test_files/spark/datetime/date_add.slt | 12 +-
.../test_files/spark/datetime/hour.slt | 23 +-
.../test_files/spark/datetime/minute.slt | 23 +-
.../test_files/spark/datetime/second.slt | 23 +-
.../sqllogictest/test_files/spark/hash/crc32.slt | 6 +-
.../spark/string/{char.slt => space.slt} | 23 +-
.../test_files/string/string_query.slt.part | 10 +-
datafusion/sqllogictest/test_files/struct.slt | 210 ++-
.../sqllogictest/test_files/table_functions.slt | 24 +
.../test_files/to_timestamp_timezone.slt | 204 +++
.../test_files/tpch/plans/q16.slt.part | 11 +-
.../test_files/tpch/plans/q18.slt.part | 34 +-
.../sqllogictest/test_files/tpch/plans/q3.slt.part | 30 +-
datafusion/sqllogictest/test_files/unnest.slt | 30 +-
datafusion/sqllogictest/test_files/update.slt | 20 +-
datafusion/substrait/src/physical_plan/consumer.rs | 12 +-
datafusion/substrait/src/serializer.rs | 1 +
datafusion/wasmtest/src/lib.rs | 11 +-
dev/changelog/52.0.0.md | 745 +++++++++
docs/requirements.txt | 4 +-
docs/source/_static/theme_overrides.css | 18 +
docs/source/contributor-guide/architecture.md | 2 +-
docs/source/contributor-guide/communication.md | 68 +-
docs/source/index.rst | 3 +-
docs/source/library-user-guide/upgrading.md | 348 ++--
docs/source/user-guide/cli/functions.md | 50 +
docs/source/user-guide/configs.md | 253 +--
docs/source/user-guide/introduction.md | 5 +-
docs/source/user-guide/sql/format_options.md | 102 +-
docs/source/user-guide/sql/scalar_functions.md | 245 ++-
603 files changed, 41161 insertions(+), 14363 deletions(-)
create mode 100755 ci/scripts/check_examples_docs.sh
create mode 100644 datafusion-examples/data/README.md
copy {datafusion/core/tests/data => datafusion-examples/data/csv}/cars.csv
(100%)
copy {datafusion/physical-expr/tests/data =>
datafusion-examples/data/csv}/regex.csv (100%)
create mode 100644 datafusion-examples/examples/dataframe/cache_factory.rs
create mode 100644 datafusion-examples/examples/sql_ops/custom_sql_parser.rs
delete mode 100644 datafusion-examples/examples/sql_ops/dialect.rs
copy datafusion/physical-expr/src/statistics/mod.rs =>
datafusion-examples/src/lib.rs (91%)
create mode 100644 datafusion-examples/src/utils/csv_to_parquet.rs
copy datafusion/spark/src/function/collection/mod.rs =>
datafusion-examples/src/utils/datasets/cars.rs (66%)
create mode 100644 datafusion-examples/src/utils/datasets/mod.rs
copy datafusion/spark/src/function/conditional/mod.rs =>
datafusion-examples/src/utils/datasets/regex.rs (67%)
copy {datafusion/physical-expr/src/statistics =>
datafusion-examples/src/utils}/mod.rs (90%)
create mode 100644 datafusion/common/src/parquet_config.rs
create mode 100644 datafusion/core/benches/range_and_generate_series.rs
create mode 100644 datafusion/core/tests/custom_sources_cases/dml_planning.rs
rename datafusion/core/tests/parquet/{schema_adapter.rs => expr_adapter.rs}
(70%)
create mode 100644 datafusion/core/tests/parquet/ordering.rs
delete mode 100644 datafusion/core/tests/schema_adapter/mod.rs
delete mode 100644
datafusion/core/tests/schema_adapter/schema_adapter_integration_tests.rs
create mode 100644 datafusion/core/tests/sql/unparser.rs
create mode 100644
datafusion/datasource-parquet/benches/parquet_nested_filter_pushdown.rs
create mode 100644 datafusion/datasource-parquet/src/supported_predicates.rs
create mode 100644 datafusion/ffi/tests/utils/mod.rs
create mode 100644 datafusion/functions-window/benches/nth_value.rs
copy datafusion/functions/benches/{concat.rs => concat_ws.rs} (56%)
create mode 100644 datafusion/functions/benches/contains.rs
copy datafusion/functions/benches/{upper.rs => crypto.rs} (55%)
create mode 100644 datafusion/functions/benches/ends_with.rs
copy datafusion/functions/benches/{uuid.rs => factorial.rs} (52%)
create mode 100644 datafusion/functions/benches/floor_ceil.rs
create mode 100644 datafusion/functions/benches/left.rs
create mode 100644 datafusion/functions/benches/levenshtein.rs
create mode 100644 datafusion/functions/benches/regexp_count.rs
copy datafusion/functions/benches/{repeat.rs => replace.rs} (56%)
create mode 100644 datafusion/functions/benches/split_part.rs
create mode 100644 datafusion/functions/benches/starts_with.rs
create mode 100644 datafusion/functions/benches/translate.rs
rename datafusion/functions/benches/{ltrim.rs => trim.rs} (56%)
create mode 100644 datafusion/functions/src/core/arrow_metadata.rs
create mode 100644 datafusion/functions/src/datetime/to_time.rs
create mode 100644 datafusion/functions/src/math/ceil.rs
create mode 100644 datafusion/functions/src/math/decimal.rs
create mode 100644 datafusion/functions/src/math/floor.rs
create mode 100644
datafusion/optimizer/src/simplify_expressions/simplify_literal.rs
rename datafusion/{execution => physical-expr-common}/src/metrics/baseline.rs
(98%)
rename datafusion/{execution => physical-expr-common}/src/metrics/builder.rs
(99%)
rename datafusion/{execution => physical-expr-common}/src/metrics/custom.rs
(98%)
create mode 100644 datafusion/physical-expr-common/src/metrics/expression.rs
rename datafusion/{execution => physical-expr-common}/src/metrics/mod.rs (99%)
rename datafusion/{execution => physical-expr-common}/src/metrics/value.rs
(100%)
delete mode 100644 datafusion/physical-optimizer/src/coalesce_batches.rs
create mode 100644 datafusion/physical-plan/src/joins/array_map.rs
create mode 100644 datafusion/physical-plan/src/joins/chain.rs
copy datafusion/spark/benches/{char.rs => space.rs} (83%)
create mode 100644 datafusion/spark/src/function/aggregate/collect.rs
create mode 100644 datafusion/spark/src/function/array/repeat.rs
create mode 100644 datafusion/spark/src/function/collection/size.rs
create mode 100644 datafusion/spark/src/function/datetime/extract.rs
create mode 100644 datafusion/spark/src/function/null_utils.rs
create mode 100644 datafusion/spark/src/function/string/space.rs
delete mode 100644 datafusion/sqllogictest/src/engines/postgres_engine/types.rs
create mode 100644 datafusion/sqllogictest/test_files/cte_quoted_reference.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_date_date.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_date_integer.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_date_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_date_time.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_interval_double.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_interval_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_negate_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_time_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_time_time.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_timestamp_duration.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_timestamp_interval.slt
create mode 100644
datafusion/sqllogictest/test_files/datetime/arith_timestamp_timestamp.slt
rename datafusion/sqllogictest/test_files/{ =>
datetime}/current_date_timezone.slt (100%)
rename datafusion/sqllogictest/test_files/{ =>
datetime}/current_time_timezone.slt (100%)
rename datafusion/sqllogictest/test_files/{expr => datetime}/date_part.slt
(100%)
rename datafusion/sqllogictest/test_files/{ => datetime}/dates.slt (98%)
rename datafusion/sqllogictest/test_files/{ => datetime}/interval.slt (100%)
rename datafusion/sqllogictest/test_files/{ => datetime}/interval_mysql.slt
(100%)
rename datafusion/sqllogictest/test_files/{ => datetime}/timestamps.slt (82%)
create mode 100644 datafusion/sqllogictest/test_files/dml_delete.slt
create mode 100644 datafusion/sqllogictest/test_files/dml_update.slt
create mode 100644 datafusion/sqllogictest/test_files/null_aware_anti_join.slt
create mode 100644
datafusion/sqllogictest/test_files/repartition_subset_satisfaction.slt
create mode 100644 datafusion/sqllogictest/test_files/run_end_encoded.slt
create mode 100644 datafusion/sqllogictest/test_files/sort_pushdown.slt
create mode 100644
datafusion/sqllogictest/test_files/spark/aggregate/collect.slt
create mode 100644 datafusion/sqllogictest/test_files/spark/collection/size.slt
copy datafusion/sqllogictest/test_files/spark/string/{char.slt => space.slt}
(80%)
create mode 100644 datafusion/sqllogictest/test_files/to_timestamp_timezone.slt
create mode 100644 dev/changelog/52.0.0.md
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
