This is an automated email from the ASF dual-hosted git repository.
alamb pushed a change to branch string-view
in repository https://gitbox.apache.org/repos/asf/datafusion.git
from 9e6cd31a1f Improve filter predicates with `Utf8View` literals (#11043)
add a8847e1a82 fix: Support `NOT <field> IN (<subquery>)` via anti join
(#10936)
add ac161bba33 fix: CTEs defined in a subquery can escape their scope
(#10954)
add 0c177d18dc Use shorter aliases in CSE (#10939)
add b26c1b819d fix: Fix the incorrect null joined rows for SMJ outer join
with join filter (#10892)
add 500b73f996 Substrait support for ParquetExec round trip for simple
select (#10949)
add 41a788238c Support to unparse `ScalarValue::IntervalMonthDayNano` to
String (#10956)
add a873f51563 Convert `StringAgg` to UDAF (#10945)
add a2c9d1a8ba Minor: Return option from row_group_row_count (#10973)
add e9f9a239ae Minor: Add routine to debug join fuzz tests (#10970)
add fbf793434e Support to unparse `ScalarValue::TimestampNanosecond` to
String (#10984)
add 2a49d61658 build(deps-dev): bump ws in
/datafusion/wasmtest/datafusion-wasm-app (#10988)
add 80f4322429 Minor: reuse Rows buffer in GroupValuesRows (#10980)
add 5cb1917f8c Add example for writing SQL analysis using DataFusion
structures (#10938)
add 8fda4a6163 feat(optimizer): handle partial anchored regex cases and
improve doc (#10977)
add 4109f581ce Push down filter for Unnest plan (#10974)
add 268f648db9 Minor: add parquet page stats for float{16, 32, 64} (#10982)
add ea0ba99d94 Fix `file_stream_provider` example compilation failure on
windows (#10975)
add c6b2efccf6 Stop copying LogicalPlan and Exprs in
`CommonSubexprEliminate` (2-3% planning speed improvement) (#10835)
add 0f80b9261f feat: Update documentation link in physical optimizer
module (#11002)
add 61e2ddbf29 fix push down logic for unnest (#10991)
add 1e7c38b4f0 Minor: add test for pushdown past unnest (#11017)
add 5bfc11ba4a Update docs for `protoc` minimum installed version (#11006)
add 89def2c6e5 Convert `bool_and` & `bool_or` to UDAF (#11009)
add 58d23c5c05 feat: support uint data page extraction (#11018)
add 5316278cea propagate error instead of panicking on out of bounds in
physical-expr/src/analysis.rs (#10992)
add 1155b0b15e Minor: Add more docs and examples for `Transformed` and
`TransformedResult` (#11003)
add 18042fd691 feat: propagate EmptyRelation for more join types (#10963)
add 1f3ba116a4 doc: Update links in the documantation (#11044)
add 5498a02853 Add drop_columns to dataframe api (#11010)
add fd5a68f802 Push down filter plan for non-unnest column (#11019)
add 4a0c7f35a0 Consider timezones with `UTC` and `+00:00` to be the same
(#10960)
add 6dffc53e76 Deprecate `OptimizerRule::try_optimize` (#11022)
add 098ba30ce5 Relax combine partial final rule (#10913)
add 8aad936e3b Compute gcd with u64 instead of i64 because of overflows
(#11036)
add 30a6ed557d Add distinct_on to dataframe api (#11012)
add ce4940d0c8 chore: add test to show current behavior of string to
timezone vs. timestamp to timezone (#11056)
add 4916e891c2 Boolean parquet get datapage stat (#11054)
add a4799c093c Using display_name for Expr::Aggregation (#11020)
add 569be9eb1b Minor: Convert `Count`'s name to lowercase (#11028)
add accd75b49a Minor: Move `function::Hint` to `datafusion-expr` crate to
avoid physical-expr dependency for `datafusion-function` crate (#11061)
add 81611ad2c5 Support to unparse ScalarValue::TimestampMillisecond to
String (#11046)
add 8a98307f2a support to unparse interval to string (#11065)
add a22423d526 feat: Add method to add analyzer rules to SessionContext
(#10849)
add 6c0e4fb5d9 SMJ: fix streaming row concurrency issue for LEFT SEMI
filtered join (#11041)
add ea46e82088 Add `advanced_parquet_index.rs` example of index in into
parquet files (#10701)
add 98373ab5af Add Expr::column_refs to find column references without
copying (#10948)
add 9f8b731827 Give `OptimizerRule::try_optimize` default implementation
and cleanup duplicated custom implementations (#11059)
add 08e4e6ad02 Fix `FormatOptions::CSV` propagation (#10912)
add 6f10dbc124 Support parsing SQL strings to Exprs (#10995)
add 8d8dd9075d Support dictionary data type in array_to_string (#10908)
add c2ea6b34aa Implement min/max for interval types (#11015)
add fdd1e3db71 Convert Average to UDAF #10942 (#10964)
add a19fc621d1 fix: remove the Sized requirement on ExecutionPlan::name()
(#11047)
add b872080989 Improve LIKE performance for Dictionary arrays (#11058)
add 8aad208299 handle overflow in gcd and return this as an error (#11057)
add d32747d09a Convert Correlation to UDAF (#11064)
add 3051d1928e Migrate more code from `Expr::to_columns` to
`Expr::column_refs` (#11067)
add c7ac8b8221 Minor: Examples cleanup + more docs in pruning example
(#11086)
add 3ff0bfe35d feat: Support duplicate column names in Joins in Substrait
consumer (#11049)
add f0ef0e6f6c decimal support for unparser (#11092)
add ede5598d55 Improve `CommonSubexprEliminate` identifier management
(10% faster planning) (#10473)
add a202a0170a Change wildcard qualifier type from `String` to
`TableReference` (#11073)
add 528c4ab6ad Allow access to UDTF in `SessionContext` (#11071)
add ed7c884d64 Strip table qualifiers from schema in `UNION ALL` for
unparser (#11082)
add 459afbb3a1 Update ListingTable to use StatisticsConverter (#11068)
add e26601811f to_timestamp functions should preserve timezone (#11038)
add 31daf25576 Rewrite array operator to function in parser (#11101)
add 8b244eee3a Resolve empty relation opt for join types (#11066)
add d44c7f2ac3 Add composed extension codec example (#11095)
add 58e2904dde Minor: Avoid some repetition in to_timestamp (#11116)
add 49aee89796 : (#11110)
add 83c80a9d6b fix: ScalarValue::new_ten error cites one not ten (#11126)
add db64743db1 Deprecate Expr::column_refs (#11115)
add 5f02c8a065 Return `&Arc` reference to inner trait object (#11103)
add ec3c71dc6c Overflow in negate operator (#11084)
add 26b646f3dd Minor: Add Architectural Goals to the docs (#11109)
add 1daf007462 fix: gcd returns negative results (#11099)
add 7adc940245 Fix overflow in pow (#11124)
add dd56dbe67c feat: Add support for Timestamp data types in data page
statistics. (#11123)
add 82f7bf40ad Support to unparse Time scalar value to String (#11121)
add 550c936de9 Support to unparse `TimestampSecond` and
`TimestampMicrosecond` to String (#11120)
add aff777b668 Add standalone example for `OptimizerRule` (#11087)
add f6f63b97cd Fix overflow in factorial (#11134)
add 7e49ccf3dd Temporary Fix: Query error when grouping by case
expressions (#11133)
add e9e2951abb Fix nullability of array_agg. (#11093)
add 2d1e8505ea fix: LCM panicked due to overflow (#11131)
add ff116c3da6 Support filter for List (#11091)
add 8216e32e87 [MINOR]: Fix some minor silent bugs (#11127)
add b468ba7883 remove `derive(Copy)` from `Operator` (#11132)
add 0c4e4a1795 Minor Fix for Logical and Physical Expr Conversions (#11142)
add ad56b7ef6f Support Date Parquet Data Page Statistics (#11135)
add 7dcef2251f fix flaky array query slt test (#11140)
add 4d1665550f Fix running in Docker instructions (#11141)
add 64b8eeafde feat: Add support for
`Binary`/`LargeBinary`/`Utf8`/`LargeUtf8` data types in data page statistics
(#11136)
add 10948ca2f0 Support Decimal and Decimal256 Parquet Data Page Statistics
(#11138)
add f58df32753 feat: Support Map type in Substrait conversions (#11129)
add d2ff2189df Implement comparisons on nested data types such that
distinct/except would work (#11117)
add acadfbf25f Minor: dont panic with bad arguments to round (#10899)
add 5501e8eb32 Support COPY TO Externally Defined File Formats, add
FileType trait (#11060)
add 57280e42dc Minor: reduce replication for nested comparison (#11149)
add 838e0f7504 Initial commit (#11158)
add 3bd720033e expose table name in proto extension codec (#11139)
add ca9c322ab3 adding config to control Varchar behavior (#11090)
add 47db63fb02 minor: consolidate `gcd` related tests (#11164)
add f1360b8486 Minor: move batch spilling methods to `lib.rs` to make it
reusable (#11154)
add 09b3c7342c Move schema projection to where it's used in ListingTable
(#11167)
add 330ece8e43 feat: Conditionally allow to keep partition_by columns when
using PARTITIONED BY enhancement (#11107)
add 7a7797c891 Make running in docker instruction be copy-pastable (#11148)
add c80da91e0f fix(typo): unqualifed to unqualified (#11159)
add 14d39734dc Rewrite `array @> array` and `array <@ array` in
sql_expr_to_logical_expr (#11155)
add 27d3aa6a62 Minor: make some physical_optimizer rules public (#11171)
add 1164a37262 Remove pr_benchmarks.yml (#11165)
add 90145dfc6c Optionally display schema in explain plan (#11177)
add 61ba6550e5 fix: Support dictionary type in parquet metadata
statistics. (#11169)
add e52b5e581b fix: Ignore nullability in Substrait structs (#11130)
add d19487c968 Minor: Add more support for ScalarValue::Float16 (#11156)
add 78055fe13b Minor: fix SQLOptions::with_allow_ddl comments (#11166)
add a64df83502 fix doc (#11181)
add 14696a0500 Update sqllogictest requirement from 0.20.0 to 0.21.0
(#11189)
add d624c0d03e Support Time Parquet Data Page Statistics (#11187)
add e40c8a8915 fix: Support Substrait's compound names also for window
functions (#11163)
add 65006b2a81 Adds support for Dictionary statistics from parquet data
pages. (#11195)
add 9fc53121a3 Make function public (#11191)
add 48a1754b33 Introduce user defined SQL planner API (#11180)
add 09cdb7834d Consolidate `Filter::remove_aliases` into
`Expr::unalias_nested` (#11001)
add ab8761d8b9 docs: add example for custom file format with `COPY TO`
(#11174)
add 4bc322819f Covert grouping to udaf (#11147)
add 75b9c9bea2 Make statistics_from_parquet_meta a sync function (#11205)
add 43ea68208b Allow user defined SQL planners to be registered (#11208)
add 58f79e143e Recursive `unnest` (#11062)
add 1840ab5331 Document how to test examples in user guide, add some more
coverage (#11178)
add a4796fa078 Minor: Move MemoryCatalog*Provider into a module, improve
comments (#11183)
add 4f4cd81de7 Fix docs wordings (#11226)
add 3421b52605 Add standalone example of using the SQL frontend (#11088)
add b76c1b7050 Add Optimizer Sanity Checker, improve sortedness
equivalence properties (#11196)
add 03c8db0a98 fix: Incorrect LEFT JOIN evaluation result on OR conditions
(#11203)
add 9d48045654 Implement user defined planner for extract (#11215)
add 03848c52f0 Move basic SQL query examples to user guide (#11217)
add dc7535a598 Support FixedSizedBinaryArray Parquet Data Page Statistics
(#11200)
add c049a94d0e Implement ScalarValue::Map (#11224)
add 699356178c fix: Be more lenient in interpreting input args for builtin
window functions (#11199)
add 4aa584c677 Remove unmaintained python pre-commit configuration (#11255)
add 4615a2d56c Enable clone_on_ref_ptr clippy lint on execution crate
(#11239)
add c6eee61918 Minor: Improve documentation about pushdown join predicates
(#11209)
add 0922d4a624 Minor: clean up data page statistics tests and fix bugs
(#11236)
add 1ffe0535ad Replacing pattern matching through downcast with trait
method (#11257)
add a753c373ee Update substrait requirement from 0.34.0 to 0.35.0 (#11206)
add 5bdc7454d9 Enhance short circuit handling in `CommonSubexprEliminate`
(#11197)
add b4afa182e7 Add bench for data page statistics parquet extraction
(#10950)
add ecc1c01f76 Register SQL planners in `SessionState` constructor (#11253)
add fe66daaa81 Support DuckDB style struct syntax (#11214)
add 6e63748818 Enable clone_on_ref_ptr clippy lint on expr crate (#11238)
add b46d5b7fc6 Optimize PushDownFilter to avoid recreating schema columns
(#11211)
add 351e5f9564 Remove outdated `rewrite_expr.rs` example (#11085)
add 0d2525e6ea Implement TPCH substrait integration teset, support tpch_2
(#11234)
add 9355f4a5c7 Enable `clone_on_ref_ptr` clippy lint on physical-expr
crate (#11240)
add dce77db316 Add standalone `AnalyzerRule` example that implements row
level access control (#11089)
add df999d6675 Replace println! with assert! if possible in DataFusion
examples (#11237)
add a0dd0a1442 minor: format `Expr::get_type()` (#11267)
add 7df000a333 Fix hash join for nested types (#11232)
add 13cb65e441 Infer count() aggregation is not null (#11256)
add 1b3a7af673 Fix count() docs around including null values (#11293)
add 5657886121 Remove unnecessary qualified names (#11292)
add 682fc05452 Fix running examples readme (#11225)
add 6f86bfad2f feat: enable "substring" as a UDF in addition to "substr"
(#11277)
add 00bbb42c96 fix: correctly handle Substrait windows with rows bounds
(and validate executability of test plans) (#11278)
add b9fdc53ac8 Minor: Add `ConstExpr`::from, use in places (#11283)
add a3e1c3d055 Implement TPCH substrait integration teset, support tpch_3
(#11298)
add 2af3d3a55b Implement user defined planner for position (#11243)
add 08c5345e93 Upgrade to arrow 52.1.0 (and fix clippy issues on main)
(#11302)
add e693ed7a3c AggregateExec: Take grouping sets into account for
InputOrderMode (#11301)
add 9f8ba6ab68 Add user_defined_sql_planners(..) to FunctionRegistry
(#11296)
add 45599ce310 use safe cast in propagate_constraints (#11297)
add 5aa7c4ae79 Minor: Remove clone in optimizer (#11315)
add 229c1398d6 minor: Add `PhysicalSortExpr::new` (#11310)
add 6f330c98b3 Fix data page statistics when all rows are null in a data
page (#11295)
add 940efd3b42 Made userDefinedFunctionPlanner to uniform the usages
(#11318)
add 1e39a8575b Implement user defined planner for `create_struct` &
`create_named_struct` (#11273)
add c254b8bb88 Improve stats convert performance for Binary/String/Boolean
arrays (#11319)
add 2f02c43b62 Fix typos in datafusion-examples/datafusion-cli/docs
(#11259)
add 1ea6545868 Minor: Fix Failing TPC-DS Test (#11331)
add 8b8452298d HashJoin can preserve the right ordering when join type is
Right (#11276)
add 8891be4694 Update substrait requirement from 0.35.0 to 0.36.0 (#11328)
add 2bd14cf62c Improve timestamp predicates support (#11326)
add f5e114e4f7 Implement user defined planner for sql_substring_to_expr
(#11327)
add b6281b54bb Improve volatile expression handling in
`CommonSubexprEliminate` (#11265)
add 0c02cad2b5 Support `IS NULL` and `IS NOT NULL` on Unions (#11321)
add 4ac1428025 Convert `nth_value` to UDAF (#11287)
add e4b54f6eae Implement TPCH substrait integration test, support tpch_4
and tpch_5 (#11311)
add 37428bb203 Enable `clone_on_ref_ptr` clippy lint on physical-plan
crate (#11241)
add 894a8794d1 fix: When consuming Substrait, temporarily rename clashing
duplicate columns (#11329)
add 99911449bc Remove any aliases in `Filter::try_new` rather than
erroring (#11307)
add 8ae56fc2b8 Improve `DataFrame` Users Guide (#11324)
add 782df39007 chore: Rename UserDefinedSQLPlanner to ExprPlanner (#11338)
add a6898d36c4 Revert "remove `derive(Copy)` from `Operator` (#11132)"
(#11341)
add 4123ad6ca7 Prepare 40.0.0 Release (#11343)
add fa0191772e Support `NULL` literals in where clause (#11266)
add e65c3e9198 Minor: Add link to blog to main DataFusion website (#11356)
add 3d1792fb9f Implement TPCH substrait integration test, support tpch_6,
tpch_10, tpch_11 (#11349)
add 6cca0f7f37 Fix bug when pushing projection under joins (#11333)
add 7f25d9dd63 Minor: some cosmetics in `filter.rs`, fix clippy due to
logical conflict (#11368)
add 9df393ea55 Update prost-derive requirement from 0.12 to 0.13 (#11355)
add c018c74ae1 Minor: update dashmap (#11335)
add 1e0c06e14a Improve and test dataframe API examples in docs (#11290)
add 16a3148354 Remove redundant `unalias_nested` calls for creating
Filter's (#11340)
add 146b679aa1 Enable `clone_on_ref_ptr` clippy lint on optimizer (#11346)
add 9a8f8b7188 fix: Fix eq properties regression from #10434 (#11363)
add d99002cf0b Update termtree requirement from 0.4.1 to 0.5.0 (#11383)
add b96186fdef Introduce `resources_err!` error macro (#11374)
add 585504a31f Enable clone_on_ref_ptr clippy lint on common (#11384)
add 6038f4cfac Track parquet writer encoding memory usage on MemoryPool
(#11345)
add 32cb3c5a54 Minor: remove clones and unnecessary Arcs in
`from_substrait_rex` (#11337)
add cc7484e0b7 Minor: Change no-statement error message to be clearer
(#11394)
add d522d5bf39 Merge remote-tracking branch 'apache/main' into string-view
add 2f0a7eceda Remove arrow-patch
No new revisions were added by this update.
Summary of changes:
.github/workflows/pr_benchmarks.yml | 101 --
.github/workflows/rust.yml | 2 +-
.pre-commit-config.yaml | 69 -
Cargo.toml | 74 +-
ci/scripts/rust_example.sh | 2 +-
datafusion-cli/Cargo.lock | 665 +++-----
datafusion-cli/Cargo.toml | 25 +-
datafusion-cli/Dockerfile | 13 +-
datafusion-cli/src/catalog.rs | 9 +-
datafusion-cli/src/exec.rs | 31 +-
datafusion-cli/src/helper.rs | 4 +-
datafusion-cli/src/main.rs | 2 +-
datafusion-examples/Cargo.toml | 4 +-
datafusion-examples/README.md | 30 +-
.../examples/advanced_parquet_index.rs | 664 ++++++++
datafusion-examples/examples/advanced_udaf.rs | 4 +-
datafusion-examples/examples/advanced_udwf.rs | 2 +-
datafusion-examples/examples/analyzer_rule.rs | 200 +++
datafusion-examples/examples/avro_sql.rs | 51 -
datafusion-examples/examples/catalog.rs | 2 +-
.../examples/composed_extension_codec.rs | 291 ++++
datafusion-examples/examples/csv_sql.rs | 70 -
datafusion-examples/examples/custom_file_format.rs | 234 +++
datafusion-examples/examples/dataframe_subquery.rs | 1 +
datafusion-examples/examples/expr_api.rs | 8 +-
.../external_dependency/dataframe-to-s3.rs | 6 +-
.../examples/file_stream_provider.rs | 326 ++--
datafusion-examples/examples/optimizer_rule.rs | 220 +++
datafusion-examples/examples/parquet_index.rs | 6 +-
datafusion-examples/examples/parquet_sql.rs | 51 -
datafusion-examples/examples/parse_sql_expr.rs | 157 ++
datafusion-examples/examples/pruning.rs | 5 +
datafusion-examples/examples/rewrite_expr.rs | 255 ----
datafusion-examples/examples/simple_udwf.rs | 2 +-
.../examples/simplify_udaf_expression.rs | 38 +-
.../examples/simplify_udwf_expression.rs | 10 +-
datafusion-examples/examples/sql_analysis.rs | 309 ++++
datafusion-examples/examples/sql_frontend.rs | 207 +++
datafusion/common-runtime/src/lib.rs | 2 +
datafusion/common/src/column.rs | 7 +
datafusion/common/src/config.rs | 108 +-
datafusion/common/src/dfschema.rs | 27 +-
datafusion/common/src/display/mod.rs | 14 +-
datafusion/common/src/error.rs | 3 +
datafusion/common/src/file_options/csv_writer.rs | 6 +
datafusion/common/src/file_options/file_type.rs | 116 +-
datafusion/common/src/file_options/mod.rs | 12 +-
datafusion/common/src/hash_utils.rs | 21 +-
datafusion/common/src/lib.rs | 6 +-
datafusion/common/src/scalar/mod.rs | 595 ++++++--
datafusion/common/src/scalar/struct_builder.rs | 2 +-
datafusion/common/src/tree_node.rs | 88 +-
datafusion/common/src/utils/mod.rs | 18 +-
datafusion/core/Cargo.toml | 2 +-
datafusion/core/benches/parquet_statistic.rs | 167 +-
datafusion/core/benches/sql_planner.rs | 2 +-
datafusion/core/src/catalog/information_schema.rs | 2 +-
datafusion/core/src/catalog/listing_schema.rs | 2 +-
datafusion/core/src/catalog/memory.rs | 352 +++++
datafusion/core/src/catalog/mod.rs | 193 +--
datafusion/core/src/catalog/schema.rs | 155 +-
datafusion/core/src/dataframe/mod.rs | 431 +++++-
datafusion/core/src/dataframe/parquet.rs | 19 +-
.../core/src/datasource/file_format/arrow.rs | 57 +-
datafusion/core/src/datasource/file_format/avro.rs | 56 +
datafusion/core/src/datasource/file_format/csv.rs | 92 +-
.../file_format/file_compression_type.rs | 83 +-
datafusion/core/src/datasource/file_format/json.rs | 76 +-
datafusion/core/src/datasource/file_format/mod.rs | 92 +-
.../core/src/datasource/file_format/parquet.rs | 579 ++++---
.../core/src/datasource/file_format/write/demux.rs | 13 +-
.../datasource/file_format/write/orchestration.rs | 1 +
datafusion/core/src/datasource/function.rs | 5 +
datafusion/core/src/datasource/listing/helpers.rs | 2 +-
datafusion/core/src/datasource/listing/table.rs | 103 +-
.../core/src/datasource/listing_table_factory.rs | 41 +-
.../core/src/datasource/physical_plan/csv.rs | 24 +-
.../core/src/datasource/physical_plan/json.rs | 14 +-
.../core/src/datasource/physical_plan/mod.rs | 2 +
.../physical_plan/parquet/access_plan.rs | 5 +
.../src/datasource/physical_plan/parquet/mod.rs | 20 +-
.../datasource/physical_plan/parquet/row_filter.rs | 25 +-
.../datasource/physical_plan/parquet/row_groups.rs | 26 +-
.../datasource/physical_plan/parquet/statistics.rs | 607 +++++++-
datafusion/core/src/datasource/statistics.rs | 21 +-
datafusion/core/src/execution/context/csv.rs | 4 +-
datafusion/core/src/execution/context/mod.rs | 91 +-
datafusion/core/src/execution/session_state.rs | 230 ++-
datafusion/core/src/lib.rs | 143 +-
.../src/physical_optimizer/aggregate_statistics.rs | 2 +-
.../combine_partial_final_agg.rs | 66 +-
datafusion/core/src/physical_optimizer/mod.rs | 6 +-
.../core/src/physical_optimizer/optimizer.rs | 18 +-
.../src/physical_optimizer/output_requirements.rs | 4 +-
.../src/physical_optimizer/pipeline_checker.rs | 334 ----
.../src/physical_optimizer/projection_pushdown.rs | 50 +-
datafusion/core/src/physical_optimizer/pruning.rs | 26 +-
.../core/src/physical_optimizer/sanity_checker.rs | 666 ++++++++
.../core/src/physical_optimizer/sort_pushdown.rs | 31 +-
.../src/physical_optimizer/update_aggr_exprs.rs | 1 +
datafusion/core/src/physical_planner.rs | 140 +-
datafusion/core/src/test/mod.rs | 23 +-
datafusion/core/src/test_util/mod.rs | 40 +
datafusion/core/tests/custom_sources_cases/mod.rs | 6 +-
.../provider_filter_pushdown.rs | 4 +
.../core/tests/custom_sources_cases/statistics.rs | 4 +
datafusion/core/tests/data/double_quote.csv | 5 +
datafusion/core/tests/dataframe/mod.rs | 186 ++-
datafusion/core/tests/expr_api/mod.rs | 1 +
datafusion/core/tests/expr_api/parse_sql_expr.rs | 93 ++
datafusion/core/tests/fifo/mod.rs | 59 +-
datafusion/core/tests/fuzz_cases/join_fuzz.rs | 282 +++-
.../fuzz_cases/sort_preserving_repartition_fuzz.rs | 6 +-
datafusion/core/tests/memory_limit/mod.rs | 25 +
datafusion/core/tests/parquet/arrow_statistics.rs | 405 +++--
datafusion/core/tests/parquet/mod.rs | 15 +
datafusion/core/tests/path_partition.rs | 4 +-
datafusion/core/tests/sql/aggregates.rs | 38 +-
datafusion/core/tests/sql/explain_analyze.rs | 8 +-
datafusion/core/tests/sql/joins.rs | 2 +-
datafusion/core/tests/sql/select.rs | 2 +-
datafusion/core/tests/sql/sql_api.rs | 34 +
datafusion/core/tests/tpcds_planning.rs | 5 +-
datafusion/core/tests/user_defined/expr_planner.rs | 123 ++
datafusion/core/tests/user_defined/mod.rs | 3 +
.../tests/user_defined/user_defined_aggregates.rs | 3 +-
.../core/tests/user_defined/user_defined_plan.rs | 142 +-
.../user_defined/user_defined_scalar_functions.rs | 15 +-
.../user_defined/user_defined_table_functions.rs | 15 +
datafusion/execution/src/cache/cache_manager.rs | 4 +-
datafusion/execution/src/cache/cache_unit.rs | 6 +-
datafusion/execution/src/disk_manager.rs | 8 +-
datafusion/execution/src/lib.rs | 2 +
datafusion/execution/src/memory_pool/mod.rs | 4 +-
datafusion/execution/src/memory_pool/pool.rs | 5 +-
datafusion/execution/src/object_store.rs | 2 +-
datafusion/execution/src/task.rs | 21 +-
datafusion/expr/src/aggregate_function.rs | 87 +-
datafusion/expr/src/expr.rs | 332 +++-
datafusion/expr/src/expr_fn.rs | 32 +-
datafusion/expr/src/expr_rewriter/mod.rs | 3 +-
datafusion/expr/src/expr_rewriter/order_by.rs | 8 +-
datafusion/expr/src/expr_schema.rs | 102 +-
datafusion/expr/src/function.rs | 8 +
datafusion/expr/src/lib.rs | 4 +
datafusion/expr/src/logical_plan/builder.rs | 49 +-
datafusion/expr/src/logical_plan/display.rs | 4 +-
datafusion/expr/src/logical_plan/dml.rs | 6 +-
datafusion/expr/src/logical_plan/plan.rs | 149 +-
datafusion/expr/src/logical_plan/tree_node.rs | 4 +-
datafusion/expr/src/planner.rs | 207 +++
datafusion/expr/src/registry.rs | 16 +
datafusion/expr/src/table_source.rs | 19 +-
datafusion/expr/src/test/function_stub.rs | 80 +-
datafusion/expr/src/tree_node.rs | 3 +-
datafusion/expr/src/type_coercion/aggregates.rs | 139 +-
datafusion/expr/src/type_coercion/binary.rs | 42 +-
datafusion/expr/src/type_coercion/functions.rs | 12 +-
datafusion/expr/src/udaf.rs | 13 +-
datafusion/expr/src/udf.rs | 12 +-
datafusion/expr/src/udwf.rs | 10 +-
datafusion/expr/src/utils.rs | 24 +-
.../functions-aggregate/src/approx_distinct.rs | 2 +-
.../src/approx_percentile_cont.rs | 2 +-
.../src}/average.rs | 408 +++--
.../functions-aggregate/src/bit_and_or_xor.rs | 10 +-
datafusion/functions-aggregate/src/bool_and_or.rs | 343 +++++
datafusion/functions-aggregate/src/correlation.rs | 225 +++
datafusion/functions-aggregate/src/count.rs | 29 +-
datafusion/functions-aggregate/src/grouping.rs | 97 ++
datafusion/functions-aggregate/src/lib.rs | 23 +-
datafusion/functions-aggregate/src/median.rs | 4 +-
.../src}/nth_value.rs | 213 +--
datafusion/functions-aggregate/src/string_agg.rs | 153 ++
datafusion/functions-aggregate/src/sum.rs | 2 +-
datafusion/functions-array/Cargo.toml | 1 +
datafusion/functions-array/src/lib.rs | 4 +-
datafusion/functions-array/src/make_array.rs | 4 +-
datafusion/functions-array/src/planner.rs | 160 ++
datafusion/functions-array/src/rewrite.rs | 181 ---
datafusion/functions-array/src/string.rs | 29 +-
datafusion/functions-array/src/utils.rs | 12 +-
datafusion/functions/Cargo.toml | 1 -
datafusion/functions/src/core/mod.rs | 2 +-
datafusion/functions/src/core/planner.rs | 59 +
datafusion/functions/src/datetime/to_timestamp.rs | 150 +-
datafusion/functions/src/lib.rs | 3 +
datafusion/functions/src/math/factorial.rs | 44 +-
datafusion/functions/src/math/gcd.rs | 73 +-
datafusion/functions/src/math/lcm.rs | 50 +-
datafusion/functions/src/math/power.rs | 33 +-
datafusion/functions/src/math/round.rs | 109 +-
datafusion/functions/src/planner.rs | 51 +
datafusion/functions/src/regex/regexpreplace.rs | 2 +-
datafusion/functions/src/string/btrim.rs | 2 +-
datafusion/functions/src/string/ltrim.rs | 2 +-
datafusion/functions/src/string/rtrim.rs | 2 +-
datafusion/functions/src/unicode/substr.rs | 6 +
datafusion/functions/src/utils.rs | 2 +-
datafusion/optimizer/README.md | 6 +-
.../optimizer/src/analyzer/count_wildcard_rule.rs | 40 +-
datafusion/optimizer/src/analyzer/mod.rs | 2 +-
datafusion/optimizer/src/analyzer/subquery.rs | 28 +-
datafusion/optimizer/src/analyzer/type_coercion.rs | 64 +-
.../optimizer/src/common_subexpr_eliminate.rs | 1600 +++++++++++++-------
datafusion/optimizer/src/decorrelate.rs | 28 +-
.../src/decorrelate_predicate_subquery.rs | 98 +-
datafusion/optimizer/src/eliminate_cross_join.rs | 14 +-
.../optimizer/src/eliminate_duplicated_expr.rs | 10 +-
datafusion/optimizer/src/eliminate_filter.rs | 13 +-
.../optimizer/src/eliminate_group_by_constant.rs | 30 +-
datafusion/optimizer/src/eliminate_join.rs | 10 +-
datafusion/optimizer/src/eliminate_limit.rs | 13 +-
datafusion/optimizer/src/eliminate_nested_union.rs | 46 +-
datafusion/optimizer/src/eliminate_one_union.rs | 15 +-
datafusion/optimizer/src/eliminate_outer_join.rs | 25 +-
.../optimizer/src/extract_equijoin_predicate.rs | 13 +-
datafusion/optimizer/src/filter_null_join_keys.rs | 10 +-
datafusion/optimizer/src/lib.rs | 3 +
.../optimizer/src/optimize_projections/mod.rs | 46 +-
.../src/optimize_projections/required_indices.rs | 4 +-
datafusion/optimizer/src/optimizer.rs | 63 +-
datafusion/optimizer/src/plan_signature.rs | 4 +-
.../optimizer/src/propagate_empty_relation.rs | 258 +++-
datafusion/optimizer/src/push_down_filter.rs | 374 +++--
datafusion/optimizer/src/push_down_limit.rs | 10 +-
.../optimizer/src/replace_distinct_aggregate.rs | 10 +-
.../optimizer/src/rewrite_disjunctive_predicate.rs | 9 -
.../optimizer/src/scalar_subquery_to_join.rs | 10 +-
.../src/simplify_expressions/expr_simplifier.rs | 25 +-
.../optimizer/src/simplify_expressions/regex.rs | 68 +-
.../src/simplify_expressions/simplify_exprs.rs | 10 +-
.../optimizer/src/single_distinct_to_groupby.rs | 78 +-
datafusion/optimizer/src/test/mod.rs | 41 +-
.../optimizer/src/unwrap_cast_in_comparison.rs | 12 +-
datafusion/optimizer/src/utils.rs | 36 +-
.../optimizer/tests/optimizer_integration.rs | 33 +-
.../src/aggregate/count_distinct/bytes.rs | 4 +-
.../src/aggregate/count_distinct/native.rs | 6 +-
.../src/aggregate/merge_arrays.rs | 195 +++
.../physical-expr-common/src/aggregate/mod.rs | 15 +
.../physical-expr-common/src/aggregate/tdigest.rs | 2 +-
datafusion/physical-expr-common/src/binary_map.rs | 4 +-
datafusion/physical-expr-common/src/datum.rs | 147 ++
.../physical-expr-common/src/expressions/cast.rs | 100 +-
datafusion/physical-expr-common/src/lib.rs | 1 +
datafusion/physical-expr-common/src/sort_expr.rs | 23 +-
datafusion/physical-expr-common/src/utils.rs | 5 +-
datafusion/physical-expr/Cargo.toml | 1 -
.../physical-expr/src/aggregate/array_agg.rs | 21 +-
.../src/aggregate/array_agg_distinct.rs | 17 +-
.../src/aggregate/array_agg_ordered.rs | 219 +--
.../physical-expr/src/aggregate/bool_and_or.rs | 394 -----
datafusion/physical-expr/src/aggregate/build_in.rs | 210 +--
.../physical-expr/src/aggregate/correlation.rs | 524 -------
.../physical-expr/src/aggregate/covariance.rs | 227 ---
datafusion/physical-expr/src/aggregate/grouping.rs | 103 --
.../src/aggregate/groups_accumulator/mod.rs | 4 +-
datafusion/physical-expr/src/aggregate/min_max.rs | 151 +-
datafusion/physical-expr/src/aggregate/mod.rs | 9 -
datafusion/physical-expr/src/aggregate/stddev.rs | 87 --
.../physical-expr/src/aggregate/string_agg.rs | 246 ---
datafusion/physical-expr/src/aggregate/variance.rs | 176 ---
datafusion/physical-expr/src/analysis.rs | 14 +-
datafusion/physical-expr/src/equivalence/class.rs | 130 +-
datafusion/physical-expr/src/equivalence/mod.rs | 32 +-
.../physical-expr/src/equivalence/ordering.rs | 26 +-
.../physical-expr/src/equivalence/projection.rs | 59 +-
.../physical-expr/src/equivalence/properties.rs | 447 ++++--
datafusion/physical-expr/src/expressions/binary.rs | 74 +-
datafusion/physical-expr/src/expressions/case.rs | 36 +-
datafusion/physical-expr/src/expressions/datum.rs | 58 -
.../physical-expr/src/expressions/in_list.rs | 120 +-
.../physical-expr/src/expressions/is_not_null.rs | 56 +-
.../physical-expr/src/expressions/is_null.rs | 131 +-
datafusion/physical-expr/src/expressions/like.rs | 16 +-
datafusion/physical-expr/src/expressions/mod.rs | 9 -
.../physical-expr/src/expressions/negative.rs | 2 +-
datafusion/physical-expr/src/expressions/not.rs | 2 +-
.../physical-expr/src/expressions/try_cast.rs | 4 +-
datafusion/physical-expr/src/functions.rs | 10 +-
.../physical-expr/src/intervals/cp_solver.rs | 30 +-
.../physical-expr/src/intervals/test_utils.rs | 18 +-
datafusion/physical-expr/src/lib.rs | 4 +-
datafusion/physical-expr/src/partitioning.rs | 8 +-
datafusion/physical-expr/src/physical_expr.rs | 20 +-
datafusion/physical-expr/src/planner.rs | 40 +-
datafusion/physical-expr/src/scalar_function.rs | 2 +-
datafusion/physical-expr/src/utils/guarantee.rs | 30 +-
datafusion/physical-expr/src/utils/mod.rs | 6 +-
datafusion/physical-expr/src/window/built_in.rs | 4 +-
datafusion/physical-expr/src/window/lead_lag.rs | 6 +-
datafusion/physical-expr/src/window/nth_value.rs | 4 +-
.../physical-expr/src/window/sliding_aggregate.rs | 2 +-
datafusion/physical-expr/src/window/window_expr.rs | 6 +-
datafusion/physical-plan/Cargo.toml | 2 +-
.../src/aggregates/group_values/row.rs | 16 +-
datafusion/physical-plan/src/aggregates/mod.rs | 299 ++--
.../physical-plan/src/aggregates/no_grouping.rs | 9 +-
.../physical-plan/src/aggregates/order/partial.rs | 3 +-
.../physical-plan/src/aggregates/row_hash.rs | 16 +-
.../physical-plan/src/aggregates/topk_stream.rs | 10 +-
datafusion/physical-plan/src/analyze.rs | 16 +-
datafusion/physical-plan/src/coalesce_batches.rs | 8 +-
.../physical-plan/src/coalesce_partitions.rs | 12 +-
datafusion/physical-plan/src/common.rs | 7 +-
datafusion/physical-plan/src/display.rs | 51 +-
datafusion/physical-plan/src/empty.rs | 15 +-
datafusion/physical-plan/src/explain.rs | 6 +-
datafusion/physical-plan/src/filter.rs | 78 +-
datafusion/physical-plan/src/insert.rs | 14 +-
datafusion/physical-plan/src/joins/cross_join.rs | 14 +-
datafusion/physical-plan/src/joins/hash_join.rs | 273 +++-
.../physical-plan/src/joins/nested_loop_join.rs | 27 +-
.../physical-plan/src/joins/sort_merge_join.rs | 369 +++--
.../physical-plan/src/joins/stream_join_utils.rs | 10 +-
.../physical-plan/src/joins/symmetric_hash_join.rs | 20 +-
datafusion/physical-plan/src/joins/test_utils.rs | 51 +-
datafusion/physical-plan/src/joins/utils.rs | 147 +-
datafusion/physical-plan/src/lib.rs | 101 +-
datafusion/physical-plan/src/limit.rs | 12 +-
datafusion/physical-plan/src/memory.rs | 9 +-
datafusion/physical-plan/src/placeholder_row.rs | 13 +-
datafusion/physical-plan/src/projection.rs | 20 +-
datafusion/physical-plan/src/recursive_query.rs | 23 +-
datafusion/physical-plan/src/repartition/mod.rs | 61 +-
datafusion/physical-plan/src/sorts/builder.rs | 6 +-
datafusion/physical-plan/src/sorts/merge.rs | 3 +-
datafusion/physical-plan/src/sorts/partial_sort.rs | 53 +-
datafusion/physical-plan/src/sorts/sort.rs | 160 +-
.../src/sorts/sort_preserving_merge.rs | 50 +-
datafusion/physical-plan/src/sorts/stream.rs | 2 +-
datafusion/physical-plan/src/stream.rs | 28 +-
datafusion/physical-plan/src/streaming.rs | 6 +-
datafusion/physical-plan/src/test.rs | 5 +-
datafusion/physical-plan/src/test/exec.rs | 38 +-
datafusion/physical-plan/src/topk/mod.rs | 4 +-
datafusion/physical-plan/src/tree_node.rs | 2 +-
datafusion/physical-plan/src/union.rs | 106 +-
datafusion/physical-plan/src/unnest.rs | 36 +-
datafusion/physical-plan/src/values.rs | 10 +-
.../src/windows/bounded_window_agg_exec.rs | 62 +-
datafusion/physical-plan/src/windows/mod.rs | 87 +-
.../physical-plan/src/windows/window_agg_exec.rs | 15 +-
datafusion/physical-plan/src/work_table.rs | 8 +-
datafusion/proto-common/Cargo.toml | 2 +-
datafusion/proto-common/gen/Cargo.toml | 2 +-
.../proto-common/proto/datafusion_common.proto | 10 +-
datafusion/proto-common/src/from_proto/mod.rs | 30 +-
datafusion/proto-common/src/generated/pbjson.rs | 87 ++
datafusion/proto-common/src/generated/prost.rs | 18 +-
datafusion/proto-common/src/to_proto/mod.rs | 12 +-
datafusion/proto/CONTRIBUTING.md | 2 +-
datafusion/proto/Cargo.toml | 2 +-
datafusion/proto/gen/Cargo.toml | 2 +-
datafusion/proto/gen/src/main.rs | 1 +
datafusion/proto/proto/datafusion.proto | 27 +-
datafusion/proto/src/bytes/mod.rs | 5 +
datafusion/proto/src/bytes/registry.rs | 5 +
.../proto/src/generated/datafusion_proto_common.rs | 18 +-
datafusion/proto/src/generated/pbjson.rs | 168 +-
datafusion/proto/src/generated/prost.rs | 67 +-
datafusion/proto/src/logical_plan/file_formats.rs | 409 +++++
datafusion/proto/src/logical_plan/from_proto.rs | 23 +-
datafusion/proto/src/logical_plan/mod.rs | 49 +-
datafusion/proto/src/logical_plan/to_proto.rs | 34 +-
datafusion/proto/src/physical_plan/from_proto.rs | 23 +-
datafusion/proto/src/physical_plan/mod.rs | 12 +-
datafusion/proto/src/physical_plan/to_proto.rs | 83 +-
.../proto/tests/cases/roundtrip_logical_plan.rs | 164 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 99 +-
datafusion/sql/src/cte.rs | 11 +-
datafusion/sql/src/expr/function.rs | 6 +-
datafusion/sql/src/expr/mod.rs | 314 ++--
datafusion/sql/src/expr/subquery.rs | 6 +-
datafusion/sql/src/expr/substring.rs | 24 +-
datafusion/sql/src/expr/value.rs | 31 +-
datafusion/sql/src/parser.rs | 29 +-
datafusion/sql/src/planner.rs | 97 +-
datafusion/sql/src/query.rs | 18 +-
datafusion/sql/src/relation/join.rs | 31 +-
datafusion/sql/src/select.rs | 106 +-
datafusion/sql/src/statement.rs | 61 +-
datafusion/sql/src/unparser/expr.rs | 328 +++-
datafusion/sql/src/unparser/mod.rs | 1 +
datafusion/sql/src/unparser/plan.rs | 7 +-
datafusion/sql/src/unparser/rewrite.rs | 101 ++
datafusion/sql/src/utils.rs | 143 +-
datafusion/sql/tests/cases/plan_to_sql.rs | 14 +-
datafusion/sql/tests/common/mod.rs | 32 +-
datafusion/sql/tests/sql_integration.rs | 113 +-
datafusion/sqllogictest/Cargo.toml | 2 +-
.../test_files/agg_func_substitute.slt | 30 +-
datafusion/sqllogictest/test_files/aggregate.slt | 339 +++--
datafusion/sqllogictest/test_files/array.slt | 70 +
datafusion/sqllogictest/test_files/array_query.slt | 128 +-
datafusion/sqllogictest/test_files/avro.slt | 6 +-
datafusion/sqllogictest/test_files/cast.slt | 20 +
datafusion/sqllogictest/test_files/copy.slt | 21 +
datafusion/sqllogictest/test_files/cse.slt | 173 +++
datafusion/sqllogictest/test_files/csv_files.slt | 67 +
datafusion/sqllogictest/test_files/cte.slt | 26 +-
datafusion/sqllogictest/test_files/errors.slt | 2 +-
datafusion/sqllogictest/test_files/explain.slt | 38 +-
datafusion/sqllogictest/test_files/expr.slt | 37 +-
datafusion/sqllogictest/test_files/group_by.slt | 112 +-
.../sqllogictest/test_files/information_schema.slt | 6 +
datafusion/sqllogictest/test_files/insert.slt | 28 +-
.../sqllogictest/test_files/insert_to_external.slt | 20 +-
datafusion/sqllogictest/test_files/join.slt | 251 +++
datafusion/sqllogictest/test_files/joins.slt | 448 +++++-
datafusion/sqllogictest/test_files/json.slt | 6 +-
datafusion/sqllogictest/test_files/limit.slt | 18 +-
datafusion/sqllogictest/test_files/math.slt | 118 ++
datafusion/sqllogictest/test_files/misc.slt | 14 +
.../test_files/optimizer_group_by_constant.slt | 16 +-
datafusion/sqllogictest/test_files/predicates.slt | 4 +-
.../sqllogictest/test_files/push_down_filter.slt | 124 ++
datafusion/sqllogictest/test_files/regexp.slt | 81 +
datafusion/sqllogictest/test_files/scalar.slt | 80 +-
datafusion/sqllogictest/test_files/select.slt | 22 +-
.../sqllogictest/test_files/sort_merge_join.slt | 29 +-
datafusion/sqllogictest/test_files/strings.slt | 49 +
datafusion/sqllogictest/test_files/struct.slt | 25 +
datafusion/sqllogictest/test_files/subquery.slt | 121 +-
datafusion/sqllogictest/test_files/timestamps.slt | 43 +
.../sqllogictest/test_files/tpch/q1.slt.part | 14 +-
.../sqllogictest/test_files/tpch/q13.slt.part | 18 +-
.../sqllogictest/test_files/tpch/q16.slt.part | 10 +-
.../sqllogictest/test_files/tpch/q17.slt.part | 14 +-
.../sqllogictest/test_files/tpch/q21.slt.part | 10 +-
.../sqllogictest/test_files/tpch/q22.slt.part | 20 +-
.../sqllogictest/test_files/tpch/q4.slt.part | 10 +-
datafusion/sqllogictest/test_files/union.slt | 18 +-
datafusion/sqllogictest/test_files/unnest.slt | 97 ++
datafusion/sqllogictest/test_files/window.slt | 243 +--
datafusion/substrait/Cargo.toml | 4 +-
datafusion/substrait/src/logical_plan/consumer.rs | 768 +++++-----
datafusion/substrait/src/logical_plan/producer.rs | 89 +-
datafusion/substrait/src/physical_plan/consumer.rs | 87 +-
datafusion/substrait/src/physical_plan/producer.rs | 93 +-
.../substrait/tests/cases/consumer_integration.rs | 355 ++++-
datafusion/substrait/tests/cases/logical_plans.rs | 34 +-
.../tests/cases/roundtrip_logical_plan.rs | 176 ++-
.../tests/cases/roundtrip_physical_plan.rs | 92 +-
datafusion/substrait/tests/testdata/Readme.md | 51 +
datafusion/substrait/tests/testdata/data.csv | 4 +-
datafusion/substrait/tests/testdata/data.parquet | Bin 0 -> 4342 bytes
datafusion/substrait/tests/testdata/empty.parquet | Bin 0 -> 976 bytes
.../non_nullable_lists.substrait.json | 0
.../select_not_bool.substrait.json | 0
.../test_plans/select_window.substrait.json | 153 ++
.../substrait/tests/testdata/tpch/customer.csv | 2 +
.../substrait/tests/testdata/tpch/nation.csv | 2 +
.../substrait/tests/testdata/tpch/orders.csv | 2 +
datafusion/substrait/tests/testdata/tpch/part.csv | 2 +
.../substrait/tests/testdata/tpch/partsupp.csv | 2 +
.../substrait/tests/testdata/tpch/region.csv | 2 +
.../substrait/tests/testdata/tpch/supplier.csv | 2 +
.../testdata/tpch_substrait_plans/query_10.json | 1257 +++++++++++++++
.../testdata/tpch_substrait_plans/query_11.json | 1059 +++++++++++++
.../testdata/tpch_substrait_plans/query_2.json | 1582 +++++++++++++++++++
.../testdata/tpch_substrait_plans/query_3.json | 851 +++++++++++
.../testdata/tpch_substrait_plans/query_4.json | 540 +++++++
.../testdata/tpch_substrait_plans/query_5.json | 1254 +++++++++++++++
.../testdata/tpch_substrait_plans/query_6.json | 585 +++++++
.../wasmtest/datafusion-wasm-app/package-lock.json | 12 +-
dev/changelog/40.0.0.md | 371 +++++
docs/source/contributor-guide/getting_started.md | 4 +-
docs/source/contributor-guide/inviting.md | 2 +-
docs/source/contributor-guide/testing.md | 25 +
docs/source/index.rst | 5 +-
docs/source/library-user-guide/adding-udfs.md | 4 +-
.../library-user-guide/using-the-dataframe-api.md | 302 +++-
.../source/library-user-guide/using-the-sql-api.md | 200 +++
.../library-user-guide/working-with-exprs.md | 6 +-
docs/source/user-guide/cli/installation.md | 7 +-
docs/source/user-guide/configs.md | 5 +-
docs/source/user-guide/dataframe.md | 121 +-
docs/source/user-guide/example-usage.md | 13 +-
docs/source/user-guide/expressions.md | 4 +-
docs/source/user-guide/introduction.md | 2 +-
docs/source/user-guide/sql/aggregate_functions.md | 6 +-
docs/source/user-guide/sql/dml.md | 5 +-
docs/source/user-guide/sql/scalar_functions.md | 8 +
docs/source/user-guide/sql/write_options.md | 10 +
486 files changed, 30214 insertions(+), 11364 deletions(-)
delete mode 100644 .github/workflows/pr_benchmarks.yml
delete mode 100644 .pre-commit-config.yaml
create mode 100644 datafusion-examples/examples/advanced_parquet_index.rs
create mode 100644 datafusion-examples/examples/analyzer_rule.rs
delete mode 100644 datafusion-examples/examples/avro_sql.rs
create mode 100644 datafusion-examples/examples/composed_extension_codec.rs
delete mode 100644 datafusion-examples/examples/csv_sql.rs
create mode 100644 datafusion-examples/examples/custom_file_format.rs
create mode 100644 datafusion-examples/examples/optimizer_rule.rs
delete mode 100644 datafusion-examples/examples/parquet_sql.rs
create mode 100644 datafusion-examples/examples/parse_sql_expr.rs
delete mode 100644 datafusion-examples/examples/rewrite_expr.rs
create mode 100644 datafusion-examples/examples/sql_analysis.rs
create mode 100644 datafusion-examples/examples/sql_frontend.rs
create mode 100644 datafusion/core/src/catalog/memory.rs
delete mode 100644 datafusion/core/src/physical_optimizer/pipeline_checker.rs
create mode 100644 datafusion/core/src/physical_optimizer/sanity_checker.rs
create mode 100644 datafusion/core/tests/data/double_quote.csv
create mode 100644 datafusion/core/tests/expr_api/parse_sql_expr.rs
create mode 100644 datafusion/core/tests/user_defined/expr_planner.rs
create mode 100644 datafusion/expr/src/planner.rs
rename datafusion/{physical-expr/src/aggregate =>
functions-aggregate/src}/average.rs (76%)
create mode 100644 datafusion/functions-aggregate/src/bool_and_or.rs
create mode 100644 datafusion/functions-aggregate/src/correlation.rs
create mode 100644 datafusion/functions-aggregate/src/grouping.rs
rename datafusion/{physical-expr/src/aggregate =>
functions-aggregate/src}/nth_value.rs (75%)
create mode 100644 datafusion/functions-aggregate/src/string_agg.rs
create mode 100644 datafusion/functions-array/src/planner.rs
delete mode 100644 datafusion/functions-array/src/rewrite.rs
create mode 100644 datafusion/functions/src/core/planner.rs
create mode 100644 datafusion/functions/src/planner.rs
create mode 100644
datafusion/physical-expr-common/src/aggregate/merge_arrays.rs
create mode 100644 datafusion/physical-expr-common/src/datum.rs
delete mode 100644 datafusion/physical-expr/src/aggregate/bool_and_or.rs
delete mode 100644 datafusion/physical-expr/src/aggregate/correlation.rs
delete mode 100644 datafusion/physical-expr/src/aggregate/covariance.rs
delete mode 100644 datafusion/physical-expr/src/aggregate/grouping.rs
delete mode 100644 datafusion/physical-expr/src/aggregate/stddev.rs
delete mode 100644 datafusion/physical-expr/src/aggregate/string_agg.rs
delete mode 100644 datafusion/physical-expr/src/aggregate/variance.rs
delete mode 100644 datafusion/physical-expr/src/expressions/datum.rs
create mode 100644 datafusion/proto/src/logical_plan/file_formats.rs
create mode 100644 datafusion/sql/src/unparser/rewrite.rs
create mode 100644 datafusion/sqllogictest/test_files/cse.slt
create mode 100644 datafusion/sqllogictest/test_files/push_down_filter.slt
create mode 100644 datafusion/substrait/tests/testdata/Readme.md
create mode 100644 datafusion/substrait/tests/testdata/data.parquet
create mode 100644 datafusion/substrait/tests/testdata/empty.parquet
rename datafusion/substrait/tests/testdata/{ =>
test_plans}/non_nullable_lists.substrait.json (100%)
rename datafusion/substrait/tests/testdata/{ =>
test_plans}/select_not_bool.substrait.json (100%)
create mode 100644
datafusion/substrait/tests/testdata/test_plans/select_window.substrait.json
create mode 100644 datafusion/substrait/tests/testdata/tpch/customer.csv
create mode 100644 datafusion/substrait/tests/testdata/tpch/nation.csv
create mode 100644 datafusion/substrait/tests/testdata/tpch/orders.csv
create mode 100644 datafusion/substrait/tests/testdata/tpch/part.csv
create mode 100644 datafusion/substrait/tests/testdata/tpch/partsupp.csv
create mode 100644 datafusion/substrait/tests/testdata/tpch/region.csv
create mode 100644 datafusion/substrait/tests/testdata/tpch/supplier.csv
create mode 100644
datafusion/substrait/tests/testdata/tpch_substrait_plans/query_10.json
create mode 100644
datafusion/substrait/tests/testdata/tpch_substrait_plans/query_11.json
create mode 100644
datafusion/substrait/tests/testdata/tpch_substrait_plans/query_2.json
create mode 100644
datafusion/substrait/tests/testdata/tpch_substrait_plans/query_3.json
create mode 100644
datafusion/substrait/tests/testdata/tpch_substrait_plans/query_4.json
create mode 100644
datafusion/substrait/tests/testdata/tpch_substrait_plans/query_5.json
create mode 100644
datafusion/substrait/tests/testdata/tpch_substrait_plans/query_6.json
create mode 100644 dev/changelog/40.0.0.md
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
