This is an automated email from the ASF dual-hosted git repository.
alamb pushed a change to branch dependabot/cargo/main/itertools-0.13
in repository https://gitbox.apache.org/repos/asf/datafusion.git
from 2d8c31926b Update Cargo.lock
add b716c09c00 feat: API for collecting statistics/index for metadata of a
parquet file + tests (#10537)
add 94b5511321 Fix: Sort Merge Join LeftSemi issues when JoinFilter is set
(#10304)
add 3ebc31d2c2 Remove `Expr::GetIndexedField`, replace
`Expr::{field,index,range}` with `FieldAccessor`, `IndexAccessor`, and
`SliceAccessor` (#10568)
add 37b0112d4d Minor: Fix ArrayFunctionRewriter name (#10581)
add 21bce33da0 Improve `UserDefinedLogicalNode::from_template` API to
return `Result` (#10575)
add f80794779e Migrate testing optimizer rules to use `rewrite` API
(#10576)
add aab40e15de Improve ContextProvider (#10577)
add 96e0ee610c test: add more tests for statistics reading (#10592)
add 4b2652f403 refactor: reduce allocations in push down filter (#10567)
add 6a75ba69d4 Fix compilation of datafusion-cli on 32bit targets (#10594)
add a59a98090e Add to_date function to scalar functions doc (#10601)
add 1e58247a0a Renaming (#10596)
add 045e8fcaea Implement Unparser for UNION ALL (#10603)
add 65e281a85b Improve UserDefinedLogicalNodeCore::from_template API to
return Result (#10597)
add 2eb38bd5e0 Minor: Move group accumulator for aggregate function to
physical-expr-common, and add ahash physical-expr-common (#10574)
add 4396d5e224 Minor: Consolidate some integration tests into
`core_integration` (#10588)
add 8cffb68d0c Stop copying LogicalPlan and Exprs in
`SingleDistinctToGroupBy` (#10527)
add cde8f59269 feat: Add eliminate group by constant optimizer rule
(#10591)
add d7e642a644 Docs: Update PR workflow documentation (#10532)
add 80084135c5 Update get range for lead lag (#10614)
add e893a2eea4 Minor: Improve documentation in sql_to_plan example (#10582)
add 513d8d1d2a Docs: add examples for `RuntimeEnv::register_object_store`,
improve error messages (#10617)
add b2a31b96ce Add support for Substrait List/EmptyList literals (#10615)
add f9ef488aec Add to_unixtime function to scalar functions doc (#10620)
add 1bf7112171 Test for reading read statistics from parquet files without
statistics and boolean & struct data type (#10608)
add b14e92f300 adding benchmark for extracting arrow statistics from
parquet (#10610)
add 03557131d2 feat: extend `unnest` to support Struct datatype (#10429)
add 7bd4b53ffa Implement a dialect-specific rule for unparsing an
identifier with or without quotes (#10573)
add 7757d63432 add catalog as part of the table path in plan_to_sql
(#10612)
add 656da8337e Refactor parquet row group pruning into a struct (use new
statistics API, part 1) (#10607)
add 529d2c0229 Extract `Date32` parquet statistics as `Date32Array`
rather than `Int32Array` (#10593)
add d2f6faf567 Omit NULLS FIRST/LAST when unparsing ORDER BY clauses for
MySQL (#10625)
add ccbd01a9b8 Fix broken build/test from merge (#10637)
add b59f4af5bf Add SessionContext::register_object_store (#10621)
add c75a95780d Minor: Move median test (#10611)
add 19d9174150 Add support for Substrait Struct literals and type (#10622)
add 8f3084a750 fix Incorrect statistics read for i8 i16 columns in parquet
(#10629)
add 4b7b5ab11a Minor: add runtime asserts to RowGroup (#10641)
add 3e4e09a7b0 Update cli Dockerfile to a newer ubuntu release, newer rust
release. (#10638)
add 8bedecc00b More properly handle nullability of types/literals in
Substrait (#10640)
add 12113354ca fix wrong type validation on unnest expr (#10657)
add d10b1a479e Fix incorrect statistics read for binary columns in parquet
(#10645)
add 0f994afff2 Support field access for literal Null (#10655)
add db3dde0c3c Update substrait requirement from 0.33.3 to 0.34.0 (#10632)
add 2fc1575cfe Fix typo in Cargo.toml (#10662)
add 4709fc65f7 Add `FileScanConfig::new()` API (#10623)
add ea92ae72f7 support null for get field (#10665)
add e40f50b991 Minor: Use slice in `ConcreteTreeNode` (#10666)
add 26b44f4dad Move Median to `functions-aggregate` and Introduce Numeric
signature (#10644)
add 08365001c0 Fix `Coalesce` casting logic to follows what Postgres and
DuckDB do. Introduce signature that do non-comparison coercion (#10268)
add c095fee4a1 fix: pass `quote` parameter to CSV writer (#10671)
add 40aabd6877 Fix compilation "comparison_binary_numeric_coercion not
found" (#10677)
add 52c4f3cce6 refactor: simplify converting List DataTypes to
`ScalarValue` (#10675)
add 6167ce915e feat: add substrait support for Interval types and literals
(#10646)
add 3a795a41d6 Minor: Improve ObjectStoreUrl docs + examples (#10619)
add 9c065065df fix: CI compilation failed on substrait (#10683)
add 7db58a900c Add tests for reading numeric limits in parquet statistics
(#10642)
add 338c6e63cd Update nix requirement from 0.28.0 to 0.29.0 (#10684)
add 7c08a6fd00 refactor: Move SchemaAdapter from parquet module to data
source (#10680)
add ae107548bd Add reference visitor `TreeNode` APIs, change
`ExecutionPlan::children()` and `PhysicalExpr::children()` return references
(#10543)
add 549cf844cc Convert first, last aggregate function to UDAF (#10648)
add 2e6beb434d Minor: CastExpr Ordering Handle (#10650)
add 3194a981e0 Factor out common datafusion types into another proto file
(#10649)
add 5afa801780 Minor: Add tests showing aggregate behavior for NaNs
(#10634)
add 7f0e194e2f Improve `ParquetExec` and related documentation (#10647)
add 3dc1773403 minor: inconsistent group by position planning (#10679)
add 98cd19ef1c Remove duplicate function name in its aliases list (#10661)
add 5a9712efd7 minor: unnest protobuf serde support (#10681)
add 27627546d1 Support Substrait's VirtualTables (#10531)
add 70a215bc8a support limit in agg exec for ser/deser (#10692)
add a51d025418 display date32/64 as YYYY-MM-DD (#10691)
add 9fba34d557 Fix: array list values are leaked on nested `unnest`
operators (#10689)
add 156a525286 Support LogicalPlan::Distinct in unparser (#10690)
add 08e19f4956 Remove redundant upper case aliases for `median`,
`first_value` and `last_value` (#10696)
add 09054263df Minor: improve Expr documentation (#10685)
add 1f15152069 chore: align re-exports in functions-aggregate (#10705)
add 20060880ba Fix typo in bench.sh (#10698)
add 6543d59e3f Fix incorrect statistics read for unsigned integers columns
in parquet (#10704)
add 5bd8751ecf Separate proto partitioning (#10708)
add 80a6b65dbb Support consuming Substrait with compound signature
function names (#10653)
add 59e2c7e024 Minor: Add examples of using TreeNode with Expr (#10686)
add 2796e01cf5 Minor: Add examples of using TreeNode with `LogicalPlan`
(#10687)
add 77352b2411 Add `ParquetExec::builder()`, deprecate `ParquetExec::new`
(#10636)
add 3d00760853 feature: Add a WindowUDFImpl::simplify() API (#9906)
add 088ad010a6 Chore: clean up udwf example && remove edundant import
(#10718)
add c775e4d6ea push down filter to partition listing (#10693)
add ad2b1dcac8 Make swap_hash_join public API (#10702)
add 313f47f30c ci: fix clippy error (#10723)
add 904f0db73c Fix clippy (#10725)
add 79481839da Remove Eager Trait for Joins (#10721)
add 100b30e135 Minor: fix signature `fn octect_length()` (#10726)
add 76f50b0ce9 docs: add documents to substrait type variation consts
(#10719)
add 075ed331bc Update rstest requirement from 0.19.0 to 0.20.0 (#10734)
add 02a76daa72 Update rstest_reuse requirement from 0.6.0 to 0.7.0 (#10733)
add 09dde27be3 Add example for building an external secondary index for
parquet files (#10549)
add a803214015 chore: move stddev test to slt (#10741)
add 7fd286b1ea fix(CLI): can not create external tables with format
options (#10739)
add 68f84761d2 Add support for `AggregateExpr`, `WindowExpr` rewrite.
(#10742)
add d6ddd23795 Fix SMJ Left Anti Join when the join filter is set (#10724)
add 7638a26979 Introduce FunctionRegistry dependency to optimize and
rewrite rule (#10714)
add 3777114192 Minor: Add SMJ to TPCH benchmark usage (#10747)
add acd7106fa4 Minor: Split physical_plan/parquet/mod.rs into smaller
modules (#10727)
add 71a99b8462 minor: consolidate unparser integration tests (#10736)
add 1db3263497 chore: move aggregate_var to slt (#10750)
add 59bfe773b6 fix: fix string repeat for negative numbers (#10760)
add 888504a8da Introduce Sum UDAF (#10651)
add 6e5344ae36 Extract parquet statistics from timestamps with timezones
(#10766)
add fbbab6c7ad Minor: Add tests for extracting dictionary parquet
statistics (#10729)
add eabbd28603 Update rstest requirement from 0.20.0 to 0.21.0 (#10774)
add a92f803298 Minor: Refactor memory size estimation for HashTable
(#10748)
add 3aae451d38 Reduce code repetition in `datafusion/functions` mod files
(#10700)
add ccf395f40c (Doc) Enable rt-multi-thread feature for sample code
(#10770)
add 180f3e8af1 Support negatives in split part (#10780)
add e4f7b9811f feat: support unparsing LogicalPlan::Window nodes (#10767)
add d3fa083acf refactor: handle LargeUtf8 statistics and add tests for
UTF8 and LargeUTF8 (#10762)
add 826331e01c Cleanup GetIndexedField (#10769)
add fe536495de Extract parquet statistics from f16 columns, add
`ScalarValue::Float16` (#10763)
add 6a0a2dce90 handle empty rows (#10786)
add 3f46b0cd49 Fix extract parquet statistics from LargeBinary columns
(#10775)
add 5a544d0909 Extract parquet statistics from Time32 and Time64 columns
(#10771)
add 729b356ef5 chore: fix `last_value` coercion (#10783)
add 76a3720095 Fix extract parquet statistics from Decimal256 columns
(#10777)
add 66ef9b92e5 Speed up arrow_statistics test (#10735)
add 1c85d6abd4 readability refactor (#10788)
add d048c8f143 Merge remote-tracking branch 'apache/main' into
dependabot/cargo/main/itertools-0.13
add 0bb0f3508d Avoid deprecated API
No new revisions were added by this update.
Summary of changes:
Cargo.toml | 9 +-
benchmarks/bench.sh | 16 +-
datafusion-cli/Cargo.lock | 119 +-
datafusion-cli/Dockerfile | 4 +-
datafusion-cli/src/exec.rs | 22 +-
datafusion-cli/src/main.rs | 8 +-
datafusion-examples/README.md | 1 +
datafusion-examples/examples/advanced_udaf.rs | 5 +-
datafusion-examples/examples/advanced_udf.rs | 2 +-
datafusion-examples/examples/csv_opener.rs | 16 +-
datafusion-examples/examples/custom_datasource.rs | 2 +-
.../external_dependency/dataframe-to-s3.rs | 3 +-
.../examples/external_dependency/query-aws-s3.rs | 3 +-
datafusion-examples/examples/function_factory.rs | 2 +-
datafusion-examples/examples/json_opener.rs | 16 +-
datafusion-examples/examples/parquet_index.rs | 705 +
.../examples/parquet_sql_multiple_files.rs | 2 +-
datafusion-examples/examples/plan_to_sql.rs | 76 +-
datafusion-examples/examples/query-http-csv.rs | 3 +-
datafusion-examples/examples/rewrite_expr.rs | 6 +-
.../examples/simplify_udaf_expression.rs | 6 +-
.../examples/simplify_udwf_expression.rs | 130 +
datafusion/common/Cargo.toml | 4 +-
datafusion/common/src/file_options/csv_writer.rs | 1 +
datafusion/common/src/scalar/mod.rs | 149 +-
datafusion/common/src/tree_node.rs | 139 +-
datafusion/common/src/utils/memory.rs | 134 +
datafusion/common/src/utils/mod.rs | 1 +
datafusion/core/Cargo.toml | 11 +-
datafusion/core/benches/parquet_statistic.rs | 205 +
datafusion/core/src/dataframe/mod.rs | 24 +-
datafusion/core/src/datasource/file_format/csv.rs | 2 -
datafusion/core/src/datasource/file_format/mod.rs | 15 +-
.../core/src/datasource/file_format/parquet.rs | 26 +-
datafusion/core/src/datasource/listing/helpers.rs | 206 +-
datafusion/core/src/datasource/listing/mod.rs | 7 +
datafusion/core/src/datasource/listing/table.rs | 17 +-
.../core/src/datasource/listing_table_factory.rs | 6 +-
datafusion/core/src/datasource/mod.rs | 1 +
.../src/datasource/physical_plan/arrow_file.rs | 2 +-
.../core/src/datasource/physical_plan/avro.rs | 61 +-
.../core/src/datasource/physical_plan/csv.rs | 24 +-
.../src/datasource/physical_plan/file_groups.rs | 2 +-
.../datasource/physical_plan/file_scan_config.rs | 124 +-
.../src/datasource/physical_plan/file_stream.rs | 16 +-
.../core/src/datasource/physical_plan/json.rs | 60 +-
.../core/src/datasource/physical_plan/mod.rs | 136 +-
.../src/datasource/physical_plan/parquet/mod.rs | 728 +-
.../src/datasource/physical_plan/parquet/opener.rs | 204 +
.../physical_plan/parquet/page_filter.rs | 15 +-
.../src/datasource/physical_plan/parquet/reader.rs | 140 +
.../datasource/physical_plan/parquet/row_groups.rs | 587 +-
.../physical_plan/parquet/schema_adapter.rs | 69 -
.../datasource/physical_plan/parquet/statistics.rs | 743 +-
.../src/datasource/physical_plan/parquet/writer.rs | 80 +
datafusion/core/src/datasource/schema_adapter.rs | 341 +
datafusion/core/src/execution/context/mod.rs | 38 +-
.../src/physical_optimizer/aggregate_statistics.rs | 2 +-
.../combine_partial_final_agg.rs | 25 +-
.../src/physical_optimizer/convert_first_last.rs | 260 -
.../src/physical_optimizer/enforce_distribution.rs | 94 +-
.../core/src/physical_optimizer/enforce_sorting.rs | 2 +-
.../core/src/physical_optimizer/join_selection.rs | 4 +-
.../limited_distinct_aggregation.rs | 4 +-
datafusion/core/src/physical_optimizer/mod.rs | 2 +-
.../core/src/physical_optimizer/optimizer.rs | 2 +-
.../src/physical_optimizer/output_requirements.rs | 6 +-
.../src/physical_optimizer/projection_pushdown.rs | 42 +-
.../replace_with_order_preserving_variants.rs | 22 +-
.../core/src/physical_optimizer/sort_pushdown.rs | 2 +-
.../core/src/physical_optimizer/test_utils.rs | 43 +-
.../src/physical_optimizer/topk_aggregation.rs | 2 +-
.../src/physical_optimizer/update_aggr_exprs.rs | 182 +
datafusion/core/src/physical_planner.rs | 55 +-
datafusion/core/src/test/mod.rs | 44 +-
datafusion/core/src/test/object_store.rs | 3 +-
datafusion/core/src/test_util/mod.rs | 2 +-
datafusion/core/src/test_util/parquet.rs | 51 +-
datafusion/core/tests/core_integration.rs | 9 +
datafusion/core/tests/custom_sources.rs | 308 -
datafusion/core/tests/custom_sources_cases/mod.rs | 289 +
.../provider_filter_pushdown.rs | 2 +-
.../core/tests/custom_sources_cases/statistics.rs | 2 +-
datafusion/core/tests/data/unnest.json | 2 +
datafusion/core/tests/dataframe/mod.rs | 22 +-
datafusion/core/tests/expr_api/mod.rs | 57 +-
.../core/tests/{ => expr_api}/simplification.rs | 2 +-
datafusion/core/tests/{fifo.rs => fifo/mod.rs} | 0
datafusion/core/tests/fuzz_cases/aggregate_fuzz.rs | 4 +-
datafusion/core/tests/fuzz_cases/window_fuzz.rs | 12 +-
.../tests/{memory_limit.rs => memory_limit/mod.rs} | 0
datafusion/core/tests/optimizer_integration.rs | 135 +-
datafusion/core/tests/parquet/arrow_statistics.rs | 1769 ++
datafusion/core/tests/parquet/custom_reader.rs | 23 +-
datafusion/core/tests/parquet/filter_pushdown.rs | 2 +-
datafusion/core/tests/parquet/mod.rs | 517 +-
datafusion/core/tests/parquet/page_pruning.rs | 25 +-
datafusion/core/tests/parquet/schema_adapter.rs | 171 -
datafusion/core/tests/parquet/schema_coercion.rs | 47 +-
datafusion/core/tests/path_partition.rs | 4 +-
.../tests/user_defined/user_defined_aggregates.rs | 7 +-
.../core/tests/user_defined/user_defined_plan.rs | 18 +-
.../user_defined/user_defined_scalar_functions.rs | 4 +-
datafusion/execution/src/lib.rs | 7 +-
datafusion/execution/src/object_store.rs | 31 +-
datafusion/execution/src/runtime_env.rs | 33 +
datafusion/expr/Cargo.toml | 4 +-
datafusion/expr/src/aggregate_function.rs | 25 +-
datafusion/expr/src/built_in_window_function.rs | 4 +-
datafusion/expr/src/expr.rs | 315 +-
datafusion/expr/src/expr_fn.rs | 23 +-
datafusion/expr/src/expr_schema.rs | 114 +-
datafusion/expr/src/field_util.rs | 100 -
datafusion/expr/src/function.rs | 16 +
datafusion/expr/src/lib.rs | 8 +-
datafusion/expr/src/logical_plan/builder.rs | 236 +-
datafusion/expr/src/logical_plan/display.rs | 35 +-
datafusion/expr/src/logical_plan/extension.rs | 72 +-
datafusion/expr/src/logical_plan/plan.rs | 204 +-
datafusion/expr/src/logical_plan/tree_node.rs | 77 +-
datafusion/{execution => expr}/src/registry.rs | 4 +-
datafusion/expr/src/signature.rs | 14 +
datafusion/expr/src/tree_node.rs | 57 +-
datafusion/expr/src/type_coercion/aggregates.rs | 7 +-
datafusion/expr/src/type_coercion/binary.rs | 261 +-
datafusion/expr/src/type_coercion/functions.rs | 148 +-
datafusion/expr/src/udaf.rs | 130 +-
datafusion/expr/src/udf.rs | 6 +-
datafusion/expr/src/udwf.rs | 34 +-
datafusion/expr/src/utils.rs | 32 +-
datafusion/functions-aggregate/Cargo.toml | 2 +
datafusion/functions-aggregate/src/first_last.rs | 504 +-
datafusion/functions-aggregate/src/lib.rs | 54 +-
datafusion/functions-aggregate/src/median.rs | 303 +
datafusion/functions-aggregate/src/sum.rs | 436 +
datafusion/functions-array/src/array_has.rs | 5 +-
datafusion/functions-array/src/cardinality.rs | 2 +-
datafusion/functions-array/src/concat.rs | 3 -
datafusion/functions-array/src/dimension.rs | 4 +-
datafusion/functions-array/src/empty.rs | 6 +-
datafusion/functions-array/src/except.rs | 2 +-
datafusion/functions-array/src/expr_ext.rs | 102 +
datafusion/functions-array/src/extract.rs | 24 +-
datafusion/functions-array/src/flatten.rs | 2 +-
datafusion/functions-array/src/length.rs | 2 +-
datafusion/functions-array/src/lib.rs | 41 +-
datafusion/functions-array/src/position.rs | 6 +-
datafusion/functions-array/src/range.rs | 4 +-
datafusion/functions-array/src/remove.rs | 9 +-
datafusion/functions-array/src/repeat.rs | 2 +-
datafusion/functions-array/src/replace.rs | 12 +-
datafusion/functions-array/src/resize.rs | 2 +-
datafusion/functions-array/src/reverse.rs | 2 +-
datafusion/functions-array/src/rewrite.rs | 2 +-
datafusion/functions-array/src/set_ops.rs | 9 +-
datafusion/functions-array/src/sort.rs | 5 +-
datafusion/functions-array/src/string.rs | 6 +-
datafusion/functions/Cargo.toml | 2 +-
datafusion/functions/src/core/coalesce.rs | 24 +-
datafusion/functions/src/core/expr_ext.rs | 68 +
datafusion/functions/src/core/getfield.rs | 46 +-
datafusion/functions/src/core/mod.rs | 83 +-
datafusion/functions/src/crypto/mod.rs | 60 +-
datafusion/functions/src/datetime/date_bin.rs | 2 +-
datafusion/functions/src/datetime/date_trunc.rs | 2 +-
datafusion/functions/src/datetime/mod.rs | 123 +-
datafusion/functions/src/encoding/mod.rs | 22 +-
datafusion/functions/src/lib.rs | 45 +-
datafusion/functions/src/macros.rs | 54 +-
datafusion/functions/src/math/abs.rs | 2 +-
datafusion/functions/src/math/log.rs | 2 +-
datafusion/functions/src/math/mod.rs | 292 +-
datafusion/functions/src/math/monotonicity.rs | 58 +-
datafusion/functions/src/math/pi.rs | 2 +-
datafusion/functions/src/math/round.rs | 2 +-
datafusion/functions/src/math/trunc.rs | 2 +-
datafusion/functions/src/string/mod.rs | 181 +-
datafusion/functions/src/string/repeat.rs | 5 +-
datafusion/functions/src/string/split_part.rs | 37 +-
datafusion/functions/src/unicode/mod.rs | 111 +-
datafusion/optimizer/src/analyzer/subquery.rs | 10 +-
datafusion/optimizer/src/analyzer/type_coercion.rs | 21 +-
.../optimizer/src/common_subexpr_eliminate.rs | 6 +-
.../optimizer/src/eliminate_group_by_constant.rs | 318 +
datafusion/optimizer/src/lib.rs | 1 +
.../optimizer/src/optimize_projections/mod.rs | 62 +-
datafusion/optimizer/src/optimizer.rs | 65 +-
datafusion/optimizer/src/push_down_filter.rs | 63 +-
datafusion/optimizer/src/push_down_limit.rs | 1 -
.../optimizer/src/replace_distinct_aggregate.rs | 83 +-
.../src/simplify_expressions/expr_simplifier.rs | 111 +-
.../src/simplify_expressions/guarantees.rs | 68 +-
.../optimizer/src/simplify_expressions/mod.rs | 3 +
.../optimizer/src/single_distinct_to_groupby.rs | 447 +-
datafusion/optimizer/src/test/user_defined.rs | 12 +-
.../optimizer/tests/optimizer_integration.rs | 10 +-
datafusion/physical-expr-common/Cargo.toml | 1 +
.../src/aggregate/groups_accumulator/accumulate.rs | 8 +-
.../src/aggregate/groups_accumulator/bool_op.rs | 5 +-
.../src/aggregate/groups_accumulator}/mod.rs | 8 +-
.../src/aggregate/groups_accumulator/prim_op.rs | 6 +-
.../physical-expr-common/src/aggregate/mod.rs | 216 +-
.../physical-expr-common/src/aggregate/utils.rs | 163 +-
.../src/expressions/cast.rs | 13 +-
.../physical-expr-common/src/expressions/column.rs | 2 +-
.../physical-expr-common/src/expressions/mod.rs | 3 +
.../physical-expr-common/src/physical_expr.rs | 2 +-
datafusion/physical-expr-common/src/sort_expr.rs | 35 +-
datafusion/physical-expr-common/src/tree_node.rs | 13 +-
datafusion/physical-expr-common/src/utils.rs | 48 +-
datafusion/physical-expr/Cargo.toml | 6 +-
.../src/aggregate/array_agg_ordered.rs | 12 +-
datafusion/physical-expr/src/aggregate/build_in.rs | 133 +-
datafusion/physical-expr/src/aggregate/count.rs | 15 +
.../src/aggregate/count_distinct/native.rs | 35 +-
.../src/aggregate/groups_accumulator/mod.rs | 17 +-
datafusion/physical-expr/src/aggregate/median.rs | 681 -
datafusion/physical-expr/src/aggregate/mod.rs | 18 +-
.../physical-expr/src/aggregate/nth_value.rs | 5 +
datafusion/physical-expr/src/aggregate/stddev.rs | 127 -
datafusion/physical-expr/src/aggregate/utils.rs | 191 -
datafusion/physical-expr/src/aggregate/variance.rs | 109 -
datafusion/physical-expr/src/equivalence/class.rs | 2 +-
.../physical-expr/src/equivalence/properties.rs | 3 +-
datafusion/physical-expr/src/expressions/binary.rs | 4 +-
datafusion/physical-expr/src/expressions/case.rs | 60 +-
datafusion/physical-expr/src/expressions/column.rs | 2 +-
.../physical-expr/src/expressions/in_list.rs | 6 +-
.../physical-expr/src/expressions/is_not_null.rs | 4 +-
.../physical-expr/src/expressions/is_null.rs | 4 +-
datafusion/physical-expr/src/expressions/like.rs | 4 +-
.../physical-expr/src/expressions/literal.rs | 2 +-
datafusion/physical-expr/src/expressions/mod.rs | 26 +-
.../physical-expr/src/expressions/negative.rs | 4 +-
datafusion/physical-expr/src/expressions/no_op.rs | 2 +-
datafusion/physical-expr/src/expressions/not.rs | 4 +-
.../physical-expr/src/expressions/try_cast.rs | 4 +-
datafusion/physical-expr/src/lib.rs | 4 +-
datafusion/physical-expr/src/planner.rs | 25 +-
datafusion/physical-expr/src/scalar_function.rs | 6 +-
datafusion/physical-expr/src/utils/mod.rs | 2 +-
datafusion/physical-expr/src/window/lead_lag.rs | 53 +-
.../physical-expr/src/window/sliding_aggregate.rs | 25 +
datafusion/physical-expr/src/window/window_expr.rs | 39 +
datafusion/physical-plan/Cargo.toml | 6 +-
datafusion/physical-plan/src/aggregates/mod.rs | 180 +-
datafusion/physical-plan/src/analyze.rs | 4 +-
datafusion/physical-plan/src/coalesce_batches.rs | 4 +-
.../physical-plan/src/coalesce_partitions.rs | 4 +-
datafusion/physical-plan/src/display.rs | 2 +-
datafusion/physical-plan/src/empty.rs | 2 +-
datafusion/physical-plan/src/explain.rs | 2 +-
datafusion/physical-plan/src/filter.rs | 4 +-
datafusion/physical-plan/src/insert.rs | 4 +-
datafusion/physical-plan/src/joins/cross_join.rs | 4 +-
datafusion/physical-plan/src/joins/hash_join.rs | 29 +-
.../physical-plan/src/joins/nested_loop_join.rs | 4 +-
.../physical-plan/src/joins/sort_merge_join.rs | 393 +-
.../physical-plan/src/joins/stream_join_utils.rs | 356 +-
.../physical-plan/src/joins/symmetric_hash_join.rs | 272 +-
datafusion/physical-plan/src/joins/utils.rs | 28 +-
datafusion/physical-plan/src/lib.rs | 10 +-
datafusion/physical-plan/src/limit.rs | 8 +-
datafusion/physical-plan/src/memory.rs | 2 +-
datafusion/physical-plan/src/placeholder_row.rs | 2 +-
datafusion/physical-plan/src/projection.rs | 4 +-
datafusion/physical-plan/src/recursive_query.rs | 8 +-
datafusion/physical-plan/src/repartition/mod.rs | 4 +-
datafusion/physical-plan/src/sorts/partial_sort.rs | 4 +-
datafusion/physical-plan/src/sorts/sort.rs | 4 +-
.../src/sorts/sort_preserving_merge.rs | 4 +-
datafusion/physical-plan/src/streaming.rs | 2 +-
datafusion/physical-plan/src/test/exec.rs | 12 +-
datafusion/physical-plan/src/tree_node.rs | 13 +-
datafusion/physical-plan/src/union.rs | 8 +-
datafusion/physical-plan/src/unnest.rs | 201 +-
datafusion/physical-plan/src/values.rs | 2 +-
.../src/windows/bounded_window_agg_exec.rs | 4 +-
datafusion/physical-plan/src/windows/mod.rs | 46 +-
.../physical-plan/src/windows/window_agg_exec.rs | 4 +-
datafusion/physical-plan/src/work_table.rs | 2 +-
datafusion/{proto => proto-common}/.gitignore | 4 +-
datafusion/{execution => proto-common}/Cargo.toml | 39 +-
datafusion/{proto => proto-common}/README.md | 7 +-
datafusion/{proto => proto-common}/gen/Cargo.toml | 2 +-
datafusion/{proto => proto-common}/gen/src/main.rs | 8 +-
.../proto-common/proto/datafusion_common.proto | 536 +
datafusion/{proto => proto-common}/regen.sh | 2 +-
.../mod.rs => proto-common/src/common.rs} | 9 +-
datafusion/proto-common/src/from_proto/mod.rs | 1094 +
.../{proto => proto-common}/src/generated/mod.rs | 2 +-
datafusion/proto-common/src/generated/pbjson.rs | 7870 +++++++
datafusion/proto-common/src/generated/prost.rs | 1126 +
datafusion/proto-common/src/lib.rs | 93 +
datafusion/proto-common/src/to_proto/mod.rs | 1006 +
datafusion/proto/.gitignore | 3 +-
datafusion/proto/Cargo.toml | 1 +
datafusion/proto/gen/src/main.rs | 25 +-
datafusion/proto/proto/datafusion.proto | 696 +-
datafusion/proto/regen.sh | 4 +-
datafusion/proto/src/common.rs | 4 -
.../proto/src/generated/datafusion_proto_common.rs | 1126 +
datafusion/proto/src/generated/mod.rs | 3 +
datafusion/proto/src/generated/pbjson.rs | 23249 +++++++------------
datafusion/proto/src/generated/prost.rs | 1393 +-
datafusion/proto/src/lib.rs | 10 +-
datafusion/proto/src/logical_plan/from_proto.rs | 716 +-
datafusion/proto/src/logical_plan/mod.rs | 142 +-
datafusion/proto/src/logical_plan/to_proto.rs | 771 +-
datafusion/proto/src/physical_plan/from_proto.rs | 470 +-
datafusion/proto/src/physical_plan/mod.rs | 115 +-
datafusion/proto/src/physical_plan/to_proto.rs | 340 +-
.../proto/tests/cases/roundtrip_logical_plan.rs | 51 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 66 +-
datafusion/sql/Cargo.toml | 1 +
datafusion/sql/examples/sql.rs | 6 +-
datafusion/sql/src/expr/function.rs | 48 +-
datafusion/sql/src/expr/mod.rs | 6 +-
datafusion/sql/src/planner.rs | 15 +-
datafusion/sql/src/select.rs | 69 +-
datafusion/sql/src/unparser/ast.rs | 3 +
datafusion/sql/src/unparser/dialect.rs | 43 +-
datafusion/sql/src/unparser/expr.rs | 157 +-
datafusion/sql/src/unparser/plan.rs | 289 +-
datafusion/sql/src/unparser/utils.rs | 81 +-
datafusion/sql/src/utils.rs | 258 +-
.../src/expressions => sql/tests/cases}/mod.rs | 2 +-
datafusion/sql/tests/cases/plan_to_sql.rs | 296 +
datafusion/sql/tests/common/mod.rs | 227 +
datafusion/sql/tests/sql_integration.rs | 367 +-
datafusion/sqllogictest/test_files/aggregate.slt | 578 +-
datafusion/sqllogictest/test_files/array.slt | 12 +
datafusion/sqllogictest/test_files/coalesce.slt | 64 +-
datafusion/sqllogictest/test_files/csv_files.slt | 41 +
datafusion/sqllogictest/test_files/distinct_on.slt | 46 +-
datafusion/sqllogictest/test_files/explain.slt | 2 +
datafusion/sqllogictest/test_files/expr.slt | 18 +
.../test_files/filter_without_sort_exec.slt | 4 +-
datafusion/sqllogictest/test_files/group_by.slt | 167 +-
datafusion/sqllogictest/test_files/joins.slt | 26 +-
.../test_files/optimizer_group_by_constant.slt | 118 +
datafusion/sqllogictest/test_files/order.slt | 48 +-
datafusion/sqllogictest/test_files/projection.slt | 17 +
.../sqllogictest/test_files/sort_merge_join.slt | 237 +
datafusion/sqllogictest/test_files/subquery.slt | 71 +-
.../sqllogictest/test_files/tpch/q1.slt.part | 6 +-
.../sqllogictest/test_files/tpch/q10.slt.part | 6 +-
.../sqllogictest/test_files/tpch/q12.slt.part | 6 +-
.../sqllogictest/test_files/tpch/q14.slt.part | 10 +-
.../sqllogictest/test_files/tpch/q15.slt.part | 12 +-
.../sqllogictest/test_files/tpch/q20.slt.part | 6 +-
.../sqllogictest/test_files/tpch/q3.slt.part | 12 +-
.../sqllogictest/test_files/tpch/q4.slt.part | 6 +-
.../sqllogictest/test_files/tpch/q5.slt.part | 6 +-
.../sqllogictest/test_files/tpch/q6.slt.part | 6 +-
.../sqllogictest/test_files/tpch/q7.slt.part | 6 +-
.../sqllogictest/test_files/tpch/q8.slt.part | 6 +-
datafusion/sqllogictest/test_files/unnest.slt | 96 +-
datafusion/sqllogictest/test_files/window.slt | 67 +-
datafusion/substrait/Cargo.toml | 5 +-
datafusion/substrait/src/logical_plan/consumer.rs | 500 +-
datafusion/substrait/src/logical_plan/producer.rs | 832 +-
datafusion/substrait/src/physical_plan/consumer.rs | 25 +-
datafusion/substrait/src/variation_const.rs | 91 +-
datafusion/substrait/tests/cases/logical_plans.rs | 63 +
datafusion/substrait/tests/cases/mod.rs | 1 +
.../tests/cases/roundtrip_logical_plan.rs | 158 +-
.../tests/cases/roundtrip_physical_plan.rs | 43 +-
.../tests/testdata/select_not_bool.substrait.json | 98 +
dev/release/rat_exclude_files.txt | 3 +
docs/source/contributor-guide/index.md | 28 +-
docs/source/user-guide/example-usage.md | 2 +-
docs/source/user-guide/sql/scalar_functions.md | 82 +
373 files changed, 38409 insertions(+), 27680 deletions(-)
create mode 100644 datafusion-examples/examples/parquet_index.rs
create mode 100644 datafusion-examples/examples/simplify_udwf_expression.rs
create mode 100644 datafusion/common/src/utils/memory.rs
create mode 100644 datafusion/core/benches/parquet_statistic.rs
create mode 100644
datafusion/core/src/datasource/physical_plan/parquet/opener.rs
create mode 100644
datafusion/core/src/datasource/physical_plan/parquet/reader.rs
delete mode 100644
datafusion/core/src/datasource/physical_plan/parquet/schema_adapter.rs
create mode 100644
datafusion/core/src/datasource/physical_plan/parquet/writer.rs
create mode 100644 datafusion/core/src/datasource/schema_adapter.rs
delete mode 100644 datafusion/core/src/physical_optimizer/convert_first_last.rs
create mode 100644 datafusion/core/src/physical_optimizer/update_aggr_exprs.rs
delete mode 100644 datafusion/core/tests/custom_sources.rs
create mode 100644 datafusion/core/tests/data/unnest.json
rename datafusion/core/tests/{ => expr_api}/simplification.rs (99%)
rename datafusion/core/tests/{fifo.rs => fifo/mod.rs} (100%)
rename datafusion/core/tests/{memory_limit.rs => memory_limit/mod.rs} (100%)
create mode 100644 datafusion/core/tests/parquet/arrow_statistics.rs
delete mode 100644 datafusion/core/tests/parquet/schema_adapter.rs
delete mode 100644 datafusion/expr/src/field_util.rs
rename datafusion/{execution => expr}/src/registry.rs (98%)
create mode 100644 datafusion/functions-aggregate/src/median.rs
create mode 100644 datafusion/functions-aggregate/src/sum.rs
create mode 100644 datafusion/functions-array/src/expr_ext.rs
create mode 100644 datafusion/functions/src/core/expr_ext.rs
create mode 100644 datafusion/optimizer/src/eliminate_group_by_constant.rs
rename datafusion/{physical-expr =>
physical-expr-common}/src/aggregate/groups_accumulator/accumulate.rs (99%)
rename datafusion/{physical-expr =>
physical-expr-common}/src/aggregate/groups_accumulator/bool_op.rs (97%)
copy datafusion/{physical-plan/src/aggregates/topk =>
physical-expr-common/src/aggregate/groups_accumulator}/mod.rs (88%)
rename datafusion/{physical-expr =>
physical-expr-common}/src/aggregate/groups_accumulator/prim_op.rs (96%)
rename datafusion/{physical-expr =>
physical-expr-common}/src/expressions/cast.rs (98%)
delete mode 100644 datafusion/physical-expr/src/aggregate/median.rs
delete mode 100644 datafusion/physical-expr/src/aggregate/utils.rs
copy datafusion/{proto => proto-common}/.gitignore (50%)
copy datafusion/{execution => proto-common}/Cargo.toml (64%)
copy datafusion/{proto => proto-common}/README.md (77%)
copy datafusion/{proto => proto-common}/gen/Cargo.toml (98%)
copy datafusion/{proto => proto-common}/gen/src/main.rs (89%)
create mode 100644 datafusion/proto-common/proto/datafusion_common.proto
copy datafusion/{proto => proto-common}/regen.sh (93%)
copy datafusion/{proto/src/generated/mod.rs => proto-common/src/common.rs}
(81%)
create mode 100644 datafusion/proto-common/src/from_proto/mod.rs
copy datafusion/{proto => proto-common}/src/generated/mod.rs (96%)
create mode 100644 datafusion/proto-common/src/generated/pbjson.rs
create mode 100644 datafusion/proto-common/src/generated/prost.rs
create mode 100644 datafusion/proto-common/src/lib.rs
create mode 100644 datafusion/proto-common/src/to_proto/mod.rs
create mode 100644 datafusion/proto/src/generated/datafusion_proto_common.rs
copy datafusion/{physical-expr-common/src/expressions =>
sql/tests/cases}/mod.rs (97%)
create mode 100644 datafusion/sql/tests/cases/plan_to_sql.rs
create mode 100644 datafusion/sql/tests/common/mod.rs
create mode 100644
datafusion/sqllogictest/test_files/optimizer_group_by_constant.slt
create mode 100644 datafusion/substrait/tests/cases/logical_plans.rs
create mode 100644
datafusion/substrait/tests/testdata/select_not_bool.substrait.json
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
