This is an automated email from the ASF dual-hosted git repository.
alamb pushed a change to branch string-view2
in repository https://gitbox.apache.org/repos/asf/datafusion.git
from 2b58fd5c32 GC `StringViewArray` in `CoalesceBatchesStream` (#11587)
add 0c39b4d2ff Replace to_lowercase with to_string in sql exmaple (#11486)
add 169a0d338c chore: switch to using proper Substrait types for
IntervalYearMonth and IntervalDayTime (#11471)
add 82fd6a7de3 Move execute_input_stream (#11449)
add c54a638585 Enable clone_on_ref_ptr clippy lints on proto (#11465)
add 382bf4f3c7 upgrade sqlparser 0.47 -> 0.48 (#11453)
add 81aff944bd feat: support UDWFs in Substrait (#11489)
add 02326998f0 Add extension hooks for encoding and decoding UDAFs and
UDWFs (#11417)
add a979f3e5d4 feat: support `unnest` in GROUP BY clause (#11469)
add d67b0fbf52 Remove element's nullability of array_agg function (#11447)
add de0765a97b Get expr planners when creating new planner (#11485)
add b0925c801e Support alternate format for Utf8 unparsing (CHAR) (#11494)
add fb34ef2cca implement retract_batch for xor accumulator (#11500)
add c95556def3 Refactor: more clearly delineate between
`TableParquetOptions` and `ParquetWriterOptions` (#11444)
add 0021356fb6 feat: support `COUNT()` (#11229)
add 81a6094753 chore: fix typos of common and core packages (#11520)
add adcfd85b77 Move spill related functions to spill.rs (#11509)
add 4b840c0883 Add tests that show the different defaults for
`ArrowWriter` and `TableParquetOptions` (#11524)
add 58da9690f2 Create `datafusion-physical-optimizer` crate (#11507)
add 1b9b35c4a5 Minor: Assert `test_enabled_backtrace` requirements to run
(#11525)
add d91a03f495 Move handlign of NULL literals in where clause to type
coercion pass (#11491)
add dff2f3c3c6 Minor: Clarify which parquet options are used for
reading/writing (#11511)
add b197449687 Update parquet page pruning code to use the
`StatisticsExtractor` (#11483)
add be130b4670 Enable SortMergeJoin LeftAnti filtered fuzz tests (#11535)
add b685e2d4f1 chore: fix typos of expr, functions, optimizer,
physical-expr-common, physical-expr, and physical-plan packages (#11538)
add 4dd8532e6c rm clone (#11532)
add 723a595528 Minor: avoid a clone in type coercion (#11530)
add 12d82c427d Move array `ArrayAgg` to a `UserDefinedAggregate` (#11448)
add 5f0dfbb8e7 Move `MAKE_MAP` to ExprPlanner (#11452)
add 28fa74bf0f feat: Optimize CASE expression for "column or null" use
case (#11534)
add cf9da76830 fix: typos of sql, sqllogictest and substrait packages
(#11548)
add a4c9bb4574 feat: consume and produce Substrait type extensions (#11510)
add af0d2baf02 make unparser Dialect Send + Sync (#11504)
add f195352818 fix: unparser generates wrong sql for derived table with
columns (#17) (#11505)
add 9189a1acdd Prevent bigger files from being checked in (#11508)
add ebe61bae2a fix: make `UnKnownColumn`s not equal to others physical
exprs (#11536)
add 827d0e3a29 Add dialect param to use double precision for float64 in
Postgres (#11495)
add 5da7ab3002 Minor: move `SessionStateDefaults` into its own module
(#11566)
add 7df2bde8fc fix: fixes trig function order by (#11559)
add d232065c3b refactor: rewrite mega type to an enum containing both
cases (#11539)
add 36660fe10d Move `sql_compound_identifier_to_expr ` to `ExprPlanner`
(#11487)
add 2587df09c3 Support `newlines_in_values` CSV option (#11533)
add 63efaee255 Support SortMergeJoin spilling (#11218)
add ecf5323eaa Fix unparser invalid sql for query with order (#11527)
add 12c0a1e2e2 chore: Minor cleanup `simplify_demo()` example (#11576)
add f9457de779 Move Datafusion Query Optimizer to library user guide
(#11563)
add 47d5d1fe1a feat: Error when a SHOW command is passed in with an
accompanying non-existant variable (#11540)
add 5c65efc799 fix: CASE with NULL (#11542)
add 51da92fb9f Provide DataFrame API for `map` and move `map` to
`functions-array` (#11560)
add 81d06f2e10 Move OutputRequirements to datafusion-physical-optimizer
crate (#11579)
add 4417a9404f Minor: move `Column` related tests and rename `column.rs`
(#11573)
add b6e55d7e9c feat: Optimize CASE expression for usage where then and
else values are literals (#11553)
add 7d078d8c11 Fix SortMergeJoin antijoin flaky condition (#11604)
add a2ac00da1b Improve Union Equivalence Propagation (#11506)
add d941dc3f01 Migrate `OrderSensitiveArrayAgg` to be a user defined
aggregate (#11564)
add deef834e7a Minor: Disable flaky antijoin test until perm fix (#11608)
add 77311a5896 support Decimal256 type in datafusion-proto (#11606)
add 7afc99f3d2 Chore/fifo tests cleanup (#11616)
add 67c6ee2fb0 fix typo in doc of Partitioning (#11612)
add f80dde06ec feat: support Map literals in Substrait consumer and
producer (#11547)
add f9ee64a0ec Fix Internal Error for an INNER JOIN query (#11578)
add 5f0ab3100f test: get file size by func metadata (#11575)
add 268be45533 Update scalar_functions.md (#11620)
add 65dd364be4 Improve unparser MySQL compatibility (#11589)
add 9f74dcc999 Change default Parquet writer settings to match arrow-rs
(except for compression & statistics) (#11558)
add 6d3521738b Push scalar functions into cross join (#11528)
add 6d8bd2cb67 Implement physical plan serialization for csv COPY plans ,
add `as_any`, `Debug` to `FileFormatFactory` (#11588)
add fc8e7b9035 Remove ArrayAgg Builtin in favor of UDF (#11611)
add 5c37d008dc refactor: simplify `DFSchema::field_with_unqualified_name`
(#11619)
add c8ef54537c fix: panic and incorrect results in
`LogFunc::output_ordering()` (#11571)
add 72c6491d25 Minor: Use upstream concat_batches (#11615)
add 1e06b91d59 Rename `functions-array` to `functions-nested` (#11602)
add 8945462ed0 Fix : `signum` function bug when `0.0` input (#11580)
add 6efdbe6d4b Enforce uniqueness of `named_struct` field names (#11614)
add e90b3ac5cf Minor: unecessary row_count calculation in `CrossJoinExec`
and `NestedLoopsJoinExec` (#11632)
add 13569340bc ExprBuilder for Physical Aggregate Expr (#11617)
add 5901df58b2 feat: add bounds for unary math scalar functions (#11584)
add bcf715c892 Minor: avoid copying order by exprs in planner (#11634)
add 20b298e9d8 perf: Optimize IsNotNullExpr (#11586)
add 76039fadd9 Unify CI and pre-commit hook settings for clippy (#11640)
add 886e8accda Consistent API to set parameters of aggregate and window
functions (`AggregateExt` --> `ExprFunctionExt`) (#11550)
add c9518245fa Parsing SQL strings to Exprs with the qualified schema
(#11562)
add f12b3db9c3 fix: expose the fluent API fn for approx_distinct instead
of the module (#11644)
add 49d9d45f36 Add some zero column tests covering LIMIT, GROUP BY, WHERE,
JOIN, and WINDOW (#11624)
add 7db4213b71 Refactor/simplify window frame utils (#11648)
add 71903e1b2c Minor: use `ready!` macro to simplify FilterExec poll loop
(#11649)
add 5b7e0aae22 Temporarily pin toolchain version to avoid clippy (#11655)
add 6fd57b2e38 Fix clippy errors for Rust 1.80 (#11654)
add d452d51bac Add parser option enable_options_value_normalization
(#11330)
add dcdcc250cc Add reference to #comet channel in Arrow Rust Discord
server (#11637)
add fab7e238e8 Add `CsvExecBuilder` for creating `CsvExec` (#11633)
add 2b2b8ab6c7 Merge remote-tracking branch 'apache/main' into string-view2
No new revisions were added by this update.
Summary of changes:
.gitattributes | 1 +
.github/workflows/large_files.yml | 55 ++
.github/workflows/rust.yml | 4 +-
Cargo.toml | 9 +-
README.md | 2 +-
datafusion-cli/Cargo.lock | 100 +--
datafusion-examples/examples/advanced_udwf.rs | 12 +-
.../examples/composed_extension_codec.rs | 80 +--
datafusion-examples/examples/custom_file_format.rs | 6 +-
datafusion-examples/examples/expr_api.rs | 14 +-
datafusion-examples/examples/simple_udwf.rs | 12 +-
datafusion/common/Cargo.toml | 1 +
datafusion/common/src/column.rs | 4 +-
datafusion/common/src/config.rs | 113 +++-
datafusion/common/src/dfschema.rs | 32 +-
datafusion/common/src/error.rs | 7 +-
.../common/src/file_options/parquet_writer.rs | 631 +++++++++++++++---
datafusion/common/src/functional_dependencies.rs | 2 +-
datafusion/common/src/hash_utils.rs | 122 +++-
.../mod.rs => common/src/scalar/consts.rs} | 31 +-
datafusion/common/src/scalar/mod.rs | 124 +++-
datafusion/common/src/table_reference.rs | 2 +-
datafusion/common/src/tree_node.rs | 8 +-
datafusion/common/src/utils/memory.rs | 2 +-
datafusion/common/src/utils/mod.rs | 2 +-
datafusion/core/Cargo.toml | 16 +-
datafusion/core/benches/map_query_sql.rs | 93 +++
datafusion/core/benches/sort.rs | 2 +-
datafusion/core/example.parquet | Bin 0 -> 976 bytes
datafusion/core/src/catalog/information_schema.rs | 4 +-
datafusion/core/src/catalog/mod.rs | 4 +-
datafusion/core/src/dataframe/mod.rs | 21 +-
.../core/src/datasource/file_format/arrow.rs | 10 +-
datafusion/core/src/datasource/file_format/avro.rs | 11 +
datafusion/core/src/datasource/file_format/csv.rs | 103 ++-
datafusion/core/src/datasource/file_format/json.rs | 12 +
datafusion/core/src/datasource/file_format/mod.rs | 16 +-
.../core/src/datasource/file_format/options.rs | 22 +
.../core/src/datasource/file_format/parquet.rs | 23 +-
.../core/src/datasource/file_format/write/demux.rs | 2 +-
.../datasource/file_format/write/orchestration.rs | 68 +-
datafusion/core/src/datasource/listing/helpers.rs | 4 +-
datafusion/core/src/datasource/listing/table.rs | 20 +-
datafusion/core/src/datasource/listing/url.rs | 2 +-
datafusion/core/src/datasource/memory.rs | 2 +-
.../core/src/datasource/physical_plan/csv.rs | 335 ++++++++--
.../datasource/physical_plan/file_scan_config.rs | 4 +-
.../src/datasource/physical_plan/parquet/mod.rs | 91 +--
.../src/datasource/physical_plan/parquet/opener.rs | 4 +-
.../physical_plan/parquet/page_filter.rs | 556 +++++++---------
.../physical_plan/parquet/row_group_filter.rs | 2 +-
.../datasource/physical_plan/parquet/statistics.rs | 40 +-
datafusion/core/src/datasource/schema_adapter.rs | 3 +-
datafusion/core/src/datasource/streaming.rs | 2 +-
datafusion/core/src/execution/context/mod.rs | 37 +-
datafusion/core/src/execution/mod.rs | 3 +
datafusion/core/src/execution/session_state.rs | 265 ++------
.../core/src/execution/session_state_defaults.rs | 203 ++++++
datafusion/core/src/lib.rs | 56 +-
.../src/physical_optimizer/aggregate_statistics.rs | 22 +-
.../src/physical_optimizer/coalesce_batches.rs | 2 +-
.../combine_partial_final_agg.rs | 45 +-
.../src/physical_optimizer/enforce_distribution.rs | 97 +--
.../core/src/physical_optimizer/enforce_sorting.rs | 4 +-
.../core/src/physical_optimizer/join_selection.rs | 10 +-
.../limited_distinct_aggregation.rs | 2 +-
datafusion/core/src/physical_optimizer/mod.rs | 11 +-
.../core/src/physical_optimizer/optimizer.rs | 28 +-
.../src/physical_optimizer/projection_pushdown.rs | 84 +--
datafusion/core/src/physical_optimizer/pruning.rs | 43 +-
.../replace_with_order_preserving_variants.rs | 32 +-
.../core/src/physical_optimizer/sanity_checker.rs | 2 +-
.../src/physical_optimizer/topk_aggregation.rs | 2 +-
.../src/physical_optimizer/update_aggr_exprs.rs | 5 +-
datafusion/core/src/physical_planner.rs | 11 +-
datafusion/core/src/prelude.rs | 4 +-
datafusion/core/src/test/mod.rs | 70 +-
datafusion/core/tests/data/newlines_in_values.csv | 13 +
.../core/tests/dataframe/dataframe_functions.rs | 22 +
datafusion/core/tests/dataframe/mod.rs | 32 +-
datafusion/core/tests/expr_api/mod.rs | 4 +-
datafusion/core/tests/expr_api/parse_sql_expr.rs | 16 +-
datafusion/core/tests/fifo/mod.rs | 174 +++--
datafusion/core/tests/fuzz_cases/aggregate_fuzz.rs | 22 +-
datafusion/core/tests/fuzz_cases/join_fuzz.rs | 41 +-
datafusion/core/tests/fuzz_cases/window_fuzz.rs | 64 +-
datafusion/core/tests/memory_limit/mod.rs | 31 +-
datafusion/core/tests/parquet/arrow_statistics.rs | 6 +-
datafusion/core/tests/parquet/schema.rs | 4 +-
datafusion/core/tests/sql/aggregates.rs | 4 +-
datafusion/core/tests/sql/explain_analyze.rs | 2 +-
.../user_defined/user_defined_scalar_functions.rs | 10 +-
datafusion/execution/src/memory_pool/mod.rs | 23 +-
datafusion/execution/src/object_store.rs | 16 +-
datafusion/expr/src/accumulator.rs | 8 +-
datafusion/expr/src/aggregate_function.rs | 20 +-
datafusion/expr/src/expr.rs | 111 +++-
datafusion/expr/src/expr_fn.rs | 291 +++++++-
datafusion/expr/src/expr_rewriter/mod.rs | 6 +-
datafusion/expr/src/expr_schema.rs | 12 +-
datafusion/expr/src/function.rs | 8 +-
datafusion/expr/src/groups_accumulator.rs | 6 +-
datafusion/expr/src/interval_arithmetic.rs | 35 +-
datafusion/expr/src/lib.rs | 3 +-
datafusion/expr/src/logical_plan/builder.rs | 6 +-
datafusion/expr/src/logical_plan/display.rs | 4 +-
datafusion/expr/src/logical_plan/plan.rs | 32 +-
datafusion/expr/src/partition_evaluator.rs | 2 +-
datafusion/expr/src/planner.rs | 31 +-
datafusion/expr/src/signature.rs | 2 +-
datafusion/expr/src/simplify.rs | 2 +-
datafusion/expr/src/tree_node.rs | 17 +-
datafusion/expr/src/type_coercion/aggregates.rs | 7 +-
datafusion/expr/src/type_coercion/binary.rs | 4 +-
datafusion/expr/src/type_coercion/functions.rs | 2 +-
datafusion/expr/src/type_coercion/mod.rs | 2 +-
datafusion/expr/src/udaf.rs | 185 +-----
datafusion/expr/src/udf.rs | 2 +-
datafusion/expr/src/udwf.rs | 51 +-
datafusion/expr/src/utils.rs | 91 +--
datafusion/expr/src/window_frame.rs | 99 ++-
datafusion/expr/src/window_function.rs | 99 +++
.../src/approx_percentile_cont.rs | 9 +-
.../src/approx_percentile_cont_with_weight.rs | 2 +-
.../src/array_agg.rs} | 388 +++++++----
.../functions-aggregate/src/bit_and_or_xor.rs | 47 ++
datafusion/functions-aggregate/src/count.rs | 8 +-
datafusion/functions-aggregate/src/first_last.rs | 15 +-
datafusion/functions-aggregate/src/lib.rs | 12 +-
datafusion/functions-aggregate/src/nth_value.rs | 7 +-
datafusion/functions-aggregate/src/stddev.rs | 6 +
.../Cargo.toml | 11 +-
.../README.md | 4 +-
.../benches/array_expression.rs | 2 +-
.../{functions => functions-nested}/benches/map.rs | 27 +-
.../src/array_has.rs | 0
.../src/cardinality.rs | 0
.../src/concat.rs | 0
.../src/dimension.rs | 0
.../src/empty.rs | 0
.../src/except.rs | 0
.../src/expr_ext.rs | 4 +-
.../src/extract.rs | 0
.../src/flatten.rs | 0
.../src/length.rs | 0
.../src/lib.rs | 17 +-
.../src/macros.rs | 0
.../src/make_array.rs | 0
.../src/core => functions-nested/src}/map.rs | 180 +----
.../src/planner.rs | 37 +-
.../src/position.rs | 0
.../src/range.rs | 0
.../src/remove.rs | 2 +-
.../src/repeat.rs | 0
.../src/replace.rs | 0
.../src/resize.rs | 0
.../src/reverse.rs | 0
.../src/set_ops.rs | 0
.../src/sort.rs | 0
.../src/string.rs | 0
.../src/utils.rs | 0
datafusion/functions/Cargo.toml | 5 -
datafusion/functions/src/core/arrow_cast.rs | 2 +-
datafusion/functions/src/core/mod.rs | 14 -
datafusion/functions/src/core/named_struct.rs | 15 +-
datafusion/functions/src/core/planner.rs | 27 +-
datafusion/functions/src/datetime/to_local_time.rs | 4 +-
datafusion/functions/src/lib.rs | 2 +-
datafusion/functions/src/macros.rs | 7 +-
datafusion/functions/src/math/bounds.rs | 108 +++
datafusion/functions/src/math/log.rs | 123 +++-
datafusion/functions/src/math/mod.rs | 305 ++++++++-
datafusion/functions/src/math/monotonicity.rs | 22 +-
datafusion/functions/src/math/signum.rs | 215 ++++++
datafusion/functions/src/regex/regexpreplace.rs | 2 +-
datafusion/functions/src/unicode/substrindex.rs | 8 +-
datafusion/optimizer/README.md | 318 +--------
.../optimizer/src/analyzer/count_wildcard_rule.rs | 24 +-
datafusion/optimizer/src/analyzer/subquery.rs | 8 +-
datafusion/optimizer/src/analyzer/type_coercion.rs | 29 +-
.../optimizer/src/common_subexpr_eliminate.rs | 6 +-
.../src/decorrelate_predicate_subquery.rs | 2 +-
datafusion/optimizer/src/lib.rs | 6 +-
.../optimizer/src/optimize_projections/mod.rs | 17 +-
.../src/optimize_projections/required_indices.rs | 2 +-
datafusion/optimizer/src/push_down_filter.rs | 18 +-
.../optimizer/src/replace_distinct_aggregate.rs | 2 +-
.../src/simplify_expressions/expr_simplifier.rs | 28 +-
.../src/simplify_expressions/guarantees.rs | 2 +-
.../optimizer/src/simplify_expressions/regex.rs | 1 +
.../optimizer/src/single_distinct_to_groupby.rs | 2 +-
.../optimizer/src/unwrap_cast_in_comparison.rs | 2 +-
.../optimizer/tests/optimizer_integration.rs | 2 +-
.../src/aggregate/groups_accumulator/accumulate.rs | 4 +-
.../physical-expr-common/src/aggregate/mod.rs | 309 +++++++--
.../physical-expr-common/src/aggregate/utils.rs | 2 +-
datafusion/physical-expr-common/src/binary_map.rs | 24 +-
.../physical-expr-common/src/expressions/column.rs | 48 +-
.../physical-expr-common/src/physical_expr.rs | 35 +-
datafusion/physical-expr-common/src/sort_expr.rs | 13 +-
datafusion/physical-expr-common/src/utils.rs | 23 +-
datafusion/physical-expr/Cargo.toml | 4 +
datafusion/physical-expr/benches/case_when.rs | 41 +-
.../benches/{case_when.rs => is_null.rs} | 91 +--
.../physical-expr/src/aggregate/array_agg.rs | 196 ------
.../src/aggregate/array_agg_distinct.rs | 446 -------------
datafusion/physical-expr/src/aggregate/build_in.rs | 104 +--
.../src/aggregate/groups_accumulator/adapter.rs | 2 +-
datafusion/physical-expr/src/aggregate/min_max.rs | 2 +-
datafusion/physical-expr/src/aggregate/mod.rs | 7 +-
datafusion/physical-expr/src/equivalence/class.rs | 2 +-
datafusion/physical-expr/src/equivalence/mod.rs | 4 +-
.../physical-expr/src/equivalence/properties.rs | 663 ++++++++++++++++---
datafusion/physical-expr/src/expressions/case.rs | 207 +++++-
.../physical-expr/src/expressions/is_not_null.rs | 4 +-
.../physical-expr/src/expressions/is_null.rs | 10 +
datafusion/physical-expr/src/expressions/mod.rs | 164 +----
.../physical-expr/src/expressions/try_cast.rs | 2 +-
.../expressions/{column.rs => unknown_column.rs} | 60 +-
datafusion/physical-expr/src/lib.rs | 2 +-
datafusion/physical-expr/src/partitioning.rs | 6 +-
datafusion/physical-expr/src/utils/guarantee.rs | 15 +-
.../Cargo.toml | 15 +-
.../physical-optimizer/README.md | 7 +-
.../src/lib.rs | 5 +-
datafusion/physical-optimizer/src/optimizer.rs | 48 ++
.../src}/output_requirements.rs | 24 +-
datafusion/physical-plan/Cargo.toml | 3 +
.../src/aggregates/group_values/row.rs | 2 +-
datafusion/physical-plan/src/aggregates/mod.rs | 248 +++----
.../physical-plan/src/aggregates/no_grouping.rs | 1 +
.../physical-plan/src/aggregates/order/mod.rs | 2 +-
.../physical-plan/src/aggregates/row_hash.rs | 3 +-
datafusion/physical-plan/src/analyze.rs | 2 +-
datafusion/physical-plan/src/coalesce_batches.rs | 38 +-
datafusion/physical-plan/src/common.rs | 356 +---------
datafusion/physical-plan/src/display.rs | 2 +-
datafusion/physical-plan/src/filter.rs | 74 +--
datafusion/physical-plan/src/insert.rs | 77 +--
datafusion/physical-plan/src/joins/cross_join.rs | 40 +-
datafusion/physical-plan/src/joins/hash_join.rs | 17 +-
.../physical-plan/src/joins/nested_loop_join.rs | 20 +-
.../physical-plan/src/joins/sort_merge_join.rs | 472 ++++++++++---
.../physical-plan/src/joins/symmetric_hash_join.rs | 6 +-
datafusion/physical-plan/src/joins/utils.rs | 12 +-
datafusion/physical-plan/src/lib.rs | 205 +++---
datafusion/physical-plan/src/limit.rs | 2 +-
datafusion/physical-plan/src/repartition/mod.rs | 6 +-
datafusion/physical-plan/src/sorts/sort.rs | 31 +-
datafusion/physical-plan/src/spill.rs | 186 ++++++
datafusion/physical-plan/src/streaming.rs | 2 +-
datafusion/physical-plan/src/test/exec.rs | 2 +-
datafusion/physical-plan/src/topk/mod.rs | 8 +-
datafusion/physical-plan/src/union.rs | 121 ++--
datafusion/physical-plan/src/unnest.rs | 3 +-
datafusion/physical-plan/src/windows/mod.rs | 41 +-
.../physical-plan/src/windows/window_agg_exec.rs | 2 +-
datafusion/physical-plan/src/work_table.rs | 2 +-
.../proto-common/proto/datafusion_common.proto | 8 +
datafusion/proto-common/src/from_proto/mod.rs | 7 +-
datafusion/proto-common/src/generated/pbjson.rs | 146 ++++
datafusion/proto-common/src/generated/prost.rs | 15 +-
datafusion/proto-common/src/lib.rs | 2 +
datafusion/proto-common/src/to_proto/mod.rs | 10 +-
datafusion/proto/proto/datafusion.proto | 38 +-
.../proto/src/generated/datafusion_proto_common.rs | 15 +-
datafusion/proto/src/generated/pbjson.rs | 123 +++-
datafusion/proto/src/generated/prost.rs | 19 +-
datafusion/proto/src/lib.rs | 2 +
datafusion/proto/src/logical_plan/file_formats.rs | 230 ++++---
datafusion/proto/src/logical_plan/from_proto.rs | 113 ++--
datafusion/proto/src/logical_plan/mod.rs | 29 +-
datafusion/proto/src/logical_plan/to_proto.rs | 86 ++-
datafusion/proto/src/physical_plan/from_proto.rs | 8 +-
datafusion/proto/src/physical_plan/mod.rs | 105 +--
datafusion/proto/src/physical_plan/to_proto.rs | 159 ++---
datafusion/proto/tests/cases/mod.rs | 99 +++
.../proto/tests/cases/roundtrip_logical_plan.rs | 301 +++++----
.../proto/tests/cases/roundtrip_physical_plan.rs | 361 +++++-----
datafusion/sql/examples/sql.rs | 23 +-
datafusion/sql/src/cte.rs | 2 +-
datafusion/sql/src/expr/function.rs | 59 +-
datafusion/sql/src/expr/identifier.rs | 111 ++--
datafusion/sql/src/expr/mod.rs | 14 +-
datafusion/sql/src/expr/order_by.rs | 10 +-
datafusion/sql/src/expr/substring.rs | 2 +-
datafusion/sql/src/expr/value.rs | 2 +-
datafusion/sql/src/lib.rs | 6 +-
datafusion/sql/src/parser.rs | 26 +-
datafusion/sql/src/planner.rs | 89 ++-
datafusion/sql/src/query.rs | 2 +-
datafusion/sql/src/relation/join.rs | 2 +-
datafusion/sql/src/relation/mod.rs | 2 +-
datafusion/sql/src/select.rs | 126 +++-
datafusion/sql/src/statement.rs | 182 +++--
datafusion/sql/src/unparser/ast.rs | 7 +-
datafusion/sql/src/unparser/dialect.rs | 223 ++++++-
datafusion/sql/src/unparser/expr.rs | 466 +++++++++++--
datafusion/sql/src/unparser/plan.rs | 145 ++--
datafusion/sql/src/unparser/rewrite.rs | 80 ++-
datafusion/sql/src/unparser/utils.rs | 4 +-
datafusion/sql/src/utils.rs | 29 +-
datafusion/sql/tests/cases/plan_to_sql.rs | 138 +++-
datafusion/sql/tests/common/mod.rs | 13 +-
datafusion/sql/tests/sql_integration.rs | 87 ++-
datafusion/sqllogictest/README.md | 2 +-
datafusion/sqllogictest/bin/sqllogictests.rs | 30 +-
datafusion/sqllogictest/test_files/aggregate.slt | 56 +-
datafusion/sqllogictest/test_files/array.slt | 4 +-
.../sqllogictest/test_files/arrow_typeof.slt | 2 +-
datafusion/sqllogictest/test_files/binary.slt | 2 +-
datafusion/sqllogictest/test_files/binary_view.slt | 2 +-
datafusion/sqllogictest/test_files/case.slt | 110 +++
datafusion/sqllogictest/test_files/coalesce.slt | 2 +-
datafusion/sqllogictest/test_files/copy.slt | 6 +-
.../sqllogictest/test_files/count_star_rule.slt | 105 +++
.../test_files/create_external_table.slt | 14 +-
datafusion/sqllogictest/test_files/csv_files.slt | 44 +-
datafusion/sqllogictest/test_files/encoding.slt | 2 +-
datafusion/sqllogictest/test_files/errors.slt | 4 -
datafusion/sqllogictest/test_files/explain.slt | 2 +-
datafusion/sqllogictest/test_files/expr.slt | 2 +-
datafusion/sqllogictest/test_files/group_by.slt | 142 ++--
.../sqllogictest/test_files/information_schema.slt | 75 ++-
datafusion/sqllogictest/test_files/interval.slt | 2 +-
datafusion/sqllogictest/test_files/join.slt | 42 ++
datafusion/sqllogictest/test_files/joins.slt | 28 +-
datafusion/sqllogictest/test_files/map.slt | 22 +-
datafusion/sqllogictest/test_files/math.slt | 8 +-
datafusion/sqllogictest/test_files/misc.slt | 6 +-
datafusion/sqllogictest/test_files/options.slt | 2 +-
datafusion/sqllogictest/test_files/order.slt | 38 +-
datafusion/sqllogictest/test_files/parquet.slt | 37 +-
datafusion/sqllogictest/test_files/predicates.slt | 4 +-
datafusion/sqllogictest/test_files/scalar.slt | 48 +-
datafusion/sqllogictest/test_files/select.slt | 92 ++-
.../sqllogictest/test_files/sort_merge_join.slt | 19 +-
datafusion/sqllogictest/test_files/strings.slt | 2 +-
datafusion/sqllogictest/test_files/struct.slt | 6 +-
datafusion/sqllogictest/test_files/timestamps.slt | 2 +-
datafusion/sqllogictest/test_files/union.slt | 49 +-
datafusion/sqllogictest/test_files/unnest.slt | 144 +++-
datafusion/sqllogictest/test_files/update.slt | 2 +-
datafusion/sqllogictest/test_files/window.slt | 14 +-
datafusion/substrait/src/extensions.rs | 157 +++++
datafusion/substrait/src/lib.rs | 1 +
datafusion/substrait/src/logical_plan/consumer.rs | 454 +++++++++----
datafusion/substrait/src/logical_plan/producer.rs | 736 +++++++++++----------
datafusion/substrait/src/variation_const.rs | 24 +-
.../tests/cases/roundtrip_logical_plan.rs | 137 ++--
dev/release/README.md | 2 +-
dev/release/crate-deps.dot | 10 +-
dev/release/crate-deps.svg | 22 +-
dev/update_datafusion_versions.py | 2 +-
docs/source/contributor-guide/communication.md | 25 +-
docs/source/index.rst | 2 +-
.../source/library-user-guide/query-optimizer.md | 0
docs/source/user-guide/configs.md | 54 +-
docs/source/user-guide/expressions.md | 2 +-
docs/source/user-guide/sql/scalar_functions.md | 2 +-
pre-commit.sh | 8 +-
dev/build-set-env.sh => rust-toolchain.toml | 6 +-
362 files changed, 11909 insertions(+), 6823 deletions(-)
create mode 100644 .github/workflows/large_files.yml
copy datafusion/{core/src/execution/mod.rs => common/src/scalar/consts.rs}
(50%)
create mode 100644 datafusion/core/benches/map_query_sql.rs
create mode 100644 datafusion/core/example.parquet
create mode 100644 datafusion/core/src/execution/session_state_defaults.rs
create mode 100644 datafusion/core/tests/data/newlines_in_values.csv
create mode 100644 datafusion/expr/src/window_function.rs
rename datafusion/{physical-expr/src/aggregate/array_agg_ordered.rs =>
functions-aggregate/src/array_agg.rs} (64%)
rename datafusion/{functions-array => functions-nested}/Cargo.toml (90%)
rename datafusion/{functions-array => functions-nested}/README.md (87%)
rename datafusion/{functions-array =>
functions-nested}/benches/array_expression.rs (95%)
rename datafusion/{functions => functions-nested}/benches/map.rs (83%)
rename datafusion/{functions-array => functions-nested}/src/array_has.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/cardinality.rs
(100%)
rename datafusion/{functions-array => functions-nested}/src/concat.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/dimension.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/empty.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/except.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/expr_ext.rs (95%)
rename datafusion/{functions-array => functions-nested}/src/extract.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/flatten.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/length.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/lib.rs (92%)
rename datafusion/{functions-array => functions-nested}/src/macros.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/make_array.rs
(100%)
rename datafusion/{functions/src/core => functions-nested/src}/map.rs (52%)
rename datafusion/{functions-array => functions-nested}/src/planner.rs (84%)
rename datafusion/{functions-array => functions-nested}/src/position.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/range.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/remove.rs (99%)
rename datafusion/{functions-array => functions-nested}/src/repeat.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/replace.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/resize.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/reverse.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/set_ops.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/sort.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/string.rs (100%)
rename datafusion/{functions-array => functions-nested}/src/utils.rs (100%)
create mode 100644 datafusion/functions/src/math/bounds.rs
create mode 100644 datafusion/functions/src/math/signum.rs
copy datafusion/physical-expr/benches/{case_when.rs => is_null.rs} (50%)
delete mode 100644 datafusion/physical-expr/src/aggregate/array_agg.rs
delete mode 100644 datafusion/physical-expr/src/aggregate/array_agg_distinct.rs
rename datafusion/physical-expr/src/expressions/{column.rs =>
unknown_column.rs} (51%)
copy datafusion/{common-runtime => physical-optimizer}/Cargo.toml (76%)
copy CODE_OF_CONDUCT.md => datafusion/physical-optimizer/README.md (79%)
copy datafusion/{common-runtime => physical-optimizer}/src/lib.rs (91%)
create mode 100644 datafusion/physical-optimizer/src/optimizer.rs
rename datafusion/{core/src/physical_optimizer =>
physical-optimizer/src}/output_requirements.rs (94%)
create mode 100644 datafusion/physical-plan/src/spill.rs
create mode 100644 datafusion/sqllogictest/test_files/case.slt
create mode 100644 datafusion/sqllogictest/test_files/count_star_rule.slt
create mode 100644 datafusion/substrait/src/extensions.rs
copy datafusion/optimizer/README.md =>
docs/source/library-user-guide/query-optimizer.md (100%)
copy dev/build-set-env.sh => rust-toolchain.toml (86%)
mode change 100755 => 100644
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
