This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/cargo/main/substrait-0.61.0
in repository https://gitbox.apache.org/repos/asf/datafusion.git
omit 0924318ab2 chore(deps): bump substrait from 0.58.0 to 0.61.0
add 04af0c61c8 perf: Implement boolean group values (#17726)
add e03ba614f5 Refactor: Update enforce_sorting tests to use insta
snapshots for easier updates (#17900)
add d76a1e0bed feat: support spark udf format_string (#17561)
add 8e73844fb3 #17838 Rewrite `regexp_like` calls as `~` and `*~` operator
expressions when possible (#17839)
add c1009e61b8 chore(deps): bump maturin from 1.9.4 to 1.9.5 in /docs
(#17940)
add 09f551cb36 chore(deps): bump flate2 from 1.1.2 to 1.1.4 (#17938)
add 014fca7071 chore(deps): bump actions/stale from 10.0.0 to 10.1.0
(#17937)
add bfd5e5582a chore(deps): bump aws-credential-types from 1.2.6 to 1.2.7
(#17936)
add 82aba7f629 chore(deps): bump rustyline from 17.0.1 to 17.0.2 (#17932)
add 24bc068073 chore(deps): bump taiki-e/install-action from 2.62.17 to
2.62.21 (#17934)
add 8769d9914a chore(deps): bump crate-ci/typos from 1.37.1 to 1.37.2
(#17935)
add 28fba58a64 chore: upgrade sqlparser (#17925)
add 307f5c3696 minor: impl Clone and Debug on CaseBuilder (#17927)
add 149d3e9533 chore: Extend backtrace coverage for `Execution` and
`Internal` errors (#17921)
add 9f70df15d3 chore(deps): bump taiki-e/install-action from 2.62.21 to
2.62.22 (#17949)
add 43dafd6bb7 chore(deps): bump crate-ci/typos from 1.37.2 to 1.38.0
(#17948)
add f8e988f830 Feat: [datafusion-spark] Migrate avg from comet to
datafusion-spark and add tests. (#17871)
add 5247a16042 Update tests to use insta / make them easier to update
(#17945)
add ab22ad8bd2 Test refactor: avoid creating the same SchemaRef (#17951)
add 7ce9b6ea06 docs: `Window::try_new_with_schema` with a descriptive
error message (#17926)
add ed5f8e7f01 Precision::<usize>::{add, sub, multiply}: avoid overflows
(#17929)
add 58ddf0d439 feat(spark): implement Spark `try_parse_url` function
(#17485)
add 1cc4dafbf9 Resolve `ListingScan` projection against table schema
including partition columns (#17911)
add d682d22d30 feat: Support reading CSV files with inconsistent column
counts (#17553)
add 0f515dc709 fix: fix failing test compilation on main (#17955)
add 4904a425ca perf: add to `aggregate_vectorized`
`PrimitiveGroupValueBuilder` benchmark as well (#17930)
add c39d1c5e67 chore(deps): bump crate-ci/typos from 1.38.0 to 1.38.1
(#17960)
add 60b37b403d chore(deps): bump taiki-e/install-action from 2.62.22 to
2.62.23 (#17959)
add 07a7eb28b0 bench: fix `vectorized_equal_to` bench mutated between
iterations (#17968)
add 5472e5da30 clean up duplicate information in FileOpener trait (#17956)
add 3b37ae06d5 fix docs and broken example from #17956 (#17980)
add f8ff82a187 feat: Adds Instrumented Object Store Registry to
datafusion-cli (#17953)
add dd68592425 Refactor: Update `replace_with_order_preserving_variants`
tests to use insta snapshots for easier updates (#17962)
add 7a063d8c31 Support repartitioned() method in RepartitionExec (#17990)
add 40b72330bc Adds Instrumented Object Store to CLI (#17984)
add 5609447450 Migrate join_selection tests to snapshot-based testing
(#17974)
add 590ad293d1 fix: update `PrimitiveGroupValueBuilder` to match NaN
correctly (#17979)
add a497074eb3 bench: fix actually generate a lot of unique values in
benchmark table (#17967)
add 73b1f2bd72 Adds Instrument Mode for InstrumentedObjectStore in
datafusion-cli (#18000)
add 6479e43f86 Support `JOIN` pipe operator (#17969)
add dd4e2289b8 minor: refactor Spark ascii function to reuse DataFusion
ascii function code (#17965)
add 668eabe64e chore(deps): bump taiki-e/install-action from 2.62.23 to
2.62.24 (#17989)
add 0ffc4333f8 chore(deps): bump taiki-e/install-action from 2.62.24 to
2.62.25 (#18007)
add 37ad8a5418 Clarify documentation that ScalarUDFImpl::simplity must not
change the schema (#17981)
add 832ba76125 fix: Add overflow checks to SparkDateAdd/Sub to avoid
panics (#18013)
add 8772411de8 Expose trace_future and trace_block outside of
common-runtime (#17976)
add f210939ecc Adds Object Store Profiling options/commands to CLI (#18004)
add a1acf775a9 Adds instrumentation to get requests for datafusion-cli
(#18016)
add fd3ccf8e18 refactor : deprecate `ParquetSource::predicate()` and merge
into `FileSource::filter()` (#17971)
add f739fd8434 docs: typo in `working-with-exprs.md` (#18033)
add 556eb9b1ef feat: convert_array_to_scalar_vec respects null elements
(#17891)
add 450fbfa26e feat: add cargo-machete in CI (#18030)
add d659c48599 chore(deps): bump half from 2.6.0 to 2.7.0 (#18036)
add fb7ef1a65e chore(deps): bump maturin from 1.9.5 to 1.9.6 in /docs
(#18039)
add f6c6fe0f25 chore(deps): bump aws-config from 1.8.6 to 1.8.7 (#18038)
add b27fedcd32 chore(deps): bump taiki-e/install-action from 2.62.25 to
2.62.28 (#18037)
add 6b1583aa0c chore(deps): bump substrait from 0.58.0 to 0.61.0
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0924318ab2)
\
N -- N -- N refs/heads/dependabot/cargo/main/substrait-0.61.0
(6b1583aa0c)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/audit.yml | 2 +-
.github/workflows/dependencies.yml | 11 +
.github/workflows/rust.yml | 6 +-
.github/workflows/stale.yml | 2 +-
Cargo.lock | 52 +-
Cargo.toml | 4 +-
datafusion-cli/Cargo.toml | 5 +-
datafusion-cli/examples/cli-session-context.rs | 5 +-
datafusion-cli/src/command.rs | 93 +-
datafusion-cli/src/main.rs | 17 +
datafusion-cli/src/object_storage.rs | 7 +-
datafusion-cli/src/object_storage/instrumented.rs | 423 ++
datafusion-cli/src/print_options.rs | 88 +-
datafusion-cli/tests/cli_integration.rs | 79 +-
.../object_store_profiling@s3_url_fallback.snap | 53 +
.../examples/advanced_parquet_index.rs | 13 +-
datafusion-examples/examples/function_factory.rs | 6 +-
datafusion-examples/examples/json_shredding.rs | 14 +-
.../examples/memory_pool_execution_plan.rs | 11 +-
datafusion/catalog-listing/Cargo.toml | 1 -
datafusion/catalog/src/listing_schema.rs | 18 +-
datafusion/common-runtime/src/lib.rs | 4 +-
datafusion/common/src/config.rs | 12 +-
datafusion/common/src/dfschema.rs | 4 +-
datafusion/common/src/scalar/mod.rs | 164 +-
datafusion/common/src/stats.rs | 49 +-
datafusion/common/src/utils/memory.rs | 9 +-
datafusion/common/src/utils/mod.rs | 18 +-
datafusion/core/Cargo.toml | 5 +-
datafusion/core/benches/aggregate_query_sql.rs | 32 +
datafusion/core/benches/data_utils/mod.rs | 11 +-
datafusion/core/src/dataframe/mod.rs | 12 +-
datafusion/core/src/dataframe/parquet.rs | 3 +-
.../core/src/datasource/file_format/arrow.rs | 7 +-
datafusion/core/src/datasource/listing/table.rs | 14 +-
.../src/datasource/physical_plan/arrow_file.rs | 25 +-
.../core/src/datasource/physical_plan/mod.rs | 1 -
.../core/src/datasource/physical_plan/parquet.rs | 5 +-
datafusion/core/src/execution/context/mod.rs | 57 +-
datafusion/core/src/physical_planner.rs | 253 +-
datafusion/core/tests/csv_schema_fix_test.rs | 122 +
datafusion/core/tests/dataframe/mod.rs | 8 +-
datafusion/core/tests/execution/coop.rs | 4 +-
.../tests/fuzz_cases/aggregation_fuzzer/fuzzer.rs | 6 +-
.../core/tests/fuzz_cases/equivalence/utils.rs | 10 +-
datafusion/core/tests/fuzz_cases/once_exec.rs | 6 +-
datafusion/core/tests/parquet/custom_reader.rs | 10 +-
datafusion/core/tests/parquet/encryption.rs | 8 +-
.../tests/physical_optimizer/enforce_sorting.rs | 4404 ++++++++++----------
.../physical_optimizer/filter_pushdown/util.rs | 8 +-
.../tests/physical_optimizer/join_selection.rs | 36 +-
.../replace_with_order_preserving_variants.rs | 1664 ++++----
.../core/tests/physical_optimizer/test_utils.rs | 52 +-
datafusion/core/tests/sql/aggregates/basic.rs | 2 +-
datafusion/core/tests/sql/select.rs | 10 +-
.../user_defined/user_defined_scalar_functions.rs | 8 +-
datafusion/datasource-avro/src/source.rs | 14 +-
datafusion/datasource-csv/src/file_format.rs | 124 +-
datafusion/datasource-csv/src/source.rs | 19 +-
datafusion/datasource-json/src/source.rs | 16 +-
datafusion/datasource-parquet/src/opener.rs | 183 +-
datafusion/datasource-parquet/src/reader.rs | 39 +-
datafusion/datasource-parquet/src/source.rs | 23 +-
datafusion/datasource/Cargo.toml | 2 +-
datafusion/datasource/src/file_groups.rs | 4 +-
datafusion/datasource/src/file_meta.rs | 53 -
datafusion/datasource/src/file_scan_config.rs | 21 +-
datafusion/datasource/src/file_stream.rs | 19 +-
datafusion/datasource/src/mod.rs | 14 +-
datafusion/datasource/src/write/demux.rs | 25 +-
datafusion/datasource/src/write/orchestration.rs | 12 +-
datafusion/execution/src/object_store.rs | 6 +-
datafusion/execution/src/parquet_encryption.rs | 6 +-
datafusion/execution/src/task.rs | 6 +-
.../src/type_coercion/binary/tests/arithmetic.rs | 4 +-
datafusion/expr/src/conditional_expressions.rs | 1 +
datafusion/expr/src/logical_plan/builder.rs | 12 +-
datafusion/expr/src/logical_plan/display.rs | 15 +-
datafusion/expr/src/logical_plan/plan.rs | 14 +-
datafusion/expr/src/simplify.rs | 10 +-
datafusion/expr/src/udaf.rs | 7 +
datafusion/expr/src/udf.rs | 8 +
datafusion/expr/src/udwf.rs | 7 +
datafusion/expr/src/window_state.rs | 8 +-
datafusion/ffi/Cargo.toml | 1 +
datafusion/ffi/src/record_batch_stream.rs | 16 +-
datafusion/ffi/src/tests/async_provider.rs | 15 +-
datafusion/ffi/src/udaf/accumulator_args.rs | 10 +-
datafusion/ffi/src/udaf/mod.rs | 5 +-
datafusion/ffi/src/udwf/mod.rs | 7 +-
.../ffi/src/udwf/partition_evaluator_args.rs | 3 +-
datafusion/ffi/src/util.rs | 16 +-
.../functions-aggregate-common/src/merge_arrays.rs | 26 +-
datafusion/functions-aggregate-common/src/utils.rs | 16 +-
.../functions-aggregate/src/approx_distinct.rs | 11 +-
datafusion/functions-aggregate/src/array_agg.rs | 11 +-
datafusion/functions-aggregate/src/nth_value.rs | 6 +-
datafusion/functions-nested/src/array_has.rs | 5 +-
datafusion/functions-nested/src/extract.rs | 8 +-
datafusion/functions-table/src/generate_series.rs | 4 +-
datafusion/functions-window/src/ntile.rs | 6 +-
datafusion/functions/src/datetime/common.rs | 11 +-
datafusion/functions/src/datetime/date_trunc.rs | 18 +-
datafusion/functions/src/datetime/to_local_time.rs | 13 +-
datafusion/functions/src/datetime/to_timestamp.rs | 4 +-
datafusion/functions/src/encoding/inner.rs | 27 +-
datafusion/functions/src/math/round.rs | 2 +-
datafusion/functions/src/regex/regexplike.rs | 72 +-
datafusion/functions/src/string/ascii.rs | 6 +-
datafusion/optimizer/src/analyzer/type_coercion.rs | 17 +-
.../src/simplify_expressions/expr_simplifier.rs | 6 +-
datafusion/physical-expr/src/expressions/cast.rs | 9 +-
.../physical-expr/src/expressions/in_list.rs | 42 +-
datafusion/physical-expr/src/utils/mod.rs | 10 +-
datafusion/physical-expr/src/window/aggregate.rs | 9 +-
datafusion/physical-expr/src/window/window_expr.rs | 10 +-
.../physical-optimizer/src/filter_pushdown.rs | 46 +-
.../physical-plan/benches/aggregate_vectorized.rs | 359 +-
.../src/aggregates/group_values/mod.rs | 17 +-
.../multi_group_by/{primitive.rs => boolean.rs} | 219 +-
.../group_values/multi_group_by/bytes.rs | 8 +-
.../aggregates/group_values/multi_group_by/mod.rs | 23 +-
.../group_values/multi_group_by/primitive.rs | 56 +-
.../group_values/single_group_by/boolean.rs | 154 +
.../group_values/single_group_by/bytes.rs | 6 +-
.../aggregates/group_values/single_group_by/mod.rs | 1 +
.../src/aggregates/topk/hash_table.rs | 6 +-
.../physical-plan/src/aggregates/topk/heap.rs | 6 +-
.../physical-plan/src/aggregates/topk_stream.rs | 4 +-
datafusion/physical-plan/src/display.rs | 6 +-
datafusion/physical-plan/src/recursive_query.rs | 6 +-
datafusion/physical-plan/src/repartition/mod.rs | 49 +-
datafusion/physical-plan/src/union.rs | 10 +-
.../src/windows/bounded_window_agg_exec.rs | 36 +-
datafusion/proto/src/common.rs | 6 +-
datafusion/proto/src/logical_plan/file_formats.rs | 36 +-
datafusion/proto/src/logical_plan/mod.rs | 84 +-
datafusion/proto/src/physical_plan/from_proto.rs | 12 +-
datafusion/proto/src/physical_plan/mod.rs | 44 +-
datafusion/proto/src/physical_plan/to_proto.rs | 6 +-
.../proto/tests/cases/roundtrip_logical_plan.rs | 79 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 21 +-
datafusion/pruning/src/file_pruner.rs | 10 +-
datafusion/pruning/src/pruning_predicate.rs | 8 +-
datafusion/spark/Cargo.toml | 1 +
datafusion/spark/src/function/aggregate/avg.rs | 350 ++
datafusion/spark/src/function/aggregate/mod.rs | 17 +-
datafusion/spark/src/function/datetime/date_add.rs | 25 +-
datafusion/spark/src/function/datetime/date_sub.rs | 25 +-
.../src/function/datetime/make_dt_interval.rs | 20 +-
.../spark/src/function/datetime/make_interval.rs | 50 +-
datafusion/spark/src/function/math/factorial.rs | 6 +-
datafusion/spark/src/function/math/hex.rs | 6 +-
datafusion/spark/src/function/math/modulus.rs | 18 +-
datafusion/spark/src/function/string/ascii.rs | 117 +-
.../spark/src/function/string/format_string.rs | 2350 +++++++++++
datafusion/spark/src/function/string/mod.rs | 17 +-
datafusion/spark/src/function/url/mod.rs | 15 +-
datafusion/spark/src/function/url/parse_url.rs | 228 +-
.../src/function/url/try_parse_url.rs} | 61 +-
datafusion/spark/src/lib.rs | 11 +-
datafusion/sql/src/expr/function.rs | 5 +
datafusion/sql/src/expr/identifier.rs | 8 +-
datafusion/sql/src/expr/mod.rs | 8 +-
datafusion/sql/src/parser.rs | 8 +-
datafusion/sql/src/planner.rs | 18 +-
datafusion/sql/src/query.rs | 5 +-
datafusion/sql/src/relation/join.rs | 6 +-
datafusion/sql/src/statement.rs | 37 +-
datafusion/sql/src/unparser/expr.rs | 12 +-
datafusion/sql/src/unparser/plan.rs | 2 +-
datafusion/sql/src/utils.rs | 8 +-
datafusion/sqllogictest/bin/sqllogictests.rs | 10 +-
.../sqllogictest/regenerate/sqllogictests.rs | 2 +-
.../src/engines/datafusion_engine/normalize.rs | 12 +-
.../sqllogictest/test_files/pipe_operator.slt | 18 +
.../sqllogictest/test_files/regexp/regexp_like.slt | 60 +
.../spark/{math/expm1.slt => aggregate/avg.slt} | 36 +-
.../test_files/spark/datetime/date_add.slt | 16 +
.../test_files/spark/string/format_string.slt | 2296 +++++++++-
.../test_files/spark/string/printf.slt | 7 +-
.../test_files/spark/url/parse_url.slt | 104 +-
.../test_files/spark/url/try_parse_url.slt | 127 +
.../sqllogictest/test_files/string/string_view.slt | 2 +-
datafusion/wasmtest/Cargo.toml | 3 +
docs/requirements.txt | 2 +-
.../library-user-guide/building-logical-plans.md | 6 +-
.../library-user-guide/working-with-exprs.md | 8 +-
docs/source/user-guide/cli/usage.md | 10 +
docs/source/user-guide/sql/select.md | 26 +
190 files changed, 11778 insertions(+), 4766 deletions(-)
create mode 100644 datafusion-cli/src/object_storage/instrumented.rs
create mode 100644
datafusion-cli/tests/snapshots/object_store_profiling@s3_url_fallback.snap
create mode 100644 datafusion/core/tests/csv_schema_fix_test.rs
delete mode 100644 datafusion/datasource/src/file_meta.rs
copy
datafusion/physical-plan/src/aggregates/group_values/multi_group_by/{primitive.rs
=> boolean.rs} (64%)
create mode 100644
datafusion/physical-plan/src/aggregates/group_values/single_group_by/boolean.rs
create mode 100644 datafusion/spark/src/function/aggregate/avg.rs
create mode 100644 datafusion/spark/src/function/string/format_string.rs
copy datafusion/{functions/src/math/pi.rs =>
spark/src/function/url/try_parse_url.rs} (51%)
copy datafusion/sqllogictest/test_files/spark/{math/expm1.slt =>
aggregate/avg.slt} (56%)
create mode 100644
datafusion/sqllogictest/test_files/spark/url/try_parse_url.slt
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
