This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/cargo/main/proto-c40a7389eb
in repository https://gitbox.apache.org/repos/asf/datafusion.git
omit 7836d65144 chore(deps): bump the proto group with 3 updates
add 52bbc8afcf chore(deps): bump insta from 1.45.0 to 1.46.0 (#19643)
add 47df535d2c chore(deps): bump taiki-e/install-action from 2.65.11 to
2.65.13 (#19646)
add c8620129f3 chore(deps): bump tracing from 0.1.43 to 0.1.44 (#19644)
add fd79241633 chore(deps): bump syn from 2.0.111 to 2.0.113 (#19645)
add ada0923a39 Respect execution timezone in to_timestamp and related
functions (#19078)
add 9b2505ce6a fix(doc): close #19393, make upgrading guide match v51 api
(#19648)
add 2d56253893 fix(spark): Use wrapping addition/subtraction in
`SparkDateAdd` and `SparkDateSub` (#19377)
add ff38480f22 Refactor `percentile_cont` to clarify support input types
(#19611)
add aee5cd9f35 fix(functions): Make translate function postgres compatible
(#19630)
add adf00a6495 Add a protection to release candidate branch 52 (#19660)
add 5c2ee3650d perf: optimize `HashTableLookupExpr::evaluate` (#19602)
add c3e1c3644a Downgrade aws-smithy-runtime, update `rust_decimal`, ignore
RUSTSEC-2026-0001 to get clean CI (#19657)
add 1037f0aa20 feat: add list_files_cache table function for
`datafusion-cli` (#19388)
add 924037ea04 perf: Improve performance of `split_part` (#19570)
add a2f02f069a fix: Return Int for Date - Date instead of duration (#19563)
add 7942e751cb Update dependencies (#19667)
add ed01b67f2d Refactor PartitionedFile: add ordering field and
new_from_meta constructor (#19596)
add e8196f462f Remove coalesce batches rule and deprecate
CoalesceBatchesExec (#19622)
add 166ef81121 Perf: Optimize `substring_index` via single-byte fast path
and direct indexing (#19590)
add 1f654bbe62 feat: implement metrics for AsyncFuncExec (#19626)
add ce08307a4c refactor: Use `Signature::coercible` for isnan/iszero
(#19604)
add 680ddcc6cb feat: split BatchPartitioner::try_new into hash and
round-robin constructors (#19668)
add 566bcde9e1 Parquet: Push down supported list predicates
(array_has/any/all) during decoding (#19545)
add 3a0ca4ef79 Remove dependency on `rust_decimal`, remove ignore of
`RUSTSEC-2026-0001` (#19666)
add 142f5972d5 Store example data directly inside the datafusion-examples
(#19141) (#19319)
add 35ff4ab0a0 Allow logical optimizer to be run without evaluating now()
& refactor SimplifyInfo (#19505)
add 646213ec7c feat: add Time type support to date_trunc function (#19640)
add 102caeb226 minor: More comments to `ParquetOpener::open()` (#19677)
add d18e670e78 feat: Allow log with non-integer base on decimals (#19372)
add 1d5d63c413 Feat: Allow pow with negative & non-integer exponent on
decimals (#19369)
add e6049de5a7 Make default ListingFilesCache table scoped (#19616)
add 5194fd5eb0 chore(deps): bump taiki-e/install-action from 2.65.13 to
2.65.15 (#19676)
add 0cf45cae9c Refactor cache APIs to support ordering information (#19597)
add b9a3b9f94f Record sort order when writing Parquet with WITH ORDER
(#19595)
add a55b77e7de fix: DynamicFilterPhysicalExpr violates Hash/Eq contract
(#19659)
add 62658cd624 implement var distinct (#19706)
add c98fa5616e perfect hash join (#19411)
add b7091c0d2f Optimize `Nullstate` / accumulators (#19625)
add 07e63edfa0 Fix TopK aggregation for UTF-8/Utf8View group keys and add
safe fallback for unsupported string aggregates (#19285)
add 8ba46466d3 docs: Fix two small issues in introduction.md (#19712)
add 209a0a2e86 fix: unnest struct field with an alias failed with internal
error (#19698)
add 20870da20d infer parquet file order from metadata and use it to
optimize scans (#19433)
add 5c2b1236b5 feat(spark): implement array_repeat function (#19702)
add 821d410fc0 feat(spark): Implement collect_list/collect_set aggregate
functions (#19699)
add 3087ca8a92 perf: optimize `NthValue` when `ignore_nulls` is true
(#19496)
add afc912106b Optimize `concat/concat_ws` scalar path by pre-allocating
memory (#19547)
add 45fb0b4b98 fix(accumulators): preserve state in evaluate() for window
frame queries (#19618)
add 458b49109a perf: optimize left function by eliminating double chars()
iteration (#19571)
add 4adadce8fe chore(deps): bump the proto group with 3 updates
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (7836d65144)
\
N -- N -- N refs/heads/dependabot/cargo/main/proto-c40a7389eb
(4adadce8fe)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 3 +
.github/workflows/audit.yml | 4 +-
.github/workflows/rust.yml | 4 +-
Cargo.lock | 1205 ++++++++------------
Cargo.toml | 2 +-
benchmarks/bench.sh | 6 +-
benchmarks/compare.py | 20 +-
benchmarks/src/hj.rs | 410 +++++--
datafusion-cli/src/functions.rs | 185 ++-
datafusion-cli/src/main.rs | 113 +-
datafusion-examples/Cargo.toml | 15 +-
datafusion-examples/README.md | 9 +-
datafusion-examples/data/README.md | 25 +
.../data => datafusion-examples/data/csv}/cars.csv | 0
.../data/csv}/regex.csv | 0
.../examples/builtin_functions/function_factory.rs | 4 +-
.../examples/builtin_functions/regexp.rs | 31 +-
.../examples/custom_data_source/csv_json_opener.rs | 32 +-
.../custom_data_source/csv_sql_streaming.rs | 19 +-
.../examples/data_io/parquet_encrypted.rs | 32 +-
.../examples/data_io/parquet_exec_visitor.rs | 24 +-
.../examples/dataframe/cache_factory.rs | 42 +-
.../examples/dataframe/dataframe.rs | 84 +-
.../examples/dataframe/deserialize_to_struct.rs | 321 +++++-
datafusion-examples/examples/dataframe/main.rs | 2 +-
.../examples/execution_monitoring/tracing.rs | 34 +-
datafusion-examples/examples/flight/client.rs | 17 +-
datafusion-examples/examples/flight/server.rs | 38 +-
datafusion-examples/examples/flight/sql_server.rs | 25 +-
.../examples/query_planning/expr_api.rs | 13 +-
.../examples/query_planning/parse_sql_expr.rs | 70 +-
.../examples/query_planning/plan_to_sql.rs | 80 +-
.../examples/query_planning/planner_api.rs | 23 +-
.../examples/query_planning/thread_pools.rs | 14 +-
datafusion-examples/examples/sql_ops/query.rs | 64 +-
datafusion-examples/examples/udf/advanced_udaf.rs | 15 +-
datafusion-examples/examples/udf/advanced_udwf.rs | 49 +-
datafusion-examples/examples/udf/async_udf.rs | 3 +-
datafusion-examples/examples/udf/simple_udtf.rs | 27 +-
.../mod.rs => datafusion-examples/src/lib.rs | 4 +-
datafusion-examples/src/utils/csv_to_parquet.rs | 245 ++++
.../src/utils/datasets/cars.rs | 16 +-
datafusion-examples/src/utils/datasets/mod.rs | 139 +++
.../src/utils/datasets/regex.rs | 22 +-
.../src/utils}/mod.rs | 5 +-
datafusion/catalog-listing/src/table.rs | 336 +++++-
datafusion/common/src/config.rs | 19 +
datafusion/common/src/dfschema.rs | 6 +
datafusion/core/benches/topk_aggregate.rs | 87 +-
datafusion/core/src/datasource/file_format/mod.rs | 12 +-
datafusion/core/src/datasource/listing/table.rs | 2 +-
.../core/src/datasource/listing_table_factory.rs | 7 +-
datafusion/core/src/datasource/mod.rs | 9 +-
.../core/src/datasource/physical_plan/parquet.rs | 84 +-
datafusion/core/src/execution/context/mod.rs | 35 +-
datafusion/core/src/execution/session_state.rs | 57 +-
datafusion/core/src/test_util/parquet.rs | 15 +-
datafusion/core/tests/execution/coop.rs | 40 +-
datafusion/core/tests/expr_api/mod.rs | 5 +-
datafusion/core/tests/expr_api/simplification.rs | 156 +--
datafusion/core/tests/parquet/custom_reader.rs | 10 +-
datafusion/core/tests/parquet/mod.rs | 1 +
datafusion/core/tests/parquet/ordering.rs | 103 ++
datafusion/core/tests/parquet/page_pruning.rs | 9 +-
.../physical_optimizer/aggregate_statistics.rs | 86 ++
.../physical_optimizer/enforce_distribution.rs | 11 +-
.../tests/physical_optimizer/enforce_sorting.rs | 29 +-
.../physical_optimizer/filter_pushdown/mod.rs | 306 ++---
.../tests/physical_optimizer/limit_pushdown.rs | 84 +-
.../physical_optimizer/partition_statistics.rs | 38 -
.../physical_optimizer/projection_pushdown.rs | 56 -
.../core/tests/physical_optimizer/pushdown_sort.rs | 76 +-
.../replace_with_order_preserving_variants.rs | 228 ++--
.../core/tests/physical_optimizer/test_utils.rs | 8 -
datafusion/core/tests/sql/explain_analyze.rs | 1 -
.../user_defined_async_scalar_functions.rs | 40 +-
.../user_defined/user_defined_scalar_functions.rs | 6 +-
datafusion/datasource-parquet/Cargo.toml | 7 +
.../benches/parquet_nested_filter_pushdown.rs | 238 ++++
datafusion/datasource-parquet/src/file_format.rs | 92 +-
datafusion/datasource-parquet/src/metadata.rs | 142 ++-
datafusion/datasource-parquet/src/mod.rs | 1 +
datafusion/datasource-parquet/src/opener.rs | 43 +-
datafusion/datasource-parquet/src/row_filter.rs | 487 +++++++-
.../datasource-parquet/src/row_group_filter.rs | 9 +-
.../datasource-parquet/src/supported_predicates.rs | 144 +++
datafusion/datasource/src/display.rs | 9 +-
datafusion/datasource/src/file_format.rs | 76 ++
datafusion/datasource/src/file_scan_config.rs | 71 +-
datafusion/datasource/src/mod.rs | 44 +
datafusion/datasource/src/url.rs | 84 +-
datafusion/execution/Cargo.toml | 1 +
datafusion/execution/src/cache/cache_manager.rs | 184 ++-
datafusion/execution/src/cache/cache_unit.rs | 454 ++++++--
.../execution/src/cache/file_metadata_cache.rs | 429 +++----
datafusion/execution/src/cache/list_files_cache.rs | 917 ++++++++-------
datafusion/execution/src/cache/mod.rs | 48 +-
datafusion/expr-common/src/accumulator.rs | 23 +-
datafusion/expr-common/src/type_coercion/binary.rs | 18 +
datafusion/expr/src/execution_props.rs | 16 +-
datafusion/expr/src/function.rs | 8 +-
datafusion/expr/src/simplify.rs | 120 +-
datafusion/expr/src/udaf.rs | 2 +-
datafusion/expr/src/udf.rs | 8 +-
datafusion/expr/src/udwf.rs | 2 +-
.../src/aggregate/groups_accumulator/accumulate.rs | 214 +++-
.../src/aggregate/groups_accumulator/bool_op.rs | 2 +-
.../src/aggregate/groups_accumulator/prim_op.rs | 5 +-
datafusion/functions-aggregate/src/array_agg.rs | 2 +-
datafusion/functions-aggregate/src/average.rs | 21 +-
datafusion/functions-aggregate/src/count.rs | 4 +-
.../functions-aggregate/src/percentile_cont.rs | 505 ++++----
datafusion/functions-aggregate/src/string_agg.rs | 13 +-
datafusion/functions-aggregate/src/variance.rs | 144 ++-
datafusion/functions-nested/src/array_has.rs | 21 +-
datafusion/functions-window/Cargo.toml | 8 +
datafusion/functions-window/benches/nth_value.rs | 263 +++++
datafusion/functions-window/src/nth_value.rs | 182 +--
datafusion/functions/Cargo.toml | 13 +-
datafusion/functions/benches/concat.rs | 100 +-
.../functions/benches/{concat.rs => concat_ws.rs} | 64 +-
datafusion/functions/benches/split_part.rs | 382 +++++++
datafusion/functions/benches/substr_index.rs | 124 +-
datafusion/functions/benches/to_timestamp.rs | 24 +-
datafusion/functions/src/core/arrow_cast.rs | 4 +-
datafusion/functions/src/core/coalesce.rs | 4 +-
datafusion/functions/src/core/getfield.rs | 2 +-
datafusion/functions/src/core/nvl.rs | 4 +-
datafusion/functions/src/core/nvl2.rs | 4 +-
datafusion/functions/src/datetime/common.rs | 153 ++-
datafusion/functions/src/datetime/current_date.rs | 23 +-
datafusion/functions/src/datetime/current_time.rs | 65 +-
datafusion/functions/src/datetime/date_trunc.rs | 284 +++--
datafusion/functions/src/datetime/mod.rs | 46 +-
datafusion/functions/src/datetime/now.rs | 18 +-
datafusion/functions/src/datetime/to_date.rs | 6 +-
datafusion/functions/src/datetime/to_timestamp.rs | 840 +++++++++++---
datafusion/functions/src/datetime/to_unixtime.rs | 2 +-
datafusion/functions/src/macros.rs | 29 +
datafusion/functions/src/math/iszero.rs | 32 +-
datafusion/functions/src/math/log.rs | 249 ++--
datafusion/functions/src/math/nans.rs | 32 +-
datafusion/functions/src/math/nanvl.rs | 27 +-
datafusion/functions/src/math/power.rs | 325 +++++-
datafusion/functions/src/regex/regexplike.rs | 4 +-
datafusion/functions/src/string/concat.rs | 11 +-
datafusion/functions/src/string/concat_ws.rs | 35 +-
datafusion/functions/src/string/split_part.rs | 38 +-
datafusion/functions/src/string/starts_with.rs | 9 +-
datafusion/functions/src/unicode/left.rs | 15 +-
datafusion/functions/src/unicode/substrindex.rs | 70 +-
datafusion/functions/src/unicode/translate.rs | 15 +-
datafusion/macros/Cargo.toml | 2 +-
datafusion/optimizer/src/decorrelate.rs | 10 +-
datafusion/optimizer/src/optimizer.rs | 28 +-
.../src/simplify_expressions/expr_simplifier.rs | 176 ++-
.../optimizer/src/simplify_expressions/mod.rs | 2 +-
.../src/simplify_expressions/simplify_exprs.rs | 13 +-
.../src/simplify_expressions/simplify_literal.rs | 7 +-
.../src/simplify_expressions/unwrap_cast.rs | 22 +-
datafusion/physical-expr/src/expressions/binary.rs | 94 ++
.../src/expressions/dynamic_filters.rs | 123 +-
.../physical-optimizer/src/coalesce_batches.rs | 87 --
datafusion/physical-optimizer/src/lib.rs | 1 -
datafusion/physical-optimizer/src/optimizer.rs | 4 -
.../physical-optimizer/src/topk_aggregation.rs | 21 +-
datafusion/physical-plan/Cargo.toml | 1 +
datafusion/physical-plan/src/aggregates/mod.rs | 70 +-
.../src/aggregates/topk/hash_table.rs | 178 +--
.../physical-plan/src/aggregates/topk/heap.rs | 192 +++-
.../src/aggregates/topk/priority_map.rs | 103 ++
.../physical-plan/src/aggregates/topk_stream.rs | 14 +
datafusion/physical-plan/src/async_func.rs | 10 +-
datafusion/physical-plan/src/coalesce_batches.rs | 7 +
datafusion/physical-plan/src/joins/array_map.rs | 547 +++++++++
datafusion/physical-plan/src/joins/chain.rs | 69 ++
.../physical-plan/src/joins/hash_join/exec.rs | 995 ++++++++++++----
.../src/joins/hash_join/partitioned_hash_eval.rs | 227 ++--
.../src/joins/hash_join/shared_bounds.rs | 29 +-
.../physical-plan/src/joins/hash_join/stream.rs | 66 +-
.../physical-plan/src/joins/join_hash_map.rs | 111 +-
datafusion/physical-plan/src/joins/mod.rs | 27 +
.../physical-plan/src/joins/stream_join_utils.rs | 14 +-
datafusion/physical-plan/src/repartition/mod.rs | 117 +-
.../src/sorts/sort_preserving_merge.rs | 18 +-
datafusion/proto/src/physical_plan/from_proto.rs | 61 +-
datafusion/proto/src/physical_plan/mod.rs | 4 +
.../proto/tests/cases/roundtrip_physical_plan.rs | 19 +-
datafusion/spark/Cargo.toml | 1 +
datafusion/spark/src/function/aggregate/collect.rs | 200 ++++
datafusion/spark/src/function/aggregate/mod.rs | 19 +-
datafusion/spark/src/function/array/mod.rs | 9 +-
datafusion/spark/src/function/array/repeat.rs | 128 +++
datafusion/spark/src/function/conditional/if.rs | 2 +-
datafusion/spark/src/function/datetime/date_add.rs | 25 +-
datafusion/spark/src/function/datetime/date_sub.rs | 25 +-
datafusion/spark/src/function/mod.rs | 1 +
datafusion/spark/src/function/null_utils.rs | 122 ++
datafusion/spark/src/function/string/concat.rs | 110 +-
datafusion/sql/src/utils.rs | 25 +-
datafusion/sqllogictest/Cargo.toml | 3 -
datafusion/sqllogictest/src/engines/conversion.rs | 6 +-
.../src/engines/postgres_engine/mod.rs | 131 ++-
.../src/engines/postgres_engine/types.rs | 45 -
datafusion/sqllogictest/test_files/aggregate.slt | 150 ++-
.../sqllogictest/test_files/aggregates_topk.slt | 88 ++
datafusion/sqllogictest/test_files/async_udf.slt | 12 +-
.../test_files/datetime/arith_date_date.slt | 11 +-
.../sqllogictest/test_files/datetime/dates.slt | 14 +-
.../test_files/datetime/timestamps.slt | 65 +-
datafusion/sqllogictest/test_files/decimal.slt | 40 +-
datafusion/sqllogictest/test_files/explain.slt | 4 -
.../sqllogictest/test_files/information_schema.slt | 59 +-
.../test_files/parquet_filter_pushdown.slt | 111 ++
.../test_files/spark/aggregate/collect.slt | 93 ++
.../test_files/spark/array/array_repeat.slt | 77 +-
.../test_files/spark/datetime/date_add.slt | 12 +-
.../test_files/string/string_query.slt.part | 10 +-
.../test_files/to_timestamp_timezone.slt | 204 ++++
datafusion/sqllogictest/test_files/unnest.slt | 30 +-
datafusion/substrait/src/physical_plan/consumer.rs | 12 +-
datafusion/wasmtest/src/lib.rs | 11 +-
docs/source/library-user-guide/upgrading.md | 269 +++--
docs/source/user-guide/cli/functions.md | 50 +
docs/source/user-guide/configs.md | 2 +
docs/source/user-guide/introduction.md | 5 +-
docs/source/user-guide/sql/scalar_functions.md | 93 +-
227 files changed, 13604 insertions(+), 5710 deletions(-)
create mode 100644 datafusion-examples/data/README.md
copy {datafusion/core/tests/data => datafusion-examples/data/csv}/cars.csv
(100%)
copy {datafusion/physical-expr/tests/data =>
datafusion-examples/data/csv}/regex.csv (100%)
copy datafusion/physical-expr/src/statistics/mod.rs =>
datafusion-examples/src/lib.rs (91%)
create mode 100644 datafusion-examples/src/utils/csv_to_parquet.rs
copy datafusion/spark/src/function/collection/mod.rs =>
datafusion-examples/src/utils/datasets/cars.rs (66%)
create mode 100644 datafusion-examples/src/utils/datasets/mod.rs
copy datafusion/spark/src/function/conditional/mod.rs =>
datafusion-examples/src/utils/datasets/regex.rs (67%)
copy {datafusion/physical-expr/src/statistics =>
datafusion-examples/src/utils}/mod.rs (90%)
create mode 100644 datafusion/core/tests/parquet/ordering.rs
create mode 100644
datafusion/datasource-parquet/benches/parquet_nested_filter_pushdown.rs
create mode 100644 datafusion/datasource-parquet/src/supported_predicates.rs
create mode 100644 datafusion/functions-window/benches/nth_value.rs
copy datafusion/functions/benches/{concat.rs => concat_ws.rs} (56%)
create mode 100644 datafusion/functions/benches/split_part.rs
delete mode 100644 datafusion/physical-optimizer/src/coalesce_batches.rs
create mode 100644 datafusion/physical-plan/src/joins/array_map.rs
create mode 100644 datafusion/physical-plan/src/joins/chain.rs
create mode 100644 datafusion/spark/src/function/aggregate/collect.rs
create mode 100644 datafusion/spark/src/function/array/repeat.rs
create mode 100644 datafusion/spark/src/function/null_utils.rs
delete mode 100644 datafusion/sqllogictest/src/engines/postgres_engine/types.rs
create mode 100644
datafusion/sqllogictest/test_files/spark/aggregate/collect.slt
create mode 100644 datafusion/sqllogictest/test_files/to_timestamp_timezone.slt
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
