This is an automated email from the ASF dual-hosted git repository.
agrove pushed a change to branch branch-34
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
omit e918aadfae Fix regression with Incorrect results when reading parquet
files with different schemas and statistics (#8533)
add 11542740a9 Add `today` alias for `current_date` (#8423)
add 2102275f64 remove useless clone (#8495)
add 7f312c8a1d fix: incorrect set preserve_partitioning in SortExec (#8485)
add 2919e32b8c Explicitly mark parquet for tests in datafusion-common
(#8497)
add 861cc36eb6 Minor/Doc: Clarify DataFrame::write_table Documentation
(#8519)
add 500ab40888 fix: Pull stats in `IdentVisitor`/`GraphvizVisitor` only
when requested (#8514)
add 4578f3daee Change display of RepartitionExec from
SortPreservingRepartitionExec to RepartitionExec preserve_order=true (#8521)
add 2bc67ef5a9 Fix `DataFrame::cache` errors with `Plan("Mismatch between
schema and batches")` (#8510)
add 0678a69785 Minor: update pbjson_dependency (#8470)
add 2e93f07946 Minor: Update prost-derive dependency (#8471)
add 9a322c8bdd Add write_table to dataframe actions in user guide (#8527)
add 5bf80d655d Minor: Add repartition_file.slt end to end test for
repartitioning files, and supporting tweaks (#8505)
add 898911b995 Prepare version 34.0.0 (#8508)
add cf2de9b22c refactor: use ExprBuilder to consume substrait expr and use
macro to generate error (#8515)
add 79c17e3f1a Make tests deterministic (#8525)
add 5909866bba fix: volatile expressions should not be target of common
subexpt elimination (#8520)
add 831b2ba6e0 Add LakeSoul to the list of Known Users (#8536)
add 974d49c907 Fix regression with Incorrect results when reading parquet
files with different schemas and statistics (#8533)
add 1042095211 feat: improve string statistics display (#8535)
add a971f1e7e3 Defer file creation to write (#8539)
add efa7b3421a Minor: Improve error handling in sqllogictest runner (#8544)
add e6e84e15df regenerate changelog
new 26933842e4 merge prep-34-rc3
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (e918aadfae)
\
N -- N -- N refs/heads/branch-34 (26933842e4)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/rust.yml | 3 +
datafusion-cli/src/functions.rs | 77 +++--
datafusion-cli/src/main.rs | 24 ++
datafusion-examples/Cargo.toml | 2 +-
datafusion/common/src/error.rs | 3 +
datafusion/common/src/file_options/file_type.rs | 1 +
datafusion/common/src/file_options/mod.rs | 8 +
datafusion/common/src/test_util.rs | 1 +
datafusion/core/src/dataframe/mod.rs | 35 ++-
datafusion/core/src/datasource/listing/helpers.rs | 8 +-
datafusion/core/src/datasource/listing/mod.rs | 5 +
datafusion/core/src/datasource/listing/url.rs | 5 +
.../core/src/datasource/listing_table_factory.rs | 17 +-
datafusion/core/src/datasource/stream.rs | 10 +-
.../src/physical_optimizer/enforce_distribution.rs | 38 +--
.../core/src/physical_optimizer/enforce_sorting.rs | 6 +-
.../src/physical_optimizer/projection_pushdown.rs | 3 +-
.../replace_with_order_preserving_variants.rs | 22 +-
datafusion/core/src/physical_planner.rs | 6 +-
datafusion/expr/src/built_in_function.rs | 2 +-
datafusion/expr/src/expr.rs | 75 ++++-
.../optimizer/src/common_subexpr_eliminate.rs | 18 +-
datafusion/physical-expr/src/array_expressions.rs | 2 +-
datafusion/physical-plan/src/display.rs | 127 +++++++-
datafusion/physical-plan/src/repartition/mod.rs | 12 +-
datafusion/proto/Cargo.toml | 2 +-
datafusion/sqllogictest/bin/sqllogictests.rs | 55 ++--
datafusion/sqllogictest/test_files/distinct_on.slt | 8 +-
datafusion/sqllogictest/test_files/functions.slt | 6 +
datafusion/sqllogictest/test_files/groupby.slt | 34 +--
.../sqllogictest/test_files/insert_to_external.slt | 18 +-
datafusion/sqllogictest/test_files/join.slt | 37 +++
datafusion/sqllogictest/test_files/joins.slt | 2 +-
.../sqllogictest/test_files/repartition_scan.slt | 268 +++++++++++++++++
datafusion/sqllogictest/test_files/timestamps.slt | 24 ++
datafusion/sqllogictest/test_files/window.slt | 10 +-
datafusion/substrait/src/logical_plan/consumer.rs | 324 ++++++++++-----------
dev/changelog/34.0.0.md | 26 ++
docs/source/user-guide/dataframe.md | 1 +
docs/source/user-guide/introduction.md | 1 +
docs/source/user-guide/sql/scalar_functions.md | 9 +
docs/source/user-guide/sql/write_options.md | 8 +-
42 files changed, 1005 insertions(+), 338 deletions(-)
create mode 100644 datafusion/sqllogictest/test_files/repartition_scan.slt
