This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/cargo/main/nix-0.27.1
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
omit ac8c6bba98 Update nix requirement from 0.26.1 to 0.27.1
add 18a03bdb9b Support Configuring Arrow RecordBatch Writers via SQL
Statement Options (#7390)
add 85ec314336 Add ROLLUP and GROUPING SETS substrait support (#7382)
add 7bca665407 Refactor sort_fuzz test to clarify what is covered (#7439)
add e83ed72ef7 Use DateTime::from_naive_utc_and_offset instead of
DateTime::from_utc (#7451)
add 6b618d2a0f Update substrait requirement from 0.12.0 to 0.13.1 (#7443)
add 58fc80edda [minior fix]: adjust the projection statistics (#7428)
add f0d3d67cc1 Add new known users: Arroyo and Restate (#7464)
add fc1fd9b63f ScalarFunctionExpr Maintaining Order (#7417)
add 3ea870cf09 Bug-fix/find_orderings_of_exprs (#7457)
add b34e7cfd55 Update prost-derive requirement from 0.11 to 0.12 (#7468)
add 716e197de8 Revert "Update prost-derive requirement from 0.11 to 0.12
(#7468)" (#7476)
add a2caad0cb4 Return error if spill file does not exist in ExternalSorter
(#7479)
add 03bbc0da83 Remove unused file_type.rs (#7478)
add c4a61852d7 Minor: more flexible pool size setting for datafusion-cli
(#7454)
add 47fd9bf5b7 Bump actions/checkout from 3 to 4 (#7480)
add 87eb126eba Support Write Options in DataFrame::write_* methods (#7435)
add e83b75f65b cp_solver, Duration vs Interval cases (#7475)
add 3bbd1f0486 feat: Allow creating a ValuesExec from record batches
(#7444)
add 18aef7c165 Make `LogicalPlan::with_new_exprs,` deprecate `from_plan`
(#7396)
add dfad0afc34 refactor: change file type logic for create table (#7477)
add 4e33be5e12 minor: do not fail analyzer if subquery plan contains
extension (#7455)
add dfd685187d Make in_list work with multiple items (#7449)
add 81d7f886e3 Minor: Add doc comments and example for
`ScalarVaue::to_scalar` (#7491)
add 0b3a0545e3 minor: Add ARROW to `CREATE EXTERNAL TABLE` docs and add
example of `COMPRESSION TYPE` (#7489)
add 2ffda2a9a8 Add backtrace to error messages (#7434)
add 7c6d731281 Make sqllogictest platform-independent for the sign of NaN
(#7462)
add ec1c8b8931 Support Configuring Parquet Column Specific Options via SQL
Statement Options (#7466)
add 63e452ace0 Minor: improve error message (#7498)
add 3a52ee1dd8 paralellize parquet (#7483)
add 4d44512946 `PrimitiveGroupsAccumulator` should propagate timestamp
timezone information properly (#7494)
add 93c209f1b5 Minor: Add `ScalarValue::data_type()` for consistency with
other APIs (#7492)
add 73d6d5f50a feat: explain with statistics (#7459)
add fda561e803 fix: incorrect nullability calculation of InListExpr (#7496)
add 270dc8ba76 Prepare 31.0.0 release (#7508)
new 77c227c22a Update nix requirement from 0.26.1 to 0.27.1
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ac8c6bba98)
\
N -- N -- N refs/heads/dependabot/cargo/main/nix-0.27.1
(77c227c22a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/dev.yml | 4 +-
.github/workflows/dev_pr.yml | 2 +-
.github/workflows/docs.yaml | 4 +-
.github/workflows/docs_pr.yaml | 2 +-
.github/workflows/rust.yml | 38 +-
Cargo.toml | 2 +-
benchmarks/Cargo.toml | 8 +-
datafusion-cli/Cargo.lock | 178 +++----
datafusion-cli/Cargo.toml | 5 +-
datafusion-cli/src/helper.rs | 8 +-
datafusion-cli/src/main.rs | 138 +++++-
datafusion-examples/examples/dataframe-to-s3.rs | 12 +-
datafusion/CHANGELOG.md | 1 +
datafusion/common/Cargo.toml | 4 +-
datafusion/common/src/column.rs | 4 +-
datafusion/common/src/config.rs | 4 +
datafusion/common/src/dfschema.rs | 16 +-
datafusion/common/src/error.rs | 77 ++-
.../src/file_options/arrow_writer.rs} | 27 +-
.../src/file_options/avro_writer.rs} | 27 +-
datafusion/common/src/file_options/csv_writer.rs | 120 +++++
.../common/src/{ => file_options}/file_type.rs | 28 +-
datafusion/common/src/file_options/json_writer.rs | 59 +++
datafusion/common/src/file_options/mod.rs | 540 +++++++++++++++++++++
.../common/src/file_options/parquet_writer.rs | 216 +++++++++
datafusion/common/src/file_options/parse_utils.rs | 196 ++++++++
datafusion/common/src/lib.rs | 6 +-
datafusion/common/src/parsers.rs | 2 +-
datafusion/common/src/scalar.rs | 52 +-
datafusion/common/src/stats.rs | 21 +
datafusion/core/Cargo.toml | 14 +-
datafusion/core/src/catalog/mod.rs | 2 +-
datafusion/core/src/dataframe.rs | 160 +++++-
.../core/src/datasource/file_format/arrow.rs | 6 +-
datafusion/core/src/datasource/file_format/avro.rs | 5 +
datafusion/core/src/datasource/file_format/csv.rs | 84 ++--
datafusion/core/src/datasource/file_format/json.rs | 27 +-
datafusion/core/src/datasource/file_format/mod.rs | 5 +-
.../core/src/datasource/file_format/parquet.rs | 280 +++--------
.../core/src/datasource/file_format/write.rs | 16 +-
datafusion/core/src/datasource/listing/table.rs | 43 +-
.../core/src/datasource/listing_table_factory.rs | 85 ++--
datafusion/core/src/datasource/memory.rs | 22 +-
.../core/src/datasource/physical_plan/csv.rs | 10 +-
.../core/src/datasource/physical_plan/json.rs | 8 +-
.../core/src/datasource/physical_plan/mod.rs | 9 +-
.../core/src/datasource/physical_plan/parquet.rs | 8 +-
datafusion/core/src/execution/context.rs | 8 +-
.../core/src/physical_optimizer/join_selection.rs | 2 +-
.../src/physical_optimizer/pipeline_checker.rs | 2 +-
datafusion/core/src/physical_plan/analyze.rs | 18 +-
datafusion/core/src/physical_plan/display.rs | 54 ++-
datafusion/core/src/physical_plan/filter.rs | 2 +-
datafusion/core/src/physical_plan/projection.rs | 102 +++-
datafusion/core/src/physical_plan/sorts/sort.rs | 6 +
datafusion/core/src/physical_plan/stream.rs | 2 +-
datafusion/core/src/physical_plan/values.rs | 69 ++-
datafusion/core/src/physical_planner.rs | 58 ++-
datafusion/core/tests/config_from_env.rs | 2 +-
datafusion/core/tests/dataframe/mod.rs | 10 +-
datafusion/core/tests/fuzz_cases/mod.rs | 2 +-
.../{order_spill_fuzz.rs => sort_fuzz.rs} | 144 ++++--
datafusion/core/tests/sql/explain_analyze.rs | 18 +
datafusion/core/tests/sql/joins.rs | 2 +-
datafusion/core/tests/sql/order.rs | 4 +-
datafusion/core/tests/sql/sql_api.rs | 10 +-
datafusion/core/tests/sql/timestamp.rs | 4 +-
datafusion/execution/Cargo.toml | 4 +-
datafusion/execution/src/disk_manager.rs | 2 +-
datafusion/execution/src/memory_pool/pool.rs | 10 +-
datafusion/execution/src/object_store.rs | 10 +-
datafusion/expr/Cargo.toml | 2 +-
datafusion/expr/src/expr_rewriter/mod.rs | 2 +-
datafusion/expr/src/logical_plan/builder.rs | 18 +-
datafusion/expr/src/logical_plan/dml.rs | 49 +-
datafusion/expr/src/logical_plan/plan.rs | 444 +++++++++++++++--
datafusion/expr/src/type_coercion/aggregates.rs | 6 +-
datafusion/expr/src/type_coercion/binary.rs | 14 +-
datafusion/expr/src/utils.rs | 340 +------------
datafusion/expr/src/window_frame.rs | 10 +-
datafusion/optimizer/Cargo.toml | 10 +-
datafusion/optimizer/src/analyzer/subquery.rs | 55 +++
datafusion/optimizer/src/analyzer/type_coercion.rs | 17 +-
datafusion/optimizer/src/optimizer.rs | 8 +-
datafusion/optimizer/src/push_down_filter.rs | 6 +-
.../src/simplify_expressions/simplify_exprs.rs | 4 +-
.../optimizer/src/unwrap_cast_in_comparison.rs | 19 +-
.../optimizer/tests/optimizer_integration.rs | 2 +-
datafusion/physical-expr/Cargo.toml | 6 +-
.../src/aggregate/approx_percentile_cont.rs | 4 +-
datafusion/physical-expr/src/aggregate/average.rs | 12 +-
.../src/aggregate/groups_accumulator/prim_op.rs | 8 +-
datafusion/physical-expr/src/aggregate/min_max.rs | 57 ++-
datafusion/physical-expr/src/aggregate/utils.rs | 24 -
datafusion/physical-expr/src/array_expressions.rs | 2 +-
.../physical-expr/src/datetime_expressions.rs | 20 +-
datafusion/physical-expr/src/expressions/binary.rs | 6 +-
datafusion/physical-expr/src/expressions/column.rs | 27 +-
.../physical-expr/src/expressions/in_list.rs | 144 +++++-
datafusion/physical-expr/src/expressions/mod.rs | 54 ++-
datafusion/physical-expr/src/functions.rs | 109 ++++-
.../physical-expr/src/intervals/cp_solver.rs | 395 +++++++++++++--
.../src/intervals/interval_aritmetic.rs | 32 --
datafusion/physical-expr/src/intervals/mod.rs | 5 +-
datafusion/physical-expr/src/intervals/utils.rs | 192 ++++++++
datafusion/physical-expr/src/regex_expressions.rs | 4 +-
datafusion/physical-expr/src/scalar_function.rs | 18 +
datafusion/physical-expr/src/udf.rs | 1 +
datafusion/physical-expr/src/utils.rs | 62 ++-
datafusion/proto/Cargo.toml | 8 +-
datafusion/proto/src/physical_plan/from_proto.rs | 2 +
datafusion/proto/src/physical_plan/mod.rs | 7 +-
datafusion/sql/Cargo.toml | 4 +-
datafusion/sql/src/expr/identifier.rs | 8 +-
datafusion/sql/src/expr/value.rs | 4 +-
datafusion/sql/src/parser.rs | 2 +-
datafusion/sql/src/query.rs | 6 +-
datafusion/sql/src/statement.rs | 75 ++-
datafusion/sql/tests/sql_integration.rs | 211 ++++----
datafusion/sqllogictest/Cargo.toml | 6 +-
datafusion/sqllogictest/src/engines/conversion.rs | 24 +-
datafusion/sqllogictest/test_files/copy.slt | 120 ++++-
datafusion/sqllogictest/test_files/explain.slt | 36 ++
.../sqllogictest/test_files/information_schema.slt | 1 +
datafusion/sqllogictest/test_files/joins.slt | 68 +--
datafusion/sqllogictest/test_files/math.slt | 6 +-
datafusion/sqllogictest/test_files/order.slt | 135 +++++-
datafusion/sqllogictest/test_files/predicates.slt | 93 +++-
datafusion/sqllogictest/test_files/scalar.slt | 55 +--
datafusion/sqllogictest/test_files/select.slt | 6 +-
datafusion/sqllogictest/test_files/subquery.slt | 46 +-
datafusion/substrait/Cargo.toml | 6 +-
datafusion/substrait/src/logical_plan/consumer.rs | 43 +-
datafusion/substrait/src/logical_plan/producer.rs | 96 +++-
.../tests/cases/roundtrip_logical_plan.rs | 17 +
dev/changelog/31.0.0.md | 123 +++++
docs/source/user-guide/configs.md | 3 +-
docs/source/user-guide/introduction.md | 2 +
docs/source/user-guide/sql/ddl.md | 14 +-
139 files changed, 4903 insertions(+), 1642 deletions(-)
copy datafusion/{proto/examples/expr_serde.rs =>
common/src/file_options/arrow_writer.rs} (63%)
copy datafusion/{proto/examples/expr_serde.rs =>
common/src/file_options/avro_writer.rs} (64%)
create mode 100644 datafusion/common/src/file_options/csv_writer.rs
rename datafusion/common/src/{ => file_options}/file_type.rs (94%)
create mode 100644 datafusion/common/src/file_options/json_writer.rs
create mode 100644 datafusion/common/src/file_options/mod.rs
create mode 100644 datafusion/common/src/file_options/parquet_writer.rs
create mode 100644 datafusion/common/src/file_options/parse_utils.rs
rename datafusion/core/tests/fuzz_cases/{order_spill_fuzz.rs => sort_fuzz.rs}
(52%)
create mode 100644 datafusion/physical-expr/src/intervals/utils.rs
create mode 100644 dev/changelog/31.0.0.md
