This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/github_actions/taiki-e/install-action-2.62.36
in repository https://gitbox.apache.org/repos/asf/datafusion-sandbox.git
discard b0770a488 chore(deps): bump taiki-e/install-action from 2.61.8 to
2.62.36
add db7ce03e6 Try to unprotect main (#46)
add 432a85312 Unprotect 2 (#47)
add 631f9abf6 Use `Display` formatting of `DataType`:s in error messages
(#17565)
add 9351c7aa5 docs: Move Google Summer of Code 2025 pages to a section
(#17504)
add da893951c feat: Add `OR REPLACE` to creating external tables (#17580)
add e2a5b57cf `avg(distinct)` support for decimal types (#17560)
add 70f4d190e chore(deps): bump taiki-e/install-action from 2.61.8 to
2.61.9 (#17640)
add b293e2cac chore(deps): bump Swatinem/rust-cache from 2.8.0 to 2.8.1
(#17641)
add 13208e6a1 Validate the memory consumption in SPM created by multi
level merge (#17029)
add c910db4b9 fix(SubqueryAlias): use maybe_project_redundant_column
(#17478)
add f1765a177 minor: Ensure `datafusion-sql` package dependencies have
`sql` flag (#17644)
add 4528f2fa6 optimizer: Rewrite `IS NOT DISTINCT FROM` joins as Hash
Joins (#17319)
add 980c948d8 Upgrade to arrow 56.1.0 (#17275)
add 293bf3e42 fix: Preserves field metadata when creating logical plan for
VALUES expression (#17525)
add c1ca3c4d7 chore(deps): bump serde from 1.0.223 to 1.0.225 (#17614)
add d587b8d3b chore: Update dynamic filter formatting (#17647)
add 5f0b83cf1 chore(deps): bump taiki-e/install-action from 2.61.9 to
2.61.10 (#17660)
add 0aa6d35ce proto: don't include parquet feature by default (#17577)
add 4e94346e7 feat: add support for RightAnti and RightSemi join types
(#17604)
add ac42765ec minor: Ensure `proto` crate has datetime & unicode expr
flags in datafusion dev dependency (#17656)
add 7d50c37ed chore(deps): bump indexmap from 2.11.3 to 2.11.4 (#17661)
add 4a5b13799 docs: add xorq to list of known users (#17668)
add 35c1cfdd1 Introduce `TypeSignatureClass::Binary` to allow accepting
arbitrarily sized `FixedSizeBinary` arguments (#17531)
add b9517a1a0 docs: deduplicate links in `introduction.md` (#17669)
add d572eeb67 Add explicit PMC/committers list to governance docs page
(#17574)
add 7125e9775 fix: Ignore governance doc from typos (#17678)
add 44cd97254 Support Decimal32/64 types (#17501)
add 75c77204f minor: Improve hygiene for `datafusion-functions` macros
(#17638)
add 14656f575 feat(small): Display `NullEquality` in join executor's
`EXPLAIN` output (#17664)
add 03b67897e Custom timestamp format for DuckDB (#17653)
add 1488e1010 feat(substrait): add time literal support (#17655)
add 00f5b7d2e Support LargeList for array_sort (#17657)
add 523eadd3e Support FixedSizeList for array_except (#17658)
add d96fbde61 fix: null padding for `array_reverse` on `FixedSizeList`
(#17673)
add c2d839f59 chore: refactor array fn signatures & add more slt tests
(#17672)
add d55fb6d9f Support FixedSizeList for array_to_string (#17666)
add 60827273b fix: correct statistics for `NestedLoopJoinExec` (#17680)
add 602475fba minor: add SQLancer fuzzed SLT case for natural joins
(#17683)
add bea1b0a1c chore: Upgrade Rust version to 1.90.0 (#17677)
add 58bc7496b Support FixedSizeList for array_position (#17659)
add 78d5779c9 chore(deps): bump the proto group with 2 updates (#16806)
add 162942016 feat(spark): implement Spark `make_interval` function
(#17424)
add b63ca3e09 chore: Update READMEs of crates to be more consistent
(#17691)
add f04efd113 chore: update a bunch of dependencies (#17708)
add 68bc0ceb0 chore: fix wasm-pack installation link in wasmtest README
(#17704)
add ba50f5254 Support FixedSizeList for array_slice via coercion to List
(#17667)
add 768d23879 docs: Remove disclaimer that `datafusion` 50.0.0 is not
released (#17695)
add 3ad9da929 chore(deps): bump taiki-e/install-action from 2.61.10 to
2.62.1 (#17710)
add 79fc94a77 perf: Improve the performance of WINDOW functions with many
partitions (#17528)
add 52690c64b fix: Partial AggregateMode will generate duplicate field
names which will fail DFSchema construct (#17706)
add 95357ed44 feat: expose `udafs` and `udwfs` methods on
`FunctionRegistry` (#17650)
add a8d5016b9 Support remaining substrait time literal variations (#17707)
add eeec01798 Bump MSRV to 1.87.0 (#17724)
add 4ea760171 Avoid redundant Schema clones (#17643)
add 677a00040 Use github link instead of relative link to
optimizer_rule.rs in query-optimizer.md (#17723)
add 46a47a9dd Move misplaced upgrading entry about MSRV (#17727)
add 22a1eab83 Introduce `avg_distinct()` and `sum_distinct()` functions to
DataFrame API (#17536)
add 23d91c59b Support `WHERE`, `ORDER BY`, `LIMIT`, `SELECT`, `EXTEND`
pipe operators (#17278)
add 84b327c56 doc: add missing examples for multiple math functions
(#17018)
add 11ffa679e feat: support for null, date, and timestamp types in
approx_distinct (#17618)
add 9bc82029a fix(agg/corr): return NULL when variance is zero or samples
< 2 (#17621)
add 348d4339c chore(deps): bump taiki-e/install-action from 2.62.1 to
2.62.4 (#17739)
add 9fae60caf chore(deps): bump tempfile from 3.22.0 to 3.23.0 (#17741)
add 432995c67 chore: make `LimitPushPastWindows` public (#17736)
add 2ba2f1c46 fix: Remove parquet encryption feature from root deps
(#17700)
add d5bfcf515 fix: Remove datafusion-macros's dependency on
datafusion-expr (#17688)
add 33fe3ec91 chore: remove homebrew publish instructions from release
steps (#17735)
add 8471f5275 minor: create `OptimizerContext` with provided
`ConfigOptions` (#17742)
add 693eeaf74 Improve documentation for ordered set aggregate functions
(#17744)
add 0e3d4a719 docs: fix sidebar overlapping table on configuration page on
website (#17738)
add ded6c0dfd Add support for calling async UDF as aggregation expression
(#17620)
add 527652244 chore(deps): bump taiki-e/install-action from 2.62.4 to
2.62.5 (#17750)
add 62e6d5e25 (fix): Lag function creates unwanted projection (#17630)
(#17639)
add 6ec14e9e4 Support `LargeList` in `array_has` simplification to
`InList` (#17732)
add ea9fb26a2 chore(deps): bump wasm-bindgen-test from 0.3.51 to 0.3.53
(#17642)
add 564864bc9 feat: support `Utf8View` for more args of `regexp_replace`
(#17195)
add ac48ba3fa feat(spark): implement Spark `map` function
`map_from_arrays` (#17456)
add 4114e78d0 chore(deps): bump object_store from 0.12.3 to 0.12.4 (#17753)
add 5bbdb7eb1 Update `arrow` / `parquet` to 56.2.0 (#17631)
add 08e75a9a1 chore(deps): bump taiki-e/install-action from 2.62.5 to
2.62.6 (#17766)
add bdcfd4312 Keep aggregate udaf schema names unique when missing an
order-by (#17731)
add 78786d696 feat : Display function alias in output column name (#17690)
add 7f70ac6a5 Support join cardinality estimation less conservatively
(#17476)
add b6f2b08c5 chore(deps): bump libc from 0.2.175 to 0.2.176 (#17767)
add 05426bca6 chore(deps): bump postgres-types from 0.2.9 to 0.2.10
(#17768)
add bb3e0d8cb Use `Expr::qualified_name()` and `Column::new()` to extract
partition keys from window and aggregate operators (#17757)
add c1d6f3431 Prevent exponential planning time for Window functions - v2
(#17684)
add 350c5da87 docs: add Ballista link to landing page (#17746) (#17775)
add be1b8b675 chore(deps): bump taiki-e/install-action from 2.62.6 to
2.62.8 (#17781)
add 6c81ca005 chore(deps): bump wasm-bindgen-test from 0.3.53 to 0.3.54
(#17784)
add 0fefbb168 chore: Action some old TODOs in github actions (#17694)
add 691dd4727 dev: Add benchmark for compilation profiles (#17754)
add 9a5024d3e chore(deps): bump tokio-postgres from 0.7.13 to 0.7.14
(#17785)
add a548cce5f chore(deps): bump serde from 1.0.226 to 1.0.227 (#17783)
add 325e351d6 chore(deps): bump regex from 1.11.2 to 1.11.3 (#17782)
add e566e9724 Support `CAST` from temporal to `Utf8View` (#17535)
add 783c45b89 Add case expr simplifiers for literal comparisons (#17743)
add 6e1599440 chore: dependabot to run weekly (#17797)
add 1a6c79b92 [DOCS] Add dbt Fusion engine and R2 Query Engine to "Known
Users" (#17793)
add 22309f92f feat: change `datafusion-proto` to use `TaskContext` rather
than`SessionContext` for physical plan serialization (#17601)
add 1607561aa display window function's alias name in output (#17788)
add 5097f2c31 docs: update wasmtest README with instructions for Apple
silicon (#17755)
add 82790c5f3 chore(deps): bump sysinfo from 0.37.0 to 0.37.1 (#17800)
add deba0069a chore(deps): bump taiki-e/install-action from 2.62.8 to
2.62.9 (#17799)
add 8c8e5651d feat(spark): implement Spark `make_dt_interval` function
(#17728)
add bbb5cc79d Fix potential overflow when we print verbose physical plan
(#17798)
add 2f54f3033 Add SedonaDB as known user to Apache DataFusion (#17806)
add 320dd91c5 Extend datatype semantic equality check to include
timestamps (#17777)
add d05dcc36c fix: Filter out nulls properly in
approx_percentile_cont_with_weight (#17780)
add a8925f3ae chore: refactor usage of `reassign_predicate_columns`
(#17703)
add 2987e80ee dev: Add Apache license check to the lint script (#17787)
add 948f6b87a Fix: common_sub_expression_eliminate optimizer rule failed
(#16066)
add a1d15cc13 feat: support multi-threaded writing of Parquet files with
modular encryption (#16738)
add 2d947b31d feat(spark): implement Spark `map` function
`map_from_entries` (#17779)
add a45192084 chore: remove dialect settings in SLT tests that are
outdated (#17807)
add cc157b8b9 feat: Add Hash Join benchmarks (#17636)
add 43721dda5 chore(deps): bump thiserror from 2.0.16 to 2.0.17 (#17821)
add 89beee225 chore(deps): bump quote from 1.0.40 to 1.0.41 (#17822)
add 247450df9 chore(deps): bump taiki-e/install-action from 2.62.9 to
2.62.12 (#17823)
add 63ca0e1f3 chore(deps): bump serde from 1.0.227 to 1.0.228 (#17827)
add 5cc0be51f Disable failing benchmark query (#17809)
add 7d6d553af chore(deps): bump taiki-e/install-action from 2.62.12 to
2.62.13 (#17836)
add 032117adf More decimal 32/64 support - type coercsion and misc gaps
(#17808)
add 935fb3e2f minor: simplify docs build process & pin pip package
versions (#17816)
add 0104822e9 Implement `AsRef` for `Expr` (#17819)
add 0b160c567 Cleanup user guide known users section (#17834)
add 6c0d1c5c5 chore(deps): bump taiki-e/install-action from 2.62.13 to
2.62.14 (#17840)
add 7f25819c6 Fix the doc about row_groups pruning metrics in
explain_usage.md (#17846)
add 843e6f477 Replace `auto_doc_cfg` with `doc_cfg` (#17845)
add 52565a7ac chore(deps): bump petgraph from 0.8.2 to 0.8.3 (#17842)
add 4eacb6046 Enable Projection Pushdown Optimization for Recursive CTEs
(#16696)
add 615bcee08 docs: Add rerun.io to known users guide (#17825)
add 35f45b544 chore: fix typos & pin action hashes (#17855)
add f1246a9ea perf: Optimize CASE for any WHEN false (#17835)
add 94b68cb3c fix: correct edge case where null haystack returns false
instead of null (#17818)
add 97e00efff #17411 Relax constraint that file sort order must only
reference individual columns (#17419)
add e0222f072 Clarify email reply instructions for invitations (#17851)
add 6a6130491 fix: ignore `DataType::Null` in possible types during csv
type inference (#17796)
add bd95a6b8c feat: Support swap for `RightMark` Join (#17651)
add 297e537bd Add missing parenthesis in features documentation (#17869)
add a9d04606b minor: Include consumer name in OOM message (#17870)
add 9611ac80d Implement `partition_statistics` API for `InterleaveExec`
(#17051)
add 3ee52f85f Add `CastColumnExpr` for struct-aware column casting (#17773)
add 8e49d13bc chore(deps): bump taiki-e/install-action from 2.62.14 to
2.62.16 (#17879)
add b170aaede chore(deps): bump crate-ci/typos from 1.37.0 to 1.37.1
(#17878)
add d3ca16e9a Add notes for DataSinkExec (#17873)
add b81073ad4 feat: makes Expr::not normalize more negations (#17848)
add ae6f8eb1a Fix failing CI caused by hash collisions (#17886)
add 1ac818611 Minor: reuse test schemas in simplify tests (#17864)
add 7dff1e1f3 minor: Make `FunctionRegistry` `udafs` and `udwfs` methods
mandatory (#17847)
add 71512e676 docs: Improve documentation for FunctionFactory / CREATE
FUNCTION (#17859)
add a7b113c45 Support `AS`, `UNION`, `INTERSECTION`, `EXCEPT`, `AGGREGATE`
pipe operators (#17312)
add 10a437b82 [forward port] Change version to 50.1.0 and add changelog
(#17748) (#17826)
add 00f05c221 Make limit pushdown work for SortPreservingMergeExec (#17893)
add 21b6df19b chore(deps): bump taiki-e/install-action from 2.62.16 to
2.62.17 (#17896)
add 0f3cf27fb perf: Faster `string_agg()` aggregate function (1000x speed
for no DISTINCT and ORDER case) (#17837)
add daeb6597a Consolidate `apply_schema_adapter_tests` (#17905)
add 76904e837 optimizer: allow projection pushdown through aliased
recursive CTE references (#17875)
add 5e558b2f0 fix with_predicate() reset Metrics (#17858)
add d273ffbc9 Improve `InListExpr` plan display (#17884)
add 182d5dc5e Export JoinSetTracerError from datafusion-common-runtime
(#17877)
add 71d741504 Clippy to `extended_tests` (#17922)
add 9e8ec548f chore: rename Schema `print_schema_tree` to `tree_string`
(#17919)
add bcbc632e2 chore: utilize trait upcasting for AsyncScalarUDF PartialEq
& Hash (#17872)
add 82cd7f3cd fix: optimizer `common_sub_expression_eliminate` fails in a
window function (#17852)
add 04af0c61c perf: Implement boolean group values (#17726)
add e03ba614f Refactor: Update enforce_sorting tests to use insta
snapshots for easier updates (#17900)
add d76a1e0be feat: support spark udf format_string (#17561)
add 8e73844fb #17838 Rewrite `regexp_like` calls as `~` and `*~` operator
expressions when possible (#17839)
add c1009e61b chore(deps): bump maturin from 1.9.4 to 1.9.5 in /docs
(#17940)
add 09f551cb3 chore(deps): bump flate2 from 1.1.2 to 1.1.4 (#17938)
add 014fca707 chore(deps): bump actions/stale from 10.0.0 to 10.1.0
(#17937)
add bfd5e5582 chore(deps): bump aws-credential-types from 1.2.6 to 1.2.7
(#17936)
add 82aba7f62 chore(deps): bump rustyline from 17.0.1 to 17.0.2 (#17932)
add 24bc06807 chore(deps): bump taiki-e/install-action from 2.62.17 to
2.62.21 (#17934)
add 8769d9914 chore(deps): bump crate-ci/typos from 1.37.1 to 1.37.2
(#17935)
add 28fba58a6 chore: upgrade sqlparser (#17925)
add 307f5c369 minor: impl Clone and Debug on CaseBuilder (#17927)
add 149d3e953 chore: Extend backtrace coverage for `Execution` and
`Internal` errors (#17921)
add 9f70df15d chore(deps): bump taiki-e/install-action from 2.62.21 to
2.62.22 (#17949)
add 43dafd6bb chore(deps): bump crate-ci/typos from 1.37.2 to 1.38.0
(#17948)
add f8e988f83 Feat: [datafusion-spark] Migrate avg from comet to
datafusion-spark and add tests. (#17871)
add 5247a1604 Update tests to use insta / make them easier to update
(#17945)
add ab22ad8bd Test refactor: avoid creating the same SchemaRef (#17951)
add 7ce9b6ea0 docs: `Window::try_new_with_schema` with a descriptive error
message (#17926)
add ed5f8e7f0 Precision::<usize>::{add, sub, multiply}: avoid overflows
(#17929)
add 58ddf0d43 feat(spark): implement Spark `try_parse_url` function
(#17485)
add 1cc4dafbf Resolve `ListingScan` projection against table schema
including partition columns (#17911)
add d682d22d3 feat: Support reading CSV files with inconsistent column
counts (#17553)
add 0f515dc70 fix: fix failing test compilation on main (#17955)
add 4904a425c perf: add to `aggregate_vectorized`
`PrimitiveGroupValueBuilder` benchmark as well (#17930)
add c39d1c5e6 chore(deps): bump crate-ci/typos from 1.38.0 to 1.38.1
(#17960)
add 60b37b403 chore(deps): bump taiki-e/install-action from 2.62.22 to
2.62.23 (#17959)
add 07a7eb28b bench: fix `vectorized_equal_to` bench mutated between
iterations (#17968)
add 5472e5da3 clean up duplicate information in FileOpener trait (#17956)
add 3b37ae06d fix docs and broken example from #17956 (#17980)
add f8ff82a18 feat: Adds Instrumented Object Store Registry to
datafusion-cli (#17953)
add dd6859242 Refactor: Update `replace_with_order_preserving_variants`
tests to use insta snapshots for easier updates (#17962)
add 7a063d8c3 Support repartitioned() method in RepartitionExec (#17990)
add 40b72330b Adds Instrumented Object Store to CLI (#17984)
add 560944745 Migrate join_selection tests to snapshot-based testing
(#17974)
add 590ad293d fix: update `PrimitiveGroupValueBuilder` to match NaN
correctly (#17979)
add a497074eb bench: fix actually generate a lot of unique values in
benchmark table (#17967)
add 73b1f2bd7 Adds Instrument Mode for InstrumentedObjectStore in
datafusion-cli (#18000)
add 6479e43f8 Support `JOIN` pipe operator (#17969)
add dd4e2289b minor: refactor Spark ascii function to reuse DataFusion
ascii function code (#17965)
add 668eabe64 chore(deps): bump taiki-e/install-action from 2.62.23 to
2.62.24 (#17989)
add 0ffc4333f chore(deps): bump taiki-e/install-action from 2.62.24 to
2.62.25 (#18007)
add 37ad8a541 Clarify documentation that ScalarUDFImpl::simplity must not
change the schema (#17981)
add 832ba7612 fix: Add overflow checks to SparkDateAdd/Sub to avoid panics
(#18013)
add 8772411de Expose trace_future and trace_block outside of
common-runtime (#17976)
add f210939ec Adds Object Store Profiling options/commands to CLI (#18004)
add a1acf775a Adds instrumentation to get requests for datafusion-cli
(#18016)
add fd3ccf8e1 refactor : deprecate `ParquetSource::predicate()` and merge
into `FileSource::filter()` (#17971)
add f739fd843 docs: typo in `working-with-exprs.md` (#18033)
add 556eb9b1e feat: convert_array_to_scalar_vec respects null elements
(#17891)
add 450fbfa26 feat: add cargo-machete in CI (#18030)
add d659c4859 chore(deps): bump half from 2.6.0 to 2.7.0 (#18036)
add fb7ef1a65 chore(deps): bump maturin from 1.9.5 to 1.9.6 in /docs
(#18039)
add f6c6fe0f2 chore(deps): bump aws-config from 1.8.6 to 1.8.7 (#18038)
add b27fedcd3 chore(deps): bump taiki-e/install-action from 2.62.25 to
2.62.28 (#18037)
add c2921e0b8 refactor: cleanup naming and macro usages for binary
operator (#17985)
add 28290baa3 Impl `gather_filters_for_pushdown` for
`CoalescePartitionsExec` (#18046)
add 4e692417c Fix bug in LimitPushPastWindows (#18029)
add 978ce8ea1 Fix `SortPreservingMergeExec` tree formatting with limit
(#18009)
add 68d59c15a Prepare for 50.2.0 release (#17963) (#18057)
add 2b2113a42 Update committers on governance page (#18015)
add 77ec31981 Feat: Make current_date aware of execution timezone. (#18034)
add 72f01a059 chore(deps): bump actions/setup-node from 5.0.0 to 6.0.0
(#18049)
add 9e658facf chore(deps): bump sysinfo from 0.37.1 to 0.37.2 (#18035)
add 8b081e9d0 feat(spark): implement Spark `elt` function (#17729)
add 703c33f43 FileScanConfig: Preserve schema metadata across ser/de
boundary (#17966)
add 0ecd59bd0 physical-plan: push filters down to UnionExec children
(#18054)
add a4c95aa96 Reproduce quadratic runtime in min_max_bytes (#18041)
add ca2585e75 Adds summary output to CLI instrumented object stores
(#18045)
add ca3816b4d Impl spark bit not function (#18018)
add a65a2cbd6 chore: revert tests (#18065)
add 6d3854f69 #17972 Restore case expr/expr optimisation while ensuring
lazy evaluation (#17973)
add 057583d5c chore: Use an enum to express the different kinds of
nullability in an array (#18048)
add 6909e66d9 chore(deps): bump taiki-e/install-action from 2.62.28 to
2.62.29 (#18069)
add 424b73d9a Refactor: split
test_window_partial_constant_and_set_monotonicity into multiple tests (#17952)
add a61a9c2c1 fix: Ensure ListingTable partitions are pruned when filters
are not used (#17958)
add 41fdab9d2 Push Down Filter Subexpressions in Nested Loop Joins as
Projections (#17906)
add 264030cca feat: support Spark `concat` string function (#18063)
add 4153adf2c Add independent configs for topk/join dynamic filter (#18090)
add 3bca1bb6e Adds Trace and Summary to CLI instrumented stores (#18064)
add ec3ca719d fix: Improve null handling in array_to_string function
(#18076)
add c8e0f1cf7 feat: update .asf.yaml configuration settings (#18027)
add 0a57e017c Fix extended tests on main to get CI green (#18096)
add 9bfa2ae77 chore(deps): bump taiki-e/install-action from 2.62.29 to
2.62.31 (#18094)
add b1723e5c6 chore: run extended suite on PRs for critical areas (#18088)
add 4e03c9205 refactor: add dialect enum (#18043)
add ea83c2644 #17982 Make `nvl` a thin wrapper for `coalesce` (#17991)
add 7c3b0d0a6 minor: fix incorrect deprecation version & window docs
(#18093)
add 337378ab8 chore: use `NullBuffer::union` for Spark `concat` (#18087)
add cadf42955 feat: support `null_treatment`, `distinct`, and `filter` for
window functions in proto (#18024)
add c84e3cf5a feat: Add percentile_cont aggregate function (#17988)
add 621a24978 fix: Re-bump latest datafusion-testing module so extended
tests succeed (#18110)
add ffe64e310 chore(deps): bump taiki-e/install-action from 2.62.31 to
2.62.33 (#18113)
add 0ae9fdcb0 Adding hiop as known user (#18114)
add a9ecd6830 chore: remove unnecessary `skip_failed_rules` config in slt
(#18117)
add fe955058a move repartition to insta (#18106)
add 3272ebe99 refactor: move ListingTable over to the
catalog-listing-table crate (#18080)
add 8e1d13a9c refactor: move arrow datasource to new
`datafusion-datasource-arrow` crate (#18082)
add dce59f850 Adds instrumentation to LIST operations in CLI (#18103)
add 76050235b feat: spark udf array shuffle (#17674)
add f0ab1369a make Union::try_new pub (#18125)
add c9561049d fix: window unparsing (#17367)
add ec2402aee feat: Support configurable `EXPLAIN ANALYZE` detail level
(#18098)
add 2222abda2 refactor: remove unused `type_coercion/aggregate.rs`
functions (#18091)
add 765f2b9ea Add extra case_when benchmarks (#18097)
add 7d294f1dc fix: Add dictionary coercion support for numeric comparison
operations (#18099)
add ec3d20bff Adds instrumentation to delimited LIST operations in CLI
(#18134)
add 522403bb4 feat: add fp16 support to Substrait (#18086)
add 1b001a14a fix(substrait): schema errors for Aggregates with no
groupings (#17909)
add 5a074ea4c Improve datafusion-cli object store profiling summary
display (#18085)
add e323357b1 test: `to_timestamp(double)` for vectorized input (#18147)
add 9079bbd4f Fix `concat_elements_utf8view` capacity initialization.
(#18003)
add 0ddc82e71 Use < instead of = in case benchmark predicates, use
Integers (#18144)
add 93f136c06 Adds instrumentation to PUT ops in the CLI (#18139)
add 28a68543d [main] chore: Fix `no space left on device` (#18141) (#18151)
add b98cad616 Fix `DISTINCT ON` for tables with no columns
(ReplaceDistinctWithAggregate: do not fail when on input without columns)
(#18133)
add f199b0008 refactor: remove core crate from datafusion-proto (#18123)
add f198fc888 Fix quadratic runtime in min_max_bytes (#18044)
add 35b2e359f fix: `array_distinct` inner nullability causing type
mismatch (#18104)
add 7c215ed8d Short circuit complex case evaluation modes as soon as
possible (#17898)
add 7f75e5802 perf: Fix NLJ slow join with condition `array_has` (#18161)
add 5d2372366 chore(deps): bump getrandom from 0.3.3 to 0.3.4 (#18163)
add fcbbfa4cc chore(deps): bump tokio from 1.47.1 to 1.48.0 (#18164)
add 5c19eeda7 chore(deps): bump indexmap from 2.11.4 to 2.12.0 (#18162)
add b1deb1f37 chore(deps): bump bzip2 from 0.6.0 to 0.6.1 (#18165)
add a4acec396 fix: improve document ui (#18157)
add 54fff6080 perf: improve `ScalarValue::to_array_of_size` for Boolean
and some null values (#18180)
add 37aad2842 Feat: Make current_time aware of execution timezone. (#18040)
add 1f434dcdd feat: `ClassicJoin` for PWMJ (#17482)
add 155b56e52 fix(docs): resolve extra outline on tables (#18193)
add b5b7f9b35 chore(deps): bump taiki-e/install-action from 2.62.33 to
2.62.34 (#18194)
add 77a4cb741 Fix COPY TO does not produce an output file for the empty
set (#18074)
add 347b2b6cc Add Projection struct w/ helper methods to manipulate
projections (#18176)
add 8d54e7b2a Add TableSchema helper to encapsulate file schema +
partition fields (#18178)
add 1e30aed6f Add spilling to RepartitionExec (#18014)
add d5ea5e987 Adds DELETE and HEAD instrumentation to CLI (#18206)
add 8054bb881 [branch-50] Prepare 50.3.0 release version number and README
(#18173) (#18182)
add 31109e422 feat(docs): display compatible logo for dark mode (#18197)
add 6d52e54bc Docs: Update SQL example for current_time() and
current_date(). (#18200)
add 531af8e43 feat: Add `deregister_object_store` (#17999)
add b7a10adbc fix: Use dynamic timezone in now() function for accurate
timestamp (#18017)
add 114beec77 Fix array_has simplification with null argument (#18186)
add 4aceda0e1 chore(deps): bump taiki-e/install-action from 2.62.34 to
2.62.35 (#18215)
add 6ecf76c83 bench: create benchmark for lookup table like `CASE WHEN`
(#18203)
add 774b6fee0 Adds instrumentation to COPY operations in the CLI (#18227)
add 47fd638c7 Consolidate core_integration/datasource and rename
parquet_source --> parquet_integration (#18226)
add 340834d1f feat: Add existence join to NestedLoopJoin benchmarks
(#18005)
add 408e1e4e2 doc: Add `Metrics` section to the user-guide (#18216)
add 92c560724 fix: UnnestExec preserves relevant equivalence properties of
input (#16985)
add be85bf41a CoalescePartitionsExec fetch is not consistent with one
partition and more than one partition (#18245)
add 144f1554a fix: wrong simplification for >= >, <= < (#18222)
add d12797311 Migrate core test to insta part 3 (#16978)
add 6f330d43c Merge branch 'main' into sandbox-main
add e4cb9a7b1 chore(deps): bump taiki-e/install-action from 2.61.8 to
2.62.36
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b0770a488)
\
N -- N -- N
refs/heads/dependabot/github_actions/taiki-e/install-action-2.62.36 (e4cb9a7b1)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 5 +
.../setup-macos-aarch64-builder/action.yaml | 2 +-
.github/dependabot.yml | 5 +
.github/workflows/audit.yml | 2 +-
.github/workflows/dependencies.yml | 11 +
.github/workflows/dev.yml | 2 +-
.github/workflows/extended.yml | 8 +
.github/workflows/labeler/labeler-config.yml | 2 +-
.github/workflows/rust.yml | 23 +-
.github/workflows/stale.yml | 2 +-
Cargo.lock | 334 +-
Cargo.toml | 85 +-
benchmarks/src/imdb/run.rs | 2 +-
benchmarks/src/nlj.rs | 39 +
benchmarks/src/tpch/run.rs | 2 +-
ci/scripts/rust_clippy.sh | 2 +-
datafusion-cli/Cargo.toml | 10 +-
datafusion-cli/examples/cli-session-context.rs | 5 +-
datafusion-cli/src/command.rs | 102 +-
datafusion-cli/src/helper.rs | 19 +-
datafusion-cli/src/highlighter.rs | 10 +-
datafusion-cli/src/lib.rs | 2 +-
datafusion-cli/src/main.rs | 17 +
datafusion-cli/src/object_storage.rs | 7 +-
datafusion-cli/src/object_storage/instrumented.rs | 1261 ++++++++
datafusion-cli/src/print_options.rs | 93 +-
datafusion-cli/tests/cli_integration.rs | 89 +-
.../snapshots/[email protected] | 2 +-
.../cli_top_memory_consumers@top3_default.snap | 2 +-
.../tests/snapshots/object_store_profiling.snap | 83 +
.../examples/advanced_parquet_index.rs | 13 +-
datafusion-examples/examples/advanced_udwf.rs | 19 +-
datafusion-examples/examples/custom_file_casts.rs | 2 +-
datafusion-examples/examples/function_factory.rs | 6 +-
datafusion-examples/examples/json_shredding.rs | 16 +-
.../examples/memory_pool_execution_plan.rs | 11 +-
datafusion-examples/examples/remote_catalog.rs | 4 +-
datafusion-testing | 2 +-
datafusion/catalog-listing/Cargo.toml | 7 +-
datafusion/catalog-listing/src/config.rs | 360 +++
datafusion/catalog-listing/src/helpers.rs | 23 +-
datafusion/catalog-listing/src/mod.rs | 9 +-
datafusion/catalog-listing/src/options.rs | 411 +++
datafusion/catalog-listing/src/table.rs | 788 +++++
datafusion/catalog/src/lib.rs | 2 +-
datafusion/catalog/src/listing_schema.rs | 18 +-
datafusion/catalog/src/stream.rs | 17 +-
datafusion/common-runtime/src/lib.rs | 6 +-
datafusion/common/src/config.rs | 140 +-
datafusion/common/src/dfschema.rs | 28 +-
datafusion/common/src/format.rs | 45 +
datafusion/common/src/join_type.rs | 2 +
datafusion/common/src/lib.rs | 8 +-
datafusion/common/src/scalar/mod.rs | 206 +-
datafusion/common/src/stats.rs | 49 +-
datafusion/common/src/utils/memory.rs | 9 +-
datafusion/common/src/utils/mod.rs | 18 +-
datafusion/core/Cargo.toml | 13 +-
datafusion/core/benches/aggregate_query_sql.rs | 32 +
datafusion/core/benches/data_utils/mod.rs | 11 +-
datafusion/core/benches/sql_planner.rs | 7 +-
datafusion/core/src/dataframe/mod.rs | 12 +-
datafusion/core/src/dataframe/parquet.rs | 3 +-
datafusion/core/src/datasource/dynamic_file.rs | 1 +
.../core/src/datasource/file_format/arrow.rs | 508 +--
datafusion/core/src/datasource/file_format/csv.rs | 112 +-
datafusion/core/src/datasource/listing/mod.rs | 3 +-
datafusion/core/src/datasource/listing/table.rs | 1698 +---------
.../physical_plan/arrow.rs} | 9 +-
.../core/src/datasource/physical_plan/csv.rs | 29 +-
.../core/src/datasource/physical_plan/mod.rs | 6 +-
.../core/src/datasource/physical_plan/parquet.rs | 10 +-
datafusion/core/src/execution/context/mod.rs | 129 +-
datafusion/core/src/execution/session_state.rs | 56 +-
datafusion/core/src/lib.rs | 2 +-
datafusion/core/src/physical_planner.rs | 423 ++-
datafusion/core/tests/catalog/memory.rs | 2 +-
datafusion/core/tests/core_integration.rs | 3 +
.../some_empty_with_header/a_empty.csv} | 0
.../data/empty_files/some_empty_with_header/b.csv | 3 +
.../some_empty_with_header/c_nulls_column.csv} | 1 +
datafusion/core/tests/dataframe/mod.rs | 8 +-
datafusion/core/tests/datasource/csv.rs | 122 +
.../memory_limit_validation => datasource}/mod.rs | 9 +-
datafusion/core/tests/execution/coop.rs | 4 +-
datafusion/core/tests/expr_api/simplification.rs | 3 +-
.../tests/fuzz_cases/aggregation_fuzzer/fuzzer.rs | 6 +-
.../core/tests/fuzz_cases/equivalence/utils.rs | 10 +-
datafusion/core/tests/fuzz_cases/join_fuzz.rs | 8 +-
datafusion/core/tests/fuzz_cases/once_exec.rs | 6 +-
.../memory_limit_validation/sort_mem_validation.rs | 2 +-
datafusion/core/tests/memory_limit/mod.rs | 31 +-
.../tests/memory_limit/repartition_mem_limit.rs | 116 +
datafusion/core/tests/optimizer/mod.rs | 3 +-
datafusion/core/tests/parquet/custom_reader.rs | 10 +-
datafusion/core/tests/parquet/encryption.rs | 8 +-
datafusion/core/tests/parquet/schema_adapter.rs | 181 +-
.../{parquet_config.rs => parquet_integration.rs} | 0
.../tests/physical_optimizer/enforce_sorting.rs | 3346 ++++++--------------
.../enforce_sorting_monotonicity.rs | 1715 ++++++++++
.../physical_optimizer/filter_pushdown/mod.rs | 167 +-
.../physical_optimizer/filter_pushdown/util.rs | 10 +-
.../tests/physical_optimizer/join_selection.rs | 94 +-
datafusion/core/tests/physical_optimizer/mod.rs | 1 +
.../physical_optimizer/partition_statistics.rs | 144 +-
.../replace_with_order_preserving_variants.rs | 1664 +++++-----
.../core/tests/physical_optimizer/test_utils.rs | 52 +-
datafusion/core/tests/sql/aggregates/basic.rs | 2 +-
datafusion/core/tests/sql/explain_analyze.rs | 160 +
datafusion/core/tests/sql/select.rs | 10 +-
.../core/tests/user_defined/insert_operation.rs | 5 +-
.../tests/user_defined/user_defined_aggregates.rs | 3 +
.../user_defined/user_defined_scalar_functions.rs | 10 +-
.../user_defined/user_defined_window_functions.rs | 17 +-
.../Cargo.toml | 17 +-
.../datasource-arrow/LICENSE.txt | 0
.../datasource-arrow/NOTICE.txt | 0
datafusion/{common => datasource-arrow}/README.md | 6 +-
.../src/file_format.rs} | 120 +-
.../src/mod.rs | 0
.../src/source.rs} | 28 +-
.../tests/data/example.arrow | Bin
datafusion/datasource-avro/src/mod.rs | 2 +-
datafusion/datasource-avro/src/source.rs | 14 +-
datafusion/datasource-csv/src/file_format.rs | 138 +-
datafusion/datasource-csv/src/source.rs | 19 +-
datafusion/datasource-json/src/source.rs | 16 +-
datafusion/datasource-parquet/src/opener.rs | 183 +-
datafusion/datasource-parquet/src/reader.rs | 39 +-
datafusion/datasource-parquet/src/source.rs | 36 +-
.../tests/apply_schema_adapter_tests.rs | 206 --
datafusion/datasource/Cargo.toml | 4 +-
datafusion/datasource/src/file_groups.rs | 4 +-
datafusion/datasource/src/file_meta.rs | 53 -
datafusion/datasource/src/file_scan_config.rs | 147 +-
datafusion/datasource/src/file_stream.rs | 21 +-
datafusion/datasource/src/memory.rs | 24 +-
datafusion/datasource/src/mod.rs | 18 +-
datafusion/datasource/src/sink.rs | 5 +
datafusion/datasource/src/table_schema.rs | 146 +
datafusion/datasource/src/write/demux.rs | 47 +-
datafusion/datasource/src/write/orchestration.rs | 12 +-
datafusion/doc/src/lib.rs | 10 +-
datafusion/execution/src/lib.rs | 2 +-
datafusion/execution/src/memory_pool/pool.rs | 22 +-
datafusion/execution/src/object_store.rs | 26 +-
datafusion/execution/src/parquet_encryption.rs | 6 +-
datafusion/execution/src/runtime_env.rs | 8 +-
datafusion/execution/src/task.rs | 6 +-
datafusion/expr-common/src/lib.rs | 2 +-
datafusion/expr-common/src/signature.rs | 1 -
.../expr-common/src/type_coercion/aggregates.rs | 300 +-
datafusion/expr-common/src/type_coercion/binary.rs | 180 +-
.../src/type_coercion/binary/tests/arithmetic.rs | 134 +-
.../src/type_coercion/binary/tests/comparison.rs | 88 +
datafusion/expr/src/async_udf.rs | 134 +-
datafusion/expr/src/conditional_expressions.rs | 1 +
datafusion/expr/src/expr.rs | 60 +
datafusion/expr/src/expr_fn.rs | 9 +-
datafusion/expr/src/lib.rs | 4 +-
datafusion/expr/src/logical_plan/builder.rs | 17 +-
datafusion/expr/src/logical_plan/ddl.rs | 22 +-
datafusion/expr/src/logical_plan/display.rs | 15 +-
datafusion/expr/src/logical_plan/plan.rs | 17 +-
datafusion/expr/src/operation.rs | 15 +-
datafusion/expr/src/registry.rs | 14 +-
datafusion/expr/src/simplify.rs | 10 +-
datafusion/expr/src/test/function_stub.rs | 67 +-
datafusion/expr/src/udaf.rs | 7 +
datafusion/expr/src/udf.rs | 94 +-
datafusion/expr/src/udwf.rs | 47 +-
datafusion/expr/src/window_state.rs | 8 +-
datafusion/ffi/Cargo.toml | 1 +
datafusion/ffi/src/lib.rs | 2 +-
datafusion/ffi/src/record_batch_stream.rs | 16 +-
datafusion/ffi/src/tests/async_provider.rs | 15 +-
datafusion/ffi/src/udaf/accumulator_args.rs | 10 +-
datafusion/ffi/src/udaf/mod.rs | 5 +-
datafusion/ffi/src/udwf/mod.rs | 13 +-
.../ffi/src/udwf/partition_evaluator_args.rs | 3 +-
datafusion/ffi/src/util.rs | 16 +-
datafusion/functions-aggregate-common/src/lib.rs | 2 +-
.../functions-aggregate-common/src/merge_arrays.rs | 26 +-
datafusion/functions-aggregate-common/src/utils.rs | 16 +-
datafusion/functions-aggregate/Cargo.toml | 4 +
.../functions-aggregate/benches/min_max_bytes.rs | 92 +
.../functions-aggregate/src/approx_distinct.rs | 11 +-
.../src/approx_percentile_cont.rs | 65 +-
datafusion/functions-aggregate/src/array_agg.rs | 11 +-
datafusion/functions-aggregate/src/average.rs | 67 +-
datafusion/functions-aggregate/src/lib.rs | 12 +-
.../src/min_max/min_max_bytes.rs | 50 +-
datafusion/functions-aggregate/src/nth_value.rs | 6 +-
.../functions-aggregate/src/percentile_cont.rs | 814 +++++
datafusion/functions-aggregate/src/string_agg.rs | 165 +-
datafusion/functions-aggregate/src/utils.rs | 72 +
datafusion/functions-nested/src/array_has.rs | 135 +-
datafusion/functions-nested/src/extract.rs | 8 +-
datafusion/functions-nested/src/lib.rs | 2 +-
datafusion/functions-nested/src/set_ops.rs | 63 +-
datafusion/functions-nested/src/string.rs | 58 +-
datafusion/functions-table/src/generate_series.rs | 4 +-
datafusion/functions-table/src/lib.rs | 2 +-
datafusion/functions-window-common/src/expr.rs | 2 +-
datafusion/functions-window-common/src/lib.rs | 2 +-
.../functions-window-common/src/partition.rs | 2 +-
datafusion/functions-window/src/cume_dist.rs | 7 +-
datafusion/functions-window/src/lead_lag.rs | 37 +-
datafusion/functions-window/src/lib.rs | 2 +-
datafusion/functions-window/src/nth_value.rs | 40 +-
datafusion/functions-window/src/ntile.rs | 22 +-
datafusion/functions-window/src/rank.rs | 11 +-
datafusion/functions-window/src/row_number.rs | 10 +-
datafusion/functions/src/core/coalesce.rs | 11 +-
datafusion/functions/src/core/nvl.rs | 240 +-
datafusion/functions/src/datetime/common.rs | 11 +-
datafusion/functions/src/datetime/current_date.rs | 38 +-
datafusion/functions/src/datetime/current_time.rs | 123 +-
datafusion/functions/src/datetime/date_trunc.rs | 18 +-
datafusion/functions/src/datetime/mod.rs | 8 +-
datafusion/functions/src/datetime/now.rs | 28 +-
datafusion/functions/src/datetime/to_local_time.rs | 13 +-
datafusion/functions/src/datetime/to_timestamp.rs | 4 +-
datafusion/functions/src/encoding/inner.rs | 27 +-
datafusion/functions/src/lib.rs | 2 +-
datafusion/functions/src/macros.rs | 26 +
datafusion/functions/src/math/abs.rs | 12 +-
datafusion/functions/src/math/round.rs | 2 +-
datafusion/functions/src/planner.rs | 2 +-
datafusion/functions/src/regex/regexplike.rs | 72 +-
datafusion/functions/src/string/ascii.rs | 6 +-
datafusion/macros/src/user_doc.rs | 2 +-
datafusion/optimizer/src/analyzer/type_coercion.rs | 17 +-
.../optimizer/src/common_subexpr_eliminate.rs | 6 +-
.../src/decorrelate_predicate_subquery.rs | 49 +-
datafusion/optimizer/src/lib.rs | 2 +-
.../optimizer/src/optimize_projections/mod.rs | 84 +-
.../optimizer/src/replace_distinct_aggregate.rs | 43 +-
.../optimizer/src/scalar_subquery_to_join.rs | 254 +-
.../src/simplify_expressions/expr_simplifier.rs | 275 +-
.../src/simplify_expressions/simplify_exprs.rs | 148 +
.../simplify_expressions/simplify_predicates.rs | 6 +-
.../optimizer/tests/optimizer_integration.rs | 161 +
datafusion/physical-expr-adapter/src/lib.rs | 2 +-
datafusion/physical-expr-common/src/lib.rs | 2 +-
.../physical-expr-common/src/physical_expr.rs | 268 +-
datafusion/physical-expr/Cargo.toml | 2 +-
datafusion/physical-expr/benches/case_when.rs | 532 +++-
datafusion/physical-expr/src/equivalence/class.rs | 7 +-
datafusion/physical-expr/src/equivalence/mod.rs | 32 +-
.../physical-expr/src/equivalence/projection.rs | 998 ------
.../src/equivalence/properties/dependency.rs | 4 +-
datafusion/physical-expr/src/expressions/binary.rs | 224 +-
.../src/expressions/binary/kernels.rs | 208 +-
datafusion/physical-expr/src/expressions/case.rs | 88 +-
datafusion/physical-expr/src/expressions/cast.rs | 9 +-
.../physical-expr/src/expressions/cast_column.rs | 409 +++
.../physical-expr/src/expressions/in_list.rs | 63 +-
datafusion/physical-expr/src/expressions/mod.rs | 2 +
datafusion/physical-expr/src/lib.rs | 9 +-
datafusion/physical-expr/src/physical_expr.rs | 29 +-
datafusion/physical-expr/src/projection.rs | 2154 +++++++++++++
datafusion/physical-expr/src/utils/mod.rs | 10 +-
datafusion/physical-expr/src/window/aggregate.rs | 9 +-
.../src/window/standard_window_function_expr.rs | 4 +-
datafusion/physical-expr/src/window/window_expr.rs | 10 +-
datafusion/physical-optimizer/Cargo.toml | 1 +
.../physical-optimizer/src/filter_pushdown.rs | 46 +-
.../physical-optimizer/src/join_selection.rs | 6 +-
datafusion/physical-optimizer/src/lib.rs | 2 +-
.../src/limit_pushdown_past_window.rs | 195 +-
.../physical-optimizer/src/projection_pushdown.rs | 748 ++++-
.../physical-plan/benches/aggregate_vectorized.rs | 359 ++-
.../src/aggregates/group_values/mod.rs | 17 +-
.../multi_group_by/{primitive.rs => boolean.rs} | 232 +-
.../group_values/multi_group_by/bytes.rs | 24 +-
.../group_values/multi_group_by/bytes_view.rs | 16 +-
.../aggregates/group_values/multi_group_by/mod.rs | 33 +-
.../group_values/multi_group_by/primitive.rs | 72 +-
.../group_values/single_group_by/boolean.rs | 154 +
.../group_values/single_group_by/bytes.rs | 6 +-
.../aggregates/group_values/single_group_by/mod.rs | 1 +
.../src/aggregates/topk/hash_table.rs | 6 +-
.../physical-plan/src/aggregates/topk/heap.rs | 6 +-
.../physical-plan/src/aggregates/topk_stream.rs | 4 +-
datafusion/physical-plan/src/analyze.rs | 19 +-
.../physical-plan/src/coalesce_partitions.rs | 132 +-
datafusion/physical-plan/src/display.rs | 39 +-
datafusion/physical-plan/src/joins/cross_join.rs | 2 +-
.../physical-plan/src/joins/hash_join/exec.rs | 8 +-
.../physical-plan/src/joins/hash_join/stream.rs | 1 +
datafusion/physical-plan/src/joins/mod.rs | 2 +
.../physical-plan/src/joins/nested_loop_join.rs | 39 +-
.../src/joins/piecewise_merge_join/classic_join.rs | 1550 +++++++++
.../src/joins/piecewise_merge_join/exec.rs | 748 +++++
.../src/joins/piecewise_merge_join}/mod.rs | 9 +-
.../src/joins/piecewise_merge_join/utils.rs | 61 +
.../src/joins/sort_merge_join/stream.rs | 171 +-
.../src/joins/sort_merge_join/tests.rs | 40 +-
.../physical-plan/src/joins/symmetric_hash_join.rs | 9 +
datafusion/physical-plan/src/joins/utils.rs | 135 +-
datafusion/physical-plan/src/lib.rs | 2 +-
datafusion/physical-plan/src/metrics/baseline.rs | 12 +-
datafusion/physical-plan/src/metrics/builder.rs | 21 +-
datafusion/physical-plan/src/metrics/mod.rs | 54 +-
datafusion/physical-plan/src/projection.rs | 338 +-
datafusion/physical-plan/src/recursive_query.rs | 6 +-
datafusion/physical-plan/src/repartition/mod.rs | 605 +++-
datafusion/physical-plan/src/sorts/sort.rs | 2 +-
.../src/sorts/sort_preserving_merge.rs | 7 +-
datafusion/physical-plan/src/union.rs | 44 +-
datafusion/physical-plan/src/unnest.rs | 98 +-
.../src/windows/bounded_window_agg_exec.rs | 36 +-
datafusion/physical-plan/src/windows/mod.rs | 6 +-
datafusion/proto-common/src/lib.rs | 2 +-
datafusion/proto/Cargo.toml | 22 +-
datafusion/proto/proto/datafusion.proto | 11 +-
datafusion/proto/src/bytes/mod.rs | 22 +-
datafusion/proto/src/bytes/registry.rs | 2 +-
datafusion/proto/src/common.rs | 6 +-
datafusion/proto/src/generated/pbjson.rs | 145 +
datafusion/proto/src/generated/prost.rs | 36 +-
datafusion/proto/src/lib.rs | 8 +-
datafusion/proto/src/logical_plan/file_formats.rs | 114 +-
datafusion/proto/src/logical_plan/from_proto.rs | 89 +-
datafusion/proto/src/logical_plan/mod.rs | 161 +-
datafusion/proto/src/logical_plan/to_proto.rs | 33 +-
datafusion/proto/src/physical_plan/from_proto.rs | 62 +-
datafusion/proto/src/physical_plan/mod.rs | 152 +-
datafusion/proto/src/physical_plan/to_proto.rs | 54 +-
datafusion/proto/tests/cases/mod.rs | 10 +-
.../proto/tests/cases/roundtrip_logical_plan.rs | 251 +-
.../proto/tests/cases/roundtrip_physical_plan.rs | 70 +-
datafusion/proto/tests/cases/serialize.rs | 2 +-
datafusion/pruning/src/file_pruner.rs | 10 +-
datafusion/pruning/src/pruning_predicate.rs | 8 +-
datafusion/spark/Cargo.toml | 3 +-
datafusion/spark/src/function/aggregate/avg.rs | 362 +++
datafusion/spark/src/function/aggregate/mod.rs | 14 +-
datafusion/spark/src/function/array/mod.rs | 9 +-
.../src/function/array/shuffle.rs} | 116 +-
.../spark/src/function/bitwise/bitwise_not.rs | 109 +
datafusion/spark/src/function/bitwise/mod.rs | 8 +
datafusion/spark/src/function/datetime/date_add.rs | 25 +-
datafusion/spark/src/function/datetime/date_sub.rs | 25 +-
.../src/function/datetime/make_dt_interval.rs | 20 +-
.../spark/src/function/datetime/make_interval.rs | 50 +-
datafusion/spark/src/function/math/factorial.rs | 6 +-
datafusion/spark/src/function/math/hex.rs | 6 +-
datafusion/spark/src/function/math/modulus.rs | 18 +-
datafusion/spark/src/function/math/width_bucket.rs | 12 +-
datafusion/spark/src/function/string/ascii.rs | 117 +-
datafusion/spark/src/function/string/concat.rs | 269 ++
datafusion/spark/src/function/string/elt.rs | 251 ++
.../spark/src/function/string/format_string.rs | 2350 ++++++++++++++
datafusion/spark/src/function/string/mod.rs | 33 +-
datafusion/spark/src/function/url/mod.rs | 15 +-
datafusion/spark/src/function/url/parse_url.rs | 228 +-
.../src/function/url/try_parse_url.rs} | 61 +-
datafusion/spark/src/lib.rs | 13 +-
datafusion/sql/src/expr/function.rs | 5 +
datafusion/sql/src/expr/identifier.rs | 8 +-
datafusion/sql/src/expr/mod.rs | 8 +-
datafusion/sql/src/lib.rs | 2 +-
datafusion/sql/src/parser.rs | 8 +-
datafusion/sql/src/planner.rs | 18 +-
datafusion/sql/src/query.rs | 118 +-
datafusion/sql/src/relation/join.rs | 6 +-
datafusion/sql/src/statement.rs | 37 +-
datafusion/sql/src/unparser/dialect.rs | 23 +
datafusion/sql/src/unparser/expr.rs | 12 +-
datafusion/sql/src/unparser/plan.rs | 21 +-
datafusion/sql/src/unparser/rewrite.rs | 66 +
datafusion/sql/src/utils.rs | 8 +-
datafusion/sql/tests/cases/plan_to_sql.rs | 86 +
datafusion/sqllogictest/bin/sqllogictests.rs | 10 +-
datafusion/sqllogictest/data/composite_order.csv | 8 +
.../sqllogictest/regenerate/sqllogictests.rs | 2 +-
.../src/engines/datafusion_engine/normalize.rs | 12 +-
datafusion/sqllogictest/src/lib.rs | 2 +-
datafusion/sqllogictest/test_files/aggregate.slt | 342 +-
datafusion/sqllogictest/test_files/array.slt | 22 +-
datafusion/sqllogictest/test_files/arrow_files.slt | 8 +-
datafusion/sqllogictest/test_files/case.slt | 103 +
datafusion/sqllogictest/test_files/copy.slt | 15 +
datafusion/sqllogictest/test_files/cte.slt | 18 +-
.../test_files/current_date_timezone.slt | 82 +
.../test_files/current_time_timezone.slt | 100 +
datafusion/sqllogictest/test_files/ddl.slt | 12 +-
.../test_files/dynamic_filter_pushdown_config.slt | 339 ++
.../sqllogictest/test_files/explain_tree.slt | 48 +
.../sqllogictest/test_files/information_schema.slt | 10 +-
datafusion/sqllogictest/test_files/join.slt.part | 5 +-
datafusion/sqllogictest/test_files/join_lists.slt | 63 +
datafusion/sqllogictest/test_files/joins.slt | 79 +-
datafusion/sqllogictest/test_files/nullif.slt | 36 +
datafusion/sqllogictest/test_files/nvl.slt | 35 +
datafusion/sqllogictest/test_files/order.slt | 37 +
.../sqllogictest/test_files/pipe_operator.slt | 106 +
datafusion/sqllogictest/test_files/pwmj.slt | 354 +++
.../sqllogictest/test_files/regexp/regexp_like.slt | 60 +
.../sqllogictest/test_files/repartition_scan.slt | 4 +-
.../test_files/simplify_predicates.slt | 12 +
.../spark/{math/expm1.slt => aggregate/avg.slt} | 36 +-
.../test_files/spark/array/shuffle.slt | 113 +
.../test_files/spark/bitwise/bitwise_not.slt | 201 ++
.../test_files/spark/datetime/date_add.slt | 16 +
.../spark/string/{char.slt => concat.slt} | 30 +-
.../sqllogictest/test_files/spark/string/elt.slt | 37 +-
.../test_files/spark/string/format_string.slt | 2296 +++++++++++++-
.../test_files/spark/string/printf.slt | 7 +-
.../test_files/spark/url/parse_url.slt | 104 +-
.../test_files/spark/url/try_parse_url.slt | 127 +
.../sqllogictest/test_files/string/string_view.slt | 4 +-
datafusion/sqllogictest/test_files/subquery.slt | 2 +-
datafusion/sqllogictest/test_files/timestamps.slt | 46 +-
.../test_files/tpch/plans/q11.slt.part | 97 +-
.../test_files/tpch/plans/q16.slt.part | 2 +-
.../test_files/tpch/plans/q19.slt.part | 4 +-
.../test_files/tpch/plans/q22.slt.part | 41 +-
datafusion/sqllogictest/test_files/unnest.slt | 272 ++
datafusion/sqllogictest/test_files/window.slt | 27 +-
.../sqllogictest/test_files/window_limits.slt | 769 +++++
datafusion/substrait/Cargo.toml | 1 +
datafusion/substrait/src/lib.rs | 2 +-
.../src/logical_plan/consumer/expr/literal.rs | 34 +-
.../src/logical_plan/consumer/rel/aggregate_rel.rs | 1 +
.../substrait/src/logical_plan/consumer/types.rs | 6 +-
.../src/logical_plan/producer/expr/cast.rs | 21 +-
.../src/logical_plan/producer/expr/literal.rs | 53 +-
.../src/logical_plan/producer/expr/mod.rs | 2 +-
.../substrait/src/logical_plan/producer/plan.rs | 2 +-
.../src/logical_plan/producer/rel/read_rel.rs | 11 +-
.../logical_plan/producer/substrait_producer.rs | 19 +-
.../substrait/src/logical_plan/producer/types.rs | 48 +-
datafusion/substrait/src/physical_plan/producer.rs | 2 +-
datafusion/substrait/src/variation_const.rs | 3 +
.../{function_test.rs => aggregation_tests.rs} | 48 +-
datafusion/substrait/tests/cases/mod.rs | 1 +
.../tests/cases/roundtrip_logical_plan.rs | 2 +
.../no_groupings.json} | 73 +-
.../single_grouping.json} | 65 +-
datafusion/wasmtest/Cargo.toml | 3 +
datafusion/wasmtest/src/lib.rs | 2 +-
dev/changelog/50.1.0.md | 47 +
dev/changelog/{49.0.2.md => 50.2.0.md} | 16 +-
dev/changelog/50.3.0.md | 47 +
dev/release/README.md | 1 +
docs/.gitignore | 5 +-
docs/Makefile | 2 +-
docs/README.md | 20 +-
docs/build.sh | 9 +-
docs/make.bat | 3 +-
docs/requirements.txt | 14 +-
docs/rustdoc_trim.py | 37 +-
.../images/{original.svg => original_dark.svg} | 36 +-
docs/source/_static/theme_overrides.css | 19 +
docs/source/_templates/docs-sidebar.html | 30 +-
docs/source/conf.py | 52 +-
docs/source/contributor-guide/governance.md | 5 +-
docs/source/contributor-guide/inviting.md | 11 +-
docs/source/index.rst | 10 +-
.../library-user-guide/building-logical-plans.md | 6 +-
docs/source/library-user-guide/upgrading.md | 49 +
.../library-user-guide/working-with-exprs.md | 8 +-
docs/source/user-guide/cli/usage.md | 39 +
docs/source/user-guide/configs.md | 8 +-
docs/source/user-guide/explain-usage.md | 8 +-
docs/source/user-guide/features.md | 2 +-
docs/source/user-guide/introduction.md | 18 +-
.../README.md => docs/source/user-guide/metrics.md | 23 +-
docs/source/user-guide/sql/aggregate_functions.md | 45 +
docs/source/user-guide/sql/explain.md | 11 +-
docs/source/user-guide/sql/scalar_functions.md | 12 +-
docs/source/user-guide/sql/select.md | 117 +
typos.toml | 3 +
476 files changed, 34452 insertions(+), 11499 deletions(-)
create mode 100644 datafusion-cli/src/object_storage/instrumented.rs
create mode 100644 datafusion-cli/tests/snapshots/object_store_profiling.snap
create mode 100644 datafusion/catalog-listing/src/config.rs
create mode 100644 datafusion/catalog-listing/src/options.rs
create mode 100644 datafusion/catalog-listing/src/table.rs
copy datafusion/core/src/{scalar.rs => datasource/physical_plan/arrow.rs} (74%)
copy datafusion/core/tests/data/{empty.csv =>
empty_files/some_empty_with_header/a_empty.csv} (100%)
create mode 100644
datafusion/core/tests/data/empty_files/some_empty_with_header/b.csv
copy datafusion/core/tests/data/{empty.csv =>
empty_files/some_empty_with_header/c_nulls_column.csv} (64%)
create mode 100644 datafusion/core/tests/datasource/csv.rs
copy datafusion/core/tests/{memory_limit/memory_limit_validation =>
datasource}/mod.rs (84%)
create mode 100644 datafusion/core/tests/memory_limit/repartition_mem_limit.rs
rename datafusion/core/tests/{parquet_config.rs => parquet_integration.rs}
(100%)
create mode 100644
datafusion/core/tests/physical_optimizer/enforce_sorting_monotonicity.rs
copy datafusion/{datasource-csv => datasource-arrow}/Cargo.toml (85%)
copy LICENSE.txt => datafusion/datasource-arrow/LICENSE.txt (100%)
copy NOTICE.txt => datafusion/datasource-arrow/NOTICE.txt (100%)
copy datafusion/{common => datasource-arrow}/README.md (83%)
copy datafusion/{core/src/datasource/file_format/arrow.rs =>
datasource-arrow/src/file_format.rs} (84%)
copy datafusion/{datasource-json => datasource-arrow}/src/mod.rs (100%)
rename datafusion/{core/src/datasource/physical_plan/arrow_file.rs =>
datasource-arrow/src/source.rs} (91%)
rename datafusion/{core => datasource-arrow}/tests/data/example.arrow (100%)
delete mode 100644
datafusion/datasource-parquet/tests/apply_schema_adapter_tests.rs
delete mode 100644 datafusion/datasource/src/file_meta.rs
create mode 100644 datafusion/datasource/src/table_schema.rs
create mode 100644 datafusion/functions-aggregate/benches/min_max_bytes.rs
create mode 100644 datafusion/functions-aggregate/src/percentile_cont.rs
create mode 100644 datafusion/functions-aggregate/src/utils.rs
delete mode 100644 datafusion/physical-expr/src/equivalence/projection.rs
create mode 100644 datafusion/physical-expr/src/expressions/cast_column.rs
create mode 100644 datafusion/physical-expr/src/projection.rs
copy
datafusion/physical-plan/src/aggregates/group_values/multi_group_by/{primitive.rs
=> boolean.rs} (63%)
create mode 100644
datafusion/physical-plan/src/aggregates/group_values/single_group_by/boolean.rs
create mode 100644
datafusion/physical-plan/src/joins/piecewise_merge_join/classic_join.rs
create mode 100644
datafusion/physical-plan/src/joins/piecewise_merge_join/exec.rs
copy datafusion/{core/tests/fuzz_cases/equivalence =>
physical-plan/src/joins/piecewise_merge_join}/mod.rs (87%)
create mode 100644
datafusion/physical-plan/src/joins/piecewise_merge_join/utils.rs
create mode 100644 datafusion/spark/src/function/aggregate/avg.rs
copy datafusion/{functions-nested/src/reverse.rs =>
spark/src/function/array/shuffle.rs} (61%)
create mode 100644 datafusion/spark/src/function/bitwise/bitwise_not.rs
create mode 100644 datafusion/spark/src/function/string/concat.rs
create mode 100644 datafusion/spark/src/function/string/elt.rs
create mode 100644 datafusion/spark/src/function/string/format_string.rs
copy datafusion/{functions/src/math/pi.rs =>
spark/src/function/url/try_parse_url.rs} (51%)
create mode 100644 datafusion/sqllogictest/data/composite_order.csv
create mode 100644 datafusion/sqllogictest/test_files/current_date_timezone.slt
create mode 100644 datafusion/sqllogictest/test_files/current_time_timezone.slt
create mode 100644
datafusion/sqllogictest/test_files/dynamic_filter_pushdown_config.slt
create mode 100644 datafusion/sqllogictest/test_files/join_lists.slt
create mode 100644 datafusion/sqllogictest/test_files/pwmj.slt
copy datafusion/sqllogictest/test_files/spark/{math/expm1.slt =>
aggregate/avg.slt} (56%)
create mode 100644 datafusion/sqllogictest/test_files/spark/array/shuffle.slt
create mode 100644
datafusion/sqllogictest/test_files/spark/bitwise/bitwise_not.slt
copy datafusion/sqllogictest/test_files/spark/string/{char.slt => concat.slt}
(71%)
create mode 100644
datafusion/sqllogictest/test_files/spark/url/try_parse_url.slt
create mode 100644 datafusion/sqllogictest/test_files/window_limits.slt
copy datafusion/substrait/tests/cases/{function_test.rs =>
aggregation_tests.rs} (51%)
copy
datafusion/substrait/tests/testdata/test_plans/{aggregate_no_project.substrait.json
=> aggregate_groupings/no_groupings.json} (62%)
copy
datafusion/substrait/tests/testdata/test_plans/{aggregate_no_project_group_expression_ref.substrait.json
=> aggregate_groupings/single_grouping.json} (63%)
create mode 100644 dev/changelog/50.1.0.md
copy dev/changelog/{49.0.2.md => 50.2.0.md} (67%)
create mode 100644 dev/changelog/50.3.0.md
copy docs/source/_static/images/{original.svg => original_dark.svg} (90%)
copy datafusion/catalog/README.md => docs/source/user-guide/metrics.md (54%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
