This is an automated email from the ASF dual-hosted git repository.
martin-g pushed a change to branch
dependabot/cargo/python/main/object_store-0.14.0
in repository https://gitbox.apache.org/repos/asf/datafusion-ballista.git
from 85a2f8b2d chore(deps): bump object_store from 0.13.2 to 0.14.0 in
/python
add 6472c7f21 feat(scheduler): allow submitting an already-built physical
plan (#1924)
add b880ca419 feat: cancel running stages and tasks on job failure or
cancellation (#1903)
add 3b8701876 refactor(scheduler): keep submit_job non-breaking, add
submit_physical_plan (#1941)
add ea9adfbea feat(benchmarks): support reading TPC-H data from
S3-compatible object storage (#1942)
add 6ccbcd54c docs: add versioned upgrade guide (54.0.0) (#1939)
add e05fe5ea3 feat: bound shuffle fetch with a reduce-side in-flight
governor (#1951)
add ce9eefabc chore(deps): bump ctor from 1.0.7 to 1.0.8 (#1957)
add 182222743 chore(deps): bump bytesize from 2.4.0 to 2.4.2 (#1956)
add e0855643d chore(deps): bump astral-sh/setup-uv from 8.2.0 to 8.3.0
(#1955)
add a96c381fb chore(deps): bump taiki-e/install-action from 2.82.6 to
2.82.7 (#1918)
add 24e95f9ff fix AQE planning with S3 (#1962)
add 4f1d7da52 Add TPC-H distributed plan-stability test suite (#1966)
add 5298a9f1f feat: display input stage id on UnresolvedShuffleExec
(#1967) (#1969)
add ce768eb35 chore(deps): bump aws-config from 1.8.18 to 1.9.0 (#1976)
add 006af4b87 chore(deps): bump aws-credential-types from 1.2.14 to 1.3.0
(#1975)
add b93bcc0df chore(deps): bump astral-sh/setup-uv from 8.3.0 to 8.3.2
(#1970)
add 8a4ef0bd3 chore(deps): bump actions/labeler from 6.1.0 to 6.2.0 (#1972)
add fd30f3775 chore(deps): bump taiki-e/install-action from 2.82.7 to
2.82.11 (#1973)
add 647cefe4d docs: document TPC-H plan-stability golden files and
regeneration (#1979)
add ec79000d4 fix: satisfy clippy 1.97 useless_borrows_in_formatting
(#1978)
add 2959cb7f9 perf: avoid redundant TaskStatus clones in scheduler
update_task_status (#1983)
add 434012f2e chore(deps): bump regex from 1.12.4 to 1.13.0 (#1985)
add 4f5e7c71e chore(deps): bump taiki-e/install-action from 2.82.11 to
2.83.0 (#1984)
add aad835288 chore(deps): bump pyarrow from 23.0.0 to 23.0.1 in /python
(#1961)
add bf9d9b449 chore(deps): bump uuid from 1.23.1 to 1.23.4 in /python
(#1936)
add 2555985e4 chore(deps): bump insta from 1.47.2 to 1.48.0 in /python
(#1935)
add 2ed13c7b0 chore(deps): bump pyo3-log from 0.13.3 to 0.13.4 in /python
(#1934)
add 4bd3d0e0b chore(deps): bump rand from 0.10.1 to 0.10.2 (#1933)
add 4365c19f7 chore(deps): bump log from 0.4.29 to 0.4.33 in /python
(#1932)
add 168406090 chore(deps): bump pyo3-build-config from 0.28.3 to 0.29.0 in
/python (#1931)
add 4b2a8d4a2 chore(deps): bump rand from 0.10.1 to 0.10.2 in /python
(#1928)
add 36b661ad7 chore(deps): bump prost from 0.14.3 to 0.14.4 in /python
(#1929)
add a178e088c Merge branch 'main' into
dependabot/cargo/python/main/object_store-0.14.0
No new revisions were added by this update.
Summary of changes:
.github/workflows/build.yml | 14 +-
.github/workflows/dev_pr.yml | 2 +-
.github/workflows/rust.yml | 2 +-
.github/workflows/tpch.yml | 2 +-
.github/workflows/web-tui.yml | 4 +-
Cargo.lock | 120 ++++---
.../src/tui/ui/main/jobs/stage_tasks_popup.rs | 2 +-
ballista/client/Cargo.toml | 1 +
ballista/client/src/extension.rs | 2 +-
ballista/client/tests/context_checks.rs | 2 +-
ballista/client/tests/physical_plan_submission.rs | 132 +++++++
ballista/client/tests/sort_shuffle.rs | 55 ++-
ballista/core/proto/ballista.proto | 5 +
ballista/core/src/client.rs | 12 +
ballista/core/src/config.rs | 60 ++++
.../core/src/execution_plans/distributed_query.rs | 88 +++++
ballista/core/src/execution_plans/mod.rs | 2 +-
.../core/src/execution_plans/shuffle_reader.rs | 382 +++++++++++++++++++--
.../core/src/execution_plans/unresolved_shuffle.rs | 22 +-
ballista/core/src/object_store.rs | 2 +-
ballista/core/src/serde/generated/ballista.rs | 8 +-
ballista/core/src/utils.rs | 25 +-
ballista/executor/src/client_pool.rs | 2 +
ballista/executor/src/executor_process.rs | 2 +-
ballista/executor/src/executor_server.rs | 14 +-
ballista/scheduler/Cargo.toml | 5 +
ballista/scheduler/src/planner.rs | 20 +-
ballista/scheduler/src/scheduler_server/event.rs | 27 +-
ballista/scheduler/src/scheduler_server/grpc.rs | 48 ++-
ballista/scheduler/src/scheduler_server/mod.rs | 95 ++++-
.../src/scheduler_server/query_stage_scheduler.rs | 2 +-
ballista/scheduler/src/state/aqe/mod.rs | 19 +-
ballista/scheduler/src/state/aqe/planner.rs | 22 +-
.../scheduler/src/state/aqe/test/job_failure.rs | 139 ++++++++
ballista/scheduler/src/state/aqe/test/mod.rs | 2 +
ballista/scheduler/src/state/execution_graph.rs | 103 +++++-
ballista/scheduler/src/state/execution_stage.rs | 41 ++-
ballista/scheduler/src/state/mod.rs | 14 +-
ballista/scheduler/src/state/task_manager.rs | 147 ++++++--
.../scheduler/tests/tpch_plan_stability/README.md | 59 ++++
.../tests/tpch_plan_stability/approved/q1.txt | 18 +
.../tests/tpch_plan_stability/approved/q10.txt | 60 ++++
.../tests/tpch_plan_stability/approved/q11.txt | 87 +++++
.../tests/tpch_plan_stability/approved/q12.txt | 30 ++
.../tests/tpch_plan_stability/approved/q13.txt | 32 ++
.../tests/tpch_plan_stability/approved/q14.txt | 25 ++
.../tests/tpch_plan_stability/approved/q15.txt | 57 +++
.../tests/tpch_plan_stability/approved/q16.txt | 46 +++
.../tests/tpch_plan_stability/approved/q17.txt | 36 ++
.../tests/tpch_plan_stability/approved/q18.txt | 46 +++
.../tests/tpch_plan_stability/approved/q19.txt | 26 ++
.../tests/tpch_plan_stability/approved/q2.txt | 137 ++++++++
.../tests/tpch_plan_stability/approved/q20.txt | 69 ++++
.../tests/tpch_plan_stability/approved/q21.txt | 82 +++++
.../tests/tpch_plan_stability/approved/q22.txt | 35 ++
.../tests/tpch_plan_stability/approved/q3.txt | 40 +++
.../tests/tpch_plan_stability/approved/q4.txt | 31 ++
.../tests/tpch_plan_stability/approved/q5.txt | 89 +++++
.../tests/tpch_plan_stability/approved/q6.txt | 6 +
.../tests/tpch_plan_stability/approved/q7.txt | 89 +++++
.../tests/tpch_plan_stability/approved/q8.txt | 118 +++++++
.../tests/tpch_plan_stability/approved/q9.txt | 85 +++++
.../tests/tpch_plan_stability/fixtures.rs | 229 ++++++++++++
.../scheduler/tests/tpch_plan_stability/main.rs | 112 ++++++
.../tests/tpch_plan_stability/stats_table.rs | 152 ++++++++
benchmarks/Cargo.toml | 4 +-
benchmarks/src/bin/tpch.rs | 38 +-
dev/release/rat_exclude_files.txt | 1 +
.../update-tpch-plan-stability.sh | 10 +-
docs/source/contributors-guide/development.md | 34 ++
docs/source/index.rst | 6 +
docs/source/upgrading/54.0.0.md | 125 +++++++
.../{user-guide/python => upgrading}/index.rst | 10 +-
examples/examples/standalone-substrait.rs | 2 +
examples/tests/object_store.rs | 143 ++++++++
python/Cargo.lock | 67 ++--
python/Cargo.toml | 2 +-
python/pyproject.toml | 2 +-
python/uv.lock | 108 +++---
79 files changed, 3659 insertions(+), 335 deletions(-)
create mode 100644 ballista/client/tests/physical_plan_submission.rs
create mode 100644 ballista/scheduler/src/state/aqe/test/job_failure.rs
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/README.md
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/approved/q1.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q10.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q11.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q12.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q13.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q14.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q15.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q16.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q17.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q18.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q19.txt
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/approved/q2.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q20.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q21.txt
create mode 100644
ballista/scheduler/tests/tpch_plan_stability/approved/q22.txt
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/approved/q3.txt
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/approved/q4.txt
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/approved/q5.txt
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/approved/q6.txt
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/approved/q7.txt
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/approved/q8.txt
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/approved/q9.txt
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/fixtures.rs
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/main.rs
create mode 100644 ballista/scheduler/tests/tpch_plan_stability/stats_table.rs
copy ci/scripts/rust_fmt.sh => dev/update-tpch-plan-stability.sh (67%)
create mode 100644 docs/source/upgrading/54.0.0.md
copy docs/source/{user-guide/python => upgrading}/index.rst (85%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
