This is an automated email from the ASF dual-hosted git repository.
milenkovicm pushed a change to branch redirect-root-to-nightlies
in repository https://gitbox.apache.org/repos/asf/datafusion-ballista.git
from b7050bc91 taplo fmt
add 03fc4be2a chore(deps): bump rustls from 0.23.42 to 0.23.43 (#2209)
add 234936485 chore(deps): bump http from 1.4.2 to 1.5.0 (#2208)
add 1bbd47725 chore(deps): bump ctor from 1.0.11 to 1.0.12 (#2206)
add 017386fcc fix(cli): sort jobs by stage completion ratio (#1947)
add 39092aea1 chore(ci): bump taiki-e/install-action from 2.85.3 to 2.85.4
(#2205)
add bb9757fd4 chore(deps): bump nix from 0.29.0 to 0.31.3 (#2207)
add 187a65063 chore(deps): bump reqwest from 0.12.28 to 0.13.4 (#2210)
add 6e9ff68a5 fix(scheduler): preserve stages on transition errors (#2215)
add c43e56eb4 feat(scheduler): range-repartition rule primitives
(batteries-included) (#2196)
add 63fa496b5 chore(ci): bump the github-codeql group with 2 updates
(#2218)
add a3161a301 fix(scheduler): fail running jobs when all executors are
lost (#2029) (#2212)
add 9fecadae8 feat: make gRPC max message size consistently configurable
(#2173)
add 984513ee7 fix(executor): bind the gRPC listener before registering
with the scheduler (#2225)
add 10d4d0877 feat: add `poll_now_notify` to `poll_loop` and
`on_work_available` callback (#1893)
add 5355a36f4 fix: update mTLS example for poll loop arguments (#2229)
add 0df207e13 chore(deps): bump bytesize from 2.4.2 to 2.7.0 (#2227)
add 947f7d901 chore(ci): bump taiki-e/install-action from 2.85.4 to 2.85.7
(#2228)
add 121f136bf chore(deps): bump cryptography from 48.0.1 to 50.0.0 in
/python (#2221)
add f1bdc9f3b feat(scheduler): expose existing job state event stream
(#1891)
add a7838ef91 Merge branch 'main' into redirect-root-to-nightlies
No new revisions were added by this update.
Summary of changes:
.github/workflows/codeql.yml | 4 +-
.github/workflows/rust.yml | 2 +-
.github/workflows/tpch.yml | 2 +-
.github/workflows/web-tui.yml | 4 +-
Cargo.lock | 185 ++-----
ballista-cli/src/tui/domain/jobs.rs | 41 +-
ballista/core/src/config.rs | 26 +
ballista/core/src/execution_plans/mod.rs | 9 +-
.../src/execution_plans/per_partition_filter.rs | 153 +++++-
ballista/core/src/execution_plans/plan_algebra.rs | 70 +++
.../execution_plans/range_repartition_common.rs | 55 +-
ballista/core/src/execution_plans/runtime_stats.rs | 561 +++++++++++++++++++--
.../core/src/execution_plans/shuffle_writer.rs | 46 +-
ballista/core/src/utils.rs | 59 +++
ballista/executor/src/config.rs | 4 +-
ballista/executor/src/execution_loop.rs | 30 +-
ballista/executor/src/executor_process.rs | 1 +
ballista/executor/src/executor_server.rs | 14 +-
ballista/executor/src/standalone.rs | 1 +
ballista/scheduler/Cargo.toml | 1 +
ballista/scheduler/src/cluster/memory.rs | 66 ++-
ballista/scheduler/src/config.rs | 92 +++-
ballista/scheduler/src/scheduler_server/mod.rs | 40 +-
.../src/scheduler_server/query_stage_scheduler.rs | 419 ++++++++++++++-
ballista/scheduler/src/state/aqe/adapter.rs | 24 +-
.../src/state/aqe/execution_plan/exchange.rs | 147 ++++++
ballista/scheduler/src/state/aqe/mod.rs | 97 +++-
.../aqe/optimizer_rule/coalesce_partitions.rs | 10 +
.../aqe/optimizer_rule/distributed_exchange.rs | 336 +++++++++++-
ballista/scheduler/src/state/aqe/planner.rs | 60 ++-
.../scheduler/src/state/aqe/test/coalesce_rule.rs | 61 +++
ballista/scheduler/src/state/aqe/test/mod.rs | 3 +
.../src/state/aqe/test/range_repartition.rs | 254 ++++++++++
ballista/scheduler/src/state/execution_graph.rs | 203 +++++++-
ballista/scheduler/src/state/executor_manager.rs | 72 ++-
ballista/scheduler/src/state/task_builder.rs | 96 +++-
ballista/scheduler/src/state/task_manager.rs | 7 +-
ballista/scheduler/src/test_utils.rs | 33 +-
chaos-testing/Cargo.toml | 4 +-
chaos-testing/README.md | 21 +-
chaos-testing/src/bin/chaos-scheduler.rs | 6 +
chaos-testing/src/cluster.rs | 53 ++
chaos-testing/tests/ha.rs | 39 +-
examples/examples/mtls-cluster.rs | 2 +-
python/uv.lock | 99 ++--
45 files changed, 3105 insertions(+), 407 deletions(-)
create mode 100644 ballista/core/src/execution_plans/plan_algebra.rs
create mode 100644 ballista/scheduler/src/state/aqe/test/range_repartition.rs
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
