This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/cargo/master/configure_me_codegen-eq-0.4.3
in repository https://gitbox.apache.org/repos/asf/arrow-ballista.git
discard 173fffd6 Update configure_me_codegen requirement from =0.4.0 to =0.4.3
add 5f0cf858 Remove capture group (#527)
add f3e7e49b fix python build in CI (#528)
add b3705999 Update docs for simplified instructions (#532)
add f6ace5e5 remove --locked (#533)
add 20087c12 Bump actions/labeler from 4.0.2 to 4.1.0 (#525)
add b5fa8720 Provide a memory StateBackendClient (#523)
add acbdf608 only build docker images on rc tags (#535)
add 70d4a7bb docs: fix style in the Helm readme (#551)
add 9bfed3e9 Fix Helm chart's image format (#550)
add a7c527f3 Update datafusion requirement from 14.0.0 to 15.0.0 (#552)
add 7c7a9979 Make it concurrently to launch tasks to executors (#557)
add 20891ae0 fix(ui): fix last seen (#562)
add a2661e49 Support Alibaba Cloud OSS with ObjectStore (#567)
add 908aa5a1 Fix cargo clippy (#571)
add 7c44b5ec Super minor spelling error (#573)
add 316486d8 Update env_logger requirement from 0.9 to 0.10 (#539)
add d2f69798 Update graphviz-rust requirement from 0.4.0 to 0.5.0 (#574)
add 85dc0836 updated readme to contain correct versions of dependencies.
(#580)
add 03e351f8 Fix benchmark image link (#596)
add 7f285339 Add support for Azure (#599)
add 45034da7 Remove outdated script and use evergreen version of rust
(#597)
add f59179a0 feat: update script such that ballista-cli image is built as
well (#601)
add 18e2cc88 Fix Cargo.toml format issue (#616)
add cbad55d7 Refactor executor main (#614)
add b1713967 Refactor scheduler main (#615)
add dfc168c6 Python: add method to get explain output as a string (#593)
add a621a3f8 Update contributor guide (#617)
add 93cf14f7 Cluster state refactor part 1 (#560)
add 75b4f845 replace master with main (#621)
add a5f306e5 implement new release process (#623)
add 70032e1f add docs on who can release (#632)
add 0b6b28f8 Upgrade to DataFusion 16 (again) (#636)
add b0d8cc7e Update datafusion dependency to the latest version (#612)
add 1b0be751 Upgrade to DataFusion 17 (#639)
add 1a6bc262 check in benchmark image (#647)
add 918c3448 Remove `python` dir & python-related workflows (#654)
add 0b8d99c3 Handle job resubmission (#586)
add 8f8154f4 Add executor self-registration mechanism in the heartbeat
service (#649)
add e7f87741 Cluster state refactor Part 2 (#658)
add a3ac8575 Upgrade to DataFusion 18.0.0-rc1 (#664)
add a9fd64d8 Minor refactor to reduce duplicate code (#659)
add cdfb3a09 move test_util to ballista-examples package (#661)
add cbd6e3e1 Upgrade to DataFusion 18 (#668)
add 5b180a12 Enable physical plan round-trip tests (#666)
add eba125e1 Prep 0.11 (#682)
add 8fe416a8 Update configure_me_codegen requirement from =0.4.0 to =0.4.3
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (173fffd6)
\
N -- N -- N
refs/heads/dependabot/cargo/master/configure_me_codegen-eq-0.4.3 (8fe416a8)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.dockerignore | 1 +
.github/dependabot.yml | 2 +-
.github/workflows/comment_bot.yml | 6 +-
.github/workflows/dev.yml | 4 +-
.github/workflows/dev_pr.yml | 2 +-
.github/workflows/python_build.yml | 138 -
.github/workflows/python_test.yaml | 73 -
.github/workflows/rust.yml | 29 +-
.gitignore | 2 +-
CONTRIBUTING.md | 212 +-
Cargo.toml | 4 +-
README.md | 2 +-
ballista-cli/Cargo.toml | 12 +-
ballista-cli/src/command.rs | 6 +-
ballista-cli/src/exec.rs | 12 +-
ballista-cli/src/main.rs | 10 +-
ballista/CHANGELOG.md | 82 +
ballista/client/Cargo.toml | 17 +-
ballista/client/README.md | 4 +-
ballista/client/src/context.rs | 88 +-
ballista/core/Cargo.toml | 16 +-
ballista/core/build.rs | 37 +-
ballista/core/proto/ballista.proto | 437 +-
ballista/core/proto/datafusion.proto | 868 +-
ballista/core/src/client.rs | 23 +-
ballista/core/src/config.rs | 13 +-
ballista/core/src/error.rs | 48 +-
ballista/core/src/event_loop.rs | 7 +-
.../core/src/execution_plans/distributed_query.rs | 34 +-
.../core/src/execution_plans/shuffle_reader.rs | 28 +-
.../core/src/execution_plans/shuffle_writer.rs | 17 +-
.../core/src/execution_plans/unresolved_shuffle.rs | 4 -
ballista/core/src/lib.rs | 2 +-
ballista/core/src/plugin/mod.rs | 2 +-
ballista/core/src/plugin/plugin_manager.rs | 2 +-
ballista/core/src/plugin/udf.rs | 6 +-
ballista/core/src/serde/generated/.gitignore | 4 -
ballista/core/src/serde/generated/ballista.rs | 2423 ++++++
ballista/core/src/serde/mod.rs | 753 +-
.../core/src/serde/physical_plan/from_proto.rs | 416 -
ballista/core/src/serde/physical_plan/mod.rs | 1701 ----
ballista/core/src/serde/physical_plan/to_proto.rs | 464 --
ballista/core/src/serde/scheduler/from_proto.rs | 24 +-
ballista/core/src/serde/scheduler/mod.rs | 4 +-
ballista/core/src/serde/scheduler/to_proto.rs | 19 +-
ballista/core/src/utils.rs | 42 +-
ballista/executor/Cargo.toml | 17 +-
ballista/executor/build.rs | 2 +-
ballista/executor/src/bin/main.rs | 83 +
ballista/executor/src/collect.rs | 8 +-
ballista/executor/src/execution_loop.rs | 8 +-
ballista/executor/src/executor.rs | 4 +-
.../executor/src/{main.rs => executor_process.rs} | 139 +-
ballista/executor/src/executor_server.rs | 45 +-
ballista/executor/src/flight_service.rs | 41 +-
ballista/executor/src/lib.rs | 1 +
ballista/executor/src/standalone.rs | 3 +-
ballista/scheduler/Cargo.toml | 18 +-
ballista/scheduler/build.rs | 4 +-
ballista/scheduler/scheduler_config_spec.toml | 14 +-
ballista/scheduler/src/api/handlers.rs | 7 +-
ballista/scheduler/src/api/mod.rs | 2 +-
ballista/scheduler/src/bin/main.rs | 127 +
ballista/scheduler/src/cluster/event/mod.rs | 318 +
ballista/scheduler/src/cluster/kv.rs | 781 ++
ballista/scheduler/src/cluster/memory.rs | 588 ++
ballista/scheduler/src/cluster/mod.rs | 443 ++
.../src/{state/backend => cluster/storage}/etcd.rs | 28 +-
.../src/{state/backend => cluster/storage}/mod.rs | 77 +-
.../standalone.rs => cluster/storage/sled.rs} | 88 +-
ballista/scheduler/src/cluster/test/mod.rs | 597 ++
ballista/scheduler/src/config.rs | 60 +
ballista/scheduler/src/display.rs | 4 +-
ballista/scheduler/src/flight_sql.rs | 106 +-
ballista/scheduler/src/lib.rs | 2 +
ballista/scheduler/src/main.rs | 266 -
ballista/scheduler/src/metrics/prometheus.rs | 16 +-
ballista/scheduler/src/planner.rs | 137 +-
ballista/scheduler/src/scheduler_process.rs | 119 +
ballista/scheduler/src/scheduler_server/event.rs | 1 +
.../src/scheduler_server/external_scaler.rs | 2 +-
ballista/scheduler/src/scheduler_server/grpc.rs | 211 +-
ballista/scheduler/src/scheduler_server/mod.rs | 121 +-
.../src/scheduler_server/query_stage_scheduler.rs | 135 +-
ballista/scheduler/src/standalone.rs | 23 +-
ballista/scheduler/src/state/execution_graph.rs | 404 +-
.../src/state/execution_graph/execution_stage.rs | 16 +-
.../scheduler/src/state/execution_graph_dot.rs | 205 +-
ballista/scheduler/src/state/executor_manager.rs | 494 +-
ballista/scheduler/src/state/mod.rs | 210 +-
ballista/scheduler/src/state/session_manager.rs | 90 +-
ballista/scheduler/src/state/task_manager.rs | 369 +-
ballista/scheduler/src/test_utils.rs | 411 +-
ballista/scheduler/ui/.gitignore | 4 +-
ballista/scheduler/ui/src/components/DataTable.tsx | 3 +-
ballista/scheduler/ui/yarn.lock | 8396 ++++++++++----------
benchmarks/Cargo.toml | 14 +-
benchmarks/src/bin/nyctaxi.rs | 15 +-
benchmarks/src/bin/tpch.rs | 330 +-
ci/scripts/rust_toml_fmt.sh | 2 +-
dev/build-ballista-docker.sh | 7 +-
...lista-rust.sh => build-ballista-executables.sh} | 4 +
dev/build-ui.sh | 23 -
dev/docker/ballista-builder.Dockerfile | 2 +-
...executor.Dockerfile => ballista-cli.Dockerfile} | 11 +-
.../{executor-entrypoint.sh => cli-entrypoint.sh} | 2 +-
dev/release/README.md | 187 +-
dev/release/create-tarball.sh | 2 +-
dev/release/rat_exclude_files.txt | 1 +
dev/release/update_change_log-ballista.sh | 2 +-
dev/release/update_change_log.sh | 2 +-
dev/update_ballista_versions.py | 1 -
docs/developer/README.md | 11 -
docs/developer/configuration.md | 34 -
docs/developer/dev-env.md | 51 -
docs/developer/integration-testing.md | 29 -
docs/source/conf.py | 4 +-
docs/source/index.rst | 2 +-
.../source/user-guide/deployment/docker-compose.md | 2 +-
docs/source/user-guide/flightsql.md | 44 +-
docs/source/user-guide/python.md | 2 +-
...bench-h-workstation-10-distributed-perquery.png | Bin 0 -> 33223 bytes
examples/Cargo.toml | 8 +-
examples/examples/standalone-sql.rs | 5 +-
.../scheduler/ui/index.d.ts => examples/src/lib.rs | 2 +-
examples/src/test_util.rs | 89 +
helm/README.md | 2 +
helm/ballista/templates/executor.yaml | 2 +-
helm/ballista/templates/scheduler.yaml | 2 +-
helm/ballista/values.yaml | 2 +-
python/.cargo/config | 22 -
python/.dockerignore | 19 -
python/.gitignore | 20 -
python/CHANGELOG.md | 129 -
python/Cargo.toml | 54 -
python/LICENSE.txt | 202 -
python/README.md | 185 -
python/ballista/__init__.py | 113 -
python/ballista/functions.py | 23 -
python/ballista/tests/__init__.py | 16 -
python/ballista/tests/generic.py | 87 -
python/ballista/tests/test_aggregation.py | 48 -
python/ballista/tests/test_catalog.py | 72 -
python/ballista/tests/test_context.py | 63 -
python/ballista/tests/test_dataframe.py | 181 -
python/ballista/tests/test_functions.py | 219 -
python/ballista/tests/test_imports.py | 65 -
python/ballista/tests/test_sql.py | 250 -
python/ballista/tests/test_udaf.py | 136 -
python/pyproject.toml | 55 -
python/requirements-310.txt | 210 -
python/requirements-37.txt | 318 -
python/requirements.in | 27 -
python/requirements.txt | 282 -
python/src/ballista_context.rs | 137 -
python/src/catalog.rs | 127 -
python/src/context.rs | 257 -
python/src/dataframe.rs | 170 -
python/src/dataset.rs | 134 -
python/src/dataset_exec.rs | 282 -
python/src/errors.rs | 111 -
python/src/expression.rs | 138 -
python/src/functions.rs | 346 -
python/src/lib.rs | 71 -
python/src/pyarrow_filter_expression.rs | 219 -
python/src/udaf.rs | 146 -
python/src/udf.rs | 95 -
python/src/utils.rs | 47 -
168 files changed, 13391 insertions(+), 15929 deletions(-)
delete mode 100644 .github/workflows/python_build.yml
delete mode 100644 .github/workflows/python_test.yaml
delete mode 100644 ballista/core/src/serde/generated/.gitignore
create mode 100644 ballista/core/src/serde/generated/ballista.rs
delete mode 100644 ballista/core/src/serde/physical_plan/from_proto.rs
delete mode 100644 ballista/core/src/serde/physical_plan/mod.rs
delete mode 100644 ballista/core/src/serde/physical_plan/to_proto.rs
create mode 100644 ballista/executor/src/bin/main.rs
rename ballista/executor/src/{main.rs => executor_process.rs} (85%)
create mode 100644 ballista/scheduler/src/bin/main.rs
create mode 100644 ballista/scheduler/src/cluster/event/mod.rs
create mode 100644 ballista/scheduler/src/cluster/kv.rs
create mode 100644 ballista/scheduler/src/cluster/memory.rs
create mode 100644 ballista/scheduler/src/cluster/mod.rs
rename ballista/scheduler/src/{state/backend => cluster/storage}/etcd.rs (94%)
rename ballista/scheduler/src/{state/backend => cluster/storage}/mod.rs (79%)
rename ballista/scheduler/src/{state/backend/standalone.rs =>
cluster/storage/sled.rs} (82%)
create mode 100644 ballista/scheduler/src/cluster/test/mod.rs
delete mode 100644 ballista/scheduler/src/main.rs
create mode 100644 ballista/scheduler/src/scheduler_process.rs
rename dev/{build-ballista-rust.sh => build-ballista-executables.sh} (79%)
delete mode 100755 dev/build-ui.sh
copy dev/docker/{ballista-executor.Dockerfile => ballista-cli.Dockerfile} (76%)
copy dev/docker/{executor-entrypoint.sh => cli-entrypoint.sh} (96%)
delete mode 100644 docs/developer/configuration.md
delete mode 100644 docs/developer/dev-env.md
delete mode 100644 docs/developer/integration-testing.md
create mode 100644 docs/sqlbench-h-workstation-10-distributed-perquery.png
copy ballista/scheduler/ui/index.d.ts => examples/src/lib.rs (95%)
create mode 100644 examples/src/test_util.rs
delete mode 100644 python/.cargo/config
delete mode 100644 python/.dockerignore
delete mode 100644 python/.gitignore
delete mode 100644 python/CHANGELOG.md
delete mode 100644 python/Cargo.toml
delete mode 100644 python/LICENSE.txt
delete mode 100644 python/README.md
delete mode 100644 python/ballista/__init__.py
delete mode 100644 python/ballista/functions.py
delete mode 100644 python/ballista/tests/__init__.py
delete mode 100644 python/ballista/tests/generic.py
delete mode 100644 python/ballista/tests/test_aggregation.py
delete mode 100644 python/ballista/tests/test_catalog.py
delete mode 100644 python/ballista/tests/test_context.py
delete mode 100644 python/ballista/tests/test_dataframe.py
delete mode 100644 python/ballista/tests/test_functions.py
delete mode 100644 python/ballista/tests/test_imports.py
delete mode 100644 python/ballista/tests/test_sql.py
delete mode 100644 python/ballista/tests/test_udaf.py
delete mode 100644 python/pyproject.toml
delete mode 100644 python/requirements-310.txt
delete mode 100644 python/requirements-37.txt
delete mode 100644 python/requirements.in
delete mode 100644 python/requirements.txt
delete mode 100644 python/src/ballista_context.rs
delete mode 100644 python/src/catalog.rs
delete mode 100644 python/src/context.rs
delete mode 100644 python/src/dataframe.rs
delete mode 100644 python/src/dataset.rs
delete mode 100644 python/src/dataset_exec.rs
delete mode 100644 python/src/errors.rs
delete mode 100644 python/src/expression.rs
delete mode 100644 python/src/functions.rs
delete mode 100644 python/src/lib.rs
delete mode 100644 python/src/pyarrow_filter_expression.rs
delete mode 100644 python/src/udaf.rs
delete mode 100644 python/src/udf.rs
delete mode 100644 python/src/utils.rs
