This is an automated email from the ASF dual-hosted git repository.
timsaucer pushed a change to branch feat/distributed-extensions-example
in repository https://gitbox.apache.org/repos/asf/datafusion-python.git
from 05b40fb8 Build the example libraries on the host, and prime their deps
once
add 016ad9d6 Number the stages, and ship all of them
add 3a61caf8 Read the third codec id instead of copying it
add e3ae58e0 Make the example's run instructions work
add 6e3dc962 Cover build_session's own consistency check
add be191ac5 Point local_session.rs at a section that exists
add 4662e183 Say why table_options_mut panics on purpose
add 60bb0f81 Link the two sections this PR added
add 1774343c Decode a provider with the schema from the plan
add 628c4517 Refuse a malformed projection instead of dropping it
add 22912acd Raise Execution, not Internal, for a bad payload
add 9f9f688a Stop converting a fallback this planner never calls
add 4a8cb2ce Clear the small stuff off the distributed example
No new revisions were added by this update.
Summary of changes:
.github/workflows/build.yml | 6 +-
.github/workflows/test.yml | 10 +-
.gitignore | 3 +
docs/source/extension-guide/checklist.md | 14 +-
.../user-guide/distributing-work/expressions.md | 6 +
examples/distributed/README.md | 22 ++-
examples/distributed/engine-library/pyproject.toml | 19 +++
.../engine-library/python/dfx_engine/driver.py | 159 ++++++++++++++-------
.../engine-library/python/dfx_engine/session.py | 14 +-
.../python/tests/_test_three_libraries.py | 112 +++++++++++++--
examples/distributed/engine-library/src/codec.rs | 14 +-
.../distributed/engine-library/src/extension.rs | 32 +++--
examples/distributed/engine-library/src/lib.rs | 16 ++-
.../engine-library/src/local_session.rs | 26 +++-
examples/distributed/engine-library/src/planner.rs | 130 ++++++++++-------
examples/distributed/engine-library/src/stage.rs | 75 +++++++---
examples/distributed/run_tpch.py | 60 +++++---
.../distributed/storage-library/pyproject.toml | 14 ++
.../python/tests/_test_portable_codec.py | 101 ++++++++++++-
examples/distributed/storage-library/src/codec.rs | 114 +++++++++++----
examples/distributed/storage-library/src/exec.rs | 3 +
.../distributed/storage-library/src/extension.rs | 16 +++
examples/distributed/storage-library/src/lib.rs | 5 +-
.../storage-library/src/table_provider.rs | 37 ++++-
examples/distributed/udf-library/pyproject.toml | 7 +
25 files changed, 780 insertions(+), 235 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
