Prathamesh9284 opened a new pull request, #1376:
URL: https://github.com/apache/datafusion-python/pull/1376
# Which issue does this PR close?
Closes #508
# Rationale for this change
Other Substrait producers (Isthmus, DuckDB) support generating
JSON-formatted Substrait plans, which makes it much easier to inspect and
compare plans across engines. The substrait crate already provides serde
support via the pbjson feature, so DataFusion Python can leverage this to
expose JSON plan generation and parsing with minimal effort.
# What changes are included in this PR?
- **`python/datafusion/substrait.py`**
- Added `to_json()` instance method and `from_json()` static method on the
`Plan` class for converting plans to/from JSON strings.
- Added `serialize_json()` and `deserialize_json()` static methods on the
`Serde` class for writing/reading JSON plan files.
- **`src/substrait.rs`**
- Implemented the Rust backend for the above Python methods, including
error mapping.
- Added the following functions:
- `fn to_json(&self) -> PyDataFusionResult<String>`
- `#[staticmethod] fn from_json(json: &str) ->
PyDataFusionResult<PyPlan>`
- `#[staticmethod] pub fn serialize_json(sql: &str, ctx:
PySessionContext, path: &str, py: Python) -> PyDataFusionResult<()>`
- `#[staticmethod] pub fn deserialize_json(path: &str) ->
PyDataFusionResult<PyPlan>`
- **`src/errors.rs`**
- Added `SerdeJsonError` and `DecodeError` error variants to
`PyDataFusionError` for proper error handling.
- **`Cargo.toml`**
- Added `serde_json` as a dependency.
- **`python/tests/test_substrait.py`**
- Added tests for both in-memory and file-based JSON plan round-trips.
# Are there any user-facing changes?
Yes — four new public API methods:
- `Plan.to_json() -> str` — convert a Substrait plan to a JSON string
- `Plan.from_json(json: str) -> Plan` — create a Substrait plan from a JSON
string
- `Serde.serialize_json(sql, ctx, path)` — generate a JSON-formatted
Substrait plan file from a SQL query
- `Serde.deserialize_json(path) -> Plan` — read a Substrait plan from a JSON
file
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
