wirybeaver commented on PR #2389:
URL:
https://github.com/apache/datafusion-ballista/pull/2389#issuecomment-5648257543
Thank you for the concern. I agree that an unreleased fork should not become
part of Ballista's release dependencies. **#2389 is intentionally still a Draft
and I am not proposing that it be merged in its current state.** It should only
be reconsidered after hudi-rs has an official compatible release.
```text
#2389: validation only (Draft)
Ballista workspace / root Cargo.lock / published package
|
| no hudi-rs edge
v
+----------------+
| build gate/SPI |
+-------+--------+
|
explicit standalone build
|
v
+-----------------------+
| integrations/hudi |
| separate crate |
| excluded from workspace|
| hudi-rs pin lives here|
+-----------------------+
```
The purpose of this Draft is to prove that the #2388 boundary can execute a
realistic projected and filtered distributed read while keeping hudi-rs outside
Ballista's main crate, root lockfile, published package, and release graph. The
private-fork pin is temporary test scaffolding confined to that excluded crate.
The intended long-term convention is one independent crate per open table
format under the Apache project umbrella. Iceberg, Delta Lake, Lance, and Hudi
integrations would follow the same build-gated boundary and own their
dependencies separately. Hudi was selected because Iceberg and Delta work
already exists, but not yet under this proposed isolation model. If useful, I
can add Lance as a second example so the proposal can be evaluated as a general
convention rather than as a request to merge Hudi specifically.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
