alamb commented on issue #1544:
URL:
https://github.com/apache/arrow-datafusion/issues/1544#issuecomment-1009919116
@hntd187 -- I think the proposal sounds good.
I think you were asking about mechanics here:
> we will at least have to develop the API inside datafusion for now it
would seem unless we can do it all via contrib. I'm not totally aware of all
the package scoping that might prevent it from existing in contrib
Here is what I suggest:
1. Create your contrib repo
2. in `Crates.toml` link to a `datafusion` git repo rather than the version
published on crates.io . For example:
```toml
datafusion = { git = "https://github.com/apache/arrow-datafusion.git";,
rev="401271377cd84dc1546827f66bda1b242860a6a8", default-features = false,
package = "datafusion" }
```
Protip: while developing you can change this to point at your own local
checkout like this:
```toml
[patch.crates-io]
datafusion = { path='/path/to/you/local/arrow-datafusion/checkout' }
```
Then as you work on the streaming implementation, add whatever new
DataFusion API / changes you need to a branch (perhaps against your own
datafusion fork)
Once you have settled on an API, then open a PR to the main DataFusion repo
to incorporate your new API
This will let you iterate on the new DataFusion API without having to worry
about the test suite, reviews, delays in merging, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
