zhztheplayer opened a new pull request, #8143:
URL: https://github.com/apache/incubator-gluten/pull/8143
This is the API layer implementation of #6920.
The PR introduces a new concept `Component` int Gluten and adjusts the role
of current concept `Backend`.
After the change, `Backend` API will become a special variant of `Component`
API. Details are as below:
1. Dependency
- `Component` should have at least one parent. Each parent can either be
another `Component` or `Backend`.
- `Backend` doesn't have any parents.
2. `Component` and `Backend` instances are used as exactly the same manner
in Gluten. Gluten treats them all as regular components.
3. Having `Component`'s parents defined, Gluten will figure out a fixed
order / priority of using the components. For exmaple:
- A component's parent's loading APIs, namely `onDriverStart` /
`onExecutorStart`, will be guaranteed to call **EARLIER** than the same APIs of
the child.
- A component's parent's rule injection API, namely `injectRules`, will
be guaranteed to call **LATER** than the same API of the child.
In subsequent PRs of #6920, we will:
1. Make iceberg / hudi / delta implement component API, remove the previous
rule injection points used by their code.
2. Make uniffle / celeborn implement component API, remove the previous rule
injection points by used their code.
3. Add an example backend to work with Velox backend.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
