zhztheplayer commented on PR #11009: URL: https://github.com/apache/incubator-gluten/pull/11009#issuecomment-3491550610
> This improvement looks nice. Thanks. > > I have some thoughts based on the superficial observation, not related to this PR's purpose. Please feel free to correct me :) > > It seems that backend is a special case — all components directly or indirectly depend on it (either the Velox backend or the ClickHouse backend for now). In addition, backend itself does not depend on other components. > > If these assumptions always hold (even for future development), perhaps we could consider excluding the backend from component management. Since only one specific backend is selected at build time and component definitions are tied to that backend and isolated from those of other backends, this simplification might make sense. > > With that exclusion: > > * We can focus component management solely on the common components. > * For a component that depends only on the backend, its dependency list can be left empty (given that the backend is always implicitly present). > * For a component A that depends on component B, we simply define A’s dependency on B as usual. > > This approach would allow component management to remain independent of backend logic (and possibly make backend no longer extend component). It could simplify maintenance and make the structure clearer for developers. Thank you for the insightful comments. I would agree that only one backend could be loaded at a time, according to the current cooperation pattern between CH backend and VL backend. And maybe there's a very straightforward way to force a single-backend rule, by adding a simple restriction during the component discovering phase. There might be some nuances between this way and the way you suggested but I don't have strong preference here. Feel free to share detailed ideas as needed. I was also thinking whether we should completely exclude the possibility of having multiple backends loaded together, in the long-term future. For example, we had several new backend proposals filed in the past few months, and this reminded me about some uncertainties on how will them be integrated with Gluten and work with the existing backends. But apparently they will finally be figured out as we move forward on these new directions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
