jayzhan211 commented on issue #10074: URL: https://github.com/apache/datafusion/issues/10074#issuecomment-2089325105
@alamb First of all, we plan to pull the aggregate function out from `datafusion-physical-expr`. And, we also split `datafusion-physical-common` from `datafusion-physical`, **so the user can only import the minimum things they need**, but this is not enough. We want to handle `physical-expr` in UDFImpl, which is not possible given the current crate graph, because `UDFImpl` should be in `datafusion-expr` and we can't import `datafusion-physical-expr` to `datafusion-expr`. But, what if we allow `datafusion-expr` to import `physical-expr-common` 🤔 We now differentiate **common** as the higher level crate. `datafusion-common` is the highest. Introduce `expr-common` as the second, `datafusion-physical-common` as the third (import common and expr-common), and `aggregate-common` as the fifth, that import all other **common**. Allow `datafusion-expr` to import `datafusion-physical-common`, so we can handle `physical-expr` concept in UDFImpl. As long as we don't introduce things that depend on `datafusion-expr` into `physical-expr-common`, I think we don't need to strictly avoid any physical-expr concept things ported into `datafusion-expr`. > Can you remind me why datafusion-functions-aggregate (the implementation of aggregate functions) shouldn't depend on datafusion-physical-expr and datafusion-expr `datafusion-functions-aggregate` can import `datafusion-expr` and `datafusion-physical-expr-common` but not `datafusion-physical-expr` to keep the minimum dependencies for the user. > I thought the thing we are trying to do is avoid datafusion-expr depending on datafusion-physical-expr This idea is what blocking us! Now, I would allow `datafusion-physical-expr-common` dependency for `datafusion-expr` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
