timsaucer opened a new pull request, #18679: URL: https://github.com/apache/datafusion/pull/18679
## Which issue does this PR close? - Addresses part of https://github.com/apache/datafusion/issues/18671 but does not close it. Note: This is based on top of https://github.com/apache/datafusion/pull/18672. I will rebase after that one merges to remove the parts of the code that are not relevant to this PR. ## Rationale for this change This PR solves one of the biggest problems we have in using user defined functions together via FFI. The justification is included in the `README.md` file in this PR. In order to de/serialize any expressions, either logical or physical, we need to know what functions have been registered. This is not possible to do if we cannot access the _current_ list of registered functions. This causes a circular dependency if we want to use registered functions as parts of expressions to other registered functions. The encoding and decoding of these functions will fail. This PR adds a weak reference to a trait that can access the _current_ `TaskContext` to be used for the protobuf de/serialization. ## What changes are included in this PR? - Adds the trait `TaskContextProvider` that is used to get the current task context. - Adds a `FFI_TaskContext` and `FFI_TaskContextProvider` - Plumbs the `FFI_TaskContextProvider` throughout any of the FFI code that requires protobof encoding and decoding. - Updates unit tests to pass around the `SessionContext` as a `TaskContextProvider` ## Are these changes tested? The unit tests have been updated. Additionally this has been tested against `datafusion-python` ## Are there any user-facing changes? This is an API change for FFI. - [ ] TODO(tsaucer) Add text in migration guide about these changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
