timsaucer opened a new issue, #1181: URL: https://github.com/apache/datafusion-python/issues/1181
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** `PyLogicalPlan` can currently only serialize or deserialize built in functions and table providers. It currently uses `DefaultLogicalExtensionCodec` in [this function](https://github.com/apache/datafusion-python/blob/2e1b71369eefc97c22b82be84bbabb414f748fb9/src/sql/logical.rs#L187). Users would like to be able to serialize and deserialize plans that include custom functions and table providers. One such example is the [Iceberg python integration](https://github.com/apache/iceberg-python/pull/2075). This topic was mentioned in the #datafusion-python channel in Discord. **Describe the solution you'd like** Create a logical extension codec that can process user defined functions and table providers. This likely means some work in the upstream repository. One approach could consist of: - Add `encode` and `decode` methods to the `FFI_TableProvider`, `FFI_ScalarUDF` and so on. - Implement `LogicalExtensionCodec` on `ForeignTableProvider`, `ForeignScalarUDF`, and so on where the only methods implemented are those appropriate to table provide, udf, etc. That is for `ForeignTableProvider` it only supports `try_encode_table_provider` and `try_decode_table_provider`. - Add a `LogicalExtensionCodec` to `datafusion-python`. For the table providers, iterate through all registered table providers and see if any of them successfully decode. The encode method is more straightforward. **Describe alternatives you've considered** The user could output the logical plan as a SQL command and then pass that along as a string. **Additional context** -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org