Zouxxyy opened a new pull request, #8299: URL: https://github.com/apache/paimon/pull/8299
### Purpose Support SQL-defined scalar UDFs (`CREATE FUNCTION ... RETURN`) on Paimon catalog for Spark 4. Intercept `CreateUserDefinedFunction` at the parser stage and persist the SQL function in the Paimon catalog as a `SQLFunctionDefinition`; at query time rebuild a Spark `SQLFunctionExpression` so Spark's own `ResolveSQLFunctions` / `EliminateSQLFunctionNode` inline the body. No Spark changes required. Scalar functions with an expression body or a single-column query body are supported; table functions are rejected for now. All Spark-4-only code (`CreateUserDefinedFunction` / `SQLFunction` / `SQLFunctionExpression`) is confined to `paimon-spark4-common` behind `SparkShim`; the spark3 shim is a no-op so `paimon-spark-common` still compiles against Spark 3. Key changes: - **new (spark4-common)**: `SQLFunctionConverter`, `RewritePaimonSQLFunctionCommands`, `CreatePaimonSQLFunctionCommand` - **common**: extract `PaimonFunctionLookup`; recognize `SQLFunctionDefinition` in the function-reference rewrite; route SQL functions through `SparkShim` in `PaimonV1FunctionRegistry`; append shim parser rules - **rename**: `V1FunctionConverter` → `FileFunctionConverter` + `FunctionIdentifierConverter` (shared) ### Tests - `PaimonSQLFunctionTest` (expression body, query body, OR REPLACE / IF NOT EXISTS, qualified names, DROP, table-function rejection) in spark-4.0 and spark-4.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
