JingsongLi commented on code in PR #9265:
URL: https://github.com/apache/paimon/pull/9265#discussion_r4493236544


##########
paimon-spark/paimon-spark4-common/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/CreatePaimonSQLFunctionCommand.scala:
##########
@@ -78,6 +79,7 @@ case class CreatePaimonSQLFunctionCommand(
       exprText,
       queryText,
       comment,
+      None, // collation, added in Spark 4.2

Review Comment:
   [P2] Preserve or reject the declared SQL-function collation
   
   Spark 4.2 accepts a function-level `DEFAULT COLLATION`, but the rewrite 
drops `CreateUserDefinedFunction.collation`, this constructor hard-codes 
`None`, and `SQLFunctionConverter` also reloads with `None`. This silently 
changes query results even for a BOOLEAN function with no string parameters, so 
it does not depend on support for collated Paimon storage types.
   
   I reproduced this against `c34c840d` through public SQL and the Paimon REST 
catalog:
   ```sql
   CREATE FUNCTION paimon.test.f() RETURNS BOOLEAN
   DEFAULT COLLATION UTF8_LCASE RETURN 'a' = 'A';
   SELECT paimon.test.f();
   ```
   The declaration is accepted but returns `false` both immediately and in a 
fresh Spark session. The identical native TEMPORARY SQL function, and `SELECT 
'a' COLLATE UTF8_LCASE = 'A'`, return `true`; the native parser confirms the 
declaration carries `UTF8_LCASE`. The persisted Paimon function has no 
collation option.
   
   A diagnostic control carrying the declared value through command creation, 
persisted properties and reload makes both Paimon reads return `true`. Please 
preserve that value across these boundaries, or explicitly reject the 
unsupported clause rather than executing different semantics. A production fix 
must also retain the 4.0/4.1 shadow-class ABI; the diagnostic carry patch was 
confined to 4.2 and removed after verification.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to