JingsongLi commented on code in PR #10950:
URL: https://github.com/apache/paimon/pull/10950#discussion_r4443107295
##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/catalyst/analysis/expressions/ExpressionUtils.scala:
##########
@@ -18,5 +18,42 @@
package org.apache.paimon.spark.catalyst.analysis.expressions
+import org.apache.paimon.predicate.Predicate
+import org.apache.paimon.types.RowType
+
+import org.apache.spark.sql.SparkSession
+import org.apache.spark.sql.catalyst.optimizer.{ConstantFolding, SimplifyCasts}
+import org.apache.spark.sql.catalyst.plans.logical.Filter
+import org.apache.spark.sql.catalyst.trees.TreePattern.CURRENT_LIKE
+import org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation
+
/** This wrapper is only used in java code, e.g. Procedure. */
-object ExpressionUtils extends ExpressionHelper
+object ExpressionUtils extends ExpressionHelper {
+
+ def convertPolicyFilter(
+ spark: SparkSession,
+ relation: DataSourceV2Relation,
+ rowType: RowType,
+ filterSql: String): Predicate = {
+ val parsed = spark.sessionState.sqlParser.parseExpression(filterSql)
+ val analyzed = spark.sessionState.analyzer.execute(Filter(parsed,
relation))
+ spark.sessionState.analyzer.checkAnalysis(analyzed)
+ val filter = analyzed.asInstanceOf[Filter]
+ val condition = filter.condition
+ // Query-context values must not be frozen into a persistent authorization
rule.
+ require(
+ !condition.containsPattern(CURRENT_LIKE),
+ "filter_sql cannot contain expressions that depend on the current query
context."
+ )
+
+ val normalized =
SimplifyCasts(ConstantFolding(filter)).asInstanceOf[Filter].condition
+ // A stored policy has no engine-side residual filter, so every conjunct
must convert.
+ convertConditionToPaimonPredicate(
Review Comment:
[P1] Require exact SQL semantics before storing a converted policy
`ignorePartialFailure = false` proves that every expression can be
converted, but does not prove semantic equivalence. I reproduced an
unauthorized row through the actual REST policy/read path on this head: create
`(id INT, v DOUBLE)` with `query-auth.enabled=true`, insert `(1, CAST('-0.0' AS
DOUBLE)), (2, 0.0), (3, 1.0)`, and create a ROW_FILTER with `filter_sql => 'v
<> CAST(0 AS DOUBLE)'`. Spark's evaluation over an independent VALUES relation
returns only id 3; selecting from the policy-protected table returns ids 1 and
3. The generated NotEqual calls CompareUtils/Double.compareTo, which
distinguishes the two signs of zero, whereas Spark SQL considers them equal.
The persisted policy has no Spark residual to remove the extra row. FLOAT has
the same comparison boundary. Please reject conversions without exact policy
semantics or supply an exact conversion (including signed-zero comparisons),
and add a real authorization regression test whose expected result is evaluated
indepen
dently of Paimon pushdown.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
