JingsongLi opened a new pull request, #11132: URL: https://github.com/apache/paimon/pull/11132
### Purpose Add a read-only Spark 4.2 execution profile for Ossie `0.3.0.dev0` semantic views stored in Paimon REST Catalog, on top of #11127. ```sql SELECT region, MEASURE(revenue) FROM paimon.sales.order_metrics GROUP BY region; ``` The resolver reads definitions through the existing semantic-view management client, validates the pinned Ossie schema, binds dataset fields separately from source columns, and compiles the model into a native Spark Metric View. Spark performs the `MEASURE` rewrite and query execution. Authentication, REST prefix and source authorization use the existing catalog paths. Ordinary tables introduced while expanding SQL views are left for Spark's normal relation resolution. Definitions are read for each new analysis; already-analyzed queries retain their plan. Querying does not persist a converted model or rewrite the original YAML. The schema is pinned to Ossie commit `2692b1cc4708fdcf6a91e597ca928a9bc9d0da60`, which matches the current upstream schema despite the mutable development version. This initial profile supports one physical Paimon dataset, scalar fields, dataset/model metrics and an explicit portable expression subset. Relationships, multiple datasets, joins against a semantic relation, streaming, writes and semantic-view SQL DDL are outside the profile. Unsupported execution features are rejected. The REST server still needs to implement semantic-view storage and authorization. The schema validator and its date/time dependency are shaded into the 4.2 connector. Safe YAML loading preserves scalar types long enough to reject non-JSON extension values, non-string keys and recursive aliases. ### Tests - `mvn -pl paimon-spark/paimon-spark-4.2 -am -Pspark42 -DfailIfNoTests=false -Dtest=none -DwildcardSuites=org.apache.paimon.spark.ossie.OssieModelTest,org.apache.spark.sql.paimon.OssieSemanticViewTest,org.apache.paimon.spark.sql.Spark42CompatibilityTest package`: 29 passed with normal format/checkstyle/enforcer checks enabled. - Repeated all 29 tests with the shaded connector JAR first on the runtime classpath. - 14 Ossie tests cover schema/version validation, JSON-compatible YAML data, field/source shadowing, dataset and model metric binding, DISTINCT/FILTER, null/empty results, division/rounding, alias collisions, namespace binding, definition replacement, metadata preservation, permission/HTTP errors and unsupported operations. - The existing 15 Spark 4.2 compatibility tests cover normal table/view/function and V1/V2 row-level paths. They also caught and guard against semantic lookup intercepting source tables introduced by ordinary view expansion. - Root Apache RAT and `git diff --check` passed. The complete multi-version Spark test suite has not been run for this 4.2-specific change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
