mchades commented on code in PR #12565:
URL: https://github.com/apache/gravitino/pull/12565#discussion_r3853790909
##########
core/src/main/java/org/apache/gravitino/GravitinoEnv.java:
##########
@@ -869,6 +883,15 @@ private void initGravitinoServerComponents() {
new ViewEventDispatcher(eventBus, viewNormalizeDispatcher);
this.viewDispatcher = viewEventDispatcher;
+ // Semantic Model operation chain: SemanticModelNormalizeDispatcher ->
+ // SemanticModelOperationDispatcher -> ManagedSemanticModelOperations.
+ // TODO: Add event and hook layers with Semantic Model server integration.
+ SemanticModelOperationDispatcher semanticModelOperationDispatcher =
+ new SemanticModelOperationDispatcher(
+ catalogManager, internalSchemaDispatcher, entityStore,
idGenerator, secretManager);
Review Comment:
Good catch. `SemanticModelOperationDispatcher` now receives the raw
`schemaOperationDispatcher`, so parent schema normalization happens only once
in `SemanticModelNormalizeDispatcher`.
##########
core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java:
##########
@@ -157,7 +157,8 @@ public static Namespace applyCaseSensitive(
|| identScope == Capability.Scope.FILESET
|| identScope == Capability.Scope.TOPIC
|| identScope == Capability.Scope.MODEL
- || identScope == Capability.Scope.FUNCTION) {
+ || identScope == Capability.Scope.FUNCTION
+ || identScope == Capability.Scope.SEMANTIC_MODEL) {
Review Comment:
Good catch. I centralized the classification in `hasSchemaParent`, reused it
from both namespace normalization paths, included `MODEL`, and added regression
coverage for both `MODEL` paths.
##########
core/src/main/java/org/apache/gravitino/catalog/SemanticModelOperationDispatcher.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.catalog;
+
+import com.google.common.base.Preconditions;
+import java.util.Arrays;
+import java.util.Map;
+import javax.annotation.Nullable;
+import org.apache.gravitino.Catalog;
+import org.apache.gravitino.EntityStore;
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.Namespace;
+import org.apache.gravitino.exceptions.IllegalSemanticModelException;
+import org.apache.gravitino.exceptions.NoSuchSchemaException;
+import org.apache.gravitino.exceptions.NoSuchSemanticModelException;
+import org.apache.gravitino.exceptions.SemanticModelAlreadyExistsException;
+import org.apache.gravitino.lock.LockType;
+import org.apache.gravitino.lock.TreeLockUtils;
+import org.apache.gravitino.secret.SecretManager;
+import org.apache.gravitino.semantic.SemanticModel;
+import org.apache.gravitino.semantic.SemanticModelChange;
+import org.apache.gravitino.semantic.SemanticModelDefinition;
+import org.apache.gravitino.storage.IdGenerator;
+
+/** Dispatches always-managed Semantic Model operations to Gravitino's
EntityStore. */
+public class SemanticModelOperationDispatcher extends OperationDispatcher
+ implements SemanticModelDispatcher {
+
+ private final CatalogManager catalogManager;
+ private final SchemaDispatcher schemaDispatcher;
+ private final ManagedSemanticModelOperations managedOperations;
+
+ /**
+ * Creates a Semantic Model operation dispatcher.
+ *
+ * @param catalogManager The catalog manager.
+ * @param schemaDispatcher The internal schema dispatcher used for parent
validation.
+ * @param store The EntityStore used for Semantic Model persistence.
+ * @param idGenerator The stable entity ID generator.
+ * @param secretManager The secret manager required by the operation
dispatcher base class.
+ */
+ public SemanticModelOperationDispatcher(
+ CatalogManager catalogManager,
+ SchemaDispatcher schemaDispatcher,
+ EntityStore store,
+ IdGenerator idGenerator,
+ SecretManager secretManager) {
+ super(catalogManager, store, idGenerator, secretManager);
+ this.catalogManager = catalogManager;
+ this.schemaDispatcher = schemaDispatcher;
+ this.managedOperations = new ManagedSemanticModelOperations(store,
idGenerator);
+ }
+
+ @Override
+ public NameIdentifier[] listSemanticModels(Namespace namespace) throws
NoSuchSchemaException {
+ checkRelationalCatalog(namespace);
+ NameIdentifier schemaIdent = NameIdentifier.of(namespace.levels());
+ schemaDispatcher.loadSchema(schemaIdent);
+ return TreeLockUtils.doWithTreeLock(
+ schemaIdent, LockType.READ, () ->
managedOperations.listSemanticModels(namespace));
+ }
+
+ @Override
+ public SemanticModel loadSemanticModel(NameIdentifier ident) throws
NoSuchSemanticModelException {
+ checkRelationalCatalog(ident.namespace());
+ if (!schemaDispatcher.schemaExists(schemaIdentifier(ident))) {
Review Comment:
With the `TreeLock` wrappers removed, the specific check-before-lock race no
longer applies. Parent-schema consistency remains a database concern and will
be enforced transactionally in the follow-up persistence implementation; this
framework PR has no functional persistence path.
##########
core/src/main/java/org/apache/gravitino/GravitinoEnv.java:
##########
@@ -869,6 +883,15 @@ private void initGravitinoServerComponents() {
new ViewEventDispatcher(eventBus, viewNormalizeDispatcher);
this.viewDispatcher = viewEventDispatcher;
+ // Semantic Model operation chain: SemanticModelNormalizeDispatcher ->
+ // SemanticModelOperationDispatcher -> ManagedSemanticModelOperations.
+ // TODO: Add event and hook layers with Semantic Model server integration.
Review Comment:
Added separate tracking references: #12595 for Semantic Model event and
audit support, and #12594 for ownership and privilege hooks.
##########
core/src/main/java/org/apache/gravitino/catalog/SemanticModelOperationDispatcher.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.catalog;
+
+import com.google.common.base.Preconditions;
+import java.util.Arrays;
+import java.util.Map;
+import javax.annotation.Nullable;
+import org.apache.gravitino.Catalog;
+import org.apache.gravitino.EntityStore;
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.Namespace;
+import org.apache.gravitino.exceptions.IllegalSemanticModelException;
+import org.apache.gravitino.exceptions.NoSuchSchemaException;
+import org.apache.gravitino.exceptions.NoSuchSemanticModelException;
+import org.apache.gravitino.exceptions.SemanticModelAlreadyExistsException;
+import org.apache.gravitino.lock.LockType;
+import org.apache.gravitino.lock.TreeLockUtils;
+import org.apache.gravitino.secret.SecretManager;
+import org.apache.gravitino.semantic.SemanticModel;
+import org.apache.gravitino.semantic.SemanticModelChange;
+import org.apache.gravitino.semantic.SemanticModelDefinition;
+import org.apache.gravitino.storage.IdGenerator;
+
+/** Dispatches always-managed Semantic Model operations to Gravitino's
EntityStore. */
+public class SemanticModelOperationDispatcher extends OperationDispatcher
+ implements SemanticModelDispatcher {
+
+ private final CatalogManager catalogManager;
+ private final SchemaDispatcher schemaDispatcher;
+ private final ManagedSemanticModelOperations managedOperations;
+
+ /**
+ * Creates a Semantic Model operation dispatcher.
+ *
+ * @param catalogManager The catalog manager.
+ * @param schemaDispatcher The internal schema dispatcher used for parent
validation.
+ * @param store The EntityStore used for Semantic Model persistence.
+ * @param idGenerator The stable entity ID generator.
+ * @param secretManager The secret manager required by the operation
dispatcher base class.
+ */
+ public SemanticModelOperationDispatcher(
+ CatalogManager catalogManager,
+ SchemaDispatcher schemaDispatcher,
+ EntityStore store,
+ IdGenerator idGenerator,
+ SecretManager secretManager) {
+ super(catalogManager, store, idGenerator, secretManager);
+ this.catalogManager = catalogManager;
+ this.schemaDispatcher = schemaDispatcher;
+ this.managedOperations = new ManagedSemanticModelOperations(store,
idGenerator);
+ }
+
+ @Override
+ public NameIdentifier[] listSemanticModels(Namespace namespace) throws
NoSuchSchemaException {
+ checkRelationalCatalog(namespace);
+ NameIdentifier schemaIdent = NameIdentifier.of(namespace.levels());
+ schemaDispatcher.loadSchema(schemaIdent);
+ return TreeLockUtils.doWithTreeLock(
+ schemaIdent, LockType.READ, () ->
managedOperations.listSemanticModels(namespace));
+ }
+
+ @Override
+ public SemanticModel loadSemanticModel(NameIdentifier ident) throws
NoSuchSemanticModelException {
+ checkRelationalCatalog(ident.namespace());
+ if (!schemaDispatcher.schemaExists(schemaIdentifier(ident))) {
+ throw new NoSuchSemanticModelException("Semantic Model %s does not
exist", ident);
+ }
+ return TreeLockUtils.doWithTreeLock(
+ ident, LockType.READ, () ->
managedOperations.loadSemanticModel(ident));
+ }
+
+ @Override
+ public SemanticModel createSemanticModel(
+ NameIdentifier ident,
+ @Nullable String comment,
+ SemanticModelDefinition definition,
+ Map<String, String> properties)
+ throws NoSuchSchemaException, SemanticModelAlreadyExistsException,
+ IllegalSemanticModelException {
+ Preconditions.checkArgument(definition != null, "Definition must not be
null");
+ Preconditions.checkArgument(properties != null, "Properties must not be
null");
+ checkRelationalCatalog(ident.namespace());
+ NameIdentifier schemaIdent = schemaIdentifier(ident);
+ schemaDispatcher.loadSchema(schemaIdent);
+ return TreeLockUtils.doWithTreeLock(
+ schemaIdent,
+ LockType.WRITE,
+ () -> managedOperations.createSemanticModel(ident, comment,
definition, properties));
+ }
+
+ @Override
+ public SemanticModel alterSemanticModel(NameIdentifier ident,
SemanticModelChange... changes)
+ throws NoSuchSemanticModelException, SemanticModelAlreadyExistsException,
+ IllegalSemanticModelException {
+ Preconditions.checkArgument(
Review Comment:
Agreed. I removed the duplicate `changes` validation from
`SemanticModelOperationDispatcher`; `SemanticModelNormalizeDispatcher` remains
the single production validation point.
##########
core/src/main/java/org/apache/gravitino/catalog/SemanticModelOperationDispatcher.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.catalog;
+
+import com.google.common.base.Preconditions;
+import java.util.Arrays;
+import java.util.Map;
+import javax.annotation.Nullable;
+import org.apache.gravitino.Catalog;
+import org.apache.gravitino.EntityStore;
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.Namespace;
+import org.apache.gravitino.exceptions.IllegalSemanticModelException;
+import org.apache.gravitino.exceptions.NoSuchSchemaException;
+import org.apache.gravitino.exceptions.NoSuchSemanticModelException;
+import org.apache.gravitino.exceptions.SemanticModelAlreadyExistsException;
+import org.apache.gravitino.lock.LockType;
+import org.apache.gravitino.lock.TreeLockUtils;
+import org.apache.gravitino.secret.SecretManager;
+import org.apache.gravitino.semantic.SemanticModel;
+import org.apache.gravitino.semantic.SemanticModelChange;
+import org.apache.gravitino.semantic.SemanticModelDefinition;
+import org.apache.gravitino.storage.IdGenerator;
+
+/** Dispatches always-managed Semantic Model operations to Gravitino's
EntityStore. */
+public class SemanticModelOperationDispatcher extends OperationDispatcher
+ implements SemanticModelDispatcher {
+
+ private final CatalogManager catalogManager;
+ private final SchemaDispatcher schemaDispatcher;
+ private final ManagedSemanticModelOperations managedOperations;
+
+ /**
+ * Creates a Semantic Model operation dispatcher.
+ *
+ * @param catalogManager The catalog manager.
+ * @param schemaDispatcher The internal schema dispatcher used for parent
validation.
+ * @param store The EntityStore used for Semantic Model persistence.
+ * @param idGenerator The stable entity ID generator.
+ * @param secretManager The secret manager required by the operation
dispatcher base class.
+ */
+ public SemanticModelOperationDispatcher(
+ CatalogManager catalogManager,
+ SchemaDispatcher schemaDispatcher,
+ EntityStore store,
+ IdGenerator idGenerator,
+ SecretManager secretManager) {
+ super(catalogManager, store, idGenerator, secretManager);
+ this.catalogManager = catalogManager;
+ this.schemaDispatcher = schemaDispatcher;
+ this.managedOperations = new ManagedSemanticModelOperations(store,
idGenerator);
+ }
+
+ @Override
+ public NameIdentifier[] listSemanticModels(Namespace namespace) throws
NoSuchSchemaException {
+ checkRelationalCatalog(namespace);
+ NameIdentifier schemaIdent = NameIdentifier.of(namespace.levels());
+ schemaDispatcher.loadSchema(schemaIdent);
+ return TreeLockUtils.doWithTreeLock(
Review Comment:
Agreed. I removed all `TreeLock` wrappers from
`SemanticModelOperationDispatcher`; the persistence layer will rely on database
transactions and concurrency controls instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
