jerryshao commented on code in PR #5002:
URL: https://github.com/apache/gravitino/pull/5002#discussion_r1805781171
##########
core/src/main/java/org/apache/gravitino/GravitinoEnv.java:
##########
@@ -130,21 +132,30 @@ public static GravitinoEnv getInstance() {
}
/**
- * Initialize the Gravitino environment.
+ * Initialize base components, used for Iceberg REST server.
*
* @param config The configuration object to initialize the environment.
- * @param isGravitinoServer A boolean flag indicating whether the
initialization is for the
- * Gravitino server. If true, server-specific components will be
initialized in addition to
- * the base components.
*/
- public void initialize(Config config, boolean isGravitinoServer) {
- LOG.info("Initializing Gravitino Environment...");
+ public void initializeBaseComponents(Config config) {
+ LOG.info("Initializing Gravitino base environment...");
this.config = config;
+ this.manageAllComponents = false;
initBaseComponents();
- if (isGravitinoServer) {
- initGravitinoServerComponents();
- }
- LOG.info("Gravitino Environment is initialized.");
+ LOG.info("Gravitino base environment is initialized.");
+ }
+
+ /**
+ * Initialize all components, used for Gravitino server.
+ *
+ * @param config The configuration object to initialize the environment.
+ */
+ public void initializeAllComponents(Config config) {
+ LOG.info("Initializing Gravitino all Environment...");
Review Comment:
"full environement.."
##########
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/extension/IcebergEventLogger.java:
##########
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.iceberg.extension;
+
+import java.util.Map;
+import org.apache.gravitino.listener.api.EventListenerPlugin;
+import org.apache.gravitino.listener.api.event.Event;
+import org.apache.gravitino.listener.api.event.PreEvent;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class IcebergEventLogger implements EventListenerPlugin {
Review Comment:
I think we don't have to had a `IcebergEventLogger` here for demo purpose. A
better way is to add audit log support for iceberg rest server when Audit PR is
merged.
##########
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/IcebergRestUtils.java:
##########
@@ -71,4 +80,48 @@ public static Instant calculateNewTimestamp(Instant
currentTimestamp, int hours)
}
return nextHourDateTime.atZone(ZoneId.systemDefault()).toInstant();
}
+
+ public static String getCatalogName(String rawPrefix) {
+ String prefix = shelling(rawPrefix);
+ Preconditions.checkArgument(
+ !IcebergConstants.GRAVITINO_DEFAULT_CATALOG.equals(prefix),
+ String.format("%s is conflict with reserved key, please replace it",
prefix));
+ if (StringUtils.isBlank(prefix)) {
+ return IcebergConstants.GRAVITINO_DEFAULT_CATALOG;
+ }
+ return prefix;
+ }
+
+ public static NameIdentifier getGravitinoNameIdentifier(
+ String catalogName, TableIdentifier icebergIdentifier) {
+ Stream<String> catalogNS =
+ Stream.concat(
+ Stream.of(catalogName),
Arrays.stream(icebergIdentifier.namespace().levels()));
+ String[] catalogNSTable =
+ Stream.concat(catalogNS,
Stream.of(icebergIdentifier.name())).toArray(String[]::new);
+ return NameIdentifier.of(catalogNSTable);
+ }
+
+ // remove the last '/' from the prefix, for example transform
'iceberg_catalog/' to
+ // 'iceberg_catalog'
+ private static String shelling(String rawPrefix) {
Review Comment:
What's the meaning of `shelling`, it is hard to understand, can we use a
better name?
##########
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergTableEventDispatcher.java:
##########
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.iceberg.service.dispatcher;
+
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.iceberg.service.IcebergRestUtils;
+import org.apache.gravitino.listener.EventBus;
+import org.apache.gravitino.listener.api.event.IcebergCreateTableFailureEvent;
+import org.apache.gravitino.listener.api.event.IcebergCreateTablePostEvent;
+import org.apache.gravitino.listener.api.event.IcebergCreateTablePreEvent;
+import org.apache.gravitino.utils.PrincipalUtils;
+import org.apache.iceberg.catalog.Namespace;
+import org.apache.iceberg.catalog.TableIdentifier;
+import org.apache.iceberg.rest.requests.CreateTableRequest;
+import org.apache.iceberg.rest.responses.LoadTableResponse;
+
+/**
+ * {@code IcebergTableEventDispatcher} is a decorator for {@link
IcebergTableOperationProcessor}
+ * that not only delegates table operations to the underlying dispatcher but
also dispatches
+ * corresponding events to an {@link org.apache.gravitino.listener.EventBus}.
+ */
+public class IcebergTableEventDispatcher implements
IcebergTableOperationDispatcher {
+
+ private IcebergTableOperationDispatcher icebergTableOperationDispatcher;
+ private EventBus eventBus;
+
+ public IcebergTableEventDispatcher(
+ IcebergTableOperationDispatcher icebergTableOperationDispatcher,
EventBus eventBus) {
+ this.icebergTableOperationDispatcher = icebergTableOperationDispatcher;
+ this.eventBus = eventBus;
+ }
+
+ @Override
+ public LoadTableResponse createTable(
+ String catalogName, Namespace namespace, CreateTableRequest
createTableRequest) {
+ TableIdentifier tableIdentifier = TableIdentifier.of(namespace,
createTableRequest.name());
+ NameIdentifier nameIdentifier =
+ IcebergRestUtils.getGravitinoNameIdentifier(catalogName,
tableIdentifier);
Review Comment:
1. Do we support multiple level namespace for Iceberg rest server?
2. Do we need to add metalake to the name identifier?
##########
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergTableOperationProcessor.java:
##########
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.iceberg.service.dispatcher;
+
+import org.apache.gravitino.iceberg.service.IcebergCatalogWrapperManager;
+import org.apache.iceberg.catalog.Namespace;
+import org.apache.iceberg.rest.requests.CreateTableRequest;
+import org.apache.iceberg.rest.responses.LoadTableResponse;
+
+/**
+ * The {@code IcebergTableOperationProcessor} locates the corresponding {@code
+ * IcebergCatalogWrapper} based on the prefix in order to perform the actual
table operations.
+ */
+public class IcebergTableOperationProcessor implements
IcebergTableOperationDispatcher {
Review Comment:
Is it better to use `Executor` than `Processor`?
##########
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergTableOperationProcessor.java:
##########
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.iceberg.service.dispatcher;
+
+import org.apache.gravitino.iceberg.service.IcebergCatalogWrapperManager;
+import org.apache.iceberg.catalog.Namespace;
+import org.apache.iceberg.rest.requests.CreateTableRequest;
+import org.apache.iceberg.rest.responses.LoadTableResponse;
+
+/**
+ * The {@code IcebergTableOperationProcessor} locates the corresponding {@code
Review Comment:
What is the meaning of "locate"?
##########
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergTableOperationDispatcher.java:
##########
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.iceberg.service.dispatcher;
+
+import org.apache.iceberg.catalog.Namespace;
+import org.apache.iceberg.rest.requests.CreateTableRequest;
+import org.apache.iceberg.rest.responses.LoadTableResponse;
+
+/**
+ * The {@code IcebergTableOperationDispatcher} interface defines the public
API for managing Iceberg
+ * tables.
+ */
+public interface IcebergTableOperationDispatcher {
+ /**
+ * Creates a new Iceberg table.
+ *
+ * @param catalogName The catalog name when creating the table.
+ * @param namespace The namespace within which the table should be created.
+ * @param createTableRequest The request object containing the details for
creating the table.
+ * @return A {@link LoadTableResponse} object containing the result of the
operation.
+ */
+ LoadTableResponse createTable(
+ String catalogName, Namespace namespace, CreateTableRequest
createTableRequest);
Review Comment:
So you only support one operation for now, right?
##########
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTablePostEvent.java:
##########
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.listener.api.event;
+
+import org.apache.gravitino.NameIdentifier;
+
+/** Represents an abstract table post event in Gravitino Iceberg REST server.
*/
+public abstract class IcebergTablePostEvent extends IcebergRESTPostEvent {
Review Comment:
Also here.
##########
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTableFailureEvent.java:
##########
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.listener.api.event;
+
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.annotation.DeveloperApi;
+
+/** Represent a failure event when do Iceberg table operation failed. */
+@DeveloperApi
+public class IcebergTableFailureEvent extends IcebergRESTFailureEvent {
Review Comment:
Why do we need one `IcebergRESTFailureEvent` and another
`IcebergTableFailureEvent `?
##########
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTablePreEvent.java:
##########
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.listener.api.event;
+
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.annotation.DeveloperApi;
+import org.apache.iceberg.rest.requests.CreateTableRequest;
+
+/** Represent a pre event before creating Iceberg table. */
+@DeveloperApi
+public class IcebergCreateTablePreEvent extends IcebergTablePreEvent {
+ private CreateTableRequest createTableRequest;
+
+ public IcebergCreateTablePreEvent(
+ String user, NameIdentifier resourceIdentifier, CreateTableRequest
createTableRequest) {
+ super(user, resourceIdentifier);
+ this.createTableRequest = createTableRequest;
Review Comment:
Do you need to clone this object here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
