This is an automated email from the ASF dual-hosted git repository.
jshao pushed a commit to branch branch-0.7
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/branch-0.7 by this push:
new 015d482c6 [#5217] docs(metalake-catalog): refine docs for deleting
metalake and catalog (#5266)
015d482c6 is described below
commit 015d482c6cd444a7c1581cf72ef0a976fe3e7cf1
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Fri Oct 25 15:53:54 2024 +0800
[#5217] docs(metalake-catalog): refine docs for deleting metalake and
catalog (#5266)
### What changes were proposed in this pull request?
refine docs for deleting metalake and catalog
### Why are the changes needed?
Fix: #5217
### Does this PR introduce _any_ user-facing change?
yes, user docs changed
### How was this patch tested?
setup the gravitino-site locally
Co-authored-by: mchades <[email protected]>
---
docs/manage-fileset-metadata-using-gravitino.md | 2 +-
docs/manage-messaging-metadata-using-gravitino.md | 4 +-
docs/manage-metalake-using-gravitino.md | 111 ++++++++++++-
docs/manage-relational-metadata-using-gravitino.md | 179 +++++++++++++++++++--
docs/open-api/catalogs.yaml | 48 ++++++
docs/open-api/metalakes.yaml | 46 ++++++
docs/open-api/openapi.yaml | 28 ++++
7 files changed, 398 insertions(+), 20 deletions(-)
diff --git a/docs/manage-fileset-metadata-using-gravitino.md
b/docs/manage-fileset-metadata-using-gravitino.md
index 22b12f5b1..de478efb7 100644
--- a/docs/manage-fileset-metadata-using-gravitino.md
+++ b/docs/manage-fileset-metadata-using-gravitino.md
@@ -25,7 +25,7 @@ control mechanism without needing to set access controls
across different storag
To use fileset, please make sure that:
- Gravitino server has started, and the host and port is
[http://localhost:8090](http://localhost:8090).
- - A metalake has been created.
+ - A metalake has been created and
[enabled](./manage-metalake-using-gravitino.md#enable-a-metalake)
## Catalog operations
diff --git a/docs/manage-messaging-metadata-using-gravitino.md
b/docs/manage-messaging-metadata-using-gravitino.md
index 9fc5d6adb..953b9989c 100644
--- a/docs/manage-messaging-metadata-using-gravitino.md
+++ b/docs/manage-messaging-metadata-using-gravitino.md
@@ -16,7 +16,7 @@ Through Gravitino, you can create, update, delete, and list
topics via unified R
To use messaging catalog, please make sure that:
- Gravitino server has started, and the host and port is
[http://localhost:8090](http://localhost:8090).
- - A metalake has been created.
+ - A metalake has been created and
[enabled](./manage-metalake-using-gravitino.md#enable-a-metalake).
## Catalog operations
@@ -138,7 +138,7 @@ in relational catalog for more details. For a messaging
catalog, the list operat
## Topic operations
:::tip
-Users should create a metalake, a catalog and a schema before creating a table.
+Users should create a metalake, a catalog and a schema, then ensure that the
metalake and catalog are enabled before operating topics.
:::
### Create a topic
diff --git a/docs/manage-metalake-using-gravitino.md
b/docs/manage-metalake-using-gravitino.md
index 0aa464fe9..1e15cd731 100644
--- a/docs/manage-metalake-using-gravitino.md
+++ b/docs/manage-metalake-using-gravitino.md
@@ -151,9 +151,108 @@ The following table outlines the supported modifications
that you can make to a
| Set property |
`{"@type":"setProperty","property":"key1","value":"value1"}` |
`MetalakeChange.setProperty("key1", "value1")` |
`MetalakeChange.set_property("key1", "value1")` |
| Remove property | `{"@type":"removeProperty","property":"key1"}`
| `MetalakeChange.removeProperty("key1")` |
`MetalakeChange.remove_property("key1")` |
+## Enable a metalake
+
+Metalake has a reserved property - `in-use`, which indicates whether the
metalake is available for use. By default, the `in-use` property is set to
`true`.
+To enable a disabled metalake, you can send a `PATCH` request to the
`/api/metalakes/{metalake_name}` endpoint or use the Gravitino Admin client.
+
+The following is an example of enabling a metalake:
+
+<Tabs groupId="language" queryString>
+<TabItem value="shell" label="Shell">
+
+```shell
+curl -X PATCH -H "Accept: application/vnd.gravitino.v1+json" \
+-H "Content-Type: application/json" -d '{"in-use": true}' \
+http://localhost:8090/api/metalakes/metalake
+```
+
+</TabItem>
+<TabItem value="java" label="Java">
+
+```java
+GravitinoAdminClient gravitinoAdminClient = GravitinoAdminClient
+ .builder("http://localhost:8090";)
+ .build();
+
+gravitinoAdminClient.enableMetalake("metalake");
+ // ...
+```
+
+</TabItem>
+<TabItem value="python" label="Python">
+
+```python
+gravitino_admin_client: GravitinoAdminClient =
GravitinoAdminClient(uri="http://localhost:8090";)
+gravitino_admin_client.enable_metalake("metalake")
+```
+
+</TabItem>
+</Tabs>
+
+:::info
+This operation does nothing if the metalake is already enabled.
+:::
+
+## Disable a metalake
+
+Once a metalake is disabled:
+ - Users can only [list](#list-all-metalakes), [load](#load-a-metalake),
[drop](#drop-a-metalake), or [enable](#enable-a-metalake) it.
+ - Any other operation on the metalake or its sub-entities will result in an
error.
+
+To disable a metalake, you can send a `PATCH` request to the
`/api/metalakes/{metalake_name}` endpoint or use the Gravitino Admin client.
+
+The following is an example of disabling a metalake:
+
+<Tabs groupId="language" queryString>
+<TabItem value="shell" label="Shell">
+
+```shell
+curl -X PATCH -H "Accept: application/vnd.gravitino.v1+json" \
+-H "Content-Type: application/json" -d '{"in-use": false}' \
+http://localhost:8090/api/metalakes/metalake
+```
+
+</TabItem>
+<TabItem value="java" label="Java">
+
+```java
+GravitinoAdminClient gravitinoAdminClient = GravitinoAdminClient
+ .builder("http://localhost:8090";)
+ .build();
+
+gravitinoAdminClient.disableMetalake("metalake");
+ // ...
+```
+
+</TabItem>
+<TabItem value="python" label="Python">
+
+```python
+gravitino_admin_client: GravitinoAdminClient =
GravitinoAdminClient(uri="http://localhost:8090";)
+gravitino_admin_client.disable_metalake("metalake")
+```
+
+</TabItem>
+</Tabs>
+
+:::info
+This operation does nothing if the metalake is already disabled.
+:::
## Drop a metalake
+Deleting a metalake by "force" is not a default behavior, so please make sure:
+
+- There are no catalogs under the metalake. Otherwise, you will get an error.
+- The metalake is [disabled](#disable-a-metalake). Otherwise, you will get an
error.
+
+Deleting a metalake by "force" will:
+
+- Delete all sub-entities (tags, catalogs, schemas, etc.) under the metalake.
+- Delete the metalake itself even if it is enabled.
+- Not delete the external resources (such as database, table, etc.) associated
with sub-entities unless they are managed (such as managed fileset).
+
To drop a metalake, you can send a `DELETE` request to the
`/api/metalakes/{metalake_name}` endpoint or use the Gravitino Admin client.
The following is an example of dropping a metalake:
@@ -163,7 +262,7 @@ The following is an example of dropping a metalake:
```shell
curl -X DELETE -H "Accept: application/vnd.gravitino.v1+json" \
--H "Content-Type: application/json"
http://localhost:8090/api/metalakes/metalake
+-H "Content-Type: application/json"
http://localhost:8090/api/metalakes/metalake?force=false
```
</TabItem>
@@ -171,7 +270,8 @@ curl -X DELETE -H "Accept:
application/vnd.gravitino.v1+json" \
```java
// ...
-boolean success = gravitinoAdminClient.dropMetalake("metalake");
+// force can be true or false
+boolean success = gravitinoAdminClient.dropMetalake("metalake", false);
// ...
```
@@ -179,17 +279,12 @@ boolean success =
gravitinoAdminClient.dropMetalake("metalake");
<TabItem value="python" label="Python">
```python
-gravitino_admin_client.drop_metalake("metalake")
+gravitino_admin_client.drop_metalake("metalake", force=True)
```
</TabItem>
</Tabs>
-:::note
-Dropping a metalake in cascade mode is not allowed. That is, all the
-catalogs, schemas, and tables under a metalake must be removed before you can
drop the metalake.
-:::
-
## List all metalakes
To view all your metalakes, you can send a `GET` request to the
`/api/metalakes` endpoint or use the Gravitino Admin client.
diff --git a/docs/manage-relational-metadata-using-gravitino.md
b/docs/manage-relational-metadata-using-gravitino.md
index f810b4aa3..c4fffaea8 100644
--- a/docs/manage-relational-metadata-using-gravitino.md
+++ b/docs/manage-relational-metadata-using-gravitino.md
@@ -29,7 +29,7 @@ For more details, please refer to the related doc.
Assuming:
- Gravitino has just started, and the host and port is
[http://localhost:8090](http://localhost:8090).
- - Metalake has been created.
+ - A metalake has been created and
[enabled](./manage-metalake-using-gravitino.md#enable-a-metalake).
## Catalog operations
@@ -84,6 +84,19 @@ Catalog catalog = gravitinoClient.createCatalog("catalog",
// ...
```
+</TabItem>
+<TabItem value="python" label="Python">
+
+```python
+# Assuming you have just created a metalake named `metalake`
+gravitino_client = GravitinoClient(uri="http://localhost:8090";,
metalake_name="metalake")
+gravitino_client.create_catalog(name="catalog",
+ catalog_type=CatalogType.RELATIONAL,
+ provider="hive",
+ comment="This is a hive catalog",
+ properties={"metastore.uris":
"thrift://localhost:9083"})
+```
+
</TabItem>
</Tabs>
@@ -121,6 +134,15 @@ Catalog catalog = gravitinoClient.loadCatalog("catalog");
// ...
```
+</TabItem>
+<TabItem value="python" label="Python">
+
+```python
+# ...
+# Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
+catalog = gravitino_client.load_catalog("catalog")
+```
+
</TabItem>
</Tabs>
@@ -159,6 +181,17 @@ Catalog catalog = gravitinoClient.alterCatalog("catalog",
// ...
```
+</TabItem>
+<TabItem value="python" label="Python">
+
+```python
+# ...
+# Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
+changes = (CatalogChange.update_comment("new comment"))
+catalog = gravitino_client.alterCatalog("catalog", *changes)
+# ...
+```
+
</TabItem>
</Tabs>
@@ -180,8 +213,108 @@ Therefore, do not change the catalog's URI unless you
fully understand the conse
:::
+### Enable a catalog
+
+Catalog has a reserved property - `in-use`, which indicates whether the
catalog is available for use. By default, the `in-use` property is set to
`true`.
+To enable a disabled catalog, you can send a `PATCH` request to the
`/api/metalakes/{metalake_name}/catalogs/{catalog_name}` endpoint or use the
Gravitino Java client.
+
+The following is an example of enabling a catalog:
+
+<Tabs groupId='language' queryString>
+<TabItem value="shell" label="Shell">
+
+```shell
+curl -X PATCH -H "Accept: application/vnd.gravitino.v1+json" \
+-H "Content-Type: application/json" -d '{"in-use": true}' \
+http://localhost:8090/api/metalakes/metalake/catalogs/catalog
+```
+
+</TabItem>
+<TabItem value="java" label="Java">
+
+```java
+// ...
+// Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
+gravitinoClient.enableCatalog("catalog");
+// ...
+```
+
+</TabItem>
+<TabItem value="python" label="Python">
+
+```python
+# ...
+# Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
+gravitino_client.enable_catalog("catalog")
+# ...
+```
+
+</TabItem>
+</Tabs>
+
+:::info
+This operation does nothing if the catalog is already enabled.
+:::
+
+### Disable a catalog
+
+Once a catalog is disabled:
+- Users can only [list](#list-all-catalogs-in-a-metalake),
[load](#load-a-catalog), [drop](#drop-a-catalog), or
[enable](#enable-a-catalog) it.
+- Any other operation on the catalog or its sub-entities will result in an
error.
+
+To disable a catalog, you can send a `PATCH` request to the
`/api/metalakes/{metalake_name}/catalogs/{catalog_name}` endpoint or use the
Gravitino Java client.
+
+The following is an example of disabling a catalog:
+
+<Tabs groupId='language' queryString>
+<TabItem value="shell" label="Shell">
+
+```shell
+curl -X PATCH -H "Accept: application/vnd.gravitino.v1+json" \
+-H "Content-Type: application/json" -d '{"in-use": false}' \
+http://localhost:8090/api/metalakes/metalake/catalogs/catalog
+```
+
+</TabItem>
+<TabItem value="java" label="Java">
+
+```java
+// ...
+// Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
+gravitinoClient.disableCatalog("catalog");
+// ...
+```
+
+</TabItem>
+<TabItem value="python" label="Python">
+
+```python
+# ...
+# Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
+gravitino_client.disable_catalog("catalog")
+# ...
+```
+
+</TabItem>
+</Tabs>
+
+:::info
+This operation does nothing if the catalog is already disabled.
+:::
+
### Drop a catalog
+Deleting a catalog by "force" is not a default behavior, so please make sure:
+
+- There are no schemas under the catalog. Otherwise, you will get an error.
+- The catalog is [disabled](#disable-a-catalog). Otherwise, you will get an
error.
+
+Deleting a catalog by "force" will:
+
+- Delete all sub-entities (schemas, tables, etc.) under the catalog.
+- Delete the catalog itself even if it is enabled.
+- Not delete the external resources (such as database, table, etc.) associated
with sub-entities unless they are managed (such as managed fileset).
+
You can remove a catalog by sending a `DELETE` request to the
`/api/metalakes/{metalake_name}/catalogs/{catalog_name}` endpoint or just use
the Gravitino Java client. The following is an example of dropping a catalog:
<Tabs groupId="language" queryString>
@@ -190,7 +323,7 @@ You can remove a catalog by sending a `DELETE` request to
the `/api/metalakes/{m
```shell
curl -X DELETE -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" \
-http://localhost:8090/api/metalakes/metalake/catalogs/catalog
+http://localhost:8090/api/metalakes/metalake/catalogs/catalog?force=false
```
</TabItem>
@@ -199,17 +332,25 @@
http://localhost:8090/api/metalakes/metalake/catalogs/catalog
```java
// ...
// Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
-gravitinoClient.dropCatalog("catalog");
+// force can be true or false
+gravitinoClient.dropCatalog("catalog", false);
// ...
```
</TabItem>
-</Tabs>
+<TabItem value="python" label="Python">
-:::note
-Dropping a catalog only removes metadata about the catalog, schemas, and
tables under the catalog in Gravitino, It doesn't remove the real data (table
and schema) in Apache Hive.
-:::
+```python
+# ...
+# Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
+# force can be true or false
+gravitino_client.drop_catalog(name="catalog", force=False)
+# ...
+```
+
+</TabItem>
+</Tabs>
### List all catalogs in a metalake
@@ -235,6 +376,16 @@ String[] catalogNames = gravitinoClient.listCatalogs();
// ...
```
+</TabItem>
+<TabItem value="python" label="Python">
+
+```python
+# ...
+# Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
+catalog_names = gravitino_client.list_catalogs()
+# ...
+```
+
</TabItem>
</Tabs>
@@ -261,6 +412,16 @@ Catalog[] catalogsInfos =
gravitinoMetaLake.listCatalogsInfo();
// ...
```
+</TabItem>
+<TabItem value="python" label="Python">
+
+```python
+# ...
+# Assuming you have created a metalake named `metalake` and a catalog named
`catalog`
+catalogs_info = gravitino_client.list_catalogs_info()
+# ...
+```
+
</TabItem>
</Tabs>
@@ -268,7 +429,7 @@ Catalog[] catalogsInfos =
gravitinoMetaLake.listCatalogsInfo();
## Schema operations
:::tip
-Users should create a metalake and a catalog before creating a schema.
+Users should create a metalake and a catalog, then ensure that the metalake
and catalog are enabled before operating schemas.
:::
### Create a schema
@@ -518,7 +679,7 @@ schema_list: List[NameIdentifier] =
catalog.as_schemas().list_schemas()
## Table operations
:::tip
-Users should create a metalake, a catalog and a schema before creating a table.
+Users should create a metalake, a catalog and a schema, then ensure that the
metalake and catalog are enabled before before operating tables.
:::
### Create a table
diff --git a/docs/open-api/catalogs.yaml b/docs/open-api/catalogs.yaml
index 828b74d2d..8e48a7a39 100644
--- a/docs/open-api/catalogs.yaml
+++ b/docs/open-api/catalogs.yaml
@@ -201,11 +201,47 @@ paths:
"5xx":
$ref: "./openapi.yaml#/components/responses/ServerErrorResponse"
+ patch:
+ tags:
+ - catalog
+ summary: set catalog in-use
+ operationId: setCatalog
+ description: Set a specific catalog in-use or not in-use
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: "#/components/schemas/CatalogSetRequest"
+ responses:
+ "200":
+ $ref: "./openapi.yaml#/components/responses/BaseResponse"
+ "404":
+ description: Not Found - The metalake does not exist
+ content:
+ application/vnd.gravitino.v1+json:
+ schema:
+ $ref: "./openapi.yaml#/components/schemas/ErrorModel"
+ examples:
+ NoSuchMetalakeException:
+ $ref: "#/components/examples/NoSuchCatalogException"
+ "500":
+ description:
+ Internal server error. It is possible that the server
+ encountered a storage issue.
+ content:
+ application/vnd.gravitino.v1+json:
+ schema:
+ $ref: "./openapi.yaml#/components/schemas/ErrorModel"
+
+
+
delete:
tags:
- catalog
summary: Drop catalog
operationId: dropCatalog
+ parameters:
+ - $ref: "./openapi.yaml#/components/parameters/force"
responses:
"200":
$ref: "./openapi.yaml#/components/responses/DropResponse"
@@ -339,6 +375,18 @@ components:
additionalProperties:
type: string
+ CatalogSetRequest:
+ type: object
+ required:
+ - inUse
+ properties:
+ inUse:
+ type: boolean
+ description: The in-use status of the catalog to set
+ example: {
+ "inUse": true
+ }
+
CatalogUpdatesRequest:
type: object
required:
diff --git a/docs/open-api/metalakes.yaml b/docs/open-api/metalakes.yaml
index 78739a453..80e40e184 100644
--- a/docs/open-api/metalakes.yaml
+++ b/docs/open-api/metalakes.yaml
@@ -135,12 +135,46 @@ paths:
schema:
$ref: "./openapi.yaml#/components/schemas/ErrorModel"
+ patch:
+ tags:
+ - metalake
+ summary: set metalake in-use
+ operationId: setMetalake
+ description: Set a specified metalake in-use or not in-use
+ requestBody:
+ content:
+ application/json:
+ schema:
+ $ref: "#/components/schemas/MetalakeSetRequest"
+ responses:
+ "200":
+ $ref: "./openapi.yaml#/components/responses/BaseResponse"
+ "404":
+ description: Not Found - The metalake does not exist
+ content:
+ application/vnd.gravitino.v1+json:
+ schema:
+ $ref: "./openapi.yaml#/components/schemas/ErrorModel"
+ examples:
+ NoSuchMetalakeException:
+ $ref: "#/components/examples/NoSuchMetalakeException"
+ "500":
+ description:
+ Internal server error. It is possible that the server
+ encountered a storage issue.
+ content:
+ application/vnd.gravitino.v1+json:
+ schema:
+ $ref: "./openapi.yaml#/components/schemas/ErrorModel"
+
delete:
tags:
- metalake
summary: Drop metalake
operationId: dropMetalake
description: Drops a specified metalake
+ parameters:
+ - $ref: "./openapi.yaml#/components/parameters/force"
responses:
"200":
$ref: "./openapi.yaml#/components/responses/DropResponse"
@@ -173,6 +207,18 @@ components:
additionalProperties:
type: string
+ MetalakeSetRequest:
+ type: object
+ required:
+ - inUse
+ properties:
+ inUse:
+ type: boolean
+ description: The in-use status of the metalake to set
+ example: {
+ "inUse": true
+ }
+
MetalakeUpdatesRequest:
type: object
required:
diff --git a/docs/open-api/openapi.yaml b/docs/open-api/openapi.yaml
index 24bc0f2ce..69d3f4cbb 100644
--- a/docs/open-api/openapi.yaml
+++ b/docs/open-api/openapi.yaml
@@ -362,6 +362,25 @@ components:
type: boolean
description: Whether the set operation was successful
+ BaseResponse:
+ description: Represents a response for a base operation
+ content:
+ application/vnd.gravitino.v1+json:
+ schema:
+ type: object
+ required:
+ - code
+ properties:
+ code:
+ type: integer
+ format: int32
+ description: Status code of the response
+ enum:
+ - 0
+ example: {
+ "code": 0
+ }
+
parameters:
metalake:
name: metalake
@@ -467,6 +486,15 @@ components:
schema:
type: string
+ force:
+ name: force
+ in: query
+ description: Force the operation to be executed
+ required: false
+ schema:
+ type: boolean
+ default: false
+
securitySchemes:
OAuth2WithJWT:
