diqiu50 commented on code in PR #11523: URL: https://github.com/apache/gravitino/pull/11523#discussion_r3402440016
########## docs/aws-glue-catalog.md: ########## @@ -0,0 +1,316 @@ +--- +title: "AWS Glue catalog" +slug: /aws-glue-catalog +keywords: + - glue + - aws + - metadata +license: "This software is licensed under the Apache License version 2." +--- + +## Introduction + +Apache Gravitino uses [AWS Glue Data Catalog](https://aws.amazon.com/glue/) as a metadata catalog. + +### Requirements + +* The Glue catalog requires network access to the AWS Glue API. +* Gravitino uses the AWS SDK v2 to communicate with Glue. + +:::note +The Glue catalog is case-insensitive for schema and table names. AWS Glue folds database and table names to lowercase on storage. +::: + +## Catalog + +### Catalog Capabilities + +The Glue catalog supports creating, updating, and deleting databases and tables in the AWS Glue Data Catalog. + +- Supports all table types stored in Glue (Hive, Iceberg, Delta, Parquet, and others) by default. +- Supports Hive-format table partitioning, bucketing, and sort orders. +- Does not support views. Glue views (tables with `TableType=VIRTUAL_VIEW`) are filtered out. + +### Catalog Properties + +Besides the [common catalog properties](./gravitino-server-config.md#catalog-properties-configuration), the Glue catalog has the following properties: + +| Property Name | Description | Default Value | Required | Immutable | Since Version | +|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------|----------|-----------|---------------| +| `aws-region` | AWS region for the Glue Data Catalog (e.g. `us-east-1`). | (none) | Yes | Yes | 1.3.0 | +| `aws-glue-catalog-id` | The 12-digit AWS account ID that owns the Glue catalog. When omitted, defaults to the caller's AWS account ID. | (none) | No | Yes | 1.3.0 | +| `aws-access-key-id` | AWS access key ID for static credential authentication. When omitted, the default credential chain is used. | (none) | No | No | 1.3.0 | +| `aws-secret-access-key` | AWS secret access key paired with `aws-access-key-id`. When omitted, the default credential chain is used. | (none) | No | No | 1.3.0 | +| `aws-glue-endpoint` | Custom Glue endpoint URL for VPC endpoints or LocalStack testing (e.g. `http://localhost:4566`). | (none) | No | No | 1.3.0 | +| `warehouse` | Base storage path used as the warehouse when no explicit `location` is specified at table creation time (e.g. `s3://my-bucket/warehouse`). Table location is derived as `warehouse/database/table`. | (none) | Yes | No | 1.3.0 | +| `default-table-format` | Default format for tables created via Gravitino's `createTable()` API. Accepted values: `iceberg`, `hive`. | `hive` | No | No | 1.3.0 | +| `table-format-filter` | Comma-separated list of table formats exposed by `listTables()` and `loadTable()`. Accepted values: `all`, `hive`, `iceberg`, `delta`, `parquet`. Use to restrict visible table types. | `all` | No | No | 1.3.0 | + +:::note +**Authentication priority**: Static credentials (`aws-access-key-id` + `aws-secret-access-key`) take precedence over the default credential chain (environment variables, instance profile, container credentials). +::: + +### Connector Configuration + +- When using Gravitino with Trino, pass additional Trino connector properties using the `trino.bypass.` prefix. For example, `trino.bypass.hive.metastore.glue.max-connections` sets `hive.metastore.glue.max-connections` in the Trino runtime. Review Comment: No, I remove it. `trino.bypass` should in the `trino-connector` docs。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
