Re: [PR] [Docs] Rework docs [gravitino]
tengqm closed pull request #6849: [Docs] Rework docs URL: https://github.com/apache/gravitino/pull/6849 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
tengqm commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2842047522 superseded by #7111 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
jerqi commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2801585999 > > Is `GVFS` suitable to put in the client module? cc @diqiu50 Maybe we should have the document about fuse, too. > > Currently, 'GVFS' is relocated to `catalogs/fileset/gvfs`, next to `catalogs/fileset/hadoop`. If we have client side module(s), we may want to put it under `client/`. As always, all kinds of suggestions/ideas are welcomed. For example, maybe treating `gvfs` as a fileset variant is not good, there is a better home for it?  This is GVFS client code. For me GVFS is a implementation of the client instead of server side. It wrapped the Gravitino client and hide the communication with Gravitino server. And then it implemented the Hadoop Filesystem interface. Maybe others can give another suggestion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
FANNG1 commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2042268474 ## docs/iceberg-rest-service.md: ## @@ -1,470 +0,0 @@ Review Comment: got it, thx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
tengqm commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2801566458 > Do we need a document `java-client.md` in the module client? That would be a good idea. I'd suggest we add an overview of the Java client, with references to the generated Java docs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
tengqm commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2801643214 > > > Is `GVFS` suitable to put in the client module? cc @diqiu50 Maybe we should have the document about fuse, too. > > > > > > Currently, 'GVFS' is relocated to `catalogs/fileset/gvfs`, next to `catalogs/fileset/hadoop`. If we have client side module(s), we may want to put it under `client/`. As always, all kinds of suggestions/ideas are welcomed. For example, maybe treating `gvfs` as a fileset variant is not good, there is a better home for it? > >  > > This is GVFS client code. For me GVFS is a implementation of the client instead of server side. It wrapped the Gravitino client and hide the communication with Gravitino server. And then it implemented the Hadoop Filesystem interface. Maybe others can give another suggestion. Thanks for the clarification. My previous understanding could be wrong. If GVFS is only about client side emulation/simulation, I'm happy to find a new home for it (maybe `client/gvfs`?). All ears to other suggestions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
tengqm commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2801574549 > Is `GVFS` suitable to put in the client module? cc @diqiu50 Maybe we should have the document about fuse, too. Currently, 'GVFS' is relocated to `catalogs/fileset/gvfs`, next to `catalogs/fileset/hadoop`. If we have client side module(s), we may want to put it under `client/`. As always, all kinds of suggestions/ideas are welcomed. For example, maybe treating `gvfs` as a fileset variant is not good, there is a better home for it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
tengqm commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2042046373 ## docs/iceberg-rest-service.md: ## @@ -1,470 +0,0 @@ Review Comment: Thanks for the review. This file was renamed into `admin\iceberg-server.md`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
FANNG1 commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2041613422 ## docs/iceberg-rest-service.md: ## @@ -1,470 +0,0 @@ Review Comment: Was this document deleted by mistake? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
jerqi commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2800745768 Do we need a document `java-client.md` in the module client? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
jerqi commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2800742473 Is `GVFS` suitable to put in the client module? cc @diqiu50 Maybe we should have the document about fuse, too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034344463 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034215318 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
jerryshao commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2784800354 Thanks @tengqm . This really is a big PR, seems like everything is refactored. I will spend time on it. Greatly appreciated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034328531 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034245541 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034337846 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
tengqm commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2788589993 > Some problems mentioned above were in the original file, but more have been introduced. This is a ridiculous accusation. You are saying this as if you read the original docs carefully, you are saying this without appreciating the huge effort on restructuring the docs layout. Go back home and enjoy your little pity grammar/syntax checker, or read the sembr.org for some good things to learn. This huge pile of docs still need a lot of improvements, including accuracy and clarity. I was not pretending that I'm delivering a collection of perfect docs. There are many defects in the docs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034336665 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on PR #6849: URL: https://github.com/apache/gravitino/pull/6849#issuecomment-2788165654 So I took a look at the first file and there are multiple grammatical issues. Compared with the original the sentences have been broken over lines, and as previously discussed this can introduce errors. Some problems mentioned above were in the original file, but more have been introduced. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034348065 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034351036 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034351836 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034345843 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034343531 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034360211 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034330366 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034352566 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
and use the `gravitino.iceberg-rest.authentication.kerberos.principal`
+and `gravitino.iceberg-rest.authentication.kerberos.keytab-uri`
+to authenticate the Kerberos connection.
+The detailed configuration items are as follows:
+
+
+
+
+ Configuration item
+ Description
+ Default value
+ Required
+ Since Version
+
+
+
+
+ gravitino.iceberg-rest.authentication.type
+
+The type of authentication for Iceberg rest catalog backend.
+This configuration only applicable for for Hive backend,
+and only supports `Kerberos`, `simple` currently.
+As for JDBC backend, only username/password authentication is supported
now.
+
+ `simple`
+ No
+ `0.7.0-incubating`
+
+
+ gravitino.iceberg-rest.authentication.impersonation-enable
+ Whether impersonation is enabled for the Iceberg catalog service.
+ `false`
+ No
+ `0.7.0-incubating`
+
+
+ gravitino.iceberg-rest.hive.metastore.sasl.enabled
+
+Whether SASL authentication protocol is enabled when connecting to
Kerberos Hive metastore.
+
+This value should be `true` in most case
+when the value of `gravitino.iceberg-rest.authentication.type` is
Kerberos.
+In some very rare cases, the SSL protocol is used.
+
+ `false`
+ No
+ `0.7.0-incubating`
+
+
+ gravitino.iceberg-rest.authentication.kerberos.principal
+
+The principal of the Kerberos authentication.
+
+This field required if the value of
`gravitino.iceberg-rest.authentication.type` is `Kerberos`.
+
+ (none)
+ Yes|No
+ `0.7.0-incubating`
+
+
+ gravitino.iceberg-rest.authentication.kerberos.keytab-uri
+
+The URI of the keytab for the Kerberos authentication.
+This field required if the value of
`gravitino.iceberg-rest.authentication.type` is `Kerberos`.
+
+ (none)
+ Yes|No
+ `0.7.0-incubating`
+
+
+
gravitino.iceberg-rest.authentication.kerberos.check-interval-sec
+ The check interval in seconds of Kerberos credential for Iceberg
catalog.
+ 60
+ No
+ `0.7.0-incubating`
+
+
+
gravitino.iceberg-rest.authentication.kerberos.keytab-fetch-timeout-sec
+
+The fetch timeout in seconds when retrieving Kerberos keytab
+from `authentication.kerberos.keytab-uri`.
+
+ 60
+ No
+ `0.7.0-incubating`
+
+
+
+
+### Credential vending
+
+Please refer to [credential vending](../security/credential-vending.md) for
more details.
+
+### Storage
+
+ S3 configuration
+
+
+
+
+ Configuration item
+ Description
+ Default value
+ Required
+ Since Version
+
+
+
+
+ gravitino.iceberg-rest.io-impl
+
+The I/O implementation for `FileIO` in Iceberg.
+Use `org.apache.iceberg.aws.s3.S3FileIO` for S3.
+
+ (none)
+ No
+ `0.6.0-incubating`
+
+
+ gravitino.iceberg-rest.s3-endpoint
+
+An alternative endpoint of the S3 service.
+This could be used for S3FileIO with any s3-compatible object storage
service
+that has a different endpoint, or access a private S3 endpoint
+in a virtual private cloud.
+
+ (none)
+ No
+ `0.6.0-incubating`
+
+
+ gravitino.iceberg-rest.s3-region
+ The region of the S3 service, like `us-west-2`.
+ (none)
+ No
+ `0.6.0-incubating`
+
+
+ gravitino.iceberg-rest.s3-path-style-access
+ Whether to use path style access for S3.
+ `false`
+ No
+ `0.9.0-incubating`
+
+
+
+
+For other Iceberg s3 properties not managed by Gravitino like `s3.sse.type`,
+you could config it directly by `gravitino.iceberg-rest.s3.sse.type`.
+
+Please refer to [S3
credentials](../security/credential-vending.md#s3-credentials)
+for credential related configurations.
+
+:::info
+To configure the JDBC catalog backend, set the
`gravitino.iceberg-rest.warehouse` parameter
+to `s3://{bucket_name}/${prefix_name}`.
+For the Hive catalog backend, set `gravitino.iceberg-rest.warehouse`
+to `s3a://{bucket_name}/${prefix_name}`.
+Additionally, download the [Iceberg AWS
bundle](https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-aws-bundle)
+and place it in the CLASSPATH of Iceberg REST server.
+:::
+
+ OSS configuration
+
+
+
+
+ Configuration item
+ Description
+ Default value
+ Required
+ Since Version
+
+
+
+
+ gravitino.iceberg-rest.io-impl
+
+The I/O implementation for `FileIO` in Iceberg.
+Use `org.apache.iceberg.aliyun.oss.OSSFileIO` for OSS.
+
+ (none)
+ No
+ `0.6.0-incubating`
+
+
+ gravitino.iceberg-rest.oss-endpoint
+ The endpoint of Aliyun OSS service.
+ (none)
+ No
+ `0.7.0-incubating`
+
+
+ gravitino.iceberg-rest.oss-region
+
+The region of the OSS service, like `oss-cn-hangzhou`.
+Only used when `credential-providers` is `oss-token`.
+
+ (none)
+ No
+ `0.8.0-incubating`
+
+
+
+
+For other Iceberg OSS properties not managed by Gravitino like
`client.security-token`,
+you could config it directly by `gravitino.iceberg-rest.client.security-token`.
+
+Please refer to [OSS
credentials](../security/credential-vending.md#oss-credentials)
+for credential related configurations.
+
+Additionally, Iceberg doesn't provide Iceberg Aliyun bundle JA
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034353434 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034355207 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034358895 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034358895 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034358895 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034358124 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034356979 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034356979 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034356149 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034354037 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034350048 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034348599 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034341171 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034346780 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034345123 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034342517 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034340420 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034331463 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034333462 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034335881 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034335280 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034334340 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034332551 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034330366 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034329760 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034327576 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034315444 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034314578 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034237171 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034243218 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034255931 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034253821 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034250921 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034248153 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034241373 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034238883 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034231823 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034216709 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034231271 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034229605 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034214493 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034211802 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: Review Comment: servers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034213687 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. +You don't need to specify them if the Iceberg server is started +as a standalone server. +::: + +### HTTP server configuration + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.iceberg-rest.host + The host of the Gravitino Iceberg REST catalog service. + `0.0.0.0` + No + `0.2.0` + + + gravitino.iceberg-rest.httpPort + The port of the Gravitino Iceberg REST catalog service. + `9001` + No + `0.2.0` + + + gravitino.iceberg-rest.minThreads + +The minimum num
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034213177 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: + +- A standalone server in a standalone Gravitino Iceberg REST server package, the CLASSPATH is `libs`. +- A standalone server in the Gravitino server package, the CLASSPATH is `iceberg-rest-server/libs`. +- An auxiliary service embedded in the Gravitino server, the CLASSPATH is `iceberg-rest-server/libs`. + +For detailed instructions on how to build and install the Gravitino server package, +please refer to [the build guide](../develop/how-to-build.md) and [the installation guide](../install/install.md). +To build the Gravitino Iceberg REST server package, use the command `./gradlew compileIcebergRESTServer -x test`. +Alternatively, to create the corresponding compressed package in the distribution directory, +use `./gradlew assembleIcebergRESTServer -x test`. +The Gravitino Iceberg REST server package includes the following files: + +```text +├─ ... +└─ distribution/gravitino-iceberg-rest-server +├─ bin/ +│ └─ gravitino-iceberg-rest-server.sh# Launching scripts. +├─ conf/ # All configurations. +│ ├─ gravitino-iceberg-rest-server.conf # Server configuration. +│ ├─ gravitino-env.sh# Environment variables, e.g. JAVA_HOME, GRAVITINO_HOME, etc. +│ ├─ log4j2.properties # log4j configurations. +│ └─ hdfs-site.xml & core-site.xml # HDFS configuration files. +├─ libs/ # Dependencies libraries. +└─ logs/ # Logs directory. Auto-created after the server starts. +``` + +## Server configuration + +There are distinct configuration files for standalone and auxiliary server: + +- `gravitino-iceberg-rest-server.conf` is used for the standalone server; +- `gravitino.conf` is for the auxiliary server. + +Although the configuration files differ, the configuration items remain the same. + +Starting with version `0.6.0-incubating`, the prefix `gravitino.auxService.iceberg-rest.` +for auxiliary server configurations has been deprecated. +If both `gravitino.auxService.iceberg-rest.key` and `gravitino.iceberg-rest.key` are present, +the latter will take precedence. +The configurations listed below use the `gravitino.iceberg-rest.` prefix. + +### Configuration to enable Iceberg REST service in Gravitino server. + + + + + Configuration item + Description + Default value + Required + Since Version + + + + + gravitino.auxService.names + +The auxiliary service name of the Gravitino Iceberg REST catalog service. +Use `iceberg-rest`. + + (none) + Yes + `0.2.0` + + + gravitino.iceberg-rest.classpath + +The CLASSPATH of the Gravitino Iceberg REST catalog service, +including the directory containing JARs and configuration. +It supports both absolute and relative paths. +For example, `iceberg-rest-server/libs,iceberg-rest-server/conf`. + + (none) + Yes + `0.2.0` + + + + +:::note +These configurations only are only effective in `gravitino.conf`. Review Comment: only are only -> are only -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Docs] Rework docs [gravitino]
justinmclean commented on code in PR #6849: URL: https://github.com/apache/gravitino/pull/6849#discussion_r2034209853 ## docs/admin/iceberg-server.md: ## @@ -0,0 +1,1351 @@ +--- +title: Iceberg REST catalog service +slug: /iceberg-rest-service +keywords: + - Iceberg REST catalog +license: "This software is licensed under the Apache License version 2." +--- + +## Background + +The Apache Gravitino Iceberg REST Server follows the +[Apache Iceberg REST API specification](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml) +and acts as an Iceberg REST catalog server, +you could access the Iceberg REST endpoint at `http://$ip:$port/iceberg/`. + +### Capabilities + +- Supports the Apache Iceberg REST API defined in Iceberg 1.5, and supports all namespace and table interfaces. + The following interfaces are not implemented yet: + - multi-table transaction + - pagination +- Works as a catalog proxy, supporting `Hive` and `JDBC` as catalog backend. +- Supports credential vending for `S3`、`GCS`、`OSS` and `ADLS`. +- Supports different storages like `S3`, `HDFS`, `OSS`, `GCS`, `ADLS`. +- Capable of supporting other storages. +- Supports event listener. +- Supports Audit log. +- Supports OAuth2 and HTTPS. +- Provides a pluggable metrics store interface to store and delete Iceberg metrics. + +## Server management + +There are three deployment scenarios for Gravitino Iceberg REST server: Review Comment: for the Gravitino -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
