danhuawang opened a new issue, #11283: URL: https://github.com/apache/gravitino/issues/11283
### Version main branch ### Describe what's wrong version: main branch (docker image: `datastratosandbox/gravitino:1.3.0-SNAPSHOT-20260527024943`) When running the Gravitino server Docker image (with Iceberg REST as an auxiliary service), the `rewrite_gravitino_server_config.py` script does not map most of the documented Docker environment variables to `gravitino.conf`. As a result, environment variables like `GRAVITINO_WAREHOUSE`, `GRAVITINO_IO_IMPL`, `GRAVITINO_CATALOG_BACKEND`, `GRAVITINO_URI`, `GRAVITINO_S3_*`, etc. are silently ignored, and the Iceberg REST service falls back to default values (`catalog-backend = memory`, `warehouse = /tmp/`). The [official documentation](https://gravitino.apache.org/docs/1.1.0/iceberg-rest-service/) lists a comprehensive set of Docker environment variables that should be supported, but the `env_map` in `rewrite_gravitino_server_config.py` only contains a small subset (`GRAVITINO_ICEBERG_REST_CATALOG_BACKEND`, `GRAVITINO_ICEBERG_REST_WAREHOUSE`, `GRAVITINO_ICEBERG_REST_HTTP_PORT`), which do not match the documented variable names. ### Error message and/or stacktrace No error is raised — the environment variables are silently ignored. The resulting `gravitino.conf` contains default values instead of the user-specified ones: ``` # Expected (based on env vars): gravitino.iceberg-rest.catalog-backend = jdbc gravitino.iceberg-rest.warehouse = s3://warehouse/ gravitino.iceberg-rest.io-impl = org.apache.iceberg.aws.s3.S3FileIO gravitino.iceberg-rest.s3-endpoint = http://minio:9000 ... # Actual (in container): gravitino.iceberg-rest.catalog-backend = memory gravitino.iceberg-rest.warehouse = /tmp/ ``` This causes downstream failures such as scan planning errors: ``` RuntimeException: Scan planning failed for table default.test_rest_scan: No in-memory file found for location: /tmp/default/test_rest_scan/metadata/snap-xxx.avro ``` ### How to reproduce 1. Use the Gravitino Docker image `datastratosandbox/gravitino:1.3.0-SNAPSHOT-20260527024943` 2. Set the documented environment variables in docker-compose: ```yaml services: rest: image: docker.io/datastratosandbox/gravitino:1.3.0-SNAPSHOT-20260527024943 environment: - GRAVITINO_WAREHOUSE=s3://warehouse/ - GRAVITINO_IO_IMPL=org.apache.iceberg.aws.s3.S3FileIO - GRAVITINO_S3_ENDPOINT=http://minio:9000 - GRAVITINO_S3_ACCESS_KEY=admin - GRAVITINO_S3_SECRET_KEY=password - GRAVITINO_S3_REGION=us-east-1 - GRAVITINO_S3_PATH_STYLE_ACCESS=true - GRAVITINO_CATALOG_BACKEND=jdbc - "GRAVITINO_URI=jdbc:sqlite::memory:" - GRAVITINO_JDBC_DRIVER=org.sqlite.JDBC - GRAVITINO_ICEBERG_REST_HTTP_PORT=9001 ``` 3. Start the container and inspect the generated config: ```bash docker exec <container> cat /opt/gravitino/conf/gravitino.conf ``` 4. Observe that only `GRAVITINO_ICEBERG_REST_HTTP_PORT` is reflected in the config. All other Iceberg REST environment variables are missing. ### Additional context The `env_map` in `dev/docker/gravitino/rewrite_gravitino_server_config.py` currently contains: ```python env_map = { ... "GRAVITINO_ICEBERG_REST_CATALOG_BACKEND": "iceberg-rest.catalog-backend", "GRAVITINO_ICEBERG_REST_WAREHOUSE": "iceberg-rest.warehouse" } ``` But the documentation specifies shorter environment variable names: | Documented Env Var | Expected Config Key | |---|---| | `GRAVITINO_WAREHOUSE` | `gravitino.iceberg-rest.warehouse` | | `GRAVITINO_IO_IMPL` | `gravitino.iceberg-rest.io-impl` | | `GRAVITINO_CATALOG_BACKEND` | `gravitino.iceberg-rest.catalog-backend` | | `GRAVITINO_URI` | `gravitino.iceberg-rest.uri` | | `GRAVITINO_JDBC_DRIVER` | `gravitino.iceberg-rest.jdbc-driver` | | `GRAVITINO_S3_ACCESS_KEY` | `gravitino.iceberg-rest.s3-access-key-id` | | `GRAVITINO_S3_SECRET_KEY` | `gravitino.iceberg-rest.s3-secret-access-key` | | `GRAVITINO_S3_ENDPOINT` | `gravitino.iceberg-rest.s3-endpoint` | | `GRAVITINO_S3_REGION` | `gravitino.iceberg-rest.s3-region` | | `GRAVITINO_S3_PATH_STYLE_ACCESS` | `gravitino.iceberg-rest.s3-path-style-access` | The `env_map` should be updated to include all documented Docker environment variable mappings, or the existing `GRAVITINO_ICEBERG_REST_*` prefixed variables should be documented as the correct ones to use with the Gravitino server image. Note: The standalone Iceberg REST server image (`dev/docker/iceberg-rest-server`) likely has its own config rewrite script that handles these correctly. The issue is specific to the Gravitino server image where Iceberg REST runs as an auxiliary service. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
