snazy commented on code in PR #3482: URL: https://github.com/apache/polaris/pull/3482#discussion_r2707897285
########## getting-started/rustfs/docker-compose.yml: ########## @@ -0,0 +1,116 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +services: + + rustfs: + image: rustfs/rustfs:latest + ports: + - "9000:9000" # S3 API port + - "9001:9001" # Console port + environment: + RUSTFS_ACCESS_KEY: rustfsadmin + RUSTFS_SECRET_KEY: rustfsadmin + RUSTFS_VOLUMES: /data + RUSTFS_ADDRESS: ":9000" + RUSTFS_CONSOLE_ENABLE: "true" + RUSTFS_CONSOLE_ADDRESS: ":9001" + healthcheck: + test: + [ + "CMD", + "sh", "-c", + "curl -f http://127.0.0.1:9000/health && curl -f http://127.0.0.1:9001/rustfs/console/health"; + ] + interval: 30s + timeout: 10s + retries: 3 + start_period: 40s + + polaris: + image: apache/polaris:latest + ports: + # API port + - "8181:8181" + # Optional, allows attaching a debugger to the Polaris JVM + - "5005:5005" + depends_on: + rustfs: + condition: service_healthy + environment: + JAVA_DEBUG: true + JAVA_DEBUG_PORT: "*:5005" + AWS_REGION: us-west-2 + AWS_ACCESS_KEY_ID: rustfsadmin + AWS_SECRET_ACCESS_KEY: rustfsadmin + POLARIS_BOOTSTRAP_CREDENTIALS: POLARIS,root,s3cr3t + polaris.realm-context.realms: POLARIS + quarkus.otel.sdk.disabled: "true" + healthcheck: + test: ["CMD", "curl", "http://localhost:8182/q/health";] + interval: 2s + timeout: 10s + retries: 10 + start_period: 10s + + setup_bucket: + image: quay.io/minio/mc:latest Review Comment: Wonder why `mc` is being used here. Doesn't Rustfs provide a tool for this as well? ########## getting-started/rustfs/README.md: ########## @@ -0,0 +1,95 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Getting Started with Apache Polaris and RustFS + +## Overview + +This example uses RustFS as a storage provider with Polaris. + +Spark is used as a query engine. This example assumes a local Spark installation. +See the [Spark Notebooks Example](../spark/README.md) for a more advanced Spark setup. + +## Starting the Example + +1. Build the Polaris server image if it's not already present locally: + + ```shell + ./gradlew \ + :polaris-server:assemble \ + :polaris-server:quarkusAppPartsBuild --rerun \ + -Dquarkus.container-image.build=true + ``` + +2. Start the docker compose group by running the following command from the root of the repository: + + ```shell + docker compose -f getting-started/rustfs/docker-compose.yml up Review Comment: This one fails for me after a while: ``` docker compose -f getting-started/rustfs/docker-compose.yml up >>>> Executing external compose provider "/usr/libexec/docker/cli-plugins/docker-compose". Please see podman-compose(1) for how to disable this message. <<<< [+] Running 17/17 ✔ setup_bucket Pulled 3.1s ✔ b83ce1c86227 Already exists 0.0s ✔ b7c1efd1d2b4 Download complete 0.7s ✔ b268f447c871 Download complete 0.0s ✔ f29310fc4a8a Download complete 0.0s ✔ 9404e0df6bd3 Download complete 1.1s ✔ 4a93952ab9ce Download complete 0.1s ✔ c2ad77420d33 Download complete 0.0s ✔ rustfs Pulled 6.7s ✔ 4f4fb700ef54 Already exists 0.0s ✔ 1074353eec0d Already exists 0.0s ✔ 1d1dee9383a6 Download complete 0.0s ✔ 82f2cf490642 Download complete 0.0s ✔ d3ec0616f053 Download complete 0.0s ✔ efb5b00da6a9 Download complete 0.3s ✔ 981fcf90905c Download complete 5.5s ✔ c83e944dc36e Download complete 0.0s [+] Running 5/5 ✔ Network rustfs_default Created 0.0s ✔ Container rustfs-rustfs-1 Created 0.7s ✔ Container rustfs-setup_bucket-1 Created 0.0s ✔ Container rustfs-polaris-1 Created 0.0s ✔ Container rustfs-polaris-setup-1 Created 0.0s Attaching to polaris-1, polaris-setup-1, rustfs-1, setup_bucket-1 rustfs-1 | Initializing data directories: /data rustfs-1 | OBS log directory not configured and logs outputs to stdout rustfs-1 | !!!WARNING: Using default RUSTFS_ACCESS_KEY or RUSTFS_SECRET_KEY. Override them in production! rustfs-1 | Starting: /usr/bin/rustfs /data rustfs-1 | RustFS Http API: http://10.89.0.4:9000 http://127.0.0.1:9000 rustfs-1 | RustFS Start Time: 2026-01-20 11:14:43 rustfs-1 | {"timestamp":"2026-01-20T11:14:43.212466879Z","level":"WARN","fields":{"message":"Detected default credentials 'rustfsadmin:rustfsadmin', we recommend that you change these values with 'RUSTFS_ACCESS_KEY' and 'RUSTFS_SECRET_KEY' environment variables"},"target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":171,"threadName":"main","threadId":"ThreadId(1)"} rustfs-1 | Console WebUI Start Time: 2026-01-20 11:14:43 rustfs-1 | Console WebUI available at: http://10.89.0.4:9001/rustfs/console/index.html rustfs-1 | Console WebUI (localhost): http://127.0.0.1:9001/rustfs/console/index.html rustfs-1 | {"timestamp":"2026-01-20T11:14:43.215548374Z","level":"WARN","fields":{"message":"config not found, start to init"},"target":"rustfs_ecstore::tier::tier","filename":"crates/ecstore/src/tier/tier.rs","line_number":490,"threadName":"main","threadId":"ThreadId(1)"} rustfs-1 | {"timestamp":"2026-01-20T11:14:43.21644797Z","level":"WARN","fields":{"message":"Configuration not found (Read the main configuration): Start initializing new configuration"},"target":"rustfs_ecstore::config::com","filename":"crates/ecstore/src/config/com.rs","line_number":137,"threadName":"main","threadId":"ThreadId(1)"} rustfs-1 | {"timestamp":"2026-01-20T11:14:43.217214776Z","level":"WARN","fields":{"message":"Configuration initialization complete (Read the main configuration)"},"target":"rustfs_ecstore::config::com","filename":"crates/ecstore/src/config/com.rs","line_number":139,"threadName":"main","threadId":"ThreadId(1)"} rustfs-1 | {"timestamp":"2026-01-20T11:14:43.220595899Z","level":"WARN","fields":{"message":"Global region is not set; attempting notification configuration for all buckets with an empty region."},"target":"rustfs::init","filename":"rustfs/src/init.rs","line_number":101,"threadName":"main","threadId":"ThreadId(1)"} rustfs-1 | {"timestamp":"2026-01-20T11:14:43.220696059Z","level":"WARN","fields":{"message":"main checkpoint file is corrupted or not exists: Not found: checkpoint file not exists: \"/tmp/rustfs_scanner/scanner_checkpoint_scanner-node-a00d591b5b1a4f3e9731b123df370860.json\""},"target":"rustfs_ahm::scanner::checkpoint","filename":"crates/ahm/src/scanner/checkpoint.rs","line_number":154,"threadName":"main","threadId":"ThreadId(1)"} rustfs-1 | {"timestamp":"2026-01-20T11:14:43.220705629Z","level":"WARN","fields":{"message":"backup file is corrupted or not exists: Not found: checkpoint file not exists: \"/tmp/rustfs_scanner/scanner_checkpoint_scanner-node-a00d591b5b1a4f3e9731b123df370860.backup\""},"target":"rustfs_ahm::scanner::checkpoint","filename":"crates/ahm/src/scanner/checkpoint.rs","line_number":172,"threadName":"main","threadId":"ThreadId(1)"} rustfs-1 | RustFS server started successfully at [::]:9000, current time: 2026-01-20 11:14:43.220719429 UTC dependency failed to start: container rustfs-rustfs-1 is unhealthy Error: executing /usr/libexec/docker/cli-plugins/docker-compose -f getting-started/rustfs/docker-compose.yml up: exit status 1 ``` The health-check command itself seems to work fine though: ``` $ docker container exec c169ceab8242 sh -c 'curl -f http://127.0.0.1:9000/health && curl -f http://127.0.0.1:9001/rustfs/console/health' % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 111 100 111 0 0 285347 0 --:--:-- --:--:-- --:--:-- 111000 {"service":"rustfs-endpoint","status":"ok","timestamp":"2026-01-20T11:20:07.728989111+00:00","version":"0.0.5"} % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 204 100 204 0 0 532637 0 --:--:-- --:--:-- --:--:-- 204000 {"details":{"iam":{"status":"connected"},"storage":{"status":"connected"}},"service":"rustfs-console","status":"ok","timestamp":"2026-01-20T11:20:07.732014419+00:00","uptime":1768908007,"version":"0.0.5"} ``` ########## getting-started/rustfs/docker-compose.yml: ########## @@ -0,0 +1,116 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +services: + + rustfs: + image: rustfs/rustfs:latest Review Comment: Can you add a "real" version here and below? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
