Copilot commented on code in PR #997: URL: https://github.com/apache/skywalking-banyandb/pull/997#discussion_r2910321486
########## docs/guides/quick-start/docker-compose.yaml: ########## @@ -1,52 +1,102 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at +# Licensed to Apache Software Foundation (ASF) under one or more contributor +# license agreements. See the NOTICE file distributed with this work for +# additional information regarding copyright ownership. The ASF licenses this +# file to you under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain a +# copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. +# Unless required by applicable law or agreed to in writing, software distributed +# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR +# CONDITIONS OF ANY KIND, either express or implied. See the License for the +# specific language governing permissions and limitations under the License. + + +# This docker-compose file demonstrates how to use BanyanDB as the storage +# backend for SkyWalking OAP. It includes: +# - BanyanDB standalone instance with data persistence +# - SkyWalking OAP server with BanyanDB storage +# - SkyWalking UI +# - Demo services (agent, provider, consumer) for generating trace traffic +# +# For VM metrics collection, add an OTEL collector or use node_exporter. +# See: https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-storage/ +# +# Usage: +# docker compose up -d +# +# To use different versions, override the image tags: +# BANYANDB_IMAGE=apache/skywalking-banyandb:0.9.0 \ +# OAP_IMAGE=apache/skywalking-oap-server:10.3.0 \ +# OAP_UI_IMAGE=apache/skywalking-ui:10.3.0 \ +# docker compose up -d services: banyandb: - image: ${BANYANDB_IMAGE:-apache/skywalking-banyandb:latest} + image: ${BANYANDB_IMAGE:-apache/skywalking-banyandb:0.9.0} container_name: banyandb command: standalone expose: - 17912 ports: - - 17913:17913 + - 17913:17913 # HTTP port for bydbctl and UI + - 6060:6060 # Prometheus metrics endpoint + - 2121:2121 # Data port Review Comment: Port annotations look incorrect: 6060 is the default pprof listener (not Prometheus metrics), and 2121 is the observability/metrics listener (not a "Data" port). This is misleading for users and also conflicts with existing docs/config defaults. ########## docs/guides/quick-start/quick-start.md: ########## @@ -1,10 +1,112 @@ # Quick Start Tutorial + The following tutorial will guide you through setting up a SkyWalking OAP with BanyanDB as the storage backend using Docker Compose. It is a quick way to get started with BanyanDB if you are a SkyWalking user and want to try out BanyanDB. -## Set up quickstart cluster with Showcase +## Option 1: Use the Docker Compose Example (Recommended) + +This repository includes a ready-to-use docker-compose file at [docker-compose.yaml](docker-compose.yaml) that sets up: + +- **BanyanDB** - Standalone instance with etcd schema registry +- **SkyWalking OAP** - Configured to use BanyanDB as storage +- **SkyWalking UI** - Web interface for visualizing traces, metrics, and topology +- **Demo Services** - Sample microservices that generate trace traffic automatically Review Comment: This section says the compose sets up “BanyanDB - Standalone instance with etcd schema registry”, but `standalone` defaults to `--schema-registry-mode=property` (property-based schema registry), unless explicitly set otherwise. Please adjust the quick-start text to match the actual default behavior, or add the flag if etcd is required. ########## .github/workflows/quick-start-test.yml: ########## @@ -0,0 +1,196 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +name: Quick Start Test + +on: + schedule: + # Run daily at 3:00 UTC + - cron: '0 3 * * *' + workflow_dispatch: + pull_request: + paths: + - 'docs/guides/quick-start/**' + +concurrency: + group: quick-start-test-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +jobs: + test-quick-start: + name: Test Quick Start Example + runs-on: ubuntu-latest + timeout-minutes: 45 + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Free disk space + run: | + df -h + docker system prune -af --volumes || true + df -h + + - name: Pull images + working-directory: docs/guides/quick-start + run: | + MAX_PULL_RETRIES=5 + for i in $(seq 1 "$MAX_PULL_RETRIES"); do + if docker compose pull; then + break + fi + if [ "$i" -eq "$MAX_PULL_RETRIES" ]; then + echo "ERROR: docker compose pull failed after $MAX_PULL_RETRIES attempts" + exit 1 + fi + echo "docker compose pull failed, retrying in 10 seconds (attempt $i of $MAX_PULL_RETRIES)..." + sleep 10 + done + + - name: Start the stack + working-directory: docs/guides/quick-start + run: | + docker compose up -d + sleep 10 + + - name: Check container status + working-directory: docs/guides/quick-start + run: | + docker compose ps + + - name: Wait for BanyanDB to start + run: | + echo "Waiting for BanyanDB..." + for i in $(seq 1 30); do + if curl -sf http://localhost:17913/api/healthz > /dev/null; then + echo "BanyanDB is ready" + break + fi + sleep 2 + done + curl -sf http://localhost:17913/api/healthz + + - name: Wait for OAP to start + run: | + echo "Waiting for OAP..." + for i in $(seq 1 30); do + if curl -sf http://localhost:12800/graphql -X POST -H "Content-Type: application/json" -d '{"query":"{ __typename }"}' > /dev/null; then + echo "OAP is ready" + break + fi + sleep 2 + done + curl -sf http://localhost:12800/graphql -X POST -H "Content-Type: application/json" -d '{"query":"{ __typename }"}' + Review Comment: The readiness loops for BanyanDB/OAP only wait up to ~60s (30 × 2s). On slower pulls/starts this can be flaky for CI. Consider increasing the retry budget (e.g., 2–3 minutes), and/or adding more informative diagnostics (e.g., `docker compose logs --tail ...`) before failing. ########## docs/guides/quick-start/docker-compose.yaml: ########## @@ -1,52 +1,102 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at +# Licensed to Apache Software Foundation (ASF) under one or more contributor +# license agreements. See the NOTICE file distributed with this work for +# additional information regarding copyright ownership. The ASF licenses this +# file to you under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain a +# copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. +# Unless required by applicable law or agreed to in writing, software distributed +# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR +# CONDITIONS OF ANY KIND, either express or implied. See the License for the +# specific language governing permissions and limitations under the License. + + +# This docker-compose file demonstrates how to use BanyanDB as the storage +# backend for SkyWalking OAP. It includes: +# - BanyanDB standalone instance with data persistence +# - SkyWalking OAP server with BanyanDB storage +# - SkyWalking UI +# - Demo services (agent, provider, consumer) for generating trace traffic +# +# For VM metrics collection, add an OTEL collector or use node_exporter. +# See: https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-storage/ +# +# Usage: +# docker compose up -d +# +# To use different versions, override the image tags: +# BANYANDB_IMAGE=apache/skywalking-banyandb:0.9.0 \ +# OAP_IMAGE=apache/skywalking-oap-server:10.3.0 \ +# OAP_UI_IMAGE=apache/skywalking-ui:10.3.0 \ +# docker compose up -d services: banyandb: - image: ${BANYANDB_IMAGE:-apache/skywalking-banyandb:latest} + image: ${BANYANDB_IMAGE:-apache/skywalking-banyandb:0.9.0} container_name: banyandb command: standalone expose: - 17912 ports: - - 17913:17913 + - 17913:17913 # HTTP port for bydbctl and UI + - 6060:6060 # Prometheus metrics endpoint + - 2121:2121 # Data port + volumes: + - banyandb-data:/tmp/banyanDB networks: - demo + healthcheck: + test: ["CMD", "bash", "-c", "cat < /dev/null > /dev/tcp/127.0.0.1/17913"] + interval: 10s + timeout: 30s + retries: 10 + start_period: 30s oap: - image: ${OAP_IMAGE:-apache/skywalking-oap-server:latest} + image: ${OAP_IMAGE:-apache/skywalking-oap-server:10.3.0} + container_name: skywalking-oap environment: + # Storage configuration - use BanyanDB as the backend SW_STORAGE: banyandb SW_STORAGE_BANYANDB_TARGETS: banyandb:17912 + + # Enable VM metrics collection via OpenTelemetry + # These settings enable the OTel receiver and VM metrics rules + SW_OTEL_RECEIVER: default + SW_OTEL_RECEIVER_ENABLED_OC_RULES: vm + SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES: vm + + # Telemetry and performance settings + SW_TELEMETRY: prometheus + SW_CORE_RECORD_DATA_TTL: 7 + SW_CORE_METRICS_DATA_TTL: 7 + + # Optional: Increase max message size for high throughput scenarios + SW_DCS_MAX_INBOUND_MESSAGE_SIZE: 5000000000 expose: - - 11800 + - 11800 # gRPC port for agents + - 12800 # HTTP port for OAP API + - 9090 # Prometheus metrics ports: - 12800:12800 + - 11800:11800 networks: - demo depends_on: banyandb: - condition: service_healthy + condition: service_started healthcheck: Review Comment: `depends_on` is set to `service_started` even though a healthcheck is defined for `banyandb`. This removes readiness gating and can race OAP startup before BanyanDB is actually accepting traffic. Consider switching to `condition: service_healthy` (and similarly for other dependent services) to make the quick-start more reliable. ########## scripts/test-quick-start-local.sh: ########## @@ -0,0 +1,143 @@ +#!/usr/bin/env bash +# Licensed to Apache Software Foundation (ASF) under one or more contributor +# license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright +# ownership. Apache Software Foundation (ASF) licenses this file to you under +# the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Local test script for the quick-start workflow. +# Run this to verify the quick-start docker-compose setup without using act/GitHub. + +set -e + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +QUICK_START_DIR="$REPO_ROOT/docs/guides/quick-start" + +cd "$REPO_ROOT" + +echo "=== Free disk space ===" +df -h +docker system prune -af --volumes || true +df -h Review Comment: This script runs `docker system prune -af --volumes`, which will delete all local Docker images/containers/volumes on the machine. That’s risky for a “local test” helper. Consider gating it behind an explicit env var (e.g., `PRUNE_DOCKER=1`) and/or printing a prominent warning + confirmation prompt. ########## docs/guides/quick-start/docker-compose.yaml: ########## @@ -1,52 +1,102 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at +# Licensed to Apache Software Foundation (ASF) under one or more contributor +# license agreements. See the NOTICE file distributed with this work for +# additional information regarding copyright ownership. The ASF licenses this +# file to you under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain a +# copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. +# Unless required by applicable law or agreed to in writing, software distributed +# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR +# CONDITIONS OF ANY KIND, either express or implied. See the License for the +# specific language governing permissions and limitations under the License. + + +# This docker-compose file demonstrates how to use BanyanDB as the storage +# backend for SkyWalking OAP. It includes: +# - BanyanDB standalone instance with data persistence +# - SkyWalking OAP server with BanyanDB storage +# - SkyWalking UI +# - Demo services (agent, provider, consumer) for generating trace traffic +# +# For VM metrics collection, add an OTEL collector or use node_exporter. +# See: https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-storage/ +# +# Usage: +# docker compose up -d +# +# To use different versions, override the image tags: +# BANYANDB_IMAGE=apache/skywalking-banyandb:0.9.0 \ +# OAP_IMAGE=apache/skywalking-oap-server:10.3.0 \ +# OAP_UI_IMAGE=apache/skywalking-ui:10.3.0 \ +# docker compose up -d services: banyandb: - image: ${BANYANDB_IMAGE:-apache/skywalking-banyandb:latest} + image: ${BANYANDB_IMAGE:-apache/skywalking-banyandb:0.9.0} container_name: banyandb command: standalone expose: - 17912 ports: - - 17913:17913 + - 17913:17913 # HTTP port for bydbctl and UI + - 6060:6060 # Prometheus metrics endpoint + - 2121:2121 # Data port + volumes: + - banyandb-data:/tmp/banyanDB networks: - demo + healthcheck: + test: ["CMD", "bash", "-c", "cat < /dev/null > /dev/tcp/127.0.0.1/17913"] + interval: 10s + timeout: 30s + retries: 10 + start_period: 30s oap: - image: ${OAP_IMAGE:-apache/skywalking-oap-server:latest} + image: ${OAP_IMAGE:-apache/skywalking-oap-server:10.3.0} + container_name: skywalking-oap environment: + # Storage configuration - use BanyanDB as the backend SW_STORAGE: banyandb SW_STORAGE_BANYANDB_TARGETS: banyandb:17912 + + # Enable VM metrics collection via OpenTelemetry + # These settings enable the OTel receiver and VM metrics rules + SW_OTEL_RECEIVER: default + SW_OTEL_RECEIVER_ENABLED_OC_RULES: vm + SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES: vm + + # Telemetry and performance settings + SW_TELEMETRY: prometheus + SW_CORE_RECORD_DATA_TTL: 7 + SW_CORE_METRICS_DATA_TTL: 7 + + # Optional: Increase max message size for high throughput scenarios + SW_DCS_MAX_INBOUND_MESSAGE_SIZE: 5000000000 Review Comment: `SW_DCS_MAX_INBOUND_MESSAGE_SIZE` is set to 5,000,000,000 by default. For a quick-start compose this is an unusually large limit and increases the risk of excessive memory use / oversized request handling. Since the comment says this is optional, consider omitting it by default or using a much smaller, documented value. ########## scripts/test-quick-start-local.sh: ########## @@ -0,0 +1,143 @@ +#!/usr/bin/env bash +# Licensed to Apache Software Foundation (ASF) under one or more contributor +# license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright +# ownership. Apache Software Foundation (ASF) licenses this file to you under +# the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Local test script for the quick-start workflow. +# Run this to verify the quick-start docker-compose setup without using act/GitHub. + +set -e + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +QUICK_START_DIR="$REPO_ROOT/docs/guides/quick-start" + +cd "$REPO_ROOT" + +echo "=== Free disk space ===" +df -h +docker system prune -af --volumes || true +df -h + +echo "=== Pull images ===" +cd "$QUICK_START_DIR" +PULL_RETRIES=5 +for i in $(seq 1 "$PULL_RETRIES"); do + if docker compose pull; then + break + fi + if [ "$i" -eq "$PULL_RETRIES" ]; then + echo "ERROR: docker compose pull failed after $PULL_RETRIES attempts" + exit 1 + fi + echo "docker compose pull failed, retrying in 10 seconds (attempt $i of $PULL_RETRIES)..." + sleep 10 +done + +echo "=== Start the stack ===" +docker compose up -d +sleep 10 + +echo "=== Check container status ===" +docker compose ps + +echo "=== Wait for BanyanDB to start ===" +for i in $(seq 1 30); do + if curl -sf "http://localhost:17913/api/healthz"; > /dev/null 2>&1; then + echo "BanyanDB is ready" + break + fi + sleep 2 +done +curl -sf "http://localhost:17913/api/healthz"; > /dev/null || { echo "BanyanDB not ready"; exit 1; } + +echo "=== Wait for OAP to start ===" +for i in $(seq 1 30); do + if curl -sf "http://localhost:12800/internal/v3/health"; > /dev/null 2>&1 || curl -sf "http://localhost:12800/graphql"; -X POST -H "Content-Type: application/json" -d '{"query":"{ __typename }"}' > /dev/null 2>&1; then + echo "OAP is ready" + break + fi + sleep 2 +done +curl -sf "http://localhost:12800/graphql"; -X POST -H "Content-Type: application/json" -d '{"query":"{ __typename }"}' > /dev/null || { echo "OAP not ready"; exit 1; } + +echo "=== Wait for services to initialize ===" +sleep 60 +docker compose ps + +echo "=== Verify BanyanDB is running ===" +docker logs banyandb 2>&1 | tail -20 +curl -sf "http://localhost:17913/api/healthz"; > /dev/null || { echo "BanyanDB not reachable"; exit 1; } + +echo "=== Verify OAP is running ===" +docker logs skywalking-oap 2>&1 | grep -i "BanyanDB\|Health status" | tail -10 +curl -sf "http://localhost:12800/graphql"; -X POST -H "Content-Type: application/json" -d '{"query":"{ __typename }"}' > /dev/null || { echo "OAP not reachable"; exit 1; } + +echo "=== Verify UI is running ===" +curl -s http://localhost:8080/ | head -5 +curl -sf "http://localhost:8080/"; > /dev/null + +echo "=== Verify demo services ===" +docker compose ps | grep -E "provider|consumer|traffic_loader" + +echo "=== Check OAP logs for BanyanDB connection ===" +docker logs skywalking-oap 2>&1 | grep -i "banyandb" | tail -5 + +echo "=== Install swctl ===" +# shellcheck source=test/e2e-v2/script/env +set -a +. "$REPO_ROOT/test/e2e-v2/script/env" +set +a +bash "$REPO_ROOT/test/e2e-v2/script/prepare/setup-e2e-shell/install-swctl.sh" /tmp /usr/local +swctl --version + Review Comment: `install-swctl.sh` is invoked with `DESTDIR=/usr/local`, which commonly requires elevated permissions on developer machines. For a local helper script, consider installing into a user-writable directory (e.g., `$HOME/.local`) and updating `PATH`, to avoid requiring sudo. ########## docs/guides/quick-start/quick-start.md: ########## @@ -1,10 +1,112 @@ # Quick Start Tutorial + The following tutorial will guide you through setting up a SkyWalking OAP with BanyanDB as the storage backend using Docker Compose. It is a quick way to get started with BanyanDB if you are a SkyWalking user and want to try out BanyanDB. -## Set up quickstart cluster with Showcase +## Option 1: Use the Docker Compose Example (Recommended) + +This repository includes a ready-to-use docker-compose file at [docker-compose.yaml](docker-compose.yaml) that sets up: + +- **BanyanDB** - Standalone instance with etcd schema registry +- **SkyWalking OAP** - Configured to use BanyanDB as storage +- **SkyWalking UI** - Web interface for visualizing traces, metrics, and topology +- **Demo Services** - Sample microservices that generate trace traffic automatically + +### Prerequisites + +- Docker Engine 20.10+ +- Docker Compose v2.0+ + +### Quick Start + +1. Navigate to the quick-start directory: + ```shell + cd docs/guides/quick-start + ``` + +2. Start the stack: + ```shell + docker compose up -d + ``` + +3. Wait for all services to be healthy (may take 1-2 minutes): + ```shell + docker compose ps + ``` + +### Accessing the Services + +| Service | URL | Description | +|---------|-----|-------------| +| SkyWalking UI | http://localhost:8080 | Web interface for traces, metrics, topology | +| BanyanDB gRPC | localhost:17913 | gRPC endpoint for bydbctl | +| BanyanDB UI | http://localhost:17913 | Embedded UI for querying data | +| OAP HTTP | http://localhost:12800 | OAP REST API | +| OAP gRPC | localhost:11800 | gRPC endpoint for agents | Review Comment: The service table mislabels `localhost:17913` as “BanyanDB gRPC”. 17913 is the BanyanDB HTTP endpoint (used by bydbctl and the embedded UI); the gRPC port is 17912 and is not published in the provided docker-compose.yaml. Please correct the table (and the related health check example) to avoid confusing users. ########## docs/api-reference.md: ########## @@ -5607,7 +5607,7 @@ WriteResponse is the response contract for write | event_type | [SchemaEventType](#banyandb-schema-v1-SchemaEventType) | | | | property | [banyandb.property.v1.Property](#banyandb-property-v1-Property) | | | | metadata_only | [bool](#bool) | | | -| delete_time | [int64](#int64) | | delete_time is the deletion timestamp in nanoseconds. 0 means not deleted, >0 means the property was deleted at this time. | +| delete_time | [int64](#int64) | | | Review Comment: The `delete_time` field description was removed, leaving an empty description in the generated API reference. If this was unintentional, consider restoring the field comment in the corresponding proto and regenerating `docs/api-reference.md` so consumers understand the semantics (e.g., 0 = not deleted, >0 = deletion timestamp). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
