This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/skywalking-banyandb-helm.git
The following commit(s) were added to refs/heads/master by this push: new 70694ad Several Important Feats and Fixes (#35) 70694ad is described below commit 70694adbfe723c9d69036ef3bb78871a4f0215f1 Author: Gao Hongtao <hanahm...@gmail.com> AuthorDate: Thu Sep 11 09:46:28 2025 +0800 Several Important Feats and Fixes (#35) * Add trace mount target support for persistent volume claims - Introduced a new mount target for trace mode in the storage configuration. - Updated relevant YAML files to include trace as a mount target for data, liaison, and standalone persistent volume claims. * Add lifecycle sidecar configuration options - Introduced new parameters for lifecycle sidecar in `values-lifecycle.yaml` and `values.yaml` to specify `progressFile` and `reportDir`. - Updated the StatefulSet template to conditionally include these parameters in the lifecycle command. - Enhanced documentation in `parameters.md` to reflect the new configuration options. - Updated E2E test values to include the new lifecycle sidecar parameters for testing. * Add authentication support for BanyanDB - Introduced basic authentication configuration in `values.yaml` and `parameters.md`, allowing users to enable authentication and specify credentials. - Updated StatefulSet templates to mount the credentials file and pass the `--auth-config-file` argument to the BanyanDB containers. - Enhanced documentation in `README.md` and `NOTES.txt` to guide users on configuring authentication. - Updated E2E test configurations to include authentication parameters for testing. This change improves security by allowing users to manage access to BanyanDB instances. --- .github/workflows/e2e.ci.yaml | 4 +- CHANGES.md | 2 + README.md | 35 ++++++++ chart/templates/NOTES.txt | 15 ++++ chart/templates/{NOTES.txt => auth_secret.yaml} | 18 +++- chart/templates/cluster_data_statefulset.yaml | 6 ++ chart/templates/cluster_liaison_statefulset.yaml | 56 +++++++++++-- chart/templates/standalone_statefulset.yaml | 56 ++++++++++++- chart/values-lifecycle.yaml | 28 ++++++- chart/values.yaml | 47 +++++++++++ doc/parameters.md | 100 +++++++++++++---------- test/e2e/e2e-banyandb-cluster.yaml | 2 + test/e2e/e2e-banyandb-standalone.yaml | 2 + test/e2e/values.cluster.yaml | 16 +++- test/e2e/values.lifecycle.yaml | 35 +++++++- test/e2e/values.standalone.yaml | 19 +++-- 16 files changed, 372 insertions(+), 69 deletions(-) diff --git a/.github/workflows/e2e.ci.yaml b/.github/workflows/e2e.ci.yaml index e11bf3a..ea0fb9f 100644 --- a/.github/workflows/e2e.ci.yaml +++ b/.github/workflows/e2e.ci.yaml @@ -43,9 +43,9 @@ jobs: config: test/e2e/e2e-banyandb-lifecycle.yaml name: ${{ matrix.test.name }} env: - OAP_TAG: bf04afdb2a841c60d5e27f5a9fc62d0879a5600c + OAP_TAG: 79860ca5c76a77bbd93e76ce4861b24707dd5ee3 OAP_REPO: ghcr.io/apache/skywalking/oap - UI_TAG: bf04afdb2a841c60d5e27f5a9fc62d0879a5600c + UI_TAG: 79860ca5c76a77bbd93e76ce4861b24707dd5ee3 UI_REPO: ghcr.io/apache/skywalking/ui steps: - uses: actions/checkout@v2 diff --git a/CHANGES.md b/CHANGES.md index 8015e14..13f2543 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -16,6 +16,8 @@ Release Notes. - Enable etcd defragmentation by default with daily scheduling (0 0 * * *) to maintain optimal etcd performance - Enhance pod hostname configuration using headless services for improved service discovery and networking - Implement volume permissions init containers for proper file ownership and permissions on mounted volumes +- Add the mount target for the trace mode +- Add `auth` to configure the basic credential file. #### Chores diff --git a/README.md b/README.md index 0274796..b918a54 100644 --- a/README.md +++ b/README.md @@ -90,6 +90,41 @@ $ helm install my-release \ > **Tip**: You can use the default [values.yaml](chart/values.yaml) +## Authentication + +Basic authentication can be enabled for liaison and standalone modes. When enabled, the chart mounts a credentials file and adds `--auth-config-file` to BanyanDB. + +- Enable auth and provide users (plaintext passwords): + +```yaml +auth: + enabled: true + users: + - username: admin + password: "changeme" +``` + +- Use an existing Secret (recommended for production). The Secret must contain a key `credentials.yaml` (configurable via `auth.credentialsFileKey`) whose value is the YAML content in the format required by BanyanDB ([docs](https://github.com/apache/skywalking-banyandb/blob/main/docs/operation/security.md#basic-authentication)): + +```yaml +auth: + enabled: true + existingSecret: my-banyandb-auth + credentialsFileKey: credentials.yaml +``` + +The Secret name defaults to `<release>-banyandb-auth` when auto-created. + +### Retrieve credentials after install + +If the chart created the Secret (no `auth.existingSecret`), you can decode it: + +```bash +kubectl get secret <release-name>-banyandb-auth -n <namespace> -o jsonpath='{.data.credentials\.yaml}' | base64 --decode +``` + +Adjust the key if you changed `auth.credentialsFileKey`. + ## Use external certificate authorities for TLS If you'd like to use external certificate authorities, such as Vault, corresponding annotations can be injected into [banyandb](./chart/templates/statefulset.yaml). diff --git a/chart/templates/NOTES.txt b/chart/templates/NOTES.txt index 90d1748..dc20101 100644 --- a/chart/templates/NOTES.txt +++ b/chart/templates/NOTES.txt @@ -1,3 +1,18 @@ +{{- if .Values.auth.enabled }} +1. BanyanDB authentication is enabled. + + Credentials file is mounted into liaison/standalone and picked by `--auth-config-file`. + +{{- $useExisting := ne .Values.auth.existingSecret "" -}} +{{- $secretName := ternary .Values.auth.existingSecret (printf "%s-auth" (include "banyandb.fullname" .)) $useExisting -}} +{{- $key := .Values.auth.credentialsFileKey -}} + +2. To view the credentials YAML stored in the Secret: + + kubectl get secret {{ $secretName }} -n {{ .Release.Namespace }} -o jsonpath='{.data.{{- printf "%s" $key | replace "." "\\." -}}}' | base64 --decode | sed -e 's/^/ /' + + Note: adjust the key if you changed `auth.credentialsFileKey`. +{{- end }} {{/* Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with diff --git a/chart/templates/NOTES.txt b/chart/templates/auth_secret.yaml similarity index 59% copy from chart/templates/NOTES.txt copy to chart/templates/auth_secret.yaml index 90d1748..25342fe 100644 --- a/chart/templates/NOTES.txt +++ b/chart/templates/auth_secret.yaml @@ -13,4 +13,20 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -*/}} \ No newline at end of file +*/}} + +{{- if and .Values.auth.enabled (not .Values.auth.existingSecret) }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "banyandb.fullname" . }}-auth + labels: {{ include "banyandb.labels" . | nindent 4 }} +type: Opaque +data: + {{- $users := .Values.auth.users }} + {{- if not $users }} + {{- $users = list (dict "username" "admin" "password" (randAlphaNum 16)) }} + {{- end }} + {{- $cred := (dict "users" $users | toYaml | b64enc) }} + {{ .Values.auth.credentialsFileKey }}: "{{ $cred }}" +{{- end }} diff --git a/chart/templates/cluster_data_statefulset.yaml b/chart/templates/cluster_data_statefulset.yaml index 3fc3163..84aa009 100644 --- a/chart/templates/cluster_data_statefulset.yaml +++ b/chart/templates/cluster_data_statefulset.yaml @@ -361,6 +361,12 @@ spec: command: - "/lifecycle" - "--schedule={{ $roleConfig.lifecycleSidecar.schedule }}" + {{- if $roleConfig.lifecycleSidecar.progressFile }} + - "--progress-file={{ $roleConfig.lifecycleSidecar.progressFile }}" + {{- end }} + {{- if $roleConfig.lifecycleSidecar.reportDir }} + - "--report-dir={{ $roleConfig.lifecycleSidecar.reportDir }}" + {{- end }} {{- if $.Values.storage.data.enabled }} volumeMounts: {{- range $claim := $.Values.storage.data.persistentVolumeClaims }} diff --git a/chart/templates/cluster_liaison_statefulset.yaml b/chart/templates/cluster_liaison_statefulset.yaml index 0c1ad4e..80f0c3b 100644 --- a/chart/templates/cluster_liaison_statefulset.yaml +++ b/chart/templates/cluster_liaison_statefulset.yaml @@ -45,7 +45,7 @@ spec: {{- toYaml . | nindent 8 }} {{- end }} priorityClassName: {{ .Values.cluster.liaison.priorityClassName }} - {{- if and .Values.storage.liaison.enabled .Values.cluster.liaison.volumePermissions.enabled }} + {{- if or (and .Values.storage.liaison.enabled .Values.cluster.liaison.volumePermissions.enabled) .Values.auth.enabled }} initContainers: - name: volume-permissions image: {{ default "busybox:1.36" .Values.cluster.liaison.volumePermissions.image }} @@ -72,6 +72,29 @@ spec: name: {{ $claim.claimName }} {{- end }} {{- end }} + {{- if .Values.auth.enabled }} + - name: auth-config-prepare + image: busybox:1.36 + imagePullPolicy: IfNotPresent + securityContext: + runAsUser: 0 + command: + - sh + - -c + - | + set -euo pipefail + cp "/var/run/banyandb-auth/{{ .Values.auth.credentialsFileKey }}" "/etc/banyandb/{{ .Values.auth.credentialsFileKey }}" + CHOWN_UID={{ default 1000 .Values.cluster.liaison.securityContext.runAsUser }} + CHOWN_GID={{ default 1000 .Values.cluster.liaison.securityContext.runAsGroup }} + chown ${CHOWN_UID}:${CHOWN_GID} "/etc/banyandb/{{ .Values.auth.credentialsFileKey }}" + chmod 0600 "/etc/banyandb/{{ .Values.auth.credentialsFileKey }}" + volumeMounts: + - mountPath: /var/run/banyandb-auth + name: banyandb-auth + readOnly: true + - mountPath: /etc/banyandb + name: banyandb-auth-writable + {{- end }} containers: - name: liaison {{- if eq .Values.cluster.ui.type "Embedded" }} @@ -125,11 +148,11 @@ spec: - name: BYDB_ETCD_PASSWORD value: {{ .Values.etcd.auth.rbac.rootPassword }} {{- end }} - {{- if .Values.etcd.auth.client.secureTransport }} + {{- if and .Values.cluster.liaison.tls .Values.cluster.liaison.tls.etcdSecretName .Values.etcd.auth.client.secureTransport }} - name: BYDB_ETCD_TLS_CA_FILE value: "/etc/tls/{{ .Values.cluster.liaison.tls.etcdSecretName }}/ca.crt" {{- end }} - {{- if .Values.etcd.auth.client.enableAuthentication }} + {{- if and .Values.cluster.liaison.tls .Values.cluster.liaison.tls.etcdSecretName .Values.etcd.auth.client.enableAuthentication }} - name: BYDB_ETCD_TLS_CERT_FILE value: "/etc/tls/{{ .Values.cluster.liaison.tls.etcdSecretName }}/tls.crt" - name: BYDB_ETCD_TLS_KEY_FILE @@ -147,6 +170,9 @@ spec: {{- end }} args: - liaison + {{- if .Values.auth.enabled }} + - --auth-config-file=/etc/banyandb/{{ .Values.auth.credentialsFileKey }} + {{- end }} ports: - containerPort: 17912 name: grpc @@ -216,7 +242,7 @@ spec: {{- end }} {{- end }} - {{- if or .Values.storage.liaison.enabled .Values.cluster.liaison.tls }} + {{- if or .Values.storage.liaison.enabled .Values.cluster.liaison.tls .Values.auth.enabled }} volumeMounts: {{- if .Values.storage.liaison.enabled }} {{- range $claim := .Values.storage.liaison.persistentVolumeClaims }} @@ -241,10 +267,15 @@ spec: name: {{ .Values.cluster.liaison.tls.httpSecretName }}-volume {{- end }} {{- end }} + {{- if .Values.auth.enabled }} + - mountPath: /etc/banyandb + name: banyandb-auth-writable + {{- end }} {{- end }} - {{- if .Values.cluster.liaison.tls }} + {{- if or .Values.cluster.liaison.tls .Values.auth.enabled }} volumes: + {{- if .Values.cluster.liaison.tls }} {{- if .Values.cluster.liaison.tls.grpcSecretName }} - name: {{ .Values.cluster.liaison.tls.grpcSecretName }}-volume secret: @@ -260,6 +291,21 @@ spec: secret: secretName: {{ .Values.cluster.liaison.tls.httpSecretName }} {{- end }} + {{- end }} + {{- if .Values.auth.enabled }} + {{- $useExisting := ne .Values.auth.existingSecret "" }} + {{- $authSecret := ternary .Values.auth.existingSecret (printf "%s-auth" (include "banyandb.fullname" .)) $useExisting }} + - name: banyandb-auth + secret: + secretName: {{ $authSecret }} + defaultMode: 0600 + items: + - key: {{ .Values.auth.credentialsFileKey }} + path: {{ .Values.auth.credentialsFileKey }} + mode: 0600 + - name: banyandb-auth-writable + emptyDir: {} + {{- end }} {{- end }} {{- if .Values.cluster.liaison.tolerations }} diff --git a/chart/templates/standalone_statefulset.yaml b/chart/templates/standalone_statefulset.yaml index ebfcdb1..0864e43 100644 --- a/chart/templates/standalone_statefulset.yaml +++ b/chart/templates/standalone_statefulset.yaml @@ -43,8 +43,9 @@ spec: {{- toYaml . | nindent 8 }} {{- end }} priorityClassName: {{ .Values.standalone.priorityClassName }} - {{- if and .Values.storage.standalone.enabled .Values.standalone.volumePermissions.enabled }} + {{- if or (and .Values.storage.standalone.enabled .Values.standalone.volumePermissions.enabled) .Values.auth.enabled }} initContainers: + {{- if and .Values.storage.standalone.enabled .Values.standalone.volumePermissions.enabled }} - name: volume-permissions image: {{ default "busybox:1.36" .Values.standalone.volumePermissions.image }} imagePullPolicy: IfNotPresent @@ -69,6 +70,30 @@ spec: - mountPath: /mnt/{{ $claim.claimName }} name: {{ $claim.claimName }} {{- end }} + {{- end }} + {{- if .Values.auth.enabled }} + - name: auth-config-prepare + image: busybox:1.36 + imagePullPolicy: IfNotPresent + securityContext: + runAsUser: 0 + command: + - sh + - -c + - | + set -euo pipefail + cp "/var/run/banyandb-auth/{{ .Values.auth.credentialsFileKey }}" "/etc/banyandb/{{ .Values.auth.credentialsFileKey }}" + CHOWN_UID={{ default 1000 .Values.standalone.securityContext.runAsUser }} + CHOWN_GID={{ default 1000 .Values.standalone.securityContext.runAsGroup }} + chown ${CHOWN_UID}:${CHOWN_GID} "/etc/banyandb/{{ .Values.auth.credentialsFileKey }}" + chmod 0600 "/etc/banyandb/{{ .Values.auth.credentialsFileKey }}" + volumeMounts: + - mountPath: /var/run/banyandb-auth + name: banyandb-auth + readOnly: true + - mountPath: /etc/banyandb + name: banyandb-auth-writable + {{- end }} {{- end }} containers: - name: standalone @@ -105,6 +130,9 @@ spec: {{- end }} args: - standalone + {{- if .Values.auth.enabled }} + - --auth-config-file=/etc/banyandb/{{ .Values.auth.credentialsFileKey }} + {{- end }} ports: - containerPort: 17912 name: grpc @@ -172,7 +200,7 @@ spec: {{- end }} {{- end }} - {{- if or .Values.storage.standalone.enabled .Values.standalone.tls }} + {{- if or .Values.storage.standalone.enabled .Values.standalone.tls .Values.auth.enabled }} volumeMounts: {{- if .Values.storage.standalone.enabled }} {{- range $claim := .Values.storage.standalone.persistentVolumeClaims }} @@ -202,20 +230,40 @@ spec: name: {{ .Values.standalone.tls.httpSecretName }}-volume {{- end }} {{- end -}} + {{- if .Values.auth.enabled }} + - mountPath: /etc/banyandb + name: banyandb-auth-writable + {{- end }} {{- end }} - {{- if .Values.standalone.tls }} + {{- if or .Values.standalone.tls .Values.auth.enabled }} volumes: + {{- if .Values.standalone.tls }} {{- if .Values.standalone.tls.grpcSecretName }} - name: {{ .Values.standalone.tls.grpcSecretName }}-volume secret: secretName: {{ .Values.standalone.tls.grpcSecretName }} {{- end }} - {{- if and .Values.standalone.tls.httpSecretName (ne .Values.standalone.tls.httpSecretName .Values.standalone.tls.grpcSecretName) }} + {{- if and .Values.standalone.tls.httpSecretName (or (not .Values.standalone.tls.grpcSecretName) (ne .Values.standalone.tls.httpSecretName .Values.standalone.tls.grpcSecretName)) }} - name: {{ .Values.standalone.tls.httpSecretName }}-volume secret: secretName: {{ .Values.standalone.tls.httpSecretName }} {{- end }} + {{- end }} + {{- if .Values.auth.enabled }} + {{- $useExisting := ne .Values.auth.existingSecret "" }} + {{- $authSecret := ternary .Values.auth.existingSecret (printf "%s-auth" (include "banyandb.fullname" .)) $useExisting }} + - name: banyandb-auth + secret: + secretName: {{ $authSecret }} + defaultMode: 0600 + items: + - key: {{ .Values.auth.credentialsFileKey }} + path: {{ .Values.auth.credentialsFileKey }} + mode: 0600 + - name: banyandb-auth-writable + emptyDir: {} + {{- end }} {{- end }} {{- if .Values.standalone.tolerations }} diff --git a/chart/values-lifecycle.yaml b/chart/values-lifecycle.yaml index b801608..2bf00a0 100644 --- a/chart/values-lifecycle.yaml +++ b/chart/values-lifecycle.yaml @@ -473,6 +473,12 @@ cluster: ## @param cluster.data.nodeTemplate.lifecycleSidecar.schedule Schedule for lifecycle sidecar (cron format) ## schedule: "@hourly" + ## @param cluster.data.nodeTemplate.lifecycleSidecar.progressFile Progress file path for lifecycle sidecar + ## + progressFile: "/tmp/stream/lifecycle/progress.json" + ## @param cluster.data.nodeTemplate.lifecycleSidecar.reportDir Report directory path for lifecycle sidecar + ## + reportDir: "/tmp/stream/lifecycle/reports" ## @param cluster.data.nodeTemplate.lifecycleSidecar.resources Resources for lifecycle sidecar for data pods ## resources: {} @@ -774,6 +780,24 @@ storage: storageClass: null ## @param storage.data.persistentVolumeClaims[2].volumeMode Volume mode for the PVC volumeMode: Filesystem + ## @param storage.data.persistentVolumeClaims[3].mountTargets Mount targets for the PVC + - mountTargets: [ "trace" ] + ## @param storage.data.persistentVolumeClaims[3].nodeRole Node role this PVC is bound to + ## + nodeRole: hot + ## @param storage.data.persistentVolumeClaims[3].existingClaimName Existing PVC name (if any) + existingClaimName: null + ## @param storage.data.persistentVolumeClaims[3].claimName Name of the PVC + claimName: hot-trace-data + ## @param storage.data.persistentVolumeClaims[3].size Size of the PVC + size: 50Gi + ## @param storage.data.persistentVolumeClaims[3].accessModes Access modes for the PVC + accessModes: + - ReadWriteOnce + ## @param storage.data.persistentVolumeClaims[3].storageClass Storage class for the PVC + storageClass: null + ## @param storage.data.persistentVolumeClaims[3].volumeMode Volume mode for the PVC + volumeMode: Filesystem ## Liaison storage configuration ## @@ -786,7 +810,7 @@ storage: persistentVolumeClaims: ## @param storage.liaison.persistentVolumeClaims[0].mountTargets Mount targets for the PVC ## - - mountTargets: [ "measure", "stream" ] + - mountTargets: [ "measure", "stream", "trace" ] ## @param storage.liaison.persistentVolumeClaims[0].claimName Name of the PVC claimName: liaison-data ## @param storage.liaison.persistentVolumeClaims[0].size Size of the PVC @@ -810,7 +834,7 @@ storage: persistentVolumeClaims: ## @param storage.standalone.persistentVolumeClaims[0].mountTargets Mount targets for the PVC ## - - mountTargets: [ "measure", "stream", "property" ] + - mountTargets: [ "measure", "stream", "property", "trace" ] ## @param storage.standalone.persistentVolumeClaims[0].claimName Name of the PVC claimName: standalone-data ## @param storage.standalone.persistentVolumeClaims[0].size Size of the PVC diff --git a/chart/values.yaml b/chart/values.yaml index 2d91078..8bce7d5 100644 --- a/chart/values.yaml +++ b/chart/values.yaml @@ -40,6 +40,40 @@ image: ## pullPolicy: IfNotPresent +## @section Authentication configuration for BanyanDB +## +auth: + ## @param auth.enabled Enable basic authentication (boolean) + ## Defaults to false. When enabled, the chart mounts a credentials file and + ## passes --auth-config-file to liaison/standalone. + ## + enabled: false + + ## @param auth.existingSecret Use an existing Secret for credentials + ## If set, the chart will NOT create a Secret and will mount this Secret + ## instead. The Secret must contain a key specified by auth.credentialsFileKey + ## whose value is the YAML-formatted credentials content. + ## + existingSecret: "" + + ## @param auth.credentialsFileKey Key name in the Secret that stores the + ## credentials YAML. This filename will also be used inside the pod via subPath. + ## + credentialsFileKey: "credentials.yaml" + + ## @param auth.users List of users to configure when not using existingSecret + ## Each user item must include plaintext fields: username, password. + ## If empty and auth.enabled=true, a default 'admin' user with a random + ## password will be generated. + ## Example: + ## users: + ## - username: admin + ## password: "changeme" + ## - username: viewer + ## password: "123456" + ## + users: [] + ## @section Configuration for standalone deployment ## standalone: @@ -55,6 +89,13 @@ standalone: ## @param standalone.containerSecurityContext Container-level security context ## e.g. { readOnlyRootFilesystem: true, allowPrivilegeEscalation: false, runAsNonRoot: true } containerSecurityContext: {} + ## @param standalone.tls TLS configuration for the standalone pod + ## When set, provide optional secret names to mount TLS materials. + ## Example: + ## tls: + ## grpcSecretName: "my-grpc-tls" + ## httpSecretName: "my-http-tls" + tls: {} ## Volume permissions init container ## @param standalone.volumePermissions.enabled Enable volume permissions init container volumePermissions: @@ -471,6 +512,12 @@ cluster: ## @param cluster.data.nodeTemplate.lifecycleSidecar.schedule Schedule for lifecycle sidecar (cron format) ## schedule: "@hourly" + ## @param cluster.data.nodeTemplate.lifecycleSidecar.progressFile Progress file path for lifecycle sidecar + ## + progressFile: "" + ## @param cluster.data.nodeTemplate.lifecycleSidecar.reportDir Report directory path for lifecycle sidecar + ## + reportDir: "" ## @param cluster.data.nodeTemplate.lifecycleSidecar.resources Resources for lifecycle sidecar for data pods ## resources: {} diff --git a/doc/parameters.md b/doc/parameters.md index 22e7b8d..b33c469 100644 --- a/doc/parameters.md +++ b/doc/parameters.md @@ -19,6 +19,15 @@ The content of this document describes the parameters that can be configured in | `image.tag` | Image tag/version (empty for latest) | `""` | | `image.pullPolicy` | Image pull policy (e.g. IfNotPresent) | `IfNotPresent` | +### Authentication configuration for BanyanDB + +| Name | Description | Value | +| ------------------------- | -------------------------------------------------------- | ------------------ | +| `auth.enabled` | Enable basic authentication (boolean) | `false` | +| `auth.existingSecret` | Use an existing Secret for credentials | `""` | +| `auth.credentialsFileKey` | Key name in the Secret that stores the | `credentials.yaml` | +| `auth.users` | List of users to configure when not using existingSecret | `[]` | + ### Configuration for standalone deployment | Name | Description | Value | @@ -27,6 +36,7 @@ The content of this document describes the parameters that can be configured in | `standalone.podAnnotations` | Additional pod annotations | `{}` | | `standalone.securityContext` | Security context for the pod | `{}` | | `standalone.containerSecurityContext` | Container-level security context | `{}` | +| `standalone.tls` | TLS configuration for the standalone pod | `{}` | | `standalone.volumePermissions.enabled` | Enable volume permissions init container | `false` | | `standalone.volumePermissions.chownUser` | User ID to chown the mounted volumes | `1000` | | `standalone.volumePermissions.chownGroup` | Group ID to chown the mounted volumes | `1000` | @@ -167,6 +177,8 @@ The content of this document describes the parameters that can be configured in | `cluster.data.nodeTemplate.backupSidecar.resources` | Resources for backup sidecar for data pods | `{}` | | `cluster.data.nodeTemplate.lifecycleSidecar.enabled` | Enable lifecycle sidecar for data pods (boolean) | `false` | | `cluster.data.nodeTemplate.lifecycleSidecar.schedule` | Schedule for lifecycle sidecar (cron format) | `@hourly` | +| `cluster.data.nodeTemplate.lifecycleSidecar.progressFile` | Progress file path for lifecycle sidecar | `""` | +| `cluster.data.nodeTemplate.lifecycleSidecar.reportDir` | Report directory path for lifecycle sidecar | `""` | | `cluster.data.nodeTemplate.lifecycleSidecar.resources` | Resources for lifecycle sidecar for data pods | `{}` | | `cluster.data.nodeTemplate.restoreInitContainer.enabled` | Enable restore init container for data pods (boolean) | `false` | | `cluster.data.nodeTemplate.restoreInitContainer.customFlags` | Custom flags for restore init container (e.g., S3, Azure, GCS configuration) | `[]` | @@ -240,50 +252,50 @@ The content of this document describes the parameters that can be configured in ### Storage configuration for persistent volumes -| Name | Description | Value | -| ----------------------------------------------------------- | ------------------------------------------------------- | --------------------------------- | -| `storage.data.enabled` | Enable persistent storage for data nodes (boolean) | `true` | -| `storage.data.persistentVolumeClaims` | List of PVC configurations for data nodes | | -| `storage.data.persistentVolumeClaims[0].mountTargets` | Mount targets for the PVC | `["measure"]` | -| `storage.data.persistentVolumeClaims[0].nodeRole` | Node role this PVC is bound to (hot, warm, cold) | `hot` | -| `storage.data.persistentVolumeClaims[0].existingClaimName` | Existing PVC name (if any) | `nil` | -| `storage.data.persistentVolumeClaims[0].claimName` | Name of the PVC | `hot-measure-data` | -| `storage.data.persistentVolumeClaims[0].size` | Size of the PVC | `50Gi` | -| `storage.data.persistentVolumeClaims[0].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | -| `storage.data.persistentVolumeClaims[0].storageClass` | Storage class for the PVC | `nil` | -| `storage.data.persistentVolumeClaims[0].volumeMode` | Volume mode for the PVC | `Filesystem` | -| `storage.data.persistentVolumeClaims[1].mountTargets` | Mount targets for the PVC | `["stream"]` | -| `storage.data.persistentVolumeClaims[1].nodeRole` | Node role this PVC is bound to | `hot` | -| `storage.data.persistentVolumeClaims[1].existingClaimName` | Existing PVC name (if any) | `nil` | -| `storage.data.persistentVolumeClaims[1].claimName` | Name of the PVC | `hot-stream-data` | -| `storage.data.persistentVolumeClaims[1].size` | Size of the PVC | `50Gi` | -| `storage.data.persistentVolumeClaims[1].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | -| `storage.data.persistentVolumeClaims[1].storageClass` | Storage class for the PVC | `nil` | -| `storage.data.persistentVolumeClaims[1].volumeMode` | Volume mode for the PVC | `Filesystem` | -| `storage.data.persistentVolumeClaims[2].mountTargets` | Mount targets for the PVC | `["property"]` | -| `storage.data.persistentVolumeClaims[2].nodeRole` | Node role this PVC is bound to | `hot` | -| `storage.data.persistentVolumeClaims[2].existingClaimName` | Existing PVC name (if any) | `nil` | -| `storage.data.persistentVolumeClaims[2].claimName` | Name of the PVC | `hot-property-data` | -| `storage.data.persistentVolumeClaims[2].size` | Size of the PVC | `5Gi` | -| `storage.data.persistentVolumeClaims[2].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | -| `storage.data.persistentVolumeClaims[2].storageClass` | Storage class for the PVC | `nil` | -| `storage.data.persistentVolumeClaims[2].volumeMode` | Volume mode for the PVC | `Filesystem` | -| `storage.liaison.enabled` | Enable persistent storage for liaison nodes (boolean) | `true` | -| `storage.liaison.persistentVolumeClaims` | List of PVC configurations for liaison nodes | | -| `storage.liaison.persistentVolumeClaims[0].mountTargets` | Mount targets for the PVC | `["measure","stream"]` | -| `storage.liaison.persistentVolumeClaims[0].claimName` | Name of the PVC | `liaison-data` | -| `storage.liaison.persistentVolumeClaims[0].size` | Size of the PVC | `10Gi` | -| `storage.liaison.persistentVolumeClaims[0].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | -| `storage.liaison.persistentVolumeClaims[0].storageClass` | Storage class for the PVC | `nil` | -| `storage.liaison.persistentVolumeClaims[0].volumeMode` | Volume mode for the PVC | `Filesystem` | -| `storage.standalone.enabled` | Enable persistent storage for standalone mode (boolean) | `false` | -| `storage.standalone.persistentVolumeClaims` | List of PVC configurations for standalone | | -| `storage.standalone.persistentVolumeClaims[0].mountTargets` | Mount targets for the PVC | `["measure","stream","property"]` | -| `storage.standalone.persistentVolumeClaims[0].claimName` | Name of the PVC | `standalone-data` | -| `storage.standalone.persistentVolumeClaims[0].size` | Size of the PVC | `200Gi` | -| `storage.standalone.persistentVolumeClaims[0].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | -| `storage.standalone.persistentVolumeClaims[0].storageClass` | Storage class for the PVC | `nil` | -| `storage.standalone.persistentVolumeClaims[0].volumeMode` | Volume mode for the PVC | `Filesystem` | +| Name | Description | Value | +| ----------------------------------------------------------- | ------------------------------------------------------- | -------------------------------------------- | +| `storage.data.enabled` | Enable persistent storage for data nodes (boolean) | `true` | +| `storage.data.persistentVolumeClaims` | List of PVC configurations for data nodes | | +| `storage.data.persistentVolumeClaims[0].mountTargets` | Mount targets for the PVC | `["measure"]` | +| `storage.data.persistentVolumeClaims[0].nodeRole` | Node role this PVC is bound to (hot, warm, cold) | `hot` | +| `storage.data.persistentVolumeClaims[0].existingClaimName` | Existing PVC name (if any) | `nil` | +| `storage.data.persistentVolumeClaims[0].claimName` | Name of the PVC | `hot-measure-data` | +| `storage.data.persistentVolumeClaims[0].size` | Size of the PVC | `50Gi` | +| `storage.data.persistentVolumeClaims[0].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | +| `storage.data.persistentVolumeClaims[0].storageClass` | Storage class for the PVC | `nil` | +| `storage.data.persistentVolumeClaims[0].volumeMode` | Volume mode for the PVC | `Filesystem` | +| `storage.data.persistentVolumeClaims[1].mountTargets` | Mount targets for the PVC | `["stream"]` | +| `storage.data.persistentVolumeClaims[1].nodeRole` | Node role this PVC is bound to | `hot` | +| `storage.data.persistentVolumeClaims[1].existingClaimName` | Existing PVC name (if any) | `nil` | +| `storage.data.persistentVolumeClaims[1].claimName` | Name of the PVC | `hot-stream-data` | +| `storage.data.persistentVolumeClaims[1].size` | Size of the PVC | `50Gi` | +| `storage.data.persistentVolumeClaims[1].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | +| `storage.data.persistentVolumeClaims[1].storageClass` | Storage class for the PVC | `nil` | +| `storage.data.persistentVolumeClaims[1].volumeMode` | Volume mode for the PVC | `Filesystem` | +| `storage.data.persistentVolumeClaims[2].mountTargets` | Mount targets for the PVC | `["property"]` | +| `storage.data.persistentVolumeClaims[2].nodeRole` | Node role this PVC is bound to | `hot` | +| `storage.data.persistentVolumeClaims[2].existingClaimName` | Existing PVC name (if any) | `nil` | +| `storage.data.persistentVolumeClaims[2].claimName` | Name of the PVC | `hot-property-data` | +| `storage.data.persistentVolumeClaims[2].size` | Size of the PVC | `5Gi` | +| `storage.data.persistentVolumeClaims[2].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | +| `storage.data.persistentVolumeClaims[2].storageClass` | Storage class for the PVC | `nil` | +| `storage.data.persistentVolumeClaims[2].volumeMode` | Volume mode for the PVC | `Filesystem` | +| `storage.liaison.enabled` | Enable persistent storage for liaison nodes (boolean) | `true` | +| `storage.liaison.persistentVolumeClaims` | List of PVC configurations for liaison nodes | | +| `storage.liaison.persistentVolumeClaims[0].mountTargets` | Mount targets for the PVC | `["measure","stream"]` | +| `storage.liaison.persistentVolumeClaims[0].claimName` | Name of the PVC | `liaison-data` | +| `storage.liaison.persistentVolumeClaims[0].size` | Size of the PVC | `10Gi` | +| `storage.liaison.persistentVolumeClaims[0].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | +| `storage.liaison.persistentVolumeClaims[0].storageClass` | Storage class for the PVC | `nil` | +| `storage.liaison.persistentVolumeClaims[0].volumeMode` | Volume mode for the PVC | `Filesystem` | +| `storage.standalone.enabled` | Enable persistent storage for standalone mode (boolean) | `false` | +| `storage.standalone.persistentVolumeClaims` | List of PVC configurations for standalone | | +| `storage.standalone.persistentVolumeClaims[0].mountTargets` | Mount targets for the PVC | `["measure","stream","metadata","property"]` | +| `storage.standalone.persistentVolumeClaims[0].claimName` | Name of the PVC | `standalone-data` | +| `storage.standalone.persistentVolumeClaims[0].size` | Size of the PVC | `200Gi` | +| `storage.standalone.persistentVolumeClaims[0].accessModes` | Access modes for the PVC | `["ReadWriteOnce"]` | +| `storage.standalone.persistentVolumeClaims[0].storageClass` | Storage class for the PVC | `nil` | +| `storage.standalone.persistentVolumeClaims[0].volumeMode` | Volume mode for the PVC | `Filesystem` | ### Service account configuration diff --git a/test/e2e/e2e-banyandb-cluster.yaml b/test/e2e/e2e-banyandb-cluster.yaml index d84b8ed..73c743a 100644 --- a/test/e2e/e2e-banyandb-cluster.yaml +++ b/test/e2e/e2e-banyandb-cluster.yaml @@ -62,6 +62,8 @@ setup: --set oap.env.SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS=k8s-mesh \ --set oap.env.SW_ENVOY_METRIC_ALS_TCP_ANALYSIS=k8s-mesh \ --set oap.env.K8S_SERVICE_NAME_RULE='e2e::${service.metadata.name}' \ + --set oap.env.SW_STORAGE_BANYANDB_USER=admin \ + --set oap.env.SW_STORAGE_BANYANDB_PASSWORD=banyandb \ --set oap.replicas=1 \ --set ui.image.repository=$UI_REPO \ --set ui.image.tag=$UI_TAG \ diff --git a/test/e2e/e2e-banyandb-standalone.yaml b/test/e2e/e2e-banyandb-standalone.yaml index 3cce8e5..8788128 100644 --- a/test/e2e/e2e-banyandb-standalone.yaml +++ b/test/e2e/e2e-banyandb-standalone.yaml @@ -62,6 +62,8 @@ setup: --set oap.env.SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS=k8s-mesh \ --set oap.env.SW_ENVOY_METRIC_ALS_TCP_ANALYSIS=k8s-mesh \ --set oap.env.K8S_SERVICE_NAME_RULE='e2e::${service.metadata.name}' \ + --set oap.env.SW_STORAGE_BANYANDB_USER=admin \ + --set oap.env.SW_STORAGE_BANYANDB_PASSWORD=banyandb \ --set oap.replicas=1 \ --set ui.image.repository=$UI_REPO \ --set ui.image.tag=$UI_TAG \ diff --git a/test/e2e/values.cluster.yaml b/test/e2e/values.cluster.yaml index e4b73a2..1fe46d1 100644 --- a/test/e2e/values.cluster.yaml +++ b/test/e2e/values.cluster.yaml @@ -19,7 +19,7 @@ image: repository: ghcr.io/apache/skywalking-banyandb - tag: 7e5b2d0404e8ad6d5835eee6fe589a2544d0decb + tag: 46083529398b73504e9ca929ef367cd1776aef82 pullPolicy: IfNotPresent cluster: @@ -407,7 +407,7 @@ storage: data: enabled: true persistentVolumeClaims: - - mountTargets: [ "measure", "stream", "property" ] + - mountTargets: [ "measure", "stream", "property", "trace" ] nodeRole: hot existingClaimName: null claimName: data @@ -428,7 +428,7 @@ storage: liaison: enabled: true persistentVolumeClaims: - - mountTargets: [ "measure", "stream" ] + - mountTargets: [ "measure", "stream", "trace" ] claimName: liaison-data size: 10Gi accessModes: @@ -438,7 +438,7 @@ storage: standalone: enabled: false persistentVolumeClaims: - - mountTargets: [ "measure", "stream", "property" ] + - mountTargets: [ "measure", "stream", "property", "trace" ] claimName: standalone-data size: 200Gi accessModes: @@ -479,3 +479,11 @@ etcd: fullnameOverride: "" nameOverride: "banyandb" + +auth: + enabled: true + existingSecret: "" + credentialsFileKey: "credentials.yaml" + users: + - username: admin + password: banyandb diff --git a/test/e2e/values.lifecycle.yaml b/test/e2e/values.lifecycle.yaml index 90cb4ca..c547c6a 100644 --- a/test/e2e/values.lifecycle.yaml +++ b/test/e2e/values.lifecycle.yaml @@ -19,7 +19,7 @@ image: repository: ghcr.io/apache/skywalking-banyandb - tag: 7e5b2d0404e8ad6d5835eee6fe589a2544d0decb + tag: 46083529398b73504e9ca929ef367cd1776aef82 pullPolicy: IfNotPresent cluster: @@ -156,6 +156,8 @@ cluster: lifecycleSidecar: enabled: true schedule: "@hourly" + progressFile: "/tmp/stream/lifecycle/progress.json" + reportDir: "/tmp/stream/lifecycle/reports" resources: {} restoreInitContainer: @@ -189,10 +191,14 @@ cluster: lifecycleSidecar: schedule: "@daily" enabled: true + progressFile: "/tmp/stream/lifecycle/progress.json" + reportDir: "/tmp/stream/lifecycle/reports" warm: lifecycleSidecar: schedule: "@daily" enabled: true + progressFile: "/tmp/stream/lifecycle/progress.json" + reportDir: "/tmp/stream/lifecycle/reports" cold: replicas: 1 @@ -287,6 +293,15 @@ storage: - ReadWriteOnce storageClass: null volumeMode: Filesystem + - mountTargets: [ "trace" ] + nodeRole: hot + existingClaimName: null + claimName: hot-trace-data + size: 50Gi + accessModes: + - ReadWriteOnce + storageClass: null + volumeMode: Filesystem - mountTargets: [ "measure" ] nodeRole: warm existingClaimName: null @@ -314,6 +329,15 @@ storage: - ReadWriteOnce storageClass: null volumeMode: Filesystem + - mountTargets: [ "trace" ] + nodeRole: warm + existingClaimName: null + claimName: warm-trace-data + size: 100Gi + accessModes: + - ReadWriteOnce + storageClass: null + volumeMode: Filesystem - mountTargets: [ "measure" ] nodeRole: cold existingClaimName: null @@ -341,6 +365,15 @@ storage: - ReadWriteOnce storageClass: null volumeMode: Filesystem + - mountTargets: [ "trace" ] + nodeRole: cold + existingClaimName: null + claimName: cold-trace-data + size: 200Gi + accessModes: + - ReadWriteOnce + storageClass: null + volumeMode: Filesystem - mountTargets: [ "backups" ] nodeRole: hot existingClaimName: null diff --git a/test/e2e/values.standalone.yaml b/test/e2e/values.standalone.yaml index 7b702a0..0839a30 100644 --- a/test/e2e/values.standalone.yaml +++ b/test/e2e/values.standalone.yaml @@ -25,7 +25,7 @@ etcd: image: repository: ghcr.io/apache/skywalking-banyandb - tag: 7e5b2d0404e8ad6d5835eee6fe589a2544d0decb + tag: 46083529398b73504e9ca929ef367cd1776aef82 pullPolicy: IfNotPresent standalone: @@ -90,8 +90,7 @@ standalone: # - key: memory # value: "512Mi" # tls: - # grpcSecretName: "" - # httpSecretName: "" + tls: {} livenessProbe: initialDelaySeconds: 20 periodSeconds: 5 @@ -146,7 +145,7 @@ storage: data: enabled: false persistentVolumeClaims: - - mountTargets: ["measure", "stream", "property"] + - mountTargets: ["measure", "stream", "property", "trace"] nodeRole: hot existingClaimName: null claimName: data @@ -167,7 +166,7 @@ storage: liaison: enabled: false persistentVolumeClaims: - - mountTargets: [ "measure", "stream" ] + - mountTargets: [ "measure", "stream", "trace" ] claimName: liaison-data size: 10Gi accessModes: @@ -177,7 +176,7 @@ storage: standalone: enabled: false persistentVolumeClaims: - - mountTargets: [ "measure", "stream", "metadata", "property" ] + - mountTargets: [ "measure", "stream", "metadata", "property", "trace" ] claimName: standalone-data size: 200Gi accessModes: @@ -196,3 +195,11 @@ serviceAccount: fullnameOverride: "" nameOverride: "banyandb" + +auth: + enabled: true + existingSecret: "" + credentialsFileKey: "credentials.yaml" + users: + - username: admin + password: banyandb