This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 0fbef83665a k8s_doc_add_network_and_PVC_2.0 (#30423)
0fbef83665a is described below
commit 0fbef83665a98908485e8bdfa929195aa549ac0d
Author: catpineapple <[email protected]>
AuthorDate: Fri Jan 26 18:12:00 2024 +0800
k8s_doc_add_network_and_PVC_2.0 (#30423)
---
docs/en/docs/install/k8s-deploy/network.md | 377 ++++++++++++++++++
docs/en/docs/install/k8s-deploy/operator-deploy.md | 2 +-
.../docs/install/k8s-deploy/persistent-volume.md | 420 ++++++++++++++++++++
docs/sidebars.json | 4 +-
docs/zh-CN/docs/install/k8s-deploy/network.md | 387 +++++++++++++++++++
.../docs/install/k8s-deploy/operator-deploy.md | 2 +-
.../docs/install/k8s-deploy/persistent-volume.md | 421 +++++++++++++++++++++
7 files changed, 1610 insertions(+), 3 deletions(-)
diff --git a/docs/en/docs/install/k8s-deploy/network.md
b/docs/en/docs/install/k8s-deploy/network.md
new file mode 100644
index 00000000000..82031c89efc
--- /dev/null
+++ b/docs/en/docs/install/k8s-deploy/network.md
@@ -0,0 +1,377 @@
+---
+{
+ "title": "Access the Doris cluster",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+In Kubernetes, a Service is a resource that defines a set of pods and provides
stable access to this set of pods, so it can be associated with a set of pods.
+Deploying a cluster through Doris-Operator will automatically generate
corresponding Service resources according to the `spec.*Spec.service`
configuration. Currently, ClusterIP, LoadBalancer and NodePort modes are
supported. Support users’ access needs in different scenarios.
+
+## Access within the Kubernetes cluster
+### ClusterIP mode
+Doris provides `ClusterIP` access within the kubernetes cluster by default on
kubernetes. For FE and BE components, we provide corresponding Service
resources for users to use on demand on kubernetes. Use the following command
to view the Service of the corresponding component. The Service naming rule
provided by Doris-Operator is `{clusterName}-{fe/be}-service`.
+```shell
+$ kubectl -n {namespace} get service
+```
+During use, please replace {namespace} with the namespace specified during
deployment. Take our default sample deployed Doris cluster as an example:
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ app.kubernetes.io/instance: doriscluster-sample
+ app.kubernetes.io/part-of: doris-operator
+ name: doriscluster-sample
+spec:
+ feSpec:
+ replicas: 3
+ limits:
+ cpu: 6
+ memory: 12Gi
+ requests:
+ cpu: 6
+ memory: 12Gi
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ beSpec:
+ replicas: 3
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ image: selectdb/doris.be-ubuntu:2.0.2
+```
+We view `kubectl get service` through the command as follows:
+```shell
+$ kubectl get service
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
+doriscluster-sample-be-internal ClusterIP None <none>
9050/TCP 12h
+doriscluster-sample-be-service ClusterIP 172.20.217.234 <none>
9060/TCP,8040/TCP,9050/TCP,8060/TCP 12h
+doriscluster-sample-fe-internal ClusterIP None <none>
9030/TCP 12h
+doriscluster-sample-fe-service ClusterIP 172.20.183.136 <none>
8030/TCP,9020/TCP,9030/TCP,9010/TCP 12h
+```
+There are two types of services, FE and BE, displayed through the command. The
service with the suffix `internal` is the service used by Doris for internal
communication and is not available externally. The suffix `-service` is a
Service for users to use.
+In this example, the `CLUSTER-IP` corresponding to
`doriscluster-sample-fe-service` and the corresponding `PORT` can be used on
the K8s cluster to access different port services of FE. Using
`doriscluster-sample-be-service` Service
+And the corresponding `PORT` port to access BE's services.
+
+## Access outside the Kubernetes cluster
+### LoadBalancer Mode
+If the cluster is created on a relevant cloud platform, it is recommended to
use the `LoadBalancer` mode to access the FE and BE services within the
cluster. The `ClusterIP` mode is used by default. If you want to use the
`LoadBalancer` mode, please configure the following configuration in the spec
of each component:
+```yaml
+service:
+ type: LoadBalancer
+```
+Taking the default configuration as a modification blueprint for example, we
use `LoadBalancer` as the access mode of FE and BE on the cloud platform. The
deployment configuration is as follows:
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ app.kubernetes.io/instance: doriscluster-sample
+ app.kubernetes.io/part-of: doris-operator
+ name: doriscluster-sample
+spec:
+ feSpec:
+ replicas: 3
+ service:
+ type: LoadBalancer
+ limits:
+ cpu: 6
+ memory: 12Gi
+ requests:
+ cpu: 6
+ memory: 12Gi
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ beSpec:
+ replicas: 3
+ service:
+ type: LoadBalancer
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ image: selectdb/doris.be-ubuntu:2.0.2
+```
+By viewing the `kubectl get service` command, view the corresponding Service
display as follows:
+```shell
+$ kubectl get service
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S)
AGE
+doriscluster-sample-be-internal ClusterIP None <none>
9050/TCP
14h
+doriscluster-sample-be-service LoadBalancer 172.20.217.234
a46bbcd6998c7436bab8ee8fba9f5e81-808549982.us-east-1.elb.amazonaws.com
9060:32060/TCP,8040:30615/TCP,9050:31742/TCP,8060:31127/TCP 14h
+doriscluster-sample-fe-internal ClusterIP None <none>
9030/TCP
14h
+doriscluster-sample-fe-service LoadBalancer 172.20.183.136
ac48284932b044251bfac389b453118f-1412731848.us-east-1.elb.amazonaws.com
8030:32213/TCP,9020:31080/TCP,9030:31433/TCP,9010:30585/TCP 14h
+```
+External ports corresponding to `EXTERNAL-IP` and `PORT` can be used outside
K8s to access the services of various components within K8s. For example, to
access the `mysql` client service corresponding to FE's 9030, you can use the
following command to connect:
+```shell
+mysql -h
ac48284932b044251bfac389b453118f-1412731848.us-east-1.elb.amazonaws.com -P 9030
-uroot
+```
+### NodePort Mode
+In a private network environment, to access internal services outside K8s, it
is recommended to use the `NodePort` mode of K8s. Use the default configuration
as a blueprint to configure the `NodePort` access mode in the private network
as follows:
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ app.kubernetes.io/instance: doriscluster-sample
+ app.kubernetes.io/part-of: doris-operator
+ name: doriscluster-sample
+spec:
+ feSpec:
+ replicas: 3
+ service:
+ type: NodePort
+ limits:
+ cpu: 6
+ memory: 12Gi
+ requests:
+ cpu: 6
+ memory: 12Gi
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ beSpec:
+ replicas: 3
+ service:
+ type: NodePort
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ image: selectdb/doris.be-ubuntu:2.0.2
+```
+After deployment, view the corresponding Service by viewing the `kubectl get
service` command:
+```shell
+$ kubectl get service
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
+kubernetes ClusterIP 10.152.183.1 <none>
443/TCP 169d
+doriscluster-sample-fe-internal ClusterIP None <none>
9030/TCP 2d
+doriscluster-sample-fe-service NodePort 10.152.183.58 <none>
8030:31041/TCP,9020:30783/TCP,9030:31545/TCP,9010:31610/TCP 2d
+doriscluster-sample-be-internal ClusterIP None <none>
9050/TCP 2d
+doriscluster-sample-be-service NodePort 10.152.183.244 <none>
9060:30940/TCP,8040:32713/TCP,9050:30621/TCP,8060:30926/TCP 2d
+```
+The above command obtains the port that can be used outside K8s, and obtains
the host managed by K8s through the following command:
+```shell
+$ kubectl get nodes -owide
+NAME STATUS ROLES AGE VERSION
INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION
CONTAINER-RUNTIME
+vm-10-7-centos Ready <none> 88d v1.23.17-2+40cc20cc310518
10.16.10.7 <none> TencentOS Server 3.1 (Final) 5.4.119-19.0009.25
containerd://1.5.13
+vm-10-8-centos Ready <none> 169d v1.23.17-2+40cc20cc310518
10.16.10.8 <none> TencentOS Server 3.1 (Final) 5.4.119-19-0009.3
containerd://1.5.13
+```
+In a private network environment, use the K8s host and mapped ports to access
K8s internal services. For example, we use the host's IP and FE's 9030 mapped
port (31545) to connect to `mysql`:
+```shell
+$ mysql -h 10.16.10.8 -P 31545 -uroot
+```
+
+In addition, you can specify the nodePort you need according to your own
platform needs.
+The Kubernetes master will allocate a port from the given configuration range
(general default: 30000-32767) and each Node will proxy to the Service from
that port (the same port on each Node). Like the example above, a random port
will be automatically generated if not specified.
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ app.kubernetes.io/instance: doriscluster-sample
+ app.kubernetes.io/part-of: doris-operator
+ name: doriscluster-sample
+spec:
+ feSpec:
+ replicas: 3
+ service:
+ type: NodePort
+ servicePorts:
+ - nodePort: 31001
+ targetPort: 8030
+ - nodePort: 31002
+ targetPort: 9020
+ - nodePort: 31003
+ targetPort: 9030
+ - nodePort: 31004
+ targetPort: 9010
+ limits:
+ cpu: 6
+ memory: 12Gi
+ requests:
+ cpu: 6
+ memory: 12Gi
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ beSpec:
+ replicas: 3
+ service:
+ type: NodePort
+ servicePorts:
+ - nodePort: 31005
+ targetPort: 9060
+ - nodePort: 31006
+ targetPort: 8040
+ - nodePort: 31007
+ targetPort: 9050
+ - nodePort: 31008
+ targetPort: 8060
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ image: selectdb/doris.be-ubuntu:2.0.2
+```
+After deployment, check the corresponding Service by viewing the `kubectl get
service` command. For access methods, please refer to the above:
+```shell
+$ kubectl get service
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
+kubernetes ClusterIP 10.152.183.1 <none>
443/TCP 169d
+doriscluster-sample-fe-internal ClusterIP None <none>
9030/TCP 2d
+doriscluster-sample-fe-service NodePort 10.152.183.67 <none>
8030:31001/TCP,9020:31002/TCP,9030:31003/TCP,9010:31004/TCP 2d
+doriscluster-sample-be-internal ClusterIP None <none>
9050/TCP 2d
+doriscluster-sample-be-service NodePort 10.152.183.24 <none>
9060:31005/TCP,8040:31006/TCP,9050:31007/TCP,8060:31008/TCP 2d
+```
+
+## Doris data exchange
+### Stream load
+[Stream
load](https://doris.apache.org/docs/data-operate/import/import-way/stream-load-manual)
is a synchronous import method. Users send requests to import local files or
data streams into Doris by sending HTTP protocol.
+In a regular deployment, users submit import commands via the HTTP protocol.
Generally, users will submit the request to FE, and FE will forward the request
to a certain BE through the HTTP redirect command. However, in a
Kubernetes-based deployment scenario, it is recommended that users **directly
submit the import command to BE's Srevice**, and then the Service will be load
balanced to a certain BE pod based on Kubernetes rules.
+The actual effects of these two operations are the same. When Flink or Spark
uses the official connector to submit, the write request can also be submitted
to the BE Service.
+
+### ErrorURL
+When import methods such as [Stream
load](https://doris.apache.org/docs/data-operate/import/import-way/stream-load-manual)
and [Routine
load](https://doris.apache.org/docs/data-operate/import/import-way/routine-load-manual)
+These import methods will print `errorURL` or `tracking_url` in the return
structure or log when encountering errors such as incorrect data format. You
can locate the cause of the import error by visiting this link.
+However, this URL is only accessible within the internal environment of a
specific BE node container in a Kubernetes deployed cluster.
+
+The following scenario takes the errorURL returned by Doris as an example:
+```http://doriscluster-sample-be-2.doriscluster-sample-be-internal.doris.svc.cluster.local:8040/api/_load_error_log?file=__shard_1/error_log_insert_stmt_af474190276a2e9c-49bb9d175b8e968e_af474190276a2e9c_49bb9d175b8e968e```
+
+**1. Kubernetes cluster internal access**
+
+You need to obtain the access method of BE's Service or pod through the
`kubectl get service` or `kubectl get pod -o wide` command, replace the domain
name and port of the original URL, and then access again.
+
+for example:
+```shell
+$ kubectl get pod -o wide
+NAME READY STATUS RESTARTS AGE IP
NODE NOMINATED NODE READINESS GATES
+doriscluster-sample-be-0 1/1 Running 0 9h 10.0.2.105
10-0-2-47.ec2.internal <none> <none>
+doriscluster-sample-be-1 1/1 Running 0 9h 10.0.2.104
10-0-2-5.ec2.internal <none> <none>
+doriscluster-sample-be-2 1/1 Running 0 9h 10.0.2.103
10-0-2-6.ec2.internal <none> <none>
+doriscluster-sample-fe-0 1/1 Running 0 9h 10.0.2.102
10-0-2-47.ec2.internal <none> <none>
+doriscluster-sample-fe-1 1/1 Running 0 9h 10.0.2.101
10-0-2-5.ec2.internal <none> <none>
+doriscluster-sample-fe-2 1/1 Running 0 9h 10.0.2.100
10-0-2-6.ec2.internal <none> <none>
+```
+The above errorURL is changed to:
+```http://10.0.2.103:8040/api/_load_error_log?file=__shard_1/error_log_insert_stmt_af474190276a2e9c-49bb9d175b8e968e_af474190276a2e9c_49bb9d175b8e968e```
+
+
+**2. NodePort mode for external access to Kubernetes cluster**
+
+Obtaining error report details from outside Kubernetes requires additional
bridging means. The following are the processing steps for using NodePort mode
Service when deploying Doris. Obtain error report details by creating a new
Service:
+Handle Service template be_streamload_errror_service.yaml:
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+ labels:
+ app.doris.service/role: debug
+ app.kubernetes.io/component: be
+ name: doriscluster-detail-error
+spec:
+ externalTrafficPolicy: Cluster
+ internalTrafficPolicy: Cluster
+ ipFamilies:
+ - IPv4
+ ipFamilyPolicy: SingleStack
+ ports:
+ - name: webserver-port
+ port: 8040
+ protocol: TCP
+ targetPort: 8040
+ selector:
+ app.kubernetes.io/component: be
+ statefulset.kubernetes.io/pod-name: ${podName}
+ sessionAffinity: None
+ type: NodePort
+```
+Use the following command to view the mapping of Service 8040 port on the host
machine
+```shell
+$ kubectl get service -n doris doriscluster-detail-error
+NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
+doriscluster-detail-error NodePort 10.152.183.35 <none> 8040:31201/TCP 32s
+```
+Use the IP of any host and the NodePort (31201) port obtained above to access
and replace errorURL to obtain a detailed error report.
+The above errorURL is changed to:
+```http://10.152.183.35:31201/api/_load_error_log?file=__shard_1/error_log_insert_stmt_af474190276a2e9c-49bb9d175b8e968e_af474190276a2e9c_49bb9d175b8e968e```
+
+**3. Access LoadBalancer mode from outside the Kubernetes cluster**
+
+Obtaining error report details from outside Kubernetes requires additional
bridging means. The following are the processing steps for using the
LoadBalancer mode Service when deploying Doris. Obtain error report details by
creating a new Service:
+Handle Service template be_streamload_errror_service.yaml:
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+ labels:
+ app.doris.service/role: debug
+ app.kubernetes.io/component: be
+ name: doriscluster-detail-error
+spec:
+ externalTrafficPolicy: Cluster
+ internalTrafficPolicy: Cluster
+ ipFamilies:
+ - IPv4
+ ipFamilyPolicy: SingleStack
+ ports:
+ - name: webserver-port
+ port: 8040
+ protocol: TCP
+ targetPort: 8040
+ selector:
+ app.kubernetes.io/component: be
+ statefulset.kubernetes.io/pod-name: ${podName}
+ sessionAffinity: None
+ type: LoadBalancer
+```
+`podName` is replaced with the highest-level domain name of errorURL:
`doriscluster-sample-be-2`.
+
+In the `namespace` deployed by Doris (generally the default is `doris`, use
`doris` for the following operations), use the following command to deploy the
service processed above:
+
+```shell
+$ kubectl apply -n doris -f be_streamload_errror_service.yaml
+```
+
+Use the following command to view the mapping of Service 8040 port on the host
machine
+```shell
+$ kubectl get service -n doris doriscluster-detail-error
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
+doriscluster-detail-error LoadBalancer 172.20.183.136
ac4828493dgrftb884g67wg4tb68gyut-1137856348.us-east-1.elb.amazonaws.com
8040:32003/TCP 14s
+```
+
+Use `EXTERNAL-IP` and Port (8040) port access to replace errorURL to obtain a
detailed error report.
+The above errorURL is changed to:
+```http://ac4828493dgrftb884g67wg4tb68gyut-1137856348.us-east-1.elb.amazonaws.com:8040/api/_load_error_log?file=__shard_1/error_log_insert_stmt_af474190276a2e9c-49bb9d175b8e968
e_af474190276a2e9c_49bb9d175b8e968e```
+
+Note: For the above method of setting up access outside the cluster, it is
recommended to clear the current Service after use. The reference command is as
follows:
+```shell
+$ kubectl delete service -n doris doriscluster-detail-error
+```
\ No newline at end of file
diff --git a/docs/en/docs/install/k8s-deploy/operator-deploy.md
b/docs/en/docs/install/k8s-deploy/operator-deploy.md
index 659fcd0d544..8da45278780 100644
--- a/docs/en/docs/install/k8s-deploy/operator-deploy.md
+++ b/docs/en/docs/install/k8s-deploy/operator-deploy.md
@@ -45,7 +45,7 @@ Hosted Kubernetes on cloud platform or set-up by yourself are
all good choice.
Complete all the
[prerequisites](https://cloud.google.com/kubernetes-engine/docs/deploy-app-cluster#before-you-begin)
when [create a GKE
cluster](https://cloud.google.com/kubernetes-engine/docs/deploy-app-cluster#create_cluster).
**Create as Kubernetes recommend**
-Kubernetes official documents recommends some ways to set up Kubernetes, as
[minikube](https://minikube.sigs.k8s.io/docs/start/),[kOps](https://kubernetes.io/zh-cn/docs/setup/production-environment/tools/kops/).
+Kubernetes official documents recommends some ways to set up Kubernetes, as
[minikube](https://minikube.sigs.k8s.io/docs/start/),[kOps](https://kops.sigs.k8s.io/).
### Deploy Doris-Operator on Kubernetes
**1. Apply the [Custom Resource
Definition(CRD)](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/#customresourcedefinitions)**
diff --git a/docs/en/docs/install/k8s-deploy/persistent-volume.md
b/docs/en/docs/install/k8s-deploy/persistent-volume.md
new file mode 100644
index 00000000000..e5757e5c891
--- /dev/null
+++ b/docs/en/docs/install/k8s-deploy/persistent-volume.md
@@ -0,0 +1,420 @@
+---
+{
+ "title": "Persistent Volume and ConfigMap",
+ "language": "en"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Doris-Operator supports mounting PV (Persistent Volume) on pods of various
Doris components.
+
+PV is generally created by the kubernetes system administrator. Doris-Operator
does not use PV directly when deploying Doris services. Instead, it declares a
set of resources through PVC to apply for PV from the kubernetes cluster.
+When a PVC is created, Kubernetes will attempt to bind it to an available PV
that meets the requirements.
+StorageClass shields administrators from the process of manually creating PVs.
When there are no ready-made PVs that meet PVC requirements, PVs can be
dynamically allocated based on StorageClass.
+PV provides a variety of storage types, mainly divided into two categories:
network storage and local storage. Based on their respective principles and
implementations, the two provide users with different performance and usage
experiences. Users can choose according to their own containerized service
types and their own needs.
+
+If PVC is not configured during deployment, Doris-Operator uses the `emptyDir`
mode by default to store metadata data files and run logs. When the pod is
restarted, related data will be lost.
+
+Recommended node directory type for persistent storage:
+* FE: doris-meta, log
+* BE: storage, log
+* CN: storage, log
+* BROKER: log
+
+Doris-Operator outputs logs to the console and the specified directory at the
same time. If the user's Kubernetes system has complete log collection
capabilities, log information at the Doris INFO level (default) can be
collected through console output.
+However, it is still recommended to configure PVC to persist log files,
because in addition to INFO level logs, there are also logs such as fe.out,
be.out, audit.log and garbage collection logs, which facilitates quick problem
location and audit log backtracking.
+
+ConfigMap is a resource object used to store configuration files in
Kubernetes. It allows dynamically mounting configuration files and decouples
configuration files from applications, making configuration management more
flexible and maintainable.
+Like PVCs, ConfigMap can be referenced by Pods in order to use configuration
data in the application.
+
+
+## StorageClass
+
+Doris-Operator provides Kubernetes default `StorageClass` mode to support FE
and BE data storage, where the storage path (mountPath) uses the default
configuration in the image.
+If users need to specify the StorageClass themselves, they need to modify
`persistentVolumeClaimSpec.storageClassName` in
`spec.feSpec.persistentVolumes`, as shown below:
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ name: doriscluster-sample-storageclass1
+spec:
+ feSpec:
+ replicas: 3
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ persistentVolumes:
+ - mountPath: /opt/apache-doris/fe/doris-meta
+ name: storage0
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ storageClassName: ${your_storageclass}
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ # notice: if the storage size less 5G, fe will not start normal.
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/fe/log
+ name: storage1
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ storageClassName: ${your_storageclass}
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ beSpec:
+ replicas: 3
+ image: selectdb/doris.be-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ persistentVolumes:
+ - mountPath: /opt/apache-doris/be/storage
+ name: storage2
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ storageClassName: ${your_storageclass}
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/be/log
+ name: storage3
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ storageClassName: ${your_storageclass}
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+```
+
+## Customized ConfigMap
+Doris uses `ConfigMap` in Kubernetes to decouple configuration files and
services. Before deploying `doriscluster`, you need to deploy the `ConfigMap`
you want to use under the same `namespace` in advance. The following example
shows that FE uses `ConfigMap` named fe-configmap and BE uses `ConfigMap` named
be-configmap. Cluster related yaml:
+
+ConfigMap sample for FE
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: fe-configmap
+ labels:
+ app.kubernetes.io/component: fe
+data:
+ fe.conf: |
+ CUR_DATE=`date +%Y%m%d-%H%M%S`
+
+ # the output dir of stderr and stdout
+ LOG_DIR = ${DORIS_HOME}/log
+
+ JAVA_OPTS="-Djavax.security.auth.useSubjectCredsOnly=false -Xss4m
-Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7
-XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC
-XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled
-XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0
-Xloggc:$DORIS_HOME/log/fe.gc.log.$CUR_DATE"
+
+ # For jdk 9+, this JAVA_OPTS will be used as default JVM options
+ JAVA_OPTS_FOR_JDK_9="-Djavax.security.auth.useSubjectCredsOnly=false
-Xss4m -Xmx8192m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7
-XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled
-XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0
-Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$CUR_DATE:time"
+
+ # INFO, WARN, ERROR, FATAL
+ sys_log_level = INFO
+
+ # NORMAL, BRIEF, ASYNC
+ sys_log_mode = NORMAL
+
+ # Default dirs to put jdbc drivers,default value is
${DORIS_HOME}/jdbc_drivers
+ # jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers
+
+ http_port = 8030
+ rpc_port = 9020
+ query_port = 9030
+ edit_log_port = 9010
+
+ enable_fqdn_mode = true
+```
+
+Note that when using FE's ConfigMap, you must add `enable_fqdn_mode = true` to
`fe.conf`. For specific reasons, please refer to [document
here](https://doris.apache.org/docs/admin-manual/cluster-management/fqdn)
+
+BE's ConfigMap sample
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: be-configmap
+ labels:
+ app.kubernetes.io/component: be
+data:
+ be.conf: |
+ CUR_DATE=`date +%Y%m%d-%H%M%S`
+
+ PPROF_TMPDIR="$DORIS_HOME/log/"
+
+ JAVA_OPTS="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log
-Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100
-DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
+
+ # For jdk 9+, this JAVA_OPTS will be used as default JVM options
+ JAVA_OPTS_FOR_JDK_9="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log
-Xlog:gc:$DORIS_HOME/log/be.gc.log.$CUR_DATE
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100
-DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
+
+ # since 1.2, the JAVA_HOME need to be set to run BE process.
+ # JAVA_HOME=/path/to/jdk/
+
+ #
https://github.com/apache/doris/blob/master/docs/zh-CN/community/developer-guide/debug-tool.md#jemalloc-heap-profile
+ # https://jemalloc.net/jemalloc.3.html
+
JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:15000,dirty_decay_ms:15000,oversize_threshold:0,lg_tcache_max:20,prof:false,lg_prof_interval:32,lg_prof_sample:19,prof_gdump:false,prof_accum:false,prof_leak:false,prof_final:false"
+ JEMALLOC_PROF_PRFIX=""
+
+ # INFO, WARNING, ERROR, FATAL
+ sys_log_level = INFO
+
+ # ports for admin, web, heartbeat service
+ be_port = 9060
+ webserver_port = 8040
+ heartbeat_service_port = 9050
+ brpc_port = 8060
+```
+`doriscluster` deployment example using the above two `ConfigMap`:
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ name: doriscluster-sample-configmap
+spec:
+ feSpec:
+ replicas: 3
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ configMapInfo:
+ # use kubectl create configmap fe-configmap --from-file=fe.conf
+ configMapName: fe-configmap
+ resolveKey: fe.conf
+ beSpec:
+ replicas: 3
+ image: selectdb/doris.be-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ configMapInfo:
+ # use kubectl create configmap be-configmap --from-file=be.conf
+ configMapName: be-configmap
+ resolveKey: be.conf
+ brokerSpec:
+ replicas: 3
+ image: selectdb/doris.broker-ubuntu:2.0.2
+ limits:
+ cpu: 2
+ memory: 4Gi
+ requests:
+ cpu: 2
+ memory: 4Gi
+ configMapInfo:
+ # use kubectl create configmap broker-configmap
--from-file=apache_hdfs_broker.conf
+ configMapName: broker-configmap
+ resolveKey: apache_hdfs_broker.conf
+
+```
+The `resolveKey` here is the name of the incoming configuration file (must be
`fe.conf`, `be.conf` or `apache_hdfs_broker.conf`, the cn node is also
`be.conf`) used to parse the incoming Doris cluster configuration file,
doris-operator will parse the file to guide the customized deployment of
doriscluster.
+
+## Add special configuration files to the conf directory
+This paragraph is for reference. Containerized deployment solutions that
configure other files need to be placed in the conf directory of the Doris
node. For example, the common HDFS/Hive configuration file mapping of [Data
Lake Multi-catalog](https://doris.apache.org/docs/lakehouse/multi-catalog/hive).
+
+Here we take BE's ConfigMap and the core-site.xml file that needs to be added
as an example:
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: be-configmap
+ labels:
+ app.kubernetes.io/component: be
+data:
+ be.conf: |
+ be_port = 9060
+ webserver_port = 8040
+ heartbeat_service_port = 9050
+ brpc_port = 8060
+ core-site.xml: |
+ <?xml version="1.0" encoding="UTF-8"?>
+ <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+ <configuration>
+ <property>
+ <name>hadoop.security.authentication</name>
+ <value>kerberos</value>
+ </property>
+ </configuration>
+ ...
+```
+Note that the data structure in `data` is as follows: key-value pair mapping:
+```yaml
+data:
+ file_name_1:
+ file_content_1
+ file_name_2:
+ file_content_2
+ file_name_3:
+ file_content_3
+```
+
+## BE multi-disk configuration
+Doris' BE service supports multi-disk mounting, which can well solve the
problem of mismatch between computing resources and storage resources in the
server era. At the same time, using multiple disks can also greatly improve the
storage efficiency of doris. On Kubernetes, Doris can also mount multiple disks
to maximize storage efficiency. Using multiple disks on Kubernetes requires
using configuration files.
+In order to achieve decoupling of service and configuration, doris uses
`ConfigMap` as the bearer of configuration to dynamically mount configuration
files for service use.
+The following is the doriscluster configuration in which the BE service uses
`ConfigMap` to host the configuration file and mount two disks for BE use:
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ name: doriscluster-sample-storageclass1
+spec:
+ feSpec:
+ replicas: 3
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ persistentVolumes:
+ - mountPath: /opt/apache-doris/fe/doris-meta
+ name: storage0
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ # notice: if the storage size less 5G, fe will not start normal.
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/fe/log
+ name: storage1
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ beSpec:
+ replicas: 3
+ image: selectdb/doris.be-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ configMapInfo:
+ configMapName: be-configmap
+ resolveKey: be.conf
+ persistentVolumes:
+ - mountPath: /opt/apache-doris/be/storage
+ name: storage2
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/be/storage1
+ name: storage3
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/be/log
+ name: storage4
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+```
+Compared with the default example, the configuration of `configMapInfo` is
added, and a configuration of `persistentVolumeClaimSpec` is also added,
[`persistentVolumeClaimSpec`](https://kubernetes.io/docs/reference/kubernetes-api/config-and-storage-resources/persistent-volume-claim-v1/#PersistentVolumeClaimSpec)
fully follows the definition format of the Kubernetes native resource PVC spec.
+In the example, `configMapInfo` identifies which ConfigMap under the same
`namespace` and which key corresponding content will be used as the
configuration file after BE is deployed, where the key must be be.conf. The
following is an example of the above `doriscluster` ConfigMap that needs to be
pre-deployed:
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: be-configmap
+ labels:
+ app.kubernetes.io/component: be
+data:
+ be.conf: |
+ CUR_DATE=`date +%Y%m%d-%H%M%S`
+
+ PPROF_TMPDIR="$DORIS_HOME/log/"
+
+ JAVA_OPTS="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log
-Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100
-DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
+
+ # For jdk 9+, this JAVA_OPTS will be used as default JVM options
+ JAVA_OPTS_FOR_JDK_9="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log
-Xlog:gc:$DORIS_HOME/log/be.gc.log.$CUR_DATE
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100
-DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
+
+ # since 1.2, the JAVA_HOME need to be set to run BE process.
+ # JAVA_HOME=/path/to/jdk/
+
+ #
https://github.com/apache/doris/blob/master/docs/zh-CN/community/developer-guide/debug-tool.md#jemalloc-heap-profile
+ # https://jemalloc.net/jemalloc.3.html
+
JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:15000,dirty_decay_ms:15000,oversize_threshold:0,lg_tcache_max:20,prof:false,lg_prof_interval:32,lg_prof_sample:19,prof_gdump:false,prof_accum:false,prof_leak:false,prof_final:false"
+ JEMALLOC_PROF_PRFIX=""
+
+ # INFO, WARNING, ERROR, FATAL
+ sys_log_level = INFO
+
+ # ports for admin, web, heartbeat service
+ be_port = 9060
+ webserver_port = 8040
+ heartbeat_service_port = 9050
+ brpc_port = 8060
+
+ storage_root_path =
/opt/apache-doris/be/storage,medium:ssd;/opt/apache-doris/be/storage1,medium:ssd
+```
+When using multiple disks, the path in the corresponding value of
`storage_root_path` in `ConfigMap` should correspond to each mounting path of
`persistentVolume` in `doriscluster`.
[`storage_root_path`](https://doris.apache.org/docs/admin-manual/config/be-config/?_highlight=storage_root_path#%E6%9C%8D%E5%8A%A1)
For the corresponding writing rules, please refer to the document in the link.
+When using cloud disks, the media is uniformly `SSD`.
\ No newline at end of file
diff --git a/docs/sidebars.json b/docs/sidebars.json
index 55eb0183e57..b09f5a84072 100644
--- a/docs/sidebars.json
+++ b/docs/sidebars.json
@@ -26,7 +26,9 @@
"label": "Deploy Doris on Kubernetes",
"items": [
"install/k8s-deploy/operator-deploy",
- "install/k8s-deploy/helm-chart-deploy"
+ "install/k8s-deploy/helm-chart-deploy",
+ "install/k8s-deploy/network",
+ "install/k8s-deploy/persistent-volume"
]
},
{
diff --git a/docs/zh-CN/docs/install/k8s-deploy/network.md
b/docs/zh-CN/docs/install/k8s-deploy/network.md
new file mode 100644
index 00000000000..f70054707f0
--- /dev/null
+++ b/docs/zh-CN/docs/install/k8s-deploy/network.md
@@ -0,0 +1,387 @@
+---
+{
+ "title": "访问 Doris 集群",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+在 Kubernetes 中，Service 是用于定义一组 pod，并提供对这组 pod 的稳定访问能力的资源，因此它可以与一组 Pod 关联。
+通过 Doris-Operator 部署集群，会根据 `spec.*Spec.service` 配置自动生成对应的 Service 资源，目前支持
ClusterIP、LoadBalancer 和 NodePort 模式。支持用户不同场景的访问需求。
+
+## 在 Kubernetes 集群内部访问
+### ClusterIP 模式
+Doris 在 kubernetes 上默认提供 kubernetes 集群内部 `ClusterIP` 的访问方式，对于 FE 和 BE
组件我们分别提供相应的 Service 资源供用户在 kubernetes 上按需使用。使用如下命令查看对应组件的
Service，Doris-Operator 提供的 Service 命名规则为 `{clusterName}-{fe/be}-service` 的方式。
+```shell
+$ kubectl -n {namespace} get service
+```
+在使用过程中，请将 {namespace} 替换成部署时候指定的 namespace。以我们默认的样例部署的 Doris 集群为例：
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ app.kubernetes.io/instance: doriscluster-sample
+ app.kubernetes.io/part-of: doris-operator
+ name: doriscluster-sample
+spec:
+ feSpec:
+ replicas: 3
+ limits:
+ cpu: 6
+ memory: 12Gi
+ requests:
+ cpu: 6
+ memory: 12Gi
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ beSpec:
+ replicas: 3
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ image: selectdb/doris.be-ubuntu:2.0.2
+```
+我们通过命令查看 `kubectl get service` 如下：
+```shell
+$ kubectl get service
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
+doriscluster-sample-be-internal ClusterIP None <none>
9050/TCP 12h
+doriscluster-sample-be-service ClusterIP 172.20.217.234 <none>
9060/TCP,8040/TCP,9050/TCP,8060/TCP 12h
+doriscluster-sample-fe-internal ClusterIP None <none>
9030/TCP 12h
+doriscluster-sample-fe-service ClusterIP 172.20.183.136 <none>
8030/TCP,9020/TCP,9030/TCP,9010/TCP 12h
+```
+通过命令展示有 FE 和 BE 的两类 Service，后缀为 `internal` 的 Service 为 Doris 内部通信使用的
Service，外部不可用。后缀为 `-service` 为供用户使用的 Service。
+在这个例子中， 在 Kubernetes 集群之上可以使用 `doriscluster-sample-fe-service` 对应的
`CLUSTER-IP` 以及后面对应的 `PORT` 来访问 FE 的不同端口服务。使用 `doriscluster-sample-be-service`
Service
+以及对应的 `PORT` 的端口来访问 BE 的服务。
+
+## 在 Kubernetes 集群外部访问
+### LoadBalancer 模式
+如果集群在相关云平台创建，建议使用 `LoadBalancer` 的模式来访问集群内部的 FE 和 BE 服务。 默认情况下使用的是 `ClusterIP`
的模式，如果想要使用 `LoadBalancer` 模式，请在每个组件的 spec 配置如下配置：
+```yaml
+service:
+ type: LoadBalancer
+```
+以默认的配置作为修改蓝本为例，我们在云平台上使用 `LoadBalancer` 作为 FE 和 BE 的访问模式，部署配置如下：
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ app.kubernetes.io/instance: doriscluster-sample
+ app.kubernetes.io/part-of: doris-operator
+ name: doriscluster-sample
+spec:
+ feSpec:
+ replicas: 3
+ service:
+ type: LoadBalancer
+ limits:
+ cpu: 6
+ memory: 12Gi
+ requests:
+ cpu: 6
+ memory: 12Gi
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ beSpec:
+ replicas: 3
+ service:
+ type: LoadBalancer
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ image: selectdb/doris.be-ubuntu:2.0.2
+```
+通过查看 `kubectl get service` 的命令，查看相应的 Service 展示如下：
+```shell
+$ kubectl get service
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S)
AGE
+doriscluster-sample-be-internal ClusterIP None <none>
9050/TCP
14h
+doriscluster-sample-be-service LoadBalancer 172.20.217.234
a46bbcd6998c7436bab8ee8fba9f5e81-808549982.us-east-1.elb.amazonaws.com
9060:32060/TCP,8040:30615/TCP,9050:31742/TCP,8060:31127/TCP 14h
+doriscluster-sample-fe-internal ClusterIP None <none>
9030/TCP
14h
+doriscluster-sample-fe-service LoadBalancer 172.20.183.136
ac48284932b044251bfac389b453118f-1412731848.us-east-1.elb.amazonaws.com
8030:32213/TCP,9020:31080/TCP,9030:31433/TCP,9010:30585/TCP 14h
+```
+在 Kubernetes 外部可以使用 `EXTERNAL-IP` 以及 `PORT` 对应的外部端口来访问 Kubernetes 内部各个组件的服务。
比如访问 FE 的 9030 对应的 `mysql` client 服务，就可以用如下命令连接：
+```shell
+mysql -h
ac48284932b044251bfac389b453118f-1412731848.us-east-1.elb.amazonaws.com -P 9030
-uroot
+```
+### NodePort 模式
+私网环境下，在 Kubernetes 外部访问内部服务，推荐使用 Kubernetes 的 `NodePort` 模式， 使用默认的配置为蓝本配置私网下
`NodePort` 访问模式如下：
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ app.kubernetes.io/instance: doriscluster-sample
+ app.kubernetes.io/part-of: doris-operator
+ name: doriscluster-sample
+spec:
+ feSpec:
+ replicas: 3
+ service:
+ type: NodePort
+ limits:
+ cpu: 6
+ memory: 12Gi
+ requests:
+ cpu: 6
+ memory: 12Gi
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ beSpec:
+ replicas: 3
+ service:
+ type: NodePort
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ image: selectdb/doris.be-ubuntu:2.0.2
+```
+部署后，通过查看 `kubectl get service` 的命令查看相应的 Service ：
+```shell
+$ kubectl get service
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
+kubernetes ClusterIP 10.152.183.1 <none>
443/TCP 169d
+doriscluster-sample-fe-internal ClusterIP None <none>
9030/TCP 2d
+doriscluster-sample-fe-service NodePort 10.152.183.58 <none>
8030:31041/TCP,9020:30783/TCP,9030:31545/TCP,9010:31610/TCP 2d
+doriscluster-sample-be-internal ClusterIP None <none>
9050/TCP 2d
+doriscluster-sample-be-service NodePort 10.152.183.244 <none>
9060:30940/TCP,8040:32713/TCP,9050:30621/TCP,8060:30926/TCP 2d
+```
+上述命令获取到在 Kubernetes 外部可使用的端口，通过如下命令获取 Kubernetes 管理的宿主机：
+```shell
+$ kubectl get nodes -owide
+NAME STATUS ROLES AGE VERSION
INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION
CONTAINER-RUNTIME
+vm-10-7-centos Ready <none> 88d v1.23.17-2+40cc20cc310518
10.16.10.7 <none> TencentOS Server 3.1 (Final) 5.4.119-19.0009.25
containerd://1.5.13
+vm-10-8-centos Ready <none> 169d v1.23.17-2+40cc20cc310518
10.16.10.8 <none> TencentOS Server 3.1 (Final) 5.4.119-19-0009.3
containerd://1.5.13
+```
+私网环境下，使用 Kubernetes 的宿主机和映射端口访问 Kubernetes 内部的服务。例如，我们使用宿主机的 IP 和 FE 的 9030
映射端口（31545）进行 `mysql` 的连接：
+```shell
+$ mysql -h 10.16.10.8 -P 31545 -uroot
+```
+
+另外，可以根据自身平台需要，指定自己需要的 nodePort。
+Kubernetes master 将从给定的配置范围内（一般默认：30000-32767）分配端口，每个 Node 将从该端口（每个 Node
上的同一端口）代理到 Service。像上面的例子那样如果不指定的话会自动生成一个随机的端口。
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ app.kubernetes.io/instance: doriscluster-sample
+ app.kubernetes.io/part-of: doris-operator
+ name: doriscluster-sample
+spec:
+ feSpec:
+ replicas: 3
+ service:
+ type: NodePort
+ servicePorts:
+ - nodePort: 31001
+ targetPort: 8030
+ - nodePort: 31002
+ targetPort: 9020
+ - nodePort: 31003
+ targetPort: 9030
+ - nodePort: 31004
+ targetPort: 9010
+ limits:
+ cpu: 6
+ memory: 12Gi
+ requests:
+ cpu: 6
+ memory: 12Gi
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ beSpec:
+ replicas: 3
+ service:
+ type: NodePort
+ servicePorts:
+ - nodePort: 31005
+ targetPort: 9060
+ - nodePort: 31006
+ targetPort: 8040
+ - nodePort: 31007
+ targetPort: 9050
+ - nodePort: 31008
+ targetPort: 8060
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ image: selectdb/doris.be-ubuntu:2.0.2
+```
+部署后，通过查看 `kubectl get service` 的命令查看相应的 Service，访问方式可以参考上文 ：
+```shell
+$ kubectl get service
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
+kubernetes ClusterIP 10.152.183.1 <none>
443/TCP 169d
+doriscluster-sample-fe-internal ClusterIP None <none>
9030/TCP 2d
+doriscluster-sample-fe-service NodePort 10.152.183.67 <none>
8030:31001/TCP,9020:31002/TCP,9030:31003/TCP,9010:31004/TCP 2d
+doriscluster-sample-be-internal ClusterIP None <none>
9050/TCP 2d
+doriscluster-sample-be-service NodePort 10.152.183.24 <none>
9060:31005/TCP,8040:31006/TCP,9050:31007/TCP,8060:31008/TCP 2d
+```
+
+## Doris 数据交互
+### Stream load
+[Stream
load](https://doris.apache.org/zh-CN/docs/data-operate/import/import-way/stream-load-manual)
是一个同步的导入方式，用户通过发送 HTTP 协议发送请求将本地文件或数据流导入到 Doris 中。
+在常规部署中，用户通过 HTTP 协议提交导入命令。一般用户会将请求提交到 FE，则 FE 会通过 HTTP redirect 指令将请求转发给某一个
BE。但是，在基于 Kubernetes 部署的场景下，推荐用户 **直接提交导入命令 BE 的 Srevice** ，再由 Service 依据
Kubernetes 规则负载均衡到某一 BE 的 pod 上。
+这两种操作效果的实际效果都是一样的，在 Flink 或 Spark 使用官方 connecter 提交的时候，也可以将写入请求提交给 BE Service。
+
+### ErrorURL 查看
+诸如 [Stream
load](https://doris.apache.org/zh-CN/docs/data-operate/import/import-way/stream-load-manual)
，[Routine
load](https://doris.apache.org/zh-CN/docs/data-operate/import/import-way/routine-load-manual)
+这些导入方式，在遇到像数据格式有误等错误的时候，会在返回结构体或者日志中打印 `errorURL` 或 `tracking_url`。
通过访问此链接可以定位导入错误原因。
+但是此 URL 是仅在 Kubernetes 部署的集群中某一个特定的 BE 节点容器内部环境可访问。
+
+以下方案，以 Doris 返回的 errorURL 为例展开：
+```http://doriscluster-sample-be-2.doriscluster-sample-be-internal.doris.svc.cluster.local:8040/api/_load_error_log?file=__shard_1/error_log_insert_stmt_af474190276a2e9c-49bb9d175b8e968e_af474190276a2e9c_49bb9d175b8e968e```
+
+**1. Kubernetes 集群内部访问**
+
+需要通过 `kubectl get service` 或 `kubectl get pod -o wide` 命令获取 BE 的 Service 或 pod
的访问方式来进行原 URL 的域名端口替换，然后再访问。
+
+比如：
+```shell
+$ kubectl get pod -o wide
+NAME READY STATUS RESTARTS AGE IP
NODE NOMINATED NODE READINESS GATES
+doriscluster-sample-be-0 1/1 Running 0 9h 10.0.2.105
10-0-2-47.ec2.internal <none> <none>
+doriscluster-sample-be-1 1/1 Running 0 9h 10.0.2.104
10-0-2-5.ec2.internal <none> <none>
+doriscluster-sample-be-2 1/1 Running 0 9h 10.0.2.103
10-0-2-6.ec2.internal <none> <none>
+doriscluster-sample-fe-0 1/1 Running 0 9h 10.0.2.102
10-0-2-47.ec2.internal <none> <none>
+doriscluster-sample-fe-1 1/1 Running 0 9h 10.0.2.101
10-0-2-5.ec2.internal <none> <none>
+doriscluster-sample-fe-2 1/1 Running 0 9h 10.0.2.100
10-0-2-6.ec2.internal <none> <none>
+```
+上述 errorURL 则改为：
+```http://10.0.2.103:8040/api/_load_error_log?file=__shard_1/error_log_insert_stmt_af474190276a2e9c-49bb9d175b8e968e_af474190276a2e9c_49bb9d175b8e968e```
+
+
+**2. Kubernetes 集群外部访问 NodePort 模式**
+
+从 Kubernetes 外部获取报错详情 需要额外的桥接⼿段实现，以下是在部署 Doris 时采用 NodePort 模式的 Service
的处理步骤，通过新建 Service 的⽅式来获取报错详情：
+处理 Service 模板 be_streamload_errror_service.yaml :
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+ labels:
+ app.doris.service/role: debug
+ app.kubernetes.io/component: be
+ name: doriscluster-detail-error
+spec:
+ externalTrafficPolicy: Cluster
+ internalTrafficPolicy: Cluster
+ ipFamilies:
+ - IPv4
+ ipFamilyPolicy: SingleStack
+ ports:
+ - name: webserver-port
+ port: 8040
+ protocol: TCP
+ targetPort: 8040
+ selector:
+ app.kubernetes.io/component: be
+ statefulset.kubernetes.io/pod-name: ${podName}
+ sessionAffinity: None
+ type: NodePort
+```
+
+`podName` 则替换为 errorURL 的最⾼级域名：`doriscluster-sample-be-2`。
+
+在 Doris 部署的 `namespace`（一般默认 `doris` ， 以下操作使用 `doris`） 使⽤如下命令部署上述处理过的 Service：
+
+```shell
+$ kubectl apply -n doris -f be_streamload_errror_service.yaml
+```
+
+通过如下命令查看 Service 8040 端⼝在宿主机的映射
+```shell
+$ kubectl get service -n doris doriscluster-detail-error
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
+doriscluster-detail-error NodePort 10.152.183.35 <none>
8040:31201/TCP 32s
+```
+使⽤任何⼀台宿主机的 IP 和上面获得的 NodePort（31201）端⼝访问 替换 errorURL 获取详细报错报告。
+上述 errorURL 则改为：
+```http://10.152.183.35:31201/api/_load_error_log?file=__shard_1/error_log_insert_stmt_af474190276a2e9c-49bb9d175b8e968e_af474190276a2e9c_49bb9d175b8e968e```
+
+**3. Kubernetes 集群外部访问 LoadBalancer 模式**
+
+从 Kubernetes 外部获取报错详情 需要额外的桥接⼿段实现，以下是在部署 Doris 时采用 LoadBalancer 模式的 Service
的处理步骤，通过新建 Service 的⽅式来获取报错详情：
+处理 Service 模板 be_streamload_errror_service.yaml :
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+ labels:
+ app.doris.service/role: debug
+ app.kubernetes.io/component: be
+ name: doriscluster-detail-error
+spec:
+ externalTrafficPolicy: Cluster
+ internalTrafficPolicy: Cluster
+ ipFamilies:
+ - IPv4
+ ipFamilyPolicy: SingleStack
+ ports:
+ - name: webserver-port
+ port: 8040
+ protocol: TCP
+ targetPort: 8040
+ selector:
+ app.kubernetes.io/component: be
+ statefulset.kubernetes.io/pod-name: ${podName}
+ sessionAffinity: None
+ type: LoadBalancer
+```
+
+`podName` 则替换为 errorURL 的最⾼级域名：`doriscluster-sample-be-2`。
+
+在 Doris 部署的 `namespace`（一般默认 `doris` ， 以下操作使用 `doris`） 使⽤如下命令部署上述处理过的 Service：
+
+```shell
+$ kubectl apply -n doris -f be_streamload_errror_service.yaml
+```
+
+通过如下命令查看 Service 8040 端⼝在宿主机的映射
+```shell
+$ kubectl get service -n doris doriscluster-detail-error
+NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
+doriscluster-detail-error LoadBalancer 172.20.183.136
ac4828493dgrftb884g67wg4tb68gyut-1137856348.us-east-1.elb.amazonaws.com
8040:32003/TCP 14s
+```
+
+使⽤ `EXTERNAL-IP` 和 Port（8040）端⼝访问 替换 errorURL 获取详细报错报告。
+上述 errorURL 则改为：
+```http://ac4828493dgrftb884g67wg4tb68gyut-1137856348.us-east-1.elb.amazonaws.com:8040/api/_load_error_log?file=__shard_1/error_log_insert_stmt_af474190276a2e9c-49bb9d175b8e968e_af474190276a2e9c_49bb9d175b8e968e```
+
+注意：上述设置集群外访问的方法，建议使用完毕后清除掉当前 Service，参考命令如下：
+```shell
+$ kubectl delete service -n doris doriscluster-detail-error
+```
\ No newline at end of file
diff --git a/docs/zh-CN/docs/install/k8s-deploy/operator-deploy.md
b/docs/zh-CN/docs/install/k8s-deploy/operator-deploy.md
index 36093d758d6..91ae72f4f5d 100644
--- a/docs/zh-CN/docs/install/k8s-deploy/operator-deploy.md
+++ b/docs/zh-CN/docs/install/k8s-deploy/operator-deploy.md
@@ -39,7 +39,7 @@ Doris-Operator 是按照 Kubernetes 原则构建的在 Kubernetes 平台之上
- 创建 TKE 集群
如果你使用腾讯云可以按照腾讯云TKE相关文档创建 [TKE
集群](https://cloud.tencent.com/document/product/457/54231)。
- 创建私有集群
-私有集群搭建，我们建议按照官方推荐的方式搭建，比如：[minikube](https://minikube.sigs.k8s.io/docs/start/)，[kOps](https://kubernetes.io/zh-cn/docs/setup/production-environment/tools/kops/)。
+
私有集群搭建，我们建议按照官方推荐的方式搭建，比如：[minikube](https://minikube.sigs.k8s.io/docs/start/)，[kOps](https://kops.sigs.k8s.io/)。
### 部署 Doris-Operator
**1. 添加 DorisCluster
[资源定义](https://kubernetes.io/zh-cn/docs/concepts/extend-kubernetes/api-extension/custom-resources/)**
diff --git a/docs/zh-CN/docs/install/k8s-deploy/persistent-volume.md
b/docs/zh-CN/docs/install/k8s-deploy/persistent-volume.md
new file mode 100644
index 00000000000..ed753ae6435
--- /dev/null
+++ b/docs/zh-CN/docs/install/k8s-deploy/persistent-volume.md
@@ -0,0 +1,421 @@
+---
+{
+ "title": "持久化存储 与 ConfigMap",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+Doris-Operator 支持 Doris 各个组件的 pod 挂载 PV（Persistent Volume）。
+
+PV 一般由 kubernetes 系统管理员创建，Doris-Operator 部署 Doris 服务的时候不直接使用 PV，而是通过 PVC
声明一组资源来向 kubernetes 集群申请 PV。
+当 PVC 被创建时，Kubernetes 将尝试将其与符合要求的可用 PV 进行绑定。
+StorageClass 屏蔽了管理员手动创建 PV 的过程，对于没有现成的 PV 满足 PVC 需求时，可以根据 StorageClass 动态分配 PV。
+PV
提供多种存储类型，主要分为两大类：网络存储、本地存储。两者基于各自原理和实现，为用户提供不同的性能和使用方式的体验，用户可以依据自己的容器化的服务类型和自身需求选择。
+
+如果部署时未对 PVC 进行配置，Doris-Operator 默认 使用 `emptyDir` 模式来存储 元数据 数据文件 和 运行日志。当 pod
重新启动时，相关数据将会丢失。
+
+建议持久化存储的节点目录类型：
+* FE：doris-meta、log
+* BE：storage、log
+* CN：storage、log
+* BROKER：log
+
+Doris-Operator 同时将日志输出到 console 和 指定目录下。如果用户的 Kubernetes 系统有完整的日志收集能力，可通过
console 输出来收集 Doris INFO 级别（默认）的日志信息。
+但是这里仍然推荐配置 PVC 来持久化日志文件，因为除了 INFO 级别日志还会有诸如 fe.out、be.out、audit.log 以及
垃圾回收日志，便于快速定位问题和审计日志回溯。
+
+
+ConfigMap 是 Kubernetes
中用于存储配置文件的资源对象，它允许动态挂载配置文件，并将配置文件与应用程序解耦，使得配置的管理更加灵活和可维护。
+像 PVC 一样 ConfigMap 可以被 Pod 引用，以便在应用程序中使用配置数据。
+
+
+## StorageClass
+
+Doris-Operator 提供了使用 Kubernetes 默认 `StorageClass` 模式来支持 FE 和 BE
数据存储，其中存储路径（mountPath）使用镜像里的默认配置。
+如果用户需要自己指定 StorageClass 则需要在 `spec.feSpec.persistentVolumes` 内修改
`persistentVolumeClaimSpec.storageClassName`，参考如下：
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ name: doriscluster-sample-storageclass1
+spec:
+ feSpec:
+ replicas: 3
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ persistentVolumes:
+ - mountPath: /opt/apache-doris/fe/doris-meta
+ name: storage0
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ storageClassName: ${your_storageclass}
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ # notice: if the storage size less 5G, fe will not start normal.
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/fe/log
+ name: storage1
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ storageClassName: ${your_storageclass}
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ beSpec:
+ replicas: 3
+ image: selectdb/doris.be-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ persistentVolumes:
+ - mountPath: /opt/apache-doris/be/storage
+ name: storage2
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ storageClassName: ${your_storageclass}
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/be/log
+ name: storage3
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ storageClassName: ${your_storageclass}
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+```
+
+## 定制化 ConfigMap
+Doris 在 Kubernetes 使用 `ConfigMap` 实现配置文件和服务解耦。 在部署 `doriscluster` 之前需要提前在同
`namespace` 下部署想要使用的 `ConfigMap`，以下样例展示了 FE 使用名称为 fe-configmap 的 `ConfigMap`，
BE 使用名称为 be-configmap 的 `ConfigMap` 的集群相关 yaml:
+
+FE 的 ConfigMap 样例
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: fe-configmap
+ labels:
+ app.kubernetes.io/component: fe
+data:
+ fe.conf: |
+ CUR_DATE=`date +%Y%m%d-%H%M%S`
+
+ # the output dir of stderr and stdout
+ LOG_DIR = ${DORIS_HOME}/log
+
+ JAVA_OPTS="-Djavax.security.auth.useSubjectCredsOnly=false -Xss4m
-Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7
-XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC
-XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled
-XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0
-Xloggc:$DORIS_HOME/log/fe.gc.log.$CUR_DATE"
+
+ # For jdk 9+, this JAVA_OPTS will be used as default JVM options
+ JAVA_OPTS_FOR_JDK_9="-Djavax.security.auth.useSubjectCredsOnly=false
-Xss4m -Xmx8192m -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7
-XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled
-XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0
-Xlog:gc*:$DORIS_HOME/log/fe.gc.log.$CUR_DATE:time"
+
+ # INFO, WARN, ERROR, FATAL
+ sys_log_level = INFO
+
+ # NORMAL, BRIEF, ASYNC
+ sys_log_mode = NORMAL
+
+ # Default dirs to put jdbc drivers,default value is
${DORIS_HOME}/jdbc_drivers
+ # jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers
+
+ http_port = 8030
+ rpc_port = 9020
+ query_port = 9030
+ edit_log_port = 9010
+
+ enable_fqdn_mode = true
+```
+
+注意，使用 FE 的 ConfigMap ，必须为 `fe.conf` 添加 `enable_fqdn_mode = true`，具体原因可参考
[此处文档](https://doris.apache.org/zh-CN/docs/admin-manual/cluster-management/fqdn)
+
+BE 的 ConfigMap 样例
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: be-configmap
+ labels:
+ app.kubernetes.io/component: be
+data:
+ be.conf: |
+ CUR_DATE=`date +%Y%m%d-%H%M%S`
+
+ PPROF_TMPDIR="$DORIS_HOME/log/"
+
+ JAVA_OPTS="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log
-Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100
-DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
+
+ # For jdk 9+, this JAVA_OPTS will be used as default JVM options
+ JAVA_OPTS_FOR_JDK_9="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log
-Xlog:gc:$DORIS_HOME/log/be.gc.log.$CUR_DATE
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100
-DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
+
+ # since 1.2, the JAVA_HOME need to be set to run BE process.
+ # JAVA_HOME=/path/to/jdk/
+
+ #
https://github.com/apache/doris/blob/master/docs/zh-CN/community/developer-guide/debug-tool.md#jemalloc-heap-profile
+ # https://jemalloc.net/jemalloc.3.html
+
JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:15000,dirty_decay_ms:15000,oversize_threshold:0,lg_tcache_max:20,prof:false,lg_prof_interval:32,lg_prof_sample:19,prof_gdump:false,prof_accum:false,prof_leak:false,prof_final:false"
+ JEMALLOC_PROF_PRFIX=""
+
+ # INFO, WARNING, ERROR, FATAL
+ sys_log_level = INFO
+
+ # ports for admin, web, heartbeat service
+ be_port = 9060
+ webserver_port = 8040
+ heartbeat_service_port = 9050
+ brpc_port = 8060
+```
+使用以上两个 `ConfigMap` 的 `doriscluster` 部署样例:
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ name: doriscluster-sample-configmap
+spec:
+ feSpec:
+ replicas: 3
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ configMapInfo:
+ # use kubectl create configmap fe-configmap --from-file=fe.conf
+ configMapName: fe-configmap
+ resolveKey: fe.conf
+ beSpec:
+ replicas: 3
+ image: selectdb/doris.be-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ configMapInfo:
+ # use kubectl create configmap be-configmap --from-file=be.conf
+ configMapName: be-configmap
+ resolveKey: be.conf
+ brokerSpec:
+ replicas: 3
+ image: selectdb/doris.broker-ubuntu:2.0.2
+ limits:
+ cpu: 2
+ memory: 4Gi
+ requests:
+ cpu: 2
+ memory: 4Gi
+ configMapInfo:
+ # use kubectl create configmap broker-configmap
--from-file=apache_hdfs_broker.conf
+ configMapName: broker-configmap
+ resolveKey: apache_hdfs_broker.conf
+
+```
+这里的 `resolveKey` 是传入配置文件名（必须是`fe.conf`，`be.conf` 或
`apache_hdfs_broker.conf`，cn 节点也是 `be.conf`） 用以解析传入的 Doris
集群配置的文件，doris-operator 会去解析该文件去指导 doriscluster 的定制化部署。
+
+## 为 conf 目录添加特殊配置文件
+本段落用来供参考 需要在 Doris 节点的 conf 目录放置配置其他文件的容器化部署方案。比如常见的
[数据湖联邦查询](https://doris.apache.org/zh-CN/docs/lakehouse/multi-catalog/hive) 的
hdfs 配置文件映射。
+
+这里以 BE 的 ConfigMap 和 需要添加的 core-site.xml 文件为例：
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: be-configmap
+ labels:
+ app.kubernetes.io/component: be
+data:
+ be.conf: |
+ be_port = 9060
+ webserver_port = 8040
+ heartbeat_service_port = 9050
+ brpc_port = 8060
+ core-site.xml: |
+ <?xml version="1.0" encoding="UTF-8"?>
+ <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
+ <configuration>
+ <property>
+ <name>hadoop.security.authentication</name>
+ <value>kerberos</value>
+ </property>
+ </configuration>
+ ...
+```
+注意，`data` 内数据结构如下键值对映射：
+ ```yaml
+data:
+ 文件名_1:
+ 文件文本内容_1
+ 文件名_2:
+ 文件文本内容_2
+ 文件名_3:
+ 文件文本内容_3
+ ```
+
+## BE 多盘配置
+Doris 的 BE 服务支持多盘挂载，在服务器时代能够很好满足一个计算资源和存储资源不匹配的问题，同时使用多盘也能够很好提高 Doris 的存储效率。在
Kubernetes 上 Doris 同样可以挂载多盘来实现存储效益最大化。在 Kubernetes 上使用多盘需要配合配置文件一起使用。
+为实现服务和配置解耦，Doris 采用 `ConfigMap` 来作为配置的承载，实现配置文件动态挂载给服务使用。
+以下为 BE 服务使用 `ConfigMap` 来承载配置文件，挂载两块盘供BE使用的 doriscluster 配置：
+```yaml
+apiVersion: doris.selectdb.com/v1
+kind: DorisCluster
+metadata:
+ labels:
+ app.kubernetes.io/name: doriscluster
+ name: doriscluster-sample-storageclass1
+spec:
+ feSpec:
+ replicas: 3
+ image: selectdb/doris.fe-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ persistentVolumes:
+ - mountPath: /opt/apache-doris/fe/doris-meta
+ name: storage0
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ # notice: if the storage size less 5G, fe will not start normal.
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/fe/log
+ name: storage1
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ beSpec:
+ replicas: 3
+ image: selectdb/doris.be-ubuntu:2.0.2
+ limits:
+ cpu: 8
+ memory: 16Gi
+ requests:
+ cpu: 8
+ memory: 16Gi
+ configMapInfo:
+ configMapName: be-configmap
+ resolveKey: be.conf
+ persistentVolumes:
+ - mountPath: /opt/apache-doris/be/storage
+ name: storage2
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/be/storage1
+ name: storage3
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+ - mountPath: /opt/apache-doris/be/log
+ name: storage4
+ persistentVolumeClaimSpec:
+ # when use specific storageclass, the storageClassName should
reConfig, example as annotation.
+ #storageClassName: openebs-jiva-csi-default
+ accessModes:
+ - ReadWriteOnce
+ resources:
+ requests:
+ storage: 100Gi
+```
+与默认样例相比增加了 `configMapInfo` 的配置，同时也增加了一个 `persistentVolumeClaimSpec`
的配置，[`persistentVolumeClaimSpec`](https://kubernetes.io/docs/reference/kubernetes-api/config-and-storage-resources/persistent-volume-claim-v1/#PersistentVolumeClaimSpec)
完全遵循 Kubernetes 原生资源 PVC spec 的定义格式。
+样例中 `configMapInfo` 标识 BE 部署后使用同 `namespace` 下哪一个 ConfigMap 以及 哪一个 key
对应的内容作为配置文件启动，其中 key 为必须为 be.conf。以下为需要预先部署的配合上述 `doriscluster` ConfigMap 样例：
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: be-configmap
+ labels:
+ app.kubernetes.io/component: be
+data:
+ be.conf: |
+ CUR_DATE=`date +%Y%m%d-%H%M%S`
+
+ PPROF_TMPDIR="$DORIS_HOME/log/"
+
+ JAVA_OPTS="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log
-Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100
-DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
+
+ # For jdk 9+, this JAVA_OPTS will be used as default JVM options
+ JAVA_OPTS_FOR_JDK_9="-Xmx1024m -DlogPath=$DORIS_HOME/log/jni.log
-Xlog:gc:$DORIS_HOME/log/be.gc.log.$CUR_DATE
-Djavax.security.auth.useSubjectCredsOnly=false -Dsun.java.command=DorisBE
-XX:-CriticalJNINatives -DJDBC_MIN_POOL=1 -DJDBC_MAX_POOL=100
-DJDBC_MAX_IDLE_TIME=300000 -DJDBC_MAX_WAIT_TIME=5000"
+
+ # since 1.2, the JAVA_HOME need to be set to run BE process.
+ # JAVA_HOME=/path/to/jdk/
+
+ #
https://github.com/apache/doris/blob/master/docs/zh-CN/community/developer-guide/debug-tool.md#jemalloc-heap-profile
+ # https://jemalloc.net/jemalloc.3.html
+
JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:15000,dirty_decay_ms:15000,oversize_threshold:0,lg_tcache_max:20,prof:false,lg_prof_interval:32,lg_prof_sample:19,prof_gdump:false,prof_accum:false,prof_leak:false,prof_final:false"
+ JEMALLOC_PROF_PRFIX=""
+
+ # INFO, WARNING, ERROR, FATAL
+ sys_log_level = INFO
+
+ # ports for admin, web, heartbeat service
+ be_port = 9060
+ webserver_port = 8040
+ heartbeat_service_port = 9050
+ brpc_port = 8060
+
+ storage_root_path =
/opt/apache-doris/be/storage,medium:ssd;/opt/apache-doris/be/storage1,medium:ssd
+```
+在使用多盘时，`ConfigMap` 中 `storage_root_path` 对应值中的路径要与 `doriscluster` 中
`persistentVolume`
各个挂载路径对应。[`storage_root_path`](https://doris.apache.org/zh-CN/docs/admin-manual/config/be-config/?_highlight=storage_root_path#%E6%9C%8D%E5%8A%A1)
对应的书写规则请参考链接中文档。
+在使用云盘的情形下，介质统一使用 `SSD`。
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
