Gallardot commented on code in PR #17339:
URL:
https://github.com/apache/dolphinscheduler/pull/17339#discussion_r2281242993
##########
.github/workflows/e2e-k8s.yml:
##########
@@ -94,65 +94,98 @@ jobs:
kind create cluster --name dolphinscheduler --image
${kind_node_image}
kubectl version
kubectl get all --all-namespaces
- - name: Load images
+
+ - name: Build dolphinscheduler images with MySQL driver
run: |
+ wget -O mysql-connector-java-8.0.16.jar
https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar
components=("master" "worker" "api" "tools" "alert-server")
+ registry="apache"
for component in "${components[@]}"; do
- kind load docker-image apache/dolphinscheduler-${component}:ci
--name dolphinscheduler
+ if [ "$component" = "tools" ]; then
+ mkdir -p dolphinscheduler-tools
+ cp mysql-connector-java-8.0.16.jar dolphinscheduler-tools/
+ echo -e "FROM $registry/dolphinscheduler-tools:latest\nCOPY
../mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/tools/libs/" >
dolphinscheduler-tools/Dockerfile
+ docker build -t $registry/dolphinscheduler-tools:ci
dolphinscheduler-tools
+ else
+ mkdir -p dolphinscheduler-$component
+ cp mysql-connector-java-8.0.16.jar dolphinscheduler-$component/
+ echo -e "FROM
$registry/dolphinscheduler-$component:latest\nCOPY
../mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/libs/" >
dolphinscheduler-$component/Dockerfile
+ docker build -t $registry/dolphinscheduler-$component:ci
dolphinscheduler-$component
+ fi
+ kind load docker-image apache/dolphinscheduler-$component:ci
--name dolphinscheduler
done
- - name: Helm install dolphinscheduler
- working-directory: ${{ github.workspace
}}/deploy/kubernetes/dolphinscheduler
+ - name: Deploy standalone MySQL on K8s
Review Comment:
Why was Mysql used to replace PostgreSQL? Wouldn't it be better to apply the
matrix feature of github actions to test both Mysql and PostgreSQL at the same
time?
##########
.github/workflows/e2e-k8s.yml:
##########
@@ -94,65 +94,98 @@ jobs:
kind create cluster --name dolphinscheduler --image
${kind_node_image}
kubectl version
kubectl get all --all-namespaces
- - name: Load images
+
+ - name: Build dolphinscheduler images with MySQL driver
run: |
+ wget -O mysql-connector-java-8.0.16.jar
https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar
components=("master" "worker" "api" "tools" "alert-server")
+ registry="apache"
for component in "${components[@]}"; do
- kind load docker-image apache/dolphinscheduler-${component}:ci
--name dolphinscheduler
+ if [ "$component" = "tools" ]; then
+ mkdir -p dolphinscheduler-tools
+ cp mysql-connector-java-8.0.16.jar dolphinscheduler-tools/
+ echo -e "FROM $registry/dolphinscheduler-tools:latest\nCOPY
../mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/tools/libs/" >
dolphinscheduler-tools/Dockerfile
+ docker build -t $registry/dolphinscheduler-tools:ci
dolphinscheduler-tools
+ else
+ mkdir -p dolphinscheduler-$component
+ cp mysql-connector-java-8.0.16.jar dolphinscheduler-$component/
+ echo -e "FROM
$registry/dolphinscheduler-$component:latest\nCOPY
../mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/libs/" >
dolphinscheduler-$component/Dockerfile
+ docker build -t $registry/dolphinscheduler-$component:ci
dolphinscheduler-$component
+ fi
+ kind load docker-image apache/dolphinscheduler-$component:ci
--name dolphinscheduler
done
- - name: Helm install dolphinscheduler
- working-directory: ${{ github.workspace
}}/deploy/kubernetes/dolphinscheduler
+ - name: Deploy standalone MySQL on K8s
run: |
- # install helm
- curl
https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
+ # Create namespace first
+ kubectl create namespace dolphinscheduler
- # install ds chart via helm
- helm repo add bitnami https://charts.bitnami.com/bitnami
- helm dependency update .
- helm upgrade --install --create-namespace -n dolphinscheduler
dolphinscheduler . -f - <<EOF
- image:
- registry: apache
- tag: ci
- master:
+ # Create Kubernetes manifests for MySQL based on docker-compose
+ cat <<EOF > mysql-k8s.yaml
+ apiVersion: v1
+ kind: Service
+ metadata:
+ name: mysql
+ namespace: dolphinscheduler
+ spec:
+ ports:
+ - port: 3306
+ selector:
+ app: mysql
+ ---
+ apiVersion: apps/v1
+ kind: Deployment
+ metadata:
+ name: mysql
+ namespace: dolphinscheduler
+ spec:
replicas: 1
- livenessProbe:
- initialDelaySeconds: 120
- readinessProbe:
- initialDelaySeconds: 120
- worker:
- replicas: 1
- livenessProbe:
- initialDelaySeconds: 120
- readinessProbe:
- initialDelaySeconds: 120
- alert:
- livenessProbe:
- initialDelaySeconds: 120
- readinessProbe:
- initialDelaySeconds: 120
- api:
- livenessProbe:
- initialDelaySeconds: 120
- readinessProbe:
- initialDelaySeconds: 120
+ selector:
+ matchLabels:
+ app: mysql
+ template:
+ metadata:
+ labels:
+ app: mysql
+ spec:
+ containers:
+ - name: mysql
+ image: mysql:8.0.33
+ ports:
+ - containerPort: 3306
+ env:
+ - name: MYSQL_ROOT_PASSWORD
+ value: "root"
+ - name: MYSQL_DATABASE
+ value: "dolphinscheduler"
EOF
- - name: Wait for pods
+ # Apply the manifests to deploy MySQL
+ kubectl apply -f mysql-k8s.yaml
+
+ # Wait for MySQL to be ready before proceeding
+ echo "Waiting for MySQL to be ready..."
+ kubectl wait --for=condition=available --timeout=300s
deployment/mysql -n dolphinscheduler
+ echo "MySQL is ready. Pods:"
+ kubectl get pods -n dolphinscheduler
+ kubectl get svc -n dolphinscheduler
+
+ - name: Helm upgrade dolphinscheduler with mysql config
Review Comment:
The action `Wait for pods` should not be deleted.
Deployment is not enough; we also need to confirm that the service can
start normally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
