[GitHub] [cloudstack] weizhouapache commented on pull request #7118: CKS: fix upgrade of HA cluster

2023-01-23 Thread via GitHub


weizhouapache commented on PR #7118:
URL: https://github.com/apache/cloudstack/pull/7118#issuecomment-1399942220

   @DaanHoogland @davidjumani @shwstppr 
   the testing results look fine (all pass without smoke test failures). This 
is ready for review/testing
   
   This PR has been tested with upgrade
   - v1.23.3 -> v1.24.0
   - v1.24.0 -> v1.25.0
   - v1.25.0 -> v1.26.0
   
   main ideas
   (1) the current command to get k8s versions does not work well, see 
https://github.com/kubernetes/kubernetes/issues/115130, 
   (2) therefore use 'kubectl get nodes' on the primary control master to get 
the k8s version of all nodes. However the version is not updated immediately 
after upgrade. so wait 10times*15seconds=150seconds (a bit too long) for the 
k8s version to be updated. For example during upgrade from v1.23.3 to v1.24.0, 
the output of k8s version is in the order:
   
   - Ready,SchedulingDisabled with v1.23.3
   - Ready with v1.23.3
   - NotReady with v1.23.3
   - Ready with v1.24.0
   
   see below log
   ```
   = loop 228 at Fri 20 Jan 2023 04:33:45 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready,SchedulingDisabled   control-plane  
 3h37m   v1.23.3
   = loop 229 at Fri 20 Jan 2023 04:33:47 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready,SchedulingDisabled   control-plane  
 3h37m   v1.23.3
   = loop 230 at Fri 20 Jan 2023 04:33:48 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready,SchedulingDisabled   control-plane  
 3h37m   v1.23.3
   = loop 231 at Fri 20 Jan 2023 04:33:49 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready,SchedulingDisabled   control-plane  
 3h37m   v1.23.3
   
   = loop 232 at Fri 20 Jan 2023 04:33:50 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Readycontrol-plane   3h37m   v1.23.3
   = loop 233 at Fri 20 Jan 2023 04:33:52 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Readycontrol-plane   3h37m   v1.23.3
   = loop 234 at Fri 20 Jan 2023 04:33:53 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Readycontrol-plane   3h37m   v1.23.3
   = loop 235 at Fri 20 Jan 2023 04:33:54 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Readycontrol-plane   3h37m   v1.23.3
   = loop 236 at Fri 20 Jan 2023 04:33:56 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Readycontrol-plane   3h37m   v1.23.3
   = loop 237 at Fri 20 Jan 2023 04:33:57 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Readycontrol-plane   3h37m   v1.23.3
   = loop 238 at Fri 20 Jan 2023 04:33:58 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Readycontrol-plane   3h37m   v1.23.3
   = loop 239 at Fri 20 Jan 2023 04:33:59 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Readycontrol-plane   3h37m   v1.23.3
   = loop 240 at Fri 20 Jan 2023 04:34:01 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Readycontrol-plane   3h37m   v1.23.3
   
   = loop 241 at Fri 20 Jan 2023 04:34:02 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   NotReady   control-plane   3h37m   v1.24.0
   = loop 242 at Fri 20 Jan 2023 04:34:03 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   NotReady   control-plane   3h37m   v1.24.0
   = loop 243 at Fri 20 Jan 2023 04:34:04 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   NotReady   control-plane   3h37m   v1.24.0
   = loop 244 at Fri 20 Jan 2023 04:34:06 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   NotReady   control-plane   3h37m   v1.24.0
   
   = loop 245 at Fri 20 Jan 2023 04:34:07 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready  control-plane  
 3h37m   v1.24.0
   = loop 246 at Fri 20 Jan 2023 04:34:13 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready  control-plane  
 3h37m   v1.24.0
   = loop 247 at Fri 20 Jan 2023 04:34:14 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready  control-plane  
 3h37m   v1.24.0
   = loop 248 at Fri 20 Jan 2023 04:34:16 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready  control-plane  
 3h37m   v1.24.0
   = loop 249 at Fri 20 Jan 2023 04:34:17 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready  control-plane  
 3h38m   v1.24.0
   = loop 250 at Fri 20 Jan 2023 04:34:18 PM UTC ==
   cloud-1-23-3-control-185cf3d5af9   Ready  control-plane  
 3h38m   v1.24.0
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[GitHub] [cloudstack] weizhouapache commented on pull request #7118: CKS: fix upgrade of HA cluster

2023-01-21 Thread via GitHub


weizhouapache commented on PR #7118:
URL: https://github.com/apache/cloudstack/pull/7118#issuecomment-1399224517

   @blueorangutan test matrix


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[GitHub] [cloudstack] weizhouapache commented on pull request #7118: CKS: fix upgrade of HA cluster

2023-01-20 Thread via GitHub


weizhouapache commented on PR #7118:
URL: https://github.com/apache/cloudstack/pull/7118#issuecomment-1398980751

   @blueorangutan test keepEnv


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[GitHub] [cloudstack] weizhouapache commented on pull request #7118: CKS: fix upgrade of HA cluster

2023-01-20 Thread via GitHub


weizhouapache commented on PR #7118:
URL: https://github.com/apache/cloudstack/pull/7118#issuecomment-1398878961

   @blueorangutan package


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@cloudstack.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org