[GitHub] cloudstack pull request #2030: WIP: CLOUDSTACK-9864 cleanup stale worker VMs...

[DaanHoogland]

GitHub user DaanHoogland reopened a pull request:

https://github.com/apache/cloudstack/pull/2030

WIP: CLOUDSTACK-9864 cleanup stale worker VMs after job expiry time



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shapeblue/cloudstack snapshot-housekeeping

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/2030.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2030


commit 40869570fc510fac0d2357f272e96cd4a4518176
Author: Daan Hoogland 
Date:   2017-03-30T14:35:37Z

CE-113 trace logging and rethrow instead of nesting CloudRuntimeException

commit 66d7d846352d52cc539b1dafb5e4d0f1620829a5
Author: Daan Hoogland 
Date:   2017-04-05T12:19:14Z

CE-113 configure workervm gc based on job expiry

commit 996f5834e6a0a9e4dc57d436ceeb5b89e6dc9974
Author: Daan Hoogland 
Date:   2017-04-05T15:35:41Z

CE-113 extra trace log of worker VMs

commit 9a8ea7c0d1c9775ad7e4200db2b3eca93e121909
Author: Daan Hoogland 
Date:   2017-04-06T09:33:53Z

CE-113 removed TODOs

commit e2c0f09609b48f4539f13edcc742ca7e06f0cca2
Author: Daan Hoogland 
Date:   2017-04-07T12:54:19Z

CE-113 use of duration (instead of the old clock-tick-based code




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #2030: WIP: CLOUDSTACK-9864 cleanup stale worker VMs...

[DaanHoogland]

Github user DaanHoogland closed the pull request at:

https://github.com/apache/cloudstack/pull/2030


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #2030: WIP: CLOUDSTACK-9864 cleanup stale worker VMs...

[abhinandanprateek]

Github user abhinandanprateek commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/2030#discussion_r110145877
  
--- Diff: 
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/manager/VmwareManagerImpl.java
 ---
@@ -550,15 +552,21 @@ public boolean needRecycle(String workerTag) {
 return true;
 }
 
-// disable time-out check until we have found out a VMware API 
that can check if
-// there are pending tasks on the subject VM
-/*
-if(System.currentTimeMillis() - startTick > 
_hungWorkerTimeout) {
-if(s_logger.isInfoEnabled())
-s_logger.info("Worker VM expired, seconds elapsed: 
" + (System.currentTimeMillis() - startTick) / 1000);
-return true;
-}
- */
+// this time-out check was disabled
+// "until we have found out a VMware API that can check if there 
are pending tasks on the subject VM"
+// but as we expire jobs and those stale worker VMs stay around 
untill an MS reboot we opt in to have them removed anyway
+Long hungWorkerTimeout = 2 * 
(AsyncJobManagerImpl.JobExpireMinutes.value() + 
AsyncJobManagerImpl.JobCancelThresholdMinutes.value()) * MILISECONDS_PER_MINUTE;
+Long letsSayNow = System.currentTimeMillis();
+if(s_vmwareCleanOldWorderVMs.value() && letsSayNow - startTick > 
hungWorkerTimeout) {
+if(s_logger.isInfoEnabled()) {
+s_logger.info("Worker VM expired, seconds elapsed: " + 
(System.currentTimeMillis() - startTick) / 1000);
+}
--- End diff --

For timeouts you may want to use java Duration, that is much cleaner.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #2030: WIP: CLOUDSTACK-9864 cleanup stale worker VMs...

[abhinandanprateek]

Github user abhinandanprateek commented on a diff in the pull request:

https://github.com/apache/cloudstack/pull/2030#discussion_r110145761
  
--- Diff: 
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/manager/VmwareManagerImpl.java
 ---
@@ -128,6 +129,7 @@
 public class VmwareManagerImpl extends ManagerBase implements 
VmwareManager, VmwareStorageMount, Listener, VmwareDatacenterService, 
Configurable {
 private static final Logger s_logger = 
Logger.getLogger(VmwareManagerImpl.class);
 
+private static final long MILISECONDS_PER_MINUTE = 6;
--- End diff --

MILI typo MILLISECONDS ..


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request #2030: WIP: CLOUDSTACK-9864 cleanup stale worker VMs...

[DaanHoogland]

GitHub user DaanHoogland opened a pull request:

https://github.com/apache/cloudstack/pull/2030

WIP: CLOUDSTACK-9864 cleanup stale worker VMs after job expiry time



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/shapeblue/cloudstack snapshot-housekeeping

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/2030.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2030


commit 40869570fc510fac0d2357f272e96cd4a4518176
Author: Daan Hoogland 
Date:   2017-03-30T14:35:37Z

CE-113 trace logging and rethrow instead of nesting CloudRuntimeException

commit 66d7d846352d52cc539b1dafb5e4d0f1620829a5
Author: Daan Hoogland 
Date:   2017-04-05T12:19:14Z

CE-113 configure workervm gc based on job expiry

commit 996f5834e6a0a9e4dc57d436ceeb5b89e6dc9974
Author: Daan Hoogland 
Date:   2017-04-05T15:35:41Z

CE-113 extra trace log of worker VMs




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---