[JIRA] (JENKINS-57204) Provision failures does not clean up deployments
Title: Message Title Jie Shen commented on JENKINS-57204 Re: Provision failures does not clean up deployments I was trying to fix this at https://github.com/jenkinsci/azure-container-agents-plugin/pull/42 . The fix has been released in 1.1.0. If it does not work, is there any log for why the cleanup action fails? Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.198961.155637878.17916.1571812320579%40Atlassian.JIRA.
[JIRA] (JENKINS-57204) Provision failures does not clean up deployments
Title: Message Title
Olivier Vernin commented on JENKINS-57204
Re: Provision failures does not clean up deployments
It appears this issue is back on ci.jenkins.io, for some reason, deployments are not cleaned up 2019-10-21 10:50:55.713+ [id=55] WARNING hudson.slaves.NodeProvisioner#lambda$update$6: Unexpected exception encountered while provisioning agent aci-maven java.lang.Exception: Status code 409, {"error":{"code":"DeploymentQuotaExceeded","message":"Creating the deployment 'aci-maven-h3c19t5n' would exceed the quota of '800'. The current deployment count is '800', please delete some deployments before creating a new one. Please see https://aka.ms/arm-debug for usage details."}} at com.microsoft.jenkins.containeragents.aci.AciService.createDeployment(AciService.java:141) at com.microsoft.jenkins.containeragents.aci.AciContainerTemplate.provisionAgents(AciContainerTemplate.java:128) at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:104) Caused: java.lang.Exception at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:140) at com.microsoft.jenkins.containeragents.aci.AciCloud$1.call(AciCloud.java:80) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)
Add Comment
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
[JIRA] (JENKINS-57204) Provision failures does not clean up deployments
Title: Message Title Olivier Vernin updated JENKINS-57204 Jenkins / JENKINS-57204 Provision failures does not clean up deployments Change By: Olivier Vernin Status: In Review Progress Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.198961.155637878.11639.1571655601306%40Atlassian.JIRA.
[JIRA] (JENKINS-57204) Provision failures does not clean up deployments
Title: Message Title Olivier Vernin updated JENKINS-57204 Jenkins / JENKINS-57204 Provision failures does not clean up deployments Change By: Olivier Vernin Resolution: Fixed Status: Resolved In Review Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.198961.155637878.11636.1571655480410%40Atlassian.JIRA.
[JIRA] (JENKINS-57204) Provision failures does not clean up deployments
Title: Message Title Jie Shen resolved as Fixed Jenkins / JENKINS-57204 Provision failures does not clean up deployments Change By: Jie Shen Status: In Progress Resolved Resolution: Fixed Released As: 1.1.0 Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.198961.155637878.565.1564041600365%40Atlassian.JIRA.
[JIRA] (JENKINS-57204) Provision failures does not clean up deployments
Title: Message Title Jie Shen commented on JENKINS-57204 Re: Provision failures does not clean up deployments Thanks for reporting this issue. It is similar as JENKINS-48045 in VM agent plugin. I will make a PR soon to fix this. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-57204) Provision failures does not clean up deployments
Title: Message Title Azure DevOps started work on JENKINS-57204 Change By: Azure DevOps Status: Open In Progress Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-57204) Provision failures does not clean up deployments
Title: Message Title
R. Tyler Croy created an issue
Jenkins / JENKINS-57204
Provision failures does not clean up deployments
Issue Type:
Bug
Assignee:
Azure DevOps
Components:
azure-container-agents-plugin
Created:
2019-04-27 15:26
Priority:
Critical
Reporter:
R. Tyler Croy
We're seeing a number of failures on ci.jenkins.io which stem from Azure Container Agents failing to provision. It looks like failed ACI deployments don't get cleaned up automatically by the plugin. ACI is returning the following failure:
{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.","details":[{"code":"Conflict","message":"{\r\n \"error\": {\r\n \"code\": \"ServiceUnavailable\",\r\n \"message\": \"The requested resource is not available in the location 'eastus2' at this moment. Please retry with a different resource request or in another location. Resource requested: '2' CPU '4' GB memory 'Linux' OS\"\r\n }\r\n}"}]}
Which is annoying, but whatever. The bug here is that the plugin does not clean up this failed deployment, so we have exceeded our quota of Deployments of 800, all failures like this one! I recall this exact same issue in the Azure VM Agents plugin at one point as well.
