[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Bernhard Kaszt commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 I think the problem was already fixed, now with the latest versions it's happening again... Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Dax Games commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 I can confirm this is still happening with Kubernetes Plugin 1.15.1 and 1.15.2 on Jenkins 2.150.3. In my case it spins up a new pod every 10 seconds until the first one connects. I just watched a single job spin up 10 pods and the moment the first pod connected the plugin stopped spinning new pods. This is a HUGE problem. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Carlos Sanchez reopened an issue Jenkins / JENKINS-47501 2 pods are started instead of 1 after update to version 1.1 Change By: Carlos Sanchez Resolution: Fixed Status: Resolved Reopened Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Erik Kristensen commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 I can confirm this is happening in 1.12.4 still. I'm seeing that if you have multiple jobs pending, and your jnlp container is crashing or and additional containers crash, the plugin exponentially tries to create more agents. It basically DDoS my k8s cluster. I had 64 workloads in the process of being added/removed due to a typo in the Jenkins Hostname for the jnlp agents to connect to. I had 20 jobs pending and the maximum for containers and instances are both set to 1. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Nicolas De Loof commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 Sounds to me this is a very common issue and this mechanism should be implemented by NodeProvisioner, and not re-implemented with various approach by each and every cloud-plugin Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Aiman Alsari commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 Just to update: I have found a suitable workaround. The premature call to terminate seems to come from the OnceRetentionStrategy. So if you set "idleMinutes: 1" in your podTemplate, it will use a different retention strategy. Unfortunately this means your containers get re-used so you can't guarantee the clean state of them, but it's better than nothing. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Aiman Alsari commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 I found that someone else has also seen this issue. See Yuan Yao's comment on JENKINS-44042 Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Aiman Alsari commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 I can confirm this is still happening on 1.5.2 - The most reliable way of reproducing this issue is to run the test case provided by the OP, have a single job running, queue up 2 or 3 behind it using the same pod template, then abort the first one. What happens is that 2 or more pods are provisioned simultaneously, disregarding the limit. This isn't that big of a deal (for me at least) but then it gets into a crazy crash loop. Something is calling "KubernetesSlave._terminate()" on one of the newly provisioned pods, it can then fail in one of two ways: The pod is terminated and the provisioning halts resulting in "java.lang.IllegalStateException: Pod no longer exists:" The master has removed the node from it's list of nodes and so JNLP fails. On the master it has "Refusing headers from remote: Unknown client name: foo-bar". On the actual JNLP pod it has "The server rejected the connection: None of the protocols were accepted". There seems to be a race condition somewhere, something is calling the _terminate method for this pod right when it starts up. My first guess was that when the first aborted job is terminating, provisioning of the two new pods happens simultaneously and the termination code of the first job then kills the new ones as well as the first one. This doesn't seem to be the case though as I can not replicate the issue reliably with just one active + one queued job, it has to be 2 or 3. Once in this state the only way to fix it is to delete all the queued jobs and clean all Error'ed pods. jenkins_log Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Aiman Alsari updated an issue Jenkins / JENKINS-47501 2 pods are started instead of 1 after update to version 1.1 Change By: Aiman Alsari Attachment: jenkins_log Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Carlos Rojas commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 I have installed the latest version 1.5.2, and I experimented the same issue, i setup only 3 for container cap and run more than 5 containers at the same time Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Carlos Sanchez updated JENKINS-47501 Fixed in 1.3.1 Jenkins / JENKINS-47501 2 pods are started instead of 1 after update to version 1.1 Change By: Carlos Sanchez Status: In Review Resolved Resolution: Fixed Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Vincent Latombe updated JENKINS-47501 Jenkins / JENKINS-47501 2 pods are started instead of 1 after update to version 1.1 Change By: Vincent Latombe Status: In Progress Review Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Vincent Latombe assigned an issue to Vincent Latombe Jenkins / JENKINS-47501 2 pods are started instead of 1 after update to version 1.1 Change By: Vincent Latombe Assignee: Jae Gangemi Vincent Latombe Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Vincent Latombe started work on JENKINS-47501 Change By: Vincent Latombe Status: Open In Progress Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
