[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Aiman Alsari commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 Just to update: I have found a suitable workaround. The premature call to terminate seems to come from the OnceRetentionStrategy. So if you set "idleMinutes: 1" in your podTemplate, it will use a different retention strategy. Unfortunately this means your containers get re-used so you can't guarantee the clean state of them, but it's better than nothing. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Aiman Alsari commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 I found that someone else has also seen this issue. See Yuan Yao's comment on JENKINS-44042 Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Aiman Alsari commented on JENKINS-47501 Re: 2 pods are started instead of 1 after update to version 1.1 I can confirm this is still happening on 1.5.2 - The most reliable way of reproducing this issue is to run the test case provided by the OP, have a single job running, queue up 2 or 3 behind it using the same pod template, then abort the first one. What happens is that 2 or more pods are provisioned simultaneously, disregarding the limit. This isn't that big of a deal (for me at least) but then it gets into a crazy crash loop. Something is calling "KubernetesSlave._terminate()" on one of the newly provisioned pods, it can then fail in one of two ways: The pod is terminated and the provisioning halts resulting in "java.lang.IllegalStateException: Pod no longer exists:" The master has removed the node from it's list of nodes and so JNLP fails. On the master it has "Refusing headers from remote: Unknown client name: foo-bar". On the actual JNLP pod it has "The server rejected the connection: None of the protocols were accepted". There seems to be a race condition somewhere, something is calling the _terminate method for this pod right when it starts up. My first guess was that when the first aborted job is terminating, provisioning of the two new pods happens simultaneously and the termination code of the first job then kills the new ones as well as the first one. This doesn't seem to be the case though as I can not replicate the issue reliably with just one active + one queued job, it has to be 2 or 3. Once in this state the only way to fix it is to delete all the queued jobs and clean all Error'ed pods. jenkins_log Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
[JIRA] (JENKINS-47501) 2 pods are started instead of 1 after update to version 1.1
Title: Message Title Aiman Alsari updated an issue Jenkins / JENKINS-47501 2 pods are started instead of 1 after update to version 1.1 Change By: Aiman Alsari Attachment: jenkins_log Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
