[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title Alexander Komarov commented on JENKINS-53625 Re: Provisioning an advanced linux image never continues after init.sh You may find this useful: This init script causes Jenkins to hang (although JDK gets installed and the script completes): wget https://cdn.azul.com/zulu/bin/zulu8.31.0.1-jdk8.0.181-linux_amd64.deb -O zulu8.deb sudo apt-get install -y ./zulu8.deb But this one results in a correctly provisioned slave: sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0xB1998361219BD9C9 sudo apt-add-repository 'deb http://repos.azulsystems.com/ubuntu stable main' sudo apt-get install -y zulu-8 Even though I'm installing the same package, how I'm installing it matters somehow. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title Alexander Komarov commented on JENKINS-53625 Re: Provisioning an advanced linux image never continues after init.sh I just tried with the recommended init.sh (OpenJDK install script copied from the help text in the Jenkins configure screen), and provisioning worked normally. Perhaps there is something about Zulu JDK installation that hangs the shell? It doesn't happen when I try it interactively. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title Jie Shen commented on JENKINS-53625 Re: Provisioning an advanced linux image never continues after init.sh Alexander Komarov, thanks for your reporting and investigation. I will look at it. Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title Jie Shen assigned an issue to Jie Shen Jenkins / JENKINS-53625 Provisioning an advanced linux image never continues after init.sh Change By: Jie Shen Assignee: Azure DevOps Jie Shen Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title Alexander Komarov commented on JENKINS-53625 Re: Provisioning an advanced linux image never continues after init.sh Same behavior with plugin version 0.7.1 Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title
Alexander Komarov edited a comment on JENKINS-53625
Re: Provisioning an advanced linux image never continues after init.sh
I was able to make the slave launching work not block by modifying *AzureVMAgentSSHLauncher.java* (executeRemoteCommand()) as follows (wrapping the IOUtils code in a new Runnable). {code:java}// Read from input streamnew Thread(new Runnable() {@Overridepublic void run() {try {try {IOUtils.copy(inputStream, logger);} finally {IOUtils.closeQuietly(inputStream);}// Read from error streamtry {IOUtils.copy(errorStream, logger);} finally {IOUtils.closeQuietly(errorStream);}} catch (Throwable t) {LOGGER.info("AzureVMAgentSSHLauncher: executeRemoteCommand: failed in read thread: " +t);}}}).start();{code} However, this also makes the code not wait for init.sh to finish, which is also bad.
Add Comment
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title
Alexander Komarov commented on JENKINS-53625
Re: Provisioning an advanced linux image never continues after init.sh
I was able to make the slave launching work by modifying AzureVMAgentSSHLauncher.java (executeRemoteCommand()) as follows (wrapping the IOUtils code in a new Runnable).
// Read from input stream
new Thread(new Runnable() {
@Override
public void run() {
try {
try {
IOUtils.copy(inputStream, logger);
} finally {
IOUtils.closeQuietly(inputStream);
}
// Read from error stream
try {
IOUtils.copy(errorStream, logger);
} finally {
IOUtils.closeQuietly(errorStream);
}
} catch (Throwable t) {
LOGGER.info("AzureVMAgentSSHLauncher: executeRemoteCommand: failed in read thread: " +t);
}
}
}).start();
Add Comment
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title
Alexander Komarov edited a comment on JENKINS-53625
Re: Provisioning an advanced linux image never continues after init.sh
After some investigation, I know what line hangs... (inputStream copy after running init.sh). I do not know why it hangs though.{noformat}"Computer.threadPoolForRemoting [#1]" daemon prio=5 TIMED_WAITING java.lang.Object.wait(Native Method) java.io.PipedInputStream.read(PipedInputStream.java:326) java.io.PipedInputStream.read(PipedInputStream.java:377) java.io.InputStream.read(InputStream.java:101) org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792) org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher.executeRemoteCommand(AzureVMAgentSSHLauncher.java:409) com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher.launch(AzureVMAgentSSHLauncher.java:172) hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294) jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748){noformat} Interestingly, nothing unblocks those threads (I now have one for each time I clicked "Relaunch Agent"). I tried stopping SSHd and even shutting down the VM - the threads keep waiting.
Add Comment
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title
Alexander Komarov edited a comment on JENKINS-53625
Re: Provisioning an advanced linux image never continues after init.sh
After some investigation, I know what line hangs... (inputStream copy after running init.sh). I do not know why it hangs though.{noformat}"Computer.threadPoolForRemoting [#1]" daemon prio=5 TIMED_WAITING java.lang.Object.wait(Native Method) java.io.PipedInputStream.read(PipedInputStream.java:326) java.io.PipedInputStream.read(PipedInputStream.java:377) java.io.InputStream.read(InputStream.java:101) org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792) org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher.executeRemoteCommand(AzureVMAgentSSHLauncher.java:409) com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher.launch(AzureVMAgentSSHLauncher.java:172) hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294) jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748){noformat}
Add Comment
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title
Alexander Komarov edited a comment on JENKINS-53625
Re: Provisioning an advanced linux image never continues after init.sh
After some investigation, I know what line hangs... (inputStream copy after running init.sh). I do not know why it hangs though.{noformat}"Computer.threadPoolForRemoting [#1]" daemon prio=5 TIMED_WAITING java.lang.Object.wait(Native Method) java.io.PipedInputStream.read(PipedInputStream.java:326) java.io.PipedInputStream.read(PipedInputStream.java:377) java.io.InputStream.read(InputStream.java:101) org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792) org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher.executeRemoteCommand(AzureVMAgentSSHLauncher.java:409) com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher.launch(AzureVMAgentSSHLauncher.java:172) hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294) jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748) {noformat}
Add Comment
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title Alexander Komarov commented on JENKINS-53625 Re: Provisioning an advanced linux image never continues after init.sh After some investigation, I know what line hangs... (inputStream copy after running init.sh). I do not know why it hangs though. "Computer.threadPoolForRemoting [#1]" daemon prio=5 TIMED_WAITING java.lang.Object.wait(Native Method) java.io.PipedInputStream.read(PipedInputStream.java:326) java.io.PipedInputStream.read(PipedInputStream.java:377) java.io.InputStream.read(InputStream.java:101) org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792) org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher.executeRemoteCommand(AzureVMAgentSSHLauncher.java:409) com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher.launch(AzureVMAgentSSHLauncher.java:172) hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294) jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748) Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title
Alexander Komarov updated an issue
Jenkins / JENKINS-53625
Provisioning an advanced linux image never continues after init.sh
Change By:
Alexander Komarov
h1. ConfigurationIn Azure, I've defined a Resource Group with a storage account and a virtual network. Launching VMs manually works. In Jenkins, I've configured a Cloud with all the settings above. I'm using an Advanced Image Configuration because I need to specify a virtual network to launch into. Settings are as follows: * Size: Standard A0 * Storage Account Type: Standard_LRS * Storage Account Name: Use existing (pre-created) * Disk: Managed * Image Reference: ** Canonical ** UbuntuServer ** 16.04.0-LTS ** latest * Launch Method: SSH * Initialization Script: {color:#00875a}wget [https://cdn.azul.com/zulu/bin/zulu8.31.0.1-jdk8.0.181-linux_amd64.deb] -O zulu8.deb{color} {color:#00875a}sudo apt install -y ./zulu8.deb{color} {color:#00875a}sudo apt install -y git{color} {color:#00875a}logger "$0 is done!!!"{color} {color:#00875a}echo "This is a success, ignoring failure"{color} * Run as root: false * Don't use VM if initialization script fails: false h1. What Happens # Instance is launched from a pipeline job ( node('matching-label') {} ) # init.sh is run (to the end). My log statement is in /var/log/syslog. # Nothing else happens. Slave is never provisioned.h1. h1. LogsI've created a Log Recorder for com.microsoft.azure.vmagent.remote and all log levels:{noformat}Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher connectToSsh AzureVMAgentSSHLauncher: connectToSsh: start Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher getRemoteSession AzureVMAgentSSHLauncher: getRemoteSession: getting remote session for user XXX to host XX.eastus.cloudapp.azure.com:22 Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher getRemoteSession AzureVMAgentSSHLauncher: getRemoteSession: Got remote session for user XXX to host XX.eastus.cloudapp.azure.com:22 Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher connectToSsh AzureVMAgentSSHLauncher: connectToSsh: Got remote connection Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher executeRemoteCommand AzureVMAgentSSHLauncher: executeRemoteCommand: starting test -e ~/.azure-agent-init Sep 17, 2018 9:59:59 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher executeRemoteCommand AzureVMAgentSSHLauncher: executeRemoteCommand: executed successfully Sep 17, 2018 9:59:59 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher launch AzureVMAgentSSHLauncher: launch: Init script is not null, preparing to execute script remotely Sep 17, 2018 9:59:59 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher copyFileToRemote AzureVMAgentSSHLauncher: copyFileToRemote: Initiating file transfer to init.sh Sep 17, 2018 10:00:00 AM FINE
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title
Alexander Komarov updated an issue
Jenkins / JENKINS-53625
Provisioning an advanced linux image never continues after init.sh
Change By:
Alexander Komarov
h1. ConfigurationIn Azure, I've defined a Resource Group with a storage account and a virtual network. Launching VMs manually works. In Jenkins, I've configured a Cloud with all the settings above. I'm using an Advanced Image Configuration because I need to specify a virtual network to launch into. Settings are as follows: * Size: Standard A0 * Storage Account Type: Standard_LRS * Storage Account Name: Use existing (pre-created) * Disk: Managed * Image Reference: ** Canonical ** UbuntuServer ** 16.04.0-LTS ** latest * Launch Method: SSH * Initialization Script: {color:#00875a}wget [https://cdn.azul.com/zulu/bin/zulu8.31.0.1-jdk8.0.181-linux_amd64.deb] -O zulu8.deb{color} {color:#00875a}sudo apt install -y ./zulu8.deb{color} {color:#00875a}sudo apt install -y git{color} {color:#00875a}logger "$0 is done!!!"{color} {color:#00875a}echo "This is a success, ignoring failure"{color} * Run as root: false * Don't use VM if initialization script fails: false h1. What Happens # Instance is launched from a pipeline job ( node('matching-label') {} ) # init.sh is run (to the end) . My log statement is in /var/log/syslog. # Nothing else happens. Slave is never provisioned.h1. LogsI've created a Log Recorder for com.microsoft.azure.vmagent.remote and all log levels:{noformat}Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher connectToSsh AzureVMAgentSSHLauncher: connectToSsh: start Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher getRemoteSession AzureVMAgentSSHLauncher: getRemoteSession: getting remote session for user XXX to host XX.eastus.cloudapp.azure.com:22 Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher getRemoteSession AzureVMAgentSSHLauncher: getRemoteSession: Got remote session for user XXX to host XX.eastus.cloudapp.azure.com:22 Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher connectToSsh AzureVMAgentSSHLauncher: connectToSsh: Got remote connection Sep 17, 2018 9:59:58 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher executeRemoteCommand AzureVMAgentSSHLauncher: executeRemoteCommand: starting test -e ~/.azure-agent-init Sep 17, 2018 9:59:59 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher executeRemoteCommand AzureVMAgentSSHLauncher: executeRemoteCommand: executed successfully Sep 17, 2018 9:59:59 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher launch AzureVMAgentSSHLauncher: launch: Init script is not null, preparing to execute script remotely Sep 17, 2018 9:59:59 AM INFO com.microsoft.azure.vmagent.remote.AzureVMAgentSSHLauncher copyFileToRemote AzureVMAgentSSHLauncher: copyFileToRemote: Initiating file transfer to init.sh Sep 17, 2018 10:00:00 AM FINE
[JIRA] (JENKINS-53625) Provisioning an advanced linux image never continues after init.sh
Title: Message Title Alexander Komarov updated an issue Jenkins / JENKINS-53625 Provisioning an advanced linux image never continues after init.sh Change By: Alexander Komarov Summary: Provisioning a custom an advanced linux image never continues after init.sh Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
