weizhouapache commented on issue #3838: CS 4.11.2+ SSVM bug when scanning templates URL: https://github.com/apache/cloudstack/issues/3838#issuecomment-578809423 > @weizhouapache As I fixed (workaround) the problem yesterday, I don't have a chance to debug it any longer. But I suppose that If you are able to add `sleep 60` in front of `listvmtmplt.sh` before it returns anything, you will get the result I have. @bwsw I am able to reproduce the issue steps below 1. add "sleep 60" in /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh, for example ``` root@s-2-VM:~# diff -u /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh.old /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh --- /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh.old 2020-01-27 10:54:11.561025172 +0000 +++ /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh 2020-01-27 10:32:14.665038400 +0000 @@ -48,6 +48,8 @@ exit 2 fi +sleep 60 + for i in $(find /$rootdir -name template.properties ); do d=$(dirname $i) ``` 2. change log level in SSVM ``` sed -i "s/INFO/DEBUG/g" /usr/local/cloud/systemvm/conf/log4j-cloud.xml ``` 3. restart cloud service in SSVM, by "systemctl restart cloud", and monitor /var/log/cloud.log 4. When you see the following logs (sleep 60),restart cloud service again. ``` 2020-01-27 10:51:20,982 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:null) Executing: /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/485ccf0d-3724-3575-a65e-318913593f8e/template/tmpl/ 2020-01-27 10:51:20,985 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:null) Executing while with timeout : 3600000 ``` 5. you will see the logs below in /var/log/cloud.log ``` 2020-01-27 10:51:25,967 INFO [cloud.agent.Agent] (AgentShutdownThread:null) Stopping the agent: Reason = sig.kill 2020-01-27 10:51:25,975 DEBUG [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:null) Exit value is 143 2020-01-27 10:51:25,992 WARN [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:null) Exception: /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r /mnt/SecStorage/485ccf0d-3724-3575-a65e-318913593f8e/template/tmpl/ java.io.IOException: Stream closed at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170) at java.io.BufferedInputStream.read(BufferedInputStream.java:336) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.readLine(BufferedReader.java:324) at java.io.BufferedReader.readLine(BufferedReader.java:389) at com.cloud.utils.script.OutputInterpreter.processError(OutputInterpreter.java:37) at com.cloud.utils.script.Script.execute(Script.java:289) at org.apache.cloudstack.storage.template.DownloadManagerImpl.listTemplates(DownloadManagerImpl.java:826) at org.apache.cloudstack.storage.template.DownloadManagerImpl.gatherTemplateInfo(DownloadManagerImpl.java:841) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:2052) at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:306) at com.cloud.agent.Agent.processRequest(Agent.java:640) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1053) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2020-01-27 10:51:26,005 DEBUG [cloud.agent.Agent] (AgentShutdownThread:null) Sending shutdown to management server 2020-01-27 10:51:26,019 INFO [storage.template.DownloadManagerImpl] (agentRequest-Handler-1:null) found 0 templates[] 2020-01-27 10:51:26,025 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) Seq 7-2120350999561371653: { Ans: , MgmtId: 6694243406180, via: 7, Ver: v1, Flags: 110, [{"com.cloud.agent.api.storage.ListTemplateAnswer":{"secUrl":"nfs://10.11.118.41:/data/secondary1","templateInfos":{},"result":true,"details":"success","wait":0}}] } 2020-01-27 10:51:26,066 DEBUG [utils.nio.NioConnection] (Agent-NioConnectionHandler-1:null) Location 1: Socket Socket[addr=/10.11.118.40,port=8250,localport=52080] closed on read. Probably -1 returned: Connection closed with -1 on reading size. 2020-01-27 10:51:26,067 DEBUG [utils.nio.NioConnection] (Agent-NioConnectionHandler-1:null) Closing socket Socket[addr=/10.11.118.40,port=8250,localport=52080] 2020-01-27 10:51:29,243 INFO [cloud.agent.AgentShell] (main:null) Agent started ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
