[
https://issues.apache.org/jira/browse/CLOUDSTACK-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940408#comment-14940408
]
ASF GitHub Bot commented on CLOUDSTACK-8808:
--------------------------------------------
Github user remibergsma commented on the pull request:
https://github.com/apache/cloudstack/pull/901#issuecomment-144847280
@karuturi Thanks a lot, works great! LGTM
I registered a template with invalid headers, as per CLOUDSTACK-8808.
At first everything is normal:
<img width="715" alt="screen shot 2015-09-30 at 21 09 29"
src="https://cloud.githubusercontent.com/assets/1630096/10233628/027aa5da-6890-11e5-97fc-981033df91f5.png";>
It then starts installing (and unzipping):
<img width="719" alt="screen shot 2015-09-30 at 21 11 22"
src="https://cloud.githubusercontent.com/assets/1630096/10233631/0bdec520-6890-11e5-995e-f193691eb1e7.png";>
And eventually disappears from the UI. The logs show `Format is invalid` as
expected.
Full logs:
```
2015-10-01 20:55:00,624 WARN [storage.template.TemplateLocation]
(pool-1-thread-2:null) Format is invalid
2015-10-01 20:55:00,625 DEBUG [storage.template.TemplateLocation]
(pool-1-thread-2:null) Format: VHD size: 21475270656 virtualsize: 0 filename:
4db55a50-1775-3f5d-b982-962eb2d0f282.vhd
2015-10-01 20:55:00,625 DEBUG [storage.template.TemplateLocation]
(pool-1-thread-2:null) format, filename cannot be null and size, virtual size
should be > 0
2015-10-01 20:55:02,241 DEBUG [storage.template.TemplateLocation]
(pool-1-thread-2:null) Remove
/mnt/SecStorage/6cc407de-bf89-3f72-8985-1b9df6a63827/template/tmpl/2/201/4db55a50-1775-3f5d-b982-962eb2d0f282.vhd
2015-10-01 20:55:02,302 DEBUG [storage.template.TemplateLocation]
(pool-1-thread-2:null) Remove
/mnt/SecStorage/6cc407de-bf89-3f72-8985-1b9df6a63827/template/tmpl/2/201/template.properties
2015-10-01 20:55:02,867 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) Processing command:
org.apache.cloudstack.storage.command.DownloadProgressCommand
2015-10-01 20:55:02,870 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:null) Seq 3-5925611209712730174: { Ans: , MgmtId:
90520732674659, via: 3, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.storage.DownloadAnswer":{"jobId":"d0965877-d8b8-4e12-a132-703fb7ee6b54","downloadPct":100,"errorString":"Failed
post download script: Unable to install due to invalid file
format","downloadStatus":"DOWNLOAD_ERROR","downloadPath":"/mnt/SecStorage/6cc407de-bf89-3f72-8985-1b9df6a63827/template/tmpl/2/201/dnld3817046421300354449tmp_","installPath":"template/tmpl/2/201/4db55a50-1775-3f5d-b982-962eb2d0f282.vhd","templateSize":0,"templatePhySicalSize":0,"checkSum":"49c737f858448e11eb181f250e4efaeb","result":true,"details":"Failed
post download script: Unable to install due to invalid file
format","wait":0}}] }
```
When someone else reviews this we can merge it and solve another blocker.
> Successfully registered VHD template is downloaded again due to missing
> virtualsize property in template.properties
> -------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-8808
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8808
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Secondary Storage
> Affects Versions: 4.4.4, 4.6.0
> Environment: Seen on NFS as sec storage
> Reporter: Remi Bergsma
> Assignee: Rajani Karuturi
> Priority: Blocker
>
> We noticed all of our templates are downloaded again as soon as we restart
> SSVM, its Cloud service or the management server it connects to.
> A scan done by the SSVM (listvmtmplt.sh) returns the template, but it is
> rejected later (Post download installation was not completed) because (Format
> is invalid) due to missing virtualSize property in template.properties.
> The initial registration did succeed however. I'd either want the
> registration to fail, or it to succeed. Not first succeed (and spin VMs
> without a problem) then fail unexpectedly later.
> This is the script processing the download:
> services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java
> 759 private List<String> listTemplates(String rootdir) {
>
> 760 List<String> result = new ArrayList<String>();
>
> 761
>
> 762 Script script = new Script(listTmpltScr, s_logger);
>
> 763 script.add("-r", rootdir);
> For example this becomes:
> ==> /usr/local/cloud/systemvm/scripts/storage/secondary/listvmtmplt.sh -r
> /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0
> In this log file, it processes the output:
> less /var/log/cloud/cloud.out
> 2015-09-04 08:39:54,622 WARN [storage.template.DownloadManagerImpl]
> (agentRequest-Handler-1:null) Post download installation was not completed
> for /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607
> This error message is generated here:
> services/secondary-storage/server/src/org/apache/cloudstack/storage/template/DownloadManagerImpl.java
>
> 780 List<String> publicTmplts = listTemplates(templateDir);
>
> 781 for (String tmplt : publicTmplts) {
>
> 782 String path = tmplt.substring(0,
> tmplt.lastIndexOf(File.separator));
> 783 TemplateLocation loc = new TemplateLocation(_storage, path);
>
> 784 try {
>
> 785 if (!loc.load()) {
>
> 786 s_logger.warn("Post download installation was not
> completed for " + path);
> 787 // loc.purge();
>
> 788 _storage.cleanup(path, templateDir);
>
> 789 continue;
>
> 790 }
>
> 791 } catch (IOException e) {
>
> 792 s_logger.warn("Unable to load template location " +
> path, e);
> 793 continue;
>
> 794 }
> In the logs this message is also seen:
> MCCP-ADMIN-1|s-32436-VM CLOUDSTACK: 10:09:17,333 WARN TemplateLocation:196 -
> Format is invalid
> It is generated here:
> .//core/src/com/cloud/storage/template/TemplateLocation.java
> 192 public boolean addFormat(FormatInfo newInfo) {
>
> 193 deleteFormat(newInfo.format);
>
> 194
>
> 195 if (!checkFormatValidity(newInfo)) {
>
> 196 s_logger.warn("Format is invalid ");
>
> 197 return false;
>
> 198 }
>
> 199
>
> 200 _props.setProperty("virtualsize",
> Long.toString(newInfo.virtualSize));
> 201 _formats.add(newInfo);
>
> 202 return true;
>
> 203 }
> This returns false if checkFormatValidity is false.
> checkFormatValidity:
> 209 protected boolean checkFormatValidity(FormatInfo info) {
>
> 210 return (info.format != null && info.size > 0 && info.virtualSize
> > 0 && info.filename != null);
> 211 }
> This returns false if virtualSize is missing in template.properties. And
> indeed it is missing.
> Examples:
> Working KVM
> root@s-44134-VM:/mnt/SecStorage/884db36b-cd00-3cf0-b812-831e9e50a4b3/template/tmpl/2/2932#
> cat template.properties
> #
> #Thu Sep 03 11:41:44 UTC 2015
> filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2
> id=2932
> qcow2.size=1808203776
> public=true
> uniquename=2932-2-5c53bbe1-0c32-32a9-a82c-a5eefcb472e5
> qcow2.virtualsize=21474836480
> virtualsize=21474836480
> checksum=4c28fc94a7a9cbe858918a97f3b78294
> hvm=true
> description=Centos7-x86_64-Sbp_cis-KVM release 2015-35 build
> qcow2=true
> qcow2.filename=125672aa-62c0-30f1-ae1b-153486aa10eb.qcow2
> Working XenServer
> cat
> /mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/9/603/template.properties
> filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
> vhd=true
> id=603
> vhd.filename=16447323-ae24-4956-ac67-316eed8a2626.vhd
> public=false
> uniquename=16447323-ae24-4956-ac67-316eed8a2626
> vhd.virtualsize=107374182400
> virtualsize=107374182400
> checksum=
> hvm=true
> description=570a3650f0-c982-334a-81b1-e8cf961363e4
> vhd.size=69028311552
> size=69028311552
> Not working XenServer
> root@s-32435-VM:/mnt/SecStorage/ee8633dd-5dbd-39a3-b3ea-801ca0a20da0/template/tmpl/2/1607#
> cat template.properties
> #
> #Fri Sep 04 09:05:24 UTC 2015
> filename=39794796-3f92-3168-9c4e-a64da5ee06e8.vhd
> uniquename=1607-2-0a13984d-6724-3b9f-8bd1-2798ef859167
> size=21475270656
> checksum=6eb85d82cb18032eba0cc8fe8a84a583
> description=Centos7-x86_64-Sbp_cis-XenServer release 2015-35 build
> hvm=true
> public=true
> id=1607
> Confirmed the virtualsize is missing. But why is this successfully registered
> in the first place?
> Steps to reproduce:
> - register template
> - have it install successfully
> - check templates.properties file
> sometimes virtualsize is missing
> - if it's there, remove it
> - restart cloud service on SSVM
> - the download will start again
> in my case virtualsize is still missing
> - this becomes an endless loop.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
![https://cloud.githubusercontent.com/assets/1630096/10233628/027aa5da-6890-11e5-97fc-981033df91f5.png"](https://cloud.githubusercontent.com/assets/1630096/10233628/027aa5da-6890-11e5-97fc-981033df91f5.png"){kind=link}
![https://cloud.githubusercontent.com/assets/1630096/10233631/0bdec520-6890-11e5-995e-f193691eb1e7.png"](https://cloud.githubusercontent.com/assets/1630096/10233631/0bdec520-6890-11e5-995e-f193691eb1e7.png"){kind=link}