WanzenBug opened a new issue, #10658: URL: https://github.com/apache/cloudstack/issues/10658
### problem The secondary storage VM fails to download ISOs for servers that do not report a `Content-Length`, but rather use "chunked transfers". An example for such an ISO would be https://releases.rancher.com/harvester/v1.5.0-rc3/harvester-v1.5.0-rc3-amd64.iso. Logs from the secondary storage VM: ``` Apr 03 12:51:06 s-2-VM _run.sh[1965]: 12:51:06,684 INFO NfsSecondaryStorageResource:3154 - Determined host 10.43.240.100 corresponds to IP 10.43.240.100 Apr 03 12:51:06 s-2-VM _run.sh[1965]: 12:51:06,745 INFO HttpTemplateDownloader:145 - No credentials configured for host=releases.rancher.com:443 Apr 03 12:51:06 s-2-VM _run.sh[1965]: 12:51:06,978 INFO HttpTemplateDownloader:211 - Starting download from https://releases.rancher.com/harvester/v1.5.0-rc3/harvester-v1.5.0-rc3-amd64.iso to /mnt/SecStorage/ede8774c-67e5-3bec-b8eb-c16854008752/template/tmpl/6/270/dnld18314603760407157687tmp_ remoteSize=(50.00 GB) 53687091200 , max size=(50.00 GB) 53687091200 Apr 03 12:51:28 s-2-VM _run.sh[1965]: 12:51:28,275 INFO NfsSecondaryStorageResource:3154 - Determined host 10.43.240.100 corresponds to IP 10.43.240.100 Apr 03 12:52:28 s-2-VM _run.sh[1965]: 12:52:28,451 INFO NfsSecondaryStorageResource:3154 - Determined host 10.43.240.100 corresponds to IP 10.43.240.100 Apr 03 12:53:28 s-2-VM _run.sh[1965]: 12:53:28,611 INFO NfsSecondaryStorageResource:3154 - Determined host 10.43.240.100 corresponds to IP 10.43.240.100 Apr 03 12:54:28 s-2-VM _run.sh[1965]: 12:54:28,783 INFO NfsSecondaryStorageResource:3154 - Determined host 10.43.240.100 corresponds to IP 10.43.240.100 Apr 03 12:54:34 s-2-VM _run.sh[1965]: 12:54:34,447 INFO DownloadManagerImpl:292 - Download Completion for jobId: 73770031-0cef-4d79-b00d-82ec7e97afaf, status=IN_PROGRESS Apr 03 12:54:34 s-2-VM _run.sh[1965]: 12:54:34,450 INFO DownloadManagerImpl:293 - local: /mnt/SecStorage/ede8774c-67e5-3bec-b8eb-c16854008752/template/tmpl/6/270/dnld18314603760407157687tmp_, bytes=(6.65 GB) 7142113280, error=Downloaded (6.65 GB) 7142113280 bytes (incomplete download), pct=13 Apr 03 12:54:34 s-2-VM _run.sh[1965]: 12:54:34,450 INFO DownloadManagerImpl:306 - Resuming jobId: 73770031-0cef-4d79-b00d-82ec7e97afaf, status=IN_PROGRESS Apr 03 12:54:34 s-2-VM _run.sh[1965]: 12:54:34,455 INFO HttpTemplateDownloader:358 - Resuming download to file (current size)=(6.65 GB) 7142113280 Apr 03 12:55:01 s-2-VM CRON[265722]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0) Apr 03 12:55:01 s-2-VM CRON[265723]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) Apr 03 12:55:01 s-2-VM CRON[265722]: pam_unix(cron:session): session closed for user root Apr 03 12:55:28 s-2-VM _run.sh[1965]: 12:55:28,914 INFO NfsSecondaryStorageResource:3154 - Determined host 10.43.240.100 corresponds to IP 10.43.240.100 Apr 03 12:56:29 s-2-VM _run.sh[1965]: 12:56:29,026 INFO NfsSecondaryStorageResource:3154 - Determined host 10.43.240.100 corresponds to IP 10.43.240.100 Apr 03 12:57:07 s-2-VM _run.sh[1965]: 12:57:07,422 INFO DownloadManagerImpl:292 - Download Completion for jobId: 73770031-0cef-4d79-b00d-82ec7e97afaf, status=UNRECOVERABLE_ERROR Apr 03 12:57:07 s-2-VM _run.sh[1965]: 12:57:07,424 INFO DownloadManagerImpl:293 - local: /mnt/SecStorage/ede8774c-67e5-3bec-b8eb-c16854008752/template/tmpl/6/270/dnld18314603760407157687tmp_, bytes=(6.65 GB) 7142113280, error=HTTP Server does not support partial get, pct=13 ``` Note that `7142113280` is the expected number of bytes, so the download is actually complete. ### versions CloudStack 4.19.1.3 ### The steps to reproduce the bug 1. Try to register an ISO from a source not sending a `Content-Length` header and and not supporting partial transfers, such as https://releases.rancher.com/harvester/v1.5.0-rc3/harvester-v1.5.0-rc3-amd64.iso 2. See the download fail ### What to do about it? It looks like the HTTP handling in `HttpTemplateDownloader` is not equipped to deal with chunked transfers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
