RosiKyu opened a new issue, #14726:
URL: https://github.com/apache/cloudstack/issues/14726
### problem
After `patchSystemVm` on the secondary storage VM reports "Successfully
patched systemVM", creating a new download link fails with "mkdir: cannot
create directory '/var/www/html/userdata/{id}/': Permission denied". A reboot
of the SSVM makes downloads work again.
### versions
**Build:** Apache CloudStack 24.0.0-SNAPSHOT (main), nightly
24.0.0-SNAPSHOT.20261118, management server and KVM agents.
- CloudStack: 24.0.0-SNAPSHOT (cloudstack-management and cloudstack-agent
24.0.0-SNAPSHOT.20261118)
- System VM: "SystemVM Template (KVM)", system VM version 24.0.0-SNAPSHOT,
one secondary storage VM
- Management server: Oracle Linux 9, MySQL 8.0.46
- Hypervisor: KVM on Oracle Linux 9, libvirt 9.0.0, qemu-kvm 7.2.0
- Storage: NFS primary storage (cluster scope), NFS secondary storage
- Network: advanced zone
### The steps to reproduce the bug
1. Make sure the SSVM is Running and its agent is Up (Infrastructure >
System VMs).
2. Patch the SSVM: Infrastructure > System VMs > SSVM S > Patch, or `cmk
patch systemvm id={ssvm id} forced=true`. The result is "Successfully patched
systemVM".
3. Download a volume that has not been downloaded before: Storage > Volumes
> volume V (of a stopped instance) > Download Volume, or `cmk extract volume
id={volume id} zoneid={zone id} mode=HTTP_DOWNLOAD`.
4. Reboot the SSVM: `cmk reboot systemvm id={ssvm id}`.
5. Download volume V again.
### Expected Result
A new download link is created right after the patch, the same as after a
reboot.
### Actual Result
Downloading a volume after the patch fails with error 530 "Unable to create
a link for entity at volumes/... on ssvm, Error in creating directory =mkdir:
cannot create directory '/var/www/html/userdata/{id}/': Permission denied".
After a reboot of the SSVM the same download creates the link and the file can
be downloaded.
### Evidence
Lab mapping: `SSVM S = s-1-VM`
(**ref-trl-12930-k-Mol9-rositsa-kyuchukova**), `volume V = ROOT-20`, `secondary
storage public IP = 10.0.60.221`.
**Management server, step 2: the patch reports success and the SSVM is Up.**
```
[root@ref-trl-12930-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# cmk patch systemvm
id=79217cb8-2146-48b6-964e-11b370fa1044 forced=true
{
"details": "Successfully patched systemVM s-1-VM ",
"success": true
}
[root@ref-trl-12930-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# cmk list systemvms
systemvmtype=secondarystoragevm filter=name,state,agentstate,version
{
"count": 1,
"systemvm": [
{
"agentstate": "Up",
"name": "s-1-VM",
"state": "Running",
"version": "24.0.0-SNAPSHOT"
}
]
}
```
**Management server, step 3: downloading volume V right after the patch
fails with Permission denied.**
```
[root@ref-trl-12930-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# cmk extract volume
id=80d39571-1693-40af-99c6-22e7b2847a15
zoneid=fe97e06a-c522-41e0-8a01-40aa677d0562 mode=HTTP_DOWNLOAD
filter=state,url,status
{
"account": "admin",
"accountid": "4d6f477b-dec8-11f1-a260-1e00580003b2",
"cmd": "org.apache.cloudstack.api.command.user.volume.ExtractVolumeCmd",
"completed": "2026-11-18T10:43:19+0000",
"created": "2026-11-18T10:42:58+0000",
"domainid": "99f0ba66-dec7-11f1-a260-1e00580003b2",
"domainpath": "ROOT",
"jobid": "6664e9c7-fe37-4930-9dfd-3dedb47e0ab0",
"jobinstanceid": "80d39571-1693-40af-99c6-22e7b2847a15",
"jobinstancetype": "Volume",
"jobprocstatus": 0,
"jobresult": {
"errorcode": 530,
"errortext": "Unable to create a link for entity at
volumes/8/18/1be61cc8-ff48-4632-b9dc-99eaaa66f2e9.qcow2 on ssvm, Error in
creating directory =mkdir: cannot create directory
‘/var/www/html/userdata/21c835f3-030e-4a29-9d16-1c228d839f2e/’: Permission
denied"
},
"jobresultcode": 530,
"jobresulttype": "object",
"jobstatus": 2,
"userid": "4d710ca5-dec8-11f1-a260-1e00580003b2"
}
🙈 Error: async API failed for job 6664e9c7-fe37-4930-9dfd-3dedb47e0ab0
```
**Management server, steps 4 and 5: after a reboot of the SSVM the same
download works and the file is served.**
```
[root@ref-trl-12930-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# cmk reboot systemvm
id=79217cb8-2146-48b6-964e-11b370fa1044 filter=name,state
{
"systemvm": {
"name": "s-1-VM",
"state": "Running"
}
}
[root@ref-trl-12930-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# cmk extract volume
id=80d39571-1693-40af-99c6-22e7b2847a15
zoneid=fe97e06a-c522-41e0-8a01-40aa677d0562 mode=HTTP_DOWNLOAD
filter=state,url,status | tee /root/ssvm-t4.json
{
"volume": {
"state": "DOWNLOAD_URL_CREATED",
"url":
"http://10.0.60.221/userdata/189b7b21-1530-47ae-8b29-6598e6464a08/ROOT-20.qcow2"
}
}
[root@ref-trl-12930-k-Mol9-rositsa-kyuchukova-mgmt1 ~]# curl -s -o /dev/null
-w '%{http_code} %{size_download}\n' --max-time 30 -r 0-1048575
'http://10.0.60.221/userdata/189b7b21-1530-47ae-8b29-6598e6464a08/ROOT-20.qcow2'
206 1048576
```
### What to do about it?
Downloads (volumes, templates, ISOs, backup files) should keep working right
after a successful patch of the SSVM, without a reboot. Until this is fixed,
rebooting the SSVM after a patch makes downloads work again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]