RosiKyu commented on issue #12415:
URL: https://github.com/apache/cloudstack/issues/12415#issuecomment-3745478071
## Additional Testing/Details
### 1. Fresh install has `value = NULL`
```sql
mysql> SELECT name, value, default_value FROM configuration WHERE
name='secstorage.nfs.version';
+------------------------+-------+---------------+
| name | value | default_value |
+------------------------+-------+---------------+
| secstorage.nfs.version | NULL | NULL |
+------------------------+-------+---------------+
1 row in set (0.00 sec)
```
### 2. Configuration description states NULL should trigger default NFS
negotiation
```sql
mysql> SELECT name, value, default_value, description FROM configuration
WHERE name='secstorage.nfs.version';
+------------------------+-------+---------------+-------------------------------------------------------------------------------------------------------+
| name | value | default_value | description
|
+------------------------+-------+---------------+-------------------------------------------------------------------------------------------------------+
| secstorage.nfs.version | 4.2 | NULL | Enforces specific NFS
version when mounting Secondary Storage. If NULL default selection is performed
|
+------------------------+-------+---------------+-------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
```
The description says: *"If NULL default selection is performed"* - implying
the OS/NFS should auto-negotiate the version (i.e., omit the `vers=` option
entirely). But instead, CloudStack appends `vers=null` as a literal string.
### 3. No option to set NFS version when adding secondary storage
The `addImageStore` API does not have an NFS version parameter:
```
(localcloud) 🐱 > add imagestore
details= exclude= filter= name= provider= url= zoneid=
```
The UI "Add Secondary Storage" dialog also has no NFS version field - only:
- Name
- Provider
- Zone
- Server
- Path
### 4. SSVM generates mount command with `vers=null` (string literal)
From SSVM `/var/log/cloud.log`:
```
2026-01-13T07:34:23,526 WARN [storage.resource.NfsSecondaryStorageResource]
(AgentRequest-Handler-8:[]) Execution of process [10617] for command [mount -t
nfs -o soft,timeo=133,retrans=2147483647,tcp,acdirmax=0,acdirmin=0,vers=null
10.0.32.4:/acs/secondary/ref-trl-10584-k-Mol9-rositsa-kyuchukova/ref-trl-10584-k-Mol9-rositsa-kyuchukova-sec3
/mnt/SecStorage/6ff0eb2d-ffed-31c8-8f8c-e2c5c1fb2db5 ] failed.
2026-01-13T07:34:23,528 WARN [storage.resource.NfsSecondaryStorageResource]
(AgentRequest-Handler-8:[]) Process [10617] for command [mount -t nfs -o
soft,timeo=133,retrans=2147483647,tcp,acdirmax=0,acdirmin=0,vers=null
10.0.32.4:/acs/secondary/ref-trl-10584-k-Mol9-rositsa-kyuchukova/ref-trl-10584-k-Mol9-rositsa-kyuchukova-sec3
/mnt/SecStorage/6ff0eb2d-ffed-31c8-8f8c-e2c5c1fb2db5 ] encountered the error:
[mount.nfs: parsing error on 'vers=' option].
2026-01-13T07:34:23,542 ERROR [storage.resource.NfsSecondaryStorageResource]
(AgentRequest-Handler-8:[]) Unable to mount
10.0.32.4:/acs/secondary/ref-trl-10584-k-Mol9-rositsa-kyuchukova/ref-trl-10584-k-Mol9-rositsa-kyuchukova-sec3
at /mnt/SecStorage/6ff0eb2d-ffed-31c8-8f8c-e2c5c1fb2db5 due to mount.nfs:
parsing error on 'vers=' option
2026-01-13T07:34:23,582 ERROR [storage.resource.NfsSecondaryStorageResource]
(AgentRequest-Handler-8:[]) GetRootDir for
nfs://10.0.32.4/acs/secondary/ref-trl-10584-k-Mol9-rositsa-kyuchukova/ref-trl-10584-k-Mol9-rositsa-kyuchukova-sec3
failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to mount
10.0.32.4:/acs/secondary/ref-trl-10584-k-Mol9-rositsa-kyuchukova/ref-trl-10584-k-Mol9-rositsa-kyuchukova-sec3
at /mnt/SecStorage/6ff0eb2d-ffed-31c8-8f8c-e2c5c1fb2db5 due to mount.nfs:
parsing error on 'vers=' option
2026-01-13T07:34:23,605 ERROR [storage.resource.NfsSecondaryStorageResource]
(AgentRequest-Handler-8:[]) failed to copy
filetemplate/tmpl/2/202/3a2e733b-94e9-3864-8064-f7c0ac8fabca.qcow2
com.cloud.utils.exception.CloudRuntimeException: GetRootDir for
nfs://10.0.32.4/acs/secondary/ref-trl-10584-k-Mol9-rositsa-kyuchukova/ref-trl-10584-k-Mol9-rositsa-kyuchukova-sec3
failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to mount
10.0.32.4:/acs/secondary/ref-trl-10584-k-Mol9-rositsa-kyuchukova/ref-trl-10584-k-Mol9-rositsa-kyuchukova-sec3
at /mnt/SecStorage/6ff0eb2d-ffed-31c8-8f8c-e2c5c1fb2db5 due to mount.nfs:
parsing error on 'vers=' option
```
### 5. After manually setting `secstorage.nfs.version=4.2`, mount works
From Management Server
`/var/log/cloudstack/management/management-server.log`:
```
2026-01-13 09:11:05,683 DEBUG [o.a.c.s.NfsMountManagerImpl]
(BackgroundTaskPollManager-5:[ctx-2dae6be3]) (logid:4d52290c) Executing command
[sudo mount -t nfs -o vers=4.2
10.0.32.4:/acs/secondary/ref-trl-10584-k-Mol9-rositsa-kyuchukova/ref-trl-10584-k-Mol9-rositsa-kyuchukova-sec1
/var/cloudstack/mnt/32986271580862.6f4a3271 ].
2026-01-13 09:11:05,846 DEBUG [o.a.c.s.NfsMountManagerImpl]
(BackgroundTaskPollManager-5:[ctx-2dae6be3]) (logid:4d52290c) Successfully
executed process [90191] for command [sudo mount -t nfs -o vers=4.2
10.0.32.4:/acs/secondary/ref-trl-10584-k-Mol9-rositsa-kyuchukova/ref-trl-10584-k-Mol9-rositsa-kyuchukova-sec1
/var/cloudstack/mnt/32986271580862.6f4a3271 ].
```
### Root Cause
The code is converting Java `null` to the string `"null"` instead of
omitting the `vers=` option entirely when `secstorage.nfs.version` is NULL.
### Why this is a bug (not just a config issue)
1. **Fresh installs are affected** - `default_value` is NULL, so every new
CloudStack deployment has this problem
2. **Description is misleading** - says "If NULL default selection is
performed" but actually appends `vers=null` which breaks the mount
3. **No way to set version per-storage** - API and UI don't have NFS version
parameter when adding secondary storage
4. **Workaround must be applied BEFORE adding storage** - if you hit the bug
first, the storage is in a broken state
### Workaround
Set the NFS version globally BEFORE adding any secondary storage:
```bash
cmk update configuration name=secstorage.nfs.version value=4.2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
