fermosan opened a new issue, #14366: URL: https://github.com/apache/cloudstack/issues/14366
### The required feature described as a wish ## Summary KVM-hosted VM disks in CloudStack have no way to configure libvirt/QEMU's `error_policy` (`rerror`/`werror` in domain XML). Every disk defaults to QEMU's `report` policy, which means any I/O error from the backing storage — even a brief, otherwise-recoverable one — is passed straight through to the guest as a real disk error. This proposes adding first-class `error_policy` support to disk/compute offerings, modeled on the existing `cacheMode` offering attribute. ## Problem / Motivation Tested against a KVM cluster backed by NFS primary storage during a *planned, successful* storage-side HA failover (floating IP moved between NFS nodes in ~6s, well within normal NFS client timeouts): a VM with a continuous write workload hit a hard `blk_update_request: I/O error` on its data disk within **3-4 seconds** of the failover starting — long before any NFS-level recovery (RPC retry, lock reclaim, grace period) had a chance to run. The guest's ext4 filesystem aborted its journal and remounted read-only (`JBD2: Error -5`, `EXT4-fs: Remounting filesystem read-only`). Root cause: the disk's libvirt XML is `<driver name='qemu' type='qcow2' cache='none'/>` with no `rerror`/`werror` attribute, so QEMU's default `report` error policy applies — any hiccup on the backing file (NFS, iSCSI, etc.) is surfaced to the guest immediately and unconditionally. This is independent of how well the storage/HA layer itself is tuned: the guest reacts and can corrupt its own filesystem state before storage-level recovery even starts. We confirmed (manually patching a live domain's persisted XML with `error_policy='stop'` and re-running the same failover) that QEMU's `stop` policy — which pauses the VM on I/O error and auto-resumes once the backing storage responds, instead of surfacing the error to the guest — eliminates the problem entirely: identical failover, zero guest-visible impact, domain never even needed to pause. This affects any KVM deployment where primary storage can have a brief interruption that the storage layer itself would otherwise recover from transparently: HA/failover storage (our case), network blips to NFS/iSCSI targets, storage maintenance, etc. Today the only way to get `error_policy='stop'` is manually patching each VM's persisted domain XML outside of CloudStack's orchestration — which gets silently overwritten the next time CloudStack redefines the domain. ## Current state (verified against `main`) `LibvirtVMDef.DiskDef` (`plugins/hypervisors/kvm/.../LibvirtVMDef.java`) has no `error_policy`/`rerror`/`werror` field at all today — only `cache`, `discard`, `io`, `iothread`, block-size, etc. are supported on the `<driver>` element. ## Proposed solution Add `error_policy` as a first-class disk/compute offering attribute, following the exact existing plumbing for `cacheMode` (`DiskOffering.DiskCacheMode`) end to end: 1. **Schema**: new `disk_offering.error_policy` column (`VARCHAR(16)`, default `'report'`), added to `disk_offering_view`, `service_offering_view`, `volume_view` — same pattern as the `cache_mode` column. 2. **Domain model**: `DiskOffering.DiskErrorPolicy` enum (`REPORT`, `STOP`, `IGNORE`, `ENOSPACE` — QEMU's valid `error_policy` values), `getErrorPolicy()`/`setErrorPolicy()` on `DiskOffering`/`DiskOfferingVO`. 3. **API**: new `errorpolicy` param on `createDiskOffering`/`updateDiskOffering` (and compute-offering equivalents), surfaced on `DiskOfferingResponse`/`ServiceOfferingResponse`, validated in `ConfigurationManagerImpl` the same way `cacheMode` is today. 4. **Orchestration**: carry the value `DiskOfferingVO` → `VolumeObject`/`VolumeObjectTO` → hypervisor command, mirroring `getCacheMode()`'s path through `VolumeObjectTO`. 5. **KVM agent**: `LibvirtVMDef.DiskDef` gets an `_errorPolicy` field + setter, rendered as `error_policy='<value>' ` on the `<driver>` element in `toString()`, set from the volume's offering in `LibvirtComputingResource`/`KVMStorageProcessor` next to the existing `setCacheMode(...)` calls. 6. **Other hypervisors**: this is a libvirt/QEMU-specific concept; XenServer/VMware plugins would simply ignore the new offering field, consistent with other KVM-only offering attributes. ## Open questions for discussion - Should this live on **disk offering** only, or also on **compute offering** (root disk)? `cacheMode` supports both — proposing the same here. - Default value: `report` (current implicit behavior, fully backward compatible) vs. defaulting new offerings to `stop` for safety? Leaning toward `report` as default to avoid surprising existing deployments with VMs that can now pause. - Operational concern: a VM with `stop` policy can sit **paused indefinitely** if the underlying storage problem never clears. Should this feature ship with (or require) companion alerting on paused-domain state, or is that left to operators? Happy to put together a PR for this if the general direction sounds right — wanted to float the approach first since it touches the offering schema/API surface. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
