On Tue, Sep 26, 2017 at 04:31:22PM +0200, Nicolas Ecarnot wrote: > Le 21/09/2017 à 16:31, Stefan Hajnoczi a écrit : > > On Tue, Sep 19, 2017 at 12:09:06PM +0200, Nicolas Ecarnot wrote: > > > Hello, > > > > > > First post here, so maybe I should introduce myself : > > > - I'm a sysadmin for decades and currently managing 4 oVirt clusters, made > > > out of tens of hypervisors, all are CentOS 7.2+ based. > > > - I'm very happy with this solution we choose especially because it is > > > based > > > on qemu-kvm (open source, reliable, documented). > > > > > > On one VM, we experienced the following : > > > - oVirt/vdsm is detecting an issue on the image > > > - following this hints https://access.redhat.com/solutions/1173623, I > > > managed to detect one error and fix it > > > - the VM is now running perfectly > > > > > > On two other VMs, we experienced a similar situation, except the check > > > stage > > > is showing something like 14000+ errors, and the relevant logs are : > > > > > > Repairing refcount block 14 is outside image > > > ERROR could not resize image: Invalid argument > > > ERROR cluster 425984 refcount=0 reference=1 > > > ERROR cluster 425985 refcount=0 reference=1 > > > [... repeating the previous line 7000+ times...] > > > ERROR cluster 457166 refcount=0 reference=1 > > > Rebuilding refcount structure > > > ERROR writing refblock: No space left on device > > > qemu-img: Check failed: No space left on device > > > > Please run strace qemu-img info /the/relevant/logical/volume/path. It
Sorry, "qemu-img info" should be your "qemu-img check" command. > > will print all the syscalls that qemu-img makes. That way we'll be able > > to verify that the ENOSPC error is coming from a pwritev syscall. > I did but I'm not skilled enough to ensure where the ENOSPC error is coming > from. > > Is your question meaning the reads and/or the writes may come from or go to > places outside the expected boundaries? I was interested in the syscall (probably pwritev or similar) related to the following output from qemu-img check: ERROR writing refblock: No space left on device Feel free to post your strace log so we can analyze it. > > > You surely know that oVirt/RHEV is storing its qcow2 images in dedicated > > > logical volumes. > > > > > > pvs/vgs/lvs are all showing there is plenty of space available, so I > > > understand that I don't understand what "No space left on device" means. > > > > After you have the strace data you can look at the file offset from the > > failing pwritev syscall and check that it's really within the LV. > > > > I think there is no fancy thin provisioning going on at the LVM level > > with oVirt, but if there is then perhaps a write within the LV could > > still result in an ENOSPC error. It would be worth confirming that > > these are class "thick" LVs. > > I think there is no such thin prov. at the LVM level, but I wouldn't swear. > Don't you mind if I forward your question to the oVirt mailing-list? Sure, feel free to CC other mailing lists. I have added oVirt devel. Stefan
