Re: [ceph-users] Hanging VMs with Qemu + RBD

2015-01-07 Thread Nico Schottelius
Hello Achim, good to hear someone else running this setup. We have changed the number of backfills using ceph tell osd.\* injectargs '--osd-max-backfills 1' and it seems to work mostly in regards of issues when rebalancing. One unsolved problem we have is machines kernel panic'ing, when

Re: [ceph-users] Hanging VMs with Qemu + RBD

2015-01-07 Thread Achim Ledermüller
Hi, We have the same setup including OpenNebula 4.10.1. We had some backfilling due to node failures and node expansion. If we throttle osd_max_backfills there is not a problem at all. If the value for backfilling jobs is too high, we can see delayed reactions within the shell, eg. `ls -lh` needs

[ceph-users] Hanging VMs with Qemu + RBD

2014-12-19 Thread Nico Schottelius
Hello, another issue we have experienced with qemu VMs (qemu 2.0.0) with ceph-0.80 on Ubuntu 14.04 managed by opennebula 4.10.1: The VMs are completly frozen when rebalancing takes place, they do not even respond to ping anymore. Looking at the qemu processes they are in state Sl. Is this a

Re: [ceph-users] Hanging VMs with Qemu + RBD

2014-12-19 Thread Robert LeBlanc
I think smaller clusters get chocked up with the default backfill. I've seen latency on a four node cluster with 10 OSD each improve by setting osd_max_backfills to 2. I would try lowering it and see if it helps. Also, if you are running both cluster and VM traffic on the same network, you could