I added two new new disks: sdb1 in 192.168.0.12 and sdb1 in 192.168.0.13. (both of these storage nodes already had sda3 which was in zone 2 and 3 respectively). I added a new zone and these two new disks in that zone. Then did rebalance and distributed the ring files all across the cluster. Did I miss any other step?
I started noticing these errors (there are some successful rsync as well). > Nov 11 12:12:48 b001 object-replicator Successful rsync of > /srv/node/sda3/objects/19498/86d at [192.168.0.12]::object/sdb1/objects/19498 > (0.209) > Nov 11 12:12:48 b001 object-replicator Bad rsync return code: ['rsync', > '--recursive', '--whole-file', '--human-readable', '--xattrs', > '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', > '/srv/node/sda3/objects/51119/350', > '[192.168.0.12]::object/sdb1/objects/51119'] -> 5 > Nov 11 12:12:48 b001 object-replicator Bad rsync return code: ['rsync', > '--recursive', '--whole-file', '--human-readable', '--xattrs', > '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', > '/srv/node/sda3/objects/32937/efe', > '[192.168.0.12]::object/sdb1/objects/32937'] -> 5 Same with .13: > Nov 11 12:13:43 b001 object-replicator Successful rsync of > /srv/node/sda3/objects/11047/10d at [192.168.0.13]::object/sdb1/objects/11047 > (0.168) > Nov 11 12:13:43 b001 object-replicator Bad rsync return code: ['rsync', > '--recursive', '--whole-file', '--human-readable', '--xattrs', > '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', > '/srv/node/sda3/objects/92247/966', > '[192.168.0.13]::object/sdb1/objects/92247'] -> 5 > Nov 11 12:13:43 b001 object-replicator Bad rsync return code: ['rsync', > '--recursive', '--whole-file', '--human-readable', '--xattrs', > '--itemize-changes', '--ignore-existing', '--timeout=30', '--contimeout=30', > '/srv/node/sda3/objects/50409/be4', > '[192.168.0.13]::object/sdb1/objects/50409'] -> 5 any suggestions? I also noticed this in .12: Nov 11 12:16:16 b002 object-replicator @ERROR: max connections (2) reached -- try again later --sharif -- Sharif Islam Senior Systems Analyst/Programmer FutureGrid (http://futuregrid.org) Pervasive Technology Institute, Indiana University Bloomington _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp