That was it! Sorry the 10.20.4.x NICs weren't configured correctly on those two nodes. I'll admit this one was definitely my mistake.
Thanks for pointing it out. Tom 2013/7/9 Gregory Farnum <[email protected]> > On Tue, Jul 9, 2013 at 3:08 AM, Tom Verdaat <[email protected]> wrote: > > Hi all, > > > > I've set up a new Ceph cluster for testing and it doesn't seem to be > working > > out-of-the-box. If I check the status it tells me that of the 3 defined > > OSD's, only 1 is in: > > > >> health HEALTH_WARN 392 pgs degraded; 392 pgs stuck unclean > >> monmap e1: 3 mons at > >> {controller-01= > 10.20.3.110:6789/0,controller-02=10.20.3.111:6789/0,controller-03=10.20.3.112:6789/0 > }, > >> election epoch 6, quorum 0,1,2 controller-01,controller-02,controller-03 > >> osdmap e20: 3 osds: 1 up, 1 in > >> pgmap v35: 392 pgs: 392 active+degraded; 0 bytes data, 37444 KB > used, > >> 15312 MB / 15348 MB avail > >> mdsmap e1: 0/0/1 up > > > > > > Turns out this is true because if I run "service ceph restart" on my OSD > > nodes, osd.0 will restart just fine but osd.1 and osd.2 give me the > > follwoing error: > > > >> Starting Ceph osd.0 on storage-02... > >> > >> starting osd.1 at :/0 osd_data /var/lib/ceph/osd/ceph-1 > >> /var/lib/ceph/osd/ceph-1/journal > >> 2013-07-09 11:54:26.497639 7f5b18813780 -1 accepter.accepter.bind unable > >> to bind to 10.20.4.121:7100 on any port in range 6800-7100: Cannot > assign > >> requested address > >> failed: 'ulimit -n 8192; /usr/bin/ceph-osd -i 1 --pid-file > >> /var/run/ceph/osd.1.pid -c /etc/ceph/ceph.conf ' > > > > > > My ceph.conf just has a very limited configuration. The OSD section > > basically contains: > > > >> [osd] > >> public network=10.20.3.0/24 > >> cluster network=10.20.4.0/24 > >> [osd.0] > >> host = storage-01 > >> public addr = 10.20.3.120 > >> cluster addr = 10.20.4.120 > >> [osd.1] > >> host = storage-02 > >> public addr = 10.20.3.121 > >> cluster addr = 10.20.4.121 > >> [osd.2] > >> host = storage-03 > >> public addr = 10.20.3.122 > >> cluster addr = 10.20.4.122 > > > > > > A quick Google search on that port binding error doesn't really yield and > > results so I'm reaching out to you guys. Any thoughts on how to fix this? > > Do the nodes in question actually have those IPs assigned? I wonder if > maybe the NIC isn't set up right or something. > -Greg > Software Engineer #42 @ http://inktank.com | http://ceph.com >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
