[..] > Seeing this in the firefly cluster as well. Tried a couple of rados > commands on the .rgw.root pool this is what is happening: > > abhi@st:~$ sudo rados -p .rgw.root put test.txt test.txt > error putting .rgw.root/test.txt: (6) No such device or address > > abhi@st:~$ sudo ceph osd map .rgw.root test.txt > osdmap e83 pool '.rgw.root' (6) object 'test.txt' -> pg 6.8b0b6108 > (6.0) -> up ([1,2,0], p1) acting ([1,2,0], p1) > > abhi@st:~$ sudo ceph pg map 6.8b0b6108 > osdmap e83 pg 6.8b0b6108 (6.0) -> up [0,2,1] acting [0,2,1] > > Looks like the osd map says the object must go to primary osd as 1, > whereas pg map says that the pg is hosted with 0 as primary. > [..]
Solved the problem; just posting it here in case anyone comes across this same error. Primarily the issue was due a misconfiguration from our config management system, where `osd pool default pgp num` got set in ceph.conf and `pg num` didn't, which led to the rgw pools having the default pg num (8) and pgp_num set to a value of 128. Though trying out commands like `ceph osd pool create` will fail without specifying the pg count; `rados mkpool` does allow pool creation without the specification of pg count and falling back to the default values; which probably explains what happened to the .rgw.default pool. An easy way to simulate this error would be to just do a setting like `ceph tell mon.0 injectargs --osd_pool_default_pgp_num 128` and then starting a fresh radosgw (assuming its not installed previously); or creating any pool with rados commands, which will fail when putting different objects because of the increased pgp count compared to the pg count. -- Abhishek
signature.asc
Description: PGP signature
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
