Re: [ceph-users] Incorrect output from ceph osd map command

2016-02-23 Thread Robert LeBlanc 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

ceph pg dump

Since all objects map to a PG, as long as you can verify that no PG is
on the same host/chassis/rack, you are good.
-BEGIN PGP SIGNATURE-
Version: Mailvelope v1.3.5
Comment: https://www.mailvelope.com

wsFcBAEBCAAQBQJWzS0NCRDmVDuy+mK58QAANfQP/19WHCUa2wPK6cHwx6zC
msfy+zipJ86qvqTgAh5azy0VRIk5lo1GknwMJhulox5vk5M+GQo0ermR/yfw
MbXKXy1f81NeZgQSqDX+GD3V19c/mb1WYuA0SLatPKkvv6L5BxPzHoGm6HYE
1hr3VSMYixCE2JZubQxj8EA+RnrJXYPue+e9aRXGbFymXIGHNdW5A3wU/vlp
IJ18E3vTIrAdmpyKlLFYhI6w2sMPUSwGllqfBpuo+OxVE+9Wa+AptZIClNXB
CI2Ozs02V9aRwUiCf6qPIBUAIPUE6/uDqzcS3mId8KUs4IxGi0pCr/t2irr5
jdc3u4WLtmZISo7RC/yyftvFFWvUkH0+2tr3lLQXHaDc+RaJPdlj5v5tylJp
j5HTywmzz/vIPKFnn9OmVimMHfFJyWinShixVWI4ORKnPFD0gT0Qlg0yC2Hx
PmtFE/OxUvYYM65WKONhAUTrjOlLAjbibFHDwhuXfQ/1Pxuh28YWkAyX/wdE
cFZxoq6E6DePuKNO3xw1EqBUVsncW3+PltN7b+CWVOawEp+me42Ovetq7OqU
B8aQhqQB0/T8bRYeIzINkkB60k6gSvrF5TO2Kq+x7UiYUQ82KyHE+zlTryXW
0BEj2bK9s4NtAItkx3F7bcmnusOOlb1AMMJFssMQV/LmjDOR9xJUYiuqXxrb
6AB3
=hv6I
-END PGP SIGNATURE-

Robert LeBlanc
PGP Fingerprint 79A2 9CA4 6CC4 45DD A904  C70E E654 3BB2 FA62 B9F1


On Tue, Feb 23, 2016 at 3:33 PM, Vickey Singh
 wrote:
> Adding community for further help on this.
>
> On Tue, Feb 23, 2016 at 10:57 PM, Vickey Singh 
> wrote:
>>
>>
>>
>> On Tue, Feb 23, 2016 at 9:53 PM, Gregory Farnum 
>> wrote:
>>>
>>>
>>>
>>> On Tuesday, February 23, 2016, Vickey Singh 
>>> wrote:

 Thanks Greg,

 Do you mean ceph osd map command is not displaying accurate information
 ?

 I guess, either of these things are happening with my cluster
 - ceph osd map is not printing true information
 - Object to PG mapping is not correct ( one object is mapped to multiple
 PG's )

 This is happening for several objects , but the cluster is Healthy.
>>>
>>>
>>> No, you're looking for the map command to do something it was not
>>> designed for. If you want to see if an object exists, you will need to use a
>>> RADOS client to fetch the object and see if it's there. "map" is a mapping
>>> command: given an object name, which PG/OSD does CRUSH map that name to?
>>
>>
>> well your 6th sense is amazing :)
>>
>> This is exactly i want to achieve , i wan to see my PG/OSD mapping for
>> objects. ( basically i have changed my crush hierarchy , now i want to
>> verify that no 2 objects should go to a single host / chassis / rack ) so to
>> verify them i was using ceph osd map command.
>>
>> Is there a smarter way to achieve this ?
>>
>>
>>
>>
>>>
>>>


 Need expert suggestion.


 On Tue, Feb 23, 2016 at 7:20 PM, Gregory Farnum 
 wrote:
>
> This is not a bug. The map command just says which PG/OSD an object
> maps to; it does not go out and query the osd to see if there actually is
> such an object.
> -Greg
>
>
> On Tuesday, February 23, 2016, Vickey Singh
>  wrote:
>>
>> Hello Guys
>>
>> I am getting wired output from osd map. The object does not exists on
>> pool but osd map still shows its PG and OSD on which its stored.
>>
>> So i have rbd device coming from pool 'gold' , this image has an
>> object 'rb.0.10f61.238e1f29.2ac5'
>>
>> The below commands verifies this
>>
>> [root@ceph-node1 ~]# rados -p gold ls | grep -i
>> rb.0.10f61.238e1f29.2ac5
>> rb.0.10f61.238e1f29.2ac5
>> [root@ceph-node1 ~]#
>>
>> This object lives on pool gold and OSD 38,0,20 , which is correct
>>
>> [root@ceph-node1 ~]# ceph osd map gold
>> rb.0.10f61.238e1f29.2ac5
>> osdmap e1357 pool 'gold' (1) object 'rb.0.10f61.238e1f29.2ac5'
>> -> pg 1.11692600 (1.0) -> up ([38,0,20], p38) acting ([38,0,20], p38)
>> [root@ceph-node1 ~]#
>>
>>
>> Since i don't have object 'rb.0.10f61.238e1f29.2ac5' in data
>> and rbd pools , rados ls will not list it. Which is expected.
>>
>> [root@ceph-node1 ~]# rados -p data ls | grep -i
>> rb.0.10f61.238e1f29.2ac5
>> [root@ceph-node1 ~]# rados -p rbd ls | grep -i
>> rb.0.10f61.238e1f29.2ac5
>>
>>
>> But , how come the object is showing in osd map of pool data and rbd.
>>
>> [root@ceph-node1 ~]# ceph osd map data
>> rb.0.10f61.238e1f29.2ac5
>> osdmap e1357 pool 'data' (2) object 'rb.0.10f61.238e1f29.2ac5'
>> -> pg 2.11692600 (2.0) -> up ([3,51,29], p3) acting ([3,51,29], p3)
>> [root@ceph-node1 ~]#
>>
>> [root@ceph-node1 ~]# ceph osd map rbd rb.0.10f61.238e1f29.2ac5
>> osdmap e1357 pool 'rbd' (0) object 'rb.0.10f61.238e1f29.2ac5'
>> -> pg 0.11692600 (0.0) -> up ([41,20,3], p41) acting ([41,20,3], p41)
>> [root@ceph-node1 ~]#
>>
>>
>> In ceph,

Re: [ceph-users] Incorrect output from ceph osd map command

2016-02-23 Thread Vickey Singh 
Adding community for further help on this.

On Tue, Feb 23, 2016 at 10:57 PM, Vickey Singh 
wrote:

>
>
> On Tue, Feb 23, 2016 at 9:53 PM, Gregory Farnum 
> wrote:
>
>>
>>
>> On Tuesday, February 23, 2016, Vickey Singh 
>> wrote:
>>
>>> Thanks Greg,
>>>
>>> Do you mean ceph osd map command is not displaying accurate information ?
>>>
>>> I guess, either of these things are happening with my cluster
>>> - ceph osd map is not printing true information
>>> - Object to PG mapping is not correct ( one object is mapped to multiple
>>> PG's )
>>>
>>> This is happening for several objects , but the cluster is Healthy.
>>>
>>
>> No, you're looking for the map command to do something it was not
>> designed for. If you want to see if an object exists, you will need to use
>> a RADOS client to fetch the object and see if it's there. "map" is a
>> mapping command: given an object name, which PG/OSD does CRUSH map that
>> name to?
>>
>
> well your 6th sense is amazing :)
>
> This is exactly i want to achieve , i wan to see my PG/OSD mapping for
> objects. ( basically i have changed my crush hierarchy , now i want to
> verify that no 2 objects should go to a single host / chassis / rack ) so
> to verify them i was using ceph osd map command.
>
> Is there a smarter way to achieve this ?
>
>
>
>
>
>>
>>
>>>
>>> Need expert suggestion.
>>>
>>>
>>> On Tue, Feb 23, 2016 at 7:20 PM, Gregory Farnum 
>>> wrote:
>>>
 This is not a bug. The map command just says which PG/OSD an object
 maps to; it does not go out and query the osd to see if there actually is
 such an object.
 -Greg


 On Tuesday, February 23, 2016, Vickey Singh <
 vickey.singh22...@gmail.com> wrote:

> Hello Guys
>
> I am getting wired output from osd map. The object does not exists on
> pool but osd map still shows its PG and OSD on which its stored.
>
> So i have rbd device coming from pool 'gold' , this image has an
> object 'rb.0.10f61.238e1f29.2ac5'
>
> The below commands verifies this
>
> *[root@ceph-node1 ~]# rados -p gold ls | grep -i
> rb.0.10f61.238e1f29.2ac5*
> *rb.0.10f61.238e1f29.2ac5*
> *[root@ceph-node1 ~]#*
>
> This object lives on pool gold and OSD 38,0,20 , which is correct
>
> *[root@ceph-node1 ~]# ceph osd map gold
> rb.0.10f61.238e1f29.2ac5*
> *osdmap e1357 pool 'gold' (1) object
> 'rb.0.10f61.238e1f29.2ac5' -> pg 1.11692600 (1.0) -> up 
> ([38,0,20],
> p38) acting ([38,0,20], p38)*
> *[root@ceph-node1 ~]#*
>
>
> Since i don't have object 'rb.0.10f61.238e1f29.2ac5' in data
> and rbd pools , rados ls will not list it. Which is expected.
>
> *[root@ceph-node1 ~]# rados -p data ls | grep -i
> rb.0.10f61.238e1f29.2ac5*
> *[root@ceph-node1 ~]# rados -p rbd ls | grep -i
> rb.0.10f61.238e1f29.2ac5*
>
>
> But , how come the object is showing in osd map of pool data and rbd.
>
> *[root@ceph-node1 ~]# ceph osd map data
> rb.0.10f61.238e1f29.2ac5*
> *osdmap e1357 pool 'data' (2) object
> 'rb.0.10f61.238e1f29.2ac5' -> pg 2.11692600 (2.0) -> up 
> ([3,51,29],
> p3) acting ([3,51,29], p3)*
> *[root@ceph-node1 ~]#*
>
> *[root@ceph-node1 ~]# ceph osd map rbd
> rb.0.10f61.238e1f29.2ac5*
> *osdmap e1357 pool 'rbd' (0) object 'rb.0.10f61.238e1f29.2ac5'
> -> pg 0.11692600 (0.0) -> up ([41,20,3], p41) acting ([41,20,3], p41)*
> *[root@ceph-node1 ~]#*
>
>
> In ceph, object is unique and belongs to only one pool. So why does it
> shows up in all pool's osd map.
>
> Is this some kind of BUG in Ceph
>
> Ceph Hammer 0.94.5
> CentOS 7.2
>

>>>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Incorrect output from ceph osd map command

2016-02-23 Thread Gregory Farnum 
On Tuesday, February 23, 2016, Vickey Singh 
wrote:

> Thanks Greg,
>
> Do you mean ceph osd map command is not displaying accurate information ?
>
> I guess, either of these things are happening with my cluster
> - ceph osd map is not printing true information
> - Object to PG mapping is not correct ( one object is mapped to multiple
> PG's )
>
> This is happening for several objects , but the cluster is Healthy.
>

No, you're looking for the map command to do something it was not designed
for. If you want to see if an object exists, you will need to use a RADOS
client to fetch the object and see if it's there. "map" is a mapping
command: given an object name, which PG/OSD does CRUSH map that name to?



>
> Need expert suggestion.
>
>
> On Tue, Feb 23, 2016 at 7:20 PM, Gregory Farnum  > wrote:
>
>> This is not a bug. The map command just says which PG/OSD an object maps
>> to; it does not go out and query the osd to see if there actually is such
>> an object.
>> -Greg
>>
>>
>> On Tuesday, February 23, 2016, Vickey Singh > > wrote:
>>
>>> Hello Guys
>>>
>>> I am getting wired output from osd map. The object does not exists on
>>> pool but osd map still shows its PG and OSD on which its stored.
>>>
>>> So i have rbd device coming from pool 'gold' , this image has an object
>>> 'rb.0.10f61.238e1f29.2ac5'
>>>
>>> The below commands verifies this
>>>
>>> *[root@ceph-node1 ~]# rados -p gold ls | grep -i
>>> rb.0.10f61.238e1f29.2ac5*
>>> *rb.0.10f61.238e1f29.2ac5*
>>> *[root@ceph-node1 ~]#*
>>>
>>> This object lives on pool gold and OSD 38,0,20 , which is correct
>>>
>>> *[root@ceph-node1 ~]# ceph osd map gold rb.0.10f61.238e1f29.2ac5*
>>> *osdmap e1357 pool 'gold' (1) object 'rb.0.10f61.238e1f29.2ac5'
>>> -> pg 1.11692600 (1.0) -> up ([38,0,20], p38) acting ([38,0,20], p38)*
>>> *[root@ceph-node1 ~]#*
>>>
>>>
>>> Since i don't have object 'rb.0.10f61.238e1f29.2ac5' in data and
>>> rbd pools , rados ls will not list it. Which is expected.
>>>
>>> *[root@ceph-node1 ~]# rados -p data ls | grep -i
>>> rb.0.10f61.238e1f29.2ac5*
>>> *[root@ceph-node1 ~]# rados -p rbd ls | grep -i
>>> rb.0.10f61.238e1f29.2ac5*
>>>
>>>
>>> But , how come the object is showing in osd map of pool data and rbd.
>>>
>>> *[root@ceph-node1 ~]# ceph osd map data rb.0.10f61.238e1f29.2ac5*
>>> *osdmap e1357 pool 'data' (2) object 'rb.0.10f61.238e1f29.2ac5'
>>> -> pg 2.11692600 (2.0) -> up ([3,51,29], p3) acting ([3,51,29], p3)*
>>> *[root@ceph-node1 ~]#*
>>>
>>> *[root@ceph-node1 ~]# ceph osd map rbd rb.0.10f61.238e1f29.2ac5*
>>> *osdmap e1357 pool 'rbd' (0) object 'rb.0.10f61.238e1f29.2ac5'
>>> -> pg 0.11692600 (0.0) -> up ([41,20,3], p41) acting ([41,20,3], p41)*
>>> *[root@ceph-node1 ~]#*
>>>
>>>
>>> In ceph, object is unique and belongs to only one pool. So why does it
>>> shows up in all pool's osd map.
>>>
>>> Is this some kind of BUG in Ceph
>>>
>>> Ceph Hammer 0.94.5
>>> CentOS 7.2
>>>
>>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Incorrect output from ceph osd map command

2016-02-23 Thread Vickey Singh 
Thanks Greg,

Do you mean ceph osd map command is not displaying accurate information ?

I guess, either of these things are happening with my cluster
- ceph osd map is not printing true information
- Object to PG mapping is not correct ( one object is mapped to multiple
PG's )

This is happening for several objects , but the cluster is Healthy.

Need expert suggestion.


On Tue, Feb 23, 2016 at 7:20 PM, Gregory Farnum  wrote:

> This is not a bug. The map command just says which PG/OSD an object maps
> to; it does not go out and query the osd to see if there actually is such
> an object.
> -Greg
>
>
> On Tuesday, February 23, 2016, Vickey Singh 
> wrote:
>
>> Hello Guys
>>
>> I am getting wired output from osd map. The object does not exists on
>> pool but osd map still shows its PG and OSD on which its stored.
>>
>> So i have rbd device coming from pool 'gold' , this image has an object
>> 'rb.0.10f61.238e1f29.2ac5'
>>
>> The below commands verifies this
>>
>> *[root@ceph-node1 ~]# rados -p gold ls | grep -i
>> rb.0.10f61.238e1f29.2ac5*
>> *rb.0.10f61.238e1f29.2ac5*
>> *[root@ceph-node1 ~]#*
>>
>> This object lives on pool gold and OSD 38,0,20 , which is correct
>>
>> *[root@ceph-node1 ~]# ceph osd map gold rb.0.10f61.238e1f29.2ac5*
>> *osdmap e1357 pool 'gold' (1) object 'rb.0.10f61.238e1f29.2ac5'
>> -> pg 1.11692600 (1.0) -> up ([38,0,20], p38) acting ([38,0,20], p38)*
>> *[root@ceph-node1 ~]#*
>>
>>
>> Since i don't have object 'rb.0.10f61.238e1f29.2ac5' in data and
>> rbd pools , rados ls will not list it. Which is expected.
>>
>> *[root@ceph-node1 ~]# rados -p data ls | grep -i
>> rb.0.10f61.238e1f29.2ac5*
>> *[root@ceph-node1 ~]# rados -p rbd ls | grep -i
>> rb.0.10f61.238e1f29.2ac5*
>>
>>
>> But , how come the object is showing in osd map of pool data and rbd.
>>
>> *[root@ceph-node1 ~]# ceph osd map data rb.0.10f61.238e1f29.2ac5*
>> *osdmap e1357 pool 'data' (2) object 'rb.0.10f61.238e1f29.2ac5'
>> -> pg 2.11692600 (2.0) -> up ([3,51,29], p3) acting ([3,51,29], p3)*
>> *[root@ceph-node1 ~]#*
>>
>> *[root@ceph-node1 ~]# ceph osd map rbd rb.0.10f61.238e1f29.2ac5*
>> *osdmap e1357 pool 'rbd' (0) object 'rb.0.10f61.238e1f29.2ac5' ->
>> pg 0.11692600 (0.0) -> up ([41,20,3], p41) acting ([41,20,3], p41)*
>> *[root@ceph-node1 ~]#*
>>
>>
>> In ceph, object is unique and belongs to only one pool. So why does it
>> shows up in all pool's osd map.
>>
>> Is this some kind of BUG in Ceph
>>
>> Ceph Hammer 0.94.5
>> CentOS 7.2
>>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Incorrect output from ceph osd map command

2016-02-23 Thread Gregory Farnum 
This is not a bug. The map command just says which PG/OSD an object maps
to; it does not go out and query the osd to see if there actually is such
an object.
-Greg

On Tuesday, February 23, 2016, Vickey Singh 
wrote:

> Hello Guys
>
> I am getting wired output from osd map. The object does not exists on pool
> but osd map still shows its PG and OSD on which its stored.
>
> So i have rbd device coming from pool 'gold' , this image has an object
> 'rb.0.10f61.238e1f29.2ac5'
>
> The below commands verifies this
>
> *[root@ceph-node1 ~]# rados -p gold ls | grep -i
> rb.0.10f61.238e1f29.2ac5*
> *rb.0.10f61.238e1f29.2ac5*
> *[root@ceph-node1 ~]#*
>
> This object lives on pool gold and OSD 38,0,20 , which is correct
>
> *[root@ceph-node1 ~]# ceph osd map gold rb.0.10f61.238e1f29.2ac5*
> *osdmap e1357 pool 'gold' (1) object 'rb.0.10f61.238e1f29.2ac5' ->
> pg 1.11692600 (1.0) -> up ([38,0,20], p38) acting ([38,0,20], p38)*
> *[root@ceph-node1 ~]#*
>
>
> Since i don't have object 'rb.0.10f61.238e1f29.2ac5' in data and
> rbd pools , rados ls will not list it. Which is expected.
>
> *[root@ceph-node1 ~]# rados -p data ls | grep -i
> rb.0.10f61.238e1f29.2ac5*
> *[root@ceph-node1 ~]# rados -p rbd ls | grep -i
> rb.0.10f61.238e1f29.2ac5*
>
>
> But , how come the object is showing in osd map of pool data and rbd.
>
> *[root@ceph-node1 ~]# ceph osd map data rb.0.10f61.238e1f29.2ac5*
> *osdmap e1357 pool 'data' (2) object 'rb.0.10f61.238e1f29.2ac5' ->
> pg 2.11692600 (2.0) -> up ([3,51,29], p3) acting ([3,51,29], p3)*
> *[root@ceph-node1 ~]#*
>
> *[root@ceph-node1 ~]# ceph osd map rbd rb.0.10f61.238e1f29.2ac5*
> *osdmap e1357 pool 'rbd' (0) object 'rb.0.10f61.238e1f29.2ac5' ->
> pg 0.11692600 (0.0) -> up ([41,20,3], p41) acting ([41,20,3], p41)*
> *[root@ceph-node1 ~]#*
>
>
> In ceph, object is unique and belongs to only one pool. So why does it
> shows up in all pool's osd map.
>
> Is this some kind of BUG in Ceph
>
> Ceph Hammer 0.94.5
> CentOS 7.2
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com