On one of our staging ceph clusters (firefly 0.80.10) I've noticed that
some
of the statistics in the 'ceph df' output don't seem to match up. For
example
in the output below the amount of raw used is 8,402G, which with triple
replication would be 2,800.7G used (all the pools are triple replication).
However, if you add up the numbers used by all the pools (424G + 2538G +
103G)
you get 3,065G used (a difference of +264.3G).
GLOBAL:
SIZE AVAIL RAW USED %RAW USED
50275G 41873G 8402G 16.71
POOLS:
NAME ID USED %USED MAX AVAIL OBJECTS
data 0 0 0 13559G 0
metadata 1 0 0 13559G 0
rbd 2 0 0 13559G 0
volumes 3 424G 0.84 13559G 159651
images 4 2538G 5.05 13559G 325198
backups 5 0 0 13559G 0
instances 6 103G 0.21 13559G 25310
The max avail amount doesn't line up either. If you take 3 * 13,559G you
get
40,677G available, but the global stat is 41,873G (a difference of 1,196G).
On another staging cluster the numbers are closer to what I would expect.
The
amount of raw used is 7,037G, which with triple replication should be
2,345.7G. However, adding up the amounts used by all the pools (102G +
1749G
+ 478G + 14G) is 2,343G (a difference of just -2.7G).
GLOBAL:
SIZE AVAIL RAW USED %RAW USED
50275G 43238G 7037G 14.00
POOLS:
NAME ID USED %USED MAX AVAIL OBJECTS
data 0 0 0 13657G 0
metadata 1 0 0 13657G 0
rbd 2 0 0 13657G 0
volumes 3 102G 0.20 13657G 27215
images 4 1749G 3.48 13657G 224259
instances 5 478G 0.95 13657G 79221
backups 6 0 0 13657G 0
scbench 8 14704M 0.03 13657G 3677
The max avail is a little further off. Taking 3 * 13,657G you get 40,971G,
but the global stat is 43,238G (a difference of 2,267G).
My guess would have been that the global numbers would include some of the
overhead involved which lines up with the second cluster, but the first
cluster would have -264.3G of overhead which just doesn't make sense. Any
ideas where these stats might be getting off?
Thanks,
Bryan
________________________________
This E-mail and any of its attachments may contain Time Warner Cable
proprietary information, which is privileged, confidential, or subject to
copyright belonging to Time Warner Cable. This E-mail is intended solely for
the use of the individual or entity to which it is addressed. If you are not
the intended recipient of this E-mail, you are hereby notified that any
dissemination, distribution, copying, or action taken in relation to the
contents of and attachments to this E-mail is strictly prohibited and may be
unlawful. If you have received this E-mail in error, please notify the sender
immediately and permanently delete the original and any copy of this E-mail and
any printout.
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
