Thanks. I hadn’t actually found ‘ceph df’. It probably just needs a brief description of what the raw totals include.
One question relating to this, the documentation you’ve linked to suggests that the pool usage stats are converted to megabytes and gigabytes where relevant, are they also converted to terabytes when we get that big? George From: John Wilkins [mailto:[email protected]] Sent: 19 June 2014 18:56 To: Gregory Farnum Cc: Ryall, George (STFC,RAL,SC); [email protected] Subject: Re: [ceph-users] understanding rados df statistics George, I'll look into writing up some additional detail. We do have a description for 'ceph df' here: http://ceph.com/docs/master/rados/operations/monitoring/#checking-a-cluster-s-usage-stats On Thu, Jun 19, 2014 at 8:07 AM, Gregory Farnum <[email protected]<mailto:[email protected]>> wrote: Yeah, the journal is a fixed size; it won't grow! On Thursday, June 19, 2014, <[email protected]<mailto:[email protected]>> wrote: Having looked at a sample of OSDs it appears that it is indeed the case that for every GB of data we have 9 GB of Journal. Is this normal? Or are we not doing some Journal/cluster management that we should be? George From: Gregory Farnum [mailto:[email protected]] Sent: 19 June 2014 13:53 To: Ryall, George (STFC,RAL,SC) Cc: [email protected]<mailto:[email protected]> Subject: Re: [ceph-users] understanding rados df statistics The total used/available/capacity is calculated by running the syscall which "df" uses across all OSDs and summing the results. The "total data" is calculated by summing the sizes of the objects stored. It depends on how you've configured your system, but I'm guessing the markup is due to the (constant size) overhead of your journals. Or anything else which you might have stored on the disks besides Ceph? -Greg On Thursday, June 19, 2014, <[email protected]<mailto:[email protected]>> wrote: Hi all, I’m struggling to understand some Ceph usage statistics and I was hoping someone might be able to explain them to me. If I run ‘rados df’ I get the following: # rados df pool name category KB objects clones degraded unfound rd rd KB wr wr KB pool-1 - 0 0 0 0 0 0 0 0 0 pool-2 - 2339809 1299 0 0 0 300 540600 3301 2340798 pool-3 - 4095749 14654 0 0 0 3969 17256 3337952 70296734 pool-4 - 1802832 39332 0 0 0 0 0 2205979 0 pool-5 - 193102485 82397 0 0 0 668938 102410614 5230404 254457331 total used 5402116076<tel:5402116076> 137682 total avail 854277445084 total space 859679561160 Pools 2 and 4 have a size of 2, whilst pools 3 and 5 have a size of 3. ‘ceph status’ tells me the following stats: “192 GB data, 134 kobjects, 5151 GB used, 795 TB / 800 TB avail” The 192 GB of data is equal to the sum of the ‘KB’ column of the rados df data. The used and available numbers are the same the totals given by rados df. What I don’t understand is how we have used 5,151 GB of data. Given the sizes of each pool I would expect it to be closer to 572 GB (sum of the size of each pool multiplied by pool ‘size’) plus some overhead of some kind. This is a factor of 9 different. So my question is: what have I missed? Cheers, George Ryall Scientific Computing | STFC Rutherford Appleton Laboratory | Harwell Oxford | Didcot | OX11 0QX (01235 44) 5021 -- Scanned by iCritical. -- Software Engineer #42 @ http://inktank.com | http://ceph.com -- Software Engineer #42 @ http://inktank.com | http://ceph.com _______________________________________________ ceph-users mailing list [email protected]<mailto:[email protected]> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- John Wilkins Senior Technical Writer Intank [email protected]<mailto:[email protected]> (415) 425-9599 http://inktank.com
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
