> ). >>> >>> There is ~5.6PiB stored on /ceph, shown by ceph.dir.rbytes with 132M files >>> and 139M rentries. The pool shows 7PiB stored and 9.7PiB used consistent >>> with 8+3 EC. >>> The layout for most files: >>> ceph.dir.layout="stripe_unit=16777216 stripe_count=1 object_size=16777216 >>> pool=cephfs_data_echdd" >>> >>> But there is 1.4PiB discrepancy between the pool and the filesystem
Do you have scrubs enabled? Which if any non-default config options do you have set? Any undersized or degraded or backfilling PGs? Which Ceph release? Do you have a sizable fraction of small files? If you’re running Squid or earlier or don’t have EC optimizations enabled, even a tiny file will allocate a multiple of 11*16=176 KB. An 129KB file will consume 352KB, etc. If I understand those layout options correctly. If that’s what’s going on, going to Tentacle with EC optimizations would gain you some efficiency for files newly [re]written. You could also migrate small files to a replicated pool. >>> which I cannot explain and I suspect there are a lot of orphan objects >>> there. I have run mds scrub on / and ~mdsdir as well. There is some mds >>> damage on some old small files (~400 files), which I do not think it's >>> relevant here. >>> >> Hello, >> We had a similar issue last year with a group of users that created and >> removed files at a very high rate. >> >> Have you read >> https://docs.clyso.com/docs/kb/cephfs/#cephfs-pool-data-usage-growth-without-explanation >> ? >> >> We increased the purge rate parameters (very) aggressively to get back to a >> comfortable situation (i.e. not a pool w/ near full warnings). >> >> >> Loïc. > > > -- > _____________________________________________________________ > prof. dr. Andrej Filipcic, E-mail: [email protected] > Department of Experimental High Energy Physics - F9 > Jozef Stefan Institute, Jamova 39, P.o.Box 3000 > SI-1001 Ljubljana, Slovenia > Tel.: +386-1-477-3674 Fax: +386-1-477-3166 > ------------------------------------------------------------- > _______________________________________________ > ceph-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
