On 5/18/20 1:51 PM, Zhenshi Zhou wrote: > Hi Wido, > > I did a research on the nfs files. I found that it contains much > pictures about > 50KB, and much video files around 30MB. The amount of the files is more than > 1 million. Maybe I can find a way to seperate the files in more buckets > so that > there is no more than 1M objects in each bucket. But how about the small > files > around 50KB. Does rgw serve well on small files?
I would recommend using different buckets. What I've done in such cases is use the year+month for sharding. For example: video-2020-05 RGW can serve objects which are 50kB in size, but there is overhead involved. Storing a lot of such small objects comes at a price of overhead. Wido > > Wido den Hollander <w...@42on.com <mailto:w...@42on.com>> 于2020年5月12 > 日周二 下午2:41写道： > > > > On 5/12/20 4:22 AM, Zhenshi Zhou wrote: > > Hi all, > > > > We have several nfs servers providing file storage. There is a > nginx in > > front of > > nfs servers in order to serve the clients. The files are mostly > small files > > and > > nearly about 30TB in total. > > > > What is small? How many objects/files are you talking about? > > > I'm gonna use ceph rgw as the storage. I wanna know if it's > appropriate to > > do so. > > The data migrating from nfs to rgw is a huge job. Besides I'm not sure > > whether > > ceph rgw is suitable in this scenario or not. > > > > Yes, it is. But make sure you don't put millions of objects into a > single bucket. Make sure that you spread them out so that you have let's > say 1M of objects per bucket at max. > > Wido > > > Thanks > > _______________________________________________ > > ceph-users mailing list -- ceph-users@ceph.io > <mailto:ceph-users@ceph.io> > > To unsubscribe send an email to ceph-users-le...@ceph.io > <mailto:ceph-users-le...@ceph.io> > > > _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
