Thanks Josh. Sharding the keys over many buckets does make sense, but then the question is over how many buckets? Every amazon user has a limit (1000), by default, on the number of buckets he can create. Is there any good reason for restricting the number of buckets created by a user? Unfortunately, I couldn't find any documentation detailing the architecture of how rados gateway provides the Amazon S3 abstraction. Moreover, is there any good practice on naming keys? We are inclined to use some uuid as a key and distribute keys across 256 buckets. I am just concerned about any long term repercussions of this design from rados gateway's architectural perspective.
Aniket On Mon, Sep 30, 2013 at 7:54 PM, Josh Durgin <[email protected]>wrote: > On 09/29/2013 07:34 PM, Aniket Nanhe wrote: > >> Hi, >> We have a Ceph cluster set up and are trying to evaluate Ceph for it's S3 >> compatible object storage. I came across this best practices document for >> Amazon S3, which goes over how naming keys in a particular way can improve >> performance of object GET and PUT operations ( >> http://aws.amazon.com/**articles/1904/<http://aws.amazon.com/articles/1904/> >> ). >> I wonder if this also applies to the object store in Ceph. I am also >> curious about the best strategy to organize objects in buckets i.e. >> whether >> it's a good idea to distribute objects to predefined number of buckets >> (say for instance 256 or 1024 buckets) or it just doesn't matter how many >> objects you put in a bucket (i.e. just put all objects in a single >> bucket). >> We have objects of size ranging from 50KB to 10 MB. >> > > You probably want to shard them over many buckets. See > http://lists.ceph.com/**pipermail/ceph-users-ceph.com/** > 2013-March/000595.html<http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-March/000595.html> > > -- Aniket M Nanhe
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
