Re: [openstack-dev] [swift] Optimizing storage for small objects in Swift

2017-06-19 Thread Clint Byrum
Excerpts from Alexandre Lécuyer's message of 2017-06-19 11:36:15 +0200: > Hello Clint, > > Thanks for your feedback, replying in the email inline. > > On 06/16/2017 10:54 PM, Clint Byrum wrote: > > Excerpts from John Dickinson's message of 2017-06-16 11:35:39 -0700: > >> On 16 Jun 2017, at

Re: [openstack-dev] [swift] Optimizing storage for small objects in Swift

2017-06-19 Thread Alexandre Lécuyer
Hello John, Thanks for your comments! Replying inline On 06/16/2017 07:06 PM, John Dickinson wrote: Alex, this is fantastic work and great info. Thanks for sharing it. Additional comments inline. On 16 Jun 2017, at 6:54, Alexandre Lécuyer wrote: Swift stores objects on a regular filesystem

Re: [openstack-dev] [swift] Optimizing storage for small objects in Swift

2017-06-19 Thread Alexandre Lécuyer
Hello Clint, Thanks for your feedback, replying in the email inline. On 06/16/2017 10:54 PM, Clint Byrum wrote: Excerpts from John Dickinson's message of 2017-06-16 11:35:39 -0700: On 16 Jun 2017, at 10:51, Clint Byrum wrote: This is great work. I'm sure you've already thought of this, but

Re: [openstack-dev] [swift] Optimizing storage for small objects in Swift

2017-06-16 Thread Clint Byrum
Excerpts from John Dickinson's message of 2017-06-16 11:35:39 -0700: > > On 16 Jun 2017, at 10:51, Clint Byrum wrote: > > > This is great work. > > > > I'm sure you've already thought of this, but could you explain why > > you've chosen not to put the small objects in the k/v store as part of >

Re: [openstack-dev] [swift] Optimizing storage for small objects in Swift

2017-06-16 Thread John Dickinson
On 16 Jun 2017, at 10:51, Clint Byrum wrote: > This is great work. > > I'm sure you've already thought of this, but could you explain why > you've chosen not to put the small objects in the k/v store as part of > the value rather than in secondary large files? I don't want to co-opt an answer

Re: [openstack-dev] [swift] Optimizing storage for small objects in Swift

2017-06-16 Thread Clint Byrum
This is great work. I'm sure you've already thought of this, but could you explain why you've chosen not to put the small objects in the k/v store as part of the value rather than in secondary large files? Excerpts from Alexandre Lécuyer's message of 2017-06-16 15:54:08 +0200: > Swift stores

Re: [openstack-dev] [swift] Optimizing storage for small objects in Swift

2017-06-16 Thread John Dickinson
Alex, this is fantastic work and great info. Thanks for sharing it. Additional comments inline. On 16 Jun 2017, at 6:54, Alexandre Lécuyer wrote: > Swift stores objects on a regular filesystem (XFS is recommended), one file > per object. While it works fine for medium or big objects, when you

[openstack-dev] [swift] Optimizing storage for small objects in Swift

2017-06-16 Thread Alexandre Lécuyer
Swift stores objects on a regular filesystem (XFS is recommended), one file per object. While it works fine for medium or big objects, when you have lots of small objects you can run into issues: because of the high count of inodes on the object servers, they can’t stay in cache, implying lot