Re: [ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-26 Thread Simon Leinen
cmonty14 writes: > due to performance issues RGW is not an option. This statement may be > wrong, but there's the following aspect to consider. > If I write a backup that is typically a large file, this is normally a > single IO stream. > This causes massive performance issues on Ceph because

Re: [ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-23 Thread cmonty14
Hi, due to performance issues RGW is not an option. This statement may be wrong, but there's the following aspect to consider. If I write a backup that is typically a large file, this is normally a single IO stream. This causes massive performance issues on Ceph because this single IO stream is

Re: [ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-22 Thread Christian Wuerdig
If you use librados directly it's up to you to ensure you can identify your objects. Generally RADOS stores objects and not files so when you provide your object ids you need to come up with a convention so you can correctly identify them. If you need to provide meta data (i.e. a list of all

Re: [ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-22 Thread Jack
AFAIK, the only AAA available with librados works on a pool granularity So, if you create a ceph user with access to your pool, he will get access to all the content stored in this pool If you want to use librados for your use case, you will need to implement, on your code, the application logic

Re: [ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-22 Thread cmonty14
My backup client is using librados. I understand that defining a pool for the same application is recommended. However this would not answer my other questions: How can I identify a backup created by client A that I want to restore on another client Z? I mean typically client A would write a

Re: [ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-22 Thread ceph
Hi, Ceph's pool are meant to let you define specific engineering rules and/or application (rbd, cephfs, rgw) They are not designed to be created in a massive fashion (see pgs etc) So, create a pool for each engineering ruleset, and store your data in them For what is left of your project, I

[ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-22 Thread cmonty14
Hi, my use case for Ceph is providing a central backup storage. This means I will backup multiple databases in Ceph storage cluster. This is my question: What is the best practice for creating pools & images? Should I create multiple pools, means one pool per database? Or should I create a

Re: [ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-22 Thread Eugen Block
Hi Thomas, What is the best practice for creating pools & images? Should I create multiple pools, means one pool per database? Or should I create a single pool "backup" and use namespace when writing data in the pool? I don't think one pool per DB is reasonable. If the number of DBs

[ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-21 Thread Thomas
Hi,   my use case for Ceph is serving a central backup storage. This means I will backup multiple databases in Ceph storage cluster.   This is my question: What is the best practice for creating pools & images? Should I create multiple pools, means one pool per database? Or should I create a

[ceph-users] Using Ceph central backup storage - Best practice creating pools

2019-01-21 Thread Thomas
Hi,   my use case for Ceph is serving a central backup storage. This means I will backup multiple databases in Ceph storage cluster.   This is my question: What is the best practice for creating pools & images? Should I create multiple pools, means one pool per database? Or should I create a