Re: [ceph-users] a big cluster or several small

2018-05-14 Thread João Paulo Sacchetto Ribeiro Bastos 
Hello Marc,

In my beliefs that's exactly the main reason why people use Ceph: its gets
more reliable the more nodes we put in the cluster. You should take a look
in documentation and try to make use of placement rules, erasure codes or
whatever fits your needs. I'm yet new in Ceph (been using for about 1 year)
and I strongly tell you that your ideia just *may be* good, but may be a
little overkill too =D

Regards,

On Mon, May 14, 2018 at 2:26 PM Michael Kuriger  wrote:

> The more servers you have in your cluster, the less impact a failure
> causes to the cluster. Monitor your systems and keep them up to date.  You
> can also isolate data with clever crush rules and creating multiple zones.
>
>
>
> *Mike Kuriger*
>
>
>
> *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf
> Of *Marc Boisis
> *Sent:* Monday, May 14, 2018 9:50 AM
> *To:* ceph-users
> *Subject:* [ceph-users] a big cluster or several small
>
>
>
>
> Hi,
>
>
>
> Hello,
>
> Currently we have a 294 OSD (21 hosts/3 racks) cluster with RBD clients
> only, 1 single pool (size=3).
>
>
>
> We want to divide this cluster into several to minimize the risk in case
> of failure/crash.
>
> For example, a cluster for the mail, another for the file servers, a test
> cluster ...
>
> Do you think it's a good idea ?
>
>
>
> Do you have experience feedback on multiple clusters in production on the
> same hardware:
>
> - containers (LXD or Docker)
>
> - multiple cluster on the same host without virtualization (with
> ceph-deploy ... --cluster ...)
>
> - multilple pools
>
> ...
>
>
>
> Do you have any advice?
>
>
>
>
>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
-- 

João Paulo Bastos
DevOps Engineer at Mav Tecnologia
Belo Horizonte - Brazil
+55 31 99279-7092
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Shared WAL/DB device partition for multiple OSDs?

2018-05-11 Thread João Paulo Sacchetto Ribeiro Bastos 
Actually, if you go to https://ceph.com/community/new-luminous-bluestore/ you
will see that DB/WAL work on a XFS partition, while the data itself goes on
a raw block.

Also, I told you the wrong command in the last mail. When i said --osd-db
it should be --block-db.

On Fri, May 11, 2018 at 11:51 AM Oliver Schulz <oliver.sch...@tu-dortmund.de>
wrote:

> Hi,
>
> thanks for the advice! I'm a bit confused now, though. ;-)
> I thought DB and WAL were supposed to go on raw block
> devices, not file systems?
>
>
> Cheers,
>
> Oliver
>
>
> On 11.05.2018 16:01, João Paulo Sacchetto Ribeiro Bastos wrote:
> > Hello Oliver,
> >
> > As far as I know yet, you can use the same DB device for about 4 or 5
> > OSDs, just need to be aware of the free space. I'm also developing a
> > bluestore cluster, and our DB and WAL will be in the same SSD of about
> > 480GB serving 4 OSD HDDs of 4 TB each. About the sizes, its just a
> > feeling because I couldn't find yet any clear rule about how to measure
> > the requirements.
> >
> > * The only concern that took me some time to realize is that you should
> > create a XFS partition if using ceph-deploy because if you don't it will
> > simply give you a RuntimeError that doesn't give any hint about what's
> > going on.
> >
> > So, answering your question, you could do something like:
> > $ ceph-deploy osd create --bluestore --data=/dev/sdb --block-db
> > /dev/nvme0n1p1 $HOSTNAME
> > $ ceph-deploy osd create --bluestore --data=/dev/sdc --block-db
> > /dev/nvme0n1p1 $HOSTNAME
> >
> > On Fri, May 11, 2018 at 10:35 AM Oliver Schulz
> > <oliver.sch...@tu-dortmund.de <mailto:oliver.sch...@tu-dortmund.de>>
> wrote:
> >
> > Dear Ceph Experts,
> >
> > I'm trying to set up some new OSD storage nodes, now with
> > bluestore (our existing nodes still use filestore). I'm
> > a bit unclear on how to specify WAL/DB devices: Can
> > several OSDs share one WAL/DB partition? So, can I do
> >
> >   ceph-deploy osd create --bluestore --osd-db=/dev/nvme0n1p2
> > --data=/dev/sdb HOSTNAME
> >
> >   ceph-deploy osd create --bluestore --osd-db=/dev/nvme0n1p2
> > --data=/dev/sdc HOSTNAME
> >
> >   ...
> >
> > Or do I need to use osd-db=/dev/nvme0n1p2 for data=/dev/sdb,
> > osd-db=/dev/nvme0n1p3 for data=/dev/sdc, and so on?
> >
> > And just to make sure - if I specify "--osd-db", I don't need
> > to set "--osd-wal" as well, since the WAL will end up on the
> > DB partition automatically, correct?
> >
> >
> > Thanks for any hints,
> >
> > Oliver
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com>
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> > --
> >
> > João Paulo Sacchetto Ribeiro Bastos
> > +55 31 99279-7092 <+55%2031%2099279-7092>
> >
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
-- 

João Paulo Bastos
DevOps Engineer at Mav Tecnologia
Belo Horizonte - Brazil
+55 31 99279-7092
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Shared WAL/DB device partition for multiple OSDs?

2018-05-11 Thread João Paulo Sacchetto Ribeiro Bastos 
Hello Oliver,

As far as I know yet, you can use the same DB device for about 4 or 5 OSDs,
just need to be aware of the free space. I'm also developing a bluestore
cluster, and our DB and WAL will be in the same SSD of about 480GB serving
4 OSD HDDs of 4 TB each. About the sizes, its just a feeling because I
couldn't find yet any clear rule about how to measure the requirements.

* The only concern that took me some time to realize is that you should
create a XFS partition if using ceph-deploy because if you don't it will
simply give you a RuntimeError that doesn't give any hint about what's
going on.

So, answering your question, you could do something like:
$ ceph-deploy osd create --bluestore --data=/dev/sdb --block-db
/dev/nvme0n1p1 $HOSTNAME
$ ceph-deploy osd create --bluestore --data=/dev/sdc --block-db
/dev/nvme0n1p1 $HOSTNAME

On Fri, May 11, 2018 at 10:35 AM Oliver Schulz <oliver.sch...@tu-dortmund.de>
wrote:

> Dear Ceph Experts,
>
> I'm trying to set up some new OSD storage nodes, now with
> bluestore (our existing nodes still use filestore). I'm
> a bit unclear on how to specify WAL/DB devices: Can
> several OSDs share one WAL/DB partition? So, can I do
>
>  ceph-deploy osd create --bluestore --osd-db=/dev/nvme0n1p2
> --data=/dev/sdb HOSTNAME
>
>  ceph-deploy osd create --bluestore --osd-db=/dev/nvme0n1p2
> --data=/dev/sdc HOSTNAME
>
>  ...
>
> Or do I need to use osd-db=/dev/nvme0n1p2 for data=/dev/sdb,
> osd-db=/dev/nvme0n1p3 for data=/dev/sdc, and so on?
>
> And just to make sure - if I specify "--osd-db", I don't need
> to set "--osd-wal" as well, since the WAL will end up on the
> DB partition automatically, correct?
>
>
> Thanks for any hints,
>
> Oliver
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
-- 

João Paulo Sacchetto Ribeiro Bastos
+55 31 99279-7092
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] howto: multiple ceph filesystems

2018-05-10 Thread João Paulo Sacchetto Ribeiro Bastos 
Hey John, thanks for you answer. For sure the hardware robustness will be
nice enough. My true concern was actually the two FS ecosystem coexistence.
In fact I realized that we may not use this as well because it may be
represent a high overhead, despite the fact that it's a experiental feature
yet.
On Thu, 10 May 2018 at 15:48 John Spray <jsp...@redhat.com> wrote:

> On Thu, May 10, 2018 at 7:38 PM, João Paulo Sacchetto Ribeiro Bastos
> <joaopaulos...@gmail.com> wrote:
> > Hello guys,
> >
> > My company is about to rebuild its whole infrastructure, so I was called
> in
> > order to help on the planning. We are essentially an corporate mail
> > provider, so we handle daily lots of clients using dovecot and roundcube
> and
> > in order to do so we want to design a better plant of our cluster. Today,
> > using Jewel, we have a single cephFS for both index and mail from
> dovecot,
> > but we want to split it into an index_FS and a mail_FS to handle the
> > workload a little better, is it profitable nowadays? From my research I
> > realized that we will need data and metadata individual pools for each FS
> > such as a group of MDS for each of then, also.
> >
> > The one thing that really scares me about all of this is: we are
> planning to
> > have four machines at full disposal to handle our MDS instances. We
> started
> > to think if an idea like the one below is valid, can anybody give a hint
> on
> > this? We basically want to handle two MDS instances on each machine (one
> for
> > each FS) and wonder if we'll be able to have them swapping between active
> > and standby simultaneously without any trouble.
> >
> > index_FS: (active={machines 1 and 3}, standby={machines 2 and 4})
> > mail_FS: (active={machines 2 and 4}, standby={machines 1 and 3})
>
> Nothing wrong with that setup, but remember that those servers are
> going to have to be well-resourced enough to run all four at once
> (when a failure occurs), so it might not matter very much exactly
> which servers are running which daemons.
>
> With a filesystem's MDS daemons (i.e. daemons with the same
> standby_for_fscid setting), Ceph will activate whichever daemon comes
> up first, so if it's important to you to have particular daemons
> active then you would need to take care of that at the point you're
> starting them up.
>
> John
>
> >
> > Regards,
> > --
> >
> > João Paulo Sacchetto Ribeiro Bastos
> > +55 31 99279-7092
> >
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
-- 

João Paulo Sacchetto Ribeiro Bastos
+55 31 99279-7092
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] howto: multiple ceph filesystems

2018-05-10 Thread João Paulo Sacchetto Ribeiro Bastos 
Hello guys,

My company is about to rebuild its whole infrastructure, so I was called in
order to help on the planning. We are essentially an corporate mail
provider, so we handle daily lots of clients using dovecot and roundcube
and in order to do so we want to design a better plant of our cluster.
Today, using Jewel, we have a single cephFS for both index and mail from
dovecot, but we want to split it into an index_FS and a mail_FS to handle
the workload a little better, is it profitable nowadays? From my research I
realized that we will need data and metadata individual pools for each FS
such as a group of MDS for each of then, also.

The one thing that really scares me about all of this is: we are planning
to have four machines at full disposal to handle our MDS instances. We
started to think if an idea like the one below is valid, can anybody give a
hint on this? We basically want to handle two MDS instances on each machine
(one for each FS) and wonder if we'll be able to have them swapping between
active and standby simultaneously without any trouble.

index_FS: (active={machines 1 and 3}, standby={machines 2 and 4})
mail_FS: (active={machines 2 and 4}, standby={machines 1 and 3})

Regards,
-- 

João Paulo Sacchetto Ribeiro Bastos
+55 31 99279-7092
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com