Re: [ceph-users] Ceph with Clos IP fabric

Hi, sorry for the delay, but in the meantime we were able to find a workaround. Inspired by this: > Side note: Configuring the loopback IP on the physical interfaces is > workable if you set it on **all** parallel links. Example with server1: > > > > “iface enp3s0f0 inet static > >

Re: [ceph-users] Ceph with Clos IP fabric

...@elits.com>> > > To: Richard Hesse > > <richard.he...@weebly.com<mailto:richard.he...@weebly.com>>, Jan Marquardt > > <j...@artfiles.de<mailto:j...@artfiles.de>> > > Cc: "ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>&quo

Re: [ceph-users] Ceph with Clos IP fabric

display/DOCS/Redistribute+ > > > Neighbor#RedistributeNeighbor-ConfiguringtheHost(s) > > > > > > Honestly the trickiest part is getting the routing on the hosts right, > you > > > essentially set static routes over each link and the kernel takes care

Re: [ceph-users] Ceph with Clos IP fabric

akes care of > > the ECMP. > > > > I understand this is a bit different from your setup, but Ceph has no > > trouble at all with the IPs on multiple interfaces. > > > > Aaron > > > > Date: Sat, 22 Apr 2017 17:37:01 + > > From: Maxime Guyot &l

Re: [ceph-users] Ceph with Clos IP fabric

t; I understand this is a bit different from your setup, but Ceph has no > > trouble at all with the IPs on multiple interfaces. > > > > Aaron > > > > Date: Sat, 22 Apr 2017 17:37:01 + > > From: Maxime Guyot <maxime.gu...@elits.com> > > To: Rich

Re: [ceph-users] Ceph with Clos IP fabric

.@weebly.com<mailto:richard.he...@weebly.com>>, Jan Marquardt <j...@artfiles.de<mailto:j...@artfiles.de>> Cc: "ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>" <ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>> Subject: Re: [ceph-user

Re: [ceph-users] Ceph with Clos IP fabric

; > Aaron > > Date: Sat, 22 Apr 2017 17:37:01 + > From: Maxime Guyot <maxime.gu...@elits.com> > To: Richard Hesse <richard.he...@weebly.com>, Jan Marquardt > <j...@artfiles.de> > Cc: "ceph-users@lists.ceph.com" <ceph-users@lists.ceph.com>

Re: [ceph-users] Ceph with Clos IP fabric

iles.de>> Cc: "ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>" <ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>> Subject: Re: [ceph-users] Ceph with Clos IP fabric Message-ID: <919c8615-c50b-4611-9b6b-13b4fbf69...@elits.com<mailto:91

Re: [ceph-users] Ceph with Clos IP fabric

<ceph-users@lists.ceph.com> Subject: Re: [ceph-users] Ceph with Clos IP fabric On Thu, Apr 20, 2017 at 2:13 AM, Maxime Guyot <maxime.gu...@elits.com<mailto:maxime.gu...@elits.com>> wrote: >2) Why did you choose to run the ceph nodes on loopback interfaces as opposed >t

Re: [ceph-users] Ceph with Clos IP fabric

On Thu, Apr 20, 2017 at 2:13 AM, Maxime Guyot wrote: > >2) Why did you choose to run the ceph nodes on loopback interfaces as > opposed to the /24 for the "public" interface? > > I can’t speak for this example, but in a clos fabric you generally want to > assign the

Re: [ceph-users] Ceph with Clos IP fabric

I'd suggest running your Ceph network on the "public" interface /24 you have assigned to that rack. Assign loopback IP's from another (not otherwise routed) network to your MONs. I have no experience with the RBD client, but with RGW you can assign the same IP to all of them and let your ToR

Re: [ceph-users] Ceph with Clos IP fabric

Am 17.04.17 um 22:12 schrieb Richard Hesse: > A couple of questions: > > 1) What is your rack topology? Are all ceph nodes in the same rack > communicating with the same top of rack switch? The cluster is planned for one rack with two ToR-/cluster-internal switches. The cluster will be accessed

Re: [ceph-users] Ceph with Clos IP fabric

A couple of questions: 1) What is your rack topology? Are all ceph nodes in the same rack communicating with the same top of rack switch? 2) Why did you choose to run the ceph nodes on loopback interfaces as opposed to the /24 for the "public" interface? 3) Are you planning on using RGW at all?

[ceph-users] Ceph with Clos IP fabric

Hi, I am currently working on Ceph with an underlying Clos IP fabric and I am hitting some issues. The setup looks as follows: There are 3 Ceph nodes which are running OSDs and MONs. Each server has one /32 loopback ip, which it announces via BGP to its uplink switches. Besides the loopback ip