AFAIK, you only have 2 networks for Ceph. The private internal traffic between the OSDs. Only servers running OSD daemons need access to this vlan/subnet. The other is the public network. The following things need access to this subnet/vlan: 1) Anything that accesses data like rbds, cephfs, or using librados (or any other Ceph library to access data). 2) Any server running Ceph CLI commands. 3) Anything else running a Ceph Damon needs access to this subnet (Mon, mds, rgw, etc)
Every single one of the above needs to be able to access all of the mons and osds. I don't think you can have multiple subnets for this, but you can do this via routing. Say your private osd network is xxx.xxx.10.0, your public ceph network is .11. Now the only things with an IP on this public network are your osds, mons, and router. Now you can have an isolated client on the .12 subnet with firewall rules allowing it to access .11. Another client can be isolated on .13 that also has firewall rules allowing it to access .11. Now the servers on .12 and .13 cannot communicate with each other, unless you set up firewall rules allowing it. The firewall I would use for this would be pfsense as I'm familiar with it and it can be installed and quickly configured with all of these vlans. Whatever firewall solution you use, it will become your network bandwidth cap into your cluster as all traffic goes through it. On Fri, May 26, 2017, 11:43 AM Deepak Naidu <[email protected]> wrote: > Hi Vlad, > > Thanks for chiming in. > > >>It's not clear what you want to achieve from the ceph point of view? > Multiple tenancy. We will have multiple tenants from different isolated > subnet/network accessing single ceph cluster which can support multiple > tenants. The only problem I see with ceph in a physical env setup is I > cannot isolate public networks , example mon,mds for multiple > subnet/network/tenants. > > >>For example, for the network isolation you can use managed switches, set > different VLANs and put ceph hosts to the every VLAN. > Yes we have managed switches with VLAN. And if I add for example 2x public > interferences on Net1(subnet 192.168.1.0/24) and Net2(subnet > 192.168.2.0/24) how does the ceph.conf look like. How does my mon and MDS > server config look like, that's the challenge/question. > > >>But it's a shoot in the dark as I don't know what exactly you need. For > example, what services (block storage, object storage, API etc) you want to > offer to your tenants and so on > > CephFS and Object. I am familiar on how to get the ceph storage part > "tenant friendly", it's just the network part I need to isolate. > > -- > Deepak > > > On May 26, 2017, at 12:03 AM, Дробышевский, Владимир <[email protected]> > wrote: > > > > It's not clear what you want to achieve from the ceph point of view? > For example, for the network isolation you can use managed switches, set > different VLANs and put ceph hosts to the every VLAN. But it's a shoot in > the dark as I don't know what exactly you need. For example, what services > (block storage, object storage, API etc) you want to offer to your tenants > and so on > > ----------------------------------------------------------------------------------- > This email message is for the sole use of the intended recipient(s) and > may contain > confidential information. Any unauthorized review, use, disclosure or > distribution > is prohibited. If you are not the intended recipient, please contact the > sender by > reply email and destroy all copies of the original message. > > ----------------------------------------------------------------------------------- > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
