Re: [ceph-users] Add one more public networks for ceph

[luckydog xf]

OK, thanks.


On Fri, Oct 25, 2019 at 6:07 PM Wido den Hollander  wrote:

>
>
> On 10/25/19 5:27 AM, luckydog xf wrote:
> > Hi, list,
> >
> > Currently my ceph nodes with 3 MON and 9 OSDs, everything is fine.
> > Now I plan to add onre more public network, the initial public network
> > is 103.x/24, and the target network is 109.x/24.  And 103 cannot reach
> > 109, as I don't config route table for them.
> >
> >I add 109.x for 3 MON nodes and they could reach one another, and I
> add
> > 
>
> That will not work. Make sure there is routing between the IP-space.
> That's the easiest and probably only way to make this work.
>
> Wido
>
> > public_network = 172.16.103.0/24 ,
> > 172.16.109.0/24 
> > 
> > After I manually change monmap file and run `ceph -s`
> > -
> >   services:
> > mon: 6 daemons, quorum
> >
> cephnode001,cephnode002,cephnode003,cephnode001-109,cephnode002-109,cephnode003-109
> > 
> > I check 6789 for both 103.x and 109.x is listening properly. But I mount
> > cephfs through IP 109.x, it fails.
> >
> > I use tcpdump and find there does have traffic directed to 109.x
> > interfaces of 3 MON host.
> >
> >  Anything wrong? Is it doable to have 2 public networks(subnets), served
> > for connections sourced from different subnets? I.E, 109 network for
> > connection from 109.x, while 103 network for 103.x?
> >
> > Thanks.
> >
> >
> >
> >
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Add one more public networks for ceph

[Wido den Hollander]

On 10/25/19 5:27 AM, luckydog xf wrote:
> Hi, list, 
> 
>     Currently my ceph nodes with 3 MON and 9 OSDs, everything is fine.
> Now I plan to add onre more public network, the initial public network
> is 103.x/24, and the target network is 109.x/24.  And 103 cannot reach
> 109, as I don't config route table for them.
> 
>    I add 109.x for 3 MON nodes and they could reach one another, and I add 
> 

That will not work. Make sure there is routing between the IP-space.
That's the easiest and probably only way to make this work.

Wido

> public_network = 172.16.103.0/24 ,
> 172.16.109.0/24 
> 
>     After I manually change monmap file and run `ceph -s`
> -
>   services:
>     mon: 6 daemons, quorum
> cephnode001,cephnode002,cephnode003,cephnode001-109,cephnode002-109,cephnode003-109
> 
> I check 6789 for both 103.x and 109.x is listening properly. But I mount
> cephfs through IP 109.x, it fails.
> 
> I use tcpdump and find there does have traffic directed to 109.x
> interfaces of 3 MON host.
> 
>  Anything wrong? Is it doable to have 2 public networks(subnets), served
> for connections sourced from different subnets? I.E, 109 network for
> connection from 109.x, while 103 network for 103.x?
> 
> Thanks.
> 
> 
> 
> 
> 
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Add one more public networks for ceph

[luckydog xf]

I follow  https://docs.ceph.com/docs/mimic/rados/operations/add-or-rm-mons/
to  add MON for 109.x.

On Fri, Oct 25, 2019 at 11:27 AM luckydog xf  wrote:

> Hi, list,
>
> Currently my ceph nodes with 3 MON and 9 OSDs, everything is fine. Now
> I plan to add onre more public network, the initial public network is
> 103.x/24, and the target network is 109.x/24.  And 103 cannot reach 109, as
> I don't config route table for them.
>
>I add 109.x for 3 MON nodes and they could reach one another, and I add
> 
> public_network = 172.16.103.0/24, 172.16.109.0/24
> 
> After I manually change monmap file and run `ceph -s`
> -
>   services:
> mon: 6 daemons, quorum
> cephnode001,cephnode002,cephnode003,cephnode001-109,cephnode002-109,cephnode003-109
> 
> I check 6789 for both 103.x and 109.x is listening properly. But I mount
> cephfs through IP 109.x, it fails.
>
> I use tcpdump and find there does have traffic directed to 109.x
> interfaces of 3 MON host.
>
>  Anything wrong? Is it doable to have 2 public networks(subnets), served
> for connections sourced from different subnets? I.E, 109 network for
> connection from 109.x, while 103 network for 103.x?
>
> Thanks.
>
>
>
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Add one more public networks for ceph

[luckydog xf]

Hi, list,

Currently my ceph nodes with 3 MON and 9 OSDs, everything is fine. Now
I plan to add onre more public network, the initial public network is
103.x/24, and the target network is 109.x/24.  And 103 cannot reach 109, as
I don't config route table for them.

   I add 109.x for 3 MON nodes and they could reach one another, and I add

public_network = 172.16.103.0/24, 172.16.109.0/24

After I manually change monmap file and run `ceph -s`
-
  services:
mon: 6 daemons, quorum
cephnode001,cephnode002,cephnode003,cephnode001-109,cephnode002-109,cephnode003-109

I check 6789 for both 103.x and 109.x is listening properly. But I mount
cephfs through IP 109.x, it fails.

I use tcpdump and find there does have traffic directed to 109.x interfaces
of 3 MON host.

 Anything wrong? Is it doable to have 2 public networks(subnets), served
for connections sourced from different subnets? I.E, 109 network for
connection from 109.x, while 103 network for 103.x?

Thanks.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com