This [*] is my ceph.conf
10.70.42.9 is the public address
And it is indeed the IP used by the MON daemon:
[root@c-mon-02 ~]# netstat -anp | grep 6789
tcp 0 0 10.70.42.9:6789 0.0.0.0:* LISTEN
3835/ceph-mon
tcp 0 0 10.70.42.9:33592 10.70.42.10:6789
ESTABLISHED 3835/ceph-mon
tcp 0 0 10.70.42.9:41786 10.70.42.8:6789
ESTABLISHED 3835/ceph-mon
tcp 106008 0 10.70.42.9:33210 10.70.42.10:6789
CLOSE_WAIT 1162/ceph-mgr
tcp 100370 0 10.70.42.9:33218 10.70.42.10:6789
CLOSE_WAIT 1162/ceph-mgr
tcp 0 0 10.70.42.9:33578 10.70.42.10:6789
ESTABLISHED 1162/ceph-mgr
But the command:
/usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon.
--keyring=/var/lib/ceph/mon/ceph-c-mon-02/keyring auth get client.admin
exported keyring for client.admin
fails with c-mon-02 resolved to the management IP.
As wokaround I can add in /etc/hosts the mapping with the public address:
10.70.42.9 c-mon-02
but I wonder if this is the expected behavior
Cheers, Massimo
[*]
[global]
fsid = 7a8cb8ff-562b-47da-a6aa-507136587dcf
public network = 10.70.42.0/24
cluster network = 10.69.42.0/24
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
osd pool default size = 3 # Write an object 3 times.
osd pool default min size = 2
osd pool default pg num = 128
osd pool default pgp num = 128
[mon]
mon host = c-mon-01, c-mon-02, c-mon-03
mon addr = 10.70.42.10:6789, 10.70.42.9:6789, 10.70.42.8:6789
[mon.c-mon-01]
mon addr = 10.70.42.10:6789
host = c-mon-01
[mon.c-mon-02]
mon addr = 10.70.42.9:6789
host = c-mon-02
[mon.c-mon-03]
mon addr = 10.70.42.8:6789
host = c-mon-03
[osd]
osd mount options xfs = rw,noatime,inode64,logbufs=8,logbsize=256k
On Thu, May 10, 2018 at 1:12 PM, Paul Emmerich <paul.emmer...@croit.io>
wrote:
> check ceph.conf, it controls to which mon IP the client tries to connect.
>
> 2018-05-10 12:57 GMT+02:00 Massimo Sgaravatto <
> massimo.sgarava...@gmail.com>:
>
>> I configured the "public network" attribute in the ceph configuration
>> file.
>>
>> But it looks like to me that in the "auth get client.admin" command [*]
>> issued by ceph-deploy the address of the management network is used (I
>> guess because c-mon-02 gets resolved to the IP management address)
>>
>> Cheers, Massimo
>>
>> [*]
>> /usr/bin/ceph --connect-timeout=25 --cluster=ceph --name mon.
>> --keyring=/var/lib/ceph/mon/ceph-c-mon-02/keyring auth get client.admin
>>
>> On Thu, May 10, 2018 at 12:49 PM, Paul Emmerich <paul.emmer...@croit.io>
>> wrote:
>>
>>> Monitors can use only exactly one IP address. ceph-deploy uses some
>>> heuristics
>>> based on hostname resolution and ceph public addr configuration to guess
>>> which
>>> one to use during setup. (Which I've always found to be a quite annoying
>>> feature.)
>>>
>>> The mon's IP must be reachable from all ceph daemons and clients, so it
>>> should be
>>> on your "public" network. Changing the IP of a mon is possible but
>>> annoying, it is
>>> often easier to remove and then re-add with a new IP (if possible):
>>>
>>> http://docs.ceph.com/docs/master/rados/operations/add-or-rm-
>>> mons/#changing-a-monitor-s-ip-address
>>>
>>>
>>> Paul
>>>
>>> 2018-05-10 12:36 GMT+02:00 Massimo Sgaravatto <
>>> massimo.sgarava...@gmail.com>:
>>>
>>>> I have a ceph cluster that I manually deployed, and now I am trying to
>>>> see if I can use ceph-deploy to deploy new nodes (in particular the object
>>>> gw).
>>>>
>>>> The network configuration is the following:
>>>>
>>>> Each MON node has two network IP: one on a "management network" (not
>>>> used for ceph related stuff) and one on a "public network",
>>>> The MON daemon listens to on the pub network
>>>>
>>>> Each OSD node has three network IPs: one on a "management network"
>>>> (not used for ceph related stuff), one on a "public network" and the third
>>>> one is an internal network to be used as ceph cluster network (for ceph
>>>> internal traffic: replication, recovery, etc)
>>>>
>>>>
>>>> Name resolution works, but names are resolved to the IP address of the
>>>> management network.
>>>> And it looks like this is a problem. E.g. the following command (used
>>>> in ceph-deploy gatherkeys) issued on a MON host (c-mon-02) doesn't work:
>>>>
>>>> /usr/bin/ceph --verbose --connect-timeout=25 --cluster=ceph --name mon.
>>>> --keyring=/var/lib/ceph/mon/ceph-c-mon-02/keyring auth get client.admin
>>>>
>>>> unless I change the name resolution of c-mon-02 to the public address
>>>>
>>>>
>>>> Is it a requirement (at least for ceph-deploy) that the name of each
>>>> node of the ceph cluster must be resolved to the public IP address ?
>>>>
>>>>
>>>> Thanks, Massimo
>>>>
>>>> _______________________________________________
>>>> ceph-users mailing list
>>>> ceph-users@lists.ceph.com
>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>>
>>>>
>>>
>>>
>>> --
>>> --
>>> Paul Emmerich
>>>
>>> Looking for help with your Ceph cluster? Contact us at https://croit.io
>>>
>>> croit GmbH
>>> Freseniusstr. 31h
>>> <https://maps.google.com/?q=Freseniusstr.+31h+81247+M%C3%BCnchen&entry=gmail&source=g>
>>> 81247 München
>>> <https://maps.google.com/?q=Freseniusstr.+31h+81247+M%C3%BCnchen&entry=gmail&source=g>
>>> www.croit.io
>>> Tel: +49 89 1896585 90
>>>
>>
>>
>
>
> --
> --
> Paul Emmerich
>
> Looking for help with your Ceph cluster? Contact us at https://croit.io
>
> croit GmbH
> Freseniusstr. 31h
> <https://maps.google.com/?q=Freseniusstr.+31h+81247+M%C3%BCnchen&entry=gmail&source=g>
> 81247 München
> <https://maps.google.com/?q=Freseniusstr.+31h+81247+M%C3%BCnchen&entry=gmail&source=g>
> www.croit.io
> Tel: +49 89 1896585 90
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
