Thank you..
After sending the post, I totally removed the mon and issued the build with
ceph-deploy:
In the logs now:
2017-01-07 21:12:38.113534 7fa9613fd700 0 cephx: verify_reply couldn't
decrypt with error: error decoding block for decryption
2017-01-07 21:12:38.113546 7fa9613fd700 0 -- 10.10.10.138:6789/0 >>
10.10.10.103:6789/0 pipe(0x55feb2e90000 sd=12 :50266 s=1 pgs=0 cs=0 l=0
c=0x55feb2ca0a80).failed verifying authorize reply
2017-01-07 21:12:38.114529 7fa95787b700 0 cephx: verify_reply couldn't
decrypt with error: error decoding block for decryption
2017-01-07 21:12:38.114567 7fa95787b700 0 -- 10.10.10.138:6789/0 >>
10.10.10.252:6789/0 pipe(0x55feb2e91400 sd=11 :38690 s=1 pgs=0 cs=0 l=0
c=0x55feb2ca0c00).failed verifying authorize reply
2017-01-07 21:12:40.114522 7fa9613fd700 0 cephx: verify_reply couldn't
decrypt with error: error decoding block for decryption
2017-01-07 21:12:40.114542 7fa9613fd700 0 -- 10.10.10.138:6789/0 >>
10.10.10.103:6789/0 pipe(0x55feb2e90000 sd=11 :50278 s=1 pgs=0 cs=0 l=0
c=0x55feb2ca0a80).failed verifying authorize reply
2017-01-07 21:12:40.115706 7fa95787b700 0 cephx: verify_reply couldn't
decrypt with error: error decoding block for decryption
2017-01-07 21:12:40.115721 7fa95787b700 0 -- 10.10.10.138:6789/0 >>
10.10.10.252:6789/0 pipe(0x55feb2e91400 sd=12 :38702 s=1 pgs=0 cs=0 l=0
c=0x55feb2ca0c00).failed verifying authorize reply
2017-01-07 21:12:41.621916 7fa956f79700 0 cephx: verify_authorizer could
not decrypt ticket info: error: NSS AES final round failed: -8190
2017-01-07 21:12:41.621929 7fa956f79700 0 mon.alex-desktop@1(probing) e0
ms_verify_authorizer bad authorizer from mon 10.10.10.103:6789/0
2017-01-07 21:12:41.621944 7fa956f79700 0 -- 10.10.10.138:6789/0 >>
10.10.10.103:6789/0 pipe(0x55feb2fb5400 sd=21 :6789 s=0 pgs=0 cs=0 l=0
c=0x55feb2ca1500).accept: got bad authorizer
$ sudo ceph -s
cluster f5aba719-4856-4ae2-a5d4-f9ff0f614b60
health HEALTH_WARN
512 pgs degraded
348 pgs stale
512 pgs stuck unclean
512 pgs undersized
6 requests are blocked > 32 sec
recovery 25013/50026 objects degraded (50.000%)
mds cluster is degraded
1 mons down, quorum 0,2 alpha,toshiba-laptop
monmap e17: 3 mons at {alex-desktop=
10.10.10.138:6789/0,alpha=10.10.10.103:6789/0,toshiba-laptop=10.10.10.252:6789/0
}
election epoch 806, quorum 0,2 alpha,toshiba-laptop
fsmap e201858: 1/1/1 up {0=1=up:replay}
osdmap e200229: 3 osds: 2 up, 2 in; 85 remapped pgs
flags sortbitwise
pgmap v4088774: 512 pgs, 4 pools, 50883 MB data, 25013 objects
59662 MB used, 476 GB / 563 GB avail
25013/50026 objects degraded (50.000%)
348 stale+active+undersized+degraded
164 active+undersized+degraded
root@alex-desktop:/var/lib/ceph/mon/ceph-alex-desktop# ls -ls
total 8
0 -rw-r--r-- 1 ceph ceph 0 Jan 7 21:11 done
4 -rw------- 1 ceph ceph 77 Jan 7 21:05 keyring
4 drwxr-xr-x 2 ceph ceph 4096 Jan 7 21:10 store.db
0 -rw-r--r-- 1 ceph ceph 0 Jan 7 21:05 systemd
Very odd... never seen this issue on the other monitor deployments...
Alex F. Evonosky
<https://twitter.com/alexevon> <https://www.linkedin.com/in/alexevonosky>
On Sat, Jan 7, 2017 at 8:54 PM, Shinobu Kinjo <[email protected]> wrote:
> Using ``ceph-deploy`` will save your life:
>
> # https://github.com/ceph/ceph/blob/master/doc/start/quick-
> ceph-deploy.rst
> * Please look at: Adding Monitors
>
> If you are using centos or similar, the latest package is available here:
>
> # http://download.ceph.com/rpm-jewel/el7/noarch/ceph-deploy-
> 1.5.37-0.noarch.rpm
>
> Regards,
>
>
> On Sun, Jan 8, 2017 at 9:53 AM, Alex Evonosky <[email protected]>
> wrote:
>
>> Thank you for the reply!
>>
>> I followed this article:
>>
>> http://docs.ceph.com/docs/jewel/rados/operations/add-or-rm-mons/
>>
>>
>> Under the section: ADDING A MONITOR (MANUAL)
>>
>>
>>
>> Alex F. Evonosky
>>
>> <https://twitter.com/alexevon> <https://www.linkedin.com/in/alexevonosky>
>>
>> On Sat, Jan 7, 2017 at 6:36 PM, Shinobu Kinjo <[email protected]> wrote:
>>
>>> How did you add a third MON?
>>>
>>> Regards,
>>>
>>> On Sun, Jan 8, 2017 at 7:01 AM, Alex Evonosky <[email protected]>
>>> wrote:
>>> > Anyone see this before?
>>> >
>>> >
>>> > 2017-01-07 16:55:11.406047 7f095b379700 0 cephx: verify_reply couldn't
>>> > decrypt with error: error decoding block for decryption
>>> > 2017-01-07 16:55:11.406053 7f095b379700 0 -- 10.10.10.138:6789/0 >>
>>> > 10.10.10.252:6789/0 pipe(0x55cf8d028000 sd=11 :47548 s=1 pgs=0 cs=0
>>> l=0
>>> > c=0x55cf8ce28f00).failed verifying authorize reply
>>> >
>>> >
>>> >
>>> > Two monitors are up just fine, just trying to add a third and a quorum
>>> > cannot be met. NTP is running and no iptables running at all on
>>> internal
>>> > cluster.
>>> >
>>> >
>>> > Thank you.
>>> > -Alex
>>> >
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > ceph-users mailing list
>>> > [email protected]
>>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>> >
>>>
>>
>>
>
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
