Yeah, some of the message formats changed (incompatibly) during development. If you update all your nodes it should go away; that one I think is just ephemeral state.
On Tue, Jul 18, 2017 at 3:09 AM Marc Roos <[email protected]> wrote: > > I just updated packages on one CentOS7 node and getting these errors: > > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.537510 7f4fa1c14e40 -1 > WARNING: the following dangerous and experimental features are enabled: > bluestore > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.537510 7f4fa1c14e40 -1 > WARNING: the following dangerous and experimental features are enabled: > bluestore > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.537725 7f4fa1c14e40 -1 > WARNING: the following dangerous and experimental features are enabled: > bluestore > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.537725 7f4fa1c14e40 -1 > WARNING: the following dangerous and experimental features are enabled: > bluestore > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.567250 7f4fa1c14e40 -1 > WARNING: the following dangerous and experimental features are enabled: > bluestore > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.567250 7f4fa1c14e40 -1 > WARNING: the following dangerous and experimental features are enabled: > bluestore > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.589008 7f4fa1c14e40 -1 > mon.a@-1(probing).mgrstat failed to decode mgrstat state; luminous dev > version? > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.589008 7f4fa1c14e40 -1 > mon.a@-1(probing).mgrstat failed to decode mgrstat state; luminous dev > version? > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.724836 7f4f977d9700 -1 > mon.a@0(synchronizing).mgrstat failed to decode mgrstat state; luminous > dev version? > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.724836 7f4f977d9700 -1 > mon.a@0(synchronizing).mgrstat failed to decode mgrstat state; luminous > dev version? > Jul 18 12:03:34 c01 ceph-mon: > /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARC > H/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/1 > 2.1.1/rpm/el7/BUILD/ceph-12.1.1/src/messages/MForward.h: In function > 'PaxosServiceMessage* MForward::claim_message()' thread 7f4f977d9700 > time 2017-07-18 12:03:34.870230 > Jul 18 12:03:34 c01 ceph-mon: > /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARC > H/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/1 > 2.1.1/rpm/el7/BUILD/ceph-12.1.1/src/messages/MForward.h: 100: FAILED > assert(msg) > Jul 18 12:03:34 c01 ceph-mon: > /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARC > H/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/1 > 2.1.1/rpm/el7/BUILD/ceph-12.1.1/src/messages/MForward.h: In function > 'PaxosServiceMessage* MForward::claim_message()' thread 7f4f977d9700 > time 2017-07-18 12:03:34.870230 > Jul 18 12:03:34 c01 ceph-mon: > /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARC > H/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/1 > 2.1.1/rpm/el7/BUILD/ceph-12.1.1/src/messages/MForward.h: 100: FAILED > assert(msg) > Jul 18 12:03:34 c01 ceph-mon: ceph version 12.1.1 > (f3e663a190bf2ed12c7e3cda288b9a159572c800) luminous (rc) > Jul 18 12:03:34 c01 ceph-mon: 1: (ceph::__ceph_assert_fail(char const*, > char const*, int, char const*)+0x110) [0x7f4fa21f4310] > Jul 18 12:03:34 c01 ceph-mon: 2: > (Monitor::handle_forward(boost::intrusive_ptr<MonOpRequest>)+0xd70) > [0x7f4fa1fddcd0] > Jul 18 12:03:34 c01 ceph-mon: 3: > (Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)+0xd8d) > [0x7f4fa1fdb29d] > Jul 18 12:03:34 c01 ceph-mon: 4: (Monitor::_ms_dispatch(Message*)+0x7de) > [0x7f4fa1fdc06e] > Jul 18 12:03:34 c01 ceph-mon: 5: (Monitor::ms_dispatch(Message*)+0x23) > [0x7f4fa2004303] > Jul 18 12:03:34 c01 ceph-mon: 6: (DispatchQueue::entry()+0x792) > [0x7f4fa242c812] > Jul 18 12:03:34 c01 ceph-mon: 7: > (DispatchQueue::DispatchThread::entry()+0xd) [0x7f4fa229a3cd] > Jul 18 12:03:34 c01 ceph-mon: 8: (()+0x7dc5) [0x7f4fa0fbedc5] > Jul 18 12:03:34 c01 ceph-mon: 9: (clone()+0x6d) [0x7f4f9e34a76d] > Jul 18 12:03:34 c01 ceph-mon: NOTE: a copy of the executable, or > `objdump -rdS <executable>` is needed to interpret this. > Jul 18 12:03:34 c01 ceph-mon: 2017-07-18 12:03:34.872654 7f4f977d9700 -1 > /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARC > H/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/1 > 2.1.1/rpm/el7/BUILD/ceph-12.1.1/src/messages/MForward.h: In function > 'PaxosServiceMessage* MForward::claim_message()' thread 7f4f977d9700 > time 2017-07-18 12:03:34.870230 > Jul 18 12:03:34 c01 ceph-mon: ceph version 12.1.1 > (f3e663a190bf2ed12c7e3cda288b9a159572c800) luminous (rc) > Jul 18 12:03:34 c01 ceph-mon: 1: (ceph::__ceph_assert_fail(char const*, > char const*, int, char const*)+0x110) [0x7f4fa21f4310] > Jul 18 12:03:34 c01 ceph-mon: 2: > (Monitor::handle_forward(boost::intrusive_ptr<MonOpRequest>)+0xd70) > [0x7f4fa1fddcd0] > Jul 18 12:03:34 c01 ceph-mon: 3: > (Monitor::dispatch_op(boost::intrusive_ptr<MonOpRequest>)+0xd8d) > [0x7f4fa1fdb29d] > Jul 18 12:03:34 c01 ceph-mon: 4: (Monitor::_ms_dispatch(Message*)+0x7de) > [0x7f4fa1fdc06e] > Jul 18 12:03:34 c01 ceph-mon: 5: (Monitor::ms_dispatch(Message*)+0x23) > [0x7f4fa2004303] > Jul 18 12:03:34 c01 ceph-mon: 6: (DispatchQueue::entry()+0x792) > [0x7f4fa242c812] > Jul 18 12:03:34 c01 ceph-mon: 7: > (DispatchQueue::DispatchThread::entry()+0xd) [0x7f4fa229a3cd] > Jul 18 12:03:34 c01 ceph-mon: 8: (()+0x7dc5) [0x7f4fa0fbedc5] > Jul 18 12:03:34 c01 ceph-mon: 9: (clone()+0x6d) [0x7f4f9e34a76d] > Jul 18 12:03:34 c01 ceph-mon: NOTE: a copy of the executable, or > `objdump -rdS <executable>` is needed to interpret this. > > > > > > Packages Altered: > Updated ceph-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated ceph-base-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated ceph-common-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated ceph-mds-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated ceph-mgr-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated ceph-mon-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated ceph-osd-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated ceph-selinux-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated libcephfs2-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated librados2-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated libradosstriper1-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated librbd1-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated librgw2-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated python-cephfs-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated python-rados-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated python-rbd-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > Updated python-rgw-1:12.1.0-0.el7.x86_64 @ceph > Update 2:12.1.1-0.el7.x86_64 @ceph > > > > > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
