Dear Ceph folks,
I encountered a strange behavior with Luminous 12.2.13, when running the
following
ceph-osd -i ${osd_id} --mkfs -k
/var/lib/ceph/osd/ceph-${osd_id}/keyring
to create OSD,
what could be the root cause? Any ideas or suggestions?
best regards,
Samuel
*******************************************************Error log messages as
below****************************************************************
+ ceph-osd -i 3 --mkkey
+ ceph-osd -i 3 --mkfs -k /var/lib/ceph/osd/ceph-3/keyring --osd-uuid
b9e67369-1d36-4d1d-9f81-1aa679d4c5bf
/var/lib/ceph-bin/cephosd.sh: line 451: 33918 Aborted (core
dumped) ceph-osd -i "${OSD_ID}" --mkfs -k "${OSD_DIR}"/keyring --osd-uuid
"${OSD_UUID}" > ${log_file} 2>&1
+ '[' 134 -eq 0 ']'
+ echo osd mkfs fail
osd mkfs fail
------------------------------------------------ ${log_file}其中挂起的相关信息：
2022-03-05 05:53:40.450234 7f7368465f80 -1 WARNING: the following dangerous and
experimental features are enabled: bluestore,rocksdb
2022-03-05 05:53:40.450356 7f7368465f80 -1 WARNING: the following dangerous and
experimental features are enabled: bluestore,rocksdb
2022-03-05 05:53:40.451535 7f7368465f80 -1 WARNING: the following dangerous and
experimental features are enabled: bluestore,rocksdb
2022-03-05 05:53:40.452372 7f7368465f80 -1
bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open
/var/lib/ceph/osd/ceph-1/block: (2) No such file or directory
2022-03-05 05:53:40.452390 7f7368465f80 -1
bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open
/var/lib/ceph/osd/ceph-1/block: (2) No such file or directory
2022-03-05 05:53:40.452395 7f7368465f80 -1
bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open
/var/lib/ceph/osd/ceph-1/block: (2) No such file or directory
2022-03-05 05:53:40.460573 7f7368465f80 -1 bluestore(/var/lib/ceph/osd/ceph-1)
_read_fsid unparsable uuid
/build/ceph-12.2.13/src/os/bluestore/fastbmap_allocator_impl.h: In function
'void AllocatorLevel02<T>::_mark_allocated(uint64_t, uint64_t) [with L1 =
AllocatorLevel01Loose; uint64_t = long unsigned int]' thread 7f7368465f80 time
2022-03-05 05:53:40.486908
/build/ceph-12.2.13/src/os/bluestore/fastbmap_allocator_impl.h: 757: FAILED
assert(available >= allocated)
ceph version 12.2.13 (584a20eb0237c657dc0567da126be145106aa47e) luminous
(stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x102) [0x559be313d662]
2: (BitmapAllocator::init_rm_free(unsigned long, unsigned long)+0x685)
[0x559be30ec865]
3: (BlueFS::mount()+0x3d1) [0x559be30b4bc1]
4: (BlueStore::_open_db(bool)+0x16ac) [0x559be2fc157c]
5: (BlueStore::mkfs()+0x1225) [0x559be2ff7215]
6: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x164)
[0x559be2b1da24]
7: (main()+0x11aa) [0x559be2a4424a]
8: (__libc_start_main()+0xf0) [0x7f73656cb840]
9: (_start()+0x29) [0x559be2ad3339]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to
interpret this.
2022-03-05 05:53:40.489602 7f7368465f80 -1
/build/ceph-12.2.13/src/os/bluestore/fastbmap_allocator_impl.h: In function
'void AllocatorLevel02<T>::_mark_allocated(uint64_t, uint64_t) [with L1 =
AllocatorLevel01Loose; uint64_t = long unsigned int]' thread 7f7368465f80 time
2022-03-05 05:53:40.486908
/build/ceph-12.2.13/src/os/bluestore/fastbmap_allocator_impl.h: 757: FAILED
assert(available >= allocated)
ceph version 12.2.13 (584a20eb0237c657dc0567da126be145106aa47e) luminous
(stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x102) [0x559be313d662]
2: (BitmapAllocator::init_rm_free(unsigned long, unsigned long)+0x685)
[0x559be30ec865]
3: (BlueFS::mount()+0x3d1) [0x559be30b4bc1]
4: (BlueStore::_open_db(bool)+0x16ac) [0x559be2fc157c]
5: (BlueStore::mkfs()+0x1225) [0x559be2ff7215]
6: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x164)
[0x559be2b1da24]
7: (main()+0x11aa) [0x559be2a4424a]
8: (__libc_start_main()+0xf0) [0x7f73656cb840]
9: (_start()+0x29) [0x559be2ad3339]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to
interpret this.
-17> 2022-03-05 05:53:40.450234 7f7368465f80 -1 WARNING: the following
dangerous and experimental features are enabled: bluestore,rocksdb
-16> 2022-03-05 05:53:40.450356 7f7368465f80 -1 WARNING: the following
dangerous and experimental features are enabled: bluestore,rocksdb
-14> 2022-03-05 05:53:40.451535 7f7368465f80 -1 WARNING: the following
dangerous and experimental features are enabled: bluestore,rocksdb
-13> 2022-03-05 05:53:40.452372 7f7368465f80 -1
bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open
/var/lib/ceph/osd/ceph-1/block: (2) No such file or directory
-12> 2022-03-05 05:53:40.452390 7f7368465f80 -1
bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open
/var/lib/ceph/osd/ceph-1/block: (2) No such file or directory
-11> 2022-03-05 05:53:40.452395 7f7368465f80 -1
bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open
/var/lib/ceph/osd/ceph-1/block: (2) No such file or directory
-10> 2022-03-05 05:53:40.460573 7f7368465f80 -1
bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid
0> 2022-03-05 05:53:40.489602 7f7368465f80 -1
/build/ceph-12.2.13/src/os/bluestore/fastbmap_allocator_impl.h: In function
'void AllocatorLevel02<T>::_mark_allocated(uint64_t, uint64_t) [with L1 =
AllocatorLevel01Loose; uint64_t = long unsigned int]' thread 7f7368465f80 time
2022-03-05 05:53:40.486908
/build/ceph-12.2.13/src/os/bluestore/fastbmap_allocator_impl.h: 757: FAILED
assert(available >= allocated)
ceph version 12.2.13 (584a20eb0237c657dc0567da126be145106aa47e) luminous
(stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x102) [0x559be313d662]
2: (BitmapAllocator::init_rm_free(unsigned long, unsigned long)+0x685)
[0x559be30ec865]
3: (BlueFS::mount()+0x3d1) [0x559be30b4bc1]
4: (BlueStore::_open_db(bool)+0x16ac) [0x559be2fc157c]
5: (BlueStore::mkfs()+0x1225) [0x559be2ff7215]
6: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x164)
[0x559be2b1da24]
7: (main()+0x11aa) [0x559be2a4424a]
8: (__libc_start_main()+0xf0) [0x7f73656cb840]
9: (_start()+0x29) [0x559be2ad3339]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to
interpret this.
*** Caught signal (Aborted) **
in thread 7f7368465f80 thread_name:ceph-osd
ceph version 12.2.13 (584a20eb0237c657dc0567da126be145106aa47e) luminous
(stable)
1: (()+0xab3b64) [0x559be30f9b64]
2: (()+0x11390) [0x7f7366745390]
3: (gsignal()+0x38) [0x7f73656e0438]
4: (abort()+0x16a) [0x7f73656e203a]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x28e) [0x559be313d7ee]
6: (BitmapAllocator::init_rm_free(unsigned long, unsigned long)+0x685)
[0x559be30ec865]
7: (BlueFS::mount()+0x3d1) [0x559be30b4bc1]
8: (BlueStore::_open_db(bool)+0x16ac) [0x559be2fc157c]
9: (BlueStore::mkfs()+0x1225) [0x559be2ff7215]
10: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x164)
[0x559be2b1da24]
11: (main()+0x11aa) [0x559be2a4424a]
12: (__libc_start_main()+0xf0) [0x7f73656cb840]
13: (_start()+0x29) [0x559be2ad3339]
2022-03-05 05:53:40.492413 7f7368465f80 -1 *** Caught signal (Aborted) **
in thread 7f7368465f80 thread_name:ceph-osd
[email protected]
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
