Hi
Here you go
Br,
Tuomas
-----Original Message-----
From: Sage Weil [mailto:[email protected]]
Sent: 27. huhtikuuta 2015 19:23
To: Tuomas Juntunen
Cc: 'Samuel Just'; [email protected]
Subject: Re: [ceph-users] Upgrade from Giant to Hammer and after some basic
operations most of the OSD's went down
On Mon, 27 Apr 2015, Tuomas Juntunen wrote:
> Thanks for the info.
>
> For my knowledge there was no snapshots on that pool, but cannot
> verify that.
Can you attach a 'ceph osd dump -f json-pretty'? That will shed a bit more
light on what happened (and the simplest way to fix it).
sage
> Any way to make this work again? Removing the tier and other settings
> didn't fix it, I tried it the second this happened.
>
> Br,
> Tuomas
>
> -----Original Message-----
> From: Samuel Just [mailto:[email protected]]
> Sent: 27. huhtikuuta 2015 15:50
> To: tuomas juntunen
> Cc: [email protected]
> Subject: Re: [ceph-users] Upgrade from Giant to Hammer and after some
> basic operations most of the OSD's went down
>
> So, the base tier is what determines the snapshots for the cache/base pool
amalgam. You added a populated pool complete with snapshots on top of a
base tier without snapshots. Apparently, it caused an existential crisis
for the snapshot code. That's one of the reasons why there is a
--force-nonempty flag for that operation, I think. I think the immediate
answer is probably to disallow pools with snapshots as a cache tier
altogether until we think of a good way to make it work.
> -Sam
>
> ----- Original Message -----
> From: "tuomas juntunen" <[email protected]>
> To: "Samuel Just" <[email protected]>
> Cc: [email protected]
> Sent: Monday, April 27, 2015 4:56:58 AM
> Subject: Re: [ceph-users] Upgrade from Giant to Hammer and after some
> basic operations most of the OSD's went down
>
>
>
> The following:
>
> ceph osd tier add img images --force-nonempty ceph osd tier cache-mode
> images forward ceph osd tier set-overlay img images
>
> Idea was to make images as a tier to img, move data to img then change
clients to use the new img pool.
>
> Br,
> Tuomas
>
> > Can you explain exactly what you mean by:
> >
> > "Also I created one pool for tier to be able to move data without
outage."
> >
> > -Sam
> > ----- Original Message -----
> > From: "tuomas juntunen" <[email protected]>
> > To: "Ian Colle" <[email protected]>
> > Cc: [email protected]
> > Sent: Monday, April 27, 2015 4:23:44 AM
> > Subject: Re: [ceph-users] Upgrade from Giant to Hammer and after
> > some basic operations most of the OSD's went down
> >
> > Hi
> >
> > Any solution for this yet?
> >
> > Br,
> > Tuomas
> >
> >> It looks like you may have hit http://tracker.ceph.com/issues/7915
> >>
> >> Ian R. Colle
> >> Global Director
> >> of Software Engineering
> >> Red Hat (Inktank is now part of Red Hat!)
> >> http://www.linkedin.com/in/ircolle
> >> http://www.twitter.com/ircolle
> >> Cell: +1.303.601.7713
> >> Email: [email protected]
> >>
> >> ----- Original Message -----
> >> From: "tuomas juntunen" <[email protected]>
> >> To: [email protected]
> >> Sent: Monday, April 27, 2015 1:56:29 PM
> >> Subject: [ceph-users] Upgrade from Giant to Hammer and after some
> >> basic operations most of the OSD's went down
> >>
> >>
> >>
> >> I upgraded Ceph from 0.87 Giant to 0.94.1 Hammer
> >>
> >> Then created new pools and deleted some old ones. Also I created
> >> one pool for tier to be able to move data without outage.
> >>
> >> After these operations all but 10 OSD's are down and creating this
> >> kind of messages to logs, I get more than 100gb of these in a night:
> >>
> >> -19> 2015-04-27 10:17:08.808584 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> >> les/c
> >> 16609/16659
> >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> >> crt=8480'7 lcod
> >> 0'0 inactive NOTIFY] enter Started
> >> -18> 2015-04-27 10:17:08.808596 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> >> les/c
> >> 16609/16659
> >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> >> crt=8480'7 lcod
> >> 0'0 inactive NOTIFY] enter Start
> >> -17> 2015-04-27 10:17:08.808608 7fd8e748d700 1 osd.23 pg_epoch:
> >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> >> les/c
> >> 16609/16659
> >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> >> crt=8480'7 lcod
> >> 0'0 inactive NOTIFY] state<Start>: transitioning to Stray
> >> -16> 2015-04-27 10:17:08.808621 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> >> les/c
> >> 16609/16659
> >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> >> crt=8480'7 lcod
> >> 0'0 inactive NOTIFY] exit Start 0.000025 0 0.000000
> >> -15> 2015-04-27 10:17:08.808637 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[0.189( v 8480'7 (0'0,8480'7] local-les=16609 n=0 ec=1
> >> les/c
> >> 16609/16659
> >> 16590/16590/16590) [24,3,23] r=2 lpr=17838 pi=15659-16589/42
> >> crt=8480'7 lcod
> >> 0'0 inactive NOTIFY] enter Started/Stray
> >> -14> 2015-04-27 10:17:08.808796 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> >> 17879/17879
> >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive NOTIFY]
> >> exit Reset 0.119467 4 0.000037
> >> -13> 2015-04-27 10:17:08.808817 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> >> 17879/17879
> >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive NOTIFY]
> >> enter Started
> >> -12> 2015-04-27 10:17:08.808828 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> >> 17879/17879
> >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive NOTIFY]
> >> enter Start
> >> -11> 2015-04-27 10:17:08.808838 7fd8e748d700 1 osd.23 pg_epoch:
> >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> >> 17879/17879
> >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive NOTIFY]
> >> state<Start>: transitioning to Stray
> >> -10> 2015-04-27 10:17:08.808849 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> >> 17879/17879
> >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive NOTIFY]
> >> exit Start 0.000020 0 0.000000
> >> -9> 2015-04-27 10:17:08.808861 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[10.181( empty local-les=17879 n=0 ec=17863 les/c
> >> 17879/17879
> >> 17863/17863/17863) [25,5,23] r=2 lpr=17879 crt=0'0 inactive NOTIFY]
> >> enter Started/Stray
> >> -8> 2015-04-27 10:17:08.809427 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 inactive]
> >> exit Reset 7.511623 45 0.000165
> >> -7> 2015-04-27 10:17:08.809445 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 inactive]
> >> enter Started
> >> -6> 2015-04-27 10:17:08.809456 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 inactive]
> >> enter Start
> >> -5> 2015-04-27 10:17:08.809468 7fd8e748d700 1 osd.23 pg_epoch:
> >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 inactive]
> >> state<Start>: transitioning to Primary
> >> -4> 2015-04-27 10:17:08.809479 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 inactive]
> >> exit Start 0.000023 0 0.000000
> >> -3> 2015-04-27 10:17:08.809492 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 inactive]
> >> enter Started/Primary
> >> -2> 2015-04-27 10:17:08.809502 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 inactive]
> >> enter Started/Primary/Peering
> >> -1> 2015-04-27 10:17:08.809513 7fd8e748d700 5 osd.23 pg_epoch:
> >> 17882 pg[2.189( empty local-les=16127 n=0 ec=1 les/c 16127/16344
> >> 16125/16125/16125) [23,5] r=0 lpr=17838 crt=0'0 mlcod 0'0 peering]
> >> enter Started/Primary/Peering/GetInfo
> >> 0> 2015-04-27 10:17:08.813837 7fd8e748d700 -1
./include/interval_set.h:
> >> In
> >> function 'void interval_set<T>::erase(T, T) [with T = snapid_t]'
> >> thread
> >> 7fd8e748d700 time 2015-04-27 10:17:08.809899
> >> ./include/interval_set.h: 385: FAILED assert(_size >= 0)
> >>
> >> ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff)
> >> 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char
> >> const*)+0x8b)
> >> [0xbc271b]
> >> 2: (interval_set<snapid_t>::subtract(interval_set<snapid_t>
> >> const&)+0xb0) [0x82cd50]
> >> 3: (PGPool::update(std::tr1::shared_ptr<OSDMap const>)+0x52e)
> >> [0x80113e]
> >> 4: (PG::handle_advance_map(std::tr1::shared_ptr<OSDMap const>,
> >> std::tr1::shared_ptr<OSDMap const>, std::vector<int,
> >> std::allocator<int> >&, int, std::vector<int, std::allocator<int>
> >> >&, int, PG::RecoveryCtx*)+0x282) [0x801652]
> >> 5: (OSD::advance_pg(unsigned int, PG*, ThreadPool::TPHandle&,
> >> PG::RecoveryCtx*, std::set<boost::intrusive_ptr<PG>,
> >> std::less<boost::intrusive_ptr<PG> >,
> >> std::allocator<boost::intrusive_ptr<PG> > >*)+0x2c3) [0x6b0e43]
> >> 6: (OSD::process_peering_events(std::list<PG*, std::allocator<PG*>
> >> > const&,
> >> ThreadPool::TPHandle&)+0x21c) [0x6b191c]
> >> 7: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*> >
> >> const&,
> >> ThreadPool::TPHandle&)+0x18) [0x709278]
> >> 8: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa5e) [0xbb38ae]
> >> 9: (ThreadPool::WorkThread::entry()+0x10) [0xbb4950]
> >> 10: (()+0x8182) [0x7fd906946182]
> >> 11: (clone()+0x6d) [0x7fd904eb147d]
> >>
> >> Also by monitoring (ceph -w) I get the following messages, also lots of
them.
> >>
> >> 2015-04-27 10:39:52.935812 mon.0 [INF] from='client.?
10.20.0.13:0/1174409'
> >> entity='osd.30' cmd=[{"prefix": "osd crush create-or-move", "args":
> >> ["host=ceph3", "root=default"], "id": 30, "weight": 1.82}]:
> >> dispatch
> >> 2015-04-27 10:39:53.297376 mon.0 [INF] from='client.?
10.20.0.13:0/1174483'
> >> entity='osd.26' cmd=[{"prefix": "osd crush create-or-move", "args":
> >> ["host=ceph3", "root=default"], "id": 26, "weight": 1.82}]:
> >> dispatch
> >>
> >>
> >> This is a cluster of 3 nodes with 36 OSD's, nodes are also mons and
> >> mds's to save servers. All run Ubuntu 14.04.2.
> >>
> >> I have pretty much tried everything I could think of.
> >>
> >> Restarting daemons doesn't help.
> >>
> >> Any help would be appreciated. I can also provide more logs if
> >> necessary. They just seem to get pretty large in few moments.
> >>
> >> Thank you
> >> Tuomas
> >>
> >>
> >> _______________________________________________
> >> ceph-users mailing list
> >> [email protected]
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>
> >>
> >>
> >
> >
> > _______________________________________________
> > ceph-users mailing list
> > [email protected]
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> >
> >
>
>
> _______________________________________________
> ceph-users mailing list
> [email protected]
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
>
> _______________________________________________
> ceph-users mailing list
> [email protected]
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
{
"epoch": 18128,
"fsid": "a2974742-3805-4cd3-bc79-765f2bddaefe",
"created": "2014-10-15 20:43:45.186949",
"modified": "2015-04-27 17:00:13.746815",
"flags": "",
"cluster_snapshot": "",
"pool_max": 17,
"max_osd": 71,
"pools": [
{
"pool": 0,
"pool_name": "data",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 45,
"last_change": "1112",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 0,
"cache_target_full_ratio_micro": 0,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 1,
"pool_name": "metadata",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "1114",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 0,
"cache_target_full_ratio_micro": 0,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 2,
"pool_name": "rbd",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 2,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "1116",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 0,
"cache_target_full_ratio_micro": 0,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 3,
"pool_name": "volumes",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "9974",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 23,
"snap_epoch": 9974,
"pool_snaps": [],
"removed_snaps": "[1~17]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "default",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 4,
"pool_name": "images",
"flags": 9,
"flags_names": "hashpspool,incomplete_clones",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "17905",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 17882,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 0,
"cache_target_full_ratio_micro": 0,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "default",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 6,
"pool_name": "vms",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "1122",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "default",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 1,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 7,
"pool_name": "san",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 1,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 4096,
"pg_placement_num": 4096,
"crash_replay_interval": 0,
"last_change": "14096",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 8,
"pool_name": "vol-ssd-accelerated",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 1024,
"pg_placement_num": 1024,
"crash_replay_interval": 0,
"last_change": "17861",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 14,
"pool_name": "backup",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 128,
"pg_placement_num": 128,
"crash_replay_interval": 0,
"last_change": "18018",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 15,
"pool_name": "img",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 256,
"pg_placement_num": 256,
"crash_replay_interval": 0,
"last_change": "18019",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 16,
"pool_name": "vm",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 1024,
"pg_placement_num": 1024,
"crash_replay_interval": 0,
"last_change": "18020",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
},
{
"pool": 17,
"pool_name": "infradisks",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_ruleset": 0,
"object_hash": 2,
"pg_num": 256,
"pg_placement_num": 256,
"crash_replay_interval": 0,
"last_change": "18021",
"last_force_op_resend": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"min_read_recency_for_promote": 0,
"stripe_width": 0,
"expected_num_objects": 0
}
],
"osds": [
{
"osd": 0,
"uuid": "757c3bc5-4d00-4344-8de4-82f5379c96af",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15738,
"last_clean_end": 16079,
"up_from": 16081,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6981\/245797",
"cluster_addr": "10.20.0.11:6881\/5245797",
"heartbeat_back_addr": "10.20.0.11:6883\/5245797",
"heartbeat_front_addr": "10.20.0.11:6887\/5245797",
"state": [
"autoout",
"exists"
]
},
{
"osd": 1,
"uuid": "c7eaa4ac-99fc-46db-84aa-a67274896ec8",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15740,
"last_clean_end": 16079,
"up_from": 16083,
"up_thru": 17873,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6813\/245273",
"cluster_addr": "10.20.0.11:6888\/4245273",
"heartbeat_back_addr": "10.20.0.11:6890\/4245273",
"heartbeat_front_addr": "10.20.0.11:6894\/4245273",
"state": [
"autoout",
"exists"
]
},
{
"osd": 2,
"uuid": "206b2949-4adf-4789-8e06-f68a8ee819c9",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15739,
"last_clean_end": 16079,
"up_from": 16081,
"up_thru": 17874,
"down_at": 17915,
"lost_at": 0,
"public_addr": "10.20.0.11:6808\/245269",
"cluster_addr": "10.20.0.11:6811\/4245269",
"heartbeat_back_addr": "10.20.0.11:6823\/4245269",
"heartbeat_front_addr": "10.20.0.11:6831\/4245269",
"state": [
"autoout",
"exists"
]
},
{
"osd": 3,
"uuid": "90b7c219-4dcd-48ea-a24d-f3b796a521e4",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15736,
"last_clean_end": 16125,
"up_from": 16128,
"up_thru": 17873,
"down_at": 17920,
"lost_at": 0,
"public_addr": "10.20.0.11:6848\/245357",
"cluster_addr": "10.20.0.11:6886\/6245357",
"heartbeat_back_addr": "10.20.0.11:6891\/6245357",
"heartbeat_front_addr": "10.20.0.11:6892\/6245357",
"state": [
"autoout",
"exists"
]
},
{
"osd": 4,
"uuid": "049ef94f-121a-4e71-8ba6-27eaebf0a569",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15737,
"last_clean_end": 16123,
"up_from": 16125,
"up_thru": 17871,
"down_at": 17925,
"lost_at": 0,
"public_addr": "10.20.0.11:6854\/245375",
"cluster_addr": "10.20.0.11:6849\/5245375",
"heartbeat_back_addr": "10.20.0.11:6850\/5245375",
"heartbeat_front_addr": "10.20.0.11:6853\/5245375",
"state": [
"autoout",
"exists"
]
},
{
"osd": 5,
"uuid": "2437a53b-339e-45af-b4de-0fc675d27405",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15734,
"last_clean_end": 16087,
"up_from": 16089,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6834\/245323",
"cluster_addr": "10.20.0.11:6803\/4245323",
"heartbeat_back_addr": "10.20.0.11:6804\/4245323",
"heartbeat_front_addr": "10.20.0.11:6805\/4245323",
"state": [
"autoout",
"exists"
]
},
{
"osd": 6,
"uuid": "f117ceed-b1fd-4069-99fe-b7aba9f3ef8d",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15738,
"last_clean_end": 16079,
"up_from": 16083,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6859\/245401",
"cluster_addr": "10.20.0.11:6865\/5245401",
"heartbeat_back_addr": "10.20.0.11:6868\/5245401",
"heartbeat_front_addr": "10.20.0.11:6874\/5245401",
"state": [
"autoout",
"exists"
]
},
{
"osd": 7,
"uuid": "e98e9b8a-9c62-4e3e-bdb4-c2c30103c0c1",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15730,
"last_clean_end": 16123,
"up_from": 16125,
"up_thru": 17872,
"down_at": 17925,
"lost_at": 0,
"public_addr": "10.20.0.11:6801\/245246",
"cluster_addr": "10.20.0.11:6812\/5245246",
"heartbeat_back_addr": "10.20.0.11:6814\/5245246",
"heartbeat_front_addr": "10.20.0.11:6821\/5245246",
"state": [
"autoout",
"exists"
]
},
{
"osd": 8,
"uuid": "41e471cd-fafe-4422-8bf5-22018bbe1375",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15736,
"last_clean_end": 17783,
"up_from": 17795,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6810\/458698",
"cluster_addr": "10.20.0.11:6819\/458698",
"heartbeat_back_addr": "10.20.0.11:6826\/458698",
"heartbeat_front_addr": "10.20.0.11:6833\/458698",
"state": [
"autoout",
"exists"
]
},
{
"osd": 9,
"uuid": "d68eeebd-d058-4b1c-a30a-994bf8fc8030",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15733,
"last_clean_end": 16079,
"up_from": 16081,
"up_thru": 17873,
"down_at": 17930,
"lost_at": 0,
"public_addr": "10.20.0.11:6969\/245600",
"cluster_addr": "10.20.0.11:6847\/5245600",
"heartbeat_back_addr": "10.20.0.11:6860\/5245600",
"heartbeat_front_addr": "10.20.0.11:6864\/5245600",
"state": [
"autoout",
"exists"
]
},
{
"osd": 10,
"uuid": "660747d6-3f47-449a-bc69-5399b0d54ff6",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15736,
"last_clean_end": 16124,
"up_from": 16128,
"up_thru": 17875,
"down_at": 17935,
"lost_at": 0,
"public_addr": "10.20.0.11:6822\/245281",
"cluster_addr": "10.20.0.11:6855\/4245281",
"heartbeat_back_addr": "10.20.0.11:6856\/4245281",
"heartbeat_front_addr": "10.20.0.11:6866\/4245281",
"state": [
"autoout",
"exists"
]
},
{
"osd": 11,
"uuid": "805965b1-127f-44a6-9a05-8a643eb7a512",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15737,
"last_clean_end": 17786,
"up_from": 17801,
"up_thru": 17870,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.11:6835\/458813",
"cluster_addr": "10.20.0.11:6836\/458813",
"heartbeat_back_addr": "10.20.0.11:6837\/458813",
"heartbeat_front_addr": "10.20.0.11:6838\/458813",
"state": [
"autoout",
"exists"
]
},
{
"osd": 12,
"uuid": "61fbfcbe-d642-478f-9620-f9d72ee96238",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16126,
"up_from": 16129,
"up_thru": 17872,
"down_at": 17935,
"lost_at": 0,
"public_addr": "10.20.0.12:6873\/7542",
"cluster_addr": "10.20.0.12:6947\/4007542",
"heartbeat_back_addr": "10.20.0.12:6948\/4007542",
"heartbeat_front_addr": "10.20.0.12:6949\/4007542",
"state": [
"autoout",
"exists"
]
},
{
"osd": 13,
"uuid": "6faad33b-00be-42a4-92ba-08be5ab7f995",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16705,
"up_from": 16707,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6821\/7274",
"cluster_addr": "10.20.0.12:6859\/5007274",
"heartbeat_back_addr": "10.20.0.12:6860\/5007274",
"heartbeat_front_addr": "10.20.0.12:6861\/5007274",
"state": [
"autoout",
"exists"
]
},
{
"osd": 14,
"uuid": "f301705b-e725-443d-96e1-d9ec9aafe657",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16962,
"up_from": 16966,
"up_thru": 17871,
"down_at": 17940,
"lost_at": 0,
"public_addr": "10.20.0.12:6809\/7209",
"cluster_addr": "10.20.0.12:6803\/7007209",
"heartbeat_back_addr": "10.20.0.12:6804\/7007209",
"heartbeat_front_addr": "10.20.0.12:6810\/7007209",
"state": [
"autoout",
"exists"
]
},
{
"osd": 15,
"uuid": "536bf483-10de-44b0-8e1e-4f349fbe572a",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15785,
"last_clean_end": 16700,
"up_from": 16715,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6881\/7613",
"cluster_addr": "10.20.0.12:6845\/7007613",
"heartbeat_back_addr": "10.20.0.12:6849\/7007613",
"heartbeat_front_addr": "10.20.0.12:6850\/7007613",
"state": [
"autoout",
"exists"
]
},
{
"osd": 16,
"uuid": "4185bd20-8eb0-4616-b36e-bacb181ae40e",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16688,
"up_from": 16693,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6813\/7210",
"cluster_addr": "10.20.0.12:6814\/7007210",
"heartbeat_back_addr": "10.20.0.12:6815\/7007210",
"heartbeat_front_addr": "10.20.0.12:6816\/7007210",
"state": [
"autoout",
"exists"
]
},
{
"osd": 17,
"uuid": "a6f2f5b4-477f-48f9-9acf-d5b7a6c88b98",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16123,
"up_from": 16125,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6817\/7215",
"cluster_addr": "10.20.0.12:6840\/3007215",
"heartbeat_back_addr": "10.20.0.12:6843\/3007215",
"heartbeat_front_addr": "10.20.0.12:6844\/3007215",
"state": [
"autoout",
"exists"
]
},
{
"osd": 18,
"uuid": "b31b0bd8-938a-496d-91bc-19bf4f794f82",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16700,
"up_from": 16712,
"up_thru": 17872,
"down_at": 17942,
"lost_at": 0,
"public_addr": "10.20.0.12:6805\/7185",
"cluster_addr": "10.20.0.12:6829\/5007185",
"heartbeat_back_addr": "10.20.0.12:6830\/5007185",
"heartbeat_front_addr": "10.20.0.12:6841\/5007185",
"state": [
"autoout",
"exists"
]
},
{
"osd": 19,
"uuid": "d76b6bd5-1ef3-436c-a75d-3587c515eb56",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16693,
"up_from": 16706,
"up_thru": 17872,
"down_at": 16693,
"lost_at": 0,
"public_addr": "10.20.0.12:6885\/7622",
"cluster_addr": "10.20.0.12:6802\/7007622",
"heartbeat_back_addr": "10.20.0.12:6818\/7007622",
"heartbeat_front_addr": "10.20.0.12:6819\/7007622",
"state": [
"exists",
"up"
]
},
{
"osd": 20,
"uuid": "8e4dd982-a4c5-4ca5-9fc5-243f55c4db57",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16703,
"up_from": 16705,
"up_thru": 17872,
"down_at": 16703,
"lost_at": 0,
"public_addr": "10.20.0.12:6825\/7316",
"cluster_addr": "10.20.0.12:6827\/5007316",
"heartbeat_back_addr": "10.20.0.12:6831\/5007316",
"heartbeat_front_addr": "10.20.0.12:6851\/5007316",
"state": [
"exists",
"up"
]
},
{
"osd": 21,
"uuid": "760aaf28-0a34-4bbc-af0c-2654b0a43fff",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16699,
"up_from": 16701,
"up_thru": 17870,
"down_at": 16698,
"lost_at": 0,
"public_addr": "10.20.0.12:6801\/7180",
"cluster_addr": "10.20.0.12:6820\/7007180",
"heartbeat_back_addr": "10.20.0.12:6822\/7007180",
"heartbeat_front_addr": "10.20.0.12:6828\/7007180",
"state": [
"exists",
"up"
]
},
{
"osd": 22,
"uuid": "40322a34-ab31-4760-b71e-a7672f812cb3",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16966,
"up_from": 16969,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6869\/7509",
"cluster_addr": "10.20.0.12:6824\/8007509",
"heartbeat_back_addr": "10.20.0.12:6832\/8007509",
"heartbeat_front_addr": "10.20.0.12:6834\/8007509",
"state": [
"autoout",
"exists"
]
},
{
"osd": 23,
"uuid": "e1d81949-f4b5-4cf2-b6af-dccaaeb30ed7",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15783,
"last_clean_end": 16123,
"up_from": 16125,
"up_thru": 17875,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.12:6877\/7592",
"cluster_addr": "10.20.0.12:6826\/4007592",
"heartbeat_back_addr": "10.20.0.12:6835\/4007592",
"heartbeat_front_addr": "10.20.0.12:6839\/4007592",
"state": [
"autoout",
"exists"
]
},
{
"osd": 24,
"uuid": "ede77283-a423-4c6b-9c6e-b0e807c63cb5",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15688,
"last_clean_end": 16109,
"up_from": 16112,
"up_thru": 17871,
"down_at": 16109,
"lost_at": 0,
"public_addr": "10.20.0.13:6921\/213483",
"cluster_addr": "10.20.0.13:6815\/3213483",
"heartbeat_back_addr": "10.20.0.13:6841\/3213483",
"heartbeat_front_addr": "10.20.0.13:6846\/3213483",
"state": [
"exists",
"up"
]
},
{
"osd": 25,
"uuid": "7cfe85f8-3ae9-493d-9801-025ff6c6265d",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15686,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6801\/212863",
"cluster_addr": "10.20.0.13:6918\/3212863",
"heartbeat_back_addr": "10.20.0.13:6932\/3212863",
"heartbeat_front_addr": "10.20.0.13:6934\/3212863",
"state": [
"exists",
"up"
]
},
{
"osd": 26,
"uuid": "266f6d70-519f-4c24-bca2-236495a600a7",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15692,
"last_clean_end": 16125,
"up_from": 16130,
"up_thru": 17872,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6917\/213469",
"cluster_addr": "10.20.0.13:6896\/4213469",
"heartbeat_back_addr": "10.20.0.13:6945\/4213469",
"heartbeat_front_addr": "10.20.0.13:6946\/4213469",
"state": [
"exists",
"up"
]
},
{
"osd": 27,
"uuid": "68644fa9-9459-4db0-a6c9-01661645038b",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15689,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6899\/213448",
"cluster_addr": "10.20.0.13:6876\/3213448",
"heartbeat_back_addr": "10.20.0.13:6902\/3213448",
"heartbeat_front_addr": "10.20.0.13:6904\/3213448",
"state": [
"exists",
"up"
]
},
{
"osd": 28,
"uuid": "fc3d5749-7673-4100-a0d4-f25e9cc0bc88",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15688,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6892\/213419",
"cluster_addr": "10.20.0.13:6866\/5213419",
"heartbeat_back_addr": "10.20.0.13:6867\/5213419",
"heartbeat_front_addr": "10.20.0.13:6873\/5213419",
"state": [
"exists",
"up"
]
},
{
"osd": 29,
"uuid": "cb5feda9-de3f-4e42-bb73-7945b4928b22",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15689,
"last_clean_end": 16109,
"up_from": 16113,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6937\/213527",
"cluster_addr": "10.20.0.13:6898\/2213527",
"heartbeat_back_addr": "10.20.0.13:6926\/2213527",
"heartbeat_front_addr": "10.20.0.13:6927\/2213527",
"state": [
"autoout",
"exists"
]
},
{
"osd": 30,
"uuid": "ef1e65bb-a634-4096-9466-1262af55db01",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15693,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6910\/213454",
"cluster_addr": "10.20.0.13:6823\/4213454",
"heartbeat_back_addr": "10.20.0.13:6838\/4213454",
"heartbeat_front_addr": "10.20.0.13:6852\/4213454",
"state": [
"autoout",
"exists"
]
},
{
"osd": 31,
"uuid": "3dad6393-67a8-43d4-ba8d-ffd320827396",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15688,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6929\/213515",
"cluster_addr": "10.20.0.13:6909\/4213515",
"heartbeat_back_addr": "10.20.0.13:6940\/4213515",
"heartbeat_front_addr": "10.20.0.13:6941\/4213515",
"state": [
"autoout",
"exists"
]
},
{
"osd": 32,
"uuid": "db6f3afa-53ed-453a-97e3-861e88cb818f",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15684,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17873,
"down_at": 16123,
"lost_at": 0,
"public_addr": "10.20.0.13:6884\/213261",
"cluster_addr": "10.20.0.13:6856\/3213261",
"heartbeat_back_addr": "10.20.0.13:6863\/3213261",
"heartbeat_front_addr": "10.20.0.13:6865\/3213261",
"state": [
"exists",
"up"
]
},
{
"osd": 33,
"uuid": "d5e59852-06b4-4a30-8c5e-ff7e328b5455",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15692,
"last_clean_end": 16125,
"up_from": 16130,
"up_thru": 17872,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6900\/213450",
"cluster_addr": "10.20.0.13:6936\/3213450",
"heartbeat_back_addr": "10.20.0.13:6938\/3213450",
"heartbeat_front_addr": "10.20.0.13:6939\/3213450",
"state": [
"autoout",
"exists"
]
},
{
"osd": 34,
"uuid": "f35a10c5-217a-4cfb-88b9-7334bda441b8",
"up": 0,
"in": 0,
"weight": 0.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15687,
"last_clean_end": 16125,
"up_from": 16127,
"up_thru": 17871,
"down_at": 17884,
"lost_at": 0,
"public_addr": "10.20.0.13:6831\/212981",
"cluster_addr": "10.20.0.13:6942\/2212981",
"heartbeat_back_addr": "10.20.0.13:6943\/2212981",
"heartbeat_front_addr": "10.20.0.13:6944\/2212981",
"state": [
"autoout",
"exists"
]
},
{
"osd": 35,
"uuid": "335e797f-a390-4f08-9da6-9ab76ffb12ae",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 15687,
"last_clean_end": 16109,
"up_from": 16112,
"up_thru": 17872,
"down_at": 16109,
"lost_at": 0,
"public_addr": "10.20.0.13:6877\/213258",
"cluster_addr": "10.20.0.13:6803\/2213258",
"heartbeat_back_addr": "10.20.0.13:6832\/2213258",
"heartbeat_front_addr": "10.20.0.13:6833\/2213258",
"state": [
"exists",
"up"
]
},
{
"osd": 36,
"uuid": "33c11fa1-1b03-42e4-8296-dc55ba052b35",
"up": 1,
"in": 1,
"weight": 1.000000,
"primary_affinity": 1.000000,
"last_clean_begin": 18112,
"last_clean_end": 18118,
"up_from": 18121,
"up_thru": 18121,
"down_at": 18120,
"lost_at": 0,
"public_addr": "10.20.0.12:6809\/3785178",
"cluster_addr": "10.20.0.12:6810\/3785178",
"heartbeat_back_addr": "10.20.0.12:6811\/3785178",
"heartbeat_front_addr": "10.20.0.12:6812\/3785178",
"state": [
"exists",
"up"
]
}
],
"osd_xinfo": [
{
"osd": 0,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.927101,
"laggy_interval": 46,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 1,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.896128,
"laggy_interval": 43,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 2,
"down_stamp": "2015-04-26 14:39:36.274440",
"laggy_probability": 0.882482,
"laggy_interval": 68,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 3,
"down_stamp": "2015-04-26 14:39:41.282378",
"laggy_probability": 0.943535,
"laggy_interval": 41,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 4,
"down_stamp": "2015-04-26 14:39:46.308944",
"laggy_probability": 0.917737,
"laggy_interval": 56,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 5,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.882482,
"laggy_interval": 15,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 6,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.917737,
"laggy_interval": 59,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 7,
"down_stamp": "2015-04-26 14:39:46.308944",
"laggy_probability": 0.917737,
"laggy_interval": 95,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 8,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.642416,
"laggy_interval": 58,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 9,
"down_stamp": "2015-04-26 14:39:51.653157",
"laggy_probability": 0.919336,
"laggy_interval": 24,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 10,
"down_stamp": "2015-04-26 14:39:56.661476",
"laggy_probability": 0.880998,
"laggy_interval": 69,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 11,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.604091,
"laggy_interval": 32,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 12,
"down_stamp": "2015-04-26 14:39:56.661476",
"laggy_probability": 0.849601,
"laggy_interval": 59,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 13,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.859651,
"laggy_interval": 42,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 14,
"down_stamp": "2015-04-26 14:40:01.673316",
"laggy_probability": 0.948316,
"laggy_interval": 30,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 15,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.945507,
"laggy_interval": 26,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 16,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.948413,
"laggy_interval": 19,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 17,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.773384,
"laggy_interval": 72,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 18,
"down_stamp": "2015-04-26 14:40:06.682693",
"laggy_probability": 0.889015,
"laggy_interval": 50,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 19,
"down_stamp": "2015-04-26 03:22:33.512773",
"laggy_probability": 0.931228,
"laggy_interval": 29,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 20,
"down_stamp": "2015-04-26 03:22:55.024663",
"laggy_probability": 0.894947,
"laggy_interval": 43,
"features": 1125899906842623,
"old_weight": 0
},
{
"osd": 21,
"down_stamp": "2015-04-26 03:22:41.016805",
"laggy_probability": 0.933753,
"laggy_interval": 14,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 22,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.951671,
"laggy_interval": 18,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 23,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.849840,
"laggy_interval": 65,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 24,
"down_stamp": "2015-04-26 02:03:26.390577",
"laggy_probability": 0.806616,
"laggy_interval": 47,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 25,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.810979,
"laggy_interval": 52,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 26,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.858681,
"laggy_interval": 132,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 27,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.796432,
"laggy_interval": 70,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 28,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.908335,
"laggy_interval": 39,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 29,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.732148,
"laggy_interval": 53,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 30,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.866609,
"laggy_interval": 44,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 31,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.866052,
"laggy_interval": 46,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 32,
"down_stamp": "2015-04-26 02:21:21.502025",
"laggy_probability": 0.812573,
"laggy_interval": 53,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 33,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.797734,
"laggy_interval": 62,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 34,
"down_stamp": "2015-04-26 14:21:27.940755",
"laggy_probability": 0.681026,
"laggy_interval": 108,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 35,
"down_stamp": "2015-04-26 02:03:26.390577",
"laggy_probability": 0.703038,
"laggy_interval": 47,
"features": 1125899906842623,
"old_weight": 65536
},
{
"osd": 36,
"down_stamp": "2015-04-27 16:35:38.560828",
"laggy_probability": 0.286094,
"laggy_interval": 6,
"features": 1125899906842623,
"old_weight": 0
}
],
"pg_temp": [],
"primary_temp": [],
"blacklist": [],
"erasure_code_profiles": {
"default": {
"directory": "\/usr\/lib\/ceph\/erasure-code",
"k": "2",
"m": "1",
"plugin": "jerasure",
"technique": "reed_sol_van"
}
}
}
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
