I have had the same problem before with unfound objects that happened while backfilling after losing a drive. We didn't lose drives outside of the failure domains and ultimately didn't lose any data, but we did have to wait until after all of the PGs in recovery_wait state were caught up. So if the 2 disks you lost were in the same host and your CRUSH rules are set so that you can lose a host without losing data, then the cluster will likely find all of the objects by the time it's done backfilling. With only losing 2 disks, I wouldn't worry about the missing objects not becoming found unless you're pool size=2.
On Mon, Jan 22, 2018 at 11:47 AM Nico Schottelius < [email protected]> wrote: > > Hello, > > we added about 7 new disks yesterday/today and our cluster became very > slow. While the rebalancing took place, 2 of the 7 new added disks > died. > > Our cluster is still recovering, however we spotted that there are a lot > of unfound objects. > > We lost osd.63 and osd.64, which seem not to be involved into the sample > pg that has unfound objects. > > We were wondering why there are unfound objects, where they are coming > from and if there is a way to recover them? > > Any help appreciated, > > Best, > > Nico > > > Our status is: > > cluster: > id: 26c0c5a8-d7ce-49ac-b5a7-bfd9d0ba81ab > health: HEALTH_WARN > 261953/3006663 objects misplaced (8.712%) > 9377/1002221 objects unfound (0.936%) > Reduced data availability: 176 pgs inactive > Degraded data redundancy: 609338/3006663 objects degraded > (20.266%), 243 pgs unclea > n, 222 pgs degraded, 213 pgs undersized > mon server2 is low on available space > > services: > mon: 3 daemons, quorum server5,server3,server2 > mgr: server5(active), standbys: 2, server2, 0, server3 > osd: 54 osds: 54 up, 54 in; 234 remapped pgs > > data: > pools: 3 pools, 1344 pgs > objects: 978k objects, 3823 GB > usage: 9350 GB used, 40298 GB / 49648 GB avail > pgs: 13.095% pgs not active > 609338/3006663 objects degraded (20.266%) > 261953/3006663 objects misplaced (8.712%) > 9377/1002221 objects unfound (0.936%) > 1101 active+clean > 84 recovery_wait+undersized+degraded+remapped+peered > 82 undersized+degraded+remapped+backfill_wait+peered > 23 active+undersized+degraded+remapped+backfill_wait > 18 active+remapped+backfill_wait > 14 active+undersized+degraded+remapped+backfilling > 10 undersized+degraded+remapped+backfilling+peered > 9 active+recovery_wait+degraded > 3 active+remapped+backfilling > > io: > client: 624 kB/s rd, 3255 kB/s wr, 22 op/s rd, 66 op/s wr > recovery: 90148 kB/s, 22 objects/s > > Looking at the unfound objects: > > [17:32:17] server1:~# ceph health detail > HEALTH_WARN 263745/3006663 objects misplaced (8.772%); 9377/1002221 > objects unfound (0.936%); Reduced data availability: 176 pgs inactive; > Degraded data redundancy: 612398/3006663 objects degraded (20.368%), 244 > pgs unclean, 223 pgs degraded, 214 pgs undersized; mon server2 is low on > available space > OBJECT_MISPLACED 263745/3006663 objects misplaced (8.772%) > OBJECT_UNFOUND 9377/1002221 objects unfound (0.936%) > pg 4.fa has 117 unfound objects > pg 4.ff has 107 unfound objects > pg 4.fd has 113 unfound objects > pg 4.f0 has 120 unfound objects > .... > > > Output from ceph pg 4.fa query: > > { > "state": "recovery_wait+undersized+degraded+remapped+peered", > "snap_trimq": "[]", > "epoch": 17561, > "up": [ > 8, > 17, > 25 > ], > "acting": [ > 61 > ], > "backfill_targets": [ > "8", > "17", > "25" > ], > "actingbackfill": [ > "8", > "17", > "25", > "61" > ], > "info": { > "pgid": "4.fa", > "last_update": "17529'85051", > "last_complete": "17217'77468", > "log_tail": "17091'75034", > "last_user_version": 85051, > "last_backfill": "MAX", > "last_backfill_bitwise": 0, > "purged_snaps": [ > { > "start": "1", > "length": "3" > }, > { > "start": "6", > "length": "8" > }, > { > "start": "10", > "length": "2" > } > ], > "history": { > "epoch_created": 9134, > "epoch_pool_created": 9134, > "last_epoch_started": 17528, > "last_interval_started": 17527, > "last_epoch_clean": 17079, > "last_interval_clean": 17078, > "last_epoch_split": 0, > "last_epoch_marked_full": 0, > "same_up_since": 17143, > "same_interval_since": 17530, > "same_primary_since": 17515, > "last_scrub": "17090'57357", > "last_scrub_stamp": "2018-01-20 20:45:32.616142", > "last_deep_scrub": "17082'54734", > "last_deep_scrub_stamp": "2018-01-15 21:09:34.121488", > "last_clean_scrub_stamp": "2018-01-20 20:45:32.616142" > }, > "stats": { > "version": "17529'85051", > "reported_seq": "218453", > "reported_epoch": "17561", > "state": "recovery_wait+undersized+degraded+remapped+peered", > "last_fresh": "2018-01-22 17:42:28.196701", > "last_change": "2018-01-22 15:00:46.507189", > "last_active": "2018-01-22 15:00:44.635399", > "last_peered": "2018-01-22 17:42:28.196701", > "last_clean": "2018-01-21 20:15:48.267209", > "last_became_active": "2018-01-22 14:53:07.918893", > "last_became_peered": "2018-01-22 15:00:46.437493", > "last_unstale": "2018-01-22 17:42:28.196701", > "last_undegraded": "2018-01-22 15:00:45.664468", > "last_fullsized": "2018-01-22 15:00:45.664468", > "mapping_epoch": 17530, > "log_start": "17091'75034", > "ondisk_log_start": "17091'75034", > "created": 9134, > "last_epoch_clean": 17079, > "parent": "0.0", > "parent_split_bits": 0, > "last_scrub": "17090'57357", > "last_scrub_stamp": "2018-01-20 20:45:32.616142", > "last_deep_scrub": "17082'54734", > "last_deep_scrub_stamp": "2018-01-15 21:09:34.121488", > "last_clean_scrub_stamp": "2018-01-20 20:45:32.616142", > "log_size": 10017, > "ondisk_log_size": 10017, > "stats_invalid": false, > "dirty_stats_invalid": false, > "omap_stats_invalid": false, > "hitset_stats_invalid": false, > "hitset_bytes_stats_invalid": false, > "pin_stats_invalid": false, > "stat_sum": { > "num_bytes": 6112366592, > "num_objects": 1503, > "num_object_clones": 0, > "num_object_copies": 4509, > "num_objects_missing_on_primary": 117, > "num_objects_missing": 0, > "num_objects_degraded": 3123, > "num_objects_misplaced": 1386, > "num_objects_unfound": 117, > "num_objects_dirty": 1503, > "num_whiteouts": 0, > "num_read": 165528, > "num_read_kb": 3252584, > "num_write": 87725, > "num_write_kb": 14125516, > "num_scrub_errors": 0, > "num_shallow_scrub_errors": 0, > "num_deep_scrub_errors": 0, > "num_objects_recovered": 244, > "num_bytes_recovered": 963108864, > "num_keys_recovered": 0, > "num_objects_omap": 0, > "num_objects_hit_set_archive": 0, > "num_bytes_hit_set_archive": 0, > "num_flush": 0, > "num_flush_kb": 0, > "num_evict": 0, > "num_evict_kb": 0, > "num_promote": 0, > "num_flush_mode_high": 0, > "num_flush_mode_low": 0, > "num_evict_mode_some": 0, > "num_evict_mode_full": 0, > "num_objects_pinned": 0, > "num_legacy_snapsets": 0 > }, > "up": [ > 8, > 17, > 25 > ], > "acting": [ > 61 > ], > "blocked_by": [], > "up_primary": 8, > "acting_primary": 61 > }, > "empty": 0, > "dne": 0, > "incomplete": 0, > "last_epoch_started": 17528, > "hit_set_history": { > "current_last_update": "0'0", > "history": [] > } > }, > "peer_info": [ > { > "peer": "8", > "pgid": "4.fa", > "last_update": "17529'85051", > "last_complete": "17529'85051", > "log_tail": "17091'75034", > "last_user_version": 85051, > "last_backfill": "MIN", > "last_backfill_bitwise": 1, > "purged_snaps": [ > { > "start": "1", > "length": "3" > }, > { > "start": "6", > "length": "8" > }, > { > "start": "10", > "length": "2" > } > ], > "history": { > "epoch_created": 9134, > "epoch_pool_created": 9134, > "last_epoch_started": 17528, > "last_interval_started": 17527, > "last_epoch_clean": 17079, > "last_interval_clean": 17078, > "last_epoch_split": 0, > "last_epoch_marked_full": 0, > "same_up_since": 17143, > "same_interval_since": 17530, > "same_primary_since": 17515, > "last_scrub": "17090'57357", > "last_scrub_stamp": "2018-01-20 20:45:32.616142", > "last_deep_scrub": "17082'54734", > "last_deep_scrub_stamp": "2018-01-15 21:09:34.121488", > "last_clean_scrub_stamp": "2018-01-20 20:45:32.616142" > }, > "stats": { > "version": "0'0", > "reported_seq": "3", > "reported_epoch": "17097", > "state": "peering", > "last_fresh": "2018-01-21 20:15:49.798018", > "last_change": "2018-01-21 20:15:49.786423", > "last_active": "0.000000", > "last_peered": "0.000000", > "last_clean": "0.000000", > "last_became_active": "0.000000", > "last_became_peered": "0.000000", > "last_unstale": "2018-01-21 20:15:49.798018", > "last_undegraded": "2018-01-21 20:15:49.798018", > "last_fullsized": "2018-01-21 20:15:49.798018", > "mapping_epoch": 17530, > "log_start": "0'0", > "ondisk_log_start": "0'0", > "created": 9134, > "last_epoch_clean": 17079, > "parent": "0.0", > "parent_split_bits": 0, > "last_scrub": "17090'57357", > "last_scrub_stamp": "2018-01-20 20:45:32.616142", > "last_deep_scrub": "17082'54734", > "last_deep_scrub_stamp": "2018-01-15 21:09:34.121488", > "last_clean_scrub_stamp": "2018-01-20 20:45:32.616142", > "log_size": 0, > "ondisk_log_size": 0, > "stats_invalid": false, > "dirty_stats_invalid": false, > "omap_stats_invalid": false, > "hitset_stats_invalid": false, > "hitset_bytes_stats_invalid": false, > "pin_stats_invalid": false, > "stat_sum": { > "num_bytes": 0, > "num_objects": 0, > "num_object_clones": 0, > "num_object_copies": 0, > "num_objects_missing_on_primary": 0, > "num_objects_missing": 0, > "num_objects_degraded": 0, > "num_objects_misplaced": 0, > "num_objects_unfound": 0, > "num_objects_dirty": 0, > "num_whiteouts": 0, > "num_read": 0, > "num_read_kb": 0, > "num_write": 0, > "num_write_kb": 0, > "num_scrub_errors": 0, > "num_shallow_scrub_errors": 0, > "num_deep_scrub_errors": 0, > "num_objects_recovered": 0, > "num_bytes_recovered": 0, > "num_keys_recovered": 0, > "num_objects_omap": 0, > "num_objects_hit_set_archive": 0, > "num_bytes_hit_set_archive": 0, > "num_flush": 0, > "num_flush_kb": 0, > "num_evict": 0, > "num_evict_kb": 0, > "num_promote": 0, > "num_flush_mode_high": 0, > "num_flush_mode_low": 0, > "num_evict_mode_some": 0, > "num_evict_mode_full": 0, > "num_objects_pinned": 0, > "num_legacy_snapsets": 0 > }, > "up": [ > 8, > 17, > 25 > ], > "acting": [ > 61 > ], > "blocked_by": [], > "up_primary": 8, > "acting_primary": 61 > }, > "empty": 0, > "dne": 0, > "incomplete": 1, > "last_epoch_started": 17528, > "hit_set_history": { > "current_last_update": "0'0", > "history": [] > } > }, > { > "peer": "17", > "pgid": "4.fa", > "last_update": "17529'85051", > "last_complete": "17529'85051", > "log_tail": "17091'75034", > "last_user_version": 85051, > "last_backfill": "MIN", > "last_backfill_bitwise": 1, > "purged_snaps": [ > { > "start": "1", > "length": "3" > }, > { > "start": "6", > "length": "8" > }, > { > "start": "10", > "length": "2" > } > ], > "history": { > "epoch_created": 9134, > "epoch_pool_created": 9134, > "last_epoch_started": 17528, > "last_interval_started": 17527, > "last_epoch_clean": 17079, > "last_interval_clean": 17078, > "last_epoch_split": 0, > "last_epoch_marked_full": 0, > "same_up_since": 17143, > "same_interval_since": 17530, > "same_primary_since": 17515, > "last_scrub": "17090'57357", > "last_scrub_stamp": "2018-01-20 20:45:32.616142", > "last_deep_scrub": "17082'54734", > "last_deep_scrub_stamp": "2018-01-15 21:09:34.121488", > "last_clean_scrub_stamp": "2018-01-20 20:45:32.616142" > }, > "stats": { > "version": "0'0", > "reported_seq": "0", > "reported_epoch": "0", > "state": "unknown", > "last_fresh": "0.000000", > "last_change": "0.000000", > "last_active": "0.000000", > "last_peered": "0.000000", > "last_clean": "0.000000", > "last_became_active": "0.000000", > "last_became_peered": "0.000000", > "last_unstale": "0.000000", > "last_undegraded": "0.000000", > "last_fullsized": "0.000000", > "mapping_epoch": 17530, > "log_start": "0'0", > "ondisk_log_start": "0'0", > "created": 0, > "last_epoch_clean": 0, > "parent": "0.0", > "parent_split_bits": 0, > "last_scrub": "0'0", > "last_scrub_stamp": "0.000000", > "last_deep_scrub": "0'0", > "last_deep_scrub_stamp": "0.000000", > "last_clean_scrub_stamp": "0.000000", > "log_size": 0, > "ondisk_log_size": 0, > "stats_invalid": false, > "dirty_stats_invalid": false, > "omap_stats_invalid": false, > "hitset_stats_invalid": false, > "hitset_bytes_stats_invalid": false, > "pin_stats_invalid": false, > "stat_sum": { > "num_bytes": 0, > "num_objects": 0, > "num_object_clones": 0, > "num_object_copies": 0, > "num_objects_missing_on_primary": 0, > "num_objects_missing": 0, > "num_objects_degraded": 0, > "num_objects_misplaced": 0, > "num_objects_unfound": 0, > "num_objects_dirty": 0, > "num_whiteouts": 0, > "num_read": 0, > "num_read_kb": 0, > "num_write": 0, > "num_write_kb": 0, > "num_scrub_errors": 0, > "num_shallow_scrub_errors": 0, > "num_deep_scrub_errors": 0, > "num_objects_recovered": 0, > "num_bytes_recovered": 0, > "num_keys_recovered": 0, > "num_objects_omap": 0, > "num_objects_hit_set_archive": 0, > "num_bytes_hit_set_archive": 0, > "num_flush": 0, > "num_flush_kb": 0, > "num_evict": 0, > "num_evict_kb": 0, > "num_promote": 0, > "num_flush_mode_high": 0, > "num_flush_mode_low": 0, > "num_evict_mode_some": 0, > "num_evict_mode_full": 0, > "num_objects_pinned": 0, > "num_legacy_snapsets": 0 > }, > "up": [ > 8, > 17, > 25 > ], > "acting": [ > 61 > ], > "blocked_by": [], > "up_primary": 8, > "acting_primary": 61 > }, > "empty": 0, > "dne": 0, > "incomplete": 1, > "last_epoch_started": 17528, > "hit_set_history": { > "current_last_update": "0'0", > "history": [] > } > }, > { > "peer": "20", > "pgid": "4.fa", > "last_update": "17138'75260", > "last_complete": "17138'75260", > "log_tail": "17090'73735", > "last_user_version": 75260, > "last_backfill": "MIN", > "last_backfill_bitwise": 1, > "purged_snaps": [ > { > "start": "1", > "length": "3" > }, > { > "start": "6", > "length": "8" > }, > { > "start": "10", > "length": "2" > } > ], > "history": { > "epoch_created": 9134, > "epoch_pool_created": 9134, > "last_epoch_started": 17121, > "last_interval_started": 17120, > "last_epoch_clean": 17079, > "last_interval_clean": 17078, > "last_epoch_split": 0, > "last_epoch_marked_full": 0, > "same_up_since": 17143, > "same_interval_since": 17530, > "same_primary_since": 17515, > "last_scrub": "17090'57357", > "last_scrub_stamp": "2018-01-20 20:45:32.616142", > "last_deep_scrub": "17082'54734", > "last_deep_scrub_stamp": "2018-01-15 21:09:34.121488", > "last_clean_scrub_stamp": "2018-01-20 20:45:32.616142" > }, > "stats": { > "version": "0'0", > "reported_seq": "0", > "reported_epoch": "0", > "state": "unknown", > "last_fresh": "0.000000", > "last_change": "0.000000", > "last_active": "0.000000", > "last_peered": "0.000000", > "last_clean": "0.000000", > "last_became_active": "0.000000", > "last_became_peered": "0.000000", > "last_unstale": "0.000000", > "last_undegraded": "0.000000", > "last_fullsized": "0.000000", > "mapping_epoch": 17530, > "log_start": "0'0", > "ondisk_log_start": "0'0", > "created": 0, > "last_epoch_clean": 0, > "parent": "0.0", > "parent_split_bits": 0, > "last_scrub": "0'0", > "last_scrub_stamp": "0.000000", > "last_deep_scrub": "0'0", > "last_deep_scrub_stamp": "0.000000", > "last_clean_scrub_stamp": "0.000000", > "log_size": 0, > "ondisk_log_size": 0, > "stats_invalid": false, > "dirty_stats_invalid": false, > "omap_stats_invalid": false, > "hitset_stats_invalid": false, > "hitset_bytes_stats_invalid": false, > "pin_stats_invalid": false, > "stat_sum": { > "num_bytes": 0, > "num_objects": 0, > "num_object_clones": 0, > "num_object_copies": 0, > "num_objects_missing_on_primary": 0, > "num_objects_missing": 0, > "num_objects_degraded": 0, > "num_objects_misplaced": 0, > "num_objects_unfound": 0, > "num_objects_dirty": 0, > "num_whiteouts": 0, > "num_read": 0, > "num_read_kb": 0, > "num_write": 0, > "num_write_kb": 0, > "num_scrub_errors": 0, > "num_shallow_scrub_errors": 0, > "num_deep_scrub_errors": 0, > "num_objects_recovered": 0, > "num_bytes_recovered": 0, > "num_keys_recovered": 0, > "num_objects_omap": 0, > "num_objects_hit_set_archive": 0, > "num_bytes_hit_set_archive": 0, > "num_flush": 0, > "num_flush_kb": 0, > "num_evict": 0, > "num_evict_kb": 0, > "num_promote": 0, > "num_flush_mode_high": 0, > "num_flush_mode_low": 0, > "num_evict_mode_some": 0, > "num_evict_mode_full": 0, > "num_objects_pinned": 0, > "num_legacy_snapsets": 0 > }, > "up": [ > 8, > 17, > 25 > ], > "acting": [ > 61 > ], > "blocked_by": [], > "up_primary": 8, > "acting_primary": 61 > }, > "empty": 0, > "dne": 0, > "incomplete": 1, > "last_epoch_started": 17121, > "hit_set_history": { > "current_last_update": "0'0", > "history": [] > } > }, > { > "peer": "25", > "pgid": "4.fa", > "last_update": "17529'85051", > "last_complete": "17529'85051", > "log_tail": "17091'75034", > "last_user_version": 85051, > "last_backfill": "MIN", > "last_backfill_bitwise": 1, > "purged_snaps": [ > { > "start": "1", > "length": "3" > }, > { > "start": "6", > "length": "8" > }, > { > "start": "10", > "length": "2" > } > ], > "history": { > "epoch_created": 9134, > "epoch_pool_created": 9134, > "last_epoch_started": 17528, > "last_interval_started": 17527, > "last_epoch_clean": 17079, > "last_interval_clean": 17078, > "last_epoch_split": 0, > "last_epoch_marked_full": 0, > "same_up_since": 17143, > "same_interval_since": 17530, > "same_primary_since": 17515, > "last_scrub": "17090'57357", > "last_scrub_stamp": "2018-01-20 20:45:32.616142", > "last_deep_scrub": "17082'54734", > "last_deep_scrub_stamp": "2018-01-15 21:09:34.121488", > "last_clean_scrub_stamp": "2018-01-20 20:45:32.616142" > }, > "stats": { > "version": "0'0", > "reported_seq": "0", > "reported_epoch": "0", > "state": "unknown", > "last_fresh": "0.000000", > "last_change": "0.000000", > "last_active": "0.000000", > "last_peered": "0.000000", > "last_clean": "0.000000", > "last_became_active": "0.000000", > "last_became_peered": "0.000000", > "last_unstale": "0.000000", > "last_undegraded": "0.000000", > "last_fullsized": "0.000000", > "mapping_epoch": 17530, > "log_start": "0'0", > "ondisk_log_start": "0'0", > "created": 0, > "last_epoch_clean": 0, > "parent": "0.0", > "parent_split_bits": 0, > "last_scrub": "0'0", > "last_scrub_stamp": "0.000000", > "last_deep_scrub": "0'0", > "last_deep_scrub_stamp": "0.000000", > "last_clean_scrub_stamp": "0.000000", > "log_size": 0, > "ondisk_log_size": 0, > "stats_invalid": false, > "dirty_stats_invalid": false, > "omap_stats_invalid": false, > "hitset_stats_invalid": false, > "hitset_bytes_stats_invalid": false, > "pin_stats_invalid": false, > "stat_sum": { > "num_bytes": 0, > "num_objects": 0, > "num_object_clones": 0, > "num_object_copies": 0, > "num_objects_missing_on_primary": 0, > "num_objects_missing": 0, > "num_objects_degraded": 0, > "num_objects_misplaced": 0, > "num_objects_unfound": 0, > "num_objects_dirty": 0, > "num_whiteouts": 0, > "num_read": 0, > "num_read_kb": 0, > "num_write": 0, > "num_write_kb": 0, > "num_scrub_errors": 0, > "num_shallow_scrub_errors": 0, > "num_deep_scrub_errors": 0, > "num_objects_recovered": 0, > "num_bytes_recovered": 0, > "num_keys_recovered": 0, > "num_objects_omap": 0, > "num_objects_hit_set_archive": 0, > "num_bytes_hit_set_archive": 0, > "num_flush": 0, > "num_flush_kb": 0, > "num_evict": 0, > "num_evict_kb": 0, > "num_promote": 0, > "num_flush_mode_high": 0, > "num_flush_mode_low": 0, > "num_evict_mode_some": 0, > "num_evict_mode_full": 0, > "num_objects_pinned": 0, > "num_legacy_snapsets": 0 > }, > "up": [ > 8, > 17, > 25 > ], > "acting": [ > 61 > ], > "blocked_by": [], > "up_primary": 8, > "acting_primary": 61 > }, > "empty": 0, > "dne": 0, > "incomplete": 1, > "last_epoch_started": 17528, > "hit_set_history": { > "current_last_update": "0'0", > "history": [] > } > } > ], > "recovery_state": [ > { > "name": "Started/Primary/Active", > "enter_time": "2018-01-22 15:00:45.664496", > "might_have_unfound": [ > { > "osd": "8", > "status": "already probed" > }, > { > "osd": "17", > "status": "already probed" > }, > { > "osd": "20", > "status": "already probed" > }, > { > "osd": "25", > "status": "already probed" > }, > { > "osd": "63", > "status": "osd is down" > }, > { > "osd": "64", > "status": "osd is down" > } > ], > "recovery_progress": { > "backfill_targets": [ > "8", > "17", > "25" > ], > "waiting_on_backfill": [], > "last_backfill_started": "MIN", > "backfill_info": { > "begin": "MIN", > "end": "MIN", > "objects": [] > }, > "peer_backfill_info": [], > "backfills_in_flight": [], > "recovering": [], > "pg_backend": { > "pull_from_peer": [], > "pushing": [] > } > }, > "scrub": { > "scrubber.epoch_start": "17078", > "scrubber.active": false, > "scrubber.state": "INACTIVE", > "scrubber.start": "MIN", > "scrubber.end": "MIN", > "scrubber.subset_last_update": "0'0", > "scrubber.deep": false, > "scrubber.seed": 0, > "scrubber.waiting_on": 0, > "scrubber.waiting_on_whom": [] > } > }, > { > "name": "Started", > "enter_time": "2018-01-22 15:00:45.613486" > } > ], > "agent_state": {} > } > > > ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF > -1 48.94531 root default > -6 11.88382 host server1 > 25 hdd 4.59999 osd.25 up 1.00000 1.00000 > 61 hdd 4.59999 osd.61 up 1.00000 1.00000 > 48 ssd 0.22198 osd.48 up 1.00000 1.00000 > 49 ssd 0.22198 osd.49 up 1.00000 1.00000 > 50 ssd 0.22198 osd.50 up 1.00000 1.00000 > 51 ssd 0.22699 osd.51 up 1.00000 1.00000 > 52 ssd 0.22198 osd.52 up 1.00000 1.00000 > 53 ssd 0.22198 osd.53 up 1.00000 1.00000 > 54 ssd 0.22198 osd.54 up 1.00000 1.00000 > 55 ssd 0.22699 osd.55 up 1.00000 1.00000 > 56 ssd 0.22198 osd.56 up 1.00000 1.00000 > 57 ssd 0.22198 osd.57 up 1.00000 1.00000 > 58 ssd 0.22699 osd.58 up 1.00000 1.00000 > 59 ssd 0.22699 osd.59 up 1.00000 1.00000 > -2 11.95193 host server2 > 21 hdd 4.59999 osd.21 up 1.00000 1.00000 > 24 hdd 4.59999 osd.24 up 1.00000 1.00000 > 0 ssd 0.68799 osd.0 up 1.00000 1.00000 > 4 ssd 0.68799 osd.4 up 1.00000 1.00000 > 6 ssd 0.68799 osd.6 up 1.00000 1.00000 > 10 ssd 0.68799 osd.10 up 1.00000 1.00000 > -3 6.61287 host server3 > 20 hdd 4.59999 osd.20 up 1.00000 1.00000 > 1 ssd 0.22198 osd.1 up 1.00000 1.00000 > 7 ssd 0.22198 osd.7 up 1.00000 1.00000 > 12 ssd 0.22198 osd.12 up 1.00000 1.00000 > 15 ssd 0.22699 osd.15 up 1.00000 1.00000 > 23 ssd 0.22198 osd.23 up 1.00000 1.00000 > 27 ssd 0.22198 osd.27 up 1.00000 1.00000 > 29 ssd 0.22699 osd.29 up 1.00000 1.00000 > 33 ssd 0.22198 osd.33 up 1.00000 1.00000 > 42 ssd 0.22699 osd.42 up 1.00000 1.00000 > -5 6.61287 host server4 > 17 hdd 4.59999 osd.17 up 1.00000 1.00000 > 3 ssd 0.22198 osd.3 up 1.00000 1.00000 > 11 ssd 0.22198 osd.11 up 1.00000 1.00000 > 16 ssd 0.22699 osd.16 up 1.00000 1.00000 > 19 ssd 0.22198 osd.19 up 1.00000 1.00000 > 28 ssd 0.22198 osd.28 up 1.00000 1.00000 > 37 ssd 0.22198 osd.37 up 1.00000 1.00000 > 41 ssd 0.22198 osd.41 up 1.00000 1.00000 > 43 ssd 0.22699 osd.43 up 1.00000 1.00000 > 46 ssd 0.22699 osd.46 up 1.00000 1.00000 > -4 11.88382 host server5 > 8 hdd 4.59999 osd.8 up 1.00000 1.00000 > 13 hdd 4.59999 osd.13 up 1.00000 1.00000 > 2 ssd 0.22198 osd.2 up 1.00000 1.00000 > 5 ssd 0.22198 osd.5 up 1.00000 1.00000 > 9 ssd 0.22198 osd.9 up 1.00000 1.00000 > 14 ssd 0.22699 osd.14 up 1.00000 1.00000 > 18 ssd 0.22198 osd.18 up 1.00000 1.00000 > 22 ssd 0.22198 osd.22 up 1.00000 1.00000 > 26 ssd 0.22198 osd.26 up 1.00000 1.00000 > 30 ssd 0.22699 osd.30 up 1.00000 1.00000 > 36 ssd 0.22198 osd.36 up 1.00000 1.00000 > 40 ssd 0.22198 osd.40 up 1.00000 1.00000 > 45 ssd 0.22699 osd.45 up 1.00000 1.00000 > 47 ssd 0.22699 osd.47 up 1.00000 1.00000 > > > -- > Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
