Re: [ceph-users] [PG] Slow request *** seconds old,v4 currently waiting for pg to exist locally

2014-09-25 Thread Udo Lembke
Hi,
looks that some osds are down?!

What is the output of ceph osd tree

Udo

Am 25.09.2014 04:29, schrieb Aegeaner:
 The cluster healthy state is WARN:
 
  health HEALTH_WARN 118 pgs degraded; 8 pgs down; 59 pgs
 incomplete; 28 pgs peering; 292 pgs stale; 87 pgs stuck inactive;
 292 pgs stuck stale; 205 pgs stuck unclean; 22 requests are blocked
  32 sec; recovery 12474/46357 objects degraded (26.909%)
  monmap e3: 3 mons at
 
 {CVM-0-mon01=172.18.117.146:6789/0,CVM-0-mon02=172.18.117.152:6789/0,CVM-0-mon03=172.18.117.153:6789/0},
 election epoch 24, quorum 0,1,2 CVM-0-mon01,CVM-0-mon02,CVM-0-mon03
  osdmap e421: 9 osds: 9 up, 9 in
   pgmap v2261: 292 pgs, 4 pools, 91532 MB data, 23178 objects
 330 MB used, 3363 GB / 3363 GB avail
 12474/46357 objects degraded (26.909%)
   20 stale+peering
   87 stale+active+clean
8 stale+down+peering
   59 stale+incomplete
  118 stale+active+degraded
 
 
 What does these errors mean? Can these PGs be recovered?
 

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


Re: [ceph-users] [PG] Slow request *** seconds old,v4 currently waiting for pg to exist locally

2014-09-25 Thread Udo Lembke
Hi again,
sorry - forgot my post... see

osdmap e421: 9 osds: 9 up, 9 in

shows that all your 9 osds are up!

Do you have trouble with your journal/filesystem?

Udo

Am 25.09.2014 08:01, schrieb Udo Lembke:
 Hi,
 looks that some osds are down?!
 
 What is the output of ceph osd tree
 
 Udo
 
 Am 25.09.2014 04:29, schrieb Aegeaner:
 The cluster healthy state is WARN:

  health HEALTH_WARN 118 pgs degraded; 8 pgs down; 59 pgs
 incomplete; 28 pgs peering; 292 pgs stale; 87 pgs stuck inactive;
 292 pgs stuck stale; 205 pgs stuck unclean; 22 requests are blocked
  32 sec; recovery 12474/46357 objects degraded (26.909%)
  monmap e3: 3 mons at
 
 {CVM-0-mon01=172.18.117.146:6789/0,CVM-0-mon02=172.18.117.152:6789/0,CVM-0-mon03=172.18.117.153:6789/0},
 election epoch 24, quorum 0,1,2 CVM-0-mon01,CVM-0-mon02,CVM-0-mon03
  osdmap e421: 9 osds: 9 up, 9 in
   pgmap v2261: 292 pgs, 4 pools, 91532 MB data, 23178 objects
 330 MB used, 3363 GB / 3363 GB avail
 12474/46357 objects degraded (26.909%)
   20 stale+peering
   87 stale+active+clean
8 stale+down+peering
   59 stale+incomplete
  118 stale+active+degraded


 What does these errors mean? Can these PGs be recovered?


 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


Re: [ceph-users] [PG] Slow request *** seconds old,v4 currently waiting for pg to exist locally

2014-09-25 Thread Irek Fasikhov
osd_op(client.4625.1:9005787)
.


This is due to external factors. For example, the network settings.

2014-09-25 10:05 GMT+04:00 Udo Lembke ulem...@polarzone.de:

 Hi again,
 sorry - forgot my post... see

 osdmap e421: 9 osds: 9 up, 9 in

 shows that all your 9 osds are up!

 Do you have trouble with your journal/filesystem?

 Udo

 Am 25.09.2014 08:01, schrieb Udo Lembke:
  Hi,
  looks that some osds are down?!
 
  What is the output of ceph osd tree
 
  Udo
 
  Am 25.09.2014 04:29, schrieb Aegeaner:
  The cluster healthy state is WARN:
 
   health HEALTH_WARN 118 pgs degraded; 8 pgs down; 59 pgs
  incomplete; 28 pgs peering; 292 pgs stale; 87 pgs stuck inactive;
  292 pgs stuck stale; 205 pgs stuck unclean; 22 requests are blocked
   32 sec; recovery 12474/46357 objects degraded (26.909%)
   monmap e3: 3 mons at
  {CVM-0-mon01=
 172.18.117.146:6789/0,CVM-0-mon02=172.18.117.152:6789/0,CVM-0-mon03=172.18.117.153:6789/0
 },
  election epoch 24, quorum 0,1,2 CVM-0-mon01,CVM-0-mon02,CVM-0-mon03
   osdmap e421: 9 osds: 9 up, 9 in
pgmap v2261: 292 pgs, 4 pools, 91532 MB data, 23178 objects
  330 MB used, 3363 GB / 3363 GB avail
  12474/46357 objects degraded (26.909%)
20 stale+peering
87 stale+active+clean
 8 stale+down+peering
59 stale+incomplete
   118 stale+active+degraded
 
 
  What does these errors mean? Can these PGs be recovered?
 
 
  ___
  ceph-users mailing list
  ceph-users@lists.ceph.com
  http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 

 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




-- 
С уважением, Фасихов Ирек Нургаязович
Моб.: +79229045757
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


Re: [ceph-users] [PG] Slow request *** seconds old,v4 currently waiting for pg to exist locally

2014-09-25 Thread Aegeaner
Yeah, three of nine OSDs went down but I recreated them, but the pgs 
cannot be recovered.


I don't know how to erase all the pgs, so I deleted all the osd pools, 
including data and metadata … Now all pgs are active and clean...


I'm not sure if there are more elegant ways to deal with this.

===
Aegeaner


在 2014-09-25 14:11, Irek Fasikhov 写道:

osd_op(client.4625.1:9005787)
.


This is due to external factors. For example, the network settings.

2014-09-25 10:05 GMT+04:00 Udo Lembke ulem...@polarzone.de 
mailto:ulem...@polarzone.de:


Hi again,
sorry - forgot my post... see

osdmap e421: 9 osds: 9 up, 9 in

shows that all your 9 osds are up!

Do you have trouble with your journal/filesystem?

Udo

Am 25.09.2014 08:01, schrieb Udo Lembke:
 Hi,
 looks that some osds are down?!

 What is the output of ceph osd tree

 Udo

 Am 25.09.2014 04:29, schrieb Aegeaner:
 The cluster healthy state is WARN:

  health HEALTH_WARN 118 pgs degraded; 8 pgs down; 59 pgs
 incomplete; 28 pgs peering; 292 pgs stale; 87 pgs stuck
inactive;
 292 pgs stuck stale; 205 pgs stuck unclean; 22 requests are
blocked
  32 sec; recovery 12474/46357 objects degraded (26.909%)
  monmap e3: 3 mons at
   
 {CVM-0-mon01=172.18.117.146:6789/0,CVM-0-mon02=172.18.117.152:6789/0,CVM-0-mon03=172.18.117.153:6789/0


http://172.18.117.146:6789/0,CVM-0-mon02=172.18.117.152:6789/0,CVM-0-mon03=172.18.117.153:6789/0},
 election epoch 24, quorum 0,1,2
CVM-0-mon01,CVM-0-mon02,CVM-0-mon03
  osdmap e421: 9 osds: 9 up, 9 in
   pgmap v2261: 292 pgs, 4 pools, 91532 MB data, 23178
objects
 330 MB used, 3363 GB / 3363 GB avail
 12474/46357 objects degraded (26.909%)
   20 stale+peering
   87 stale+active+clean
8 stale+down+peering
   59 stale+incomplete
  118 stale+active+degraded


 What does these errors mean? Can these PGs be recovered?


 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com mailto:ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com mailto:ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




--
С уважением, Фасихов Ирек Нургаязович
Моб.: +79229045757


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com