No. This will no help (((
I try to found data, but it's look exist with same time stamp on all osd or
missing on all osd ...
So, need advice , what I need to do...
вторник, 18 августа 2015 г. пользователь Abhishek L написал:
>
> Voloshanenko Igor writes:
>
> > Hi Irek, Please read careful )))
> > You proposal was the first, i try to do... That's why i asked about
> > help... (
> >
> > 2015-08-18 8:34 GMT+03:00 Irek Fasikhov <malm...@gmail.com
> <javascript:;>>:
> >
> >> Hi, Igor.
> >>
> >> You need to repair the PG.
> >>
> >> for i in `ceph pg dump| grep inconsistent | grep -v
> 'inconsistent+repair'
> >> | awk {'print$1'}`;do ceph pg repair $i;done
> >>
> >> С уважением, Фасихов Ирек Нургаязович
> >> Моб.: +79229045757
> >>
> >> 2015-08-18 8:27 GMT+03:00 Voloshanenko Igor <
> igor.voloshane...@gmail.com <javascript:;>>:
> >>
> >>> Hi all, at our production cluster, due high rebalancing ((( we have 2
> pgs
> >>> in inconsistent state...
> >>>
> >>> root@temp:~# ceph health detail | grep inc
> >>> HEALTH_ERR 2 pgs inconsistent; 18 scrub errors
> >>> pg 2.490 is active+clean+inconsistent, acting [56,15,29]
> >>> pg 2.c4 is active+clean+inconsistent, acting [56,10,42]
> >>>
> >>> From OSD logs, after recovery attempt:
> >>>
> >>> root@test:~# ceph pg dump | grep -i incons | cut -f 1 | while read i;
> do
> >>> ceph pg repair ${i} ; done
> >>> dumped all in format plain
> >>> instructing pg 2.490 on osd.56 to repair
> >>> instructing pg 2.c4 on osd.56 to repair
> >>>
> >>> /var/log/ceph/ceph-osd.56.log:51:2015-08-18 07:26:37.035910
> 7f94663b3700
> >>> -1 log_channel(cluster) log [ERR] : deep-scrub 2.490
> >>> f5759490/rbd_data.1631755377d7e.00000000000004da/head//2 expected clone
> >>> 90c59490/rbd_data.eb486436f2beb.0000000000007a65/141//2
> >>> /var/log/ceph/ceph-osd.56.log:52:2015-08-18 07:26:37.035960
> 7f94663b3700
> >>> -1 log_channel(cluster) log [ERR] : deep-scrub 2.490
> >>> fee49490/rbd_data.12483d3ba0794b.000000000000522f/head//2 expected
> clone
> >>> f5759490/rbd_data.1631755377d7e.00000000000004da/141//2
> >>> /var/log/ceph/ceph-osd.56.log:53:2015-08-18 07:26:37.036133
> 7f94663b3700
> >>> -1 log_channel(cluster) log [ERR] : deep-scrub 2.490
> >>> a9b39490/rbd_data.12483d3ba0794b.00000000000037b3/head//2 expected
> clone
> >>> fee49490/rbd_data.12483d3ba0794b.000000000000522f/141//2
> >>> /var/log/ceph/ceph-osd.56.log:54:2015-08-18 07:26:37.036243
> 7f94663b3700
> >>> -1 log_channel(cluster) log [ERR] : deep-scrub 2.490
> >>> bac19490/rbd_data.1238e82ae8944a.000000000000032e/head//2 expected
> clone
> >>> a9b39490/rbd_data.12483d3ba0794b.00000000000037b3/141//2
> >>> /var/log/ceph/ceph-osd.56.log:55:2015-08-18 07:26:37.036289
> 7f94663b3700
> >>> -1 log_channel(cluster) log [ERR] : deep-scrub 2.490
> >>> 98519490/rbd_data.123e9c2ae8944a.0000000000000807/head//2 expected
> clone
> >>> bac19490/rbd_data.1238e82ae8944a.000000000000032e/141//2
> >>> /var/log/ceph/ceph-osd.56.log:56:2015-08-18 07:26:37.036314
> 7f94663b3700
> >>> -1 log_channel(cluster) log [ERR] : deep-scrub 2.490
> >>> c3c09490/rbd_data.1238e82ae8944a.0000000000000c2b/head//2 expected
> clone
> >>> 98519490/rbd_data.123e9c2ae8944a.0000000000000807/141//2
> >>> /var/log/ceph/ceph-osd.56.log:57:2015-08-18 07:26:37.036363
> 7f94663b3700
> >>> -1 log_channel(cluster) log [ERR] : deep-scrub 2.490
> >>> 28809490/rbd_data.edea7460fe42b.00000000000001d9/head//2 expected clone
> >>> c3c09490/rbd_data.1238e82ae8944a.0000000000000c2b/141//2
> >>> /var/log/ceph/ceph-osd.56.log:58:2015-08-18 07:26:37.036432
> 7f94663b3700
> >>> -1 log_channel(cluster) log [ERR] : deep-scrub 2.490
> >>> e1509490/rbd_data.1423897545e146.00000000000009a6/head//2 expected
> clone
> >>> 28809490/rbd_data.edea7460fe42b.00000000000001d9/141//2
> >>> /var/log/ceph/ceph-osd.56.log:59:2015-08-18 07:26:38.548765
> 7f94663b3700
> >>> -1 log_channel(cluster) log [ERR] : 2.490 deep-scrub 17 errors
> >>>
> >>> So, how i can solve "expected clone" situation by hand?
> >>> Thank in advance!
>
> I've had an inconsistent pg once, but it was a different sort of an
> error (some sort of digest mismatch, where the secondary object copies
> had later timestamps). This was fixed by moving the object away and
> restarting, the osd which got fixed when the osd peered, similar to what
> was mentioned in Sebastian Han's blog[1].
>
> I'm guessing the same method will solve this error as well, but not
> completely sure, maybe someone else who has seen this particular error
> could guide you better.
>
> [1]:
> http://www.sebastien-han.fr/blog/2015/04/27/ceph-manually-repair-object/
>
> --
> Abhishek
>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
