Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-03-07 Thread Ritter Sławomir
> > Yes, > > > > fix for #8269 also has been included in our version: Dumpling 0.67.11. > > Guys from #13764 are using patched Hammer version > > I didn't notice that you were actually running Dumpling (which we > haven't supported and backported fixes for a while). Here's one issue > that you mig

Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-03-04 Thread Yehuda Sadeh-Weinraub
On Fri, Mar 4, 2016 at 7:26 AM, Ritter Sławomir wrote: >> From: Robin H. Johnson [mailto:robb...@gentoo.org] >> Sent: Friday, March 04, 2016 12:40 AM >> To: Ritter Sławomir >> Cc: ceph-us...@ceph.com; ceph-devel >> Subject: Re: [ceph-users] Problem: silently corrupte

Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-03-04 Thread Ritter Sławomir
> From: Robin H. Johnson [mailto:robb...@gentoo.org] > Sent: Friday, March 04, 2016 12:40 AM > To: Ritter Sławomir > Cc: ceph-us...@ceph.com; ceph-devel > Subject: Re: [ceph-users] Problem: silently corrupted RadosGW objects caused > by slow requests > > On Thu, Mar 03,

Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-03-04 Thread Ritter Sławomir
Thnx for contact. > > 2016-02-23 13:49:58.818640 osd.260 10.176.67.27:6800/688083 2119 : [WRN] 4 > > slow requests, 4 included below; oldest blocked for > 30.727096 secs > > 2016-02-23 13:49:58.818673 osd.260 10.176.67.27:6800/688083 2120 : [WRN] > > slow request 30.727096 seconds old, received at

Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-03-03 Thread Robin H. Johnson
On Thu, Mar 03, 2016 at 01:55:13PM +0100, Ritter Sławomir wrote: > Hi, > > I think this is really serious problem - again: > > - we silently lost S3/RGW objects in clusters > > Moreover, it our situation looks very similiar to described in > uncorrected bug #13764 (Hammer) and in corrected #8

Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-03-03 Thread Yehuda Sadeh-Weinraub
On Thu, Feb 25, 2016 at 7:17 AM, Ritter Sławomir wrote: > Hi, > > > > We have two CEPH clusters running on Dumpling 0.67.11 and some of our > "multipart objects" are incompleted. It seems that some slow requests could > cause corruption of related S3 objects. Moveover GETs for that objects are > w

Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-03-03 Thread Ritter Sławomir
[mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Dominik Mostowiec Sent: Friday, February 26, 2016 3:33 PM To: ceph-us...@ceph.com; ceph-devel Subject: Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests Hi, Maybe this is the reason of another bug? http

Re: [ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-02-26 Thread Dominik Mostowiec
Hi, Maybe this is the reason of another bug? http://tracker.ceph.com/issues/13764 The situation is very similiar... -- Regards Dominik 2016-02-25 16:17 GMT+01:00 Ritter Sławomir : > Hi, > > > > We have two CEPH clusters running on Dumpling 0.67.11 and some of our > "multipart objects" are incompl

[ceph-users] Problem: silently corrupted RadosGW objects caused by slow requests

2016-02-25 Thread Ritter Sławomir
Hi, We have two CEPH clusters running on Dumpling 0.67.11 and some of our "multipart objects" are incompleted. It seems that some slow requests could cause corruption of related S3 objects. Moveover GETs for that objects are working without any error messages. There are only HTTP 200 in logs as