Hi,
I have also seen the same warning even when I use v0.56.1 (both kernel
rbd and OSD side) when write stress is high enough(Say I have 3 osds but having
4~5 clients doing dd on top of the rbd).
2013-01-15 15:54:05.990052 7ff97dd0c700 0 log [WRN] : slow request 32.545624
seconds old, received at 2013-01-15 15:53:33.444348: osd_op(client.4125.1:51574
rb.0.100f.6b8b4567.000000000827 [write 3670016~524288] 2.1ae8252c) currently
waiting for sub ops
2013-01-15 15:54:05.990055 7ff97dd0c700 0 log [WRN] : slow request 32.533197
seconds old, received at 2013-01-15 15:53:33.456775: osd_op(client.4125.1:51599
rb.0.100e.6b8b4567.0000000007a7 [write 3522560~524288] 2.54bcf797) currently
waiting for sub ops
2013-01-15 15:54:05.990058 7ff97dd0c700 0 log [WRN] : slow request 32.519738
seconds old, received at 2013-01-15 15:53:33.470234: osd_op(client.4125.1:51600
rb.0.100e.6b8b4567.0000000007a7 [write 4046848~147456] 2.54bcf797) currently
waiting for sub ops
2013-01-15 15:54:06.990355 7ff97dd0c700 0 log [WRN] : 2 slow requests, 2
included below; oldest blocked for > 33.225773 secs
2013-01-15 15:54:06.990366 7ff97dd0c700 0 log [WRN] : slow request 33.225773
seconds old, received at 2013-01-15 15:53:33.764540: osd_op(client.4125.1:51665
rb.0.100f.6b8b4567.00000000082e [write 3145728~524288] 2.3c00a0fb) currently
waiting for sub ops
2013-01-15 15:54:06.990371 7ff97dd0c700 0 log [WRN] : slow request 33.221343
seconds old, received at 2013-01-15 15:53:33.768970: osd_op(client.4125.1:51666
rb.0.100f.6b8b4567.00000000082e [write 3670016~524288] 2.3c00a0fb) currently
waiting for sub ops
2013-01-15 15:54:18.993759 7ff97dd0c700 0 log [WRN] : 6 slow requests, 6
included below; oldest blocked for > 30.313194 secs
2013-01-15 15:54:18.993769 7ff97dd0c700 0 log [WRN] : slow request 30.313194
seconds old, received at 2013-01-15 15:53:48.680488: osd_op(client.4125.1:52333
rb.0.100d.6b8b4567.000000000851 [write 1048576~524288] 2.43b76099) currently
waiting for sub ops
2013-01-15 15:54:18.993774 7ff97dd0c700 0 log [WRN] : slow request 30.308800
seconds old, received at 2013-01-15 15:53:48.684882: osd_op(client.4125.1:52334
rb.0.100d.6b8b4567.000000000851 [write 1572864~524288] 2.43b76099) currently
waiting for sub ops
2013-01-15 15:54:18.993778 7ff97dd0c700 0 log [WRN] : slow request 30.304389
seconds old, received at 2013-01-15 15:53:48.689293: osd_op(client.4125.1:52335
rb.0.100d.6b8b4567.000000000851 [write 2097152~524288] 2.43b76099) currently
waiting for sub ops
2013-01-15 15:54:18.993782 7ff97dd0c700 0 log [WRN] : slow request 30.299783
seconds old, received at 2013-01-15 15:53:48.693899: osd_op(client.4125.1:52336
rb.0.100d.6b8b4567.000000000851 [write 2621440~524288] 2.43b76099) currently
waiting for sub ops
2013-01-15 15:54:18.993787 7ff97dd0c700 0 log [WRN] : slow request 30.295340
seconds old, received at 2013-01-15 15:53:48.698342: osd_op(client.4125.1:52337
rb.0.100d.6b8b4567.000000000851 [write 3145728~524288] 2.43b76099) currently
waiting for sub ops
Xiaoxi
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Sage Weil
Sent: 2012年12月17日 6:55
To: Holcombe, Christopher
Cc: Jens Kristian Søgaard; [email protected]
Subject: RE: Slow requests
v0.52 is also quite old. This was resolved in v0.55, I believe. Where are you
looking for the RPMs? v0.55 should be there...
sage
On Sun, 16 Dec 2012, Holcombe, Christopher wrote:
> I heard the solution for this was to restart the osd's. That fixed it for me.
>
> -Chris
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Jens Kristian
> S?gaard
> Sent: Sunday, December 16, 2012 9:00 AM
> To: [email protected]
> Subject: Slow requests
>
> Hi,
>
> My log is filling up with warnings about a single slow request that has been
> around for a very long time:
>
> osd.1 10.0.0.2:6800/900 162926 : [WRN] 1 slow requests, 1 included
> below; oldest blocked for > 84446.312051 secs
>
> osd.1 10.0.0.2:6800/900 162927 : [WRN] slow request 84446.312051 seconds old,
> received at 2012-12-15 15:27:56.891437:
> osd_sub_op(client.4528.0:19602219 0.fe
> 3807b5fe/rb.0.11b7.4a933baa.00000008629e/head//0 [] v 53'185888
> snapset=0=[]:[] snapc=0=[]) v7 currently started
>
>
> How can I identify the cause of this and how can I cancel this request?
>
> I'm running Ceph on Fedora 17 using the latest RPMs available from ceph.com
> (0.52-6).
>
>
> Thanks in advance,
> --
> Jens Kristian S?gaard, Mermaid Consulting ApS,
> [email protected], http://www.mermaidconsulting.com/
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in the body of a message to [email protected] More majordomo
> info at http://vger.kernel.org/majordomo-info.html
>
> ________________________________
>
> NOTICE: This e-mail and any attachments is intended only for use by the
> addressee(s) named herein and may contain legally privileged, proprietary or
> confidential information. If you are not the intended recipient of this
> e-mail, you are hereby notified that any dissemination, distribution or
> copying of this email, and any attachments thereto, is strictly prohibited.
> If you receive this email in error please immediately notify me via reply
> email or at (800) 927-9800 and permanently delete the original copy and any
> copy of any e-mail, and any printout.
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in the body of a message to [email protected] More majordomo
> info at http://vger.kernel.org/majordomo-info.html
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the
body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
