Re: RAID extremely slow

2012-08-17 Thread Jan Engelhardt
On Thursday 2012-07-26 03:00, Phil Turmel wrote: >> I used atop to show the transfer speeds to each drive. Here's a >> screenshot: >> http://img402.imageshack.us/img402/6484/screenshotfrom201207251.png > >[ The output of "lsdrv" [1] might be useful here, along with >"mdadm -D /dev/md0" and "mdadm

Re: RAID extremely slow

2012-08-17 Thread Jan Engelhardt
On Thursday 2012-07-26 03:00, Phil Turmel wrote: I used atop to show the transfer speeds to each drive. Here's a screenshot: http://img402.imageshack.us/img402/6484/screenshotfrom201207251.png [ The output of lsdrv [1] might be useful here, along with mdadm -D /dev/md0 and mdadm -E /dev/[b-j]

Re: RAID extremely slow

2012-07-31 Thread Bill Davidsen
Kevin Ross wrote: On 07/27/2012 09:45 PM, Grant Coady wrote: On Fri, 27 Jul 2012 14:45:18 -0700, you wrote: On 07/27/2012 12:08 PM, Bill Davidsen wrote: Have you set the io scheduler to deadline on all members of the array? That's kind of "job one" on older kernels. I have not, thanks for

Re: RAID extremely slow

2012-07-31 Thread Bill Davidsen
Kevin Ross wrote: On 07/27/2012 09:45 PM, Grant Coady wrote: On Fri, 27 Jul 2012 14:45:18 -0700, you wrote: On 07/27/2012 12:08 PM, Bill Davidsen wrote: Have you set the io scheduler to deadline on all members of the array? That's kind of job one on older kernels. I have not, thanks for

Re: RAID extremely slow

2012-07-28 Thread Kevin Ross
On 07/27/2012 09:45 PM, Grant Coady wrote: On Fri, 27 Jul 2012 14:45:18 -0700, you wrote: On 07/27/2012 12:08 PM, Bill Davidsen wrote: Have you set the io scheduler to deadline on all members of the array? That's kind of "job one" on older kernels. I have not, thanks for the tip, I'll look

Re: RAID extremely slow

2012-07-28 Thread Kevin Ross
On 07/27/2012 09:45 PM, Grant Coady wrote: On Fri, 27 Jul 2012 14:45:18 -0700, you wrote: On 07/27/2012 12:08 PM, Bill Davidsen wrote: Have you set the io scheduler to deadline on all members of the array? That's kind of job one on older kernels. I have not, thanks for the tip, I'll look

Re: RAID extremely slow

2012-07-27 Thread Grant Coady
On Fri, 27 Jul 2012 14:45:18 -0700, you wrote: >On 07/27/2012 12:08 PM, Bill Davidsen wrote: >> Have you set the io scheduler to deadline on all members of the array? >> That's kind of "job one" on older kernels. >> > >I have not, thanks for the tip, I'll look into that now. Plus I disable the

Re: RAID extremely slow

2012-07-27 Thread Kevin Ross
On 07/27/2012 12:08 PM, Bill Davidsen wrote: Have you set the io scheduler to deadline on all members of the array? That's kind of "job one" on older kernels. I have not, thanks for the tip, I'll look into that now. Thanks! -- Kevin -- To unsubscribe from this list: send the line

Re: RAID extremely slow

2012-07-27 Thread Bill Davidsen
Kevin Ross wrote: unused devices: # cat /proc/sys/dev/raid/speed_limit_min 1 MD is unable to reach its minimum rebuild rate while other system activity is ongoing. You might want to lower this number to see if that gets you out of the stalls. Or temporarily shut down mythtv. I will

Re: RAID extremely slow

2012-07-27 Thread Bill Davidsen
Kevin Ross wrote: unused devices:none # cat /proc/sys/dev/raid/speed_limit_min 1 MD is unable to reach its minimum rebuild rate while other system activity is ongoing. You might want to lower this number to see if that gets you out of the stalls. Or temporarily shut down mythtv. I

Re: RAID extremely slow

2012-07-27 Thread Kevin Ross
On 07/27/2012 12:08 PM, Bill Davidsen wrote: Have you set the io scheduler to deadline on all members of the array? That's kind of job one on older kernels. I have not, thanks for the tip, I'll look into that now. Thanks! -- Kevin -- To unsubscribe from this list: send the line unsubscribe

Re: RAID extremely slow

2012-07-27 Thread Grant Coady
On Fri, 27 Jul 2012 14:45:18 -0700, you wrote: On 07/27/2012 12:08 PM, Bill Davidsen wrote: Have you set the io scheduler to deadline on all members of the array? That's kind of job one on older kernels. I have not, thanks for the tip, I'll look into that now. Plus I disable the on-drive

Re: RAID extremely slow

2012-07-26 Thread Kevin Ross
On 07/26/2012 07:53 PM, Kevin Ross wrote: On 07/26/2012 07:27 PM, David Dillow wrote: On Thu, 2012-07-26 at 19:17 -0700, Kevin Ross wrote: On 07/26/2012 07:17 PM, David Dillow wrote: On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: If you can, upgrade to the latest 3.4 stable kernel

Re: RAID extremely slow

2012-07-26 Thread David Dillow
On Wed, 2012-07-25 at 18:55 -0700, Kevin Ross wrote: > On 07/25/2012 06:00 PM, Phil Turmel wrote: > > Piles of small reads scattered across multiple drives, and a > > concentration of queued writes to /dev/sda. What's on /dev/sda? > > It's not a member of the raid, so it must be some other

Re: RAID extremely slow

2012-07-26 Thread David Dillow
On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: > If you can, upgrade to the latest 3.4 stable kernel (3.4.6 right now). > As far as I can see, the latest 3.2 stable does not contain the delayed > stripe fix. And I was looking at the wrong version; 3.2.24 does indeed have the fix. -- To

Re: RAID extremely slow

2012-07-26 Thread Kevin Ross
On 07/26/2012 07:27 PM, David Dillow wrote: On Thu, 2012-07-26 at 19:17 -0700, Kevin Ross wrote: On 07/26/2012 07:17 PM, David Dillow wrote: On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: If you can, upgrade to the latest 3.4 stable kernel (3.4.6 right now). As far as I can see, the

Re: RAID extremely slow

2012-07-26 Thread David Dillow
On Thu, 2012-07-26 at 19:17 -0700, Kevin Ross wrote: > On 07/26/2012 07:17 PM, David Dillow wrote: > > On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: > >> If you can, upgrade to the latest 3.4 stable kernel (3.4.6 right now). > >> As far as I can see, the latest 3.2 stable does not contain

Re: RAID extremely slow

2012-07-26 Thread Kevin Ross
On 07/26/2012 07:17 PM, David Dillow wrote: On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: If you can, upgrade to the latest 3.4 stable kernel (3.4.6 right now). As far as I can see, the latest 3.2 stable does not contain the delayed stripe fix. And I was looking at the wrong version;

Re: RAID extremely slow

2012-07-26 Thread Kevin Ross
On 07/25/2012 10:00 PM, Kevin Ross wrote: unused devices: # cat /proc/sys/dev/raid/speed_limit_min 1 MD is unable to reach its minimum rebuild rate while other system activity is ongoing. You might want to lower this number to see if that gets you out of the stalls. Or temporarily

Re: RAID extremely slow

2012-07-26 Thread Kevin Ross
On 07/25/2012 10:00 PM, Kevin Ross wrote: unused devices:none # cat /proc/sys/dev/raid/speed_limit_min 1 MD is unable to reach its minimum rebuild rate while other system activity is ongoing. You might want to lower this number to see if that gets you out of the stalls. Or

Re: RAID extremely slow

2012-07-26 Thread Kevin Ross
On 07/26/2012 07:17 PM, David Dillow wrote: On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: If you can, upgrade to the latest 3.4 stable kernel (3.4.6 right now). As far as I can see, the latest 3.2 stable does not contain the delayed stripe fix. And I was looking at the wrong version;

Re: RAID extremely slow

2012-07-26 Thread David Dillow
On Thu, 2012-07-26 at 19:17 -0700, Kevin Ross wrote: On 07/26/2012 07:17 PM, David Dillow wrote: On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: If you can, upgrade to the latest 3.4 stable kernel (3.4.6 right now). As far as I can see, the latest 3.2 stable does not contain the

Re: RAID extremely slow

2012-07-26 Thread Kevin Ross
On 07/26/2012 07:27 PM, David Dillow wrote: On Thu, 2012-07-26 at 19:17 -0700, Kevin Ross wrote: On 07/26/2012 07:17 PM, David Dillow wrote: On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: If you can, upgrade to the latest 3.4 stable kernel (3.4.6 right now). As far as I can see, the

Re: RAID extremely slow

2012-07-26 Thread David Dillow
On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: If you can, upgrade to the latest 3.4 stable kernel (3.4.6 right now). As far as I can see, the latest 3.2 stable does not contain the delayed stripe fix. And I was looking at the wrong version; 3.2.24 does indeed have the fix. -- To

Re: RAID extremely slow

2012-07-26 Thread David Dillow
On Wed, 2012-07-25 at 18:55 -0700, Kevin Ross wrote: On 07/25/2012 06:00 PM, Phil Turmel wrote: Piles of small reads scattered across multiple drives, and a concentration of queued writes to /dev/sda. What's on /dev/sda? It's not a member of the raid, so it must be some other system task

Re: RAID extremely slow

2012-07-26 Thread Kevin Ross
On 07/26/2012 07:53 PM, Kevin Ross wrote: On 07/26/2012 07:27 PM, David Dillow wrote: On Thu, 2012-07-26 at 19:17 -0700, Kevin Ross wrote: On 07/26/2012 07:17 PM, David Dillow wrote: On Thu, 2012-07-26 at 22:15 -0400, David Dillow wrote: If you can, upgrade to the latest 3.4 stable kernel

Re: RAID extremely slow

2012-07-25 Thread Kevin Ross
unused devices: # cat /proc/sys/dev/raid/speed_limit_min 1 MD is unable to reach its minimum rebuild rate while other system activity is ongoing. You might want to lower this number to see if that gets you out of the stalls. Or temporarily shut down mythtv. I will try lowering those

Re: RAID extremely slow

2012-07-25 Thread Kevin Ross
On 07/25/2012 07:09 PM, CoolCold wrote: You might be interested in write intent bitmap then, it gonna help a lot. (resending in plain text) Thanks, I'll look into that! -- Kevin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to

Re: RAID extremely slow

2012-07-25 Thread CoolCold
On Thu, Jul 26, 2012 at 5:55 AM, Kevin Ross wrote: > > Thank you very much for taking the time to look into this. > > > On 07/25/2012 06:00 PM, Phil Turmel wrote: >> >> Piles of small reads scattered across multiple drives, and a >> concentration of queued writes to /dev/sda. What's on

Re: RAID extremely slow

2012-07-25 Thread Kevin Ross
Thank you very much for taking the time to look into this. On 07/25/2012 06:00 PM, Phil Turmel wrote: Piles of small reads scattered across multiple drives, and a concentration of queued writes to /dev/sda. What's on /dev/sda? It's not a member of the raid, so it must be some other system

Re: RAID extremely slow

2012-07-25 Thread Phil Turmel
[Added linux-raid to the CC] Hi Kevin, Notes interleaved: On 07/25/2012 06:52 PM, Kevin Ross wrote: > Hello, > > I'm having a problem. After a while, my software RAID rebuild becomes > extremely slow, and the filesystem on the RAID is essentially blocked. > I don't know what is causing this.

RAID extremely slow

2012-07-25 Thread Kevin Ross
Hello, I'm having a problem. After a while, my software RAID rebuild becomes extremely slow, and the filesystem on the RAID is essentially blocked. I don't know what is causing this. I guess it could be a bad drive, but how can I find out? I used atop to show the transfer speeds to each

RAID extremely slow

2012-07-25 Thread Kevin Ross
Hello, I'm having a problem. After a while, my software RAID rebuild becomes extremely slow, and the filesystem on the RAID is essentially blocked. I don't know what is causing this. I guess it could be a bad drive, but how can I find out? I used atop to show the transfer speeds to each

Re: RAID extremely slow

2012-07-25 Thread Phil Turmel
[Added linux-raid to the CC] Hi Kevin, Notes interleaved: On 07/25/2012 06:52 PM, Kevin Ross wrote: Hello, I'm having a problem. After a while, my software RAID rebuild becomes extremely slow, and the filesystem on the RAID is essentially blocked. I don't know what is causing this. I

Re: RAID extremely slow

2012-07-25 Thread Kevin Ross
Thank you very much for taking the time to look into this. On 07/25/2012 06:00 PM, Phil Turmel wrote: Piles of small reads scattered across multiple drives, and a concentration of queued writes to /dev/sda. What's on /dev/sda? It's not a member of the raid, so it must be some other system

Re: RAID extremely slow

2012-07-25 Thread CoolCold
On Thu, Jul 26, 2012 at 5:55 AM, Kevin Ross ke...@familyross.net wrote: Thank you very much for taking the time to look into this. On 07/25/2012 06:00 PM, Phil Turmel wrote: Piles of small reads scattered across multiple drives, and a concentration of queued writes to /dev/sda. What's on

Re: RAID extremely slow

2012-07-25 Thread Kevin Ross
On 07/25/2012 07:09 PM, CoolCold wrote: You might be interested in write intent bitmap then, it gonna help a lot. (resending in plain text) Thanks, I'll look into that! -- Kevin -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to

Re: RAID extremely slow

2012-07-25 Thread Kevin Ross
unused devices:none # cat /proc/sys/dev/raid/speed_limit_min 1 MD is unable to reach its minimum rebuild rate while other system activity is ongoing. You might want to lower this number to see if that gets you out of the stalls. Or temporarily shut down mythtv. I will try lowering