Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-08-01 Thread Tejun Heo
Hello, On Thu, Aug 01, 2013 at 10:01:01AM +0800, Shaohua Li wrote: > It does have lock, but when a stripe is queued to handle, no lock is required. H sorry but can you please explain it a bit further? Why wouldn't it require a lock? Perhaps because it has to be on some queue already?

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-08-01 Thread Tejun Heo
Hello, On Thu, Aug 01, 2013 at 10:01:01AM +0800, Shaohua Li wrote: It does have lock, but when a stripe is queued to handle, no lock is required. H sorry but can you please explain it a bit further? Why wouldn't it require a lock? Perhaps because it has to be on some queue already?

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-31 Thread Shaohua Li
On Wed, Jul 31, 2013 at 06:33:32AM -0400, Tejun Heo wrote: > Hello, > > On Wed, Jul 31, 2013 at 09:24:34AM +0800, Shaohua Li wrote: > > stripe is the work unit actually. As I said, if I queue a work for each > > stripe, > > just queue_work() will make the system blast because of the

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-31 Thread Tejun Heo
Hello, On Wed, Jul 31, 2013 at 09:24:34AM +0800, Shaohua Li wrote: > stripe is the work unit actually. As I said, if I queue a work for each > stripe, > just queue_work() will make the system blast because of the pwq->pool->lock > contention. dispatching one work has another side effect that I

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-31 Thread Tejun Heo
Hello, On Wed, Jul 31, 2013 at 09:24:34AM +0800, Shaohua Li wrote: stripe is the work unit actually. As I said, if I queue a work for each stripe, just queue_work() will make the system blast because of the pwq-pool-lock contention. dispatching one work has another side effect that I can't

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-31 Thread Shaohua Li
On Wed, Jul 31, 2013 at 06:33:32AM -0400, Tejun Heo wrote: Hello, On Wed, Jul 31, 2013 at 09:24:34AM +0800, Shaohua Li wrote: stripe is the work unit actually. As I said, if I queue a work for each stripe, just queue_work() will make the system blast because of the pwq-pool-lock

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Shaohua Li
On Tue, Jul 30, 2013 at 09:57:51AM -0400, Tejun Heo wrote: > Hello, > > On Tue, Jul 30, 2013 at 09:07:08PM +0800, Shaohua Li wrote: > > Ok, I should explain here. I can't add a work_struct for each stripe, > > because > > this will stress workqueue very hard. My system handles > 1M/s stripes, >

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Tejun Heo
Hello, On Tue, Jul 30, 2013 at 09:07:08PM +0800, Shaohua Li wrote: > Ok, I should explain here. I can't add a work_struct for each stripe, because > this will stress workqueue very hard. My system handles > 1M/s stripes, which > makes workqueue pool lock contended very hard. It doesn't have to

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Shaohua Li
On Tue, Jul 30, 2013 at 08:53:06AM -0400, Tejun Heo wrote: > Hello, > > On Tue, Jul 30, 2013 at 01:52:08PM +0800, s...@kernel.org wrote: > > +static void raid5_wakeup_stripe_thread(struct stripe_head *sh) > > +{ > > + struct r5conf *conf = sh->raid_conf; > > + struct r5worker_group *group; >

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Tejun Heo
Hello, On Tue, Jul 30, 2013 at 01:52:08PM +0800, s...@kernel.org wrote: > +static void raid5_wakeup_stripe_thread(struct stripe_head *sh) > +{ > + struct r5conf *conf = sh->raid_conf; > + struct r5worker_group *group; > + int i; > + > + if (conf->worker_cnt_per_group == 0) { > +

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Tejun Heo
Hello, On Tue, Jul 30, 2013 at 01:52:08PM +0800, s...@kernel.org wrote: > static int __init raid5_init(void) > { > + raid5_wq = alloc_workqueue("raid5wq", > + WQ_NON_REENTRANT|WQ_UNBOUND|WQ_MEM_RECLAIM| > + WQ_CPU_INTENSIVE|WQ_SYSFS, 0); Workqueues are now

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Tejun Heo
Hello, On Tue, Jul 30, 2013 at 01:52:08PM +0800, s...@kernel.org wrote: static int __init raid5_init(void) { + raid5_wq = alloc_workqueue(raid5wq, + WQ_NON_REENTRANT|WQ_UNBOUND|WQ_MEM_RECLAIM| + WQ_CPU_INTENSIVE|WQ_SYSFS, 0); Workqueues are now always

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Tejun Heo
Hello, On Tue, Jul 30, 2013 at 01:52:08PM +0800, s...@kernel.org wrote: +static void raid5_wakeup_stripe_thread(struct stripe_head *sh) +{ + struct r5conf *conf = sh-raid_conf; + struct r5worker_group *group; + int i; + + if (conf-worker_cnt_per_group == 0) { +

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Shaohua Li
On Tue, Jul 30, 2013 at 08:53:06AM -0400, Tejun Heo wrote: Hello, On Tue, Jul 30, 2013 at 01:52:08PM +0800, s...@kernel.org wrote: +static void raid5_wakeup_stripe_thread(struct stripe_head *sh) +{ + struct r5conf *conf = sh-raid_conf; + struct r5worker_group *group; + int i;

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Tejun Heo
Hello, On Tue, Jul 30, 2013 at 09:07:08PM +0800, Shaohua Li wrote: Ok, I should explain here. I can't add a work_struct for each stripe, because this will stress workqueue very hard. My system handles 1M/s stripes, which makes workqueue pool lock contended very hard. It doesn't have to be

Re: [patch 1/3] raid5: offload stripe handle to workqueue

2013-07-30 Thread Shaohua Li
On Tue, Jul 30, 2013 at 09:57:51AM -0400, Tejun Heo wrote: Hello, On Tue, Jul 30, 2013 at 09:07:08PM +0800, Shaohua Li wrote: Ok, I should explain here. I can't add a work_struct for each stripe, because this will stress workqueue very hard. My system handles 1M/s stripes, which

[patch 1/3] raid5: offload stripe handle to workqueue

2013-07-29 Thread shli
This is another attempt to create multiple threads to handle raid5 stripes. This time I use workqueue. raid5 handles request (especially write) in stripe unit. A stripe is page size aligned/long and acrosses all disks. Writing to any disk sector, raid5 runs a state machine for the corresponding

[patch 1/3] raid5: offload stripe handle to workqueue

2013-07-29 Thread shli
This is another attempt to create multiple threads to handle raid5 stripes. This time I use workqueue. raid5 handles request (especially write) in stripe unit. A stripe is page size aligned/long and acrosses all disks. Writing to any disk sector, raid5 runs a state machine for the corresponding