Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-02 Thread Jens Axboe
On 05/02/2014 10:43 AM, Christoph Hellwig wrote: > On Fri, May 02, 2014 at 10:41:40AM -0600, Jens Axboe wrote: >>> At least for SCSI devices _tag space_ is plenty, it's just the we >>> artifically limit our tag space to the queue depth to avoid having to >>> track that one separately. In addition

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-02 Thread Christoph Hellwig
On Fri, May 02, 2014 at 10:41:40AM -0600, Jens Axboe wrote: > > At least for SCSI devices _tag space_ is plenty, it's just the we > > artifically limit our tag space to the queue depth to avoid having to > > track that one separately. In addition we also preallocaste a request > > for each tag,

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-02 Thread Jens Axboe
On 05/01/2014 11:05 PM, Christoph Hellwig wrote: > On Thu, May 01, 2014 at 08:19:39PM -0600, Jens Axboe wrote: >> I've taken the consequence of this and implemented another tagging >> scheme that blk-mq will use if it deems that percpu_ida isn't going >> to be effective for the device being

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-02 Thread Jens Axboe
On 05/01/2014 11:05 PM, Christoph Hellwig wrote: On Thu, May 01, 2014 at 08:19:39PM -0600, Jens Axboe wrote: I've taken the consequence of this and implemented another tagging scheme that blk-mq will use if it deems that percpu_ida isn't going to be effective for the device being initialized.

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-02 Thread Christoph Hellwig
On Fri, May 02, 2014 at 10:41:40AM -0600, Jens Axboe wrote: At least for SCSI devices _tag space_ is plenty, it's just the we artifically limit our tag space to the queue depth to avoid having to track that one separately. In addition we also preallocaste a request for each tag, so even

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-02 Thread Jens Axboe
On 05/02/2014 10:43 AM, Christoph Hellwig wrote: On Fri, May 02, 2014 at 10:41:40AM -0600, Jens Axboe wrote: At least for SCSI devices _tag space_ is plenty, it's just the we artifically limit our tag space to the queue depth to avoid having to track that one separately. In addition we also

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Christoph Hellwig
On Thu, May 01, 2014 at 08:19:39PM -0600, Jens Axboe wrote: > I've taken the consequence of this and implemented another tagging > scheme that blk-mq will use if it deems that percpu_ida isn't going > to be effective for the device being initialized. But I really hate > to have both of them in

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Jens Axboe
On 2014-05-01 20:38, Kent Overstreet wrote: On Thu, May 01, 2014 at 08:19:39PM -0600, Jens Axboe wrote: On 2014-05-01 16:47, Kent Overstreet wrote: On Tue, Apr 29, 2014 at 03:13:38PM -0600, Jens Axboe wrote: On 04/29/2014 05:35 AM, Ming Lei wrote: On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Kent Overstreet
On Thu, May 01, 2014 at 08:19:39PM -0600, Jens Axboe wrote: > On 2014-05-01 16:47, Kent Overstreet wrote: > >On Tue, Apr 29, 2014 at 03:13:38PM -0600, Jens Axboe wrote: > >>On 04/29/2014 05:35 AM, Ming Lei wrote: > >>>On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe wrote: > On 2014-04-25 18:01,

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Jens Axboe
On 2014-05-01 16:47, Kent Overstreet wrote: On Tue, Apr 29, 2014 at 03:13:38PM -0600, Jens Axboe wrote: On 04/29/2014 05:35 AM, Ming Lei wrote: On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe wrote: On 2014-04-25 18:01, Ming Lei wrote: Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Kent Overstreet
On Tue, Apr 29, 2014 at 03:13:38PM -0600, Jens Axboe wrote: > On 04/29/2014 05:35 AM, Ming Lei wrote: > > On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe wrote: > >> On 2014-04-25 18:01, Ming Lei wrote: > >>> > >>> Hi Jens, > >>> > >>> On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe wrote: > >

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Alexander Gordeev
On Thu, May 01, 2014 at 11:24:42PM +0200, Alexander Gordeev wrote: > On Tue, Apr 22, 2014 at 01:56:29PM +0200, Peter Zijlstra wrote: > > I've not had time to revisit/finish them, but you should definitely > > clean up the percpu_ida stuff and reduce existing contention before > > going overboard.

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Alexander Gordeev
On Tue, Apr 22, 2014 at 01:56:29PM +0200, Peter Zijlstra wrote: > I've not had time to revisit/finish them, but you should definitely > clean up the percpu_ida stuff and reduce existing contention before > going overboard. Hi Peter, I tried to combine your series into a single patch against

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Alexander Gordeev
On Tue, Apr 22, 2014 at 01:56:29PM +0200, Peter Zijlstra wrote: I've not had time to revisit/finish them, but you should definitely clean up the percpu_ida stuff and reduce existing contention before going overboard. Hi Peter, I tried to combine your series into a single patch against

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Alexander Gordeev
On Thu, May 01, 2014 at 11:24:42PM +0200, Alexander Gordeev wrote: On Tue, Apr 22, 2014 at 01:56:29PM +0200, Peter Zijlstra wrote: I've not had time to revisit/finish them, but you should definitely clean up the percpu_ida stuff and reduce existing contention before going overboard. Hi

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Kent Overstreet
On Tue, Apr 29, 2014 at 03:13:38PM -0600, Jens Axboe wrote: On 04/29/2014 05:35 AM, Ming Lei wrote: On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe ax...@kernel.dk wrote: On 2014-04-25 18:01, Ming Lei wrote: Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe ax...@kernel.dk wrote:

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Jens Axboe
On 2014-05-01 16:47, Kent Overstreet wrote: On Tue, Apr 29, 2014 at 03:13:38PM -0600, Jens Axboe wrote: On 04/29/2014 05:35 AM, Ming Lei wrote: On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe ax...@kernel.dk wrote: On 2014-04-25 18:01, Ming Lei wrote: Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM,

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Kent Overstreet
On Thu, May 01, 2014 at 08:19:39PM -0600, Jens Axboe wrote: On 2014-05-01 16:47, Kent Overstreet wrote: On Tue, Apr 29, 2014 at 03:13:38PM -0600, Jens Axboe wrote: On 04/29/2014 05:35 AM, Ming Lei wrote: On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe ax...@kernel.dk wrote: On 2014-04-25 18:01,

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Jens Axboe
On 2014-05-01 20:38, Kent Overstreet wrote: On Thu, May 01, 2014 at 08:19:39PM -0600, Jens Axboe wrote: On 2014-05-01 16:47, Kent Overstreet wrote: On Tue, Apr 29, 2014 at 03:13:38PM -0600, Jens Axboe wrote: On 04/29/2014 05:35 AM, Ming Lei wrote: On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-05-01 Thread Christoph Hellwig
On Thu, May 01, 2014 at 08:19:39PM -0600, Jens Axboe wrote: I've taken the consequence of this and implemented another tagging scheme that blk-mq will use if it deems that percpu_ida isn't going to be effective for the device being initialized. But I really hate to have both of them in there.

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-30 Thread Ming Lei
On Wed, Apr 30, 2014 at 5:13 AM, Jens Axboe wrote: > On 04/29/2014 05:35 AM, Ming Lei wrote: >> On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe wrote: >>> On 2014-04-25 18:01, Ming Lei wrote: Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe wrote: > > On 04/25/2014

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-30 Thread Ming Lei
On Wed, Apr 30, 2014 at 5:13 AM, Jens Axboe ax...@kernel.dk wrote: On 04/29/2014 05:35 AM, Ming Lei wrote: On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe ax...@kernel.dk wrote: On 2014-04-25 18:01, Ming Lei wrote: Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe ax...@kernel.dk wrote:

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-29 Thread Jens Axboe
On 04/29/2014 05:35 AM, Ming Lei wrote: > On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe wrote: >> On 2014-04-25 18:01, Ming Lei wrote: >>> >>> Hi Jens, >>> >>> On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe wrote: On 04/25/2014 03:10 AM, Ming Lei wrote: Sorry, I did run it the

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-29 Thread Ming Lei
On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe wrote: > On 2014-04-25 18:01, Ming Lei wrote: >> >> Hi Jens, >> >> On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe wrote: >>> >>> On 04/25/2014 03:10 AM, Ming Lei wrote: >>> >>> Sorry, I did run it the other day. It has little to no effect here, but >>>

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-29 Thread Ming Lei
On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe ax...@kernel.dk wrote: On 2014-04-25 18:01, Ming Lei wrote: Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe ax...@kernel.dk wrote: On 04/25/2014 03:10 AM, Ming Lei wrote: Sorry, I did run it the other day. It has little to no effect here,

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-29 Thread Jens Axboe
On 04/29/2014 05:35 AM, Ming Lei wrote: On Sat, Apr 26, 2014 at 10:03 AM, Jens Axboe ax...@kernel.dk wrote: On 2014-04-25 18:01, Ming Lei wrote: Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe ax...@kernel.dk wrote: On 04/25/2014 03:10 AM, Ming Lei wrote: Sorry, I did run it the

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-25 Thread Jens Axboe
On 2014-04-25 18:01, Ming Lei wrote: Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe wrote: On 04/25/2014 03:10 AM, Ming Lei wrote: Sorry, I did run it the other day. It has little to no effect here, but that's mostly because there's so much other crap going on in there. The most

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-25 Thread Ming Lei
Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe wrote: > On 04/25/2014 03:10 AM, Ming Lei wrote: > > Sorry, I did run it the other day. It has little to no effect here, but > that's mostly because there's so much other crap going on in there. The > most effective way to currently make it

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-25 Thread Jens Axboe
On 04/25/2014 03:10 AM, Ming Lei wrote: > Hi Jens, > > On Wed, Apr 23, 2014 at 9:25 AM, Jens Axboe wrote: >> On 2014-04-22 18:53, Ming Lei wrote: >> >> >>> In my null_blk test on a quad core SMP VM: >>> >>> - 4 hw queue >>> - timer mode >>> >>> With the above approach, tag allocation

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-25 Thread Ming Lei
Hi Jens, On Wed, Apr 23, 2014 at 9:25 AM, Jens Axboe wrote: > On 2014-04-22 18:53, Ming Lei wrote: > > >> In my null_blk test on a quad core SMP VM: >> >> - 4 hw queue >> - timer mode >> >> With the above approach, tag allocation from local CPU can be >> improved from: >> >>

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-25 Thread Ming Lei
Hi Jens, On Wed, Apr 23, 2014 at 9:25 AM, Jens Axboe ax...@kernel.dk wrote: On 2014-04-22 18:53, Ming Lei wrote: In my null_blk test on a quad core SMP VM: - 4 hw queue - timer mode With the above approach, tag allocation from local CPU can be improved from: 5%

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-25 Thread Jens Axboe
On 04/25/2014 03:10 AM, Ming Lei wrote: Hi Jens, On Wed, Apr 23, 2014 at 9:25 AM, Jens Axboe ax...@kernel.dk wrote: On 2014-04-22 18:53, Ming Lei wrote: In my null_blk test on a quad core SMP VM: - 4 hw queue - timer mode With the above approach, tag allocation from local

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-25 Thread Ming Lei
Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe ax...@kernel.dk wrote: On 04/25/2014 03:10 AM, Ming Lei wrote: Sorry, I did run it the other day. It has little to no effect here, but that's mostly because there's so much other crap going on in there. The most effective way to currently

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-25 Thread Jens Axboe
On 2014-04-25 18:01, Ming Lei wrote: Hi Jens, On Sat, Apr 26, 2014 at 5:23 AM, Jens Axboe ax...@kernel.dk wrote: On 04/25/2014 03:10 AM, Ming Lei wrote: Sorry, I did run it the other day. It has little to no effect here, but that's mostly because there's so much other crap going on in there.

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Jens Axboe
On 2014-04-22 18:53, Ming Lei wrote: Hi Jens, On Tue, Apr 22, 2014 at 11:57 PM, Jens Axboe wrote: On 04/22/2014 08:03 AM, Jens Axboe wrote: On 2014-04-22 01:10, Alexander Gordeev wrote: On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: But other systems (more dense?)

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Ming Lei
Hi Jens, On Tue, Apr 22, 2014 at 11:57 PM, Jens Axboe wrote: > On 04/22/2014 08:03 AM, Jens Axboe wrote: >> On 2014-04-22 01:10, Alexander Gordeev wrote: >>> On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: But other systems (more dense?) showed increased cache-hit rate

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Peter Zijlstra
On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: > Hello, > > This series is against 3.14.0-rc7. > > It is amied to further improve 'percpu_ida' tags locality by taking > into account system's CPU topology when stealing tags. That is try > to steal from a CPU which is 'closest'

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Jens Axboe
On 04/22/2014 08:03 AM, Jens Axboe wrote: > On 2014-04-22 01:10, Alexander Gordeev wrote: >> On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: >>> But other systems (more dense?) showed increased cache-hit rate >>> up to 20%, i.e. this one: >> >> Hello Gentlemen, >> >> Any

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Jens Axboe
On 2014-04-22 01:10, Alexander Gordeev wrote: On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: But other systems (more dense?) showed increased cache-hit rate up to 20%, i.e. this one: Hello Gentlemen, Any feedback on this? Sorry for dropping the ball on this.

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Alexander Gordeev
On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: > But other systems (more dense?) showed increased cache-hit rate > up to 20%, i.e. this one: Hello Gentlemen, Any feedback on this? Thanks! -- Regards, Alexander Gordeev agord...@redhat.com -- To unsubscribe from this list:

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Alexander Gordeev
On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: But other systems (more dense?) showed increased cache-hit rate up to 20%, i.e. this one: Hello Gentlemen, Any feedback on this? Thanks! -- Regards, Alexander Gordeev agord...@redhat.com -- To unsubscribe from this list:

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Jens Axboe
On 2014-04-22 01:10, Alexander Gordeev wrote: On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: But other systems (more dense?) showed increased cache-hit rate up to 20%, i.e. this one: Hello Gentlemen, Any feedback on this? Sorry for dropping the ball on this.

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Jens Axboe
On 04/22/2014 08:03 AM, Jens Axboe wrote: On 2014-04-22 01:10, Alexander Gordeev wrote: On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: But other systems (more dense?) showed increased cache-hit rate up to 20%, i.e. this one: Hello Gentlemen, Any feedback on this?

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Peter Zijlstra
On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: Hello, This series is against 3.14.0-rc7. It is amied to further improve 'percpu_ida' tags locality by taking into account system's CPU topology when stealing tags. That is try to steal from a CPU which is 'closest' to the

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Ming Lei
Hi Jens, On Tue, Apr 22, 2014 at 11:57 PM, Jens Axboe ax...@kernel.dk wrote: On 04/22/2014 08:03 AM, Jens Axboe wrote: On 2014-04-22 01:10, Alexander Gordeev wrote: On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: But other systems (more dense?) showed increased cache-hit

Re: [PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-04-22 Thread Jens Axboe
On 2014-04-22 18:53, Ming Lei wrote: Hi Jens, On Tue, Apr 22, 2014 at 11:57 PM, Jens Axboe ax...@kernel.dk wrote: On 04/22/2014 08:03 AM, Jens Axboe wrote: On 2014-04-22 01:10, Alexander Gordeev wrote: On Wed, Mar 26, 2014 at 02:34:22PM +0100, Alexander Gordeev wrote: But other systems

[PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-03-26 Thread Alexander Gordeev
Hello, This series is against 3.14.0-rc7. It is amied to further improve 'percpu_ida' tags locality by taking into account system's CPU topology when stealing tags. That is try to steal from a CPU which is 'closest' to the stealing one. I would not bother to post this, since on several system

[PATCH RFC 0/2] percpu_ida: Take into account CPU topology when stealing tags

2014-03-26 Thread Alexander Gordeev
Hello, This series is against 3.14.0-rc7. It is amied to further improve 'percpu_ida' tags locality by taking into account system's CPU topology when stealing tags. That is try to steal from a CPU which is 'closest' to the stealing one. I would not bother to post this, since on several system