Re: [rfc] lockless pagecache
On Tue, Jul 05, 2005 at 08:31:40AM -0700, Martin J. Bligh wrote: > >> > On Mon, 27 Jun 2005, Chen, Kenneth W wrote: > >> > > I don't recall seeing tree_lock to be a problem for DSS workload > >> > > either. > >> > > >> > I have seen the tree_lock being a problem a number of times with large > >> > scale NUMA type workloads. > >> > >> I totally agree! My earlier posts are strictly referring to industry > >> standard db workloads (OLTP, DSS). I'm not saying it's not a problem > >> for everyone :-) Obviously you just outlined a few > > > > I'm a bit late to the party here (was gone on vacation), but I do have > > profiles from DSS workloads using page-cache rather than O_DIRECT and > > I do see spin_lock_irq() in the profiles which I'm pretty certain are > > locks spinning for access to the radix_tree. I'll talk about it a bit > > more up in Ottawa but here's the top 5 on my profile (sorry don't have > > the number of ticks at the momement): > > > > 1. dedicated_idle (waiting for I/O) > > 2. __copy_tofrom_user > > 3. radix_tree_delete > > 4. _spin_lock_irq > > 5. __find_get_block > > > > So, yes, if the page-cache is used in a DSS workload then one will see > > the tree-lock. BTW, this was on a PPC64 machine w/ a fairly small > > NUMA factor. > > The easiest way to confirm the spin-lock thing is to recompile with > CONFIG_SPINLINE, and take a new profile, then diff the two ... Yep... Unfortunately, this is broken in PPC64 since 2.6.9-rc2 or something like that, I never had a chance to track down what the issue was exactly. IIRC, there was a lot of churn in the spinlocking code around that time. Sonny - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [rfc] lockless pagecache
>> > On Mon, 27 Jun 2005, Chen, Kenneth W wrote: >> > > I don't recall seeing tree_lock to be a problem for DSS workload either. >> > >> > I have seen the tree_lock being a problem a number of times with large >> > scale NUMA type workloads. >> >> I totally agree! My earlier posts are strictly referring to industry >> standard db workloads (OLTP, DSS). I'm not saying it's not a problem >> for everyone :-) Obviously you just outlined a few > > I'm a bit late to the party here (was gone on vacation), but I do have > profiles from DSS workloads using page-cache rather than O_DIRECT and > I do see spin_lock_irq() in the profiles which I'm pretty certain are > locks spinning for access to the radix_tree. I'll talk about it a bit > more up in Ottawa but here's the top 5 on my profile (sorry don't have > the number of ticks at the momement): > > 1. dedicated_idle (waiting for I/O) > 2. __copy_tofrom_user > 3. radix_tree_delete > 4. _spin_lock_irq > 5. __find_get_block > > So, yes, if the page-cache is used in a DSS workload then one will see > the tree-lock. BTW, this was on a PPC64 machine w/ a fairly small > NUMA factor. The easiest way to confirm the spin-lock thing is to recompile with CONFIG_SPINLINE, and take a new profile, then diff the two ... M. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [rfc] lockless pagecache
On Mon, Jun 27, 2005 at 12:42:44PM -0700, Chen, Kenneth W wrote: > Christoph Lameter wrote on Monday, June 27, 2005 12:23 PM > > On Mon, 27 Jun 2005, Chen, Kenneth W wrote: > > > I don't recall seeing tree_lock to be a problem for DSS workload either. > > > > I have seen the tree_lock being a problem a number of times with large > > scale NUMA type workloads. > > I totally agree! My earlier posts are strictly referring to industry > standard db workloads (OLTP, DSS). I'm not saying it's not a problem > for everyone :-) Obviously you just outlined a few I'm a bit late to the party here (was gone on vacation), but I do have profiles from DSS workloads using page-cache rather than O_DIRECT and I do see spin_lock_irq() in the profiles which I'm pretty certain are locks spinning for access to the radix_tree. I'll talk about it a bit more up in Ottawa but here's the top 5 on my profile (sorry don't have the number of ticks at the momement): 1. dedicated_idle (waiting for I/O) 2. __copy_tofrom_user 3. radix_tree_delete 4. _spin_lock_irq 5. __find_get_block So, yes, if the page-cache is used in a DSS workload then one will see the tree-lock. BTW, this was on a PPC64 machine w/ a fairly small NUMA factor. Sonny - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [rfc] lockless pagecache
On Mon, Jun 27, 2005 at 12:42:44PM -0700, Chen, Kenneth W wrote: Christoph Lameter wrote on Monday, June 27, 2005 12:23 PM On Mon, 27 Jun 2005, Chen, Kenneth W wrote: I don't recall seeing tree_lock to be a problem for DSS workload either. I have seen the tree_lock being a problem a number of times with large scale NUMA type workloads. I totally agree! My earlier posts are strictly referring to industry standard db workloads (OLTP, DSS). I'm not saying it's not a problem for everyone :-) Obviously you just outlined a few I'm a bit late to the party here (was gone on vacation), but I do have profiles from DSS workloads using page-cache rather than O_DIRECT and I do see spin_lock_irq() in the profiles which I'm pretty certain are locks spinning for access to the radix_tree. I'll talk about it a bit more up in Ottawa but here's the top 5 on my profile (sorry don't have the number of ticks at the momement): 1. dedicated_idle (waiting for I/O) 2. __copy_tofrom_user 3. radix_tree_delete 4. _spin_lock_irq 5. __find_get_block So, yes, if the page-cache is used in a DSS workload then one will see the tree-lock. BTW, this was on a PPC64 machine w/ a fairly small NUMA factor. Sonny - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [rfc] lockless pagecache
On Mon, 27 Jun 2005, Chen, Kenneth W wrote: I don't recall seeing tree_lock to be a problem for DSS workload either. I have seen the tree_lock being a problem a number of times with large scale NUMA type workloads. I totally agree! My earlier posts are strictly referring to industry standard db workloads (OLTP, DSS). I'm not saying it's not a problem for everyone :-) Obviously you just outlined a few I'm a bit late to the party here (was gone on vacation), but I do have profiles from DSS workloads using page-cache rather than O_DIRECT and I do see spin_lock_irq() in the profiles which I'm pretty certain are locks spinning for access to the radix_tree. I'll talk about it a bit more up in Ottawa but here's the top 5 on my profile (sorry don't have the number of ticks at the momement): 1. dedicated_idle (waiting for I/O) 2. __copy_tofrom_user 3. radix_tree_delete 4. _spin_lock_irq 5. __find_get_block So, yes, if the page-cache is used in a DSS workload then one will see the tree-lock. BTW, this was on a PPC64 machine w/ a fairly small NUMA factor. The easiest way to confirm the spin-lock thing is to recompile with CONFIG_SPINLINE, and take a new profile, then diff the two ... M. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [rfc] lockless pagecache
On Tue, Jul 05, 2005 at 08:31:40AM -0700, Martin J. Bligh wrote: On Mon, 27 Jun 2005, Chen, Kenneth W wrote: I don't recall seeing tree_lock to be a problem for DSS workload either. I have seen the tree_lock being a problem a number of times with large scale NUMA type workloads. I totally agree! My earlier posts are strictly referring to industry standard db workloads (OLTP, DSS). I'm not saying it's not a problem for everyone :-) Obviously you just outlined a few I'm a bit late to the party here (was gone on vacation), but I do have profiles from DSS workloads using page-cache rather than O_DIRECT and I do see spin_lock_irq() in the profiles which I'm pretty certain are locks spinning for access to the radix_tree. I'll talk about it a bit more up in Ottawa but here's the top 5 on my profile (sorry don't have the number of ticks at the momement): 1. dedicated_idle (waiting for I/O) 2. __copy_tofrom_user 3. radix_tree_delete 4. _spin_lock_irq 5. __find_get_block So, yes, if the page-cache is used in a DSS workload then one will see the tree-lock. BTW, this was on a PPC64 machine w/ a fairly small NUMA factor. The easiest way to confirm the spin-lock thing is to recompile with CONFIG_SPINLINE, and take a new profile, then diff the two ... Yep... Unfortunately, this is broken in PPC64 since 2.6.9-rc2 or something like that, I never had a chance to track down what the issue was exactly. IIRC, there was a lot of churn in the spinlocking code around that time. Sonny - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
