[PATCH] Allow lazy unmapping by taking extra page references V3

2007-10-24 Thread David Chinner
Allow lazy unmapping of vmap()d regions be taking a reference on each page in the region being vmap()ed. The page references get released after the region is vunmap()ed, thereby ensuring we don't leave stray mappings on freed pages that could lead to problems pages being reallocated with incorrect

[PATCH] Allow lazy unmapping by taking extra page references V3

2007-10-24 Thread David Chinner
Allow lazy unmapping of vmap()d regions be taking a reference on each page in the region being vmap()ed. The page references get released after the region is vunmap()ed, thereby ensuring we don't leave stray mappings on freed pages that could lead to problems pages being reallocated with incorrect

Re: [PATCH] Allow lazy unmapping by taking extra page references V3

2007-10-24 Thread David Chinner
On Wed, Oct 24, 2007 at 03:46:16PM -0700, Jeremy Fitzhardinge wrote: David Chinner wrote: Version 3: - compile on latest -git Not quite: CC mm/vmalloc.o /home/jeremy/hg/xen/paravirt/linux/mm/vmalloc.c: In function 'vm_area_alloc_pagearray': /home/jeremy/hg/xen/paravirt

[PATCH] Allow lazy unmapping by taking extra page references V2

2007-10-23 Thread David Chinner
Allow lazy unmapping of vmap()d regions be taking a reference on each page in the region being vmap()ed. The page references get released after the region is vunmap()ed, thereby ensuring we don't leave stray mappings on freed pages that could lead to problems pages being reallocated with

Re: [patch] Re: Interaction between Xen and XFS: stray RW mappings

2007-10-23 Thread David Chinner
On Tue, Oct 23, 2007 at 11:30:35AM +0200, Andi Kleen wrote: > On Tue, Oct 23, 2007 at 05:04:14PM +1000, David Chinner wrote: > > > You mean like vmap() could record the pages passed to it in the > > > area->pages > > > array, and we walk and release than i

[patch] Re: Interaction between Xen and XFS: stray RW mappings

2007-10-23 Thread David Chinner
On Tue, Oct 23, 2007 at 10:36:41AM +1000, David Chinner wrote: > On Tue, Oct 23, 2007 at 01:35:14AM +0200, Andi Kleen wrote: > > On Tue, Oct 23, 2007 at 08:32:25AM +1000, David Chinner wrote: > > > Could vmap()/vunmap() take references to the pages that are mapped? That

[patch] Re: Interaction between Xen and XFS: stray RW mappings

2007-10-23 Thread David Chinner
On Tue, Oct 23, 2007 at 10:36:41AM +1000, David Chinner wrote: On Tue, Oct 23, 2007 at 01:35:14AM +0200, Andi Kleen wrote: On Tue, Oct 23, 2007 at 08:32:25AM +1000, David Chinner wrote: Could vmap()/vunmap() take references to the pages that are mapped? That way delaying the unmap would

Re: [patch] Re: Interaction between Xen and XFS: stray RW mappings

2007-10-23 Thread David Chinner
On Tue, Oct 23, 2007 at 11:30:35AM +0200, Andi Kleen wrote: On Tue, Oct 23, 2007 at 05:04:14PM +1000, David Chinner wrote: You mean like vmap() could record the pages passed to it in the area-pages array, and we walk and release than in __vunmap() like it already does for vfree

[PATCH] Allow lazy unmapping by taking extra page references V2

2007-10-23 Thread David Chinner
Allow lazy unmapping of vmap()d regions be taking a reference on each page in the region being vmap()ed. The page references get released after the region is vunmap()ed, thereby ensuring we don't leave stray mappings on freed pages that could lead to problems pages being reallocated with

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-22 Thread David Chinner
On Tue, Oct 23, 2007 at 01:35:14AM +0200, Andi Kleen wrote: > On Tue, Oct 23, 2007 at 08:32:25AM +1000, David Chinner wrote: > > On Mon, Oct 22, 2007 at 09:07:40PM +0200, Andi Kleen wrote: > > > It's hidden now so it causes any obvious failures any more. Just subtle > > &g

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-22 Thread David Chinner
On Mon, Oct 22, 2007 at 09:07:40PM +0200, Andi Kleen wrote: > On Mon, Oct 22, 2007 at 11:40:52AM -0700, Jeremy Fitzhardinge wrote: > > Andi Kleen wrote: > > > Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes: > > > > > >> Yes, that's precisely the problem. xfs does delay the unmap, leaving > >

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-22 Thread David Chinner
On Mon, Oct 22, 2007 at 09:07:40PM +0200, Andi Kleen wrote: On Mon, Oct 22, 2007 at 11:40:52AM -0700, Jeremy Fitzhardinge wrote: Andi Kleen wrote: Jeremy Fitzhardinge [EMAIL PROTECTED] writes: Yes, that's precisely the problem. xfs does delay the unmap, leaving stray mappings,

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-22 Thread David Chinner
On Tue, Oct 23, 2007 at 01:35:14AM +0200, Andi Kleen wrote: On Tue, Oct 23, 2007 at 08:32:25AM +1000, David Chinner wrote: On Mon, Oct 22, 2007 at 09:07:40PM +0200, Andi Kleen wrote: It's hidden now so it causes any obvious failures any more. Just subtle ones which is much worse

Re: [RFC][PATCH] block: Isolate the buffer cache in it's own mappings.

2007-10-21 Thread David Chinner
On Sun, Oct 21, 2007 at 02:24:46PM +1000, Nick Piggin wrote: > On Saturday 20 October 2007 07:27, Eric W. Biederman wrote: > > Currently only > > metadata is more or less in sync with the contents of /dev/hda1. > > It either is or it isn't, right? And it is, isn't it? (at least > for the common

Re: [RFC][PATCH] block: Isolate the buffer cache in it's own mappings.

2007-10-21 Thread David Chinner
On Sun, Oct 21, 2007 at 02:24:46PM +1000, Nick Piggin wrote: On Saturday 20 October 2007 07:27, Eric W. Biederman wrote: Currently only metadata is more or less in sync with the contents of /dev/hda1. It either is or it isn't, right? And it is, isn't it? (at least for the common

Re: More Large blocksize benchmarks

2007-10-15 Thread David Chinner
On Mon, Oct 15, 2007 at 08:22:31PM -0400, Chris Mason wrote: > Hello everyone, > > I'm stealing the cc list and reviving and old thread because I've > finally got some numbers to go along with the Btrfs variable blocksize > feature. The basic idea is to create a read/write interface to > map a

Re: [PATCH 12/12] xfs: eagerly remove vmap mappings to avoid upsetting Xen

2007-10-15 Thread David Chinner
gt; the page. > > This patch solves the problem in a brute force way, by making XFS > always eagerly unmap its mappings. David Chinner says this shouldn't > have any performance impact on filesystems with default block sizes; > it will only affect filesystems with large block sizes. Lo

Re: [PATCH 12/12] xfs: eagerly remove vmap mappings to avoid upsetting Xen

2007-10-15 Thread David Chinner
. This patch solves the problem in a brute force way, by making XFS always eagerly unmap its mappings. David Chinner says this shouldn't have any performance impact on filesystems with default block sizes; it will only affect filesystems with large block sizes. Looks fine, Jeremy. I'll pull

Re: More Large blocksize benchmarks

2007-10-15 Thread David Chinner
On Mon, Oct 15, 2007 at 08:22:31PM -0400, Chris Mason wrote: Hello everyone, I'm stealing the cc list and reviving and old thread because I've finally got some numbers to go along with the Btrfs variable blocksize feature. The basic idea is to create a read/write interface to map a range

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-14 Thread David Chinner
On Sun, Oct 14, 2007 at 09:18:17PM -0700, Jeremy Fitzhardinge wrote: > David Chinner wrote: > > With defaults - little effect as vmap should never be used. It's > > only when you start using larger block sizes for metadata that this > > becomes an issue. The CONFIG_XEN work

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-14 Thread David Chinner
On Sun, Oct 14, 2007 at 08:42:34PM -0700, Jeremy Fitzhardinge wrote: > Nick Piggin wrote: > > That's not going to > > happen for at least a cycle or two though, so in the meantime maybe > > an ifdef for that XFS vmap batching code would help? > > > > For now I've proposed a patch to simply

Re: [PATCH 01/52] CRED: Introduce a COW credentials record

2007-10-14 Thread David Chinner
On Fri, Oct 12, 2007 at 05:05:24PM +0100, David Howells wrote: > diff --git a/fs/xfs/xfs_acl.c b/fs/xfs/xfs_acl.c > index 4ca4beb..a460508 100644 > --- a/fs/xfs/xfs_acl.c > +++ b/fs/xfs/xfs_acl.c > @@ -383,7 +383,7 @@ xfs_acl_allow_set( > error = bhv_vop_getattr(vp, , 0, NULL); > if

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-14 Thread David Chinner
On Sun, Oct 14, 2007 at 04:12:20PM -0700, Jeremy Fitzhardinge wrote: > David Chinner wrote: > > You mean xfs_buf.c. > > > > Yes, sorry. > > > And yes, we delay unmapping pages until we have a batch of them > > to unmap. vmap and vunmap do not scale, so

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-14 Thread David Chinner
On Fri, Oct 12, 2007 at 09:58:43AM -0700, Jeremy Fitzhardinge wrote: > Hi Dave & other XFS folk, > > I'm tracking down a bug which appears to be a bad interaction between XFS > and Xen. It looks like XFS is holding RW mappings on free pages, which Xen > is trying to get an exclusive RO mapping

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-14 Thread David Chinner
On Fri, Oct 12, 2007 at 09:58:43AM -0700, Jeremy Fitzhardinge wrote: Hi Dave other XFS folk, I'm tracking down a bug which appears to be a bad interaction between XFS and Xen. It looks like XFS is holding RW mappings on free pages, which Xen is trying to get an exclusive RO mapping on so

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-14 Thread David Chinner
On Sun, Oct 14, 2007 at 04:12:20PM -0700, Jeremy Fitzhardinge wrote: David Chinner wrote: You mean xfs_buf.c. Yes, sorry. And yes, we delay unmapping pages until we have a batch of them to unmap. vmap and vunmap do not scale, so this is batching helps alleviate some of the worst

Re: [PATCH 01/52] CRED: Introduce a COW credentials record

2007-10-14 Thread David Chinner
On Fri, Oct 12, 2007 at 05:05:24PM +0100, David Howells wrote: diff --git a/fs/xfs/xfs_acl.c b/fs/xfs/xfs_acl.c index 4ca4beb..a460508 100644 --- a/fs/xfs/xfs_acl.c +++ b/fs/xfs/xfs_acl.c @@ -383,7 +383,7 @@ xfs_acl_allow_set( error = bhv_vop_getattr(vp, va, 0, NULL); if (error)

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-14 Thread David Chinner
On Sun, Oct 14, 2007 at 08:42:34PM -0700, Jeremy Fitzhardinge wrote: Nick Piggin wrote: That's not going to happen for at least a cycle or two though, so in the meantime maybe an ifdef for that XFS vmap batching code would help? For now I've proposed a patch to simply eagerly

Re: Interaction between Xen and XFS: stray RW mappings

2007-10-14 Thread David Chinner
On Sun, Oct 14, 2007 at 09:18:17PM -0700, Jeremy Fitzhardinge wrote: David Chinner wrote: With defaults - little effect as vmap should never be used. It's only when you start using larger block sizes for metadata that this becomes an issue. The CONFIG_XEN workaround should be fine until we

Re: RFC: reviewer's statement of oversight

2007-10-09 Thread David Chinner
On Tue, Oct 09, 2007 at 10:49:20AM -0600, Jonathan Corbet wrote: > Neil Brown <[EMAIL PROTECTED]> wrote: > > > + (b) Any problems, concerns, or questions relating to the patch have been > > > + communicated back to the submitter. I am satisfied with how the > > > + submitter has responded

Re: RFC: reviewer's statement of oversight

2007-10-09 Thread David Chinner
On Tue, Oct 09, 2007 at 10:49:20AM -0600, Jonathan Corbet wrote: Neil Brown [EMAIL PROTECTED] wrote: + (b) Any problems, concerns, or questions relating to the patch have been + communicated back to the submitter. I am satisfied with how the + submitter has responded to my

Re: [PATCH] mm: set_page_dirty_balance() vs ->page_mkwrite()

2007-10-08 Thread David Chinner
On Mon, Oct 08, 2007 at 04:37:00PM +1000, Nick Piggin wrote: > On Tuesday 09 October 2007 02:54, Peter Zijlstra wrote: > > It seems that with the recent usage of ->page_mkwrite() a little detail > > was overlooked. > > > > .22-rc1 merged OCFS2 usage of this hook > > .23-rc1 merged XFS usage > >

Re: [PATCH] mm: set_page_dirty_balance() vs -page_mkwrite()

2007-10-08 Thread David Chinner
On Mon, Oct 08, 2007 at 04:37:00PM +1000, Nick Piggin wrote: On Tuesday 09 October 2007 02:54, Peter Zijlstra wrote: It seems that with the recent usage of -page_mkwrite() a little detail was overlooked. .22-rc1 merged OCFS2 usage of this hook .23-rc1 merged XFS usage .24-rc1 will

Re: XFS internal error

2007-10-07 Thread David Chinner
[please cc [EMAIL PROTECTED] on XFS bug reports. thx.] On Sun, Oct 07, 2007 at 09:09:58AM +0800, Max Waterman wrote: > Hi, > > I have just had an XFS error occur while deleting some directory > hierarchy. I hope this is the correct place to report it. . > This is in syslog : > > > Oct 6

Re: [PATCH] remove throttle_vm_writeout()

2007-10-07 Thread David Chinner
On Fri, Oct 05, 2007 at 08:30:28PM +0800, Fengguang Wu wrote: > The improvement could be: > - kswapd is now explicitly preferred to do the writeout; Careful. kswapd is much less efficient at writeout than pdflush because it does not do low->high offset writeback per address space. It just flushes

Re: Crash on 2.6.21.7 Vanilla + DRBD 0.7

2007-10-07 Thread David Chinner
On Thu, Oct 04, 2007 at 09:29:40AM +0200, Laurent Caron wrote: > > Hi, > > I did compile a fresh 2.6.21.7 kernel from kernel.org (no distro patch, > ), and latest svn (3062) 0.7.X drbd. > > After just 2 days of uptime, I did experience another crash. > > I wonder if it is an XFS related

Re: [NFS] What's slated for inclusion in 2.6.24-rc1 from the NFS client git tree...

2007-10-07 Thread David Chinner
On Fri, Oct 05, 2007 at 02:12:22PM -0400, Jeff Layton wrote: > On Fri, 05 Oct 2007 13:30:10 -0400 > [EMAIL PROTECTED] wrote: > > > On Thu, 04 Oct 2007 10:00:50 EDT, Trond Myklebust said: > > > > > How about a boot/module parameter to turn it on or off? > > > > > > I don't see any point in

Re: [NFS] What's slated for inclusion in 2.6.24-rc1 from the NFS client git tree...

2007-10-07 Thread David Chinner
On Fri, Oct 05, 2007 at 02:12:22PM -0400, Jeff Layton wrote: On Fri, 05 Oct 2007 13:30:10 -0400 [EMAIL PROTECTED] wrote: On Thu, 04 Oct 2007 10:00:50 EDT, Trond Myklebust said: How about a boot/module parameter to turn it on or off? I don't see any point in having a sysctl for

Re: Crash on 2.6.21.7 Vanilla + DRBD 0.7

2007-10-07 Thread David Chinner
On Thu, Oct 04, 2007 at 09:29:40AM +0200, Laurent Caron wrote: Hi, I did compile a fresh 2.6.21.7 kernel from kernel.org (no distro patch, ), and latest svn (3062) 0.7.X drbd. After just 2 days of uptime, I did experience another crash. I wonder if it is an XFS related bug, a

Re: [PATCH] remove throttle_vm_writeout()

2007-10-07 Thread David Chinner
On Fri, Oct 05, 2007 at 08:30:28PM +0800, Fengguang Wu wrote: The improvement could be: - kswapd is now explicitly preferred to do the writeout; Careful. kswapd is much less efficient at writeout than pdflush because it does not do low-high offset writeback per address space. It just flushes

Re: XFS internal error

2007-10-07 Thread David Chinner
[please cc [EMAIL PROTECTED] on XFS bug reports. thx.] On Sun, Oct 07, 2007 at 09:09:58AM +0800, Max Waterman wrote: Hi, I have just had an XFS error occur while deleting some directory hierarchy. I hope this is the correct place to report it. . This is in syslog : Oct 6 23:40:33

Re: [PATCH 5/5] writeback: introduce writeback_control.more_io to indicate more io

2007-10-05 Thread David Chinner
On Fri, Oct 05, 2007 at 11:36:52AM +0800, Fengguang Wu wrote: > On Thu, Oct 04, 2007 at 03:03:44PM +1000, David Chinner wrote: > > On Thu, Oct 04, 2007 at 10:21:33AM +0800, Fengguang Wu wrote: > > > OK, I guess it is the focus of all your questions: Why should we sleep > &

Re: [PATCH 5/5] writeback: introduce writeback_control.more_io to indicate more io

2007-10-05 Thread David Chinner
On Fri, Oct 05, 2007 at 11:36:52AM +0800, Fengguang Wu wrote: On Thu, Oct 04, 2007 at 03:03:44PM +1000, David Chinner wrote: On Thu, Oct 04, 2007 at 10:21:33AM +0800, Fengguang Wu wrote: OK, I guess it is the focus of all your questions: Why should we sleep in congestion_wait

Re: SLUB performance regression vs SLAB

2007-10-04 Thread David Chinner
On Thu, Oct 04, 2007 at 03:07:18PM -0700, David Miller wrote: > From: Chuck Ebbert <[EMAIL PROTECTED]> Date: Thu, 04 Oct 2007 17:47:48 > -0400 > > > On 10/04/2007 05:11 PM, David Miller wrote: > > > From: Chuck Ebbert <[EMAIL PROTECTED]> Date: Thu, 04 Oct 2007 17:02:17 > > > -0400 > > > > > >>

Re: SLUB performance regression vs SLAB

2007-10-04 Thread David Chinner
On Thu, Oct 04, 2007 at 03:07:18PM -0700, David Miller wrote: From: Chuck Ebbert [EMAIL PROTECTED] Date: Thu, 04 Oct 2007 17:47:48 -0400 On 10/04/2007 05:11 PM, David Miller wrote: From: Chuck Ebbert [EMAIL PROTECTED] Date: Thu, 04 Oct 2007 17:02:17 -0400 How do you simulate

Re: [PATCH 5/5] writeback: introduce writeback_control.more_io to indicate more io

2007-10-03 Thread David Chinner
On Thu, Oct 04, 2007 at 10:21:33AM +0800, Fengguang Wu wrote: > On Wed, Oct 03, 2007 at 12:41:19PM +1000, David Chinner wrote: > > On Wed, Oct 03, 2007 at 09:34:39AM +0800, Fengguang Wu wrote: > > > On Wed, Oct 03, 2007 at 07:47:45AM +1000, David Chinner wrote: > > > &

Re: [PATCH 5/5] writeback: introduce writeback_control.more_io to indicate more io

2007-10-03 Thread David Chinner
On Thu, Oct 04, 2007 at 10:21:33AM +0800, Fengguang Wu wrote: On Wed, Oct 03, 2007 at 12:41:19PM +1000, David Chinner wrote: On Wed, Oct 03, 2007 at 09:34:39AM +0800, Fengguang Wu wrote: On Wed, Oct 03, 2007 at 07:47:45AM +1000, David Chinner wrote: On Tue, Oct 02, 2007 at 04:41:48PM

Re: [PATCH 5/5] writeback: introduce writeback_control.more_io to indicate more io

2007-10-02 Thread David Chinner
On Wed, Oct 03, 2007 at 09:34:39AM +0800, Fengguang Wu wrote: > On Wed, Oct 03, 2007 at 07:47:45AM +1000, David Chinner wrote: > > On Tue, Oct 02, 2007 at 04:41:48PM +0800, Fengguang Wu wrote: > > > wbc.pages_skipped = 0; > > > @@ -560,8 +561,9 @@ static void b

Re: [PATCH 4/5] writeback: remove pages_skipped accounting in __block_write_full_page()

2007-10-02 Thread David Chinner
On Wed, Oct 03, 2007 at 09:43:33AM +0800, Fengguang Wu wrote: > On Wed, Oct 03, 2007 at 07:55:18AM +1000, David Chinner wrote: > > > > > > do not quite agree with each other. The page writeback should be skipped > > > for > > > 'locked buffer', but her

Re: [PATCH 4/5] writeback: remove pages_skipped accounting in __block_write_full_page()

2007-10-02 Thread David Chinner
> > do not quite agree with each other. The page writeback should be skipped for > 'locked buffer', but here it is 'clean buffer'! Ok, so that means we need an equivalent fix in xfs_start_page_writeback() as it will skip pages with clean buffers just like this. Something like this (untested)?

Re: [PATCH 5/5] writeback: introduce writeback_control.more_io to indicate more io

2007-10-02 Thread David Chinner
On Tue, Oct 02, 2007 at 04:41:48PM +0800, Fengguang Wu wrote: > wbc.pages_skipped = 0; > @@ -560,8 +561,9 @@ static void background_writeout(unsigned > min_pages -= MAX_WRITEBACK_PAGES - wbc.nr_to_write; > if (wbc.nr_to_write > 0 || wbc.pages_skipped > 0)

Re: XFS Fails Quality Assurance Tests on ARM

2007-10-02 Thread David Chinner
On Fri, Sep 28, 2007 at 07:40:48AM +1000, Nick Piggin wrote: > On Sunday 02 September 2007 08:14, Andi Kleen wrote: > > David Miller <[EMAIL PROTECTED]> writes: > > > From: Byron Bradley <[EMAIL PROTECTED]> > > > Date: Fri, 31 Aug 2007 03:12:46 + (UTC) > > > > Anybody got any ideas of how we

Re: XFS Fails Quality Assurance Tests on ARM

2007-10-02 Thread David Chinner
On Fri, Sep 28, 2007 at 07:40:48AM +1000, Nick Piggin wrote: On Sunday 02 September 2007 08:14, Andi Kleen wrote: David Miller [EMAIL PROTECTED] writes: From: Byron Bradley [EMAIL PROTECTED] Date: Fri, 31 Aug 2007 03:12:46 + (UTC) Anybody got any ideas of how we fix this? I

Re: [PATCH 5/5] writeback: introduce writeback_control.more_io to indicate more io

2007-10-02 Thread David Chinner
On Tue, Oct 02, 2007 at 04:41:48PM +0800, Fengguang Wu wrote: wbc.pages_skipped = 0; @@ -560,8 +561,9 @@ static void background_writeout(unsigned min_pages -= MAX_WRITEBACK_PAGES - wbc.nr_to_write; if (wbc.nr_to_write 0 || wbc.pages_skipped 0) {

Re: [PATCH 4/5] writeback: remove pages_skipped accounting in __block_write_full_page()

2007-10-02 Thread David Chinner
do not quite agree with each other. The page writeback should be skipped for 'locked buffer', but here it is 'clean buffer'! Ok, so that means we need an equivalent fix in xfs_start_page_writeback() as it will skip pages with clean buffers just like this. Something like this (untested)? ---

Re: [PATCH 4/5] writeback: remove pages_skipped accounting in __block_write_full_page()

2007-10-02 Thread David Chinner
On Wed, Oct 03, 2007 at 09:43:33AM +0800, Fengguang Wu wrote: On Wed, Oct 03, 2007 at 07:55:18AM +1000, David Chinner wrote: do not quite agree with each other. The page writeback should be skipped for 'locked buffer', but here it is 'clean buffer'! Ok, so that means we need

Re: [PATCH 5/5] writeback: introduce writeback_control.more_io to indicate more io

2007-10-02 Thread David Chinner
On Wed, Oct 03, 2007 at 09:34:39AM +0800, Fengguang Wu wrote: On Wed, Oct 03, 2007 at 07:47:45AM +1000, David Chinner wrote: On Tue, Oct 02, 2007 at 04:41:48PM +0800, Fengguang Wu wrote: wbc.pages_skipped = 0; @@ -560,8 +561,9 @@ static void background_writeout(unsigned

Re: Disk I/O degraded performance

2007-09-20 Thread David Chinner
On Wed, Sep 19, 2007 at 10:09:12PM +0200, Ramon Chimeno wrote: > Hi all > > I migrated one of my server from kernel 2.6.18 to the latest 2.6.22 > and I experienced lower disk performance for processes that open file > with the O_DIRECT flag. > > I did a very simple test program that opens two

Re: 2.6.20 (XFS? related) crash after uptime of > 180 days during apt-get dist-upgrade on Debian Testing

2007-09-20 Thread David Chinner
On Wed, Sep 19, 2007 at 04:47:38AM -0400, Justin Piszcz wrote: > On Mon, 17 Sep 2007, Justin Piszcz wrote: > > >Including the XFS mailing list in here too because it may be an XFS bug > >looking at the call trace. > > > >System: Debian Testing > >Kernel: 2.6.20 > >Config: Attached > > > >I was

Re: 2.6.20 (XFS? related) crash after uptime of 180 days during apt-get dist-upgrade on Debian Testing

2007-09-20 Thread David Chinner
On Wed, Sep 19, 2007 at 04:47:38AM -0400, Justin Piszcz wrote: On Mon, 17 Sep 2007, Justin Piszcz wrote: Including the XFS mailing list in here too because it may be an XFS bug looking at the call trace. System: Debian Testing Kernel: 2.6.20 Config: Attached I was running apt-get

Re: Disk I/O degraded performance

2007-09-20 Thread David Chinner
On Wed, Sep 19, 2007 at 10:09:12PM +0200, Ramon Chimeno wrote: Hi all I migrated one of my server from kernel 2.6.18 to the latest 2.6.22 and I experienced lower disk performance for processes that open file with the O_DIRECT flag. I did a very simple test program that opens two files

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-19 Thread David Chinner
On Wed, Sep 19, 2007 at 04:04:30PM +0200, Andrea Arcangeli wrote: > On Wed, Sep 19, 2007 at 03:09:10PM +1000, David Chinner wrote: > > Ok, let's step back for a moment and look at a basic, fundamental > > constraint of disks - seek capacity. A decade ago, a terabyte of > > fi

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-19 Thread David Chinner
On Wed, Sep 19, 2007 at 04:04:30PM +0200, Andrea Arcangeli wrote: On Wed, Sep 19, 2007 at 03:09:10PM +1000, David Chinner wrote: Ok, let's step back for a moment and look at a basic, fundamental constraint of disks - seek capacity. A decade ago, a terabyte of filesystem had 30 disks behind

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-18 Thread David Chinner
On Tue, Sep 18, 2007 at 06:06:52PM -0700, Linus Torvalds wrote: > > especially as the Linux > > kernel limitations in this area are well known. There's no "16K mess" > > that SGI is trying to clean up here (and SGI have offered both IA64 and > > x86_64

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-18 Thread David Chinner
On Tue, Sep 18, 2007 at 11:00:40AM +0100, Mel Gorman wrote: > We still lack data on what sort of workloads really benefit from large > blocks (assuming there are any that cannot also be solved by improving > order-0). No we don't. All workloads benefit from larger block sizes when you've got a

Re: 2.6.20 (XFS? related) crash after uptime of > 180 days during apt-get dist-upgrade on Debian Testing

2007-09-18 Thread David Chinner
On Tue, Sep 18, 2007 at 10:20:13AM +0100, Christoph Hellwig wrote: > On Tue, Sep 18, 2007 at 11:45:37AM +1000, David Chinner wrote: > > No idea - it looks like dkpg was trying to remove a directory on the > > same path the lookup was and both have gone splat in __d_lookup on >

Re: 2.6.20 (XFS? related) crash after uptime of 180 days during apt-get dist-upgrade on Debian Testing

2007-09-18 Thread David Chinner
On Tue, Sep 18, 2007 at 10:20:13AM +0100, Christoph Hellwig wrote: On Tue, Sep 18, 2007 at 11:45:37AM +1000, David Chinner wrote: No idea - it looks like dkpg was trying to remove a directory on the same path the lookup was and both have gone splat in __d_lookup on the same dentry

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-18 Thread David Chinner
On Tue, Sep 18, 2007 at 11:00:40AM +0100, Mel Gorman wrote: We still lack data on what sort of workloads really benefit from large blocks (assuming there are any that cannot also be solved by improving order-0). No we don't. All workloads benefit from larger block sizes when you've got a btree

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-18 Thread David Chinner
On Tue, Sep 18, 2007 at 06:06:52PM -0700, Linus Torvalds wrote: especially as the Linux kernel limitations in this area are well known. There's no 16K mess that SGI is trying to clean up here (and SGI have offered both IA64 and x86_64 systems

Re: 2.6.20 (XFS? related) crash after uptime of > 180 days during apt-get dist-upgrade on Debian Testing

2007-09-17 Thread David Chinner
On Mon, Sep 17, 2007 at 01:20:17PM -0400, Justin Piszcz wrote: > Including the XFS mailing list in here too because it may be an XFS bug > looking at the call trace. > > System: Debian Testing > Kernel: 2.6.20 > Config: Attached > > I was running apt-get dist-upgrade as I always do to get the

Re: 2.6.20 (XFS? related) crash after uptime of 180 days during apt-get dist-upgrade on Debian Testing

2007-09-17 Thread David Chinner
On Mon, Sep 17, 2007 at 01:20:17PM -0400, Justin Piszcz wrote: Including the XFS mailing list in here too because it may be an XFS bug looking at the call trace. System: Debian Testing Kernel: 2.6.20 Config: Attached I was running apt-get dist-upgrade as I always do to get the latest

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-16 Thread David Chinner
On Fri, Sep 14, 2007 at 06:48:55AM +1000, Nick Piggin wrote: > On Thursday 13 September 2007 12:01, Nick Piggin wrote: > > On Thursday 13 September 2007 23:03, David Chinner wrote: > > > Then just do operations on directories with lots of files in them > > > (tens of

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-16 Thread David Chinner
On Fri, Sep 14, 2007 at 06:48:55AM +1000, Nick Piggin wrote: On Thursday 13 September 2007 12:01, Nick Piggin wrote: On Thursday 13 September 2007 23:03, David Chinner wrote: Then just do operations on directories with lots of files in them (tens of thousands). Every directory operation

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-13 Thread David Chinner
On Thu, Sep 13, 2007 at 03:23:21AM +1000, Nick Piggin wrote: > On Thursday 13 September 2007 11:49, David Chinner wrote: > > On Wed, Sep 12, 2007 at 01:27:33AM +1000, Nick Piggin wrote: > > > > I just gave 4 things which combined might easily reduce xfs vmap overhead &g

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-13 Thread David Chinner
On Thu, Sep 13, 2007 at 03:23:21AM +1000, Nick Piggin wrote: On Thursday 13 September 2007 11:49, David Chinner wrote: On Wed, Sep 12, 2007 at 01:27:33AM +1000, Nick Piggin wrote: I just gave 4 things which combined might easily reduce xfs vmap overhead by several orders of magnitude

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-12 Thread David Chinner
On Wed, Sep 12, 2007 at 01:27:33AM +1000, Nick Piggin wrote: > > IOWs, we already play these vmap harm-minimisation games in the places > > where we can, but still the overhead is high and something we'd prefer > > to be able to avoid. > > I don't think you've looked nearly far enough with all

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-12 Thread David Chinner
On Wed, Sep 12, 2007 at 01:27:33AM +1000, Nick Piggin wrote: IOWs, we already play these vmap harm-minimisation games in the places where we can, but still the overhead is high and something we'd prefer to be able to avoid. I don't think you've looked nearly far enough with all this low

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-11 Thread David Chinner
On Tue, Sep 11, 2007 at 04:00:17PM +1000, Nick Piggin wrote: > > > OTOH, I'm not sure how much buy-in there was from the filesystems guys. > > > Particularly Christoph H and XFS (which is strange because they already > > > do vmapping in places). > > > > I think they use vmapping because they have

Re: [00/41] Large Blocksize Support V7 (adds memmap support)

2007-09-11 Thread David Chinner
On Tue, Sep 11, 2007 at 04:00:17PM +1000, Nick Piggin wrote: OTOH, I'm not sure how much buy-in there was from the filesystems guys. Particularly Christoph H and XFS (which is strange because they already do vmapping in places). I think they use vmapping because they have to, not

Re: limiting to UDMA/33 instead of UDMA/100 - pata_pdc202xx_old (also XFS error)?

2007-09-07 Thread David Chinner
On Thu, Aug 30, 2007 at 04:11:03AM -0700, n wrote: > hello, with kernel 2.6.22.5 using the new pata_pdc202xx_old driver it > doesn detect the cable right on this seagate drive (i tried switching ports > / cables ...etc) . > XFS mounting filesystem sdc3 > Ending clean XFS mount for

Re: limiting to UDMA/33 instead of UDMA/100 - pata_pdc202xx_old (also XFS error)?

2007-09-07 Thread David Chinner
On Thu, Aug 30, 2007 at 04:11:03AM -0700, n wrote: hello, with kernel 2.6.22.5 using the new pata_pdc202xx_old driver it doesn detect the cable right on this seagate drive (i tried switching ports / cables ...etc) . XFS mounting filesystem sdc3 Ending clean XFS mount for filesystem:

Re: [PATCH] Increase lockdep MAX_LOCK_DEPTH

2007-08-31 Thread David Chinner
On Fri, Aug 31, 2007 at 05:09:21PM +0200, Peter Zijlstra wrote: > On Sat, 2007-09-01 at 01:05 +1000, David Chinner wrote: > > > > Trouble is, we'd like to have a sane upper bound on the amount of held > > > locks at any one time, obviously this is just wanting, because a

Re: [PATCH] Increase lockdep MAX_LOCK_DEPTH

2007-08-31 Thread David Chinner
On Fri, Aug 31, 2007 at 04:33:51PM +0200, Peter Zijlstra wrote: > On Fri, 2007-08-31 at 23:50 +1000, David Chinner wrote: > > On Fri, Aug 31, 2007 at 08:39:49AM +0200, Peter Zijlstra wrote: > > > On Thu, 2007-08-30 at 23:43 -0500, Eric Sandeen wrote: > > > >

Re: [PATCH] Increase lockdep MAX_LOCK_DEPTH

2007-08-31 Thread David Chinner
On Fri, Aug 31, 2007 at 08:39:49AM +0200, Peter Zijlstra wrote: > On Thu, 2007-08-30 at 23:43 -0500, Eric Sandeen wrote: > > The xfs filesystem can exceed the current lockdep > > MAX_LOCK_DEPTH, because when deleting an entire cluster of inodes, > > they all get locked in xfs_ifree_cluster().

Re: [PATCH] Increase lockdep MAX_LOCK_DEPTH

2007-08-31 Thread David Chinner
On Fri, Aug 31, 2007 at 04:33:51PM +0200, Peter Zijlstra wrote: On Fri, 2007-08-31 at 23:50 +1000, David Chinner wrote: On Fri, Aug 31, 2007 at 08:39:49AM +0200, Peter Zijlstra wrote: On Thu, 2007-08-30 at 23:43 -0500, Eric Sandeen wrote: The xfs filesystem can exceed the current lockdep

Re: [PATCH] Increase lockdep MAX_LOCK_DEPTH

2007-08-31 Thread David Chinner
On Fri, Aug 31, 2007 at 05:09:21PM +0200, Peter Zijlstra wrote: On Sat, 2007-09-01 at 01:05 +1000, David Chinner wrote: Trouble is, we'd like to have a sane upper bound on the amount of held locks at any one time, obviously this is just wanting, because a lot of lock chains also depend

Re: [PATCH] Increase lockdep MAX_LOCK_DEPTH

2007-08-31 Thread David Chinner
On Fri, Aug 31, 2007 at 08:39:49AM +0200, Peter Zijlstra wrote: On Thu, 2007-08-30 at 23:43 -0500, Eric Sandeen wrote: The xfs filesystem can exceed the current lockdep MAX_LOCK_DEPTH, because when deleting an entire cluster of inodes, they all get locked in xfs_ifree_cluster(). The

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread David Chinner
On Tue, Aug 28, 2007 at 11:08:20AM -0400, Chris Mason wrote: > On Wed, 29 Aug 2007 00:55:30 +1000 > David Chinner <[EMAIL PROTECTED]> wrote: > > On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: > > > On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wr

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread David Chinner
On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: > On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: > > On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: > > > On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: > > > No

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread David Chinner
On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: Notes: (1) I'm not sure inode number

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-28 Thread David Chinner
On Tue, Aug 28, 2007 at 11:08:20AM -0400, Chris Mason wrote: On Wed, 29 Aug 2007 00:55:30 +1000 David Chinner [EMAIL PROTECTED] wrote: On Fri, Aug 24, 2007 at 09:55:04PM +0800, Fengguang Wu wrote: On Thu, Aug 23, 2007 at 12:33:06PM +1000, David Chinner wrote: On Wed, Aug 22, 2007 at 09

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread David Chinner
On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: > I think we should assume a full scan of s_dirty is impossible in the > presence of concurrent writers. We want to be able to pick a start > time (right now) and find all the inodes older than that start time. > New things will come in

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread David Chinner
On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: > On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: > Notes: > (1) I'm not sure inode number is correlated to disk location in > filesystems other than ext2/3/4. Or parent dir? The correspond to the exact location on

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread David Chinner
On Wed, Aug 22, 2007 at 09:18:41AM +0800, Fengguang Wu wrote: On Tue, Aug 21, 2007 at 08:23:14PM -0400, Chris Mason wrote: Notes: (1) I'm not sure inode number is correlated to disk location in filesystems other than ext2/3/4. Or parent dir? The correspond to the exact location on disk on

Re: [PATCH 0/6] writeback time order/delay fixes take 3

2007-08-22 Thread David Chinner
On Wed, Aug 22, 2007 at 08:42:01AM -0400, Chris Mason wrote: I think we should assume a full scan of s_dirty is impossible in the presence of concurrent writers. We want to be able to pick a start time (right now) and find all the inodes older than that start time. New things will come in

Re: [PATCH 3/6] writeback: remove pages_skipped accounting in __block_write_full_page()

2007-08-12 Thread David Chinner
On Sun, Aug 12, 2007 at 05:11:23PM +0800, Fengguang Wu wrote: > Miklos Szeredi <[EMAIL PROTECTED]> and me identified a writeback bug: > Basicly they are > - during the dd: ~16M > - after 30s: ~4M > - after 5s: ~4M > - after 5s: ~176M > > The box has 2G memory. > > Question 1: >

Re: [PATCH 3/6] writeback: remove pages_skipped accounting in __block_write_full_page()

2007-08-12 Thread David Chinner
On Sun, Aug 12, 2007 at 05:11:23PM +0800, Fengguang Wu wrote: Miklos Szeredi [EMAIL PROTECTED] and me identified a writeback bug: Basicly they are - during the dd: ~16M - after 30s: ~4M - after 5s: ~4M - after 5s: ~176M The box has 2G memory. Question 1: How come the

Re: Oops on 2.6.21 + DRBD + XFS

2007-08-06 Thread David Chinner
On Mon, Aug 06, 2007 at 09:38:19AM +0200, Laurent Caron wrote: > Hi, > > I'm using an XFS filesystem over DRBD for a few weeks on this machine > and did experience an oops. .. > Aug 3 10:59:47 fileserv kernel: [] cache_flusharray+0x59/0xd0 > Aug 3 10:59:47 fileserv kernel: []

Re: Oops on 2.6.21 + DRBD + XFS

2007-08-06 Thread David Chinner
On Mon, Aug 06, 2007 at 09:38:19AM +0200, Laurent Caron wrote: Hi, I'm using an XFS filesystem over DRBD for a few weeks on this machine and did experience an oops. .. Aug 3 10:59:47 fileserv kernel: [c0164be9] cache_flusharray+0x59/0xd0 Aug 3 10:59:47 fileserv kernel: [c0164d2a]

Re: [PATCH][RESEND] fix a potential NULL pointer deref in XFS on failed mount.

2007-08-05 Thread David Chinner
On Sat, Aug 04, 2007 at 08:30:21PM +0200, Jesper Juhl wrote: > Back in 2006 (2006-10-31 to be specific, reposted on 2006-11-16), I > submitted a patch to fix a potential NULL pointer deref in XFS on > failed mount. Already checked into xfs-dev tree. Will go to next mainline merge.

<    1   2   3   4   5   6   7   8   9   >