Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-03-02 Thread Theodore Ts'o
On Sat, Mar 02, 2013 at 11:54:37AM -0800, Linus Torvalds wrote:
> 
> This needs to be fixed or reverted. I traced back some user
> interaction problems to this same issue. It literally gets so bad that
> the whole system is choppy, and a profile shows that a lot of time is
> being spent in the spin-lock protecting these data structures. Called
> from ext4_es_reclaim_extents_count, ext4_es_lru_add and
> ext4_es_shrink.

I'm doing final testing on a pull request that will fix these
problems.  I'll send it to you later this evening.

 - Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-03-02 Thread Linus Torvalds
On Thu, Feb 28, 2013 at 8:00 PM, Theodore Ts'o  wrote:
>
> ext4_es_reclaim_extents_count() is getting called out of the slab
> shrinker.  It's getting called too often when there is significant
> memory pressure.  We can optimize this so we're not calculating it all
> the time.

This needs to be fixed or reverted. I traced back some user
interaction problems to this same issue. It literally gets so bad that
the whole system is choppy, and a profile shows that a lot of time is
being spent in the spin-lock protecting these data structures. Called
from ext4_es_reclaim_extents_count, ext4_es_lru_add and
ext4_es_shrink.

We're talking very user-noticeable pauses, so it's not just the
spinlock, there's some real badness that comes in with it too.
Possibly related to other (sleeping) locks being held while the data
structures are then refilled or something.

   Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-03-02 Thread Linus Torvalds
On Thu, Feb 28, 2013 at 8:00 PM, Theodore Ts'o ty...@mit.edu wrote:

 ext4_es_reclaim_extents_count() is getting called out of the slab
 shrinker.  It's getting called too often when there is significant
 memory pressure.  We can optimize this so we're not calculating it all
 the time.

This needs to be fixed or reverted. I traced back some user
interaction problems to this same issue. It literally gets so bad that
the whole system is choppy, and a profile shows that a lot of time is
being spent in the spin-lock protecting these data structures. Called
from ext4_es_reclaim_extents_count, ext4_es_lru_add and
ext4_es_shrink.

We're talking very user-noticeable pauses, so it's not just the
spinlock, there's some real badness that comes in with it too.
Possibly related to other (sleeping) locks being held while the data
structures are then refilled or something.

   Linus
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-03-02 Thread Theodore Ts'o
On Sat, Mar 02, 2013 at 11:54:37AM -0800, Linus Torvalds wrote:
 
 This needs to be fixed or reverted. I traced back some user
 interaction problems to this same issue. It literally gets so bad that
 the whole system is choppy, and a profile shows that a lot of time is
 being spent in the spin-lock protecting these data structures. Called
 from ext4_es_reclaim_extents_count, ext4_es_lru_add and
 ext4_es_shrink.

I'm doing final testing on a pull request that will fix these
problems.  I'll send it to you later this evening.

 - Ted
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-28 Thread Theodore Ts'o
On Thu, Feb 28, 2013 at 10:30:05PM -0500, Dave Jones wrote:
> 
> This has fixed the problem I reported, but I notice now that my
> desktop is really sluggish. perf top shows it's almost constantly
> spinning in ext4_es_reclaim_extents_count
> 
> Any ideas ?

ext4_es_reclaim_extents_count() is getting called out of the slab
shrinker.  It's getting called too often when there is significant
memory pressure.  We can optimize this so we're not calculating it all
the time.

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-28 Thread Dave Jones
On Wed, Feb 27, 2013 at 03:12:17PM -0500, Linus Torvalds wrote:
 > The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:
 > 
 >   ext4: fix free clusters calculation in bigalloc filesystem (2013-02-22 
 > 15:27:52 -0500)
 > 
 > are available in the git repository at:
 > 
 >   git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git 
 > tags/ext4_for_linus
 > 
 > for you to fetch changes up to 8e919d13048cd5acaadb2b15b48acbfb8832d3c2:
 > 
 >   ext4: fix extent status tree regression for file systems > 512GB 
 > (2013-02-27 14:54:37 -0500)
 > 
 > 
 > This fixes a real brown paper bag bug which causes ext4 to choke on
 > file systems larger than 512GB.
 > 
 > 
 > Theodore Ts'o (1):
 >   ext4: fix extent status tree regression for file systems > 512GB
 > 
 >  fs/ext4/extents_status.h | 19 +++
 >  1 file changed, 11 insertions(+), 8 deletions(-)

This has fixed the problem I reported, but I notice now that my desktop is 
really
sluggish. perf top shows it's almost constantly spinning in 
ext4_es_reclaim_extents_count

Any ideas ?

Dave

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-28 Thread Dave Jones
On Wed, Feb 27, 2013 at 03:12:17PM -0500, Linus Torvalds wrote:
  The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:
  
ext4: fix free clusters calculation in bigalloc filesystem (2013-02-22 
  15:27:52 -0500)
  
  are available in the git repository at:
  
git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git 
  tags/ext4_for_linus
  
  for you to fetch changes up to 8e919d13048cd5acaadb2b15b48acbfb8832d3c2:
  
ext4: fix extent status tree regression for file systems  512GB 
  (2013-02-27 14:54:37 -0500)
  
  
  This fixes a real brown paper bag bug which causes ext4 to choke on
  file systems larger than 512GB.
  
  
  Theodore Ts'o (1):
ext4: fix extent status tree regression for file systems  512GB
  
   fs/ext4/extents_status.h | 19 +++
   1 file changed, 11 insertions(+), 8 deletions(-)

This has fixed the problem I reported, but I notice now that my desktop is 
really
sluggish. perf top shows it's almost constantly spinning in 
ext4_es_reclaim_extents_count

Any ideas ?

Dave

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-28 Thread Theodore Ts'o
On Thu, Feb 28, 2013 at 10:30:05PM -0500, Dave Jones wrote:
 
 This has fixed the problem I reported, but I notice now that my
 desktop is really sluggish. perf top shows it's almost constantly
 spinning in ext4_es_reclaim_extents_count
 
 Any ideas ?

ext4_es_reclaim_extents_count() is getting called out of the slab
shrinker.  It's getting called too often when there is significant
memory pressure.  We can optimize this so we're not calculating it all
the time.

- Ted
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Borislav Petkov
On Wed, Feb 27, 2013 at 12:23:18PM -0800, Linus Torvalds wrote:
> On Wed, Feb 27, 2013 at 12:15 PM, Theodore Ts'o  wrote:
> > On Wed, Feb 27, 2013 at 03:12:17PM -0500, Linus Torvalds wrote:
> >> The following changes since commit 
> >> 304e220f0879198b1f5309ad6f0be862b4009491:
> >
> > Obviously, this wasn't from Linus.  I screwed up on sending the pull
> > request.  Sigh...
> 
> That looked really odd in my inbox for a while ;)
> 
>   Linus "the *real* one" Torvalds

He doesn't trust anyone - he sends pull requests to himself!

:-)

-- 
Regards/Gruss,
Boris.

Sent from a fat crate under my desk. Formatting is fine.
--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Linus Torvalds
On Wed, Feb 27, 2013 at 12:15 PM, Theodore Ts'o  wrote:
> On Wed, Feb 27, 2013 at 03:12:17PM -0500, Linus Torvalds wrote:
>> The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:
>
> Obviously, this wasn't from Linus.  I screwed up on sending the pull
> request.  Sigh...

That looked really odd in my inbox for a while ;)

  Linus "the *real* one" Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Theodore Ts'o
On Wed, Feb 27, 2013 at 03:12:17PM -0500, Linus Torvalds wrote:
> The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:

Obviously, this wasn't from Linus.  I screwed up on sending the pull
request.  Sigh...

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Theodore Ts'o
The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:

  ext4: fix free clusters calculation in bigalloc filesystem (2013-02-22 
15:27:52 -0500)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git 
tags/ext4_for_linus

for you to fetch changes up to 8e919d13048cd5acaadb2b15b48acbfb8832d3c2:

  ext4: fix extent status tree regression for file systems > 512GB (2013-02-27 
14:54:37 -0500)


This fixes a real brown paper bag bug which causes ext4 to choke on
file systems larger than 512GB.


Theodore Ts'o (1):
  ext4: fix extent status tree regression for file systems > 512GB

 fs/ext4/extents_status.h | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Linus Torvalds
The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:

  ext4: fix free clusters calculation in bigalloc filesystem (2013-02-22 
15:27:52 -0500)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git 
tags/ext4_for_linus

for you to fetch changes up to 8e919d13048cd5acaadb2b15b48acbfb8832d3c2:

  ext4: fix extent status tree regression for file systems > 512GB (2013-02-27 
14:54:37 -0500)


This fixes a real brown paper bag bug which causes ext4 to choke on
file systems larger than 512GB.


Theodore Ts'o (1):
  ext4: fix extent status tree regression for file systems > 512GB

 fs/ext4/extents_status.h | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Linus Torvalds
The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:

  ext4: fix free clusters calculation in bigalloc filesystem (2013-02-22 
15:27:52 -0500)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git 
tags/ext4_for_linus

for you to fetch changes up to 8e919d13048cd5acaadb2b15b48acbfb8832d3c2:

  ext4: fix extent status tree regression for file systems  512GB (2013-02-27 
14:54:37 -0500)


This fixes a real brown paper bag bug which causes ext4 to choke on
file systems larger than 512GB.


Theodore Ts'o (1):
  ext4: fix extent status tree regression for file systems  512GB

 fs/ext4/extents_status.h | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


[GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Theodore Ts'o
The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:

  ext4: fix free clusters calculation in bigalloc filesystem (2013-02-22 
15:27:52 -0500)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git 
tags/ext4_for_linus

for you to fetch changes up to 8e919d13048cd5acaadb2b15b48acbfb8832d3c2:

  ext4: fix extent status tree regression for file systems  512GB (2013-02-27 
14:54:37 -0500)


This fixes a real brown paper bag bug which causes ext4 to choke on
file systems larger than 512GB.


Theodore Ts'o (1):
  ext4: fix extent status tree regression for file systems  512GB

 fs/ext4/extents_status.h | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Theodore Ts'o
On Wed, Feb 27, 2013 at 03:12:17PM -0500, Linus Torvalds wrote:
 The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:

Obviously, this wasn't from Linus.  I screwed up on sending the pull
request.  Sigh...

- Ted
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Linus Torvalds
On Wed, Feb 27, 2013 at 12:15 PM, Theodore Ts'o ty...@mit.edu wrote:
 On Wed, Feb 27, 2013 at 03:12:17PM -0500, Linus Torvalds wrote:
 The following changes since commit 304e220f0879198b1f5309ad6f0be862b4009491:

 Obviously, this wasn't from Linus.  I screwed up on sending the pull
 request.  Sigh...

That looked really odd in my inbox for a while ;)

  Linus the *real* one Torvalds
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [GIT PULL URGENT] ext4 regression fix for 3.9

2013-02-27 Thread Borislav Petkov
On Wed, Feb 27, 2013 at 12:23:18PM -0800, Linus Torvalds wrote:
 On Wed, Feb 27, 2013 at 12:15 PM, Theodore Ts'o ty...@mit.edu wrote:
  On Wed, Feb 27, 2013 at 03:12:17PM -0500, Linus Torvalds wrote:
  The following changes since commit 
  304e220f0879198b1f5309ad6f0be862b4009491:
 
  Obviously, this wasn't from Linus.  I screwed up on sending the pull
  request.  Sigh...
 
 That looked really odd in my inbox for a while ;)
 
   Linus the *real* one Torvalds

He doesn't trust anyone - he sends pull requests to himself!

:-)

-- 
Regards/Gruss,
Boris.

Sent from a fat crate under my desk. Formatting is fine.
--
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/