Re: [RFC][PATCH] vfs: add closefrom(2) syscall

2014-04-08 Thread Zheng Liu
On Tue, Apr 08, 2014 at 10:21:37AM +0200, Mateusz Guzik wrote: > On Tue, Apr 08, 2014 at 03:12:22PM +0800, Zheng Liu wrote: > > > > +int __close_fds(struct files_struct *files, int lowfd) > > +{ > > + struct file *file; > > + struct fdtable *fdt; > >

[RFC][PATCH] vfs: add closefrom(2) syscall

2014-04-08 Thread Zheng Liu
From: Zheng Liu This commit adds a new syscall called closefrom(2) which is almost the same as closefrom(2) in FreeBSD. This syscall will close all open file descriptors that is greater than or equal to the fd that is indicated by caller. It is really useful for an application that has opened

[RFC][PATCH] vfs: add closefrom(2) syscall

2014-04-08 Thread Zheng Liu
From: Zheng Liu wenqing...@taobao.com This commit adds a new syscall called closefrom(2) which is almost the same as closefrom(2) in FreeBSD. This syscall will close all open file descriptors that is greater than or equal to the fd that is indicated by caller. It is really useful

Re: [RFC][PATCH] vfs: add closefrom(2) syscall

2014-04-08 Thread Zheng Liu
On Tue, Apr 08, 2014 at 10:21:37AM +0200, Mateusz Guzik wrote: On Tue, Apr 08, 2014 at 03:12:22PM +0800, Zheng Liu wrote: +int __close_fds(struct files_struct *files, int lowfd) +{ + struct file *file; + struct fdtable *fdt; + int fd; + + if (lowfd 0

[PATCH] mm: improve the description for dirty_background_ratio/dirty_ratio sysctl

2013-11-03 Thread Zheng Liu
From: Zheng Liu Now dirty_backgraound_ratio/dirty_ratio contains a percentage of total avaiable memory, which contains free pages and reclaimable pages. The number of these pages is not equal to the number of total system memory. But they are described as a percentage of total system memory

[PATCH] mm: improve the description for dirty_background_ratio/dirty_ratio sysctl

2013-11-03 Thread Zheng Liu
From: Zheng Liu wenqing...@taobao.com Now dirty_backgraound_ratio/dirty_ratio contains a percentage of total avaiable memory, which contains free pages and reclaimable pages. The number of these pages is not equal to the number of total system memory. But they are described as a percentage

Re: [PATCH 2/2] fs/ext4/namei.c: reducing contention on s_orphan_lock mmutex

2013-10-02 Thread Zheng Liu
Hello, On Wed, Oct 02, 2013 at 09:36:59AM -0600, T Makphaibulchoke wrote: > Instead of using a single per super block mutex, s_orphan_lock, to serialize > all orphan list updates, a separate mutex and spinlock are used to > protect the on disk and in memory orphan lists respecvitely. > > At the

Re: [PATCH 2/2] fs/ext4/namei.c: reducing contention on s_orphan_lock mmutex

2013-10-02 Thread Zheng Liu
Hello, On Wed, Oct 02, 2013 at 09:36:59AM -0600, T Makphaibulchoke wrote: Instead of using a single per super block mutex, s_orphan_lock, to serialize all orphan list updates, a separate mutex and spinlock are used to protect the on disk and in memory orphan lists respecvitely. At the same

Re: [PATCH] ext4: fix a bug when we try to open a file with O_TMPFILE flag

2013-07-20 Thread Zheng Liu
Hi Al, On Sat, Jul 20, 2013 at 12:36:13AM +0100, Al Viro wrote: > On Fri, Jul 19, 2013 at 08:14:05PM +0800, Zheng Liu wrote: > > Hi Dave, > > > > After applied this patch, the problem has been fixed in my own sand box. > > But that would be great if you could give it

Re: [PATCH] ext4: fix a bug when we try to open a file with O_TMPFILE flag

2013-07-20 Thread Zheng Liu
Hi Al, On Sat, Jul 20, 2013 at 12:36:13AM +0100, Al Viro wrote: On Fri, Jul 19, 2013 at 08:14:05PM +0800, Zheng Liu wrote: Hi Dave, After applied this patch, the problem has been fixed in my own sand box. But that would be great if you could give it a try. I want to make sure

[PATCH] ext4: fix a bug when we try to open a file with O_TMPFILE flag

2013-07-19 Thread Zheng Liu
for ext3 file system. Thanks, - Zheng From: Zheng Liu When we try to open a file with O_TMPFILE flag, we will trigger a bug. The root cause is that in ext4_orphan_add() we check ->i_nlink == 0 and this check always fails because we set ->i_nli

[PATCH] ext4: fix a bug when we try to open a file with O_TMPFILE flag

2013-07-19 Thread Zheng Liu
for ext3 file system. Thanks, - Zheng From: Zheng Liu wenqing...@taobao.com When we try to open a file with O_TMPFILE flag, we will trigger a bug. The root cause is that in ext4_orphan_add() we check -i_nlink == 0 and this check always fails because we

Re: kernel BUG at fs/ext4/namei.c:2572!

2013-07-18 Thread Zheng Liu
Hi Dave, Thanks for reporting this. On Wed, Jul 17, 2013 at 12:19:44PM -0400, Dave Jones wrote: > Seen while fuzzing.. > > kernel BUG at fs/ext4/namei.c:2572! > invalid opcode: [#1] PREEMPT SMP DEBUG_PAGEALLOC > Modules linked in: dlci bridge stp hidp cmtp kernelcapi l2tp_ppp l2tp_netlink

Re: kernel BUG at fs/ext4/namei.c:2572!

2013-07-18 Thread Zheng Liu
Hi Dave, Thanks for reporting this. On Wed, Jul 17, 2013 at 12:19:44PM -0400, Dave Jones wrote: Seen while fuzzing.. kernel BUG at fs/ext4/namei.c:2572! invalid opcode: [#1] PREEMPT SMP DEBUG_PAGEALLOC Modules linked in: dlci bridge stp hidp cmtp kernelcapi l2tp_ppp l2tp_netlink

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-07-14 Thread Zheng Liu
On Sun, Jul 14, 2013 at 11:12:58AM +0800, Sam Ben wrote: > On 07/02/2013 10:37 AM, Zheng Liu wrote: > >On Mon, Jul 01, 2013 at 09:16:46AM -0700, Dave Hansen wrote: > >>On 06/28/2013 07:20 PM, Zheng Liu wrote: > >>>>>IOW, a process needing to do a bunch of MAP_P

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-07-14 Thread Zheng Liu
On Sun, Jul 14, 2013 at 11:12:58AM +0800, Sam Ben wrote: On 07/02/2013 10:37 AM, Zheng Liu wrote: On Mon, Jul 01, 2013 at 09:16:46AM -0700, Dave Hansen wrote: On 06/28/2013 07:20 PM, Zheng Liu wrote: IOW, a process needing to do a bunch of MAP_POPULATEs isn't parallelizable, but one using

Re: 3.9-rc6 ext4: free_rb_tree_fname oops

2013-07-09 Thread Zheng Liu
Hi Daniel, On Mon, Jun 24, 2013 at 02:34:00PM +0800, Daniel J Blueman wrote: > On 16 April 2013 15:37, Daniel J Blueman wrote: > > When using e4defrag on a ext4 filesystem created a month ago, I ran > > into this fatal page fault [1] > > while running e4defrag on 3.9-rc6 (Ubuntu mainline). > >

Re: 3.9-rc6 ext4: free_rb_tree_fname oops

2013-07-09 Thread Zheng Liu
Hi Daniel, On Mon, Jun 24, 2013 at 02:34:00PM +0800, Daniel J Blueman wrote: On 16 April 2013 15:37, Daniel J Blueman dan...@quora.org wrote: When using e4defrag on a ext4 filesystem created a month ago, I ran into this fatal page fault [1] while running e4defrag on 3.9-rc6 (Ubuntu

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-07-01 Thread Zheng Liu
On Mon, Jul 01, 2013 at 09:43:29PM -0700, Dave Hansen wrote: > On 07/01/2013 07:37 PM, Zheng Liu wrote: > > FWIW, it would be great if we can let MAP_POPULATE flag support shared > > mappings because in our product system there has a lot of applications > > that uses mmap(2

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-07-01 Thread Zheng Liu
On Mon, Jul 01, 2013 at 09:16:46AM -0700, Dave Hansen wrote: > On 06/28/2013 07:20 PM, Zheng Liu wrote: > >> > IOW, a process needing to do a bunch of MAP_POPULATEs isn't > >> > parallelizable, but one using this mechanism would be. > > I look at the code,

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-07-01 Thread Zheng Liu
On Mon, Jul 01, 2013 at 09:16:46AM -0700, Dave Hansen wrote: On 06/28/2013 07:20 PM, Zheng Liu wrote: IOW, a process needing to do a bunch of MAP_POPULATEs isn't parallelizable, but one using this mechanism would be. I look at the code, and it seems that we will handle MAP_POPULATE flag

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-07-01 Thread Zheng Liu
On Mon, Jul 01, 2013 at 09:43:29PM -0700, Dave Hansen wrote: On 07/01/2013 07:37 PM, Zheng Liu wrote: FWIW, it would be great if we can let MAP_POPULATE flag support shared mappings because in our product system there has a lot of applications that uses mmap(2) and then pre-faults

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-06-28 Thread Zheng Liu
On 06/28/2013 11:48 PM, Dave Hansen wrote: > On 06/27/2013 10:47 PM, Zheng Liu wrote: >>> I've been doing some testing involving large amounts of >>> page cache. It's quite painful to get hundreds of GB >>> of page cache mapped in, especially when I am trying to

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-06-28 Thread Zheng Liu
On 06/28/2013 11:48 PM, Dave Hansen wrote: On 06/27/2013 10:47 PM, Zheng Liu wrote: I've been doing some testing involving large amounts of page cache. It's quite painful to get hundreds of GB of page cache mapped in, especially when I am trying to do it in parallel threads. This is true

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-06-27 Thread Zheng Liu
Hi Dave, On Thu, Jun 27, 2013 at 04:16:05PM -0700, Dave Hansen wrote: > > I've been doing some testing involving large amounts of > page cache. It's quite painful to get hundreds of GB > of page cache mapped in, especially when I am trying to > do it in parallel threads. This is true even when

Re: [RFC][PATCH] mm: madvise: MADV_POPULATE for quick pre-faulting

2013-06-27 Thread Zheng Liu
Hi Dave, On Thu, Jun 27, 2013 at 04:16:05PM -0700, Dave Hansen wrote: I've been doing some testing involving large amounts of page cache. It's quite painful to get hundreds of GB of page cache mapped in, especially when I am trying to do it in parallel threads. This is true even when the

Re: [PATCH 0/3] ext4: introduce two new ioctls

2013-06-24 Thread Zheng Liu
On Mon, Jun 24, 2013 at 10:37:57AM +, Sidorov, Andrei wrote: > On 24.06.2013 13:36, Namjae Jeon wrote: > > Currently, we can try implementing dave's suggesstion of introducing a > > new flag FALLOC_FL_COLLAPSE_RANGE for falloctae instead of individual > > ioctls for both XFS and EXT4. Thanks.

Re: [PATCH 0/3] ext4: introduce two new ioctls

2013-06-24 Thread Zheng Liu
On Mon, Jun 24, 2013 at 10:37:57AM +, Sidorov, Andrei wrote: On 24.06.2013 13:36, Namjae Jeon wrote: Currently, we can try implementing dave's suggesstion of introducing a new flag FALLOC_FL_COLLAPSE_RANGE for falloctae instead of individual ioctls for both XFS and EXT4. Thanks. Hi,

Re: linux-next: manual merge of the akpm tree with the ext4 tree

2013-06-19 Thread Zheng Liu
Hi Stephen, On Jun 19, 2013, at 3:27 PM, Stephen Rothwell wrote: > Hi Andrew, > > Today's linux-next merge of the akpm tree got a conflict in > fs/ext4/extents_status.c between commit 6480bad916be ("ext4: improve > extent cache shrink mechanism to avoid to burn CPU time") from the ext > tree

Re: linux-next: manual merge of the akpm tree with the ext4 tree

2013-06-19 Thread Zheng Liu
Hi Stephen, On Jun 19, 2013, at 3:27 PM, Stephen Rothwell s...@canb.auug.org.au wrote: Hi Andrew, Today's linux-next merge of the akpm tree got a conflict in fs/ext4/extents_status.c between commit 6480bad916be (ext4: improve extent cache shrink mechanism to avoid to burn CPU time) from

Re: ext4 extent status tree LRU locking

2013-06-17 Thread Zheng Liu
On Mon, Jun 17, 2013 at 10:51:34PM -0400, Theodore Ts'o wrote: > On Tue, Jun 18, 2013 at 10:25:48AM +0800, Zheng Liu wrote: > > Ah, sorry, I forgot to mention that this patch bases against ext4/master > > branch. Now ext4/dev branch has some regression when I run xfstests. >

Re: ext4 extent status tree LRU locking

2013-06-17 Thread Zheng Liu
On Mon, Jun 17, 2013 at 02:12:10PM -0700, Dave Hansen wrote: > On 06/17/2013 03:10 AM, Zheng Liu wrote: > > Dave, that would be great if you could do your testing again to confirm > > this patch is useful. > > I was able to apply this to Ted's > > https://git.k

Re: ext4 extent status tree LRU locking

2013-06-17 Thread Zheng Liu
On Fri, Jun 14, 2013 at 02:00:54PM -0400, Theodore Ts'o wrote: > On Sat, Jun 15, 2013 at 01:00:28AM +0800, Zheng Liu wrote: > > > I have a suggestion for how to address this: Keep a timestamp of when > > > the list last has been sorted in struct ext4_super_info. When > >

Re: ext4 extent status tree LRU locking

2013-06-17 Thread Zheng Liu
On Fri, Jun 14, 2013 at 02:00:54PM -0400, Theodore Ts'o wrote: On Sat, Jun 15, 2013 at 01:00:28AM +0800, Zheng Liu wrote: I have a suggestion for how to address this: Keep a timestamp of when the list last has been sorted in struct ext4_super_info. When iterating over the list, looking

Re: ext4 extent status tree LRU locking

2013-06-17 Thread Zheng Liu
On Mon, Jun 17, 2013 at 02:12:10PM -0700, Dave Hansen wrote: On 06/17/2013 03:10 AM, Zheng Liu wrote: Dave, that would be great if you could do your testing again to confirm this patch is useful. I was able to apply this to Ted's https://git.kernel.org/cgit/linux/kernel/git/tytso

Re: ext4 extent status tree LRU locking

2013-06-17 Thread Zheng Liu
On Mon, Jun 17, 2013 at 10:51:34PM -0400, Theodore Ts'o wrote: On Tue, Jun 18, 2013 at 10:25:48AM +0800, Zheng Liu wrote: Ah, sorry, I forgot to mention that this patch bases against ext4/master branch. Now ext4/dev branch has some regression when I run xfstests. What regressions are you

Re: ext4 extent status tree LRU locking

2013-06-14 Thread Zheng Liu
On Fri, Jun 14, 2013 at 08:57:44AM -0700, Dave Hansen wrote: > On 06/14/2013 07:09 AM, Zheng Liu wrote: > > - INIT_LIST_HEAD(); > > - > > spin_lock(>s_es_lru_lock); > > + list_sort(NULL, >s_es_lru, ext4_inode_touch_time_cmp); > > list_for_each_sa

Re: ext4 extent status tree LRU locking

2013-06-14 Thread Zheng Liu
On Fri, Jun 14, 2013 at 10:02:15AM -0400, Theodore Ts'o wrote: > On Fri, Jun 14, 2013 at 10:09:40PM +0800, Zheng Liu wrote: > > This commit tries to fix this problem. Now a new member called > > i_touch_when is added into ext4_inode_info to record the last access > > time for

Re: ext4 extent status tree LRU locking

2013-06-14 Thread Zheng Liu
nt or suggestion is welcome. Thanks, - Zheng Subject: [PATCH] ext4: improve extent cache shrink mechanism to avoid to burn CPU time From: Zheng Liu Now we maintain an proper in-order LRU list in ext4 to reclaim entries from extent status tree whe

Re: ext4 extent status tree LRU locking

2013-06-14 Thread Zheng Liu
, - Zheng Subject: [PATCH] ext4: improve extent cache shrink mechanism to avoid to burn CPU time From: Zheng Liu wenqing...@taobao.com Now we maintain an proper in-order LRU list in ext4 to reclaim entries from extent status tree when we are under heavy memory

Re: ext4 extent status tree LRU locking

2013-06-14 Thread Zheng Liu
On Fri, Jun 14, 2013 at 10:02:15AM -0400, Theodore Ts'o wrote: On Fri, Jun 14, 2013 at 10:09:40PM +0800, Zheng Liu wrote: This commit tries to fix this problem. Now a new member called i_touch_when is added into ext4_inode_info to record the last access time for an inode. Meanwhile we

Re: ext4 extent status tree LRU locking

2013-06-14 Thread Zheng Liu
On Fri, Jun 14, 2013 at 08:57:44AM -0700, Dave Hansen wrote: On 06/14/2013 07:09 AM, Zheng Liu wrote: - INIT_LIST_HEAD(scanned); - spin_lock(sbi-s_es_lru_lock); + list_sort(NULL, sbi-s_es_lru, ext4_inode_touch_time_cmp); list_for_each_safe(cur, tmp, sbi-s_es_lru) { How

Re: ext4 extent status tree LRU locking

2013-06-13 Thread Zheng Liu
On Thu, Jun 13, 2013 at 09:35:17AM -0400, Theodore Ts'o wrote: > On Thu, Jun 13, 2013 at 09:27:15PM +0800, Zheng Liu wrote: > > Thanks for your suggestion. But, sorry, I couldn't get your point here. > > As you suggested above, we can tag each inode with the last acces

Re: ext4 extent status tree LRU locking

2013-06-13 Thread Zheng Liu
Hi Ted, On Wed, Jun 12, 2013 at 04:48:54PM -0400, Theodore Ts'o wrote: > On Wed, Jun 12, 2013 at 08:09:14AM -0700, Dave Hansen wrote: > > You could make per-cpu LRUs, and batch movement on and off the global > > LRU once the local ones get to be a certain size. Or, you could keep > > them

Re: ext4 extent status tree LRU locking

2013-06-13 Thread Zheng Liu
Hi Ted, On Wed, Jun 12, 2013 at 04:48:54PM -0400, Theodore Ts'o wrote: On Wed, Jun 12, 2013 at 08:09:14AM -0700, Dave Hansen wrote: You could make per-cpu LRUs, and batch movement on and off the global LRU once the local ones get to be a certain size. Or, you could keep them cpu-local

Re: ext4 extent status tree LRU locking

2013-06-13 Thread Zheng Liu
On Thu, Jun 13, 2013 at 09:35:17AM -0400, Theodore Ts'o wrote: On Thu, Jun 13, 2013 at 09:27:15PM +0800, Zheng Liu wrote: Thanks for your suggestion. But, sorry, I couldn't get your point here. As you suggested above, we can tag each inode with the last access time when ext4_map_blocks

Re: ext4 extent status tree LRU locking

2013-06-12 Thread Zheng Liu
On Wed, Jun 12, 2013 at 08:09:14AM -0700, Dave Hansen wrote: > On 06/12/2013 12:17 AM, Zheng Liu wrote: > > On Tue, Jun 11, 2013 at 04:22:16PM -0700, Dave Hansen wrote: > >> I've got a test case which I intended to use to stress the VM a bit. It > >> fills memory

Re: ext4 extent status tree LRU locking

2013-06-12 Thread Zheng Liu
Hi Dave Thanks for reporting this. On Tue, Jun 11, 2013 at 04:22:16PM -0700, Dave Hansen wrote: > I've got a test case which I intended to use to stress the VM a bit. It > fills memory up with page cache a couple of times. It essentially runs > 30 or so cp's in parallel. Could you please

Re: ext4 extent status tree LRU locking

2013-06-12 Thread Zheng Liu
Hi Dave Thanks for reporting this. On Tue, Jun 11, 2013 at 04:22:16PM -0700, Dave Hansen wrote: I've got a test case which I intended to use to stress the VM a bit. It fills memory up with page cache a couple of times. It essentially runs 30 or so cp's in parallel. Could you please share

Re: ext4 extent status tree LRU locking

2013-06-12 Thread Zheng Liu
On Wed, Jun 12, 2013 at 08:09:14AM -0700, Dave Hansen wrote: On 06/12/2013 12:17 AM, Zheng Liu wrote: On Tue, Jun 11, 2013 at 04:22:16PM -0700, Dave Hansen wrote: I've got a test case which I intended to use to stress the VM a bit. It fills memory up with page cache a couple of times

Re: [PATCH v2] ext4: Avoid unnecessarily writing back dirty pages before hole punching

2013-05-20 Thread Zheng Liu
s, no matter they are dirty or not, except the ordered case. > > Signed-off-by: Li Wang > Signed-off-by: Yunchuan Wen > Reviewed-by: Zheng Liu FWIW, you could Cc me but you couldn't add 'Reviewed-by' here because in first version I give a comment to you, and I don't think th

Re: [PATCH v2] ext4: Avoid unnecessarily writing back dirty pages before hole punching

2013-05-20 Thread Zheng Liu
they are dirty or not, except the ordered case. Signed-off-by: Li Wang liw...@ubuntukylin.com Signed-off-by: Yunchuan Wen yunchuan...@ubuntukylin.com Reviewed-by: Zheng Liu gnehzuil@gmail.com FWIW, you could Cc me but you couldn't add 'Reviewed-by' here because in first version I give a comment

Re: [PATCH 0/2] ext4: deadlocks after allocation failure in ext4_init_io_end()

2013-05-14 Thread Zheng Liu
On Mon, May 13, 2013 at 11:08:11PM +0400, Alexey Khoroshilov wrote: > Hi, Ted! > > Our tests for ext4 with targeted fault injection were stalled in > Uninterruptible Sleep State > when they simulate a memory allocation failure in ext4_init_io_end() while it > is called from >

Re: [PATCH 0/2] ext4: deadlocks after allocation failure in ext4_init_io_end()

2013-05-14 Thread Zheng Liu
On Mon, May 13, 2013 at 11:08:11PM +0400, Alexey Khoroshilov wrote: Hi, Ted! Our tests for ext4 with targeted fault injection were stalled in Uninterruptible Sleep State when they simulate a memory allocation failure in ext4_init_io_end() while it is called from mpage_da_submit_io() or

Re: Re: Re: EXT4 regression caused 4eec7

2013-05-13 Thread Zheng Liu
On Mon, May 13, 2013 at 05:17:27PM +0200, Jan Kara wrote: > On Mon 13-05-13 21:56:43, Zheng Liu wrote: > > On Mon, May 13, 2013 at 03:18:09PM +0200, Jan Kara wrote: > > > On Sun 12-05-13 13:04:59, EUNBONG SONG wrote: > > > > > > > > > > &

Re: Re: Re: EXT4 regression caused 4eec7

2013-05-13 Thread Zheng Liu
On Mon, May 13, 2013 at 03:18:09PM +0200, Jan Kara wrote: > On Sun 12-05-13 13:04:59, EUNBONG SONG wrote: > > > > > > >> Since at this point it's safer to rollback the change and we can > > >> investigate more deeply how to fix it correctly for the next > > >> development cycle, this is the

Re: Re: Re: EXT4 panic at jbd2_journal_put_journal_head() in 3.9+

2013-05-13 Thread Zheng Liu
On Mon, May 13, 2013 at 09:53:25AM +, EUNBONG SONG wrote: > > > > Hi all, > > > First of all I couldn't reproduce this regression in my sand box. So > > the following speculation is only my guess. I suspect that the commit > > (ae4647fb) isn't root cause. It just uncover a potential bug

Re: Re: EXT4 panic at jbd2_journal_put_journal_head() in 3.9+

2013-05-13 Thread Zheng Liu
00:03:42 2013 -0400 > > jbd2: reduce journal_head size > > Remove unused t_cow_tid field (ext4 copy-on-write support doesn't seem > to be happening) and change b_modified and b_jlist to bitfields thus > saving 8 bytes in the structure. > > Signed-off-by

Re: Re: EXT4 panic at jbd2_journal_put_journal_head() in 3.9+

2013-05-13 Thread Zheng Liu
Reviewed-by: Zheng Liu wenqing...@taobao.com :04 04 c39ece4341894b3daf84764ba425a87ffb90fe50 d4e8d9185c2a1b740c235ca8ed05d496a442fce3 M include Hi all, First of all I couldn't reproduce this regression in my sand box. So the following speculation is only my guess. I suspect

Re: Re: Re: EXT4 panic at jbd2_journal_put_journal_head() in 3.9+

2013-05-13 Thread Zheng Liu
On Mon, May 13, 2013 at 09:53:25AM +, EUNBONG SONG wrote: Hi all, First of all I couldn't reproduce this regression in my sand box. So the following speculation is only my guess. I suspect that the commit (ae4647fb) isn't root cause. It just uncover a potential bug that has

Re: Re: Re: EXT4 regression caused 4eec7

2013-05-13 Thread Zheng Liu
On Mon, May 13, 2013 at 03:18:09PM +0200, Jan Kara wrote: On Sun 12-05-13 13:04:59, EUNBONG SONG wrote: Since at this point it's safer to rollback the change and we can investigate more deeply how to fix it correctly for the next development cycle, this is the patch which I'm

Re: Re: Re: EXT4 regression caused 4eec7

2013-05-13 Thread Zheng Liu
On Mon, May 13, 2013 at 05:17:27PM +0200, Jan Kara wrote: On Mon 13-05-13 21:56:43, Zheng Liu wrote: On Mon, May 13, 2013 at 03:18:09PM +0200, Jan Kara wrote: On Sun 12-05-13 13:04:59, EUNBONG SONG wrote: Since at this point it's safer to rollback the change and we can

Re: [PATCH v3 17/18] ext4: make punch hole code path work with bigalloc

2013-04-24 Thread Zheng Liu
On Wed, Apr 24, 2013 at 01:08:17PM +0200, Lukáš Czerner wrote: > On Tue, 23 Apr 2013, Zheng Liu wrote: [snip] > > > > Also update respective tracepoints to use signed long long type for > > > > partial_cluster. > > > The patch looks OK. You

Re: [PATCH v3 17/18] ext4: make punch hole code path work with bigalloc

2013-04-24 Thread Zheng Liu
On Wed, Apr 24, 2013 at 01:08:17PM +0200, Lukáš Czerner wrote: On Tue, 23 Apr 2013, Zheng Liu wrote: [snip] Also update respective tracepoints to use signed long long type for partial_cluster. The patch looks OK. You can add: Reviewed-by: Jan Kara j...@suse.cz Just

Re: [PATCH v3 17/18] ext4: make punch hole code path work with bigalloc

2013-04-23 Thread Zheng Liu
On Sat, Apr 20, 2013 at 03:42:41PM +0200, Jan Kara wrote: > On Tue 09-04-13 11:14:26, Lukas Czerner wrote: > > Currently punch hole is disabled in file systems with bigalloc > > feature enabled. However the recent changes in punch hole patch should > > make it easier to support punching holes on

Re: [PATCH v3 17/18] ext4: make punch hole code path work with bigalloc

2013-04-23 Thread Zheng Liu
On Sat, Apr 20, 2013 at 03:42:41PM +0200, Jan Kara wrote: On Tue 09-04-13 11:14:26, Lukas Czerner wrote: Currently punch hole is disabled in file systems with bigalloc feature enabled. However the recent changes in punch hole patch should make it easier to support punching holes on bigalloc

Re: [PATCH 1/3] ext4: mark all metadata I/O with REQ_META

2013-04-22 Thread Zheng Liu
ore Ts'o" Reviewed-by: Zheng Liu Regards, - Zheng > --- > fs/ext4/balloc.c | 2 +- > fs/ext4/ialloc.c | 2 +- > fs/ext4/mmp.c| 4 ++-- > fs/ext4/super.c | 2 +- > 4 files changed, 5 insertions(+), 5 deletions(-) &

Re: 3.9-rc6 ext4: free_rb_tree_fname oops

2013-04-22 Thread Zheng Liu
[Cc ext4 mailing list to let other folks know] On Tue, Apr 16, 2013 at 03:37:31PM +0800, Daniel J Blueman wrote: > When using e4defrag on a ext4 filesystem created a month ago, I ran > into this fatal page fault [1] > while running e4defrag on 3.9-rc6 (Ubuntu mainline). > > e2fsdump output is

Re: 3.9-rc6 ext4: free_rb_tree_fname oops

2013-04-22 Thread Zheng Liu
[Cc ext4 mailing list to let other folks know] On Tue, Apr 16, 2013 at 03:37:31PM +0800, Daniel J Blueman wrote: When using e4defrag on a ext4 filesystem created a month ago, I ran into this fatal page fault [1] while running e4defrag on 3.9-rc6 (Ubuntu mainline). e2fsdump output is at

Re: [PATCH 1/3] ext4: mark all metadata I/O with REQ_META

2013-04-22 Thread Zheng Liu
Reviewed-by: Zheng Liu wenqing...@taobao.com Regards, - Zheng --- fs/ext4/balloc.c | 2 +- fs/ext4/ialloc.c | 2 +- fs/ext4/mmp.c| 4 ++-- fs/ext4/super.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/ext4

Re: [PATCH] ext4: unregister extents status shrinker if mount failed

2013-04-19 Thread Zheng Liu
On 04/20/2013 06:05 AM, Alexey Khoroshilov wrote: > If ext4_fill_super() failed after extents status shrinker > has been registered, the shrinker is left in a global list > while the memory, it sits in, is already freed. > Oops is not so bad scenario after that. > > Found by Linux File System

Re: [PATCH] ext4: unregister extents status shrinker if mount failed

2013-04-19 Thread Zheng Liu
On 04/20/2013 06:05 AM, Alexey Khoroshilov wrote: If ext4_fill_super() failed after extents status shrinker has been registered, the shrinker is left in a global list while the memory, it sits in, is already freed. Oops is not so bad scenario after that. Found by Linux File System

Re: s390x: kernel BUG at fs/ext4/inode.c:1591! (powerpc too!)

2013-04-03 Thread Zheng Liu
org/patch/233397/ >> I hope this should fix all known issues > > Zheng Liu also sent a patch: > > [PATCH] ext4: fix a big-endian bug when an extent is zeroed out > > When I try to apply all three of those to 3.9-4c4, the 2nd one from Dmitry > fails: > > $ cat ~

Re: [PATCH] ext4: fix big-endian bugs which could cause fs corruptions

2013-04-03 Thread Zheng Liu
On 04/04/2013 12:36 AM, Theodore Ts'o wrote: > From: Zheng Liu > > From: Zheng Liu > > When an extent was zeroed out, we forgot to do convert from cpu to le16. > It could make us hit a BUG_ON when we try to write dirty pages out. So > fix it. > > [ Also fix a b

Re: s390x: kernel BUG at fs/ext4/inode.c:1591! (powerpc too!)

2013-04-03 Thread Zheng Liu
arse to check this problem. I think the following patch could fix this bug. Regards, - Zheng Subject: [PATCH] ext4: fix a big-endian bug when an extent is zeroed out From: Zheng Liu When an extent was zeroed out, we forgot to do convert from cpu to le16. It could make us hit a BUG_ON when we try to

Re: s390x: kernel BUG at fs/ext4/inode.c:1591! (powerpc too!)

2013-04-03 Thread Zheng Liu
, - Zheng Subject: [PATCH] ext4: fix a big-endian bug when an extent is zeroed out From: Zheng Liu wenqing...@taobao.com When an extent was zeroed out, we forgot to do convert from cpu to le16. It could make us hit a BUG_ON when we try to write dirty pages out. So fix it. Signed-off

Re: [PATCH] ext4: fix big-endian bugs which could cause fs corruptions

2013-04-03 Thread Zheng Liu
On 04/04/2013 12:36 AM, Theodore Ts'o wrote: From: Zheng Liu wenqing...@taobao.com From: Zheng Liu wenqing...@taobao.com When an extent was zeroed out, we forgot to do convert from cpu to le16. It could make us hit a BUG_ON when we try to write dirty pages out. So fix it. [ Also fix

Re: s390x: kernel BUG at fs/ext4/inode.c:1591! (powerpc too!)

2013-04-03 Thread Zheng Liu
fix all known issues Zheng Liu also sent a patch: [PATCH] ext4: fix a big-endian bug when an extent is zeroed out When I try to apply all three of those to 3.9-4c4, the 2nd one from Dmitry fails: $ cat ~/dev/002-ext4_fix-cpu_vs_disk-conversions.diff | patch --dry-run -p1 patching file

Re: Excessive stall times on ext4 in 3.9-rc2

2013-04-02 Thread Zheng Liu
Hi Mel, Thanks for reporting it. On 04/02/2013 10:27 PM, Mel Gorman wrote: > I'm testing a page-reclaim-related series on my laptop that is partially > aimed at fixing long stalls when doing metadata-intensive operations on > low memory such as a git checkout. I've been running 3.9-rc2 with the

Re: s390x: kernel BUG at fs/ext4/inode.c:1591! (powerpc too!)

2013-04-02 Thread Zheng Liu
On Tue, Apr 02, 2013 at 01:47:44PM +0400, Dmitry Monakhov wrote: > On Mon, 1 Apr 2013 23:15:07 -0700 (PDT), Christian Kujau > wrote: > > Hi, > > > > my machine (PowerBook G4) just crashed and the only thing netconsole was > > able to transmit was: > > > > [ cut here

Re: s390x: kernel BUG at fs/ext4/inode.c:1591! (powerpc too!)

2013-04-02 Thread Zheng Liu
On Tue, Apr 02, 2013 at 01:47:44PM +0400, Dmitry Monakhov wrote: On Mon, 1 Apr 2013 23:15:07 -0700 (PDT), Christian Kujau li...@nerdbynature.de wrote: Hi, my machine (PowerBook G4) just crashed and the only thing netconsole was able to transmit was: [ cut here

Re: Excessive stall times on ext4 in 3.9-rc2

2013-04-02 Thread Zheng Liu
Hi Mel, Thanks for reporting it. On 04/02/2013 10:27 PM, Mel Gorman wrote: I'm testing a page-reclaim-related series on my laptop that is partially aimed at fixing long stalls when doing metadata-intensive operations on low memory such as a git checkout. I've been running 3.9-rc2 with the

Re: torrent hash failures since 3.9.0-rc1

2013-03-13 Thread Zheng Liu
On Tue, Mar 12, 2013 at 09:28:11AM -0400, Theodore Ts'o wrote: > On Tue, Mar 12, 2013 at 03:16:06PM +0800, Zheng Liu wrote: > > > > Ted, I am wandering if we need to Cc this patch to stable kernel. We > > don't receive any report to complaint it, though, but it is worth

Re: torrent hash failures since 3.9.0-rc1

2013-03-13 Thread Zheng Liu
On Tue, Mar 12, 2013 at 09:28:11AM -0400, Theodore Ts'o wrote: On Tue, Mar 12, 2013 at 03:16:06PM +0800, Zheng Liu wrote: Ted, I am wandering if we need to Cc this patch to stable kernel. We don't receive any report to complaint it, though, but it is worth backporting it I think

Re: torrent hash failures since 3.9.0-rc1

2013-03-12 Thread Zheng Liu
On Tue, Mar 12, 2013 at 07:48:10AM +0100, Markus Trippelsdorf wrote: > On 2013.03.12 at 14:44 +0800, Zheng Liu wrote: > > On Tue, Mar 12, 2013 at 07:16:24AM +0100, Markus Trippelsdorf wrote: > > > On 2013.03.11 at 23:30 -0400, Theodore Ts'o wrote: > > > > On Tue, Ma

Re: torrent hash failures since 3.9.0-rc1

2013-03-12 Thread Zheng Liu
On Tue, Mar 12, 2013 at 07:16:24AM +0100, Markus Trippelsdorf wrote: > On 2013.03.11 at 23:30 -0400, Theodore Ts'o wrote: > > On Tue, Mar 12, 2013 at 11:00:58AM +0800, Zheng Liu wrote: > > > > > > Thanks for reporting this problem. My deepest apologies. > > >

Re: torrent hash failures since 3.9.0-rc1

2013-03-12 Thread Zheng Liu
On Tue, Mar 12, 2013 at 07:16:24AM +0100, Markus Trippelsdorf wrote: On 2013.03.11 at 23:30 -0400, Theodore Ts'o wrote: On Tue, Mar 12, 2013 at 11:00:58AM +0800, Zheng Liu wrote: Thanks for reporting this problem. My deepest apologies. As Ted suggested, could you please try

Re: torrent hash failures since 3.9.0-rc1

2013-03-12 Thread Zheng Liu
On Tue, Mar 12, 2013 at 07:48:10AM +0100, Markus Trippelsdorf wrote: On 2013.03.12 at 14:44 +0800, Zheng Liu wrote: On Tue, Mar 12, 2013 at 07:16:24AM +0100, Markus Trippelsdorf wrote: On 2013.03.11 at 23:30 -0400, Theodore Ts'o wrote: On Tue, Mar 12, 2013 at 11:00:58AM +0800, Zheng Liu

Re: torrent hash failures since 3.9.0-rc1

2013-03-11 Thread Zheng Liu
. > > > > I've started a full bisection from v3.8 to todays git tree. It will take > > ~13 steps. However it's already late here in Germany. I will continue > > the bisection tomorrow and report back. > > The issue started with: > > 74cd15cd02708c7188581f279f33

Re: torrent hash failures since 3.9.0-rc1

2013-03-11 Thread Zheng Liu
~13 steps. However it's already late here in Germany. I will continue the bisection tomorrow and report back. The issue started with: 74cd15cd02708c7188581f279f33a98b2ae8d322 is the first bad commit commit 74cd15cd02708c7188581f279f33a98b2ae8d322 Author: Zheng Liu wenqing...@taobao.com

Re: [GIT PULL] ext4 updates for 3.9

2013-02-27 Thread Zheng Liu
On 02/28/2013 03:19 AM, Dave Jones wrote: > On Wed, Feb 27, 2013 at 07:56:25PM +0100, Markus Trippelsdorf wrote: > > On 2013.02.27 at 13:49 -0500, Theodore Ts'o wrote: > > > > > Markus, Dave, can you confirm that this fixes your problem? > > > > Yes, it fixes the issue. > > Looks like it's

Re: [GIT PULL] ext4 updates for 3.9

2013-02-27 Thread Zheng Liu
block number > 152052301 (from the syslog) is 0x910224D, while block number 688923213 > is 0x2910224D. Note the missing "0x2000" in the block number. > > Reported-by: Markus Trippelsdorf > Reported-by: Dave Jones > Cc: Zheng Liu > Signed-off-by: "The

Re: [GIT PULL] ext4 updates for 3.9

2013-02-27 Thread Zheng Liu
in the block number. Reported-by: Markus Trippelsdorf mar...@trippelsdorf.de Reported-by: Dave Jones da...@redhat.com Cc: Zheng Liu gnehzuil@gmail.com Signed-off-by: Theodore Ts'o ty...@mit.edu --- fs/ext4/extents_status.h | 16 1 file changed, 8 insertions(+), 8 deletions

Re: [GIT PULL] ext4 updates for 3.9

2013-02-27 Thread Zheng Liu
On 02/28/2013 03:19 AM, Dave Jones wrote: On Wed, Feb 27, 2013 at 07:56:25PM +0100, Markus Trippelsdorf wrote: On 2013.02.27 at 13:49 -0500, Theodore Ts'o wrote: Markus, Dave, can you confirm that this fixes your problem? Yes, it fixes the issue. Looks like it's fixed here

Re: [RFC][PATCH] vfs: always protect diretory file->fpos with inode mutex

2013-02-19 Thread Zheng Liu
On Tue, Feb 19, 2013 at 09:22:40AM +0800, Li Zefan wrote: > There's a long long-standing bug...As long as I don't know when it dates > from. > > I've written and attached a simple program to reproduce this bug, and it can > immediately trigger the bug in my box. It uses two threads, one keeps

Re: [RFC][PATCH] vfs: always protect diretory file-fpos with inode mutex

2013-02-19 Thread Zheng Liu
On Tue, Feb 19, 2013 at 09:22:40AM +0800, Li Zefan wrote: There's a long long-standing bug...As long as I don't know when it dates from. I've written and attached a simple program to reproduce this bug, and it can immediately trigger the bug in my box. It uses two threads, one keeps calling

Re: [PATCH 0/6 RFC] Mapping range lock

2013-02-04 Thread Zheng Liu
On Thu, Jan 31, 2013 at 04:07:57PM -0800, Andrew Morton wrote: [snip] > > c) i_mutex doesn't allow any paralellism of operations using it and some > >filesystems workaround this for specific cases (e.g. DIO reads). Using > >range locking allows for concurrent operations (e.g. writes, DIO)

Re: [PATCH 0/6 RFC] Mapping range lock

2013-02-04 Thread Zheng Liu
On Thu, Jan 31, 2013 at 04:07:57PM -0800, Andrew Morton wrote: [snip] c) i_mutex doesn't allow any paralellism of operations using it and some filesystems workaround this for specific cases (e.g. DIO reads). Using range locking allows for concurrent operations (e.g. writes, DIO) on

Re: [PATCH v2] fadvise: perform WILLNEED readahead asynchronously

2012-12-26 Thread Zheng Liu
64(3, 0, 1048576, POSIX_FADV_WILLNEED) = 0 <0.000474> > After: fadvise64(3, 0, 1048576, POSIX_FADV_WILLNEED) = 0 <0.63> I do a simple test in my desktop, which reads 128k data. W/o this patch, this syscall takes 32us, and w/ this patch it takes 7us. You can add: Tested-by: Zheng Liu >

Re: [PATCH v2] fadvise: perform WILLNEED readahead asynchronously

2012-12-26 Thread Zheng Liu
a simple test in my desktop, which reads 128k data. W/o this patch, this syscall takes 32us, and w/ this patch it takes 7us. You can add: Tested-by: Zheng Liu wenqing...@taobao.com While userspace can mimic the effect of this commit by using a background thread to perform readahead(), this allows

  1   2   >