Hi Jaegeuk,
> -----Original Message-----
> From: Jaegeuk Kim [mailto:jaeg...@kernel.org]
> Sent: Sunday, January 24, 2016 4:16 AM
> To: Chao Yu
> Cc: 'Dave Chinner'; linux-f2fs-devel@lists.sourceforge.net
> Subject: Re: [f2fs-dev] [oops, 4.4-rc8] warn + oops during generic/204
>
> Hi Chao,
>
> On Fri, Jan 22, 2016 at 02:15:26PM +0800, Chao Yu wrote:
> > > -----Original Message-----
> > > From: Jaegeuk Kim [mailto:jaeg...@kernel.org]
> > > Sent: Friday, January 22, 2016 9:53 AM
> > > To: Chao Yu
> > > Cc: 'Dave Chinner'; linux-f2fs-devel@lists.sourceforge.net
> > > Subject: Re: [f2fs-dev] [oops, 4.4-rc8] warn + oops during generic/204
> > >
> > > On Thu, Jan 21, 2016 at 07:46:10PM +0800, Chao Yu wrote:
> > > > Hi Dave,
> > > >
> > > > > -----Original Message-----
> > > > > From: Dave Chinner [mailto:da...@fromorbit.com]
> > > > > Sent: Thursday, January 21, 2016 5:31 AM
> > > > > To: linux-f2fs-devel@lists.sourceforge.net
> > > > > Subject: [f2fs-dev] [oops, 4.4-rc8] warn + oops during generic/204
> > > > >
> > > > > Hi f2fs folks,
> > > > >
> > > > > I just ran xfstests on f2fs using defaults and a pair of 4GB ram
> > > > > disks for the test and scratch devices, and it hard locked the
> > > > > machine with this failure in generic/204:
> > > >
> > > > Thanks for your report! :)
> > > >
> > > > Hi all,
> > > >
> > > > We didn't handle well with the case of inline data storm which floods
> > > > full disk. Actually the reason is: if we have 10M free space, and user
> > > > fillings the disk with ~10M inline data, then in memory there are ~10M
> > > > dirty inline datas and ~10M dirty inodes, once inodes were writebacked
> > > > before inline datas, all free space will be occupied, then we have to
> > > > write these dirty inline datas to ovp area which doesn't have enough
> > > > space there normally.
> > >
> > > Well, I think the user block count was wrong which is determined by
> > > mkfs.f2fs.
> > > When writing inline_data, gc should activate to reclaim the space, but it
> > > seems it couldn't do that cause there is no victim.
> > > So, when taking a look at mkfs, I suspect that the number of total user
> > > blocks
> > > does not consider our current segments, which is 6 by default.
> > > IOWs, we gave more blocks to users, which turns out actually gc couldn't
> > > get a
> > > victim from them in this case.
> >
> > Hmm, I don't think this could fix similar inline storm issue, I change some
> > parameters in generic/204 like below patch, and do the test with your patch
> > based on commit 9b72a388f586 ("f2fs: skip releasing nodes in chindless
> > extent
> > tree"), oops occurs again, still the same problem. :(
>
> Urg, yes.
>
> >
> > ---
> > tests/generic/204 | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/tests/generic/204 b/tests/generic/204
> > index 24a2a94..4db1533 100755
> > --- a/tests/generic/204
> > +++ b/tests/generic/204
> > @@ -56,7 +56,7 @@ _scratch_mkfs 2> /dev/null | _filter_mkfs 2> $tmp.mkfs >
> > /dev/null
> > # And v4/512 v5/1k xfs don't have enough free inodes, set imaxpct=50 at
> > mkfs
> > # time solves this problem.
> > [ $FSTYP = "xfs" ] && MKFS_OPTIONS="$MKFS_OPTIONS -l size=7m -i maxpct=50"
> > -SIZE=`expr 106 \* 1024 \* 1024`
> > +SIZE=`expr 406 \* 1024 \* 1024`
> >
> > _scratch_mkfs_sized $SIZE $dbsize 2> /dev/null \
> > | _filter_mkfs 2> $tmp.mkfs > /dev/null
> > @@ -69,7 +69,7 @@ _scratch_mount
> > # work out correctly. Space usages is based 22500 files and 1024 reserved
> > blocks
> > # on a 4k block size 256 byte inode size filesystem.
> > resv_blks=1024
> > -space=97920000
> > +space=397920000
> >
> > # decrease files for inode size.
> > # 22500 * (256 + 4k) = ~97MB
> > --
> > 2.6.3
> >
> > I traced the details of this issue, we can get some hints with below datas:
> >
> > steps in this issue:
> > a) generic/204: submit 22 * 512 inline data
> > b) f2fs_write_node_pages: f2fs_balance_fs_bg->f2fs_sync_fs
> > c) f2fs_write_data_pages: write inline data page into inode page
> > d) f2fs_write_node_pages: persist inode pages
> >
> >
> > Test #1, mkfs.f2fs: not patched, generic/204: not patched:
> > [MAIN: 45(OverProv:23 Resv:16)]
> > valid user segment = 45 - 23 = 22
> > inline data dentry node free segment
> > prefree+dirty
> > a) 22 1 22 39 0
> > b) 22 0 0 18 0
> > c) 0 0 22 18 0
> > d) 0 0 4 0 19
> >
> > Test #2, mkfs.f2fs: patched, generic/204: not patched:
> > [MAIN: 45(OverProv:23 Resv:16)]
> > valid user segment = 45 - 23 - 6 = 16
> > inline data dentry node free segment
> > prefree+dirty
> > a) 16 1 16 39 0
> > b) 16 0 0 23 0
> > c) 0 0 16 23 0
> > d) 0 0 0 7 16
> >
> > Test #3, mkfs.f2fs: patched, generic/204: patched:
> > [MAIN: 195(OverProv:44 Resv:28)]
> > valid user segment = 195 - 44 - 6 = 145
> > inline data dentry node free segment
> > prefree+dirty
> > a) 145 1 145 189 0
> > b) 145 0 0 45 0
> > c) 0 0 145 45 0
> > d) 0 0 99 0 46
> >
> > Once we arrive step c), we would face the danger, because writebacking
> > of dirty node page will exhaust free segments, result in oops.
> > During step c) to step d), inode pages were COWed, prefree will increase,
> > it's a pity these prefree segments could only be reused after a checkpoint,
> > however checkpoint needs more free space for flushing dirty nodes, that
> > also will exhaust free segments, this likes a chicken and egg problem.
> >
> > Any idea to solve this?
>
> One question is why f2fs_gc couldn't make the free space during c).
> The answer was there was no victim and no prefree segments, but the dirty node
> blocks are producing suddenly.
Yes, better to avoid entering step c).
>
> I wrote two patches regarding to this issue.
> Could you check them too?
Please see the comments on them.
thanks,
>
> Thanks,
>
> >
> > Thanks,
> >
> > >
> > > I could reproduce this issue, and once I reduced the number by 6 segments,
> > > I could avoid the issue.
> > >
> > > Here the change of f2fs-tools.
> > >
> > > From 411166a44009bb138eca937376863e9ce673278a Mon Sep 17 00:00:00 2001
> > > From: Jaegeuk Kim <jaeg...@kernel.org>
> > > Date: Thu, 21 Jan 2016 05:16:37 -0800
> > > Subject: [PATCH] mkfs.f2fs: adjust current segments for total usable
> > > blocks
> > >
> > > When calculating total number of user blocks, we should reserve the
> > > current
> > > segments.
> > >
> > > Signed-off-by: Jaegeuk Kim <jaeg...@kernel.org>
> > > ---
> > > mkfs/f2fs_format.c | 7 ++++++-
> > > 1 file changed, 6 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> > > index 2c81ecc..b25a490 100644
> > > --- a/mkfs/f2fs_format.c
> > > +++ b/mkfs/f2fs_format.c
> > > @@ -474,7 +474,12 @@ static int f2fs_write_check_point_pack(void)
> > >
> > > /* main segments - reserved segments - (node + data segments) */
> > > set_cp(free_segment_count, get_sb(segment_count_main) - 6);
> > > - set_cp(user_block_count, ((get_cp(free_segment_count) + 6 -
> > > + if (get_cp(free_segment_count) <= get_cp(overprov_segment_count)) {
> > > + MSG(0, "Error: Too small size\n");
> > > + goto free_sum_compact;
> > > + }
> > > +
> > > + set_cp(user_block_count, ((get_cp(free_segment_count) -
> > > get_cp(overprov_segment_count)) * config.blks_per_seg));
> > > /* cp page (2), data summaries (1), node summaries (3) */
> > > set_cp(cp_pack_total_block_count, 6 + get_sb(cp_payload));
> > > --
> > > 2.6.3
> >
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
