On Fri, Jun 5, 2020 at 12:57 PM Eric Biggers <[email protected]> wrote: > > On Fri, Jun 05, 2020 at 12:42:59PM -0700, Nick Desaulniers wrote: > > On Fri, Jun 5, 2020 at 12:34 PM Eric Biggers <[email protected]> wrote: > > > > > > On Fri, Jun 05, 2020 at 09:45:43AM -0700, Nick Desaulniers wrote: > > > > On Fri, Jun 5, 2020 at 9:08 AM Eric Biggers <[email protected]> wrote: > > > > > > > > > > On Fri, Jun 05, 2020 at 07:55:46AM -0700, Jaegeuk Kim wrote: > > > > > > Eric, > > > > > > > > > > > > Could you PTAL? > > > > > > > > > > > > On 06/05, kernel test robot wrote: > > > > > > > tree: > > > > > > > https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git > > > > > > > dev-test > > > > > > > head: adf3d3a53cf13d0998c699ba43d8582c875179e3 > > > > > > > commit: adf3d3a53cf13d0998c699ba43d8582c875179e3 [48/48] f2fs: > > > > > > > don't return vmalloc() memory from f2fs_kmalloc() > > > > > > > config: powerpc64-randconfig-r011-20200605 (attached as .config) > > > > > > > compiler: clang version 11.0.0 > > > > > > > (https://github.com/llvm/llvm-project > > > > > > > ac47588bc4ff5927a01ed6fcd269ce86aba52a7c) > > > > > > > reproduce (this is a W=1 build): > > > > > > > wget > > > > > > > https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross > > > > > > > -O ~/bin/make.cross > > > > > > > chmod +x ~/bin/make.cross > > > > > > > # install powerpc64 cross compiling tool for clang build > > > > > > > # apt-get install binutils-powerpc64-linux-gnu > > > > > > > git checkout adf3d3a53cf13d0998c699ba43d8582c875179e3 > > > > > > > # save the attached .config to linux build tree > > > > > > > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang > > > > > > > make.cross ARCH=powerpc64 > > > > > > > > > > > > > > If you fix the issue, kindly add following tag as appropriate > > > > > > > Reported-by: kernel test robot <[email protected]> > > > > > > > > > > I don't know what's causing this. It doesn't look at all related to > > > > > my commit, > > > > > and I don't see what's using so much stack in f2fs_fill_super(). > > > > > > > > > > @kernel test robot: the directions given above don't actually work. > > > > > First I got an error due to powerpc64-linux-gnu-elfedit not existing. > > > > > So I had to build binutils for powerpc64 myself. > > > > > > > > > > Then I still got an error: > > > > > > > > > > make: *** No rule to make target 'arch/powerpc64/Makefile'. > > > > > Stop. > > > > > > > > If you have the config, then > > > > $ ARCH=powerpc CROSS_COMPILE=powerpc64-linux-gnu- make CC=clang -j71 > > > > If you recompile with CONFIG_DEBUG_INFO, you can get the stack frame > > > > information. I wrote a tool to parse this for these cryptic warnings. > > > > https://github.com/ClangBuiltLinux/frame-larger-than > > > > > > I can build the file and get the warning now, but the frame_larger_than.py > > > script doesn't work: > > > > > > $ ARCH=powerpc CROSS_COMPILE=powerpc64-linux-gnu- make CC=clang > > > fs/f2fs/super.o > > > CALL scripts/checksyscalls.sh > > > CALL scripts/atomic/check-atomics.sh > > > CC [M] fs/f2fs/super.o > > > fs/f2fs/super.c:3303:12: warning: stack frame size of 2064 bytes in > > > function 'f2fs_fill_super' [-Wframe-larger-than=] > > > static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > > ^ > > > 1 warning generated. > > > > > > $ python3 ~/src/frame-larger-than/frame_larger_than.py fs/f2fs/super.o > > > f2fs_fill_super > > > failed to parse elf: Unsupported relocation type: 1 > > > > Looks like the python library I'm using to parse the DWARF debug info > > doesn't understand some ppc64 specific relocation kind, and is > > throwing an ELFError. Let me report it upstream. > > > > The hard part for these is inlining; it can be hard to tell which > > child has a large local allocation when looking at the warning about > > the parent. > > > > Then again, my script is just nicer output than `llvm-dwarfdump foo.o` > > or `readelf --debug-dump=info foo.o` for this specific issue. > > > > I did confirm that my commit "f2fs: don't return vmalloc() memory from > f2fs_kmalloc()" actually caused this, in particular the following part: > > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c > index a71da699cb2d55..f3c15116954218 100644 > --- a/fs/f2fs/super.c > +++ b/fs/f2fs/super.c > @@ -3033,7 +3033,7 @@ static int init_blkz_info(struct f2fs_sb_info *sbi, int > devi) > if (nr_sectors & (bdev_zone_sectors(bdev) - 1)) > FDEV(devi).nr_blkz++; > > - FDEV(devi).blkz_seq = f2fs_kzalloc(sbi, > + FDEV(devi).blkz_seq = f2fs_kvzalloc(sbi, > BITS_TO_LONGS(FDEV(devi).nr_blkz) > * sizeof(unsigned long), > GFP_KERNEL); > > This is inlined as: > > f2fs_fill_super() > => f2fs_scan_devices() > => init_blkz_info() > => f2fs_kvzalloc() > > Somehow this change increased the frame size of f2fs_fill_super() from 1984 to > 2064 bytes. That exceeds the 2048-byte limit, triggering the warning. > > So, f2fs_fill_super() was very close to the limit already. I'm not sure why; > I > don't see any huge stack allocations in it or in static functions it calls. > Maybe there is a kconfig option that is increasing the stack usage? > > If I add noinline_for_stack to f2fs_scan_devices(), the frame drops to 1744 > bytes. Or if I add noinline_for_stack to read_raw_super_block(), the frame > drops to 1776 bytes. We could do either (or both) of those to avoid the > warning. But we'd still wouldn't be that far from this 2048-byte "limit".
Right, so my script would have printed out the list of all local variables in f2fs_fill_super() and their sizes. With that information handy, we could assess if there were any smoking guns of clearly incorrect large stack allocations vs death by a thousand cuts. Your change may not have added a new large local allocation, simply tipped the scale or changed inlining decisions. They may be other local variables in this call chain that we should reassess allocation strategy; ie. dynamic or static rather than local, to avoid the potential for exhausting kernel stack. -- Thanks, ~Nick Desaulniers _______________________________________________ Linux-f2fs-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
