subject:"\[PATCH v2\] f2fs\: add bio cache for IPU"

Re: [f2fs-dev] [PATCH v2] f2fs: add bio cache for IPU

2018-12-18 Thread Jaegeuk Kim

On 12/14, Jaegeuk Kim wrote:
> On 12/14, Chao Yu wrote:
> > Hi Jaegeuk,
> > 
> > I guess ("f2fs: clear PG_writeback if IPU failed") and v2 patch can fix
> > hang issue caused by v1 one, could you please have a try with v2?
> 
> Sure.

Hang again. I'll remove this patch and run the test again without this to
confirm.

> 
> > 
> > Thanks,
> > 
> > On 2018/11/19 11:29, Chao Yu wrote:
> > > SQLite in Wal mode may trigger sequential IPU write in db-wal file, after
> > > commit d1b3e72d5490 ("f2fs: submit bio of in-place-update pages"), we
> > > lost the chance of merging page in inner managed bio cache, result in
> > > submitting more small-sized IO.
> > > 
> > > So let's add temporary bio in writepages() to cache mergeable write IO as
> > > much as possible.
> > > 
> > > Test case:
> > > 1. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"
> > > 2. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"
> > > 
> > > Before:
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65544, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65552, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65560, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65568, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65576, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65584, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65592, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65600, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65608, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65616, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65624, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65632, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65640, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65648, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65656, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65664, size = 4096
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector 
> > > = 57352, size = 4096
> > > 
> > > After:
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector 
> > > = 65544, size = 65536
> > > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector 
> > > = 57368, size = 4096
> > > 
> > > Signed-off-by: Chao Yu 
> > > ---
> > > v2:
> > > - submit cached bio for cp error case.
> > >  fs/f2fs/data.c| 61 +--
> > >  fs/f2fs/f2fs.h|  3 +++
> > >  fs/f2fs/segment.c |  5 +++-
> > >  3 files changed, 66 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> > > index 8780f3d737c4..7dffafb8b2c5 100644
> > > --- a/fs/f2fs/data.c
> > > +++ b/fs/f2fs/data.c
> > > @@ -474,6 +474,49 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)
> > >   return 0;
> > >  }
> > >  
> > > +int f2fs_merge_page_bio(struct f2fs_io_info *fio)
> > > +{
> > > + struct bio *bio = *fio->bio;
> > > + struct page *page = fio->encrypted_page ?
> > > + fio->encrypted_page : fio->page;
> > > +
> > > + if (!f2fs_is_valid_blkaddr(fio->sbi, fio->new_blkaddr,
> > > + __is_meta_io(fio) ? META_GENERIC : DATA_GENERIC))
> > > + return -EFAULT;
> > > +
> > > + trace_f2fs_submit_page_bio(page, fio);
> > > + f2fs_trace_ios(fio, 0);
> > > +
> > > + if (bio && (*fio->last_block + 1 != fio->new_blkaddr ||
> > > + !__same_bdev(fio->sbi, fio->new_blkaddr, bio))) {
> > > + __submit_bio(fio->sbi, bio, fio->type);
> > > + bio = NULL;
> > > + }
> > > +alloc_new:
> > > + if (!bio) {
> > > + bio = __bio_alloc(fio->sbi, fio->new_blkaddr, fio->io_wbc,
> > > + BIO_MAX_PAGES, false, fio->type, fio->temp);
> > > + *fio->last_block = fio->new_blkaddr;
> > > + bio_set_op_attrs(bio, fio->op, fio->op_flags);
> > > + }
> > > +
> > > + if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) {
> > > + __submit_bio(fio->sbi, bio, fio->type);
> > > + bio = NULL;
> > > + goto alloc_new;
> > > + }
> > > +
> > > + if (fio->io_wbc)
> > > + wbc_account_io(fio->io_wbc, page, PAGE_SIZE);
> > > +
> > > + *fio->last_block = fio->new_blkaddr;
> > > +
> > > +

Re: [PATCH v2] f2fs: add bio cache for IPU

2018-12-14 Thread Jaegeuk Kim

On 12/14, Chao Yu wrote:
> Hi Jaegeuk,
> 
> I guess ("f2fs: clear PG_writeback if IPU failed") and v2 patch can fix
> hang issue caused by v1 one, could you please have a try with v2?

Sure.

> 
> Thanks,
> 
> On 2018/11/19 11:29, Chao Yu wrote:
> > SQLite in Wal mode may trigger sequential IPU write in db-wal file, after
> > commit d1b3e72d5490 ("f2fs: submit bio of in-place-update pages"), we
> > lost the chance of merging page in inner managed bio cache, result in
> > submitting more small-sized IO.
> > 
> > So let's add temporary bio in writepages() to cache mergeable write IO as
> > much as possible.
> > 
> > Test case:
> > 1. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"
> > 2. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"
> > 
> > Before:
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65544, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65552, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65560, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65568, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65576, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65584, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65592, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65600, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65608, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65616, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65624, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65632, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65640, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65648, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65656, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65664, size = 4096
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector = 
> > 57352, size = 4096
> > 
> > After:
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> > 65544, size = 65536
> > f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector = 
> > 57368, size = 4096
> > 
> > Signed-off-by: Chao Yu 
> > ---
> > v2:
> > - submit cached bio for cp error case.
> >  fs/f2fs/data.c| 61 +--
> >  fs/f2fs/f2fs.h|  3 +++
> >  fs/f2fs/segment.c |  5 +++-
> >  3 files changed, 66 insertions(+), 3 deletions(-)
> > 
> > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> > index 8780f3d737c4..7dffafb8b2c5 100644
> > --- a/fs/f2fs/data.c
> > +++ b/fs/f2fs/data.c
> > @@ -474,6 +474,49 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)
> > return 0;
> >  }
> >  
> > +int f2fs_merge_page_bio(struct f2fs_io_info *fio)
> > +{
> > +   struct bio *bio = *fio->bio;
> > +   struct page *page = fio->encrypted_page ?
> > +   fio->encrypted_page : fio->page;
> > +
> > +   if (!f2fs_is_valid_blkaddr(fio->sbi, fio->new_blkaddr,
> > +   __is_meta_io(fio) ? META_GENERIC : DATA_GENERIC))
> > +   return -EFAULT;
> > +
> > +   trace_f2fs_submit_page_bio(page, fio);
> > +   f2fs_trace_ios(fio, 0);
> > +
> > +   if (bio && (*fio->last_block + 1 != fio->new_blkaddr ||
> > +   !__same_bdev(fio->sbi, fio->new_blkaddr, bio))) {
> > +   __submit_bio(fio->sbi, bio, fio->type);
> > +   bio = NULL;
> > +   }
> > +alloc_new:
> > +   if (!bio) {
> > +   bio = __bio_alloc(fio->sbi, fio->new_blkaddr, fio->io_wbc,
> > +   BIO_MAX_PAGES, false, fio->type, fio->temp);
> > +   *fio->last_block = fio->new_blkaddr;
> > +   bio_set_op_attrs(bio, fio->op, fio->op_flags);
> > +   }
> > +
> > +   if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) {
> > +   __submit_bio(fio->sbi, bio, fio->type);
> > +   bio = NULL;
> > +   goto alloc_new;
> > +   }
> > +
> > +   if (fio->io_wbc)
> > +   wbc_account_io(fio->io_wbc, page, PAGE_SIZE);
> > +
> > +   *fio->last_block = fio->new_blkaddr;
> > +
> > +   inc_page_count(fio->sbi, WB_DATA_TYPE(fio->page));
> > +
> > +   *fio->bio = bio;
> > +   return 0;
> > +}
> > +
> >  void f2fs_submit_page_write(struct f2fs_io_info *fio)
> >  {
> > struct f2fs_sb_info *sbi = fio->sbi;
> > @@ -1894,6 +1937,8 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
> >  }
>

Re: [PATCH v2] f2fs: add bio cache for IPU

2018-12-13 Thread Chao Yu

Hi Jaegeuk,

I guess ("f2fs: clear PG_writeback if IPU failed") and v2 patch can fix
hang issue caused by v1 one, could you please have a try with v2?

Thanks,

On 2018/11/19 11:29, Chao Yu wrote:
> SQLite in Wal mode may trigger sequential IPU write in db-wal file, after
> commit d1b3e72d5490 ("f2fs: submit bio of in-place-update pages"), we
> lost the chance of merging page in inner managed bio cache, result in
> submitting more small-sized IO.
> 
> So let's add temporary bio in writepages() to cache mergeable write IO as
> much as possible.
> 
> Test case:
> 1. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"
> 2. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"
> 
> Before:
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65544, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65552, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65560, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65568, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65576, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65584, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65592, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65600, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65608, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65616, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65624, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65632, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65640, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65648, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65656, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65664, size = 4096
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector = 
> 57352, size = 4096
> 
> After:
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
> 65544, size = 65536
> f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector = 
> 57368, size = 4096
> 
> Signed-off-by: Chao Yu 
> ---
> v2:
> - submit cached bio for cp error case.
>  fs/f2fs/data.c| 61 +--
>  fs/f2fs/f2fs.h|  3 +++
>  fs/f2fs/segment.c |  5 +++-
>  3 files changed, 66 insertions(+), 3 deletions(-)
> 
> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
> index 8780f3d737c4..7dffafb8b2c5 100644
> --- a/fs/f2fs/data.c
> +++ b/fs/f2fs/data.c
> @@ -474,6 +474,49 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)
>   return 0;
>  }
>  
> +int f2fs_merge_page_bio(struct f2fs_io_info *fio)
> +{
> + struct bio *bio = *fio->bio;
> + struct page *page = fio->encrypted_page ?
> + fio->encrypted_page : fio->page;
> +
> + if (!f2fs_is_valid_blkaddr(fio->sbi, fio->new_blkaddr,
> + __is_meta_io(fio) ? META_GENERIC : DATA_GENERIC))
> + return -EFAULT;
> +
> + trace_f2fs_submit_page_bio(page, fio);
> + f2fs_trace_ios(fio, 0);
> +
> + if (bio && (*fio->last_block + 1 != fio->new_blkaddr ||
> + !__same_bdev(fio->sbi, fio->new_blkaddr, bio))) {
> + __submit_bio(fio->sbi, bio, fio->type);
> + bio = NULL;
> + }
> +alloc_new:
> + if (!bio) {
> + bio = __bio_alloc(fio->sbi, fio->new_blkaddr, fio->io_wbc,
> + BIO_MAX_PAGES, false, fio->type, fio->temp);
> + *fio->last_block = fio->new_blkaddr;
> + bio_set_op_attrs(bio, fio->op, fio->op_flags);
> + }
> +
> + if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) {
> + __submit_bio(fio->sbi, bio, fio->type);
> + bio = NULL;
> + goto alloc_new;
> + }
> +
> + if (fio->io_wbc)
> + wbc_account_io(fio->io_wbc, page, PAGE_SIZE);
> +
> + *fio->last_block = fio->new_blkaddr;
> +
> + inc_page_count(fio->sbi, WB_DATA_TYPE(fio->page));
> +
> + *fio->bio = bio;
> + return 0;
> +}
> +
>  void f2fs_submit_page_write(struct f2fs_io_info *fio)
>  {
>   struct f2fs_sb_info *sbi = fio->sbi;
> @@ -1894,6 +1937,8 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
>  }
>  
>  static int __write_data_page(struct page *page, bool *submitted,
> + struct bio **bio,
> + sector_t *last_block,
>   struct

[PATCH v2] f2fs: add bio cache for IPU

2018-11-18 Thread Chao Yu

SQLite in Wal mode may trigger sequential IPU write in db-wal file, after
commit d1b3e72d5490 ("f2fs: submit bio of in-place-update pages"), we
lost the chance of merging page in inner managed bio cache, result in
submitting more small-sized IO.

So let's add temporary bio in writepages() to cache mergeable write IO as
much as possible.

Test case:
1. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"
2. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"

Before:
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65544, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65552, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65560, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65568, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65576, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65584, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65592, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65600, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65608, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65616, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65624, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65632, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65640, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65648, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65656, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65664, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector = 
57352, size = 4096

After:
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65544, size = 65536
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector = 
57368, size = 4096

Signed-off-by: Chao Yu 
---
v2:
- submit cached bio for cp error case.
 fs/f2fs/data.c| 61 +--
 fs/f2fs/f2fs.h|  3 +++
 fs/f2fs/segment.c |  5 +++-
 3 files changed, 66 insertions(+), 3 deletions(-)

diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 8780f3d737c4..7dffafb8b2c5 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -474,6 +474,49 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)
return 0;
 }
 
+int f2fs_merge_page_bio(struct f2fs_io_info *fio)
+{
+   struct bio *bio = *fio->bio;
+   struct page *page = fio->encrypted_page ?
+   fio->encrypted_page : fio->page;
+
+   if (!f2fs_is_valid_blkaddr(fio->sbi, fio->new_blkaddr,
+   __is_meta_io(fio) ? META_GENERIC : DATA_GENERIC))
+   return -EFAULT;
+
+   trace_f2fs_submit_page_bio(page, fio);
+   f2fs_trace_ios(fio, 0);
+
+   if (bio && (*fio->last_block + 1 != fio->new_blkaddr ||
+   !__same_bdev(fio->sbi, fio->new_blkaddr, bio))) {
+   __submit_bio(fio->sbi, bio, fio->type);
+   bio = NULL;
+   }
+alloc_new:
+   if (!bio) {
+   bio = __bio_alloc(fio->sbi, fio->new_blkaddr, fio->io_wbc,
+   BIO_MAX_PAGES, false, fio->type, fio->temp);
+   *fio->last_block = fio->new_blkaddr;
+   bio_set_op_attrs(bio, fio->op, fio->op_flags);
+   }
+
+   if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) {
+   __submit_bio(fio->sbi, bio, fio->type);
+   bio = NULL;
+   goto alloc_new;
+   }
+
+   if (fio->io_wbc)
+   wbc_account_io(fio->io_wbc, page, PAGE_SIZE);
+
+   *fio->last_block = fio->new_blkaddr;
+
+   inc_page_count(fio->sbi, WB_DATA_TYPE(fio->page));
+
+   *fio->bio = bio;
+   return 0;
+}
+
 void f2fs_submit_page_write(struct f2fs_io_info *fio)
 {
struct f2fs_sb_info *sbi = fio->sbi;
@@ -1894,6 +1937,8 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
 }
 
 static int __write_data_page(struct page *page, bool *submitted,
+   struct bio **bio,
+   sector_t *last_block,
struct writeback_control *wbc,
enum iostat_type io_type)
 {
@@ -1919,6 +1964,8 @@ static int __write_data_page(struct page *page, bool 
*submitted,
.need_lock = LOCK_RETRY,
.io_type = io_type,
.io_wbc = wbc,
+   .bio = bio,
+   .last_block = last_block,
};

[PATCH v2] f2fs: add bio cache for IPU

2018-11-18 Thread Chao Yu

SQLite in Wal mode may trigger sequential IPU write in db-wal file, after
commit d1b3e72d5490 ("f2fs: submit bio of in-place-update pages"), we
lost the chance of merging page in inner managed bio cache, result in
submitting more small-sized IO.

So let's add temporary bio in writepages() to cache mergeable write IO as
much as possible.

Test case:
1. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"
2. xfs_io -f /mnt/f2fs/file -c "pwrite 0 65536" -c "fsync"

Before:
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65544, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65552, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65560, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65568, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65576, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65584, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65592, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65600, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65608, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65616, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65624, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65632, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65640, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65648, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65656, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65664, size = 4096
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector = 
57352, size = 4096

After:
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), DATA, sector = 
65544, size = 65536
f2fs_submit_write_bio: dev = (251,0)/(251,0), rw = WRITE(S), NODE, sector = 
57368, size = 4096

Signed-off-by: Chao Yu 
---
v2:
- submit cached bio for cp error case.
 fs/f2fs/data.c| 61 +--
 fs/f2fs/f2fs.h|  3 +++
 fs/f2fs/segment.c |  5 +++-
 3 files changed, 66 insertions(+), 3 deletions(-)

diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c
index 8780f3d737c4..7dffafb8b2c5 100644
--- a/fs/f2fs/data.c
+++ b/fs/f2fs/data.c
@@ -474,6 +474,49 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio)
return 0;
 }
 
+int f2fs_merge_page_bio(struct f2fs_io_info *fio)
+{
+   struct bio *bio = *fio->bio;
+   struct page *page = fio->encrypted_page ?
+   fio->encrypted_page : fio->page;
+
+   if (!f2fs_is_valid_blkaddr(fio->sbi, fio->new_blkaddr,
+   __is_meta_io(fio) ? META_GENERIC : DATA_GENERIC))
+   return -EFAULT;
+
+   trace_f2fs_submit_page_bio(page, fio);
+   f2fs_trace_ios(fio, 0);
+
+   if (bio && (*fio->last_block + 1 != fio->new_blkaddr ||
+   !__same_bdev(fio->sbi, fio->new_blkaddr, bio))) {
+   __submit_bio(fio->sbi, bio, fio->type);
+   bio = NULL;
+   }
+alloc_new:
+   if (!bio) {
+   bio = __bio_alloc(fio->sbi, fio->new_blkaddr, fio->io_wbc,
+   BIO_MAX_PAGES, false, fio->type, fio->temp);
+   *fio->last_block = fio->new_blkaddr;
+   bio_set_op_attrs(bio, fio->op, fio->op_flags);
+   }
+
+   if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) {
+   __submit_bio(fio->sbi, bio, fio->type);
+   bio = NULL;
+   goto alloc_new;
+   }
+
+   if (fio->io_wbc)
+   wbc_account_io(fio->io_wbc, page, PAGE_SIZE);
+
+   *fio->last_block = fio->new_blkaddr;
+
+   inc_page_count(fio->sbi, WB_DATA_TYPE(fio->page));
+
+   *fio->bio = bio;
+   return 0;
+}
+
 void f2fs_submit_page_write(struct f2fs_io_info *fio)
 {
struct f2fs_sb_info *sbi = fio->sbi;
@@ -1894,6 +1937,8 @@ int f2fs_do_write_data_page(struct f2fs_io_info *fio)
 }
 
 static int __write_data_page(struct page *page, bool *submitted,
+   struct bio **bio,
+   sector_t *last_block,
struct writeback_control *wbc,
enum iostat_type io_type)
 {
@@ -1919,6 +1964,8 @@ static int __write_data_page(struct page *page, bool 
*submitted,
.need_lock = LOCK_RETRY,
.io_type = io_type,
.io_wbc = wbc,
+   .bio = bio,
+   .last_block = last_block,
};

Re: [f2fs-dev] [PATCH v2] f2fs: add bio cache for IPU

Re: [PATCH v2] f2fs: add bio cache for IPU

Re: [PATCH v2] f2fs: add bio cache for IPU

[PATCH v2] f2fs: add bio cache for IPU

[PATCH v2] f2fs: add bio cache for IPU

5 matches

Site Navigation

Mail list logo

Footer information