Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 09/19, Chao Yu wrote: > On 2019/9/19 0:47, Jaegeuk Kim wrote: > > On 09/18, Chao Yu wrote: > >> On 2019/9/18 11:12, Jaegeuk Kim wrote: > >>> On 09/18, Chao Yu wrote: > On 2019/9/18 4:55, Jaegeuk Kim wrote: > > On 09/17, Chao Yu wrote: > >> On 2019/9/16 23:37, Jaegeuk Kim wrote: > >>> On 09/16, Chao Yu wrote: > On 2019/9/9 20:04, Jaegeuk Kim wrote: > > On 09/09, Chao Yu wrote: > >> On 2019/9/9 16:06, Jaegeuk Kim wrote: > >>> On 09/09, Chao Yu wrote: > On 2019/9/9 9:25, Jaegeuk Kim wrote: > > GC must avoid select the same victim again. > > Blocks in previous victim will occupy addition free segment, I > doubt after this > change, FGGC may encounter out-of-free space issue more > frequently. > >>> > >>> Hmm, actually this change seems wrong by sec_usage_check(). > >>> We may be able to avoid this only in the suspicious loop? > >>> > >>> --- > >>> fs/f2fs/gc.c | 2 +- > >>> 1 file changed, 1 insertion(+), 1 deletion(-) > >>> > >>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c > >>> index e88f98ddf396..5877bd729689 100644 > >>> --- a/fs/f2fs/gc.c > >>> +++ b/fs/f2fs/gc.c > >>> @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool > >>> sync, > >>> round++; > >>> } > >>> > >>> - if (gc_type == FG_GC) > >>> + if (gc_type == FG_GC && seg_freed) > >> > >> That's original solution Sahitya provided to avoid infinite loop > >> of GC, but I > >> suggest to find the root cause first, then we added > >> .invalid_segmap for that > >> purpose. > > > > I've checked the Sahitya's patch. So, it seems the problem can > > happen due to > > is_alive or atomic_file. > > For some conditions, this doesn't help, for example, two sections > contain the > same fewest valid blocks, it will cause to loop selecting them if it > fails to > migrate blocks. > > How about keeping it as it is to find potential bug. > >>> > >>> I think it'd be fine to merge this. Could you check the above > >>> scenario in more > >>> detail? > >> > >> I haven't saw this in real scenario yet. > >> > >> What I mean is if there is a bug (maybe in is_alive()) failing us to > >> GC on one > >> section, when that bug happens in two candidates, there could be the > >> same > >> condition that GC will run into loop (select A, fail to migrate; > >> select B, fail > >> to migrate, select A...). > >> > >> But I guess the benefit of this change is, if FGGC fails to migrate > >> block due to > >> i_gc_rwsem race, selecting another section and later retrying previous > >> one may > >> avoid lock race, right? > > > > In any case, I think this can avoid potenial GC loop. At least to me, > > it'd be > > quite risky, if we remain this just for debugging purpose only. > > Yup, > > One more concern is would this cur_victim_sec remain after FGGC? then > BGGC/SSR > will always skip the section cur_victim_sec points to. > >>> > >>> Then, we can get another loop before using it by BGGC/SSR. > >> > >> I guess I didn't catch your point, do you mean, if we reset it in the end > >> of > >> FGGC, we may encounter the loop during BGGC/SSR? > > > > FGGC failed in a loop and last victim was remained in cur_victim_sec. > > It won't run into a loop because we keep below condition? The following FGGC will be likely to select this victim again, which doesn't mean "this loop" but "loop of f2fs_gc". > > + if (gc_type == FG_GC && seg_freed) > + sbi->cur_victim_sec = NULL_SEGNO; > > if (sync) > goto stop; > > I meant add below logic in addition: > > + if (gc_type == FG_GC) > + sbi->cur_victim_sec = NULL_SEGNO; > > mutex_unlock(>gc_mutex); > > Thanks, > > > Next FGGC kicked in and did the same thing again. I don't expect BGGC/SSR > > wants to select this victim much, since it will have CB policy. > > > >> > >> I meant: > >> > >> f2fs_gc() > >> ... > >> > >> + if (gc_type == FG_GC) > >> + sbi->cur_victim_sec = NULL_SEGNO; > >> > >>mutex_unlock(>gc_mutex); > >> > >>put_gc_inode(_list); > >> ... > >> > >> Thanks, > >> > >>> > > So could we reset cur_victim_sec in the end of FGGC? > > Thanks, > > > > >> > >> Thanks, > >> > >>> > >>> Thanks, > >>> > > Thanks, > > > > >> > >> Thanks, > >> > >>> sbi->cur_victim_sec = NULL_SEGNO; > >>> > >>> if (sync) > >>> >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 2019/9/19 0:47, Jaegeuk Kim wrote: > On 09/18, Chao Yu wrote: >> On 2019/9/18 11:12, Jaegeuk Kim wrote: >>> On 09/18, Chao Yu wrote: On 2019/9/18 4:55, Jaegeuk Kim wrote: > On 09/17, Chao Yu wrote: >> On 2019/9/16 23:37, Jaegeuk Kim wrote: >>> On 09/16, Chao Yu wrote: On 2019/9/9 20:04, Jaegeuk Kim wrote: > On 09/09, Chao Yu wrote: >> On 2019/9/9 16:06, Jaegeuk Kim wrote: >>> On 09/09, Chao Yu wrote: On 2019/9/9 9:25, Jaegeuk Kim wrote: > GC must avoid select the same victim again. Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently. >>> >>> Hmm, actually this change seems wrong by sec_usage_check(). >>> We may be able to avoid this only in the suspicious loop? >>> >>> --- >>> fs/f2fs/gc.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c >>> index e88f98ddf396..5877bd729689 100644 >>> --- a/fs/f2fs/gc.c >>> +++ b/fs/f2fs/gc.c >>> @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool >>> sync, >>> round++; >>> } >>> >>> - if (gc_type == FG_GC) >>> + if (gc_type == FG_GC && seg_freed) >> >> That's original solution Sahitya provided to avoid infinite loop of >> GC, but I >> suggest to find the root cause first, then we added .invalid_segmap >> for that >> purpose. > > I've checked the Sahitya's patch. So, it seems the problem can happen > due to > is_alive or atomic_file. For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks. How about keeping it as it is to find potential bug. >>> >>> I think it'd be fine to merge this. Could you check the above scenario >>> in more >>> detail? >> >> I haven't saw this in real scenario yet. >> >> What I mean is if there is a bug (maybe in is_alive()) failing us to GC >> on one >> section, when that bug happens in two candidates, there could be the same >> condition that GC will run into loop (select A, fail to migrate; select >> B, fail >> to migrate, select A...). >> >> But I guess the benefit of this change is, if FGGC fails to migrate >> block due to >> i_gc_rwsem race, selecting another section and later retrying previous >> one may >> avoid lock race, right? > > In any case, I think this can avoid potenial GC loop. At least to me, > it'd be > quite risky, if we remain this just for debugging purpose only. Yup, One more concern is would this cur_victim_sec remain after FGGC? then BGGC/SSR will always skip the section cur_victim_sec points to. >>> >>> Then, we can get another loop before using it by BGGC/SSR. >> >> I guess I didn't catch your point, do you mean, if we reset it in the end of >> FGGC, we may encounter the loop during BGGC/SSR? > > FGGC failed in a loop and last victim was remained in cur_victim_sec. It won't run into a loop because we keep below condition? + if (gc_type == FG_GC && seg_freed) + sbi->cur_victim_sec = NULL_SEGNO; if (sync) goto stop; I meant add below logic in addition: + if (gc_type == FG_GC) + sbi->cur_victim_sec = NULL_SEGNO; mutex_unlock(>gc_mutex); Thanks, > Next FGGC kicked in and did the same thing again. I don't expect BGGC/SSR > wants to select this victim much, since it will have CB policy. > >> >> I meant: >> >> f2fs_gc() >> ... >> >> +if (gc_type == FG_GC) >> +sbi->cur_victim_sec = NULL_SEGNO; >> >> mutex_unlock(>gc_mutex); >> >> put_gc_inode(_list); >> ... >> >> Thanks, >> >>> So could we reset cur_victim_sec in the end of FGGC? Thanks, > >> >> Thanks, >> >>> >>> Thanks, >>> Thanks, > >> >> Thanks, >> >>> sbi->cur_victim_sec = NULL_SEGNO; >>> >>> if (sync) >>> > . > >>> . >>> > . > >>> . >>> > . >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 09/18, Chao Yu wrote: > On 2019/9/18 11:12, Jaegeuk Kim wrote: > > On 09/18, Chao Yu wrote: > >> On 2019/9/18 4:55, Jaegeuk Kim wrote: > >>> On 09/17, Chao Yu wrote: > On 2019/9/16 23:37, Jaegeuk Kim wrote: > > On 09/16, Chao Yu wrote: > >> On 2019/9/9 20:04, Jaegeuk Kim wrote: > >>> On 09/09, Chao Yu wrote: > On 2019/9/9 16:06, Jaegeuk Kim wrote: > > On 09/09, Chao Yu wrote: > >> On 2019/9/9 9:25, Jaegeuk Kim wrote: > >>> GC must avoid select the same victim again. > >> > >> Blocks in previous victim will occupy addition free segment, I > >> doubt after this > >> change, FGGC may encounter out-of-free space issue more frequently. > > > > Hmm, actually this change seems wrong by sec_usage_check(). > > We may be able to avoid this only in the suspicious loop? > > > > --- > > fs/f2fs/gc.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c > > index e88f98ddf396..5877bd729689 100644 > > --- a/fs/f2fs/gc.c > > +++ b/fs/f2fs/gc.c > > @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool > > sync, > > round++; > > } > > > > - if (gc_type == FG_GC) > > + if (gc_type == FG_GC && seg_freed) > > That's original solution Sahitya provided to avoid infinite loop of > GC, but I > suggest to find the root cause first, then we added .invalid_segmap > for that > purpose. > >>> > >>> I've checked the Sahitya's patch. So, it seems the problem can happen > >>> due to > >>> is_alive or atomic_file. > >> > >> For some conditions, this doesn't help, for example, two sections > >> contain the > >> same fewest valid blocks, it will cause to loop selecting them if it > >> fails to > >> migrate blocks. > >> > >> How about keeping it as it is to find potential bug. > > > > I think it'd be fine to merge this. Could you check the above scenario > > in more > > detail? > > I haven't saw this in real scenario yet. > > What I mean is if there is a bug (maybe in is_alive()) failing us to GC > on one > section, when that bug happens in two candidates, there could be the same > condition that GC will run into loop (select A, fail to migrate; select > B, fail > to migrate, select A...). > > But I guess the benefit of this change is, if FGGC fails to migrate > block due to > i_gc_rwsem race, selecting another section and later retrying previous > one may > avoid lock race, right? > >>> > >>> In any case, I think this can avoid potenial GC loop. At least to me, > >>> it'd be > >>> quite risky, if we remain this just for debugging purpose only. > >> > >> Yup, > >> > >> One more concern is would this cur_victim_sec remain after FGGC? then > >> BGGC/SSR > >> will always skip the section cur_victim_sec points to. > > > > Then, we can get another loop before using it by BGGC/SSR. > > I guess I didn't catch your point, do you mean, if we reset it in the end of > FGGC, we may encounter the loop during BGGC/SSR? FGGC failed in a loop and last victim was remained in cur_victim_sec. Next FGGC kicked in and did the same thing again. I don't expect BGGC/SSR wants to select this victim much, since it will have CB policy. > > I meant: > > f2fs_gc() > ... > > + if (gc_type == FG_GC) > + sbi->cur_victim_sec = NULL_SEGNO; > > mutex_unlock(>gc_mutex); > > put_gc_inode(_list); > ... > > Thanks, > > > > >> > >> So could we reset cur_victim_sec in the end of FGGC? > >> > >> Thanks, > >> > >>> > > Thanks, > > > > > Thanks, > > > >> > >> Thanks, > >> > >>> > > Thanks, > > > sbi->cur_victim_sec = NULL_SEGNO; > > > > if (sync) > > > >>> . > >>> > > . > > > >>> . > >>> > > . > > ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 2019/9/18 11:12, Jaegeuk Kim wrote: > On 09/18, Chao Yu wrote: >> On 2019/9/18 4:55, Jaegeuk Kim wrote: >>> On 09/17, Chao Yu wrote: On 2019/9/16 23:37, Jaegeuk Kim wrote: > On 09/16, Chao Yu wrote: >> On 2019/9/9 20:04, Jaegeuk Kim wrote: >>> On 09/09, Chao Yu wrote: On 2019/9/9 16:06, Jaegeuk Kim wrote: > On 09/09, Chao Yu wrote: >> On 2019/9/9 9:25, Jaegeuk Kim wrote: >>> GC must avoid select the same victim again. >> >> Blocks in previous victim will occupy addition free segment, I doubt >> after this >> change, FGGC may encounter out-of-free space issue more frequently. > > Hmm, actually this change seems wrong by sec_usage_check(). > We may be able to avoid this only in the suspicious loop? > > --- > fs/f2fs/gc.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c > index e88f98ddf396..5877bd729689 100644 > --- a/fs/f2fs/gc.c > +++ b/fs/f2fs/gc.c > @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, > round++; > } > > - if (gc_type == FG_GC) > + if (gc_type == FG_GC && seg_freed) That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose. >>> >>> I've checked the Sahitya's patch. So, it seems the problem can happen >>> due to >>> is_alive or atomic_file. >> >> For some conditions, this doesn't help, for example, two sections >> contain the >> same fewest valid blocks, it will cause to loop selecting them if it >> fails to >> migrate blocks. >> >> How about keeping it as it is to find potential bug. > > I think it'd be fine to merge this. Could you check the above scenario in > more > detail? I haven't saw this in real scenario yet. What I mean is if there is a bug (maybe in is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...). But I guess the benefit of this change is, if FGGC fails to migrate block due to i_gc_rwsem race, selecting another section and later retrying previous one may avoid lock race, right? >>> >>> In any case, I think this can avoid potenial GC loop. At least to me, it'd >>> be >>> quite risky, if we remain this just for debugging purpose only. >> >> Yup, >> >> One more concern is would this cur_victim_sec remain after FGGC? then >> BGGC/SSR >> will always skip the section cur_victim_sec points to. > > Then, we can get another loop before using it by BGGC/SSR. I guess I didn't catch your point, do you mean, if we reset it in the end of FGGC, we may encounter the loop during BGGC/SSR? I meant: f2fs_gc() ... + if (gc_type == FG_GC) + sbi->cur_victim_sec = NULL_SEGNO; mutex_unlock(>gc_mutex); put_gc_inode(_list); ... Thanks, > >> >> So could we reset cur_victim_sec in the end of FGGC? >> >> Thanks, >> >>> Thanks, > > Thanks, > >> >> Thanks, >> >>> Thanks, > sbi->cur_victim_sec = NULL_SEGNO; > > if (sync) > >>> . >>> > . > >>> . >>> > . >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 09/18, Chao Yu wrote: > On 2019/9/18 4:55, Jaegeuk Kim wrote: > > On 09/17, Chao Yu wrote: > >> On 2019/9/16 23:37, Jaegeuk Kim wrote: > >>> On 09/16, Chao Yu wrote: > On 2019/9/9 20:04, Jaegeuk Kim wrote: > > On 09/09, Chao Yu wrote: > >> On 2019/9/9 16:06, Jaegeuk Kim wrote: > >>> On 09/09, Chao Yu wrote: > On 2019/9/9 9:25, Jaegeuk Kim wrote: > > GC must avoid select the same victim again. > > Blocks in previous victim will occupy addition free segment, I doubt > after this > change, FGGC may encounter out-of-free space issue more frequently. > >>> > >>> Hmm, actually this change seems wrong by sec_usage_check(). > >>> We may be able to avoid this only in the suspicious loop? > >>> > >>> --- > >>> fs/f2fs/gc.c | 2 +- > >>> 1 file changed, 1 insertion(+), 1 deletion(-) > >>> > >>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c > >>> index e88f98ddf396..5877bd729689 100644 > >>> --- a/fs/f2fs/gc.c > >>> +++ b/fs/f2fs/gc.c > >>> @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, > >>> round++; > >>> } > >>> > >>> - if (gc_type == FG_GC) > >>> + if (gc_type == FG_GC && seg_freed) > >> > >> That's original solution Sahitya provided to avoid infinite loop of > >> GC, but I > >> suggest to find the root cause first, then we added .invalid_segmap > >> for that > >> purpose. > > > > I've checked the Sahitya's patch. So, it seems the problem can happen > > due to > > is_alive or atomic_file. > > For some conditions, this doesn't help, for example, two sections > contain the > same fewest valid blocks, it will cause to loop selecting them if it > fails to > migrate blocks. > > How about keeping it as it is to find potential bug. > >>> > >>> I think it'd be fine to merge this. Could you check the above scenario in > >>> more > >>> detail? > >> > >> I haven't saw this in real scenario yet. > >> > >> What I mean is if there is a bug (maybe in is_alive()) failing us to GC on > >> one > >> section, when that bug happens in two candidates, there could be the same > >> condition that GC will run into loop (select A, fail to migrate; select B, > >> fail > >> to migrate, select A...). > >> > >> But I guess the benefit of this change is, if FGGC fails to migrate block > >> due to > >> i_gc_rwsem race, selecting another section and later retrying previous one > >> may > >> avoid lock race, right? > > > > In any case, I think this can avoid potenial GC loop. At least to me, it'd > > be > > quite risky, if we remain this just for debugging purpose only. > > Yup, > > One more concern is would this cur_victim_sec remain after FGGC? then BGGC/SSR > will always skip the section cur_victim_sec points to. Then, we can get another loop before using it by BGGC/SSR. > > So could we reset cur_victim_sec in the end of FGGC? > > Thanks, > > > > >> > >> Thanks, > >> > >>> > >>> Thanks, > >>> > > Thanks, > > > > >> > >> Thanks, > >> > >>> sbi->cur_victim_sec = NULL_SEGNO; > >>> > >>> if (sync) > >>> > > . > > > >>> . > >>> > > . > >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 2019/9/18 4:55, Jaegeuk Kim wrote: > On 09/17, Chao Yu wrote: >> On 2019/9/16 23:37, Jaegeuk Kim wrote: >>> On 09/16, Chao Yu wrote: On 2019/9/9 20:04, Jaegeuk Kim wrote: > On 09/09, Chao Yu wrote: >> On 2019/9/9 16:06, Jaegeuk Kim wrote: >>> On 09/09, Chao Yu wrote: On 2019/9/9 9:25, Jaegeuk Kim wrote: > GC must avoid select the same victim again. Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently. >>> >>> Hmm, actually this change seems wrong by sec_usage_check(). >>> We may be able to avoid this only in the suspicious loop? >>> >>> --- >>> fs/f2fs/gc.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c >>> index e88f98ddf396..5877bd729689 100644 >>> --- a/fs/f2fs/gc.c >>> +++ b/fs/f2fs/gc.c >>> @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, >>> round++; >>> } >>> >>> - if (gc_type == FG_GC) >>> + if (gc_type == FG_GC && seg_freed) >> >> That's original solution Sahitya provided to avoid infinite loop of GC, >> but I >> suggest to find the root cause first, then we added .invalid_segmap for >> that >> purpose. > > I've checked the Sahitya's patch. So, it seems the problem can happen due > to > is_alive or atomic_file. For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks. How about keeping it as it is to find potential bug. >>> >>> I think it'd be fine to merge this. Could you check the above scenario in >>> more >>> detail? >> >> I haven't saw this in real scenario yet. >> >> What I mean is if there is a bug (maybe in is_alive()) failing us to GC on >> one >> section, when that bug happens in two candidates, there could be the same >> condition that GC will run into loop (select A, fail to migrate; select B, >> fail >> to migrate, select A...). >> >> But I guess the benefit of this change is, if FGGC fails to migrate block >> due to >> i_gc_rwsem race, selecting another section and later retrying previous one >> may >> avoid lock race, right? > > In any case, I think this can avoid potenial GC loop. At least to me, it'd be > quite risky, if we remain this just for debugging purpose only. Yup, One more concern is would this cur_victim_sec remain after FGGC? then BGGC/SSR will always skip the section cur_victim_sec points to. So could we reset cur_victim_sec in the end of FGGC? Thanks, > >> >> Thanks, >> >>> >>> Thanks, >>> Thanks, > >> >> Thanks, >> >>> sbi->cur_victim_sec = NULL_SEGNO; >>> >>> if (sync) >>> > . > >>> . >>> > . >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 09/17, Chao Yu wrote: > On 2019/9/16 23:37, Jaegeuk Kim wrote: > > On 09/16, Chao Yu wrote: > >> On 2019/9/9 20:04, Jaegeuk Kim wrote: > >>> On 09/09, Chao Yu wrote: > On 2019/9/9 16:06, Jaegeuk Kim wrote: > > On 09/09, Chao Yu wrote: > >> On 2019/9/9 9:25, Jaegeuk Kim wrote: > >>> GC must avoid select the same victim again. > >> > >> Blocks in previous victim will occupy addition free segment, I doubt > >> after this > >> change, FGGC may encounter out-of-free space issue more frequently. > > > > Hmm, actually this change seems wrong by sec_usage_check(). > > We may be able to avoid this only in the suspicious loop? > > > > --- > > fs/f2fs/gc.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c > > index e88f98ddf396..5877bd729689 100644 > > --- a/fs/f2fs/gc.c > > +++ b/fs/f2fs/gc.c > > @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, > > round++; > > } > > > > - if (gc_type == FG_GC) > > + if (gc_type == FG_GC && seg_freed) > > That's original solution Sahitya provided to avoid infinite loop of GC, > but I > suggest to find the root cause first, then we added .invalid_segmap for > that > purpose. > >>> > >>> I've checked the Sahitya's patch. So, it seems the problem can happen due > >>> to > >>> is_alive or atomic_file. > >> > >> For some conditions, this doesn't help, for example, two sections contain > >> the > >> same fewest valid blocks, it will cause to loop selecting them if it fails > >> to > >> migrate blocks. > >> > >> How about keeping it as it is to find potential bug. > > > > I think it'd be fine to merge this. Could you check the above scenario in > > more > > detail? > > I haven't saw this in real scenario yet. > > What I mean is if there is a bug (maybe in is_alive()) failing us to GC on one > section, when that bug happens in two candidates, there could be the same > condition that GC will run into loop (select A, fail to migrate; select B, > fail > to migrate, select A...). > > But I guess the benefit of this change is, if FGGC fails to migrate block due > to > i_gc_rwsem race, selecting another section and later retrying previous one may > avoid lock race, right? In any case, I think this can avoid potenial GC loop. At least to me, it'd be quite risky, if we remain this just for debugging purpose only. > > Thanks, > > > > > Thanks, > > > >> > >> Thanks, > >> > >>> > > Thanks, > > > sbi->cur_victim_sec = NULL_SEGNO; > > > > if (sync) > > > >>> . > >>> > > . > >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 2019/9/16 23:37, Jaegeuk Kim wrote: > On 09/16, Chao Yu wrote: >> On 2019/9/9 20:04, Jaegeuk Kim wrote: >>> On 09/09, Chao Yu wrote: On 2019/9/9 16:06, Jaegeuk Kim wrote: > On 09/09, Chao Yu wrote: >> On 2019/9/9 9:25, Jaegeuk Kim wrote: >>> GC must avoid select the same victim again. >> >> Blocks in previous victim will occupy addition free segment, I doubt >> after this >> change, FGGC may encounter out-of-free space issue more frequently. > > Hmm, actually this change seems wrong by sec_usage_check(). > We may be able to avoid this only in the suspicious loop? > > --- > fs/f2fs/gc.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c > index e88f98ddf396..5877bd729689 100644 > --- a/fs/f2fs/gc.c > +++ b/fs/f2fs/gc.c > @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, > round++; > } > > - if (gc_type == FG_GC) > + if (gc_type == FG_GC && seg_freed) That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose. >>> >>> I've checked the Sahitya's patch. So, it seems the problem can happen due to >>> is_alive or atomic_file. >> >> For some conditions, this doesn't help, for example, two sections contain the >> same fewest valid blocks, it will cause to loop selecting them if it fails to >> migrate blocks. >> >> How about keeping it as it is to find potential bug. > > I think it'd be fine to merge this. Could you check the above scenario in more > detail? I haven't saw this in real scenario yet. What I mean is if there is a bug (maybe in is_alive()) failing us to GC on one section, when that bug happens in two candidates, there could be the same condition that GC will run into loop (select A, fail to migrate; select B, fail to migrate, select A...). But I guess the benefit of this change is, if FGGC fails to migrate block due to i_gc_rwsem race, selecting another section and later retrying previous one may avoid lock race, right? Thanks, > > Thanks, > >> >> Thanks, >> >>> Thanks, > sbi->cur_victim_sec = NULL_SEGNO; > > if (sync) > >>> . >>> > . > ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 09/16, Chao Yu wrote: > On 2019/9/9 20:04, Jaegeuk Kim wrote: > > On 09/09, Chao Yu wrote: > >> On 2019/9/9 16:06, Jaegeuk Kim wrote: > >>> On 09/09, Chao Yu wrote: > On 2019/9/9 9:25, Jaegeuk Kim wrote: > > GC must avoid select the same victim again. > > Blocks in previous victim will occupy addition free segment, I doubt > after this > change, FGGC may encounter out-of-free space issue more frequently. > >>> > >>> Hmm, actually this change seems wrong by sec_usage_check(). > >>> We may be able to avoid this only in the suspicious loop? > >>> > >>> --- > >>> fs/f2fs/gc.c | 2 +- > >>> 1 file changed, 1 insertion(+), 1 deletion(-) > >>> > >>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c > >>> index e88f98ddf396..5877bd729689 100644 > >>> --- a/fs/f2fs/gc.c > >>> +++ b/fs/f2fs/gc.c > >>> @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, > >>> round++; > >>> } > >>> > >>> - if (gc_type == FG_GC) > >>> + if (gc_type == FG_GC && seg_freed) > >> > >> That's original solution Sahitya provided to avoid infinite loop of GC, > >> but I > >> suggest to find the root cause first, then we added .invalid_segmap for > >> that > >> purpose. > > > > I've checked the Sahitya's patch. So, it seems the problem can happen due to > > is_alive or atomic_file. > > For some conditions, this doesn't help, for example, two sections contain the > same fewest valid blocks, it will cause to loop selecting them if it fails to > migrate blocks. > > How about keeping it as it is to find potential bug. I think it'd be fine to merge this. Could you check the above scenario in more detail? Thanks, > > Thanks, > > > > >> > >> Thanks, > >> > >>> sbi->cur_victim_sec = NULL_SEGNO; > >>> > >>> if (sync) > >>> > > . > >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 2019/9/9 20:04, Jaegeuk Kim wrote: > On 09/09, Chao Yu wrote: >> On 2019/9/9 16:06, Jaegeuk Kim wrote: >>> On 09/09, Chao Yu wrote: On 2019/9/9 9:25, Jaegeuk Kim wrote: > GC must avoid select the same victim again. Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently. >>> >>> Hmm, actually this change seems wrong by sec_usage_check(). >>> We may be able to avoid this only in the suspicious loop? >>> >>> --- >>> fs/f2fs/gc.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c >>> index e88f98ddf396..5877bd729689 100644 >>> --- a/fs/f2fs/gc.c >>> +++ b/fs/f2fs/gc.c >>> @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, >>> round++; >>> } >>> >>> - if (gc_type == FG_GC) >>> + if (gc_type == FG_GC && seg_freed) >> >> That's original solution Sahitya provided to avoid infinite loop of GC, but I >> suggest to find the root cause first, then we added .invalid_segmap for that >> purpose. > > I've checked the Sahitya's patch. So, it seems the problem can happen due to > is_alive or atomic_file. For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks. How about keeping it as it is to find potential bug. Thanks, > >> >> Thanks, >> >>> sbi->cur_victim_sec = NULL_SEGNO; >>> >>> if (sync) >>> > . >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 09/09, Chao Yu wrote: > On 2019/9/9 16:06, Jaegeuk Kim wrote: > > On 09/09, Chao Yu wrote: > >> On 2019/9/9 9:25, Jaegeuk Kim wrote: > >>> GC must avoid select the same victim again. > >> > >> Blocks in previous victim will occupy addition free segment, I doubt after > >> this > >> change, FGGC may encounter out-of-free space issue more frequently. > > > > Hmm, actually this change seems wrong by sec_usage_check(). > > We may be able to avoid this only in the suspicious loop? > > > > --- > > fs/f2fs/gc.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c > > index e88f98ddf396..5877bd729689 100644 > > --- a/fs/f2fs/gc.c > > +++ b/fs/f2fs/gc.c > > @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, > > round++; > > } > > > > - if (gc_type == FG_GC) > > + if (gc_type == FG_GC && seg_freed) > > That's original solution Sahitya provided to avoid infinite loop of GC, but I > suggest to find the root cause first, then we added .invalid_segmap for that > purpose. I've checked the Sahitya's patch. So, it seems the problem can happen due to is_alive or atomic_file. > > Thanks, > > > sbi->cur_victim_sec = NULL_SEGNO; > > > > if (sync) > >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 2019/9/9 16:06, Jaegeuk Kim wrote: > On 09/09, Chao Yu wrote: >> On 2019/9/9 9:25, Jaegeuk Kim wrote: >>> GC must avoid select the same victim again. >> >> Blocks in previous victim will occupy addition free segment, I doubt after >> this >> change, FGGC may encounter out-of-free space issue more frequently. > > Hmm, actually this change seems wrong by sec_usage_check(). > We may be able to avoid this only in the suspicious loop? > > --- > fs/f2fs/gc.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c > index e88f98ddf396..5877bd729689 100644 > --- a/fs/f2fs/gc.c > +++ b/fs/f2fs/gc.c > @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, > round++; > } > > - if (gc_type == FG_GC) > + if (gc_type == FG_GC && seg_freed) That's original solution Sahitya provided to avoid infinite loop of GC, but I suggest to find the root cause first, then we added .invalid_segmap for that purpose. Thanks, > sbi->cur_victim_sec = NULL_SEGNO; > > if (sync) >
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 09/09, Chao Yu wrote: > On 2019/9/9 9:25, Jaegeuk Kim wrote: > > GC must avoid select the same victim again. > > Blocks in previous victim will occupy addition free segment, I doubt after > this > change, FGGC may encounter out-of-free space issue more frequently. Hmm, actually this change seems wrong by sec_usage_check(). We may be able to avoid this only in the suspicious loop? --- fs/f2fs/gc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c index e88f98ddf396..5877bd729689 100644 --- a/fs/f2fs/gc.c +++ b/fs/f2fs/gc.c @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, round++; } - if (gc_type == FG_GC) + if (gc_type == FG_GC && seg_freed) sbi->cur_victim_sec = NULL_SEGNO; if (sync) -- 2.19.0.605.g01d371f741-goog
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 2019/9/9 9:25, Jaegeuk Kim wrote:
> GC must avoid select the same victim again.
Blocks in previous victim will occupy addition free segment, I doubt after this
change, FGGC may encounter out-of-free space issue more frequently.
Thanks,
>
> Signed-off-by: Jaegeuk Kim
> ---
> fs/f2fs/gc.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
> index e88f98ddf396..15ca8bbb0b22 100644
> --- a/fs/f2fs/gc.c
> +++ b/fs/f2fs/gc.c
> @@ -274,6 +274,9 @@ static unsigned int get_cb_cost(struct f2fs_sb_info *sbi,
> unsigned int segno)
> static inline unsigned int get_gc_cost(struct f2fs_sb_info *sbi,
> unsigned int segno, struct victim_sel_policy *p)
> {
> + if (sbi->cur_victim_sec == GET_SEC_FROM_SEG(sbi, segno))
> + return UINT_MAX;
> +
> if (p->alloc_mode == SSR)
> return get_seg_entry(sbi, segno)->ckpt_valid_blocks;
>
> @@ -1326,9 +1329,6 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync,
> round++;
> }
>
> - if (gc_type == FG_GC)
> - sbi->cur_victim_sec = NULL_SEGNO;
> -
> if (sync)
> goto stop;
>
>
