[U-Boot] [PATCH v3] Enable journal replay for UBIFS
Hello Heiko Thank you fro the review. I added atomic_long_read in ubifs.h as for other atomic operations. I hope (but I can't garantee) by this time the mail is well formed. If not please tell me again. During mount_ubifs the ubifs_replay_journal was disabled. This patch enables it again and fix some unrecoverable UBIFS volumes. Signed-off-by: Anton Habegger anton.habeg...@delta-es.com --- fs/ubifs/Makefile | 2 +- fs/ubifs/gc.c | 987 ++ fs/ubifs/replay.c | 4 - fs/ubifs/super.c | 8 +- fs/ubifs/tnc.c| 2 - fs/ubifs/ubifs.h | 1 + 6 files changed, 990 insertions(+), 14 deletions(-) create mode 100644 fs/ubifs/gc.c diff --git a/fs/ubifs/Makefile b/fs/ubifs/Makefile index 8c8c6ac..5efb349 100644 --- a/fs/ubifs/Makefile +++ b/fs/ubifs/Makefile @@ -12,4 +12,4 @@ obj-y := ubifs.o io.o super.o sb.o master.o lpt.o obj-y += lpt_commit.o scan.o lprops.o obj-y += tnc.o tnc_misc.o debug.o crc16.o budget.o -obj-y += log.o orphan.o recovery.o replay.o +obj-y += log.o orphan.o recovery.o replay.o gc.o diff --git a/fs/ubifs/gc.c b/fs/ubifs/gc.c new file mode 100644 index 000..159ee67 --- /dev/null +++ b/fs/ubifs/gc.c @@ -0,0 +1,987 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* + * This file implements garbage collection. The procedure for garbage collection + * is different depending on whether a LEB as an index LEB (contains index + * nodes) or not. For non-index LEBs, garbage collection finds a LEB which + * contains a lot of dirty space (obsolete nodes), and copies the non-obsolete + * nodes to the journal, at which point the garbage-collected LEB is free to be + * reused. For index LEBs, garbage collection marks the non-obsolete index nodes + * dirty in the TNC, and after the next commit, the garbage-collected LEB is + * to be reused. Garbage collection will cause the number of dirty index nodes + * to grow, however sufficient space is reserved for the index to ensure the + * commit will never run out of space. + * + * Notes about dead watermark. At current UBIFS implementation we assume that + * LEBs which have less than @c-dead_wm bytes of free + dirty space are full + * and not worth garbage-collecting. The dead watermark is one min. I/O unit + * size, or min. UBIFS node size, depending on what is greater. Indeed, UBIFS + * Garbage Collector has to synchronize the GC head's write buffer before + * returning, so this is about wasting one min. I/O unit. However, UBIFS GC can + * actually reclaim even very small pieces of dirty space by garbage collecting + * enough dirty LEBs, but we do not bother doing this at this implementation. + * + * Notes about dark watermark. The results of GC work depends on how big are + * the UBIFS nodes GC deals with. Large nodes make GC waste more space. Indeed, + * if GC move data from LEB A to LEB B and nodes in LEB A are large, GC would + * have to waste large pieces of free space at the end of LEB B, because nodes + * from LEB A would not fit. And the worst situation is when all nodes are of + * maximum size. So dark watermark is the amount of free + dirty space in LEB + * which are guaranteed to be reclaimable. If LEB has less space, the GC might + * be unable to reclaim it. So, LEBs with free + dirty greater than dark + * watermark are good LEBs from GC's point of few. The other LEBs are not so + * good, and GC takes extra care when moving them. + */ +#ifndef __UBOOT__ +#include linux/slab.h +#include linux/pagemap.h +#include linux/list_sort.h +#endif +#include ubifs.h + +#ifndef __UBOOT__ +/* + * GC may need to move more than one LEB to make progress. The below constants + * define soft and hard limits on the number of LEBs the garbage collector + * may move. + */ +#define SOFT_LEBS_LIMIT 4 +#define HARD_LEBS_LIMIT 32 + +/** + * switch_gc_head - switch the garbage collection journal head. + * @c: UBIFS file-system description object + * @buf: buffer to write + * @len: length of the buffer to write + * @lnum: LEB number written is returned here + * @offs: offset written is returned here + * + * This function switch the GC head to the next LEB which is reserved in + * @c-gc_lnum. Returns %0 in case of success,
Re: [U-Boot] [PATCH v3] Enable journal replay for UBIFS
Hello Anton, Am 20.01.2015 15:22, schrieb Anton Habegger: Hello Heiko Thank you fro the review. I added atomic_long_read in ubifs.h as for other atomic operations. Hmm.. I see, there are other missing atomic defines for UBI also ... Ok, I tend to accept your patch, but the correct way would be to import include/asm-generic/atomic-long.h from linux and drop this definitions in ubifs.h ... Could you prepare such a second patch? I hope (but I can't garantee) by this time the mail is well formed. If not please tell me again. It looks good now! During mount_ubifs the ubifs_replay_journal was disabled. This patch enables it again and fix some unrecoverable UBIFS volumes. nitpick ... Sorry, this is not a valid commit message ... please look at http://www.denx.de/wiki/view/U-Boot/Patches how to write a commit message. Please add comments to your patch under a --- line, and provide a patch history ... add also a hint from which linux version you picked up the new file fs/ubifs/gc.c Current version of the other MTD/UBI/UBIFS files is 1860e379875df: Linux 3.15 Could you use this as base for your patch? Thanks! Signed-off-by: Anton Habegger anton.habeg...@delta-es.com --- fs/ubifs/Makefile | 2 +- fs/ubifs/gc.c | 987 ++ fs/ubifs/replay.c | 4 - fs/ubifs/super.c | 8 +- fs/ubifs/tnc.c| 2 - fs/ubifs/ubifs.h | 1 + 6 files changed, 990 insertions(+), 14 deletions(-) create mode 100644 fs/ubifs/gc.c Compiling current mainline with your patch drops the follwoing warning: CC fs/ubifs/tnc.o fs/ubifs/tnc.c: In function 'ubifs_tnc_close': fs/ubifs/tnc.c:2861:8: warning: variable 'n' set but not used [-Wunused-but-set-variable] Could you please fix this too? Thanks! Beside of this nitpicks, your patch looks good. bye, Heiko diff --git a/fs/ubifs/Makefile b/fs/ubifs/Makefile index 8c8c6ac..5efb349 100644 --- a/fs/ubifs/Makefile +++ b/fs/ubifs/Makefile @@ -12,4 +12,4 @@ obj-y := ubifs.o io.o super.o sb.o master.o lpt.o obj-y += lpt_commit.o scan.o lprops.o obj-y += tnc.o tnc_misc.o debug.o crc16.o budget.o -obj-y += log.o orphan.o recovery.o replay.o +obj-y += log.o orphan.o recovery.o replay.o gc.o diff --git a/fs/ubifs/gc.c b/fs/ubifs/gc.c new file mode 100644 index 000..159ee67 --- /dev/null +++ b/fs/ubifs/gc.c @@ -0,0 +1,987 @@ +/* + * This file is part of UBIFS. + * + * Copyright (C) 2006-2008 Nokia Corporation. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 51 + * Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + * + * Authors: Adrian Hunter + * Artem Bityutskiy (Битюцкий Артём) + */ + +/* + * This file implements garbage collection. The procedure for garbage collection + * is different depending on whether a LEB as an index LEB (contains index + * nodes) or not. For non-index LEBs, garbage collection finds a LEB which + * contains a lot of dirty space (obsolete nodes), and copies the non-obsolete + * nodes to the journal, at which point the garbage-collected LEB is free to be + * reused. For index LEBs, garbage collection marks the non-obsolete index nodes + * dirty in the TNC, and after the next commit, the garbage-collected LEB is + * to be reused. Garbage collection will cause the number of dirty index nodes + * to grow, however sufficient space is reserved for the index to ensure the + * commit will never run out of space. + * + * Notes about dead watermark. At current UBIFS implementation we assume that + * LEBs which have less than @c-dead_wm bytes of free + dirty space are full + * and not worth garbage-collecting. The dead watermark is one min. I/O unit + * size, or min. UBIFS node size, depending on what is greater. Indeed, UBIFS + * Garbage Collector has to synchronize the GC head's write buffer before + * returning, so this is about wasting one min. I/O unit. However, UBIFS GC can + * actually reclaim even very small pieces of dirty space by garbage collecting + * enough dirty LEBs, but we do not bother doing this at this implementation. + * + * Notes about dark watermark. The results of GC work depends on how big are + * the UBIFS nodes GC deals with. Large nodes make GC waste more space. Indeed, + * if GC move data from LEB A to LEB B and nodes in LEB A are large, GC would + * have to waste large pieces of free space at the end of LEB B, because nodes + * from LEB A would not fit. And the worst situation is when all
