[GIT PULL] fs-verity for 5.4
The following changes since commit 609488bc979f99f805f34e9a32c1e3b71179d10b:
Linux 5.3-rc2 (2019-07-28 12:47:02 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/fs/fscrypt/fscrypt.git tags/fsverity-for-linus
for you to fetch changes up to 95ae251fe82838b85c6d37e5a1775006e2a42ae0:
f2fs: add fs-verity support (2019-08-12 19:33:51 -0700)
Hi Linus,
Please consider pulling fs-verity for 5.4.
fs-verity is a filesystem feature that provides Merkle tree based
hashing (similar to dm-verity) for individual readonly files, mainly for
the purpose of efficient authenticity verification.
This pull request includes:
(a) The fs/verity/ support layer and documentation.
(b) fs-verity support for ext4 and f2fs.
Compared to the original fs-verity patchset from last year, the UAPI to
enable fs-verity on a file has been greatly simplified. Lots of other
things were cleaned up too.
fs-verity is planned to be used by two different projects on Android;
most of the userspace code is in place already. Another userspace tool
("fsverity-utils"), and xfstests, are also available. e2fsprogs and
f2fs-tools already have fs-verity support. Other people have shown
interest in using fs-verity too.
I've tested this on ext4 and f2fs with xfstests, both the existing tests
and the new fs-verity tests. This has also been in linux-next since
July 30 with no reported issues except a couple minor ones I found
myself and folded in fixes for.
Ted and I will be co-maintaining fs-verity.
There will be some fairly straightforward merge conflicts with the ext4
and f2fs trees. I've tested the resolution of these in linux-next.
This will also "silently" conflict (compiler warning only) with the key
ACLs patchset, if you merge it again this cycle. The resolution is to
translate the key permissions to an ACL in fs/verity/signature.c. I
suggest using the resolution in linux-next, which I've tested. This
resolution avoids making any behavior changes; note that some of the old
permissions map to multiple new permissions.
Eric Biggers (17):
fs-verity: add a documentation file
fs-verity: add MAINTAINERS file entry
fs-verity: add UAPI header
fs: uapi: define verity bit for FS_IOC_GETFLAGS
fs-verity: add Kconfig and the helper functions for hashing
fs-verity: add inode and superblock fields
fs-verity: add the hook for file ->open()
fs-verity: add the hook for file ->setattr()
fs-verity: add data verification hooks for ->readpages()
fs-verity: implement FS_IOC_ENABLE_VERITY ioctl
fs-verity: implement FS_IOC_MEASURE_VERITY ioctl
fs-verity: add SHA-512 support
fs-verity: support builtin file signatures
ext4: add basic fs-verity support
ext4: add fs-verity read support
ext4: update on-disk format documentation for fs-verity
f2fs: add fs-verity support
Documentation/filesystems/ext4/inodes.rst | 6 +-
Documentation/filesystems/ext4/overview.rst | 1 +
Documentation/filesystems/ext4/super.rst| 2 +
Documentation/filesystems/ext4/verity.rst | 41 ++
Documentation/filesystems/fsverity.rst | 726
Documentation/filesystems/index.rst | 1 +
Documentation/ioctl/ioctl-number.rst| 1 +
MAINTAINERS | 12 +
fs/Kconfig | 2 +
fs/Makefile | 1 +
fs/ext4/Makefile| 1 +
fs/ext4/ext4.h | 23 +-
fs/ext4/file.c | 4 +
fs/ext4/inode.c | 55 ++-
fs/ext4/ioctl.c | 13 +
fs/ext4/readpage.c | 211 ++--
fs/ext4/super.c | 18 +-
fs/ext4/sysfs.c | 6 +
fs/ext4/verity.c| 367 ++
fs/f2fs/Makefile| 1 +
fs/f2fs/data.c | 75 ++-
fs/f2fs/f2fs.h | 20 +-
fs/f2fs/file.c | 43 +-
fs/f2fs/inode.c | 5 +-
fs/f2fs/super.c | 3 +
fs/f2fs/sysfs.c | 11 +
fs/f2fs/verity.c| 247 ++
fs/f2fs/xattr.h | 2 +
fs/verity/Kconfig | 55 +++
fs/verity/Makefile | 10 +
fs/verity/enable.c | 377 +++
fs/verity/fsverity_private.h| 185 +++
fs/verity/hash_algs.c | 280 +++
fs/verity/init.c| 61 +++
fs/verity/measure.c | 57 +++
Re: [f2fs-dev] [PATCH v4 2/2] fsck.f2fs: Check write pointer consistency with current segments
On 2019/9/12 16:16, Shinichiro Kawasaki wrote:
> On Sep 10, 2019 / 17:12, Chao Yu wrote:
>> On 2019/9/10 16:10, Shinichiro Kawasaki wrote:
>>> On Sep 09, 2019 / 15:14, Chao Yu wrote:
On 2019/9/6 16:31, Shinichiro Kawasaki wrote:
> On Sep 05, 2019 / 17:58, Chao Yu wrote:
>> Hi Shinichiro,
>>
>> Sorry for the delay.
>>
>> On 2019/9/3 16:37, Shinichiro Kawasaki wrote:
>>> On Sep 02, 2019 / 15:02, Chao Yu wrote:
On 2019/8/30 18:19, Shin'ichiro Kawasaki wrote:
> On sudden f2fs shutdown, zoned block device status and f2fs current
> segment positions in meta data can be inconsistent. When f2fs shutdown
> happens before write operations completes, write pointers of zoned
> block
> devices can go further but f2fs meta data keeps current segments at
> positions before the write operations. After remounting the f2fs, the
> inconsistency causes write operations not at write pointers and
> "Unaligned write command" error is reported. This error was observed
> when
> xfstests test case generic/388 was run with f2fs on a zoned block
> device.
>
> To avoid the error, have f2fs.fsck check consistency between each
> current
> segment's position and the write pointer of the zone the current
> segment
> points to. If the write pointer goes advance from the current segment,
> fix the current segment position setting at same as the write pointer
> position. If the write pointer goes to the zone end, find a new zone
> and
> set the current segment position at the new zone start. In case the
> write
> pointer is behind the current segment, write zero data at the write
> pointer position to make write pointer position at same as the current
> segment.
>
> When inconsistencies are found, turn on c.bug_on flag in
> fsck_verify() to
> ask users to fix them or not. When inconsistencies get fixed, turn on
> 'force' flag in fsck_verify() to enforce fixes in following checks.
> This
> position fix is done at the beginning of do_fsck() function so that
> other
> checks reflect the current segment modification.
>
> Also add GET_SEC_FROM_SEG and GET_SEG_FROM_SEC macros in fsck/fsck.h
> to
> simplify the code.
>
> Signed-off-by: Shin'ichiro Kawasaki
> ---
> fsck/f2fs.h | 5 ++
> fsck/fsck.c | 198
>
> fsck/fsck.h | 3 +
> fsck/main.c | 2 +
> 4 files changed, 208 insertions(+)
>
> diff --git a/fsck/f2fs.h b/fsck/f2fs.h
> index 4dc6698..2c1c2b3 100644
> --- a/fsck/f2fs.h
> +++ b/fsck/f2fs.h
> @@ -337,6 +337,11 @@ static inline block_t __end_block_addr(struct
> f2fs_sb_info *sbi)
> #define GET_BLKOFF_FROM_SEG0(sbi, blk_addr)
> \
> (GET_SEGOFF_FROM_SEG0(sbi, blk_addr) & (sbi->blocks_per_seg -
> 1))
>
> +#define GET_SEC_FROM_SEG(sbi, segno)
> \
> + ((segno) / (sbi)->segs_per_sec)
> +#define GET_SEG_FROM_SEC(sbi, secno)
> \
> + ((secno) * (sbi)->segs_per_sec)
> +
> #define FREE_I_START_SEGNO(sbi)
> \
> GET_SEGNO_FROM_SEG0(sbi, SM_I(sbi)->main_blkaddr)
> #define GET_R2L_SEGNO(sbi, segno)(segno +
> FREE_I_START_SEGNO(sbi))
> diff --git a/fsck/fsck.c b/fsck/fsck.c
> index 8953ca1..a0f6849 100644
> --- a/fsck/fsck.c
> +++ b/fsck/fsck.c
> @@ -2574,6 +2574,190 @@ out:
> return cnt;
> }
>
> +/*
> + * Search a free section in main area. Start search from the section
> specified
> + * with segno argument toward main area end. Return first segment of
> the found
> + * section in segno argument.
> + */
> +static int find_next_free_section(struct f2fs_sb_info *sbi,
> + unsigned int *segno)
> +{
> + unsigned int i, sec, section_valid_blocks;
> + unsigned int end_segno = GET_SEGNO(sbi, SM_I(sbi)->main_blkaddr)
> + + SM_I(sbi)->main_segments;
> + unsigned int end_sec = GET_SEC_FROM_SEG(sbi, end_segno);
> + struct seg_entry *se;
> + struct curseg_info *cs;
> +
> + for (sec = GET_SEC_FROM_SEG(sbi, *segno); sec < end_sec; sec++)
> {
> + /* find a section without valid blocks */
> +
Re: [f2fs-dev] [PATCH 1/2] f2fs: do not select same victim right again
On 2019/9/9 20:04, Jaegeuk Kim wrote: > On 09/09, Chao Yu wrote: >> On 2019/9/9 16:06, Jaegeuk Kim wrote: >>> On 09/09, Chao Yu wrote: On 2019/9/9 9:25, Jaegeuk Kim wrote: > GC must avoid select the same victim again. Blocks in previous victim will occupy addition free segment, I doubt after this change, FGGC may encounter out-of-free space issue more frequently. >>> >>> Hmm, actually this change seems wrong by sec_usage_check(). >>> We may be able to avoid this only in the suspicious loop? >>> >>> --- >>> fs/f2fs/gc.c | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c >>> index e88f98ddf396..5877bd729689 100644 >>> --- a/fs/f2fs/gc.c >>> +++ b/fs/f2fs/gc.c >>> @@ -1326,7 +1326,7 @@ int f2fs_gc(struct f2fs_sb_info *sbi, bool sync, >>> round++; >>> } >>> >>> - if (gc_type == FG_GC) >>> + if (gc_type == FG_GC && seg_freed) >> >> That's original solution Sahitya provided to avoid infinite loop of GC, but I >> suggest to find the root cause first, then we added .invalid_segmap for that >> purpose. > > I've checked the Sahitya's patch. So, it seems the problem can happen due to > is_alive or atomic_file. For some conditions, this doesn't help, for example, two sections contain the same fewest valid blocks, it will cause to loop selecting them if it fails to migrate blocks. How about keeping it as it is to find potential bug. Thanks, > >> >> Thanks, >> >>> sbi->cur_victim_sec = NULL_SEGNO; >>> >>> if (sync) >>> > . >
Re: [f2fs-dev] [PATCH 3/3] f2fs: fix inode rwsem regression
On 2019/9/14 3:46, Jaegeuk Kim wrote:
> https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git/commit/?h=dev=ebef4d7eda0d06a6ab6dc0f9e9f848276e605962
Reviewed-by: Chao Yu
Thanks,
>
> Merged. Thanks,
>
> On 09/11, Goldwyn Rodrigues wrote:
>> From: Goldwyn Rodrigues
>>
>> This is similar to 942491c9e6d6 ("xfs: fix AIM7 regression")
>> Apparently our current rwsem code doesn't like doing the trylock, then
>> lock for real scheme. So change our read/write methods to just do the
>> trylock for the RWF_NOWAIT case.
>>
>> We don't need a check for IOCB_NOWAIT and !direct-IO because it
>> is checked in generic_write_checks().
>>
>> Fixes: b91050a80cec ("f2fs: add nowait aio support")
>> Signed-off-by: Goldwyn Rodrigues
>> ---
>> fs/f2fs/file.c | 10 +++---
>> 1 file changed, 3 insertions(+), 7 deletions(-)
>>
>> diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
>> index 3e58a6f697dd..c6f3ef815c05 100644
>> --- a/fs/f2fs/file.c
>> +++ b/fs/f2fs/file.c
>> @@ -3134,16 +3134,12 @@ static ssize_t f2fs_file_write_iter(struct kiocb
>> *iocb, struct iov_iter *from)
>> goto out;
>> }
>>
>> -if ((iocb->ki_flags & IOCB_NOWAIT) && !(iocb->ki_flags & IOCB_DIRECT)) {
>> -ret = -EINVAL;
>> -goto out;
>> -}
>> -
>> -if (!inode_trylock(inode)) {
>> -if (iocb->ki_flags & IOCB_NOWAIT) {
>> +if (iocb->ki_flags & IOCB_NOWAIT) {
>> +if (!inode_trylock(inode)) {
>> ret = -EAGAIN;
>> goto out;
>> }
>> +} else {
>> inode_lock(inode);
>> }
>>
>> --
>> 2.16.4
> .
>
___
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [PATCH 2/2] f2fs: avoid infinite GC loop due to stale atomic files
On 2019/9/10 20:09, Jaegeuk Kim wrote: >> The lock is used to protect F2FS_I(inode)->inmem_pages list... it should be >> kept? > Urg.. yup. I added. Reviewed-by: Chao Yu Thanks, >
