Re: [f2fs-dev] [GIT PULL] zstd changes for v5.16
> On Nov 11, 2021, at 2:59 PM, Linus Torvalds > wrote: > > On Wed, Nov 10, 2021 at 10:47 AM Nick Terrell wrote: >> >> I just wanted to make sure that you’ve received my pull request. I’m a newbie >> here, so I want to make sure I’m not making a stupid mistake that means >> you’ve >> missed my message. I’d hate for this PR to not even be considered for merging >> in this window because of some mistake I’ve made. > > Oh, it's in my queue, but it's basically at the end of my queue > because I will need to take a much deeper look into what's going on. > > It's not just that you're a new source of pulls, it's also that this > is a big change and completely changes the organization of the zlib > stuff. So every time I look at my list of pending pulls, this always > ends up being "I'll do all the normal ones first". > > So it's not lost, but this is the kind of pull that I tend to do when > my queues have emptied. Which they haven't done yet.. Thanks for the ack, I really appreciate it! I figured that was the case. When you do get a chance to take a look, please let me know if there is anything more I can do to make it easier for you. In the meantime, I’ll focus on getting my key better signed. Best, Nick > Linus ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [GIT PULL] zstd changes for v5.16
On Wed, Nov 10, 2021 at 10:47 AM Nick Terrell wrote: > > I just wanted to make sure that you’ve received my pull request. I’m a newbie > here, so I want to make sure I’m not making a stupid mistake that means you’ve > missed my message. I’d hate for this PR to not even be considered for merging > in this window because of some mistake I’ve made. Oh, it's in my queue, but it's basically at the end of my queue because I will need to take a much deeper look into what's going on. It's not just that you're a new source of pulls, it's also that this is a big change and completely changes the organization of the zlib stuff. So every time I look at my list of pending pulls, this always ends up being "I'll do all the normal ones first". So it's not lost, but this is the kind of pull that I tend to do when my queues have emptied. Which they haven't done yet.. Linus ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [GIT PULL] zstd changes for v5.16
> On Nov 8, 2021, at 5:30 PM, Nick Terrell wrote: > > From: Nick Terrell > > Hi Linus, > > I am sending you a pull request to add myself as the maintainer of zstd and > update the zstd version in the kernel, which is now 4 years out of date, > to the latest zstd release. This includes bug fixes, much more extensive > fuzzing, > and performance improvements. And generates the kernel zstd automatically > from upstream zstd, so it is easier to keep the zstd verison up to date, and > we > don't fall so far out of date again. > > Thanks, > Nick Terrell > > The following changes since commit d2f38a3c6507b2520101f9a3807ed98f1bdc545a: > > Merge tag 'backlight-next-5.16' of > git://git.kernel.org/pub/scm/linux/kernel/git/lee/backlight (2021-11-08 > 12:21:28 -0800) > > are available in the Git repository at: > > g...@github.com:terrelln/linux.git tags/zstd-for-linus-v5.16 I’ve mistakenly sent you an authenticated git repo URL. Please use this address instead. Alternatively, I can re-send the pull request with the fixed URL if you please. git://github.com/terrelln/linux tags/zstd-for-linus-v5.16 Additionally, after some off-list feedback I am: 1. Working on getting my GPG key signed by two members of the Kernel developer PGP keyring. I hope to have at least one signature today. 2. Reaching out to Herbert Xu—because the first zstd version went through the crypto tree—and other relevant maintainers who have been involved, and trying to get a word of support for adding me as the maintainer of lib/zstd in this thread. Best, Nick Terrell > > for you to fetch changes up to 0a8ea235837cc39f27c45689930aa97ae91d5953: > > lib: zstd: Add cast to silence clang's -Wbitwise-instead-of-logical > (2021-11-08 16:55:38 -0800) > > > Update to zstd-1.4.10 > > This PR includes 5 commits that update the zstd library version: > > 1. Adds a new kernel-style wrapper around zstd. This wrapper API > is functionally equivalent to the subset of the current zstd API that is > currently used. The wrapper API changes to be kernel style so that the > symbols > don't collide with zstd's symbols. The update to zstd-1.4.10 maintains the > same > API and preserves the semantics, so that none of the callers need to be > updated. All callers are updated in the commit, because there are zero > functional changes. > 2. Adds an indirection for `lib/decompress_unzstd.c` so it > doesn't depend on the layout of `lib/zstd/` to include every source file. > This allows the next patch to be automatically generated. > 3. Imports the zstd-1.4.10 source code. This commit is automatically generated > from upstream zstd (https://github.com/facebook/zstd). > 4. Adds me (terre...@fb.com) as the maintainer of `lib/zstd`. > 5. Fixes a newly added build warning for clang. > > The discussion around this patchset has been pretty long, so I've included a > FAQ-style summary of the history of the patchset, and why we are taking this > approach. > > Why do we need to update? > - > > The zstd version in the kernel is based off of zstd-1.3.1, which is was > released > August 20, 2017. Since then zstd has seen many bug fixes and performance > improvements. And, importantly, upstream zstd is continuously fuzzed by > OSS-Fuzz, > and bug fixes aren't backported to older versions. So the only way to sanely > get > these fixes is to keep up to date with upstream zstd. There are no known > security > issues that affect the kernel, but we need to be able to update in case there > are. And while there are no known security issues, there are relevant bug > fixes. > For example the problem with large kernel decompression has been fixed > upstream > for over 2 years https://lkml.org/lkml/2020/9/29/27. > > Additionally the performance improvements for kernel use cases are > significant. > Measured for x86_64 on my Intel i9-9900k @ 3.6 GHz: > > - BtrFS zstd compression at levels 1 and 3 is 5% faster > - BtrFS zstd decompression+read is 15% faster > - SquashFS zstd decompression+read is 15% faster > - F2FS zstd compression+write at level 3 is 8% faster > - F2FS zstd decompression+read is 20% faster > - ZRAM decompression+read is 30% faster > - Kernel zstd decompression is 35% faster > - Initramfs zstd decompression+build is 5% faster > > On top of this, there are significant performance improvements coming down the > line in the next zstd release, and the new automated update patch generation > will allow us to pull them easily. > > How is the update patch generated? > -- > > The first two patches are preparation for updating the zstd version. Then the > 3rd patch in the series imports upstream zstd into the kernel. This patch is > automatically generated from upstream. A script makes the necessary changes > and > imports it into the kernel. The changes are: > > - Replace all libc
Re: [f2fs-dev] [GIT PULL] f2fs update for 5.16-rc1
On 11/11, Christoph Hellwig wrote: > So question on the direct I/O code. The perfectly fine iomap code > wasn't merged because of the pre-existing stale data exposure issuewith > f2fs direct I/O low-level implementation. This doesn't seem to be fixed > or even worked around, but instead we do get new direct I/O features? No worries. I'm working on the fix and will apply the Eric's patches on top of this change. > > On Wed, Nov 10, 2021 at 08:11:10PM -0800, Jaegeuk Kim wrote: > > Hi Linus, > > > > Could you please consider this pull request? > > > > Thanks, > > > > The following changes since commit ff1ffd71d5f0612cf194f5705c671d6b64bf5f91: > > > > Merge tag 'hyperv-fixes-signed-20210915' of > > git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux (2021-09-15 > > 17:18:56 -0700) > > > > are available in the Git repository at: > > > > git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git > > tags/f2fs-for-5.16-rc1 > > > > for you to fetch changes up to 5429c9dbc9025f9a166f64e22e3a69c94fd5b29b: > > > > f2fs: fix UAF in f2fs_available_free_memory (2021-11-09 08:23:17 -0800) > > > > > > f2fs-for-5.16-rc1 > > > > In this cycle, we've applied relatively small number of patches which fix > > subtle > > corner cases mainly, while introducing a new mount option to be able to > > fragment > > the disk intentionally for performance tests. > > > > Enhancement: > > - add a mount option to fragmente on-disk layout to understand the > > performance > > - support direct IO for multi-partitions > > - add a fault injection of dquot_initialize > > > > Bug fix: > > - address some lockdep complaints > > - fix a deadlock issue with quota > > - fix a memory tuning condition > > - fix compression condition to improve the ratio > > - fix disabling compression on the non-empty compressed file > > - invalidate cached pages before IPU/DIO writes > > > > And, we've added some minor clean-ups as usual. > > > > > > Chao Yu (7): > > f2fs: quota: fix potential deadlock > > f2fs: avoid attaching SB_ACTIVE flag during mount > > f2fs: introduce excess_dirty_threshold() > > f2fs: fix wrong condition to trigger background checkpoint correctly > > f2fs: multidevice: support direct IO > > f2fs: fix incorrect return value in f2fs_sanity_check_ckpt() > > f2fs: support fault injection for dquot_initialize() > > > > Daeho Jeong (2): > > f2fs: include non-compressed blocks in compr_written_block > > f2fs: introduce fragment allocation mode mount option > > > > Dongliang Mu (1): > > f2fs: fix UAF in f2fs_available_free_memory > > > > Fengnan Chang (1): > > f2fs: compress: fix overwrite may reduce compress ratio unproperly > > > > Gao Xiang (1): > > f2fs: fix up f2fs_lookup tracepoints > > > > Hyeong-Jun Kim (2): > > f2fs: compress: disallow disabling compress on non-empty compressed > > file > > f2fs: invalidate META_MAPPING before IPU/DIO write > > > > Jaegeuk Kim (1): > > f2fs: should use GFP_NOFS for directory inodes > > > > Keoseong Park (1): > > f2fs: fix to use WHINT_MODE > > > > Qing Wang (1): > > f2fs: replace snprintf in show functions with sysfs_emit > > > > Weichao Guo (1): > > f2fs: set SBI_NEED_FSCK flag when inconsistent node block found > > > > Documentation/ABI/testing/sysfs-fs-f2fs | 16 ++ > > Documentation/filesystems/f2fs.rst | 19 +++ > > fs/f2fs/checkpoint.c| 8 ++- > > fs/f2fs/compress.c | 20 +++ > > fs/f2fs/data.c | 95 > > + > > fs/f2fs/f2fs.h | 54 --- > > fs/f2fs/file.c | 6 +-- > > fs/f2fs/gc.c| 5 +- > > fs/f2fs/inline.c| 2 +- > > fs/f2fs/inode.c | 4 +- > > fs/f2fs/namei.c | 32 +-- > > fs/f2fs/node.c | 1 + > > fs/f2fs/node.h | 5 -- > > fs/f2fs/recovery.c | 14 ++--- > > fs/f2fs/segment.c | 83 +--- > > fs/f2fs/segment.h | 1 + > > fs/f2fs/super.c | 39 +- > > fs/f2fs/sysfs.c | 24 - > > fs/f2fs/verity.c| 2 +- > > fs/f2fs/xattr.c | 2 +- > > include/trace/events/f2fs.h | 33 > > 21 files changed, 359 insertions(+), 106 deletions(-) > ---end quoted text--- ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Re: [f2fs-dev] [GIT PULL] f2fs update for 5.16-rc1
So question on the direct I/O code. The perfectly fine iomap code wasn't merged because of the pre-existing stale data exposure issuewith f2fs direct I/O low-level implementation. This doesn't seem to be fixed or even worked around, but instead we do get new direct I/O features? On Wed, Nov 10, 2021 at 08:11:10PM -0800, Jaegeuk Kim wrote: > Hi Linus, > > Could you please consider this pull request? > > Thanks, > > The following changes since commit ff1ffd71d5f0612cf194f5705c671d6b64bf5f91: > > Merge tag 'hyperv-fixes-signed-20210915' of > git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux (2021-09-15 > 17:18:56 -0700) > > are available in the Git repository at: > > git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git > tags/f2fs-for-5.16-rc1 > > for you to fetch changes up to 5429c9dbc9025f9a166f64e22e3a69c94fd5b29b: > > f2fs: fix UAF in f2fs_available_free_memory (2021-11-09 08:23:17 -0800) > > > f2fs-for-5.16-rc1 > > In this cycle, we've applied relatively small number of patches which fix > subtle > corner cases mainly, while introducing a new mount option to be able to > fragment > the disk intentionally for performance tests. > > Enhancement: > - add a mount option to fragmente on-disk layout to understand the > performance > - support direct IO for multi-partitions > - add a fault injection of dquot_initialize > > Bug fix: > - address some lockdep complaints > - fix a deadlock issue with quota > - fix a memory tuning condition > - fix compression condition to improve the ratio > - fix disabling compression on the non-empty compressed file > - invalidate cached pages before IPU/DIO writes > > And, we've added some minor clean-ups as usual. > > > Chao Yu (7): > f2fs: quota: fix potential deadlock > f2fs: avoid attaching SB_ACTIVE flag during mount > f2fs: introduce excess_dirty_threshold() > f2fs: fix wrong condition to trigger background checkpoint correctly > f2fs: multidevice: support direct IO > f2fs: fix incorrect return value in f2fs_sanity_check_ckpt() > f2fs: support fault injection for dquot_initialize() > > Daeho Jeong (2): > f2fs: include non-compressed blocks in compr_written_block > f2fs: introduce fragment allocation mode mount option > > Dongliang Mu (1): > f2fs: fix UAF in f2fs_available_free_memory > > Fengnan Chang (1): > f2fs: compress: fix overwrite may reduce compress ratio unproperly > > Gao Xiang (1): > f2fs: fix up f2fs_lookup tracepoints > > Hyeong-Jun Kim (2): > f2fs: compress: disallow disabling compress on non-empty compressed file > f2fs: invalidate META_MAPPING before IPU/DIO write > > Jaegeuk Kim (1): > f2fs: should use GFP_NOFS for directory inodes > > Keoseong Park (1): > f2fs: fix to use WHINT_MODE > > Qing Wang (1): > f2fs: replace snprintf in show functions with sysfs_emit > > Weichao Guo (1): > f2fs: set SBI_NEED_FSCK flag when inconsistent node block found > > Documentation/ABI/testing/sysfs-fs-f2fs | 16 ++ > Documentation/filesystems/f2fs.rst | 19 +++ > fs/f2fs/checkpoint.c| 8 ++- > fs/f2fs/compress.c | 20 +++ > fs/f2fs/data.c | 95 > + > fs/f2fs/f2fs.h | 54 --- > fs/f2fs/file.c | 6 +-- > fs/f2fs/gc.c| 5 +- > fs/f2fs/inline.c| 2 +- > fs/f2fs/inode.c | 4 +- > fs/f2fs/namei.c | 32 +-- > fs/f2fs/node.c | 1 + > fs/f2fs/node.h | 5 -- > fs/f2fs/recovery.c | 14 ++--- > fs/f2fs/segment.c | 83 +--- > fs/f2fs/segment.h | 1 + > fs/f2fs/super.c | 39 +- > fs/f2fs/sysfs.c | 24 - > fs/f2fs/verity.c| 2 +- > fs/f2fs/xattr.c | 2 +- > include/trace/events/f2fs.h | 33 > 21 files changed, 359 insertions(+), 106 deletions(-) ---end quoted text--- ___ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
