Re: [PATCH 1/4] bcachefs: allow initializing a bcachefs filesystem from a source directory

2024-06-13 Thread Kent Overstreet
On Tue, Jun 04, 2024 at 10:43:22AM +0300, Ariel Miculas wrote: > Add a new source command line argument which specifies the directory > tree that will be copied onto the newly formatted bcachefs filesystem. > > This commit also fixes an issue in copy_link where uninitialized data is > copied into

Re: [syzbot] [bcachefs?] UBSAN: array-index-out-of-bounds in bch2_bkey_swab_key

2024-06-12 Thread Kent Overstreet
On Sat, Jun 08, 2024 at 03:15:24PM -0700, syzbot wrote: > Hello, > > syzbot found the following issue on: > > HEAD commit:d30d0e49da71 Merge tag 'net-6.10-rc3' of git://git.kernel... > git tree: upstream > console output: https://syzkaller.appspot.com/x/log.txt?x=17cac09698 >

Re: [syzbot] [bcachefs?] WARNING: locking bug in srcu_gp_start_if_needed

2024-06-12 Thread Kent Overstreet
On Mon, Jun 10, 2024 at 08:14:32PM -0700, syzbot wrote: > Hello, > > syzbot found the following issue on: > > HEAD commit:d30d0e49da71 Merge tag 'net-6.10-rc3' of git://git.kernel... > git tree: upstream > console output: https://syzkaller.appspot.com/x/log.txt?x=11f5d89698 >

Re: [syzbot] [bcachefs?] WARNING in bch2_fs_ioctl

2024-06-12 Thread Kent Overstreet
94cd5708292e/vmlinux-83a7eefe.xz > kernel image: > https://storage.googleapis.com/syzbot-assets/a89698812e8b/bzImage-83a7eefe.xz > mounted in repro: > https://storage.googleapis.com/syzbot-assets/2b8f6b18fd8a/mount_0.gz > > The issue was bisected to: > > commit 03ef80b469d5d835

[GIT PULL] bcachefs fixes for 6.10-rc4

2024-06-12 Thread Kent Overstreet
drastic performance improvement - assorted syzbot fixes Hongbo Li (1): bcachefs: fix the display format for show-super Kent Overstreet (16): bcachefs: Split out btree_write_submit_wq bcachefs: Fix incorrect error

Re: [PATCH v2 4/4] bcachefs: remove now spurious i_state initialization

2024-06-11 Thread Kent Overstreet
On Tue, Jun 11, 2024 at 02:06:26PM GMT, Mateusz Guzik wrote: > inode_init_always started setting the field to 0. > > Signed-off-by: Mateusz Guzik Acked-by: Kent Overstreet

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-10 Thread Kent Overstreet
On Mon, Jun 10, 2024 at 08:59:21PM +0200, Mateusz Guzik wrote: > My plan for short term is to get the cheap hack in, it already > provides vast majority of whatever win one can expect to get from any > inode hash rework for next to no price. > > As for long term, there is other stuff I intend to

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-10 Thread Kent Overstreet
On Mon, Jun 10, 2024 at 08:44:37PM +0200, Mateusz Guzik wrote: > On Mon, Jun 10, 2024 at 8:13 PM Kent Overstreet > wrote: > > > > On Sat, Jun 08, 2024 at 11:24:37AM +0200, Mateusz Guzik wrote: > > > On Fri, Jun 07, 2024 at 02:10:05PM -0400, Kent Overstreet wrote: >

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-10 Thread Kent Overstreet
On Sat, Jun 08, 2024 at 11:24:37AM +0200, Mateusz Guzik wrote: > On Fri, Jun 07, 2024 at 02:10:05PM -0400, Kent Overstreet wrote: > > Does the following patch help? I think the hammering on the key cache > > lock may be correlated with the key cache being mostly empty (and it

Re: [syzbot] [bcachefs?] kernel BUG in bch2_fs_recovery

2024-06-09 Thread Kent Overstreet
On Sun, Jun 09, 2024 at 05:24:02AM -0700, syzbot wrote: > syzbot suspects this issue was fixed by commit: > > commit 7ffec9ccdc6ad8356792f9a7823b1fe9c8a10cbf > Author: Kent Overstreet > Date: Fri May 3 14:55:17 2024 + > > bcachefs: don't free error pointers >

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-07 Thread Kent Overstreet
On Fri, Jun 07, 2024 at 09:15:31PM +0200, Mateusz Guzik wrote: > On Fri, Jun 7, 2024 at 8:43 PM Kent Overstreet > wrote: > > > > On Fri, Jun 07, 2024 at 08:28:07PM +0200, Mateusz Guzik wrote: > > > Well I did a quick skim through, it's probably a no-go: > > &

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-07 Thread Kent Overstreet
On Fri, Jun 07, 2024 at 08:28:07PM +0200, Mateusz Guzik wrote: > Well I did a quick skim through, it's probably a no-go: > 1. uses bit locks which i'm told would be problematic vs lockdep and > rt kernels (see the original patchset by Dave Chinner, linked in my > fsdevel post) The bit spinlocks

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-07 Thread Kent Overstreet
On Fri, Jun 07, 2024 at 07:59:10PM +0200, Mateusz Guzik wrote: > On Fri, Jun 7, 2024 at 7:53 PM Kent Overstreet > wrote: > > > > On Fri, Jun 07, 2024 at 06:51:05PM +0200, Mateusz Guzik wrote: > > > On Fri, Jun 7, 2024 at 6:28 PM Mateusz Guzik wrote: > > > >

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-07 Thread Kent Overstreet
On Fri, Jun 07, 2024 at 06:51:05PM +0200, Mateusz Guzik wrote: > On Fri, Jun 7, 2024 at 6:28 PM Mateusz Guzik wrote: > > > > On Fri, Jun 7, 2024 at 6:07 PM Kent Overstreet > > wrote: > > > > > > On Fri, Jun 07, 2024 at 12:10:34PM +0200, Mateusz Guzik wrot

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-07 Thread Kent Overstreet
On Fri, Jun 07, 2024 at 12:10:34PM +0200, Mateusz Guzik wrote: > On Fri, Jun 7, 2024 at 11:13 AM Kent Overstreet > wrote: > > > > On Fri, Jun 07, 2024 at 08:50:40AM +0200, Mateusz Guzik wrote: > > > On Fri, Jun 7, 2024 at 2:31 AM Kent Overstreet > > > wrote:

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-07 Thread Kent Overstreet
On Fri, Jun 07, 2024 at 08:50:40AM +0200, Mateusz Guzik wrote: > On Fri, Jun 7, 2024 at 2:31 AM Kent Overstreet > wrote: > > > > On Thu, Jun 06, 2024 at 08:40:50PM +0200, Mateusz Guzik wrote: > > > So I tried out bcachefs again and it once more fails to complete > &

Re: long stalls when creating 20 directory trees of 1 million inodes in parallel

2024-06-06 Thread Kent Overstreet
On Thu, Jun 06, 2024 at 08:40:50PM +0200, Mateusz Guzik wrote: > So I tried out bcachefs again and it once more fails to complete > parallel creation of 20 mln files -- processes doing the work use cpu > time in the kernel indefinitely. Hey thanks for the report - can you try my branch? It's

Re: [PATCH] tests/bcachefs/single_device: test bcachefs format from source

2024-06-06 Thread Kent Overstreet
On Thu, Jun 06, 2024 at 06:28:04PM +0300, Ariel Miculas wrote: > Add a test which checks formatting and initializing a bcachefs > filesystem from a source directory: > `bcachefs format --source=source_dir` > > Create a new `prepare_fs` function shared by both `migrate_from_fs` and >

Re: [PATCH] bcachefs: fix smatch data leak warning in fs usage ioctl

2024-06-06 Thread Kent Overstreet
On Thu, Jun 06, 2024 at 09:58:26AM -0400, Brian Foster wrote: > smatch warns that the copy of arg to userspace is a potential data > leak by virtue of arg.pad not being checked or zeroed. This was > introduced by the commit referenced below that switched arg from > being a zeroed runtime

[GIT PULL] bcachefs fixes for 6.10-rc3

2024-06-05 Thread Kent Overstreet
xes. ---- Kent Overstreet (3): bcachefs: Fix GFP_KERNEL allocation in break_cycle() bcachefs: Rereplicate now moves data off of durability=0 devices bcachefs: Fix trans->locked assert fs/bcachefs/btree_locking.c | 1 + fs/bcachefs/move.c | 16 +++- 2 files chang

Re: memcpy failed when __bch2_accounting_mem_mod_slowpath

2024-06-03 Thread Kent Overstreet
4 09:16:44 2024] memcpy: detected field-spanning write (size 4) of > single field "r" at fs/bcachefs/disk_accounting.c:185 (size 3) > [Tue Jun 4 09:16:44 2024] WARNING: CPU: 6 PID: 11 at Thanks, this just needs an unsafe_memcpy() annotation commit 1c39379a4cddbf8d68c588ac

Re: [PATCH] bcachefs: bch2_btree_insert() - add btree iter flags

2024-06-03 Thread Kent Overstreet
On Mon, Jun 03, 2024 at 11:47:31PM +0300, Ariel Miculas wrote: > The commit 65bd44239727 ("bcachefs: bch2_btree_insert_trans() no longer > specifies BTREE_ITER_cached") removes BTREE_ITER_cached from > bch2_btree_insert_trans, which causes the update_inode function from > bcachefs-tools to take a

Re: [PATCH 0/3] introduce getversion, getfslabel, setfslabel ioctls

2024-06-03 Thread Kent Overstreet
On Mon, Jun 03, 2024 at 09:26:17PM +0800, Hongbo Li wrote: > GETVERSION: to support lsattr command. > GETFSLABEL: to get fslabel by ioctl. > SETFSLABEL: to set fslabel by ioctl. > > Hongbo Li (3): > bcachefs: implement FS_IOC_GETVERSION to support lsattr > bcachefs: support get fs label >

Re: [PATCH] bcachefs: fix the display format for show-super

2024-06-03 Thread Kent Overstreet
On Mon, Jun 03, 2024 at 09:23:35PM +0800, Hongbo Li wrote: > There are three keys displayed in non-uniform format. > Let's fix them. > > [Before] > ``` > Label:testbcachefs > Version: 1.9: (unknown version) > Version upgrade complete: 0.0: (unknown version) > ``` > > [After] >

Re: [PATCH] bcachefs: support REMAP_FILE_DEDUP in bch2_remap_file_range

2024-06-02 Thread Kent Overstreet
On Mon, Jun 03, 2024 at 12:05:36AM +, Reed Riley wrote: > Any chance of getting this patch merged upstream? > > I've been running with it on my system (and using the fideduperange > ioctl fairly regularly) for a couple of weeks now with no issues. Yep, applied

[PATCH 1/2] fork: Export create_io_thread()

2024-05-30 Thread Kent Overstreet
create_io_thread() creates a thread for doing work on behalf of some user process - export it so it can be used in thread_with_file, which spins up a kthread connected to a file descriptor. Cc: Jens Axboe Signed-off-by: Kent Overstreet --- kernel/fork.c | 1 + 1 file changed, 1 insertion

[PATCH 2/2] bcachefs: twf: Switch to create_io_thread()

2024-05-30 Thread Kent Overstreet
create_io_thread() is more appropriate than kthread_create() when spinning up a thread to do work asynchronously on behalf of some specific userspace task. Signed-off-by: Kent Overstreet --- fs/bcachefs/chardev.c | 3 +-- fs/bcachefs/thread_with_file.c | 29

[GIT PULL] bcachefs fixes for 6.10-rc2

2024-05-30 Thread Kent Overstreet
ux and has it retry the mount - fix failure to return error on misaligned dio write; reported as an issue with coreutils shred Jeff Johnson (1): bcachefs: add missing MODULE_DESCRIPTION() Kent Overstreet (19): bcachefs:

Re: [PATCH] bcachefs: fix uninitialized variable in bch2_remount()

2024-05-29 Thread Kent Overstreet
On Wed, May 29, 2024 at 07:38:14PM -0600, Thomas Bertschinger wrote: > Reported-by: kernel test robot > Closes: > https://lore.kernel.org/oe-kbuild-all/202405291826.785pvn5c-...@intel.com/ > Fixes: d6a5e45c75bb ("bcachefs: use new mount API") > Signed-off-by: Thomas Bertschinger folded in

[PATCH 2/2] bcachefs: bch2_fs_get_tree() cleanup

2024-05-29 Thread Kent Overstreet
- improve error paths - call bch2_fs_start() separately, after applying late-parsed options Signed-off-by: Kent Overstreet --- fs/bcachefs/fs.c | 59 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/fs/bcachefs/fs.c b/fs/bcachefs

[PATCH 1/2] bcachefs: Kill bch2_mount()

2024-05-29 Thread Kent Overstreet
Fold into bch2_fs_get_tree() Signed-off-by: Kent Overstreet --- fs/bcachefs/fs.c | 46 +++--- 1 file changed, 15 insertions(+), 31 deletions(-) diff --git a/fs/bcachefs/fs.c b/fs/bcachefs/fs.c index d3984f197d9f..61aba7d53194 100644 --- a/fs/bcachefs

Re: Use of zero-length arrays in bcachefs structures inner fields

2024-05-28 Thread Kent Overstreet
On Tue, May 28, 2024 at 01:36:11PM +0200, Alexander Potapenko wrote: > On Fri, May 24, 2024 at 7:30 PM Kent Overstreet > wrote: > > > > On Fri, May 24, 2024 at 12:04:11PM -0400, Mathieu Desnoyers wrote: > > > On 2024-05-24 11:35, Mathieu Desnoyers wrote: > >

Re: [GIT PULL] bcachefs updates fro 6.10-rc1

2024-05-28 Thread Kent Overstreet
On Tue, May 28, 2024 at 09:18:24AM +0200, Geert Uytterhoeven wrote: > Hi Kent, > > On Mon, May 20, 2024 at 4:39 AM Kees Cook wrote: > > On Sun, May 19, 2024 at 12:14:34PM -0400, Kent Overstreet wrote: > > > [...] > > > bcachefs changes for 6.10-rc1

Re: [PATCH v2] bcachefs: fix last_seq and last_empty_seq in bch2_fs_journal_start()

2024-05-27 Thread Kent Overstreet
On Mon, May 27, 2024 at 03:25:39PM -0400, Camila Alvarez Inostroza wrote: > Hello, > I was wondering if you had had the time to take a look at this. > Please let me know if there's something I'm missing. Sorry, I'll get to it - I'm going to need to stare at this one and try and figure out what

Re: [PATCH 1/2] bcachefs: make offline fsck set read_only fs flag

2024-05-26 Thread Kent Overstreet
On Sun, May 26, 2024 at 01:08:19PM -0600, Thomas Bertschinger wrote: > A subsequent change will remove "read_only" as a mount option in favor > of the standard option "ro", meaning the userspace fsck command cannot > pass it to the fsck ioctl. Instead, in offline fsck, set "read_only" >

Re: [PATCH] bcachefs: allow passing full device path for target options

2024-05-25 Thread Kent Overstreet
On Sat, May 25, 2024 at 01:36:19PM -0600, Thomas Bertschinger wrote: > The output of mount options such as "metadata_target" in `/proc/mounts` > uses the full path to the device. > > mount(8) from util-linux uses the output from `/proc/mounts` to pass > existing mount options when performing a

Re: [PATCH] bcachefs: Use copy_folio_from_iter_atomic()

2024-05-24 Thread Kent Overstreet
On Fri, May 24, 2024 at 08:46:25PM +0100, Matthew Wilcox (Oracle) wrote: > copy_page_from_iter_atomic() will be removed at some point. > Also fixup a comment for folios. > > Signed-off-by: Matthew Wilcox (Oracle) Applied > --- > fs/bcachefs/fs-io-buffered.c | 6 +++--- > 1 file changed, 3

Re: Use of zero-length arrays in bcachefs structures inner fields

2024-05-24 Thread Kent Overstreet
On Fri, May 24, 2024 at 12:04:11PM -0400, Mathieu Desnoyers wrote: > On 2024-05-24 11:35, Mathieu Desnoyers wrote: > > [ Adding clang/llvm and KMSAN maintainers/reviewers in CC. ] > > > > On 2024-05-24 11:28, Kent Overstreet wrote: > > > On Thu, May 23, 2024 at 01:53

Re: Use of zero-length arrays in bcachefs structures inner fields

2024-05-24 Thread Kent Overstreet
On Thu, May 23, 2024 at 01:53:42PM -0400, Mathieu Desnoyers wrote: > Hi Kent, > > Looking around in the bcachefs code for possible causes of this KMSAN > bug report: > > https://lore.kernel.org/lkml/fd5e7006191f7...@google.com/ > > I notice the following pattern in the bcachefs

[GIT PULL] bcachefs fixes for 6.10-rc1

2024-05-24 Thread Kent Overstreet
Kent Overstreet (17): bcachefs: Fix rcu splat in check_fix_ptrs() bcachefs: Fix ref in trans_mark_dev_sbs() error path bcachefs: Fix shift overflow in btree_lost_data() bcachefs: Fix shift overflows in replicas.c bcachefs: Improve

Re: [GIT PULL] bcachefs updates fro 6.10-rc1

2024-05-19 Thread Kent Overstreet
On Sun, May 19, 2024 at 07:39:38PM -0700, Kees Cook wrote: > On Sun, May 19, 2024 at 12:14:34PM -0400, Kent Overstreet wrote: > > [...] > > bcachefs changes for 6.10-rc1 > > [...] > > bcachefs: bch2_btree_path_to_text() > > Hi Kent, > > I've asked af

Re: [PATCH] guard against unknown r->data_type in bch2_sb_dev_has_data()

2024-05-19 Thread Kent Overstreet
ez Sorry I just now saw this, already applied a different fix - BCH_DATA_NR isn't the right guard here, because we need to be able to handle unknown data types for backwards compatibilicy. Here's my version: commit 85e8b546542c9a1f9050a7f53d92b8bd39c8e9d8 Author: Kent Overstreet Date: Sun

Re: [PATCH] bcachefs: fix last_seq and last_empty_seq in bch2_fs_journal_start()

2024-05-19 Thread Kent Overstreet
On Wed, May 15, 2024 at 11:19:20PM -0400, Camila Alvarez wrote: > Values were left as the next possible sequence number when there were no > entries. > > The fix involves updating the last_seq initial value and > setting last_empty_seq to cur_seq - 1. I think this is correct, but we should try

[GIT PULL] bcachefs updates fro 6.10-rc1

2024-05-19 Thread Kent Overstreet
mong other things talks about the correct usage of assertions Daniel Hill (1): bcachefs: add counters for failed shrinker reclaim Hongbo Li (1): bcachefs: eliminate the uninitialized compilation warning in bch2_reco

Re: [PATCH] bcachefs: guard against invalid bits_per_field in bch2_bkey_format_invalid

2024-05-09 Thread Kent Overstreet
bits_per_field, > bch2_max_bits_per_field[i]); > + return -BCH_ERR_invalid; > + } > + u64 packed_max = bits_per_field > + ? ~((~0ULL << 1) << (bits_per_field - 1)) >

Re: [PATCH v3 1/1] bcachefs: Move BCACHEFS_STATFS_MAGIC value to UAPI magic.h

2024-05-08 Thread Kent Overstreet
On Wed, May 08, 2024 at 11:17:59PM +0200, Petr Vorel wrote: > HI Su, Kent, > > > > On May 7, 2024, at 23:37, Petr Vorel wrote: > > > > Move BCACHEFS_STATFS_MAGIC value to UAPI under > > > BCACHEFS_SUPER_MAGIC definition (use common approach for name) and reuse > > > the > > > definition in

Re: [PATCH v3 1/1] bcachefs: Move BCACHEFS_STATFS_MAGIC value to UAPI magic.h

2024-05-07 Thread Kent Overstreet
On Tue, May 07, 2024 at 06:44:30PM +0200, Petr Vorel wrote: > > On Tue, May 07, 2024 at 05:37:57PM +0200, Petr Vorel wrote: > > > Move BCACHEFS_STATFS_MAGIC value to UAPI under > > > BCACHEFS_SUPER_MAGIC definition (use common approach for name) and reuse > > > the > > > definition in

Re: [PATCH] bcachefs: fix oob in bch2_sb_clean_to_text

2024-05-07 Thread Kent Overstreet
On Wed, May 08, 2024 at 09:11:37AM +0800, Edward Adam Davis wrote: > On Tue, 7 May 2024 20:59:14 -0400, Kent Overstreet wrote: > > > > diff --git a/fs/bcachefs/sb-clean.c b/fs/bcachefs/sb-clean.c > > > > index 35ca3f138de6..194e55b11137 100644 > > > > --- a/

Re: [PATCH] bcachefs: fix oob in bch2_sb_clean_to_text

2024-05-07 Thread Kent Overstreet
On Wed, May 08, 2024 at 08:49:39AM +0800, Edward Adam Davis wrote: > On Tue, 7 May 2024 10:14:22 -0400, Kent Overstreet wrote: > > > When got too small clean field, entry will never equal > > > vstruct_end(>field), > > > the dead loop resulted in out of

Re: [GIT PULL] bcachefs fixes for 6.9

2024-05-07 Thread Kent Overstreet
On Tue, May 07, 2024 at 11:10:44AM -0400, Kent Overstreet wrote: > On Tue, May 07, 2024 at 10:43:11AM -0400, Kent Overstreet wrote: > > Hi Linus, another stack of fixes for you. > > I have a report from a user that the sb-downgrade validation has an > issue, so that's been yank

Re: [GIT PULL] bcachefs fixes for 6.9

2024-05-07 Thread Kent Overstreet
On Tue, May 07, 2024 at 10:43:11AM -0400, Kent Overstreet wrote: > Hi Linus, another stack of fixes for you. I have a report from a user that the sb-downgrade validation has an issue, so that's been yanked. The following changes since commit c258c08add1cc8fa7719f112c5db36c08c507f1e: bcach

[GIT PULL] bcachefs fixes for 6.9

2024-05-07 Thread Kent Overstreet
dropping and retaking bch_dev->io_ref references ---- Kent Overstreet (20): bcachefs: Fix a scheduler splat in __bch2_next_write_buffer_flush_journal_buf() bcachefs: don't free error pointers bcach

Re: [LTP] [PATCH v2 3/3] statx04: Skip STATX_ATTR_COMPRESSED on Bcachefs

2024-05-07 Thread Kent Overstreet
On Tue, May 07, 2024 at 10:18:46AM +0200, Petr Vorel wrote: > > Hi! > > > > > Quite likely, other filesystems does have an inode flag that is set > > > > > when > > > > > file has been compressed and simply report that in the foo_getattr() > > > > > callback. Looking at bcachefs I supose that we

Re: [PATCH] bcachefs: fix oob in bch2_sb_clean_to_text

2024-05-07 Thread Kent Overstreet
to_text() needs to be safe for types that fail to validate. Reported-by: syzbot+c48865e11e7e893ec...@syzkaller.appspotmail.com Signed-off-by: Kent Overstreet diff --git a/fs/bcachefs/sb-clean.c b/fs/bcachefs/sb-clean.c index 35ca3f138de6..194e55b11137 100644 --- a/fs/bcachefs/sb-clea

Re: [PATCH] bcachefs: fix overflow in fiemap

2024-05-04 Thread Kent Overstreet
On Sat, May 04, 2024 at 10:12:23PM +, Reed Riley wrote: > filefrag (and potentially other utilities that call fiemap) sometimes > pass ULONG_MAX as the length. fiemap_prep clamps excessively large > lengths - but the calculation of end can overflow if it occurs before > calling fiemap_prep.

Re: syzbot for bcachefs?

2024-05-03 Thread Kent Overstreet
On Fri, May 03, 2024 at 06:31:30PM +0200, Aleksandr Nogikh wrote: > On Fri, May 3, 2024 at 5:36 PM Kent Overstreet > wrote: > > > > On Thu, May 02, 2024 at 10:28:42PM +0200, Aleksandr Nogikh wrote: > > > > > > I think the problem was that I was trying to sa

Re: syzbot for bcachefs?

2024-05-03 Thread Kent Overstreet
On Thu, May 02, 2024 at 10:28:42PM +0200, Aleksandr Nogikh wrote: > On Fri, Apr 19, 2024 at 7:19 PM Kent Overstreet > wrote: > > > > On Fri, Apr 19, 2024 at 10:35:27AM +0200, Dmitry Vyukov wrote: > > > > > > We've enabled the configs, but we don't have good im

Re: [PATCH] bcachefs: Fix error path of bch2_link_trans()

2024-04-29 Thread Kent Overstreet
On Tue, Apr 30, 2024 at 11:28:39AM +0800, Youling Tang wrote: > From: Youling Tang > > In bch2_link_trans(), if bch2_inode_nlink_inc() fails, it needs to > call bch2_trans_iter_exit() in the error path. Thanks - applied. > > Signed-off-by: Youling Tang > --- > fs/bcachefs/fs-common.c | 4

Re: [PATCH] bcachefs: Correct the FS_IOC_GETFLAGS to FS_IOC32_GETFLAGS in bch2_compat_fs_ioctl()

2024-04-29 Thread Kent Overstreet
On Tue, Apr 30, 2024 at 11:27:44AM +0800, Youling Tang wrote: > From: Youling Tang > > It should be FS_IOC32_GETFLAGS instead of FS_IOC_GETFLAGS in > compat ioctl. Do we by chance have a test for this? > > Signed-off-by: Youling Tang > --- > fs/bcachefs/fs-ioctl.c | 2 +- > 1 file changed,

[GIT PULL] bcachefs fixes for 6.9-rc7

2024-04-29 Thread Kent Overstreet
to fetch changes up to c258c08add1cc8fa7719f112c5db36c08c507f1e: bcachefs: fix integer conversion bug (2024-04-28 21:34:29 -0400) bcachefs fixes for 6.9-rc7 Kent

[PATCH] bcachefs: Developer guide

2024-04-29 Thread Kent Overstreet
Signed-off-by: Kent Overstreet --- .../filesystems/bcachefs/developer-guide.rst | 177 ++ Documentation/filesystems/bcachefs/index.rst | 1 + 2 files changed, 178 insertions(+) create mode 100644 Documentation/filesystems/bcachefs/developer-guide.rst diff --git

Re: [PATCH v2 1/2] bcachefs: eliminate the uninitialized compilation warning in bch2_reconstruct_snapshots

2024-04-25 Thread Kent Overstreet
On Fri, Apr 26, 2024 at 11:21:35AM +0800, Hongbo Li wrote: > When compiling the bcachefs-tools, the following compilation warning > is reported: > libbcachefs/snapshot.c: In function ‘bch2_reconstruct_snapshots’: > libbcachefs/snapshot.c:915:19: warning: ‘tree_id’ may be used >

Re: [PATCH v2 2/2] bcachefs: eliminate the uninitialized compilation warning in __do_six_trylock

2024-04-25 Thread Kent Overstreet
On Fri, Apr 26, 2024 at 11:21:36AM +0800, Hongbo Li wrote: > When compiling the bcachefs-tools, the following compilation warning > is reported: > libbcachefs/six.c: In function ‘__do_six_trylock’: > libbcachefs/six.c:90:12: warning: ‘old’ may be used uninitialized in this > function

Re: [PATCH 1/2] bcachefs: eliminate the uninitialized compilation warning in bch2_reconstruct_snapshots

2024-04-25 Thread Kent Overstreet
On Thu, Apr 25, 2024 at 11:55:02AM +0800, Hongbo Li wrote: > > > On 2024/4/25 11:34, Kent Overstreet wrote: > > On Fri, Apr 19, 2024 at 03:48:50PM +0800, Hongbo Li wrote: > > > [BUG] > > > When compiling the bcachefs-tools, the following comp

Re: [PATCH 2/2] bcachefs: eliminate the uninitialized compilation warning in __do_six_trylock

2024-04-24 Thread Kent Overstreet
On Fri, Apr 19, 2024 at 03:48:51PM +0800, Hongbo Li wrote: > When compiling the bcachefs-tools, the following compilation warning > is reported: > libbcachefs/six.c: In function ‘__do_six_trylock’: > libbcachefs/six.c:90:12: warning: ‘old’ may be used uninitialized in this > function

Re: [PATCH 1/2] bcachefs: eliminate the uninitialized compilation warning in bch2_reconstruct_snapshots

2024-04-24 Thread Kent Overstreet
On Fri, Apr 19, 2024 at 03:48:50PM +0800, Hongbo Li wrote: > [BUG] > When compiling the bcachefs-tools, the following compilation warning > is reported: > libbcachefs/snapshot.c: In function ‘bch2_reconstruct_snapshots’: > libbcachefs/snapshot.c:915:19: warning: ‘tree_id’ may be used >

Re: [PATCH] bcachefs: Fix format specifiers in bch2_btree_key_cache_to_text()

2024-04-23 Thread Kent Overstreet
On Tue, Apr 23, 2024 at 01:12:27PM -0700, Nathan Chancellor wrote: > When building for a 32-bit target, for which 'size_t' is 'unsigned int', > there are two warnings around mismatched format specifiers and argument > types: Thanks! Applied.

weekly bcachefs meeting, 1PM EST

2024-04-22 Thread Kent Overstreet
Shoot me an email if you'd like to come, and something about what you're interested in or would like to talk about. Good topics: - bugs you're working on, behaviour that needs explaining, or better introspection/debug tooling - short code walkthroughs - short, focused code review - design

[GIT PULL] bcachefs fixes fro 6.9-rc6

2024-04-22 Thread Kent Overstreet
, with more instrumentation coming so we can verify that it's working correctly more easily in the future Kent Overstreet (14): bcachefs: Fix null ptr deref in twf from BCH_IOCTL_FSCK_OFFLINE bcachefs: node scan: ignore multiple

Re: [PATCH] bcachefs: Fix format specifier in validate_bset_keys()

2024-04-22 Thread Kent Overstreet
On Mon, Apr 22, 2024 at 02:47:32PM +0200, Geert Uytterhoeven wrote: > Hi Kent, > > On Wed, Apr 17, 2024 at 12:09 AM Kent Overstreet > wrote: > > On Tue, Apr 16, 2024 at 08:16:02AM -0700, Nathan Chancellor wrote: > > > When building for 32-bit platforms, for wh

[PATCH] bcachefs: idle work scheduling design doc

2024-04-20 Thread Kent Overstreet
-by: Kent Overstreet --- fs/bcachefs/idle.h | 80 ++ 1 file changed, 80 insertions(+) create mode 100644 fs/bcachefs/idle.h diff --git a/fs/bcachefs/idle.h b/fs/bcachefs/idle.h new file mode 100644 index ..b1717635581e --- /dev/null +++ b/fs

Re: [PATCH 06/30] bcachefs: Remove calls to folio_set_error

2024-04-19 Thread Kent Overstreet
On Sat, Apr 20, 2024 at 03:50:01AM +0100, Matthew Wilcox (Oracle) wrote: > Common code doesn't test the error flag, so we don't need to set it in > bcachefs. We can use folio_end_read() to combine the setting (or not) > of the uptodate flag and clearing the lock flag. > > Cc:

Re: [PATCH v3 4/4] bcachefs: add fiemap delalloc extent detection

2024-04-19 Thread Kent Overstreet
On Thu, Apr 18, 2024 at 10:46:21AM -0400, Brian Foster wrote: > So Kent had pointed out in an earlier chat that this needs to be a > little more robust wrt to transaction restart because there is the > potential for spinning under folio lock contention. I think there are a > couple ways to do this

Re: syzbot for bcachefs?

2024-04-19 Thread Kent Overstreet
On Fri, Apr 19, 2024 at 10:35:27AM +0200, Dmitry Vyukov wrote: > On Fri, 5 Apr 2024 at 18:20, Aleksandr Nogikh wrote: > > > > On Fri, Apr 5, 2024 at 1:16 AM Kent Overstreet > > wrote: > > > > > > On Thu, Apr 04, 2024 at 10:33:07AM +0200, Dmitry Vyukov wro

Re: [PATCH] bcachefs: Align the display format of `btrees/inodes/keys`

2024-04-17 Thread Kent Overstreet
On Wed, Apr 17, 2024 at 05:49:41PM +0800, Hongbo Li wrote: > > > On 2024/4/17 13:51, Youling Tang wrote: > > Hi, Kent & Hongbo > > On 17/04/2024 11:21, Hongbo Li wrote: > > > > > > > > > On 2024/4/17 11:16, Hongbo Li wrote: > > >

Re: [PATCH] bcachefs: Align the display format of `btrees/inodes/keys`

2024-04-16 Thread Kent Overstreet
On Wed, Apr 17, 2024 at 10:50:10AM +0800, Youling Tang wrote: > Hi, Kent > On 17/04/2024 10:20, Kent Overstreet wrote: > > On Wed, Apr 17, 2024 at 09:54:48AM +0800, Youling Tang wrote: > > > From: Youling Tang > > > > > > Before patch: > > > ```

Re: [PATCH] bcachefs: Align the display format of `btrees/inodes/keys`

2024-04-16 Thread Kent Overstreet
On Wed, Apr 17, 2024 at 09:54:48AM +0800, Youling Tang wrote: > From: Youling Tang > > Before patch: > ``` > #cat btrees/inodes/keys > u64s 17 type inode_v3 0:4096:U32_MAX len 0 ver 0: mode=40755 >flags= (1630) > ``` > > After patch: > ``` > #cat btrees/inodes/keys > u64s 17 type

Re: [PATCH] bcachefs: Fix format specifier in validate_bset_keys()

2024-04-16 Thread Kent Overstreet
On Tue, Apr 16, 2024 at 08:16:02AM -0700, Nathan Chancellor wrote: > When building for 32-bit platforms, for which size_t is 'unsigned int', > there is a warning from a format string in validate_bset_keys(): > > fs/bcachefs/btree_io.c: In function 'validate_bset_keys': >

Re: debugfs revoke broken?

2024-04-15 Thread Kent Overstreet
On Mon, Apr 15, 2024 at 10:47:56AM +0200, Greg Kroah-Hartman wrote: > So this file was open when debugfs_remove() was called? Seems likely. > Any chance you can bisect? We just fixed some issues here in > 952c3fce297f ("debugfs: fix wait/cancellation handling during remove") > that I thought

Re: [PATCH] bcachefs: bch_member.btree_allocated_bitmap

2024-04-14 Thread Kent Overstreet
On Mon, Apr 15, 2024 at 09:12:11AM +0800, Hongbo Li wrote: > > > On 2024/4/14 5:57, Kent Overstreet wrote: > > For those interested, this is a good tour of the facilities for rolling > > out new on disk format features. > > > > -- >8 -- > > > &

debugfs revoke broken?

2024-04-14 Thread Kent Overstreet
I recently started seeing test failures like the following; from the "shutdown complete" line we're well after the point where we called debugfs_remove_recursive() - yet from the backtrace we were still able to call into debugfs. And I see from the history the remove path has been getting

[PATCH] bcachefs: bch_member.btree_allocated_bitmap

2024-04-13 Thread Kent Overstreet
map is present - Upgrade table lists the required recovery pass and expected fsck error - Btree node scan uses the bitmap to skip ranges if we're on the new version Signed-off-by: Kent Overstreet --- fs/bcachefs/bcachefs_format.h | 7 ++-- fs/bcachefs/btree_gc.c |

Re: [PATCH v2] bcachefs: chardev: make bch_chardev_class constant

2024-04-11 Thread Kent Overstreet
On Fri, Mar 08, 2024 at 09:12:47AM -0300, Ricardo B. Marliere wrote: > Since commit 43a7206b0963 ("driver core: class: make class_register() take > a const *"), the driver core allows for struct class to be in read-only > memory, so move the bch_chardev_class structure to be declared at build >

Re: [PATCH RFC v3 for-6.8/block 07/17] bcachefs: remove dead function bdev_sectors()

2024-04-11 Thread Kent Overstreet
On Thu, Dec 21, 2023 at 04:57:02PM +0800, Yu Kuai wrote: > From: Yu Kuai > > bdev_sectors() is not used hence remove it. > > Signed-off-by: Yu Kuai Acked-by: Kent Overstreet > --- > fs/bcachefs/util.h | 5 - > 1 file changed, 5 deletions(-) > > diff -

Re: [PATCH] bcachefs: fix typo in reference to BCACHEFS_DEBUG

2024-04-11 Thread Kent Overstreet
On Thu, Apr 11, 2024 at 05:21:05PM +0800, Hongbo Li wrote: > Also, I am curious why such spelling errors cannot be detected at the > compilation stage. code coverage analysis would catch this. I'm one patch away from fully automated code coverage.

[GIT PULL] bcachefs fixes for rc4

2024-04-10 Thread Kent Overstreet
): bcachefs: fix ! vs ~ typo in __clear_bit_le64() Hongbo Li (1): bcachefs: fix the count of nr_freed_pcpu after changing bc->freed_nonpcpu list Kent Overstreet (19): bcachefs: Make snapshot_is_ancestor() safe bcachefs: Bump limit in btree_trans_too_many_iters() bcach

Re: [PATCH RFC 0/3] xfs: nodataio mount option to skip data I/O

2024-04-10 Thread Kent Overstreet
On Wed, Apr 10, 2024 at 10:09:53AM -0400, Brian Foster wrote: > Hi all, > > bcachefs has a nodataio mount option that is used for isolated metadata > performance testing purposes. When enabled, it performs all metadata I/O > as normal and shortcuts data I/O by directly invoking bio completion. >

Re: [PATCH] bcachefs: Optimize eytzinger0_sort() with bottom-up heapsort

2024-04-06 Thread Kent Overstreet
On Sun, Apr 07, 2024 at 09:33:49AM +0800, Kuan-Wei Chiu wrote: > This optimization reduces the average number of comparisons required > from 2*n*log2(n) - 3*n + o(n) to n*log2(n) + 0.37*n + o(n). When n is > sufficiently large, it results in approximately 50% fewer comparisons. > > Currently,

Re: [PATCH] bcachefs: Rename struct field swap to prevent macro naming collision

2024-04-06 Thread Kent Overstreet
On Sat, Apr 06, 2024 at 11:21:37PM +0200, Thorsten Blum wrote: > On 6. Apr 2024, at 20:48, Kent Overstreet wrote: > > > > On Sat, Apr 06, 2024 at 04:19:20PM +0200, Thorsten Blum wrote: > >> The struct field swap can collide with the swap() macro defined in > >>

Re: [PATCH] bcachefs: Rename struct field swap to prevent macro naming collision

2024-04-06 Thread Kent Overstreet
On Sat, Apr 06, 2024 at 04:19:20PM +0200, Thorsten Blum wrote: > The struct field swap can collide with the swap() macro defined in > linux/minmax.h. Rename the struct field to prevent such collisions. Same as in lib/sort.c, so what's the actual reason?

Re: [PATCH 0/2] toctree fix for bcachefs docs

2024-04-05 Thread Kent Overstreet
On Fri, Apr 05, 2024 at 02:23:17PM +0700, Bagas Sanjaya wrote: > Hi Kent, > > Building htmldocs for current Linus's tree triggers a new warning, and > that is on your docs (missing toctree). Here's the fix + MAINTAINERS > entry for bcachefs docs. Thanks! Applied.

Re: [PATCH v3 13/13] bcachefs: fiemap: emit new COMPRESSED state

2024-04-05 Thread Kent Overstreet
On Fri, Apr 05, 2024 at 01:17:45PM -0600, Andreas Dilger wrote: > On Apr 3, 2024, at 1:22 AM, Sweet Tea Dorminy > wrote: > > > > Signed-off-by: Sweet Tea Dorminy > > --- > > fs/bcachefs/fs.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/fs/bcachefs/fs.c

Re: [PATCH v3 02/13] fiemap: update fiemap_fill_next_extent() signature

2024-04-05 Thread Kent Overstreet
On Fri, Apr 05, 2024 at 01:05:01PM -0600, Andreas Dilger wrote: > On Apr 3, 2024, at 1:22 AM, Sweet Tea Dorminy > wrote: > > > > Update the signature of fiemap_fill_next_extent() to allow passing a > > physical length. Update all callers to pass a 0 physical length -- since > > none set the

Re: [PATCH] bcachefs: fix ! vs ~ typo in __clear_bit_le64()

2024-04-05 Thread Kent Overstreet
On Fri, Apr 05, 2024 at 06:01:02PM +0300, Dan Carpenter wrote: > The ! was obviously intended to be ~. As it is, this function does > the equivalent to: "addr[bit / 64] = 0;". > > Fixes: 27fcec6c27ca ("bcachefs: Clear recovery_passes_required as they > complete without errors") > Signed-off-by:

Re: syzbot for bcachefs?

2024-04-04 Thread Kent Overstreet
On Thu, Apr 04, 2024 at 10:33:07AM +0200, Dmitry Vyukov wrote: > On Thu, 4 Apr 2024 at 01:59, Kent Overstreet > wrote: > > > > Hi - what would it take to get syzbot going for bcachefs? > > Hi Kent, > > There are 2 parts: > > The easy one: we need to en

[GIT PULL] bcachefs repair code for rc3

2024-04-03 Thread Kent Overstreet
. Guenter Roeck (1): mean_and_variance: Drop always failing tests Kent Overstreet (18): bcachefs: Fix btree node reserve bcachefs: BCH_WATERMARK_interior_updates bcachefs: fix nocow lock deadlock bcachefs

Re: [PATCH v3 00/13] fiemap extension for more physical information

2024-04-03 Thread Kent Overstreet
On Wed, Apr 03, 2024 at 03:22:41AM -0400, Sweet Tea Dorminy wrote: > For many years, various btrfs users have written programs to discover > the actual disk space used by files, using root-only interfaces. > However, this information is a great fit for fiemap: it is inherently > tied to extent

Re: [PATCH v3 11/13] bcachefs: fiemap: return correct extent physical length

2024-04-03 Thread Kent Overstreet
On Wed, Apr 03, 2024 at 01:00:11PM -0400, Brian Foster wrote: > On Wed, Apr 03, 2024 at 03:22:52AM -0400, Sweet Tea Dorminy wrote: > > Signed-off-by: Sweet Tea Dorminy > > --- > > fs/bcachefs/fs.c | 18 -- > > 1 file changed, 12 insertions(+), 6 deletions(-) > > > > diff --git

[GIT PULL] bcachefs fixes for 6.9-rc3

2024-04-01 Thread Kent Overstreet
s a result of the splitbrain bug will be able to repair then. Hongbo Li (1): bcachefs: fix trans->mem realloc in __bch2_trans_kmalloc Kent Overstreet (29): bcachefs: Fix assert in bch2_backpointer_invalid()

Re: [PATCH] bcachefs: add REQ_SYNC and REQ_IDLE in write dio

2024-03-26 Thread Kent Overstreet
On Tue, Mar 26, 2024 at 08:03:45PM +0800, zhuxiaohui wrote: > when writing file with direct_IO on bcachefs, then performance is > much lower than other fs due to write back throttle in block layer: > > wbt_wait+1 > __rq_qos_throttle+32 > blk_mq_submit_bio+394 >

  1   2   3   4   5   6   >