Re: [PATCH 0/4][V4] drop the mmap_sem when doing IO in the fault path

2018-12-06 Thread Dave Chinner
do a copy-on-write operation to break physical data sharing and so the page with the file data in it physically changes during ->page_mkwrite (because DAX). Hence we need to restart the page fault to map the new page correctly because the file no longer points at the page that was originally faulted. With this stashed-page-and-retry mechanism implemented for ->page_mkwrite, we could stash the new page in the vmf and tell the fault to retry, and everything would just work. Without ->page_mkwrite support, it's just not that interesting and I have higher priority things to deal with right now Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH AUTOSEL 4.14 25/35] iomap: sub-block dio needs to zeroout beyond EOF

2018-12-02 Thread Dave Chinner
On Sat, Dec 01, 2018 at 02:49:09AM -0500, Sasha Levin wrote: > On Sat, Dec 01, 2018 at 08:50:05AM +1100, Dave Chinner wrote: > >On Fri, Nov 30, 2018 at 05:14:41AM -0500, Sasha Levin wrote: > >>On Fri, Nov 30, 2018 at 09:22:03AM +0100, Greg KH wrote: > >>>On Fri, No

Re: [PATCH AUTOSEL 4.14 25/35] iomap: sub-block dio needs to zeroout beyond EOF

2018-11-30 Thread Dave Chinner
On Fri, Nov 30, 2018 at 05:14:41AM -0500, Sasha Levin wrote: > On Fri, Nov 30, 2018 at 09:22:03AM +0100, Greg KH wrote: > >On Fri, Nov 30, 2018 at 09:40:19AM +1100, Dave Chinner wrote: > >>I stopped my tests at 5 billion ops yesterday (i.e. 20 billion ops > >>aggrega

Re: [PATCH AUTOSEL 4.14 25/35] iomap: sub-block dio needs to zeroout beyond EOF

2018-11-30 Thread Dave Chinner
On Fri, Nov 30, 2018 at 09:22:03AM +0100, Greg KH wrote: > On Fri, Nov 30, 2018 at 09:40:19AM +1100, Dave Chinner wrote: > > On Thu, Nov 29, 2018 at 01:47:56PM +0100, Greg KH wrote: > > > On Thu, Nov 29, 2018 at 11:14:59PM +1100, Dave Chinner wrote: > > > > > >

Re: [PATCH AUTOSEL 4.14 25/35] iomap: sub-block dio needs to zeroout beyond EOF

2018-11-29 Thread Dave Chinner
On Thu, Nov 29, 2018 at 01:47:56PM +0100, Greg KH wrote: > On Thu, Nov 29, 2018 at 11:14:59PM +1100, Dave Chinner wrote: > > > > Cherry picking only one of the 50-odd patches we've committed into > > late 4.19 and 4.20 kernels to fix the problems we've found r

Re: [PATCH AUTOSEL 4.19 52/68] iomap: sub-block dio needs to zeroout beyond EOF

2018-11-29 Thread Dave Chinner
ient to regression test fixes that, in some cases, took hundreds of millions of fsx ops to expose. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH AUTOSEL 4.19 52/68] iomap: sub-block dio needs to zeroout beyond EOF

2018-11-29 Thread Dave Chinner
On Thu, Nov 29, 2018 at 12:55:43AM -0500, Sasha Levin wrote: > From: Dave Chinner > > [ Upstream commit b450672fb66b4a991a5b55ee24209ac7ae7690ce ] > > If we are doing sub-block dio that extends EOF, we need to zero > the unused tail of the block to initialise the data in it

Re: [PATCH AUTOSEL 4.14 25/35] iomap: sub-block dio needs to zeroout beyond EOF

2018-11-29 Thread Dave Chinner
On Thu, Nov 29, 2018 at 01:00:59AM -0500, Sasha Levin wrote: > From: Dave Chinner > > [ Upstream commit b450672fb66b4a991a5b55ee24209ac7ae7690ce ] > > If we are doing sub-block dio that extends EOF, we need to zero > the unused tail of the block to initialise the data in it

Re: [PATCH] xfs: Remove noinline from #define STATIC

2018-11-12 Thread Dave Chinner
On Mon, Nov 12, 2018 at 08:23:42PM -0800, Joe Perches wrote: > On Tue, 2018-11-13 at 14:09 +1100, Dave Chinner wrote: > > On Mon, Nov 12, 2018 at 08:54:10PM -0500, Theodore Y. Ts'o wrote: > > > On Tue, Nov 13, 2018 at 12:18:05PM +1100, Dave Chinner wrote: > > > >

Re: [PATCH] xfs: Remove noinline from #define STATIC

2018-11-12 Thread Dave Chinner
On Mon, Nov 12, 2018 at 08:54:10PM -0500, Theodore Y. Ts'o wrote: > On Tue, Nov 13, 2018 at 12:18:05PM +1100, Dave Chinner wrote: > > I'm not interested in making code fast if distro support engineers > > can't debug problems on user systems easily. Optim

Re: [PATCH] xfs: Remove noinline from #define STATIC

2018-11-12 Thread Dave Chinner
On Mon, Nov 12, 2018 at 02:30:01PM -0800, Joe Perches wrote: > On Tue, 2018-11-13 at 08:45 +1100, Dave Chinner wrote: > > On Mon, Nov 12, 2018 at 02:12:08PM -0600, Eric Sandeen wrote: > > > On 11/10/18 7:21 PM, Joe Perches wrote: > > > > Reduce total object size quit

Re: [PATCH] xfs: Remove noinline from #define STATIC

2018-11-12 Thread Dave Chinner
stack traces. It flattens them way too much to be able to tell how we got to a specific location in the code. In reality, being able to find problems quickly and efficiently is far more important to us than being able to run everything at ludicrous speed Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH] Fix coding style issue in xfs_acl.c and xfs_aops.c

2018-11-11 Thread Dave Chinner
columns is preferred" but they are wrong. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH] fs/proc: introduce /proc/stat2 file

2018-11-07 Thread Dave Chinner
sfs, if not already available. This doesn't solve the problem. The problem is that this specific implementation of per-cpu counters need to be summed on every read. Hence when you have a huge number of CPUs each per-cpu iteration that takes a substantial amount of time. If only we had percpu counters that had a fixed, extremely low read overhead that doesn't care about the number of CPUs in the machine Oh, wait, we do: percpu_counters.[ch]. This all seems like a counter implementation deficiency to me, not an interface problem... Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-06 Thread Dave Chinner
On Tue, Nov 06, 2018 at 12:00:06PM +0100, Jan Kara wrote: > On Tue 06-11-18 13:47:15, Dave Chinner wrote: > > On Mon, Nov 05, 2018 at 04:26:04PM -0800, John Hubbard wrote: > > > On 11/5/18 1:54 AM, Jan Kara wrote: > > > > Hmm, have you tried larger buffer sizes? B

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-11-05 Thread Dave Chinner
every IO? I'd argue that the IO latency impact is far worse than the a 20% throughput drop. i.e. You can make up for throughput drops by running a deeper queue/more dispatch threads, but you can't reduce IO latency at all... Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [GIT PULL] vfs: fix many problems in vfs clone/dedupe implementation

2018-11-04 Thread Dave Chinner
On Sat, Nov 03, 2018 at 10:13:37AM -0700, Linus Torvalds wrote: > On Fri, Nov 2, 2018 at 4:36 PM Dave Chinner wrote: > > > > On Fri, Nov 02, 2018 at 09:35:23AM -0700, Linus Torvalds wrote: > > > > > > I don't love the timing of this at the end of the merge w

Re: [GIT PULL] vfs: fix many problems in vfs clone/dedupe implementation

2018-11-02 Thread Dave Chinner
On Fri, Nov 02, 2018 at 09:35:23AM -0700, Linus Torvalds wrote: > On Thu, Nov 1, 2018 at 10:15 PM Dave Chinner wrote: > > > > Can you please pull update containing a rework of the VFS clone and > > dedupe file range infrastructure from the tag listed below? > > I don&#

[GIT PULL] vfs: fix many problems in vfs clone/dedupe implementation

2018-11-01 Thread Dave Chinner
/xfs/xfs_reflink.h | 15 +- include/linux/fs.h| 55 -- mm/filemap.c | 146 +++--- 20 files changed, 734 insertions(+), 596 deletions(-) -- Dave Chinner da...@fromorbit.com

Re: linux-next: manual merge of the vfs tree with the xfs tree

2018-10-30 Thread Dave Chinner
ream maintainer when your tree > is submitted for merging. You may also want to consider cooperating > with the maintainer of the conflicting tree to minimise any particularly > complex conflicts. Looks ok. I didn't expect this conflict, but looks simple enough to resolve. Thanks! Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: linux-next: manual merge of the xfs tree with Linus' tree

2018-10-30 Thread Dave Chinner
replaced with > + ->remap_file_range(). See Documentation/filesystems/vfs.txt for more > + information. Looks good - I knew about this one from merging back into a recent Linus kernel. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: XFS: Hang and dmesg flood on mounting invalid FS image

2018-10-29 Thread Dave Chinner
ot;we didn't validate the input properly". Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: XFS: Hang and dmesg flood on mounting invalid FS image

2018-10-28 Thread Dave Chinner
cycle is not valid. And that's the problem. Neither the head or tail blocks are validated before they are used. CRC checking of the head and tail blocks comes later Cheers, Dave. -- Dave Chinner da...@fromorbit.com

[GIT PULL] xfs: 4.20 merge

2018-10-23 Thread Dave Chinner
e management in xrep_findroot_block Dave Chinner (2): xfs: issue log message on user force shutdown xfs: fix use-after-free race in xfs_buf_rele fs/xfs/libxfs/xfs_attr.c | 236 - fs/xfs/{ => libxfs}/xfs_attr.h | 2 + fs/xfs/libxfs/xfs_

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-13 Thread Dave Chinner
On Sat, Oct 13, 2018 at 12:34:12AM -0700, John Hubbard wrote: > On 10/12/18 8:55 PM, Dave Chinner wrote: > > On Thu, Oct 11, 2018 at 11:00:12PM -0700, john.hubb...@gmail.com wrote: > >> From: John Hubbard > [...] > >> diff --git a/include/linux/mm_types.h b/inclu

Re: [PATCH 4/6] mm: introduce page->dma_pinned_flags, _count

2018-10-12 Thread Dave Chinner
r to read/write direct IO and so the pages passed to gup will be on the active/inactive LRUs. hence I can't see how you can have dual use of the LRU list head like this What am I missing here? Cheers, Dave. -- Dave Chinner da...@fromorbit.com

[GIT PULL] xfs: fixes for v4.19-rc7

2018-10-10 Thread Dave Chinner
xfs: zero posteof blocks when cloning above eof xfs: update ctime and remove suid before cloning files Dave Chinner (2): xfs: fix data corruption w/ unaligned dedupe ranges xfs: fix data corruption w/ unaligned reflink ranges fs/xfs/xfs_reflink.c | 200

Re: [PATCH] writeback: fix range_cyclic writeback vs writepages deadlock

2018-10-05 Thread Dave Chinner
On Fri, Oct 05, 2018 at 12:46:40PM -0700, Andrew Morton wrote: > On Fri, 5 Oct 2018 15:45:26 +1000 Dave Chinner wrote: > > > From: Dave Chinner > > > > We've recently seen a workload on XFS filesystems with a repeatable > > deadlock between backg

[PATCH] writeback: fix range_cyclic writeback vs writepages deadlock

2018-10-04 Thread Dave Chinner
From: Dave Chinner We've recently seen a workload on XFS filesystems with a repeatable deadlock between background writeback and a multi-process application doing concurrent writes and fsyncs to a small range of a file. range_cyclic writeback Process 1 Proc

[GIT PULL] xfs: fixes for 4.19-rc6

2018-10-03 Thread Dave Chinner
xfs: don't bring in extents in xfs_bmap_punch_delalloc_range xfs: skip delalloc COW blocks in xfs_reflink_end_cow Darrick J. Wong (1): xfs: don't crash the vfs on a garbage inline symlink Dave Chinner (3): xfs: avoid lockdep false positives in xfs_trans_alloc xfs: fix

Re: Leaking Path in XFS's ioctl interface(missing LSM check)

2018-10-02 Thread Dave Chinner
On Wed, Oct 03, 2018 at 05:20:31AM +1000, James Morris wrote: > On Tue, 2 Oct 2018, Dave Chinner wrote: > > > On Tue, Oct 02, 2018 at 06:08:16AM +1000, James Morris wrote: > > > On Mon, 1 Oct 2018, Darrick J. Wong wrote: > > > > > > > If we /did/ replace

Re: [PATCH 0/4] get_user_pages*() and RDMA: first steps

2018-10-01 Thread Dave Chinner
On Mon, Oct 01, 2018 at 05:47:57AM -0700, Christoph Hellwig wrote: > On Mon, Oct 01, 2018 at 04:11:27PM +1000, Dave Chinner wrote: > > This reminds me so much of Linux mmap() in the mid-2000s - mmap() > > worked for ext3 without being aware of page faults, > > And "w

Re: Leaking Path in XFS's ioctl interface(missing LSM check)

2018-10-01 Thread Dave Chinner
st in XFS So do I - the "in root we trust" model is pretty deeply ingrained up and down the storage stack. I also suspect that most of our hardware admin (not just storage) has similar assumptions about the security model they operate in. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: Leaking Path in XFS's ioctl interface(missing LSM check)

2018-10-01 Thread Dave Chinner
orage ioctls under CAP_SYS_STORAGE_ADMIN? Maybe I'm missing something, but I don't see how that improves the situation w.r.t. locked down LSM configurations? Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH 0/3] namei: implement various scoping AT_* flags

2018-09-30 Thread Dave Chinner
On Mon, Oct 01, 2018 at 03:47:23PM +1000, Aleksa Sarai wrote: > On 2018-10-01, Dave Chinner wrote: > > > I've added some selftests for this, but it's not clear to me whether > > > they should live here or in xfstests (as far as I can tell there are no > >

Re: [PATCH 0/4] get_user_pages*() and RDMA: first steps

2018-09-30 Thread Dave Chinner
is leaves the RDMA app with direct access to the physical storage even though the filesystem has freed the space it is accessing. This is a use after free of the physical storage that the filesystem cannot control, and why DAX+RDMA is disabled right now. We could address these use-a

Re: [PATCH 0/3] namei: implement various scoping AT_* flags

2018-09-30 Thread Dave Chinner
like > generic VFS tests in xfstests). If you'd prefer them to be included in > xfstests, let me know. xfstests, please. That way the new functionality will get immediate coverage by all the main filesystem development and distro QA teams Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: Leaking Path in XFS's ioctl interface(missing LSM check)

2018-09-30 Thread Dave Chinner
can be mounted by untrusted users (i.e. FUSE) have a special flag in their fstype definition to say this is allowed. Systems restricted by LSMs to the point where CAP_SYS_ADMIN is not trusted have exactly the same issues. i.e. there's nobody trusted by the kernel to administer the storage stack, and nobody has defined a workable security model that can prevent untrusted users from violating the existing storage trust model Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: Leaking Path in XFS's ioctl interface(missing LSM check)

2018-09-27 Thread Dave Chinner
On Fri, Sep 28, 2018 at 07:23:42AM +1000, James Morris wrote: > On Thu, 27 Sep 2018, Dave Chinner wrote: > > > Sure, but there are so many CAP_SYS_ADMIN-only ioctls in the kernel > > that have no LSM coverage that this is not an isolated problem that > > people setting

Re: Leaking Path in XFS's ioctl interface(missing LSM check)

2018-09-26 Thread Dave Chinner
On Wed, Sep 26, 2018 at 09:23:03AM -0400, Stephen Smalley wrote: > On 09/25/2018 09:33 PM, Dave Chinner wrote: > >On Tue, Sep 25, 2018 at 08:51:50PM -0400, TongZhang wrote: > >>Hi, > >> > >>I'm bringing up this issue again to let of LSM developers know t

Re: Leaking Path in XFS's ioctl interface(missing LSM check)

2018-09-26 Thread Dave Chinner
On Wed, Sep 26, 2018 at 07:24:26PM +0100, Alan Cox wrote: > On Wed, 26 Sep 2018 11:33:29 +1000 > Dave Chinner wrote: > > > On Tue, Sep 25, 2018 at 08:51:50PM -0400, TongZhang wrote: > > > Hi, > > > > > > I'm bringing up this issue again to let o

Re: Leaking Path in XFS's ioctl interface(missing LSM check)

2018-09-25 Thread Dave Chinner
are essentially internal filesystem interfaces used by trusted code and not general user application facing APIs. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH 3.16 52/63] xfs: validate cached inodes are free when allocated

2018-09-21 Thread Dave Chinner
On Sat, Sep 22, 2018 at 01:15:42AM +0100, Ben Hutchings wrote: > 3.16.58-rc1 review patch. If anyone has any objections, please let me know. > > -- > > From: Dave Chinner > > commit afca6c5b2595fc44383919fba740c194b0b76aff upstream. > > A recent fuz

Re: [PATCH 3.16 51/63] xfs: catch inode allocation state mismatch corruption

2018-09-21 Thread Dave Chinner
On Sat, Sep 22, 2018 at 01:15:42AM +0100, Ben Hutchings wrote: > 3.16.58-rc1 review patch. If anyone has any objections, please let me know. > > -- > > From: Dave Chinner > > commit ee457001ed6c6f31ddad69c24c1da8f377d8472d upstream. > > We recently

Re: metadata operation reordering regards to crash

2018-09-14 Thread Dave Chinner
metadata consistency is ensured after a crash. > Thus, B is either the original B(or not exists) or has been replaced by A. > The same to D. > > Is it possible that, after a crash, D has been replaced by C but B is still > the original file(or not exists)? Not for XFS, ext4, btr

Re: [PATCH 2/2] fs: list init in the critical section

2018-09-10 Thread Dave Chinner
On Mon, Sep 10, 2018 at 05:09:52AM -0700, swkhack wrote: > the i_state init in the critical section,so as the list init should in it. Why? What bug does this fix? -Dave. -- Dave Chinner da...@fromorbit.com

Re: POSIX violation by writeback error

2018-09-05 Thread Dave Chinner
fsync operations for every physical disk IO here, as opposed to 2 IOs per fsync in the above case. Put simply, the assumption that applications can't do more flush/fsync operations than disk IOs is not valid, and that performance of open-write-flush-close workloads on modern filesystems isn't anywhere near as bad as you think it is. To mangle a common saying into storage speak: "Caches are for show, IO is for go" Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: POSIX violation by writeback error

2018-09-04 Thread Dave Chinner
vial though... > > > > Here's a crazy and potentially stupid idea: > > Implement a new class of swap space for backing dirty pages which fail > to write back. Pages in this space survive reboots, essentially backing > the implicit commitment POSIX establishes in the face of asynchronous > writeback errors. Rather than evicting these pages as clean, they are > swapped out to the persistent swap. And when that "swap" area gets write errors, too? What then? We're straight back to the same "what the hell do we do with the error" problem. Adding more turtles doesn't help solve this issue. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH v2 2/3] xfs: Prevent multiple wakeups of the same log space waiter

2018-08-27 Thread Dave Chinner
On Mon, Aug 27, 2018 at 11:34:13AM -0400, Waiman Long wrote: > On 08/26/2018 08:21 PM, Dave Chinner wrote: > > On Sun, Aug 26, 2018 at 04:53:14PM -0400, Waiman Long wrote: > >> The current log space reservation code allows multiple wakeups of the > >> same sleeping waite

Re: [PATCH v2 2/3] xfs: Prevent multiple wakeups of the same log space waiter

2018-08-27 Thread Dave Chinner
On Mon, Aug 27, 2018 at 12:39:06AM -0700, Christoph Hellwig wrote: > On Mon, Aug 27, 2018 at 10:21:34AM +1000, Dave Chinner wrote: > > tl; dr: Once you pass a certain point, ramdisks can be *much* slower > > than SSDs on journal intensive workloads like AIM7. Hence it would be >

Re: [PATCH v2 2/3] xfs: Prevent multiple wakeups of the same log space waiter

2018-08-26 Thread Dave Chinner
eup contention go away? Can you please test both of these things and report the results so we can properly evaluate the impact of these changes? Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH v2 0/3] xfs: Reduce spinlock contention in log space slowpath code

2018-08-26 Thread Dave Chinner
On Sun, Aug 26, 2018 at 04:53:12PM -0400, Waiman Long wrote: > v1->v2: > - For patch 1, remove wake_q_empty() & add task_in_wake_q(). > - Rewrite patch 2 after comments from Dave Chinner and break it down >to 2 separate patches. Now the original xfs logic was kept. The &g

Re: [PATCH 2/2] xfs: Use wake_q for waking up log space waiters

2018-08-23 Thread Dave Chinner
areful(&log->l_reserve_head.waiters)) { > @@ -1086,8 +1110,10 @@ > > spin_lock(&log->l_reserve_head.lock); > free_bytes = xlog_space_left(log, &log->l_reserve_head.grant); > - xlog_grant_head_wake(log, &log->l_reserve_head, &free_bytes); > + xlog_grant_head_wake(log, &log->l_reserve_head, &free_bytes, > + &wakeq); > spin_unlock(&log->l_reserve_head.lock); > + wake_up_q(&wakeq); > } > } Ok, what about xlog_grant_head_wake_all()? You didn't convert that to use wake queues, and so that won't remove tickets for the grant head waiter list, and so those tasks will never get out of the new inner loop you added to xlog_grant_head_wait(). That means filesystem shutdowns will just hang the filesystem and leave it unmountable. Did you run this through fstests? Cheers, Dave -- Dave Chinner da...@fromorbit.com

Re: [PATCH] mm: introduce kvvirt_to_page() helper

2018-08-20 Thread Dave Chinner
gt; that array. If the memory isn't vmapped, maybe the caller could pass > an array pointer like XFS does, or we could require them to pass GFP flags > to allocate a new array. We have code in XFS to avoid allocating the array for the common case - it ends up embedded in the struct xfs_buf instead and so we avoid needing and extra alloc/free for every single buffer in the fast path. Hence I'd prefer the interface lets the caller to supply the result array, similar to the way callers provide their own pagevecs for bulk page lookup functions... Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH RFC 01/10] rcu: Make CONFIG_SRCU unconditionally enabled

2018-08-08 Thread Dave Chinner
e've had enough recent bugs to deal with from shrinkers being called before the filesystem is set up and from trying to handle allocation errors during setup. Do we really want to make shrinker shutdown just as prone to mismanagement and subtle, hard to hit bugs? I don't think we do - unmount is simply not a critical performance path. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [LTP] [PATCH 4.17 00/67] 4.17.7-stable review

2018-07-16 Thread Dave Chinner
t; > thanks, > > > > > > greg k-h > > > > > > Results from Linaro’s test farm. > > Regressions detected. > > > > LTP syscalls failed test cases on all devices arm64, arm32 and x86_64, > > - creat08 > > - open10 > >

Re: [PATCH 2/2] fs: xfs: use BUG_ON if writepage call comes from direct reclaim

2018-07-02 Thread Dave Chinner
ituation gracefully - this is the appropriate way to handle incorrect calling contexts. There is absolutely no good reason to panic production kernels in situations like this. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: config files and how to have persistent Linux kernel Driver/File System configuration info saved

2018-06-28 Thread Dave Chinner
s, but the fact is modifying sysctl in this way exposes it to a whole bunch of stuff sysctl doesn't understand, shouldn't be accessing and/or trying modify. i.e. sysctl is disturbingly dumb, and it gets away with it because of it's restricted scope and API presented by /proc/sys.

[BUG? v4.16.12] OOM kill of qemu causes whacky pty shenanigans

2018-06-25 Thread Dave Chinner
ontexts. But something is clearly getting screwed up and the trigger is an OOM kill event. Anyone got any ideas on how to go about debugging this? Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [lkp-robot] [xfs] b027d4c97b: fio.latency_2ms% +7.1% regression

2018-06-05 Thread Dave Chinner
0.69 ± 13% fio.latency_750us% This is not a regression. The number of IOs in the 4ms IO latency bin has reduced by 7%, and inumber in the 2ms IO latency bin has increased by 7%. IOWs, there's a measurable improvement in IO latency as a result of those patches, not a regression. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: linux-next: manual merge of the xfs tree with Linus' tree

2018-06-04 Thread Dave Chinner
rged into Linus' tree. Does that mean the gfs2 tree is not being pulled into the linux-next tree? Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [GIT PULL] overlayfs update for 4.18

2018-05-30 Thread Dave Chinner
btrfs ioctl years ago and there are several dedupe applications out there that use it. That's why we pulled it up to the vfs rather than invent a new one and have to wait years for apps to start using it... Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [REVIEW][PATCH 0/6] Wrapping up the vfs support for unprivileged mounts

2018-05-29 Thread Dave Chinner
On Tue, May 29, 2018 at 09:34:35PM -0500, Eric W. Biederman wrote: > Dave Chinner writes: > > > Yeah, the are some fairly big process and policy things that > > need to be decided here. Not just at the kernel level, but at > > distro and app infrastructure level too.

Re: [REVIEW][PATCH 0/6] Wrapping up the vfs support for unprivileged mounts

2018-05-29 Thread Dave Chinner
On Tue, May 29, 2018 at 08:12:28AM -0500, Eric W. Biederman wrote: > Dave Chinner writes: > > > On Thu, May 24, 2018 at 06:23:30PM -0500, Eric W. Biederman wrote: > >> "Theodore Y. Ts'o" writes: > >> > >> > On Wed,

Re: [PATCH v2] doc: document scope NOFS, NOIO APIs

2018-05-29 Thread Dave Chinner
t; Requested-by: "Theodore Y. Ts'o" > Signed-off-by: Michal Hocko We could bikeshed forever about the exact wording, but it covers everything I think needs to be documented. Reviewed-by: Dave Chinner -- Dave Chinner da...@fromorbit.com

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-28 Thread Dave Chinner
On Mon, May 28, 2018 at 11:19:23AM +0200, Michal Hocko wrote: > On Mon 28-05-18 09:48:54, Dave Chinner wrote: > > On Fri, May 25, 2018 at 10:16:24AM +0200, Michal Hocko wrote: > > > On Fri 25-05-18 08:17:15, Dave Chinner wrote: > > > > On Thu, May 24, 2018 at 01:43:

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-27 Thread Dave Chinner
On Fri, May 25, 2018 at 10:16:24AM +0200, Michal Hocko wrote: > On Fri 25-05-18 08:17:15, Dave Chinner wrote: > > On Thu, May 24, 2018 at 01:43:41PM +0200, Michal Hocko wrote: > [...] > > > +FS/IO code then simply calls the appropriate save function right at the > >

Re: [REVIEW][PATCH 0/6] Wrapping up the vfs support for unprivileged mounts

2018-05-24 Thread Dave Chinner
make removable media > reasonably safe from privilege escalation attacks. > > There is enough code in most filesystems that I don't know what our > chances of locking down very many of them are. But I figure a few more > of them are possible. I'm not sure we need to - fus

Re: [PATCH] doc: document scope NOFS, NOIO APIs

2018-05-24 Thread Dave Chinner
e access to." I think it's also worth making a note about recursive/nested save/restore stacking, because it's not clear from this description that this is allowed and will work as long as inner save/restore calls are fully nested inside outer save/restore contexts. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: Bugs involving maliciously crafted file system

2018-05-23 Thread Dave Chinner
On Wed, May 23, 2018 at 08:59:06PM -0400, Theodore Y. Ts'o wrote: > On Thu, May 24, 2018 at 10:49:31AM +1000, Dave Chinner wrote: > > > > We've learnt this lesson the hard way over and over again: don't > > parse untrusted input in privileged contexts. How m

Re: Bugs involving maliciously crafted file system

2018-05-23 Thread Dave Chinner
overrun (and I fixed > two in ext4 in the last two days while being stuck in a T13 standards > meeting, so I do feel your pain) might be a very different story. The fact you are currently finding and fixing buffer overuns in ext4 solidly demonstrates my point about existing filesystems being largely untrustable and unfixable. :/ Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: INFO: task hung in xlog_grant_head_check

2018-05-23 Thread Dave Chinner
age-guided > fuzzers are no longer state of the art. I've been really amazed at the bugs > syzkaller been able to find in other kernel subsystems, e.g. obscure races > that > no one would have ever thought to test for. OTOH, knowing how many bugs lurk in our code base, I'm still amazed that tools like syzbot find so few of them. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: INFO: task hung in xlog_grant_head_check

2018-05-22 Thread Dave Chinner
On Tue, May 22, 2018 at 03:52:08PM -0700, Eric Biggers wrote: > On Wed, May 23, 2018 at 08:26:20AM +1000, Dave Chinner wrote: > > On Tue, May 22, 2018 at 08:31:08AM -0400, Brian Foster wrote: > > > On Mon, May 21, 2018 at 10:55:02AM -0700, syzbot wrote: > > > > Hello,

Re: INFO: task hung in xlog_grant_head_check

2018-05-22 Thread Dave Chinner
like this. Fuzzing random collections of v4 filesystem fragments will continue to trip over problems we've avoided with v5 filesystems, and this is further evidence to point to that. I'd suggest that at this point, syzbot XFS reports should be redirected to /dev/null. It's not worth our ti

Re: [RFC PATCH 08/11] xfs: have sync_fs op report writeback errors when passed a since pointer

2018-05-21 Thread Dave Chinner
* We schedule log work now (now that the disk is * active) instead of later (when it might not be). */ flush_delayed_work(&mp->m_log->l_work); } - return 0 + return errseq_check_and_advance(&sb->s_wb_err, since); } Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCHv2][SMB3] Add kernel trace support

2018-05-20 Thread Dave Chinner
On Sat, May 19, 2018 at 08:56:39PM -0500, Steve French wrote: > On Sat, May 19, 2018 at 6:22 PM, Dave Chinner wrote: > > On Fri, May 18, 2018 at 01:43:14PM -0700, Steve French wrote: > >> On Fri, May 18, 2018 at 11:46 AM, Ralph Böhme wrote: > >> > On Thu, May 17, 2

Re: [PATCHv2][SMB3] Add kernel trace support

2018-05-19 Thread Dave Chinner
ng is about this vs. systemtap these days. There was a nice > three part series > describing ftrace/trace-cmd on lwn > (https://old.lwn.net/Articles/365835/) a while ago. > > In terms of useability "trace-cmd" looked good to me and much more > powerful than the > current dmesg based printk style debugging. And then you learn about trace_printk() for putting custom one-off debug into the tracepoint stream and wonder why you didn't know about this years ago :P Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: WARNING: bad unlock balance in xfs_iunlock

2018-05-11 Thread Dave Chinner
On Fri, May 11, 2018 at 10:59:53AM +0200, Dmitry Vyukov wrote: > On Thu, May 10, 2018 at 1:22 AM, Dave Chinner wrote: > > On Wed, May 09, 2018 at 10:43:05AM +0200, Dmitry Vyukov wrote: > >> Does "xfstests fuzzing infrastructure" use coverage-guidance? > >

Re: [PATCH 1/2] bcachefs: On disk data structures

2018-05-11 Thread Dave Chinner
type; > + }; > +#else This is another worry - using "long" in the on disk structure definition. If this is in-meory structures, then use le64_to_cpu/cpu_to_le64 to convert the value from the on-disk value to the in-memory, cpu order value Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: WARNING: bad unlock balance in xfs_iunlock

2018-05-09 Thread Dave Chinner
filesystems and finding problems we haven't, then I'm going to be interested in their fuzzing tools. But (guided) random bit perturbation fuzzing of legacy filesystem formats is really not that useful or interesting to us right now. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [RFC][PATCH 0/76] vfs: 'views' for filesystems with more than one root

2018-05-08 Thread Dave Chinner
On Tue, May 08, 2018 at 10:06:44PM -0400, Jeff Mahoney wrote: > On 5/8/18 7:38 PM, Dave Chinner wrote: > > On Tue, May 08, 2018 at 11:03:20AM -0700, Mark Fasheh wrote: > >> Hi, > >> > >> The VFS's super_block covers a variety of filesystem functiona

Re: WARNING: bad unlock balance in xfs_iunlock

2018-05-08 Thread Dave Chinner
On Tue, May 08, 2018 at 09:56:01AM +0200, Dmitry Vyukov wrote: > On Wed, May 2, 2018 at 12:51 AM, Dave Chinner wrote: > >> >>> Hello, > >> >>> > >> >>> syzbot hit the following crash on upstream commit > >> >>> 86bbbe

Re: [RFC][PATCH 0/76] vfs: 'views' for filesystems with more than one root

2018-05-08 Thread Dave Chinner
d it's something we talked about at LSFMM because we need to solve it for reliable hibernation. > It's far more efficient then to pull those fields we need for a > subvolume namespace into their own structure. I'm not convinced yet - it still feels like it's the wrong layer to be solving the multiple namespace per superblock problem Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: WARNING: bad unlock balance in xfs_iunlock

2018-05-01 Thread Dave Chinner
On Mon, Apr 30, 2018 at 03:24:48PM +0200, Dmitry Vyukov wrote: > On Thu, Apr 5, 2018 at 8:54 PM, Dmitry Vyukov wrote: > > On Tue, Apr 3, 2018 at 6:38 AM, Dave Chinner wrote: > >> On Mon, Apr 02, 2018 at 07:01:02PM -0700, syzbot wrote: > >>> Hello, > >>&g

Re: WARNING in up_write

2018-04-05 Thread Dave Chinner
On Thu, Apr 05, 2018 at 05:13:25PM -0700, Eric Biggers wrote: > On Fri, Apr 06, 2018 at 08:32:26AM +1000, Dave Chinner wrote: > > On Wed, Apr 04, 2018 at 08:24:54PM -0700, Matthew Wilcox wrote: > > > On Wed, Apr 04, 2018 at 11:22:00PM -0400, Theodore Y. Ts'o wrote: > >

Re: WARNING in up_write

2018-04-05 Thread Dave Chinner
:-) > > Not really ... syzbot caught it Monday evening ;-) Rather than arguing over who reported it first, I think that time would be better spent reflecting on why the syzbot report was completely ignored until *after* Ted diagnosed the issue independently and Waiman had already fixed it Clearly there is scope for improvement here. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: WARNING: bad unlock balance in xfs_iunlock

2018-04-05 Thread Dave Chinner
On Thu, Apr 05, 2018 at 08:54:50PM +0200, Dmitry Vyukov wrote: > On Tue, Apr 3, 2018 at 6:38 AM, Dave Chinner wrote: > > On Mon, Apr 02, 2018 at 07:01:02PM -0700, syzbot wrote: > >> Hello, > >> > >> syzbot hit the following crash on upstream commit > >

Re: REGRESSION/TEST FAILURE caused by DEBUG_RWSEMS

2018-04-04 Thread Dave Chinner
nd of > cross task lock/unlock. I think we should add the corresponding > up_write_non_owner() for that. Keep in mind this is a percpu_rwsem, not a rwsem. It's the sb_freeze_unlock() call in thaw_super() that is triggering it, and it would appear that percpu_rwsems have no concept on "non_owner" operation ATM. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: WARNING: bad unlock balance in xfs_iunlock

2018-04-02 Thread Dave Chinner
ate our own filesystem image than I'm certainly not going to waste time looking at these reports... Cheers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH] xfs: always free inline data before resetting inode fork during ifree

2018-04-02 Thread Dave Chinner
On Mon, Apr 02, 2018 at 12:32:44AM +, Sasha Levin wrote: > On Sun, Apr 01, 2018 at 08:02:45AM +1000, Dave Chinner wrote: > >On Fri, Mar 30, 2018 at 02:47:05AM +, Sasha Levin wrote: > >> On Thu, Mar 29, 2018 at 10:05:35AM +1100, Dave Chinner wrote: > >> >On W

Re: [PATCH] xfs: always free inline data before resetting inode fork during ifree

2018-03-31 Thread Dave Chinner
On Fri, Mar 30, 2018 at 02:47:05AM +, Sasha Levin wrote: > On Thu, Mar 29, 2018 at 10:05:35AM +1100, Dave Chinner wrote: > >On Wed, Mar 28, 2018 at 07:30:06PM +, Sasha Levin wrote: > > This commit has been processed by the -stable helper bot and determined > > to

Re: [PATCH] xfs: always free inline data before resetting inode fork during ifree

2018-03-28 Thread Dave Chinner
On Wed, Mar 28, 2018 at 07:30:06PM +, Sasha Levin wrote: > On Wed, Mar 28, 2018 at 02:32:28PM +1100, Dave Chinner wrote: > >How much time are your test rigs going to be able to spend running > >xfstests? A single pass on a single filesysetm config on spinning > >disks w

Re: [PATCH] xfs: always free inline data before resetting inode fork during ifree

2018-03-27 Thread Dave Chinner
On Mon, Mar 26, 2018 at 07:54:31PM -0400, Sasha Levin wrote: > On Sun, Mar 25, 2018 at 6:33 PM, Dave Chinner wrote: > > On Fri, Mar 23, 2018 at 06:23:02PM +, Luis R. Rodriguez wrote: > >> On Fri, Mar 23, 2018 at 10:26:20AM -0700, Darrick J. Wong wrote: > >> >

Re: [PATCH RFC] xfs, memcg: Call xfs_fs_nr_cached_objects() only in case of global reclaim

2018-03-25 Thread Dave Chinner
On Fri, Mar 23, 2018 at 03:39:53PM +0300, Kirill Tkhai wrote: > On 23.03.2018 02:46, Dave Chinner wrote: > > On Thu, Mar 22, 2018 at 07:52:37PM +0300, Kirill Tkhai wrote: > >> Here is the problem I'm solving: https://lkml.org/lkml/2018/3/21/365. > > > > Oh, fina

Re: [PATCH] xfs: always free inline data before resetting inode fork during ifree

2018-03-25 Thread Dave Chinner
like they are simple often have subtle dependencies in them that automated backports won't ever be able to understand, and if we don't get that right, we break stuff. Filesystems aren't like drivers or memory management - you can't reboot to fix a filesystem corruption or data

Re: [PATCH RFC] xfs, memcg: Call xfs_fs_nr_cached_objects() only in case of global reclaim

2018-03-22 Thread Dave Chinner
issues... > Can't we call shrink of shared objects only for top memcg? Something like > this: What's a "shared object", and how is it any different to a normal slab cache object? CHeers, Dave. -- Dave Chinner da...@fromorbit.com

Re: [PATCH RFC] xfs, memcg: Call xfs_fs_nr_cached_objects() only in case of global reclaim

2018-03-21 Thread Dave Chinner
On Wed, Mar 21, 2018 at 07:15:14PM +0300, Kirill Tkhai wrote: > On 20.03.2018 17:34, Dave Chinner wrote: > > On Tue, Mar 20, 2018 at 04:15:16PM +0300, Kirill Tkhai wrote: > >> On 20.03.2018 03:18, Dave Chinner wrote: > >>> On Mon, Mar 19, 2018 at 02:06:01PM +0300,

Re: [PATCH RFC] xfs, memcg: Call xfs_fs_nr_cached_objects() only in case of global reclaim

2018-03-20 Thread Dave Chinner
On Tue, Mar 20, 2018 at 04:15:16PM +0300, Kirill Tkhai wrote: > On 20.03.2018 03:18, Dave Chinner wrote: > > On Mon, Mar 19, 2018 at 02:06:01PM +0300, Kirill Tkhai wrote: > >> On 17.03.2018 00:39, Dave Chinner wrote: > > Actually, it is fair, because: > > >

Re: [PATCH RFC] xfs, memcg: Call xfs_fs_nr_cached_objects() only in case of global reclaim

2018-03-19 Thread Dave Chinner
On Mon, Mar 19, 2018 at 02:06:01PM +0300, Kirill Tkhai wrote: > On 17.03.2018 00:39, Dave Chinner wrote: > > On Fri, Mar 16, 2018 at 11:55:30AM +0300, Kirill Tkhai wrote: > >> On 16.03.2018 02:03, Dave Chinner wrote: > >>> On Thu, Mar 15, 2018 at 10:28:43PM +0300,

Re: xfs: list corruption in xfs_setup_inode()

2018-03-19 Thread Dave Chinner
x.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F as that will tell us what code paths are executing on inode allocation. Cheers, Dave. -- Dave Chinner da...@fromorbit.com

<    1   2   3   4   5   6   7   8   9   10   >