[PATCH blktests -v2] Fix build failure for discontiguous-io on 32-bit platforms

2018-10-30 Thread Theodore Ts'o
Signed-off-by: Theodore Ts'o --- src/discontiguous-io.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/discontiguous-io.cpp b/src/discontiguous-io.cpp index 5e0ee0f..f51a305 100644 --- a/src/discontiguous-io.cpp +++ b/src/discontiguous-io.cpp @@ -251,7 +251,7 @@ int

[PATCH blktests] Add use of logger so that syslog files show when each test starts

2018-10-29 Thread Theodore Ts'o
Signed-off-by: Theodore Ts'o --- check | 3 +++ 1 file changed, 3 insertions(+) diff --git a/check b/check index f6c3537..ebd87c0 100755 --- a/check +++ b/check @@ -319,6 +319,7 @@ _call_test() { local dmesg_marker="" CHECK_DMESG=0 fi + $L

[PATCH blktests] Fix build failure for discontiguous-io on 32-bit platforms

2018-10-29 Thread Theodore Ts'o
Signed-off-by: Theodore Ts'o --- src/discontiguous-io.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/discontiguous-io.cpp b/src/discontiguous-io.cpp index 5e0ee0f..a59c18d 100644 --- a/src/discontiguous-io.cpp +++ b/src/discontiguous-io.cpp @@ -291,7 +291,7 @@ int

[PATCH blktests] common/rc: allow the loop driver to be built into the kernel

2018-10-22 Thread Theodore Ts'o
, _have_kernel_module which works like _have_module but will not fail if the driver is compiled directly into the kernel. Signed-off-by: Theodore Ts'o --- common/rc | 10 +- tests/nvme/002 | 4 ++-- tests/nvme/003 | 4 ++-- tests/nvme/004 | 4 ++-- tests/nvme/005 | 6 +++--- tests/nvme/006

[PATCH -v2] loop: add recursion validation to LOOP_CHANGE_FD

2018-05-07 Thread Theodore Ts'o
66f994b041...@syzkaller.appspotmail.com Reported-by: syzbot+0a89a9ce473936c57...@syzkaller.appspotmail.com Signed-off-by: Theodore Ts'o <ty...@mit.edu> --- drivers/block/loop.c | 68 +--- 1 file changed, 38 insertions(+), 30 deletions(-) diff --git a/drivers/block/

[PATCH] loop: add recursion validation to LOOP_CHANGE_FD

2018-05-03 Thread Theodore Ts'o
66f994b041...@syzkaller.appspotmail.com Reported-by: syzbot+0a89a9ce473936c57...@syzkaller.appspotmail.com Signed-off-by: Theodore Ts'o <ty...@mit.edu> --- drivers/block/loop.c | 70 1 file changed, 38 insertions(+), 32 deletions(-) diff --git a/drivers/block/

Re: EXT4 Oops (Re: [PATCH V15 06/22] mmc: block: Add blk-mq support)

2018-03-01 Thread Theodore Ts'o
On Thu, Mar 01, 2018 at 01:15:24AM -0800, Jose R R wrote: > Probably it is not wise to place all your eggs (data) in one basket > (ext4) and diversify to viable alternatives which won't be affected by > UNIX 2038 year date problem, likewise? > < >

Re: EXT4 Oops (Re: [PATCH V15 06/22] mmc: block: Add blk-mq support)

2018-03-01 Thread Theodore Ts'o
On Thu, Mar 01, 2018 at 10:55:37AM +0200, Adrian Hunter wrote: > On 27/02/18 11:28, Adrian Hunter wrote: > > On 26/02/18 23:48, Dmitry Osipenko wrote: > >> But still something is wrong... I've been getting occasional EXT4 Ooops's, > >> like > >> the one below, and __wait_on_bit() is always

Re: Panic with ext4,nbd,qemu-img,block

2018-01-20 Thread Theodore Ts'o
On Sat, Jan 20, 2018 at 04:41:00PM +0800, Hongzhi, Song wrote: > > 329.11 EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: > (null) > 329.12 block nbd0: Connection timed out > 329.13 block nbd0: shutting down sockets That's your problem. I'm guessing qemu-nbd is dying for some

Re: [LSF/MM TOPIC] A high-performance userspace block driver

2018-01-16 Thread Theodore Ts'o
On Tue, Jan 16, 2018 at 06:52:40AM -0800, Matthew Wilcox wrote: > > I see the improvements that Facebook have been making to the nbd driver, > and I think that's a wonderful thing. Maybe the outcome of this topic > is simply: "Shut up, Matthew, this is good enough". > > It's clear that there's

Re: About the try to remove cross-release feature entirely by Ingo

2018-01-02 Thread Theodore Ts'o
On Wed, Jan 03, 2018 at 11:10:37AM +0900, Byungchul Park wrote: > > The point I was trying to drive home is that "all we have to do is > > just classify everything well or just invalidate the right lock > > Just to be sure, we don't have to invalidate lock objects at all but > a problematic

Re: About the try to remove cross-release feature entirely by Ingo

2018-01-01 Thread Theodore Ts'o
On Mon, Jan 01, 2018 at 02:18:55AM -0800, Matthew Wilcox wrote: > > Clarification: all TCP connections that are used by kernel code would > > need to be in their own separate lock class. All TCP connections used > > only by userspace could be in their own shared lock class. You can't > > use a

Re: About the try to remove cross-release feature entirely by Ingo

2017-12-30 Thread Theodore Ts'o
On Sat, Dec 30, 2017 at 05:40:28PM -0500, Theodore Ts'o wrote: > On Sat, Dec 30, 2017 at 12:44:17PM -0800, Matthew Wilcox wrote: > > > > I'm not sure I agree with this part. What if we add a new TCP lock class > > for connections which are used for filesystems/network block

Re: About the try to remove cross-release feature entirely by Ingo

2017-12-30 Thread Theodore Ts'o
On Fri, Dec 29, 2017 at 10:16:24PM -0800, Matthew Wilcox wrote: > > I think this is a terminology problem. To me (and, I suspect Ted), a > waiter is a subject of a verb while a lock is an object. So Ted is asking > whether we have to classify the users, while I think you're saying we > have

Re: About the try to remove cross-release feature entirely by Ingo

2017-12-28 Thread Theodore Ts'o
On Fri, Dec 29, 2017 at 10:47:36AM +0900, Byungchul Park wrote: > >(1) The best way: To classify all waiters correctly. It's really not all waiters, but all *locks*, no? > Ultimately the problems should be solved in this way. But it > takes a lot of time so it's not easy to use

Re: [PATCH] locking/lockdep: Remove the cross-release locking checks

2017-12-15 Thread Theodore Ts'o
On Fri, Dec 15, 2017 at 05:39:25PM +0900, Byungchul Park wrote: > > All locks should belong to one class if each path of acquisition > can be switchable each other within the class at any time. > Otherwise, they should belong to a different class. OK, so let's go back to my case of a Network

Re: [PATCH] locking/lockdep: Remove the cross-release locking checks

2017-12-14 Thread Theodore Ts'o
On Fri, Dec 15, 2017 at 01:05:43PM +0900, Byungchul Park wrote: > For example, in the case of fs issues, for now we can > invalidate wait_for_completion() in submit_bio_wait() And this will spawn a checkpatch.pl ERROR: ERROR("LOCKDEP",

Re: About the try to remove cross-release feature entirely by Ingo

2017-12-13 Thread Theodore Ts'o
On Wed, Dec 13, 2017 at 04:13:07PM +0900, Byungchul Park wrote: > > Therefore, I want to say the fundamental problem > comes from classification, not cross-release > specific. You keep saying that it is "just" a matter of classificaion. However, it is not obvious how to do the classification in

Re: [PATCH] locking/lockdep: Add CONFIG_LOCKDEP_AGGRESSIVE

2017-12-12 Thread Theodore Ts'o
On Tue, Dec 12, 2017 at 02:20:32PM +0900, Byungchul Park wrote: > > The *problem* is false positives, since locks and waiters in > kernel are not classified properly, at the moment, which is just > a fact that is not related to cross-release stuff at all. IOW, > that would be useful once all

Re: [PATCH] locking/lockdep: Add CONFIG_LOCKDEP_AGGRESSIVE

2017-12-11 Thread Theodore Ts'o
On Mon, Dec 11, 2017 at 01:06:55PM -0800, Linus Torvalds wrote: > On Sun, Dec 10, 2017 at 7:50 PM, Theodore Ts'o <ty...@mit.edu> wrote: > > CONFIG_LOCKDEP_CROSSRELEASE and CONFIG_LOCKDEP_COMPLETIONS can result > > in a large number of false positives because lockdep doesn

Re: [PATCH] locking/lockdep: Add CONFIG_LOCKDEP_AGGRESSIVE

2017-12-10 Thread Theodore Ts'o
On Sun, Dec 10, 2017 at 10:50:17PM -0500, Theodore Ts'o wrote: > CONFIG_LOCKDEP_CROSSRELEASE and CONFIG_LOCKDEP_COMPLETIONS can result > in a large number of false positives because lockdep doesn't > understand how to deal with multiple stacked loop or MD devices. > > Until someone

[PATCH] locking/lockdep: Add CONFIG_LOCKDEP_AGGRESSIVE

2017-12-10 Thread Theodore Ts'o
system developers will disable LOCKDEP entirely since it results in far too many false positives when trying to use xfstests. Signed-off-by: Theodore Ts'o <ty...@mit.edu> Cc: Byungchul Park <byungchul.p...@lge.com> Cc: Linus Torvalds <torva...@linux-foundation.org> Cc: P

Re: [dm-devel] Ideas to reuse filesystem's checksum to enhance dm-raid1/10/5/6?

2017-11-20 Thread Theodore Ts'o
On Thu, Nov 16, 2017 at 03:32:05PM -0700, Chris Murphy wrote: > > XFS by default does metadata csums. But ext4 doesn't use it for either > metadata or the journal by default still, it is still optional. So for > now it mainly benefits XFS. Metadata checksums are enabled by default in the version

Re: [ANNOUNCE] fsperf: a simple fs/block performance testing framework

2017-10-09 Thread Theodore Ts'o
On Mon, Oct 09, 2017 at 08:54:16AM -0400, Josef Bacik wrote: > I purposefully used as little as possible, just json and sqlite, and I tried > to > use as little python3 isms as possible. Any rpm based systems should have > these > libraries already installed, I agree that using any of the PyPI

Re: [ANNOUNCE] fsperf: a simple fs/block performance testing framework

2017-10-09 Thread Theodore Ts'o
On Mon, Oct 09, 2017 at 02:54:34PM +0800, Eryu Guan wrote: > I have no problem either if python is really needed, after all this is a > very useful infrastructure improvement. But the python version problem > brought up by Ted made me a bit nervous, we need to work that round > carefully. > >

Re: [ANNOUNCE] fsperf: a simple fs/block performance testing framework

2017-10-08 Thread Theodore Ts'o
On Sun, Oct 08, 2017 at 10:25:10PM -0400, Josef Bacik wrote: > > Probably should have led with that shouldn't I have? There's nothing keeping > me > from doing it, but I didn't want to try and shoehorn in a python thing into > fstests. I need python to do the sqlite and the json parsing to

Re: [RFC 5/5] pm: remove kernel thread freezing

2017-10-06 Thread Theodore Ts'o
On Fri, Oct 06, 2017 at 02:07:13PM +0200, Pavel Machek wrote: > > Yeah, I was not careful enough reading cover letter. Having series > where 1-4/5 are ready to go, and 5/5 not-good-idea for years to come > is quite confusing. 4/5 is not ready to go either, at the very least

Re: [RFC 4/5] ext4: add fs freezing support on suspend/hibernation

2017-10-04 Thread Theodore Ts'o
On Wed, Oct 04, 2017 at 06:05:23PM +1100, Dave Chinner wrote: > Basically, before thawing filesystems the rest of the kernel > infrastructure needs to have been restarted. i.e. the order > needs to be: > > freeze userspace > freeze filesystems > freeze kernel threads > freeze workqueues > > thaw

Re: [RFC 4/5] ext4: add fs freezing support on suspend/hibernation

2017-10-03 Thread Theodore Ts'o
On Tue, Oct 03, 2017 at 11:53:12AM -0700, Luis R. Rodriguez wrote: > @@ -4926,7 +4926,7 @@ static int ext4_unfreeze(struct super_block *sb) > ext4_set_feature_journal_needs_recovery(sb); > } > > - ext4_commit_super(sb, 1); > + ext4_commit_super(sb, 0); > return

Re: [LSF/MM TOPIC][LSF/MM ATTEND] OCSSDs - SMR, Hierarchical Interface, and Vector I/Os

2017-01-09 Thread Theodore Ts'o
On Tue, Jan 10, 2017 at 10:42:45AM +0900, Damien Le Moal wrote: > Thank you for the information. I will check this out. Is it the > optimization that aggressively delay meta-data update by allowing > reading of meta-data blocks directly from the journal (for blocks that > are not yet updated in

Re: [LSF/MM TOPIC][LSF/MM ATTEND] OCSSDs - SMR, Hierarchical Interface, and Vector I/Os

2017-01-09 Thread Theodore Ts'o
So in the model where the Flash-side is tracking logical to physical zone mapping, and host is merely expecting the ZBC interface, one way it could work is as follows. 1) The flash signals that a particular zone should be reset soon. 2) If the host does not honor the request, eventually the

Re: [LSF/MM TOPIC][LSF/MM ATTEND] OCSSDs - SMR, Hierarchical Interface, and Vector I/Os

2017-01-04 Thread Theodore Ts'o
I agree with Damien, but I'd also add that in the future there may very well be some new Zone types added to the ZBC model. So we shouldn't assume that the ZBC model is a fixed one. And who knows? Perhaps T10 standards body will come up with a simpler model for interfacing with

Re: [PATCH 45/60] block: bio: introduce bio_for_each_segment_all_rd() and its write pair

2016-11-01 Thread Theodore Ts'o
On Tue, Nov 01, 2016 at 07:51:27AM +0800, Ming Lei wrote: > Sorry for forgetting to mention one important point: > > - after multipage bvec is introduced, the iterated bvec pointer > still points to singlge page bvec, which is generated in-flight > and is readonly actually. That is the motivation

Re: [PATCH 45/60] block: bio: introduce bio_for_each_segment_all_rd() and its write pair

2016-10-31 Thread Theodore Ts'o
On Sat, Oct 29, 2016 at 04:08:44PM +0800, Ming Lei wrote: > This patches introduce bio_for_each_segment_all_rd() and > bio_for_each_segment_all_wt(). > > bio_for_each_segment_all_rd() is for replacing > bio_for_each_segment_all() in case the bvec from bio->bi_io_vec > is accessed as readonly. >

Re: [PATCHv2, 00/41] ext4: support of huge pages

2016-08-12 Thread Theodore Ts'o
On Fri, Aug 12, 2016 at 09:37:43PM +0300, Kirill A. Shutemov wrote: > Here's stabilized version of my patchset which intended to bring huge pages > to ext4. So this patch is more about mm level changes than it is about the file system, and I didn't see any comments from the linux-mm peanut