Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Fri, Jul 21, 2023 at 06:14:04PM +1000, Finn Thain wrote: > > I'm not blaming the unstable API for the bugs, I'm blaming it for the > workload. A stable API (like a userspace API) decreases the likelihood > that overloaded maintainers have to orphan a filesystem implementation. You are incorrect. The HFS file system has gotten zero development attention and the bugs were not the result of the API changes. The main issue here is that the HFS file system does not have maintainer, and decreasing the workload will not magically make someone appear with deep knowledge of that particular part of the code base. It's also the case that the actual amount of work on the "overloaded maintainers" caused by API changes is minimal --- it's dwarfed by syzbot noise (complaints from syzbot that aren't really bugs, or for really outré threat models). API changes within the kernel are the responsibility of the people making the change. For example, consider all of the folio changes that have been landing in the kernel; the amount of extra work on the part of most file system maintainers is minimal, because it's the people making the API changes who update the file system. I won't say that it's _zero_ work, because file system maintainers review the changes, and we run regression tests, and we sometimes need to point out when a bug has been introduced --- at which point the person making the API change has the responsibility of fixing or reverting the change. An unstable API are much painful for out-of-tree kernel code. But upstream kernel developers aren't really concerned with out-of-tree kernel code, except to point out that the work of the people who are promulgated out-of-tree modules would be much less if they actually got them cleaned up and made acceptable for upstream inclusion. - Ted
Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Fri, 21 Jul 2023, Matthew Wilcox wrote: > > You've misunderstood. Google have decided to subject the entire kernel > (including obsolete unmaintained filesystems) to stress tests that it's > never had before. IOW these bugs have been there since the code was > merged. There's nothing to back out. There's no API change to blame. > It's always been buggy and it's never mattered before. > I'm not blaming the unstable API for the bugs, I'm blaming it for the workload. A stable API (like a userspace API) decreases the likelihood that overloaded maintainers have to orphan a filesystem implementation.
Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Fri, Jul 21, 2023 at 11:03:28AM +1000, Finn Thain wrote: > On Fri, 21 Jul 2023, Dave Chinner wrote: > > > > I suspect that this is one of those catch-22 situations: distros are > > > going to enable every feature under the sun. That doesn't mean that > > > anyone is actually _using_ them these days. > > I think the value of filesystem code is not just a question of how often > it gets executed -- it's also about retaining access to the data collected > in archives, museums, galleries etc. that is inevitably held in old > formats. That's an argument for adding support to tar, not for maintaining read/write support. > > We need to much more proactive about dropping support for unmaintained > > filesystems that nobody is ever fixing despite the constant stream of > > corruption- and deadlock- related bugs reported against them. > > IMO, a stream of bug reports is not a reason to remove code (it's a reason > to revert some commits). > > Anyway, that stream of bugs presumably flows from the unstable kernel API, > which is inherently high-maintenance. It seems that a stable API could be > more appropriate for any filesystem for which the on-disk format is fixed > (by old media, by unmaintained FLOSS implementations or abandoned > proprietary implementations). You've misunderstood. Google have decided to subject the entire kernel (including obsolete unmaintained filesystems) to stress tests that it's never had before. IOW these bugs have been there since the code was merged. There's nothing to back out. There's no API change to blame. It's always been buggy and it's never mattered before. It wouldn't be so bad if Google had also decided to fund people to fix those bugs, but no, they've decided to dump them on public mailing lists and berate developers into fixing them.
Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Fri, 21 Jul 2023, Dave Chinner wrote: > > I suspect that this is one of those catch-22 situations: distros are > > going to enable every feature under the sun. That doesn't mean that > > anyone is actually _using_ them these days. I think the value of filesystem code is not just a question of how often it gets executed -- it's also about retaining access to the data collected in archives, museums, galleries etc. that is inevitably held in old formats. > > We need to much more proactive about dropping support for unmaintained > filesystems that nobody is ever fixing despite the constant stream of > corruption- and deadlock- related bugs reported against them. > IMO, a stream of bug reports is not a reason to remove code (it's a reason to revert some commits). Anyway, that stream of bugs presumably flows from the unstable kernel API, which is inherently high-maintenance. It seems that a stable API could be more appropriate for any filesystem for which the on-disk format is fixed (by old media, by unmaintained FLOSS implementations or abandoned proprietary implementations). Being in userspace, I suppose FUSE could be a stable API though I imagine it's not ideal in the sense that migrating kernel code there would be difficult. Maybe userspace NFS 4 would be a better fit? (I've no idea, I'm out of my depth in /fs...) Ideally, kernel-to-userspace code migration would be done with automatic program transformation -- otherwise it would become another stream of bugs.
Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Thu, 20 Jul 2023 at 15:37, Matthew Wilcox wrote: > > I think you're missing the context. There are bugs in how this filesystem > handles intentionally-corrupted filesystems. That's being reported as > a critical bug because apparently some distributions automount HFS/HFS+ > filesystems presented to them on a USB key. Nobody is being paid to fix > these bugs. Nobody is volunteering to fix these bugs out of the kindness > of their heart. What choice do we have but to remove the filesystem, > regardless of how many happy users it has? You're being silly. We have tons of sane options. The obvious one is "just don't mount untrusted media". Now, the kernel doesn't know which media is trusted or not, since the kernel doesn't actually see things like /etc/mtab and friends. So we in the kernel can't do that, but distros should have a very easy time just fixing their crazy models. Saying that the kernel should remove a completely fine filesystem just because some crazy use-cases that nobody cares about are broken, now *that* just crazy. Now, would it be good to have a maintainer for hgs? Obviously. But no, we don't remove filesystems just because they don't have maintainers. And no, we have not suddenly started saying "users don't matter". Linus
Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Thu, Jul 20, 2023 at 05:38:52PM -0400, Jeffrey Walton wrote: > On Thu, Jul 20, 2023 at 2:39 PM Matthew Wilcox wrote: > > > > On Thu, Jul 20, 2023 at 07:50:47PM +0200, John Paul Adrian Glaubitz wrote: > > > > Then we should delete the HFS/HFS+ filesystems. They're orphaned in > > > > MAINTAINERS and if distros are going to do such a damnfool thing, > > > > then we must stop them. > > > > > > Both HFS and HFS+ work perfectly fine. And if distributions or users are > > > so > > > sensitive about security, it's up to them to blacklist individual features > > > in the kernel. > > > > > > Both HFS and HFS+ have been the default filesystem on MacOS for 30 years > > > and I don't think it's justified to introduce such a hard compatibility > > > breakage just because some people are worried about theoretical evil > > > maid attacks. > > > > > > HFS/HFS+ mandatory if you want to boot Linux on a classic Mac or PowerMac > > > and I don't think it's okay to break all these systems running Linux. > > > > If they're so popular, then it should be no trouble to find somebody > > to volunteer to maintain those filesystems. Except they've been > > marked as orphaned since 2011 and effectively were orphaned several > > years before that (the last contribution I see from Roman Zippel is > > in 2008, and his last contribution to hfs was in 2006). > > One data point may help.. I've been running Linux on an old PowerMac > and an old Intel MacBook since about 2014 or 2015 or so. I have needed > the HFS/HFS+ filesystem support for about 9 years now (including that > "blessed" support for the Apple Boot partition). > > There's never been a problem with Linux and the Apple filesystems. > Maybe it speaks to the maturity/stability of the code that already > exists. The code does not need a lot of attention nowadays. > > Maybe the orphaned status is the wrong metric to use to determine > removal. Maybe a better metric would be installation base. I.e., how > many users use the filesystem. I think you're missing the context. There are bugs in how this filesystem handles intentionally-corrupted filesystems. That's being reported as a critical bug because apparently some distributions automount HFS/HFS+ filesystems presented to them on a USB key. Nobody is being paid to fix these bugs. Nobody is volunteering to fix these bugs out of the kindness of their heart. What choice do we have but to remove the filesystem, regardless of how many happy users it has?
Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Thu, Jul 20, 2023 at 02:27:50PM -0400, Jeff Layton wrote: > On Thu, 2023-07-20 at 18:59 +0100, Matthew Wilcox wrote: > > On Thu, Jul 20, 2023 at 07:50:47PM +0200, John Paul Adrian Glaubitz wrote: > > > > Then we should delete the HFS/HFS+ filesystems. They're orphaned in > > > > MAINTAINERS and if distros are going to do such a damnfool thing, > > > > then we must stop them. > > > > > > Both HFS and HFS+ work perfectly fine. And if distributions or users are > > > so > > > sensitive about security, it's up to them to blacklist individual features > > > in the kernel. > > > > > > Both HFS and HFS+ have been the default filesystem on MacOS for 30 years > > > and I don't think it's justified to introduce such a hard compatibility > > > breakage just because some people are worried about theoretical evil > > > maid attacks. > > > > > > HFS/HFS+ mandatory if you want to boot Linux on a classic Mac or PowerMac > > > and I don't think it's okay to break all these systems running Linux. > > > > If they're so popular, then it should be no trouble to find somebody > > to volunteer to maintain those filesystems. Except they've been > > marked as orphaned since 2011 and effectively were orphaned several > > years before that (the last contribution I see from Roman Zippel is > > in 2008, and his last contribution to hfs was in 2006). > > I suspect that this is one of those catch-22 situations: distros are > going to enable every feature under the sun. That doesn't mean that > anyone is actually _using_ them these days. > > Is "staging" still a thing? Maybe we should move these drivers into the > staging directory and pick a release where we'll sunset it, and then see > who comes out of the woodwork? No, the train wreck of filesystems in staging proved that it wasn't a viable process. We should just follow the same process as we are using for reiser - mark it as deprecated in place, pick a date that we are going to remove it, then add a warning (both runtime, in kconfig and probably in the kernel filesystem documentation) that it is deprecated and support is going to be removed at a certain date. We should be applying the same criteria and process for all the other filesystems that are orphaned, too. We need to much more proactive about dropping support for unmaintained filesystems that nobody is ever fixing despite the constant stream of corruption- and deadlock- related bugs reported against them. -Dave. -- Dave Chinner da...@fromorbit.com
Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Thu, Jul 20, 2023 at 2:39 PM Matthew Wilcox wrote: > > On Thu, Jul 20, 2023 at 07:50:47PM +0200, John Paul Adrian Glaubitz wrote: > > > Then we should delete the HFS/HFS+ filesystems. They're orphaned in > > > MAINTAINERS and if distros are going to do such a damnfool thing, > > > then we must stop them. > > > > Both HFS and HFS+ work perfectly fine. And if distributions or users are so > > sensitive about security, it's up to them to blacklist individual features > > in the kernel. > > > > Both HFS and HFS+ have been the default filesystem on MacOS for 30 years > > and I don't think it's justified to introduce such a hard compatibility > > breakage just because some people are worried about theoretical evil > > maid attacks. > > > > HFS/HFS+ mandatory if you want to boot Linux on a classic Mac or PowerMac > > and I don't think it's okay to break all these systems running Linux. > > If they're so popular, then it should be no trouble to find somebody > to volunteer to maintain those filesystems. Except they've been > marked as orphaned since 2011 and effectively were orphaned several > years before that (the last contribution I see from Roman Zippel is > in 2008, and his last contribution to hfs was in 2006). One data point may help.. I've been running Linux on an old PowerMac and an old Intel MacBook since about 2014 or 2015 or so. I have needed the HFS/HFS+ filesystem support for about 9 years now (including that "blessed" support for the Apple Boot partition). There's never been a problem with Linux and the Apple filesystems. Maybe it speaks to the maturity/stability of the code that already exists. The code does not need a lot of attention nowadays. Maybe the orphaned status is the wrong metric to use to determine removal. Maybe a better metric would be installation base. I.e., how many users use the filesystem. Jeff
Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Thu, 2023-07-20 at 18:59 +0100, Matthew Wilcox wrote: > On Thu, Jul 20, 2023 at 07:50:47PM +0200, John Paul Adrian Glaubitz wrote: > > > Then we should delete the HFS/HFS+ filesystems. They're orphaned in > > > MAINTAINERS and if distros are going to do such a damnfool thing, > > > then we must stop them. > > > > Both HFS and HFS+ work perfectly fine. And if distributions or users are so > > sensitive about security, it's up to them to blacklist individual features > > in the kernel. > > > > Both HFS and HFS+ have been the default filesystem on MacOS for 30 years > > and I don't think it's justified to introduce such a hard compatibility > > breakage just because some people are worried about theoretical evil > > maid attacks. > > > > HFS/HFS+ mandatory if you want to boot Linux on a classic Mac or PowerMac > > and I don't think it's okay to break all these systems running Linux. > > If they're so popular, then it should be no trouble to find somebody > to volunteer to maintain those filesystems. Except they've been > marked as orphaned since 2011 and effectively were orphaned several > years before that (the last contribution I see from Roman Zippel is > in 2008, and his last contribution to hfs was in 2006). I suspect that this is one of those catch-22 situations: distros are going to enable every feature under the sun. That doesn't mean that anyone is actually _using_ them these days. Is "staging" still a thing? Maybe we should move these drivers into the staging directory and pick a release where we'll sunset it, and then see who comes out of the woodwork? Cheers, -- Jeff Layton
Re: [syzbot] [hfs?] WARNING in hfs_write_inode
On Thu, Jul 20, 2023 at 07:50:47PM +0200, John Paul Adrian Glaubitz wrote: > > Then we should delete the HFS/HFS+ filesystems. They're orphaned in > > MAINTAINERS and if distros are going to do such a damnfool thing, > > then we must stop them. > > Both HFS and HFS+ work perfectly fine. And if distributions or users are so > sensitive about security, it's up to them to blacklist individual features > in the kernel. > > Both HFS and HFS+ have been the default filesystem on MacOS for 30 years > and I don't think it's justified to introduce such a hard compatibility > breakage just because some people are worried about theoretical evil > maid attacks. > > HFS/HFS+ mandatory if you want to boot Linux on a classic Mac or PowerMac > and I don't think it's okay to break all these systems running Linux. If they're so popular, then it should be no trouble to find somebody to volunteer to maintain those filesystems. Except they've been marked as orphaned since 2011 and effectively were orphaned several years before that (the last contribution I see from Roman Zippel is in 2008, and his last contribution to hfs was in 2006).
