Re: stability matrix

[Chris Mason]

On 09/19/2016 04:36 PM, Christoph Anton Mitterer wrote:

On Mon, 2016-09-19 at 16:07 -0400, Chris Mason wrote:

That's in the blockdev command (blockdev --setro /dev/xxx).

Well, I know that ;-) ... but I bet most end-user don't (just as most
end-users assume mount -r is truly ro)...



It's a tradeoff, without the log replay traditional filesystems wouldn't 
be able to mount at all after a crash.  Since most init systems default 
to ro at the beginning, it would have been awkward to introduce the 
logging filesystems into established systems.


16+ years later, I still feel it's still the path of least surprise.

-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: stability matrix

[Christoph Anton Mitterer]

On Mon, 2016-09-19 at 16:07 -0400, Chris Mason wrote:
> That's in the blockdev command (blockdev --setro /dev/xxx).
Well, I know that ;-) ... but I bet most end-user don't (just as most
end-users assume mount -r is truly ro)...

At least this is nowadays documented at the mount manpage... so in a
way one can of course argue: if the user can't read you can't help him
anyway... :)

Cheers,
Chris.

smime.p7s
Description: S/MIME cryptographic signature

Re: stability matrix

[Chris Mason]

On 09/19/2016 03:52 PM, Christoph Anton Mitterer wrote:

On Mon, 2016-09-19 at 13:18 -0400, Austin S. Hemmelgarn wrote:

- even mounting a fs ro, may cause it to be changed


This would go to the UseCases

My same argument about the UUID issues applies here, just without
the
security aspect.


I personally could agree to have that "just" in the usecases.

That a fs my be changed even though it's mounted ro is not unique to
btrfs and the need for not having that happen goes probably rather
into data-forensics and rescue use cases.

IMO there's rather a general problem, namely that the different
filesystems don't provide a mount option that implies every other mount
options currently needed to get an actual "hard ro", i.e. one where the
device is never written to.

Qu was about to add such option when nologreplay was added, but IIRC he
got some resistance by linux-fs, who probably didn't care enough
whether the end-user can easily do such "hard ro" mount ;)




That's in the blockdev command (blockdev --setro /dev/xxx).

We actually try to maintain the established norms where it doesn't 
conflict with the btrfs use cases.  This is one of them ;)


-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: stability matrix

[Christoph Anton Mitterer]

On Mon, 2016-09-19 at 13:18 -0400, Austin S. Hemmelgarn wrote:
> > > - even mounting a fs ro, may cause it to be changed
> > 
> > This would go to the UseCases
> My same argument about the UUID issues applies here, just without
> the 
> security aspect.

I personally could agree to have that "just" in the usecases.

That a fs my be changed even though it's mounted ro is not unique to
btrfs and the need for not having that happen goes probably rather
into data-forensics and rescue use cases.

IMO there's rather a general problem, namely that the different
filesystems don't provide a mount option that implies every other mount
options currently needed to get an actual "hard ro", i.e. one where the
device is never written to.

Qu was about to add such option when nologreplay was added, but IIRC he
got some resistance by linux-fs, who probably didn't care enough
whether the end-user can easily do such "hard ro" mount ;)


Cheers,
Chris.

smime.p7s
Description: S/MIME cryptographic signature

Re: stability matrix (was: Is stability a joke?)

[Christoph Anton Mitterer]

+1 for all your changes with the following comments in addition...


On Mon, 2016-09-19 at 17:27 +0200, David Sterba wrote:
> That's more like a usecase, thats out of the scope of the tabular
> overview. But we have an existing page UseCases that I'd like to
> transform to a more structured and complete overview of usceases of
> various features, so the UUID collisions would build on top of that
> with
> "and this could hapen if ...".
Well I don't agree here and see it basically like Austin.

It's not that these UUID collisions can only happen in special
circumstances but plain normal situations that always used to work with
probably literally each and every fs. (So much for the accidental
corruptions).

And an attack is probably never "usecase dependant"... it always
depends on the attacker.
And since that seems to be a pretty real attack vector, I'd also say
it's mandatory to quite clearly warn about that deficiency...

TBH, I'm rather surprised that this situation seems to be kinda
"accepted".

I had a chat with CM recently and he implied things might be solved
with encryption.
While this is probably the case for at least some of the described
problems, it rather seems like a workaround:
- why making btrfs-encryption mandatory for devices who have partially
  secured access (e.g. where a systemdisk with btrfs is not physically
  accessible but a USB port is)
- what about users that rather want to use block device encryption
  instead of fs-level-encryption?


> > - in-band dedupe
> >   deduped are IIRC not bitwise compared by the kernel before de-
> > duping,
> >   as it's the case with offline dedupe.
> >   Even if this is considered safe by the community... I think users
> >   should be told.
> Only features merged are reflected. And the out-of-band dedupe does
> full
> memcpy. See btrfs_cmp_data() called from btrfs_extent_same().
Ah,... I kinda thought it was already merged ... possibly got confused
by the countless patch iterations of it ;)


> > - btrfs check --repair (and others?)
> >   Telling people that this may often cause more harm than good.
> I think userspace tools do not belong to the overview.
Well... I wouldn't mind if there was a btrfs-progs status page... (and
both link each other).
OTOH,... the user probably wants one central point where all relevant
info can be found... and not again having to dig through n websites.


> > - even mounting a fs ro, may cause it to be changed
> 
> This would go to the UseCases
Fine for me.


> 
> > 
> > - DB/VM-image like IO patterns + nodatacow + (!)checksumming
> >   + (auto)defrag + snapshots
> >   a)
> >   People typically may have the impression:
> >   btrfs = checksummed => als is guaranteed to be "valid" (or at
> > least
> >   noticed)
> >   However this isn't the case for nodatacow'ed files, which in turn
> > is
> >   kinda "mandatory" for DB/VM-image like IO patterns, cause
> > otherwise
> >   these would fragment to heavily (see (b).
> >   Unless claimed by some people, none of the major DBs or VM-image
> >   formats do general checksumming on their own, most even don't
> > support
> >   it, some that do wouldn't do it without app-support and few
> > "just"
> >   don't do it per default.
> >   Thus one should bump people to this situation and that they may
> > not
> >   get this "correctness" guarantee here.
> >   b)
> >   IIRC, it doesn't even help to simply not use nodatacow on such
> > files
> >   and using auto-defrag instead to countermeasure the fragmenting,
> > as
> >   that one doesn't perform too well on large files.
> 
> Same.
Fine for me either... you already said above you would mention the
nodatacow=>no-checksumming=>no-verification-and-no-raid-repair in the
general section... this is enough for that place.


> > For specific features:
> > - Autodefrag
> >   - didn't that also cause reflinks to be broken up?
> 
> No and never had.

Absolutely sure? One year ago, I was told that at first too so I
started using it, but later on some (IIRC) developer said auto-defrag
would also suffer from it.

> > - RAID*
> >   No userland tools for monitoring/etc.
> 
> That's a usability bug.

Well it is and it will probably go away sooner or later... but the
unaware user may not really realise that he actually has to take care
on this by himself for now.
So I though it would be helpful to have it added.



Best wishes,
Chris.

smime.p7s
Description: S/MIME cryptographic signature

Re: stability matrix

[Austin S. Hemmelgarn]

On 2016-09-19 11:27, David Sterba wrote:

Hi,

On Thu, Sep 15, 2016 at 04:14:04AM +0200, Christoph Anton Mitterer wrote:

In general:
- I think another column should be added, which tells when and for
  which kernel version the feature-status of each row was
  revised/updated the last time and especially by whom.
  If a core dev makes a statement on a particular feature, this
  probably means much more, than if it was made by "just" a list
  regular.


It's going to be revised per release. If there's a bug that affect the
status, the page will be updated. I'm going to do that among other
per-release regular boring tasks.

I'm still not decided if the kernel version will be useful enough, but
if anybody is willing to do the research and fill the table I don't
object.
Moving forwards, I think it's worth it, but I don't feel that it's worth 
looking back at anything before 4.4 to list versions.



  And yes I know, in the beginning it already says "this is for 4.7"...
  but let's be honest, it's pretty likely when this is bumped to 4.8
  that not each and every point will be thoroughly checked again.
- Optionally even one further column could be added, that lists bugs
  where the specific cases are kept record of (if any).


There's a new section under the table to write anything that would not
fit. Mostly pointers to other documentation (manual pages) or bugzilla.


- Perhaps a 3rd Status like "eats-your-data" which is worse than
  critical, e.g. for things were it's known that there is a high
  chance for still getting data corruption (RAID56?)


Perhaps there should be another section that lists general caveats
and pitfalls including:
- defrag/auto-defrag causes ref-link break up (which in turn causes
  possible extensive space being eaten up)


Updated accordingly.


- nodatacow files are not yet[0] checksummed, which in turn means
  that any errors (especially silent data corruption) will not be
  noticed AND which in turn also means the data itself cannot be
  repaired even in case of RAIDs (only the RAIDs are made consistent
  again)


Added to the table.


- subvolume UUID attacks discussed in the recent thread
- fs/device UUID collisions
  - the accidental corruption that can happen in case colliding
fs/device UUIDs appear in a system (and telling the user that
this is e.g. the case when dd'ing and image or using lvm
snapshots, probably also when having btrfs on MD RAID1 or RAID10)
  - the attacks that are possible when UUIDs are known to an attacker


That's more like a usecase, thats out of the scope of the tabular
overview. But we have an existing page UseCases that I'd like to
transform to a more structured and complete overview of usceases of
various features, so the UUID collisions would build on top of that with
"and this could hapen if ...".
I don't agree with this being use case specific.  Whether or not someone 
cares could technically be use case specific, but the use cases where 
this actually doesn't matter is pretty much limited to tight embedded 
systems with no way to attach external storage.  This behavior results 
in both a number of severe security holes for anyone without proper 
physical security (read as 'almost all desktop and laptop users, as well 
as many server admins'), and severe potential for data loss when 
performing normal recovery activities that work on every other filesystem.



- in-band dedupe
  deduped are IIRC not bitwise compared by the kernel before de-duping,
  as it's the case with offline dedupe.
  Even if this is considered safe by the community... I think users
  should be told.


Only features merged are reflected. And the out-of-band dedupe does full
memcpy. See btrfs_cmp_data() called from btrfs_extent_same().


- btrfs check --repair (and others?)
  Telling people that this may often cause more harm than good.


I think userspace tools do not belong to the overview.


- even mounting a fs ro, may cause it to be changed


This would go to the UseCases
My same argument about the UUID issues applies here, just without the 
security aspect.  The only difference here is that it's common behavior 
across most filesystems (but not widely known to most people who aren't 
FS develo9pers or sysops experts).



- DB/VM-image like IO patterns + nodatacow + (!)checksumming
  + (auto)defrag + snapshots
  a)
  People typically may have the impression:
  btrfs = checksummed => als is guaranteed to be "valid" (or at least
  noticed)
  However this isn't the case for nodatacow'ed files, which in turn is
  kinda "mandatory" for DB/VM-image like IO patterns, cause otherwise
  these would fragment to heavily (see (b).
  Unless claimed by some people, none of the major DBs or VM-image
  formats do general checksumming on their own, most even don't support
  it, some that do wouldn't do it without app-support and few "just"
  don't do it per default.
  Thus one should bump people to this situation and that they may not
  get this "correctness" guarantee here.
 

Re: stability matrix (was: Is stability a joke?)

[David Sterba]

Hi,

On Thu, Sep 15, 2016 at 04:14:04AM +0200, Christoph Anton Mitterer wrote:
> In general:
> - I think another column should be added, which tells when and for
>   which kernel version the feature-status of each row was 
>   revised/updated the last time and especially by whom.
>   If a core dev makes a statement on a particular feature, this
>   probably means much more, than if it was made by "just" a list
>   regular.

It's going to be revised per release. If there's a bug that affect the
status, the page will be updated. I'm going to do that among other
per-release regular boring tasks.

I'm still not decided if the kernel version will be useful enough, but
if anybody is willing to do the research and fill the table I don't
object.

>   And yes I know, in the beginning it already says "this is for 4.7"...
>   but let's be honest, it's pretty likely when this is bumped to 4.8
>   that not each and every point will be thoroughly checked again.
> - Optionally even one further column could be added, that lists bugs
>   where the specific cases are kept record of (if any).

There's a new section under the table to write anything that would not
fit. Mostly pointers to other documentation (manual pages) or bugzilla.

> - Perhaps a 3rd Status like "eats-your-data" which is worse than
>   critical, e.g. for things were it's known that there is a high
>   chance for still getting data corruption (RAID56?)
> 
> 
> Perhaps there should be another section that lists general caveats
> and pitfalls including:
> - defrag/auto-defrag causes ref-link break up (which in turn causes
>   possible extensive space being eaten up)

Updated accordingly.

> - nodatacow files are not yet[0] checksummed, which in turn means
>   that any errors (especially silent data corruption) will not be
>   noticed AND which in turn also means the data itself cannot be
>   repaired even in case of RAIDs (only the RAIDs are made consistent
>   again)

Added to the table.

> - subvolume UUID attacks discussed in the recent thread
> - fs/device UUID collisions
>   - the accidental corruption that can happen in case colliding
>     fs/device UUIDs appear in a system (and telling the user that
>     this is e.g. the case when dd'ing and image or using lvm
>     snapshots, probably also when having btrfs on MD RAID1 or RAID10)
>   - the attacks that are possible when UUIDs are known to an attacker

That's more like a usecase, thats out of the scope of the tabular
overview. But we have an existing page UseCases that I'd like to
transform to a more structured and complete overview of usceases of
various features, so the UUID collisions would build on top of that with
"and this could hapen if ...".

> - in-band dedupe
>   deduped are IIRC not bitwise compared by the kernel before de-duping,
>   as it's the case with offline dedupe.
>   Even if this is considered safe by the community... I think users
>   should be told.

Only features merged are reflected. And the out-of-band dedupe does full
memcpy. See btrfs_cmp_data() called from btrfs_extent_same().

> - btrfs check --repair (and others?)
>   Telling people that this may often cause more harm than good.

I think userspace tools do not belong to the overview.

> - even mounting a fs ro, may cause it to be changed

This would go to the UseCases

> - DB/VM-image like IO patterns + nodatacow + (!)checksumming
>   + (auto)defrag + snapshots
>   a)
>   People typically may have the impression:
>   btrfs = checksummed => als is guaranteed to be "valid" (or at least
>   noticed)
>   However this isn't the case for nodatacow'ed files, which in turn is
>   kinda "mandatory" for DB/VM-image like IO patterns, cause otherwise
>   these would fragment to heavily (see (b).
>   Unless claimed by some people, none of the major DBs or VM-image
>   formats do general checksumming on their own, most even don't support
>   it, some that do wouldn't do it without app-support and few "just"
>   don't do it per default.
>   Thus one should bump people to this situation and that they may not
>   get this "correctness" guarantee here.
>   b)
>   IIRC, it doesn't even help to simply not use nodatacow on such files
>   and using auto-defrag instead to countermeasure the fragmenting, as
>   that one doesn't perform too well on large files.

Same.

> For specific features:
> - Autodefrag
>   - didn't that also cause reflinks to be broken up?

No and never had.

> that should be
>     mentioned than as well, as it is (more or less) for defrag and
>     people could then assume it's not the case for autodefrag (which I
>     did initially)
>   - wasn't it said that autodefrag performs bad with files > ~1GB?
>     Perhaps that should be mentioned too
> - defrag
>   "extents get unshared" is IMO not an adequate description for the end
>   user,... it should perhaps link to the defrag article and there
>   explain in detail that any ref-linked files will be broken up, which
>   means space usage will increase, and may especially 

Re: stability matrix

[David Sterba]

On Thu, Sep 15, 2016 at 07:54:26AM -0400, Austin S. Hemmelgarn wrote:
> > I'd like to help creating/maintaining this bug overview. A good start
> > would be to just crawl through all stable kernels and some distro
> > kernels and see which commits show up in fs/btrfs.
> >
> As of right now, we kind of do have such a page:
> https://btrfs.wiki.kernel.org/index.php/Gotchas
> It's not really well labeled though, ans it's easy to overlook.

The page has been created long time ago, if you'd need to start a new
page with similar content I can add a redirect so the link still works.

A more detailed bug page would be welcome by users. The changelogs I
write per release are terse as I don't want to spend the day just on
that. All the information should be in the git log and possibly in
recent mails, so this is manual work to present in on the wiki and does
not need devs to assist.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: stability matrix

[Martin Steigerwald]

Am Donnerstag, 15. September 2016, 07:54:26 CEST schrieb Austin S. Hemmelgarn:
> On 2016-09-15 05:49, Hans van Kranenburg wrote:
> > On 09/15/2016 04:14 AM, Christoph Anton Mitterer wrote:
[…]
> I specifically do not think we should worry about distro kernels though.
>   If someone is using a specific distro, that distro's documentation
> should cover what they support and what works and what doesn't.  Some
> (like Arch and to a lesser extent Gentoo) use almost upstream kernels,
> so there's very little point in tracking them.  Some (like Ubuntu and
> Debian) use almost upstream LTS kernels, so there's little point
> tracking them either.  Many others though (like CentOS, RHEL, and OEL)
> Use forked kernels that have so many back-ported patches that it's
> impossible to track up-date to up-date what the hell they've got.  A
> rather ridiculous expression regarding herding of cats comes to mind
> with respect to the last group.

Yep. I just read through RHEL releasenotes for a RHEL 7 workshop I will hold 
for a customer… and noted that newer RHEL 7 kernels for example have device 
mapper from Kernel 4.1 (while the kernel still says its a 3.10 one), XFS from 
kernel this.that, including new incompat CRC disk format and the need to also 
upgrade xfsprogs in lockstep, and this and that from kernel this.that and so 
on. Frankenstein as an association comes to my mind, but I bet RHEL kernel 
engineers know what they are doing.

-- 
Martin
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: stability matrix

[Chris Murphy]

On Thu, Sep 15, 2016 at 5:54 AM, Austin S. Hemmelgarn
 wrote:
>
>
> I specifically do not think we should worry about distro kernels though.

It will be essentially impossible to keep such a thing up to date.
It's difficult in the best case scenario to even track upstream's own
backports to longterm kernels, whether those would actually even
change anything in the matrix.

I'd say each major version gets it's own page, and just dup the page
for each version.

So for starters, the current page is for version 4.7. If when 4.8 is
released there's no significant change in stability that affects the
color (stability status) of any listed feature, then that page could
say 4.7 through current. If it's true that the status page has no
major changes going back to 4.4 through current, label it that way.

As soon as there's a change that affects the color coding of an item
in the grid, duplicate the page. Old page gets a fixed range of
kernels, say 4.4 to 4.7. And now the newest page is 4.8 - current.

I think a column for version will lose the historical perspective of
when something goes from red to yellow, yellow to green.


> If
> someone is using a specific distro, that distro's documentation should cover
> what they support and what works and what doesn't.  Some (like Arch and to a
> lesser extent Gentoo) use almost upstream kernels, so there's very little
> point in tracking them.  Some (like Ubuntu and Debian) use almost upstream
> LTS kernels, so there's little point tracking them either.  Many others
> though (like CentOS, RHEL, and OEL) Use forked kernels that have so many
> back-ported patches that it's impossible to track up-date to up-date what
> the hell they've got.  A rather ridiculous expression regarding herding of
> cats comes to mind with respect to the last group.

Yeah you need the secret decoder ring to sort it out. Forget it, not worth it.


-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: stability matrix

[Austin S. Hemmelgarn]

On 2016-09-15 05:49, Hans van Kranenburg wrote:

On 09/15/2016 04:14 AM, Christoph Anton Mitterer wrote:

Hey.

As for the stability matrix...

In general:
- I think another column should be added, which tells when and for
  which kernel version the feature-status of each row was
  revised/updated the last time and especially by whom.
  If a core dev makes a statement on a particular feature, this
  probably means much more, than if it was made by "just" a list
  regular.
  And yes I know, in the beginning it already says "this is for 4.7"...
  but let's be honest, it's pretty likely when this is bumped to 4.8
  that not each and every point will be thoroughly checked again.
- Optionally even one further column could be added, that lists bugs
  where the specific cases are kept record of (if any).
- Perhaps a 3rd Status like "eats-your-data" which is worse than
  critical, e.g. for things were it's known that there is a high
  chance for still getting data corruption (RAID56?)


About the "for 4.7" issue... The Status page could have an extra column,
which for every OK labeled row lists the first version (kernel.org x.y.0
release) it's OK for.

The bugs make it more complicated.

* Feature A is labeled OK in kernel 5.0
* During development of kernel 8-rc, an eat my data bug is fixed. The OK
for this feature in the table is bumped to 8.0?
* kernel 5 is EOL
* kernel 6 is still supported, and the fix is applied to 6.12
* then there's distros which have their own old kernels, applying fixes
on them whenever they like, for example 5.6-distro4 which is leading its
own life

"Normal" users are using distro kernels. They shouldn't be panicing
about their data if they're running 6.14 or 5.6-distro4, but the OK in
the table is bumped to 8.0 because of the serious bugs.

At least the official kernels should be tracked in the table I think.

Separately, a list of known serious bugs per feature (like the 4 about
compression, http://www.spinics.net/lists/linux-btrfs/msg58674.html )
could be listed on another Bugs! page (lots of work) so a user, or
someone helping the user can see if the listed commits are or aren't
included in the actual whatever kernel a user is using.

This list of serious bugs could also help disussions that now sound like
"yeah, there were issues with compression which some time ago got fixed,
but noone knows what it was and when, so don't use compression".

Many of the commits which fix serious bugs (even if they're only
triggered in an edge case) have some explanation about how to trigger
them, like the excellent commit messages of Filipe in the commits
mentioned above. This helps setting up and maintaining the bug page, and
helps advanced users to decide if they're hitting the edge case or not
with their usage pattern.

I'd like to help creating/maintaining this bug overview. A good start
would be to just crawl through all stable kernels and some distro
kernels and see which commits show up in fs/btrfs.


As of right now, we kind of do have such a page:
https://btrfs.wiki.kernel.org/index.php/Gotchas
It's not really well labeled though, ans it's easy to overlook.

I specifically do not think we should worry about distro kernels though. 
 If someone is using a specific distro, that distro's documentation 
should cover what they support and what works and what doesn't.  Some 
(like Arch and to a lesser extent Gentoo) use almost upstream kernels, 
so there's very little point in tracking them.  Some (like Ubuntu and 
Debian) use almost upstream LTS kernels, so there's little point 
tracking them either.  Many others though (like CentOS, RHEL, and OEL) 
Use forked kernels that have so many back-ported patches that it's 
impossible to track up-date to up-date what the hell they've got.  A 
rather ridiculous expression regarding herding of cats comes to mind 
with respect to the last group.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: stability matrix

[Hans van Kranenburg]

On 09/15/2016 04:14 AM, Christoph Anton Mitterer wrote:
> Hey.
> 
> As for the stability matrix...
> 
> In general:
> - I think another column should be added, which tells when and for
>   which kernel version the feature-status of each row was 
>   revised/updated the last time and especially by whom.
>   If a core dev makes a statement on a particular feature, this
>   probably means much more, than if it was made by "just" a list
>   regular.
>   And yes I know, in the beginning it already says "this is for 4.7"...
>   but let's be honest, it's pretty likely when this is bumped to 4.8
>   that not each and every point will be thoroughly checked again.
> - Optionally even one further column could be added, that lists bugs
>   where the specific cases are kept record of (if any).
> - Perhaps a 3rd Status like "eats-your-data" which is worse than
>   critical, e.g. for things were it's known that there is a high
>   chance for still getting data corruption (RAID56?)

About the "for 4.7" issue... The Status page could have an extra column,
which for every OK labeled row lists the first version (kernel.org x.y.0
release) it's OK for.

The bugs make it more complicated.

* Feature A is labeled OK in kernel 5.0
* During development of kernel 8-rc, an eat my data bug is fixed. The OK
for this feature in the table is bumped to 8.0?
* kernel 5 is EOL
* kernel 6 is still supported, and the fix is applied to 6.12
* then there's distros which have their own old kernels, applying fixes
on them whenever they like, for example 5.6-distro4 which is leading its
own life

"Normal" users are using distro kernels. They shouldn't be panicing
about their data if they're running 6.14 or 5.6-distro4, but the OK in
the table is bumped to 8.0 because of the serious bugs.

At least the official kernels should be tracked in the table I think.

Separately, a list of known serious bugs per feature (like the 4 about
compression, http://www.spinics.net/lists/linux-btrfs/msg58674.html )
could be listed on another Bugs! page (lots of work) so a user, or
someone helping the user can see if the listed commits are or aren't
included in the actual whatever kernel a user is using.

This list of serious bugs could also help disussions that now sound like
"yeah, there were issues with compression which some time ago got fixed,
but noone knows what it was and when, so don't use compression".

Many of the commits which fix serious bugs (even if they're only
triggered in an edge case) have some explanation about how to trigger
them, like the excellent commit messages of Filipe in the commits
mentioned above. This helps setting up and maintaining the bug page, and
helps advanced users to decide if they're hitting the edge case or not
with their usage pattern.

I'd like to help creating/maintaining this bug overview. A good start
would be to just crawl through all stable kernels and some distro
kernels and see which commits show up in fs/btrfs.

-- 
Hans van Kranenburg
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: stability matrix (was: Is stability a joke?)

[Christoph Anton Mitterer]

Hey.

As for the stability matrix...

In general:
- I think another column should be added, which tells when and for
  which kernel version the feature-status of each row was 
  revised/updated the last time and especially by whom.
  If a core dev makes a statement on a particular feature, this
  probably means much more, than if it was made by "just" a list
  regular.
  And yes I know, in the beginning it already says "this is for 4.7"...
  but let's be honest, it's pretty likely when this is bumped to 4.8
  that not each and every point will be thoroughly checked again.
- Optionally even one further column could be added, that lists bugs
  where the specific cases are kept record of (if any).
- Perhaps a 3rd Status like "eats-your-data" which is worse than
  critical, e.g. for things were it's known that there is a high
  chance for still getting data corruption (RAID56?)


Perhaps there should be another section that lists general caveats
and pitfalls including:
- defrag/auto-defrag causes ref-link break up (which in turn causes
  possible extensive space being eaten up)
- nodatacow files are not yet[0] checksummed, which in turn means
  that any errors (especially silent data corruption) will not be
  noticed AND which in turn also means the data itself cannot be
  repaired even in case of RAIDs (only the RAIDs are made consistent
  again)
- subvolume UUID attacks discussed in the recent thread
- fs/device UUID collisions
  - the accidental corruption that can happen in case colliding
    fs/device UUIDs appear in a system (and telling the user that
    this is e.g. the case when dd'ing and image or using lvm
    snapshots, probably also when having btrfs on MD RAID1 or RAID10)
  - the attacks that are possible when UUIDs are known to an attacker
- in-band dedupe
  deduped are IIRC not bitwise compared by the kernel before de-duping,
  as it's the case with offline dedupe.
  Even if this is considered safe by the community... I think users
  should be told.
- btrfs check --repair (and others?)
  Telling people that this may often cause more harm than good.
- even mounting a fs ro, may cause it to be changed
- DB/VM-image like IO patterns + nodatacow + (!)checksumming
  + (auto)defrag + snapshots
  a)
  People typically may have the impression:
  btrfs = checksummed => als is guaranteed to be "valid" (or at least
  noticed)
  However this isn't the case for nodatacow'ed files, which in turn is
  kinda "mandatory" for DB/VM-image like IO patterns, cause otherwise
  these would fragment to heavily (see (b).
  Unless claimed by some people, none of the major DBs or VM-image
  formats do general checksumming on their own, most even don't support
  it, some that do wouldn't do it without app-support and few "just"
  don't do it per default.
  Thus one should bump people to this situation and that they may not
  get this "correctness" guarantee here.
  b)
  IIRC, it doesn't even help to simply not use nodatacow on such files
  and using auto-defrag instead to countermeasure the fragmenting, as
  that one doesn't perform too well on large files.




For specific features:
- Autodefrag
  - didn't that also cause reflinks to be broken up? that should be
    mentioned than as well, as it is (more or less) for defrag and
    people could then assume it's not the case for autodefrag (which I
    did initially)
  - wasn't it said that autodefrag performs bad with files > ~1GB?
    Perhaps that should be mentioned too
- defrag
  "extents get unshared" is IMO not an adequate description for the end
  user,... it should perhaps link to the defrag article and there
  explain in detail that any ref-linked files will be broken up, which
  means space usage will increase, and may especially explode in case
  of snapshots
- all the RAID56 related points
  wasn't there recently a thread that discussed a more serious bug,
  where parity was wrongly re-calculated which in turn caused actual
  data corruption?
  I think if that's still an issue "write hole still exists, parity
  not checksummed" is not enough but one should emphasize that data may
  easily be corrupted.
- RAID*
  No userland tools for monitoring/etc.
- Device replace 
  IIRC, CM told me that this may cause severe troubles on RAID56


Also, the current matrix talks about "auto-repair"... what's that? (=>
should be IMO explained). 


Last but not least, perhaps this article may also be the place to
document 3rd party things and how far they work stable with btrfs.
For example:
- Which grub version supports booting from it? Which features does it
  [not] support (e.g. which RAIDs, skinny-extents, etc.)?
- Which forensic tools (e.g. things like testdisk) do work with btrfs?
- Which are still maintained/working dedupe userland tools (and are
  they stable?)



Cheers,
Chris.



[0] Yeah I know, a number of list regulars constantly tried to convince
    me that this wasn't possible per se, but a recent discussion I had
    with CM seemed to have revealed (unless I