Re: Plans for BTRFS in Fedora

2011-03-06 Thread Neal Becker
I've noticed scary reports regarding fragmentation on btrfs, some fairly recent 
(within last 6 months).  I'm interested in considering btrfs for my next f15 
install, but should I be concerned about this issue?  Is it expected to be 
resolved?

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-03-04 Thread James Ralston
On 2011-02-26 at 17:33-05 Lyos Gemini Norezel lyos.gemininore...@gmail.com 
wrote:

 On 02/23/2011 06:38 PM, James Ralston wrote:
 
  Separate LVM logical volumes can help mitigate consumption-based
  DoS attacks.
  
  For example: if /tmp and /var/tmp are separate LVM logical
  volumes, then a runaway/malicious process cannot fill up the
  entire filesystem merely by filling up /tmp or /var/tmp.
 
 For the sake of brevity... I already understand the encrypted
 volumes argument... but I still fail to see why /tmp, /var/tmp/,
 /opt, /usr, etc need to have their own partitions.

I mentioned one: any filesystem tree that grants regular users write
access should have some way to prevent DoS attacks.  Making that
subdirectory tree a separate filesystem is one way to do it.

Another reason to isolate user-writable subdirectory trees to separate
filesystems is to make certain types of security attacks more
difficult (by removing the ability of a regular user to create a hard
link to a file).

 The more complex a system is... the more likely it is to fail.

Generally speaking, yes.  But sometimes the benefits provided by the
increased complexity are worth the (negligible in this case, IMHO)
increase in risk.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


BTRFS vs LVM for VM storage (was: Re: Plans for BTRFS in Fedora)

2011-03-02 Thread Richard W.M. Jones
On Tue, Feb 22, 2011 at 02:51:50PM -0500, Josef Bacik wrote:
 2) Fedora 16 ships without LVM as the volume manager and instead use
 BTRFS's built in volume management, again just for the default.

Sorry I'm a bit late on this gentle discussion, but I have one
question about this:

I use LVM to store virtual machines, one VM per LV, and it's very good
for that.

How is BTRFS's performance when used to store VMs (presumably they are
stored as files)?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: BTRFS vs LVM for VM storage (was: Re: Plans for BTRFS in Fedora)

2011-03-02 Thread Josef Bacik
On Wed, Mar 2, 2011 at 9:23 AM, Richard W.M. Jones rjo...@redhat.com wrote:
 On Tue, Feb 22, 2011 at 02:51:50PM -0500, Josef Bacik wrote:
 2) Fedora 16 ships without LVM as the volume manager and instead use
 BTRFS's built in volume management, again just for the default.

 Sorry I'm a bit late on this gentle discussion, but I have one
 question about this:

 I use LVM to store virtual machines, one VM per LV, and it's very good
 for that.

 How is BTRFS's performance when used to store VMs (presumably they are
 stored as files)?


Good, but the problem is the default behavior of virt manager is to
use fsync for everything, you have to manually go in and set the
Cache to None so it will use O_DIRECT, and then it's just as fast
as anything else.  Not a big deal if you create everything via the
command line, kind of annoying if you do it via the GUI, tho all you
have to do is say let me edit the options before starting this vm
when you first create it, set the cache type and then do the install
and you are good to go.  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-28 Thread Jon Masters
On Sat, 2011-02-26 at 17:33 -0500, Lyos Gemini Norezel wrote:
 On 02/23/2011 04:37 PM, Kevin Kofler wrote:
 
  And I'd like to counter-counter-propose that we just stop using ANY kind of
  subvolumes or volume management by default and just default to plain old
  partitions. IMHO, LVM causes more problems than it fixes. Sure, you can
  easily add storage from another disk, but in exchange there's no
  straightforward way to resize your partitions, at least none of the common
  partition editors can do it. There's also a performance penalty.

 +1
 
 This subvolume nonsense has no real place on any home computer/consumer 
 device.

With all due respect, that's the path chosen by certain other Linux
distributions (ones where if I install them I have to jump through all
kinds of hoops to turn on LVM). That is not the way we should be going.
I've made my objections known, added a comment on the wiki discussion
for the feature, and will raise an objection at the appropriate time
that it is proposed to drop LVM use by default. Until then, I'm done :)

Jon.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-28 Thread Josef Bacik
On Mon, Feb 28, 2011 at 1:10 PM, Jon Masters jonat...@jonmasters.org wrote:
 On Sat, 2011-02-26 at 17:33 -0500, Lyos Gemini Norezel wrote:
 On 02/23/2011 04:37 PM, Kevin Kofler wrote:
 
  And I'd like to counter-counter-propose that we just stop using ANY kind of
  subvolumes or volume management by default and just default to plain old
  partitions. IMHO, LVM causes more problems than it fixes. Sure, you can
  easily add storage from another disk, but in exchange there's no
  straightforward way to resize your partitions, at least none of the common
  partition editors can do it. There's also a performance penalty.

 +1

 This subvolume nonsense has no real place on any home computer/consumer
 device.

 With all due respect, that's the path chosen by certain other Linux
 distributions (ones where if I install them I have to jump through all
 kinds of hoops to turn on LVM). That is not the way we should be going.
 I've made my objections known, added a comment on the wiki discussion
 for the feature, and will raise an objection at the appropriate time
 that it is proposed to drop LVM use by default. Until then, I'm done :)


I don't see any comments on the discussion page.  And I'm not talking
about adding extra hoops, you just have to select the custom partition
scheme.  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-28 Thread Przemek Klosowski
On 02/26/2011 05:33 PM, Lyos Gemini Norezel wrote:

 This subvolume nonsense has no real place on any home computer/consumer
 device.
...
 Having more than 3 partitions on ANY system other than production
 servers seems foolish at best.

 To have it as default on a modern operating system is nothing short of
 insanity.

You state these as facts but they are really your opinions, and many 
reasonable people disagree with you to varying degrees. I for instance 
like a logical volume facility, even if it is awkward like LVM. I am 
really looking forward to LVs from BTRFS. Note that when the volume 
management comes as part of the FS, it stops being an issue: if someone 
doesn't like volumes, they may decline to use them, that's all.

I think your arguments would be more effective if you were more precise 
and stated them as your opinions:

I believe that the subvolumes are an overkill on most non-server
systems, for the following reasons:

Greetings

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-26 Thread Lyos Gemini Norezel
On 02/23/2011 04:37 PM, Kevin Kofler wrote:

 And I'd like to counter-counter-propose that we just stop using ANY kind of
 subvolumes or volume management by default and just default to plain old
 partitions. IMHO, LVM causes more problems than it fixes. Sure, you can
 easily add storage from another disk, but in exchange there's no
 straightforward way to resize your partitions, at least none of the common
 partition editors can do it. There's also a performance penalty.

  Kevin Kofler

+1

This subvolume nonsense has no real place on any home computer/consumer 
device.


On 02/23/2011 06:38 PM, James Ralston wrote:
 1.  Separate LVM logical volumes can help mitigate consumption-based
  DoS attacks.

 For example: if /tmp and /var/tmp are separate LVM logical volumes,
 then a runaway/malicious process cannot fill up the entire filesystem
 merely by filling up /tmp or /var/tmp.

Could someone please explain to me what the average home user has for 
a ridiculous amount of partitions?

For the sake of brevity... I already understand the encrypted volumes 
argument... but I still fail to see why /tmp, /var/tmp/, /opt, /usr, etc 
need to have their own partitions.

The more complex a system is... the more likely it is to fail.

Having more than 3 partitions on ANY system other than production 
servers seems foolish at best.

To have it as default on a modern operating system is nothing short of 
insanity.

Lyos Gemini Norezel
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-26 Thread Adam Williamson
On Sat, 2011-02-26 at 17:33 -0500, Lyos Gemini Norezel wrote:

 Having more than 3 partitions on ANY system other than production 
 servers seems foolish at best.
 
 To have it as default on a modern operating system is nothing short of 
 insanity.

I'm not sure why your mail is so strident, because we don't default to
that. The default Fedora layout is either /boot , swap , / or /boot ,
swap , / , and /home . Okay, that last one is four, but only if you
count swap.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-26 Thread Matej Cepl
Dne 27.2.2011 06:51, Adam Williamson napsal(a):
 I'm not sure why your mail is so strident, because we don't default to
 that. The default Fedora layout is either /boot , swap , / or /boot ,
 swap , / , and /home . Okay, that last one is four, but only if you
 count swap.

He confuses mountpoints and partitions and the rant is against numerous
tmpfs mounted directories. That's at least how I understood him, not
that I would care for his argument.

Matěj

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-25 Thread Matěj Cepl
Dne 24.2.2011 20:54, Ric Wheeler napsal(a):
 Can we have pointers to these crashes or BZ reports please? As Josef has
 noted, btrfs has been quite stable in our testing and we are certainly
 going to pursue any reports.

Will do ... I am hesitant to do so, because so many of my previous bug
reports didn't make much difference, and given fsck core dumping (for
the last three years), it seems to me that BTRFS is just nothing I would
take seriously. Yet.

 Just to answer your last question,  we do not intend to slow it down. 
 Rather, we hope to speed it up considerably by adding developers,
 testing and users :)

I meant obviously slowing down accepting BTRFS as default filesystem for
Fedora and throwing away LVM from default, not development of BTRFS itself.

Matěj

-- 
http://www.ceplovi.cz/matej/, Jabber: mceplatceplovi.cz
GPG Finger: 89EF 4BC6 288A BF43 1BAB  25C3 E09F EF25 D964 84AC




smime.p7s
Description: S/MIME Cryptographic Signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-25 Thread Peter Robinson
On Thu, Feb 24, 2011 at 7:54 PM, Ric Wheeler rwhee...@redhat.com wrote:
 On 02/24/2011 08:44 AM, Matej Cepl wrote:
 Dne 23.2.2011 20:49, Matthew Garrett napsal(a):
 btrfs does the former without anywhere near as much of the latter.
 BTRFS so far only makes my kernel panicking as it did anytime I have
 been trying it since Fedora 9 (yes, I am crazy). This is absolutely not
 meant as anything personal against Josef (I know very well how
 incredibly small group are BTRFS developers), but just a bit of
 suspicion, whether we have fsck now (or we will have fsck soon) really
 leads so quickly let's make it default.

 I am quite OK with having crashing and unstable systemd or Gnome 3 (and
 again, nothing against their developers, this is Rawhide and Fedora, so
 when my kids are alive despite me using it I am pretty happy), but
 unstable file system is quite a different matter.

 Could we slow down a bit, please?

 Matěj

 Can we have pointers to these crashes or BZ reports please? As Josef has 
 noted,
 btrfs has been quite stable in our testing and we are certainly going to 
 pursue
 any reports.

 Also note that the btrfs community of developers is not so small these days 
 and
 rivals (if not surpasses) the size of the team working on ext4.

 Just to answer your last question,  we do not intend to slow it down.  
 Rather,
 we hope to speed it up considerably by adding developers, testing and users :)

I've seen a number of crashes using 2.6.37 on a Dell 6410 using btrfs
in a luks encrypted LVM volume. Sometimes its a message in dmesg,
other times an out right crash. Each time it happens I submit the
kernel oops using abrt, but unlike RHBZ reports you don't get a URL
for the report so I have no idea where they get reported to but it
might be worthwhile reviewing that information where ever it ends up.

Peter
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-25 Thread Ric Wheeler
On 02/25/2011 04:06 AM, Peter Robinson wrote:
 On Thu, Feb 24, 2011 at 7:54 PM, Ric Wheelerrwhee...@redhat.com  wrote:
 On 02/24/2011 08:44 AM, Matej Cepl wrote:
 Dne 23.2.2011 20:49, Matthew Garrett napsal(a):
 btrfs does the former without anywhere near as much of the latter.
 BTRFS so far only makes my kernel panicking as it did anytime I have
 been trying it since Fedora 9 (yes, I am crazy). This is absolutely not
 meant as anything personal against Josef (I know very well how
 incredibly small group are BTRFS developers), but just a bit of
 suspicion, whether we have fsck now (or we will have fsck soon) really
 leads so quickly let's make it default.

 I am quite OK with having crashing and unstable systemd or Gnome 3 (and
 again, nothing against their developers, this is Rawhide and Fedora, so
 when my kids are alive despite me using it I am pretty happy), but
 unstable file system is quite a different matter.

 Could we slow down a bit, please?

 Matěj
 Can we have pointers to these crashes or BZ reports please? As Josef has 
 noted,
 btrfs has been quite stable in our testing and we are certainly going to 
 pursue
 any reports.

 Also note that the btrfs community of developers is not so small these days 
 and
 rivals (if not surpasses) the size of the team working on ext4.

 Just to answer your last question,  we do not intend to slow it down.  
 Rather,
 we hope to speed it up considerably by adding developers, testing and users 
 :)
 I've seen a number of crashes using 2.6.37 on a Dell 6410 using btrfs
 in a luks encrypted LVM volume. Sometimes its a message in dmesg,
 other times an out right crash. Each time it happens I submit the
 kernel oops using abrt, but unlike RHBZ reports you don't get a URL
 for the report so I have no idea where they get reported to but it
 might be worthwhile reviewing that information where ever it ends up.

 Peter

I think that it is probably best to report issues to the linux-btrfs list where 
the developers are. If you report them via bugzilla, we will see them directly 
there as well.

Seems that we need to figure out where these abrt generated BZ's go, I have not 
seen them come in via our normal bugzilla reports but might need to figure out 
how to do specific queries for them.

Thanks!

Ric

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-25 Thread Peter Robinson
On Fri, Feb 25, 2011 at 1:31 PM, Ric Wheeler rwhee...@redhat.com wrote:
 On 02/25/2011 04:06 AM, Peter Robinson wrote:

 On Thu, Feb 24, 2011 at 7:54 PM, Ric Wheelerrwhee...@redhat.com  wrote:

 On 02/24/2011 08:44 AM, Matej Cepl wrote:

 Dne 23.2.2011 20:49, Matthew Garrett napsal(a):

 btrfs does the former without anywhere near as much of the latter.

 BTRFS so far only makes my kernel panicking as it did anytime I have
 been trying it since Fedora 9 (yes, I am crazy). This is absolutely not
 meant as anything personal against Josef (I know very well how
 incredibly small group are BTRFS developers), but just a bit of
 suspicion, whether we have fsck now (or we will have fsck soon) really
 leads so quickly let's make it default.

 I am quite OK with having crashing and unstable systemd or Gnome 3 (and
 again, nothing against their developers, this is Rawhide and Fedora, so
 when my kids are alive despite me using it I am pretty happy), but
 unstable file system is quite a different matter.

 Could we slow down a bit, please?

 Matěj

 Can we have pointers to these crashes or BZ reports please? As Josef has
 noted,
 btrfs has been quite stable in our testing and we are certainly going to
 pursue
 any reports.

 Also note that the btrfs community of developers is not so small these
 days and
 rivals (if not surpasses) the size of the team working on ext4.

 Just to answer your last question,  we do not intend to slow it down.
  Rather,
 we hope to speed it up considerably by adding developers, testing and
 users :)

 I've seen a number of crashes using 2.6.37 on a Dell 6410 using btrfs
 in a luks encrypted LVM volume. Sometimes its a message in dmesg,
 other times an out right crash. Each time it happens I submit the
 kernel oops using abrt, but unlike RHBZ reports you don't get a URL
 for the report so I have no idea where they get reported to but it
 might be worthwhile reviewing that information where ever it ends up.

 Peter

 I think that it is probably best to report issues to the linux-btrfs list
 where the developers are. If you report them via bugzilla, we will see them
 directly there as well.

 Seems that we need to figure out where these abrt generated BZ's go, I have
 not seen them come in via our normal bugzilla reports but might need to
 figure out how to do specific queries for them.

I think the kernel ones get submitted here http://kerneloops.org/ but
if not you'd have to look closer at the abrt-addon-kerneloops for
details on where its sent.

Peter
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-25 Thread Ric Wheeler
On 02/25/2011 08:52 AM, Peter Robinson wrote:
 On Fri, Feb 25, 2011 at 1:31 PM, Ric Wheelerrwhee...@redhat.com  wrote:
 On 02/25/2011 04:06 AM, Peter Robinson wrote:
 On Thu, Feb 24, 2011 at 7:54 PM, Ric Wheelerrwhee...@redhat.comwrote:
 On 02/24/2011 08:44 AM, Matej Cepl wrote:
 Dne 23.2.2011 20:49, Matthew Garrett napsal(a):
 btrfs does the former without anywhere near as much of the latter.
 BTRFS so far only makes my kernel panicking as it did anytime I have
 been trying it since Fedora 9 (yes, I am crazy). This is absolutely not
 meant as anything personal against Josef (I know very well how
 incredibly small group are BTRFS developers), but just a bit of
 suspicion, whether we have fsck now (or we will have fsck soon) really
 leads so quickly let's make it default.

 I am quite OK with having crashing and unstable systemd or Gnome 3 (and
 again, nothing against their developers, this is Rawhide and Fedora, so
 when my kids are alive despite me using it I am pretty happy), but
 unstable file system is quite a different matter.

 Could we slow down a bit, please?

 Matěj
 Can we have pointers to these crashes or BZ reports please? As Josef has
 noted,
 btrfs has been quite stable in our testing and we are certainly going to
 pursue
 any reports.

 Also note that the btrfs community of developers is not so small these
 days and
 rivals (if not surpasses) the size of the team working on ext4.

 Just to answer your last question,  we do not intend to slow it down.
   Rather,
 we hope to speed it up considerably by adding developers, testing and
 users :)
 I've seen a number of crashes using 2.6.37 on a Dell 6410 using btrfs
 in a luks encrypted LVM volume. Sometimes its a message in dmesg,
 other times an out right crash. Each time it happens I submit the
 kernel oops using abrt, but unlike RHBZ reports you don't get a URL
 for the report so I have no idea where they get reported to but it
 might be worthwhile reviewing that information where ever it ends up.

 Peter
 I think that it is probably best to report issues to the linux-btrfs list
 where the developers are. If you report them via bugzilla, we will see them
 directly there as well.

 Seems that we need to figure out where these abrt generated BZ's go, I have
 not seen them come in via our normal bugzilla reports but might need to
 figure out how to do specific queries for them.
 I think the kernel ones get submitted here http://kerneloops.org/ but
 if not you'd have to look closer at the abrt-addon-kerneloops for
 details on where its sent.

 Peter

Not sure who monitors all kernel oops reports, but I personally don't see them. 
If you have a btrfs issue (or other issue with fedora file systems), feel free 
to drop me an email to make sure that we know about it so we can have a look at 
it :)

ric

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-25 Thread Eric Sandeen
On 2/25/11 2:54 AM, Matěj Cepl wrote:
 Dne 24.2.2011 20:54, Ric Wheeler napsal(a):
 Can we have pointers to these crashes or BZ reports please? As Josef has
 noted, btrfs has been quite stable in our testing and we are certainly
 going to pursue any reports.
 
 Will do ... I am hesitant to do so, because so many of my previous bug
 reports didn't make much difference,

Do you have pointers to those?  Were they on the kernel.org bugzilla, or
the red hat bugzilla, the list, or elsewhere?

Thanks,
-Eric

  and given fsck core dumping (for
 the last three years), it seems to me that BTRFS is just nothing I would
 take seriously. Yet.
 
 Just to answer your last question,  we do not intend to slow it down. 
 Rather, we hope to speed it up considerably by adding developers,
 testing and users :)
 
 I meant obviously slowing down accepting BTRFS as default filesystem for
 Fedora and throwing away LVM from default, not development of BTRFS itself.
 
 Matěj
 
 

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-25 Thread Matthew Miller
On Thu, Feb 24, 2011 at 02:25:26PM +0100, Lennart Poettering wrote:
  snapshotted every time we perform a package/admin operation (and
  perhaps also just on regular intervals for good measure), what would
  we then gain by adding a read-only rootfs to the mix?
 Security, robustness: you can be sure that nothing tempers with your
 basic OS tree and it is always in a defined state, unless put in a
 specific admin mode, where the image may be changed/administered,
 i.e. / is remounted rw.

It'd be nice to support a separate /usr in this case as well, because
changes to /etc are usually a different use-case than changes to /usr -- the
former is administrator configuration actions, and the latter almost
exclusively package updates, installations, or removals. (Installing
packages may or may not also entail changes to /etc, of course.)

-- 
Matthew Miller mat...@mattdm.org
Senior Systems Architect -- Instructional  Research Computing Services
Harvard School of Engineering  Applied Sciences
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-24 Thread Matej Cepl
Dne 23.2.2011 20:49, Matthew Garrett napsal(a):
 btrfs does the former without anywhere near as much of the latter.

BTRFS so far only makes my kernel panicking as it did anytime I have
been trying it since Fedora 9 (yes, I am crazy). This is absolutely not
meant as anything personal against Josef (I know very well how
incredibly small group are BTRFS developers), but just a bit of
suspicion, whether we have fsck now (or we will have fsck soon) really
leads so quickly let's make it default.

I am quite OK with having crashing and unstable systemd or Gnome 3 (and
again, nothing against their developers, this is Rawhide and Fedora, so
when my kids are alive despite me using it I am pretty happy), but
unstable file system is quite a different matter.

Could we slow down a bit, please?

Matěj

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-24 Thread Peter Jones
On 02/23/2011 06:01 PM, James Ralston wrote:
 On 2011-02-23 at 13:41-05 Peter Jonespjo...@redhat.com  wrote:
 dm-crypt still just throws REQ_FLUSH away instead of figuring out
 the block remaps involved and issuing the right bios.  Of course,
 this is a problem with dm-crypt and _any_ filesystem.
 
 Are you sure that's still the case?

Well, when I said So, don't hold me to this, but it looks as if... in
the sentence before the one you quoted, I was attempting to imply that
I was not, in fact, sure, and had only done the most trivial read of the
code in question.

 Because this patchset appears to add REQ_FLUSH support to dm:
 
 http://www.redhat.com/archives/dm-devel/2010-August/msg00303.html
 
 ...and that patchset is included in 2.6.38-rc6.

The second patch in that patchset is the one I referenced in the email
you're replying to. But I guess I misread it and it does seem to say it
fixes dm-crypt as well.

-- 
Peter

The Shuttle is now going five times the sound of speed.
-- Dan Rather, first landing of Columbia
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-24 Thread James Ralston
On 2011-02-23 at 23:32-06 Michael Cronenworth m...@cchtml.com wrote:

 On 02/23/2011 05:38 PM, James Ralston wrote:
 
  None of these issues is a dealbreaker, but they *are* losses of
  functionality versus what LVM offers.
 
 LVM isn't going anywhere. It just won't be the default during a
 fresh installation, which you would still be free to override by
 using an LVM again if you wished.

But I *don't* wish to discard btrfs and continue to use LVM.

What I want to do is *replace* my use of LVM with btrfs.

But to do that, I need to either figure out how to make btrfs do the
things I currently do with LVM, or I need to figure out how to change
the things I do to better leverage the capabilities of btrfs.  I'm
open to the former, the latter, or some combination of both.  But
right now, I've been unable to find enough information about btrfs to
do either.

A third thing that's important to me (which others have already
mentioned): disks for virtual guests.  With LVM, I simply lvcreate a
new volume, and then when I create a new guest with virt-manager, I
tell it to use the block device for the LVM logical volume.  This
avoids having a filesystem layer between the virtual guest and the LVM
block device, which I've assumed (but have not actually tested) is a
performance win.

But in the btrfs is your entire disk model, it seems that my only
option is to create a big file on the filesystem, and tell
virt-manager to use that as the guest's disk.  Even if I create the
big file efficiently (i.e., with fallocate(2)), how much of a
performance hit will I take by having the guest's disk I/O go through
the btrfs filesystem layer, versus the LVM layer?  A little?  A lot?
None at all, because it'll actually be a performance win?  I have no
idea.  And furthermore, I don't know if there's a better way to do it.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-24 Thread Ric Wheeler
On 02/24/2011 08:44 AM, Matej Cepl wrote:
 Dne 23.2.2011 20:49, Matthew Garrett napsal(a):
 btrfs does the former without anywhere near as much of the latter.
 BTRFS so far only makes my kernel panicking as it did anytime I have
 been trying it since Fedora 9 (yes, I am crazy). This is absolutely not
 meant as anything personal against Josef (I know very well how
 incredibly small group are BTRFS developers), but just a bit of
 suspicion, whether we have fsck now (or we will have fsck soon) really
 leads so quickly let's make it default.

 I am quite OK with having crashing and unstable systemd or Gnome 3 (and
 again, nothing against their developers, this is Rawhide and Fedora, so
 when my kids are alive despite me using it I am pretty happy), but
 unstable file system is quite a different matter.

 Could we slow down a bit, please?

 Matěj

Can we have pointers to these crashes or BZ reports please? As Josef has noted, 
btrfs has been quite stable in our testing and we are certainly going to pursue 
any reports.

Also note that the btrfs community of developers is not so small these days and 
rivals (if not surpasses) the size of the team working on ext4.

Just to answer your last question,  we do not intend to slow it down.  
Rather, 
we hope to speed it up considerably by adding developers, testing and users :)

Ric





-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-24 Thread Josef Bacik
On Thu, Feb 24, 2011 at 8:44 AM, Matej Cepl mc...@redhat.com wrote:
 Dne 23.2.2011 20:49, Matthew Garrett napsal(a):
 btrfs does the former without anywhere near as much of the latter.

 BTRFS so far only makes my kernel panicking as it did anytime I have
 been trying it since Fedora 9 (yes, I am crazy). This is absolutely not
 meant as anything personal against Josef (I know very well how
 incredibly small group are BTRFS developers), but just a bit of
 suspicion, whether we have fsck now (or we will have fsck soon) really
 leads so quickly let's make it default.

 I am quite OK with having crashing and unstable systemd or Gnome 3 (and
 again, nothing against their developers, this is Rawhide and Fedora, so
 when my kids are alive despite me using it I am pretty happy), but
 unstable file system is quite a different matter.

 Could we slow down a bit, please?

Sure but where's the fun in that? :).

Seriously though, when I'm not on RHEL duty most of my time has been
spent on stability vs. adding features, and honestly there aren't a
lot of OMFG this shit is broken complaints, it's more of hey when I
do this thing that is specific to my work load, BTRFS does X wrong.
At this point what is left is the normal day to day users using this
thing and breaking it in ways that we as developers have not been able
to break it using it the way we use it and test it the way we test it.
 I think that if I could get a large base to test for F15 that we
could squash most/all of the problems that crop up from that to be in
great shape for default in F16.  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-24 Thread James Ralston
On 2011-02-24 at 16:02-05 Josef Bacik jo...@toxicpanda.com wrote:
 I think that if I could get a large base to test for F15 that we
 could squash most/all of the problems that crop up from that to be
 in great shape for default in F16.

I think you'd increase your chances of getting lots of testers for
F15+btrfs if there were documentation that explained how to take
certain LVM configurations/actions and translate them into btrfs
best practices.

For examples, see my other follow-ups in this thread...

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Jóhann B. Guðmundsson
On 02/23/2011 01:26 AM, Josef Bacik wrote:
 Various things, better data integrity to start with, and if you
 install the yum-fs-snapshot you have the ability to rollback easily.

So we got the above + What Lennart mentioned as benefits to the end user.

Now if we continue to hang on to the outdated concept ( From my 
perspective ) of defaults as opposed to allowing each SIG to control 
and set it's own defaults that suits their target end user base.

For example any of the *DE might not want to switch to BTRFS until all 
GUI tool for their end user to take advantage of what it has to offer 
are in place while the Server SIG might certainly do.

Hence I propose that we open a ticket with FESCO which will request them 
to do something like this.

Review the current filesystem default

Gather feed back from various SIG with in the community to create a good 
criteria for what needs to be in place in a filesystem for it to be 
considered as an a default for Fedora.

Gather a list of all existing linux filesystems ( there seems to be a 
new one popping up every other month or so ).

Filter out those that do not live up to that criteria and any base line 
they set for their decision like must not perform worse then the 
current default must work well on SSD's etc..  settle on one that 
benefits most likely most the novice end users.
( We experienced users are able to choose tweak and turn any knob at 
install time to suit our need which makes the novice end user the lowest 
dominator)

JBG
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Josh Boyer
On Tue, Feb 22, 2011 at 10:25 PM, Jon Masters jonat...@jonmasters.org wrote:
 On Tue, 2011-02-22 at 14:51 -0500, Josef Bacik wrote:

 2) Fedora 16 ships without LVM as the volume manager and instead use
 BTRFS's built in volume management, again just for the default.

 In my personal opinion, this is a poor design decision. Yes, BTRFS can
 do a lot of volume-y things, and these are growing by the day, but I
 don't want my filesystem replacing a full volume manager and I am
 concerned that this will lead to less testing and exposure to full LVM
 use within the Fedora community. Instead, I'd like to counter-propose
 that everything stay exactly as it is, with users being able to elect to
 switch to BTRFS (sub)volumes if they are interested in doing so.

 Should the switch to BTRFS by default happen, this will be one more
 thing I will have to fix immediately during installation. The list grows
 longer and longer over time - please don't make this change.

You seem to spend a lot of time during your installs undoing all the
new things that were done for the release.  Perhaps a rapid changing,
bleeding-edge distribution isn't quite suited to your needs.  Maybe
you would be more comfortable with Debian or CentOS?

josh
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Lennart Poettering
On Tue, 22.02.11 22:25, Jon Masters (jonat...@jonmasters.org) wrote:

 
 On Tue, 2011-02-22 at 14:51 -0500, Josef Bacik wrote:
 
  2) Fedora 16 ships without LVM as the volume manager and instead use
  BTRFS's built in volume management, again just for the default.
 
 In my personal opinion, this is a poor design decision. Yes, BTRFS can
 do a lot of volume-y things, and these are growing by the day, but I
 don't want my filesystem replacing a full volume manager and I am
 concerned that this will lead to less testing and exposure to full LVM
 use within the Fedora community. Instead, I'd like to counter-propose
 that everything stay exactly as it is, with users being able to elect to
 switch to BTRFS (sub)volumes if they are interested in doing so.
 
 Should the switch to BTRFS by default happen, this will be one more
 thing I will have to fix immediately during installation. The list grows
 longer and longer over time - please don't make this change.

Aren't you exaggerating your conservatism a bit?

Are there actually new Fedora features you do support? The only signal
you appear to be sending all the time is NO!.

Lennart

-- 
Lennart Poettering - Red Hat, Inc.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Josef Bacik
On Tue, Feb 22, 2011 at 10:25 PM, Jon Masters jonat...@jonmasters.org wrote:
 On Tue, 2011-02-22 at 14:51 -0500, Josef Bacik wrote:

 2) Fedora 16 ships without LVM as the volume manager and instead use
 BTRFS's built in volume management, again just for the default.

 In my personal opinion, this is a poor design decision. Yes, BTRFS can
 do a lot of volume-y things, and these are growing by the day, but I
 don't want my filesystem replacing a full volume manager and I am
 concerned that this will lead to less testing and exposure to full LVM
 use within the Fedora community. Instead, I'd like to counter-propose
 that everything stay exactly as it is, with users being able to elect to
 switch to BTRFS (sub)volumes if they are interested in doing so.

 Should the switch to BTRFS by default happen, this will be one more
 thing I will have to fix immediately during installation. The list grows
 longer and longer over time - please don't make this change.


Theres no sense in using LVM if BTRFS can do the same job better.  I'm
not suggesting removing LVM altogether, just not using it for the
default pick the layout for me option, you can always do your own
thing with a custom layout.

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Josef Bacik
On Tue, Feb 22, 2011 at 11:57 PM, Bruno Wolff III br...@wolff.to wrote:
 On Tue, Feb 22, 2011 at 14:51:50 -0500,
  Josef Bacik jo...@toxicpanda.com wrote:

 3) All the various little tools that we have for putting together
 LiveCD's that are very ext* centered.  I've not even looked at this
 yet, but I assume it's going to be kind of a pain.

 I like to see live CDs just use squashfs directly and not a squashed copy
 of an ext3/4 image. However this will break using dd to copy the image
 for installs which will slow things down. Further however, there is at least
 some interest in having live CDs and ananconda share the same install method,
 so this might be happening in any case.


This would be a great thing in general since the default ext* image is
shrunk down to be installed which creates a bad fs layout which has
performance implications.  Sure it may be faster for the install, but
overall it hurts our users.  So moving away from this dd an image
thing would be good for everybody.  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread drago01
On Wed, Feb 23, 2011 at 4:25 AM, Jon Masters jonat...@jonmasters.org wrote:
 On Tue, 2011-02-22 at 14:51 -0500, Josef Bacik wrote:

 2) Fedora 16 ships without LVM as the volume manager and instead use
 BTRFS's built in volume management, again just for the default.

 In my personal opinion, this is a poor design decision. Yes, BTRFS can
 do a lot of volume-y things, and these are growing by the day, but I
 don't want my filesystem replacing a full volume manager and I am
 concerned that this will lead to less testing and exposure to full LVM
 use within the Fedora community. Instead, I'd like to counter-propose
 that everything stay exactly as it is, with users being able to elect to
 switch to BTRFS (sub)volumes if they are interested in doing so.

 Should the switch to BTRFS by default happen, this will be one more
 thing I will have to fix immediately during installation. The list grows
 longer and longer over time - please don't make this change.

Oh god changes how dare we even think about those evil things!

Now seriously we cannot make progress when every time a (bigger)
change is proposed people start screaming but this changes what we
had before,
and regarding this particular case I think LVM by default has pretty
much always been a bad idea anyway.
And it is not like installing an OS is something you do everyday so
you install your OS, do the changes you think are necessary and be
done with it.
Defaults should be chooses on the metric what provides the best
experience for the users not based on what we have been doing in the
past (i.e stagnation).
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Camilo Mesias
Hi

I wanted to second these questions...

2011/2/22 Jóhann B. johan...@gmail.com:
 Will there be any performance penalties making this move?
[...]
 What benefit will this switch bring to the novice desktop end users?

 Will the novice desktop end user ever take advantages of any of the features 
 that btrfs brings?

Since upgrading (downgrading?) my netbook to use an SSD I went with
the Anaconda defaults (using LVM etc) and that probably wasn't in my
best interest - judging from benchmarks at the time of read
performance in the LVM compared to the underlying device, also from
the point of not being able to add complex disk arrays to the netbook
any time soon.

I think Fedora could do more to support lower end devices *well*, in
addition to allowing people to use the very latest technology on
larger (ie. desktop and server) platforms. My impression right now is
I'd be interested to try BTRFS for the server and maybe larger
desktops, but would probably want to avoid it for anything smaller.

-Cam
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Josef Bacik
On Wed, Feb 23, 2011 at 8:29 AM, Camilo Mesias cam...@mesias.co.uk wrote:
 Hi

 I wanted to second these questions...

 2011/2/22 Jóhann B. johan...@gmail.com:
 Will there be any performance penalties making this move?
 [...]
 What benefit will this switch bring to the novice desktop end users?

 Will the novice desktop end user ever take advantages of any of the features 
 that btrfs brings?

 Since upgrading (downgrading?) my netbook to use an SSD I went with
 the Anaconda defaults (using LVM etc) and that probably wasn't in my
 best interest - judging from benchmarks at the time of read
 performance in the LVM compared to the underlying device, also from
 the point of not being able to add complex disk arrays to the netbook
 any time soon.

 I think Fedora could do more to support lower end devices *well*, in
 addition to allowing people to use the very latest technology on
 larger (ie. desktop and server) platforms. My impression right now is
 I'd be interested to try BTRFS for the server and maybe larger
 desktops, but would probably want to avoid it for anything smaller.


Your impression is wrong, there has been quite a bit of work done to
make BTRFS work well on small devices, it is the default filesystem
for meego which goes on phones, which is much smaller than anything
you are going to have on your netbook.  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Camilo Mesias
Josef,

On Wed, Feb 23, 2011 at 1:42 PM, Josef Bacik jo...@toxicpanda.com wrote:
 Your impression is wrong, there has been quite a bit of work done to
 make BTRFS work well on small devices, it is the default filesystem
 for meego which goes on phones, which is much smaller than anything
 you are going to have on your netbook.  Thanks,

thanks for the info, I will be sure to test it when it is available.
My impression was based on a quick read of the BTRFS FAQ. I saw more
related to problems of limited disk space than to SSD support (but
granted, what mention there is sounds promising). I will test.

-Cam
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Peter Jones
On 02/22/2011 10:25 PM, Jon Masters wrote:
 On Tue, 2011-02-22 at 14:51 -0500, Josef Bacik wrote:
 
 2) Fedora 16 ships without LVM as the volume manager and instead use
 BTRFS's built in volume management, again just for the default.
 
 In my personal opinion, this is a poor design decision. Yes, BTRFS can
 do a lot of volume-y things, and these are growing by the day, but I
 don't want my filesystem replacing a full volume manager and I am
 concerned that this will lead to less testing and exposure to full LVM
 use within the Fedora community. Instead, I'd like to counter-propose
 that everything stay exactly as it is, with users being able to elect to
 switch to BTRFS (sub)volumes if they are interested in doing so.

I think this is more than a little bit over the top, but there are some
legitimate concerns we do need to keep track of and figure out solutions
to.  Off the top of my head, we need to figure out what we're doing with
encrypted volumes:

1) can btrfs do encrypted volumes?  Or, somewhat weirder but possibly better,
   encrypted volumes with non-encrypted subvolumes?
   a) if encrypted volumes but not plaintext subvolumes, then we need /boot
  split out to a separate physical partition, maybe
   b) if it can do encrypted subvolumes of non-encrypted volumes, we
  could make a /boot and a /sysroot and mount /sysroot as / I guess
   c) if no encrypted volumes, or if that functionality won't land at the
  same time as the basic parts, then we need to talk about UI for disk
  encryption:
  I) keeping lvm when encrypt data is checked in the anaconda UI,
 so we can continue to use luks on a VG in those cases.
  II) it may be worth investigating identifying machine classes during
  installation and varying the defaults based on them.  We talk about
  doing this every 3 or 4 releases for various reasons and usually
  decide it's a terrible idea and do something else.

 Should the switch to BTRFS by default happen, this will be one more
 thing I will have to fix immediately during installation. The list grows
 longer and longer over time - please don't make this change.

I think you're confusing necessity and desire here.

-- 
Peter

Computers don't make errors.  What they do, they do on purpose.
-- Dale
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread John Reiser
On 02/23/2011 05:07 AM, drago01 wrote:
 Defaults should be chooses on the metric what provides the best
 experience for the users not based on what we have been doing in the
 past (i.e stagnation).

*One* data corruption constitutes EPIC FAIL.  Btrfs is too young,
and will be for yet a while longer.

-- 
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Josef Bacik
On Wed, Feb 23, 2011 at 9:18 AM, John Reiser jrei...@bitwagon.com wrote:
 On 02/23/2011 05:07 AM, drago01 wrote:
 Defaults should be chooses on the metric what provides the best
 experience for the users not based on what we have been doing in the
 past (i.e stagnation).

 *One* data corruption constitutes EPIC FAIL.  Btrfs is too young,
 and will be for yet a while longer.


Well if data corruption is the test then we shouldn't be using Ext4
either, there was one fixed as recently as the beginning of this
month.  File systems are software like anything else, there will be
bugs.  Off the top of my head I can think of 3 data corrupters we've
had in 4 years of working on BTRFS, and they've all been hard to hit
and have not to my knowledge been seen by users, only us developers in
testing.  BTRFS is young, but we have to start somewhere.  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Nathaniel McCallum
On Wed, 2011-02-23 at 09:27 -0500, Josef Bacik wrote:
 On Wed, Feb 23, 2011 at 9:18 AM, John Reiser jrei...@bitwagon.com wrote:
  On 02/23/2011 05:07 AM, drago01 wrote:
  Defaults should be chooses on the metric what provides the best
  experience for the users not based on what we have been doing in the
  past (i.e stagnation).
 
  *One* data corruption constitutes EPIC FAIL.  Btrfs is too young,
  and will be for yet a while longer.
 
 
 Well if data corruption is the test then we shouldn't be using Ext4
 either, there was one fixed as recently as the beginning of this
 month.  File systems are software like anything else, there will be
 bugs.  Off the top of my head I can think of 3 data corrupters we've
 had in 4 years of working on BTRFS, and they've all been hard to hit
 and have not to my knowledge been seen by users, only us developers in
 testing.  BTRFS is young, but we have to start somewhere.  Thanks,

From a user's perspective, I've been using btrfs for about 1.5 years on
multiple computers and I've been very happy (particularly on my netbook
where the transparent compression increases the disk writes
considerably).  I had one small problem where btrfs wouldn't mount, but
by booting off of a newer kernel I had no problems.

Thanks for your hard work on btrfs everyone!

Nathaniel

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Dennis Jacobfeuerborn
On 02/23/2011 03:27 PM, Josef Bacik wrote:
 On Wed, Feb 23, 2011 at 9:18 AM, John Reiserjrei...@bitwagon.com  wrote:
 On 02/23/2011 05:07 AM, drago01 wrote:
 Defaults should be chooses on the metric what provides the best
 experience for the users not based on what we have been doing in the
 past (i.e stagnation).

 *One* data corruption constitutes EPIC FAIL.  Btrfs is too young,
 and will be for yet a while longer.


 Well if data corruption is the test then we shouldn't be using Ext4
 either, there was one fixed as recently as the beginning of this
 month.  File systems are software like anything else, there will be
 bugs.  Off the top of my head I can think of 3 data corrupters we've
 had in 4 years of working on BTRFS, and they've all been hard to hit
 and have not to my knowledge been seen by users, only us developers in
 testing.  BTRFS is young, but we have to start somewhere.  Thanks,

I'm actually not that worried about corruption as that is something that 
can be fixed once discovered. What creeps me out about btrfs at the moment 
is this:

https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21__Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21

The fact that the FS needs manual rebalance operations and that these can 
take a while (even tough this can be done online) doesn't exactly make 
btrfs the ideal candidate for an end-user desktop system that should pretty 
much be able to look after itself.
I'm actually quite interested in btrfs especially for servers because of 
it's features but this problem really worries me.

Regards,
   Dennis
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Eric Sandeen
On 2/23/11 5:38 AM, Jóhann B. Guðmundsson wrote:
 On 02/23/2011 01:26 AM, Josef Bacik wrote:
 Various things, better data integrity to start with, and if you
 install the yum-fs-snapshot you have the ability to rollback easily.
 
 So we got the above + What Lennart mentioned as benefits to the end user.
 
 Now if we continue to hang on to the outdated concept ( From my 
 perspective ) of defaults as opposed to allowing each SIG to control 
 and set it's own defaults that suits their target end user base.
 
 For example any of the *DE might not want to switch to BTRFS until all 
 GUI tool for their end user to take advantage of what it has to offer 
 are in place while the Server SIG might certainly do.
 
 Hence I propose that we open a ticket with FESCO which will request them 
 to do something like this.
 
 Review the current filesystem default
 
 Gather feed back from various SIG with in the community to create a good 
 criteria for what needs to be in place in a filesystem for it to be 
 considered as an a default for Fedora.
 
 Gather a list of all existing linux filesystems ( there seems to be a 
 new one popping up every other month or so ).

Perhaps, but the list of possibly usable defaults for Fedora is not very long.

ext3, ext4, xfs, btrfs for now, I'd say.

Anything that popped up last month is not going to be ready for primetime
for years.

That said, I'd love to see it easy to tweak the default from among the supported
anaconda filesystems via kickstart or boot-time option, vs. having to go into
the detailed custom storage setup screen.  ext3 please is a lot simpler 
request
than custom partitioning with volume managers, and today it's all rolled into
one...

-Eric

 Filter out those that do not live up to that criteria and any base line 
 they set for their decision like must not perform worse then the 
 current default must work well on SSD's etc..  settle on one that 
 benefits most likely most the novice end users.
 ( We experienced users are able to choose tweak and turn any knob at 
 install time to suit our need which makes the novice end user the lowest 
 dominator)
 
 JBG

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Josef Bacik
On Wed, Feb 23, 2011 at 10:19 AM, Dennis Jacobfeuerborn
denni...@conversis.de wrote:
 On 02/23/2011 03:27 PM, Josef Bacik wrote:
 On Wed, Feb 23, 2011 at 9:18 AM, John Reiserjrei...@bitwagon.com  wrote:
 On 02/23/2011 05:07 AM, drago01 wrote:
 Defaults should be chooses on the metric what provides the best
 experience for the users not based on what we have been doing in the
 past (i.e stagnation).

 *One* data corruption constitutes EPIC FAIL.  Btrfs is too young,
 and will be for yet a while longer.


 Well if data corruption is the test then we shouldn't be using Ext4
 either, there was one fixed as recently as the beginning of this
 month.  File systems are software like anything else, there will be
 bugs.  Off the top of my head I can think of 3 data corrupters we've
 had in 4 years of working on BTRFS, and they've all been hard to hit
 and have not to my knowledge been seen by users, only us developers in
 testing.  BTRFS is young, but we have to start somewhere.  Thanks,

 I'm actually not that worried about corruption as that is something that
 can be fixed once discovered. What creeps me out about btrfs at the moment
 is this:

 https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21__Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21

 The fact that the FS needs manual rebalance operations and that these can
 take a while (even tough this can be done online) doesn't exactly make
 btrfs the ideal candidate for an end-user desktop system that should pretty
 much be able to look after itself.
 I'm actually quite interested in btrfs especially for servers because of
 it's features but this problem really worries me.


Yes this is one of the more complicated areas of BTRFS and tends to
blow up in our faces a lot.  That being said it's only a big deal if
you tend to run your filesystem close to full a lot, which most people
do not.  It is an area that we work very hard to make sure it's not a
problem, hopefully we have eliminated all of the big problems and you
should really only see ENOSPC when you actually fill up the disk.
Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Jóhann B. Guðmundsson
On 02/23/2011 03:33 PM, Josef Bacik wrote:

 I'm actually not that worried about corruption as that is something that
 can be fixed once discovered. What creeps me out about btrfs at the moment
 is this:

 https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21__Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21

 The fact that the FS needs manual rebalance operations and that these can
 take a while (even tough this can be done online) doesn't exactly make
 btrfs the ideal candidate for an end-user desktop system that should pretty
 much be able to look after itself.
 I'm actually quite interested in btrfs especially for servers because of
 it's features but this problem really worries me.

 Yes this is one of the more complicated areas of BTRFS and tends to
 blow up in our faces a lot.  That being said it's only a big deal if
 you tend to run your filesystem close to full a lot, which most people
 do not.  It is an area that we work very hard to make sure it's not a
 problem, hopefully we have eliminated all of the big problems and you
 should really only see ENOSPC when you actually fill up the disk.
 Thanks,

I'm pretty sure for novice end user desktops the same procedure that 
would clean up old snapshot on regular intervals could make rebalance 
operation run at the same time since it can be done online...

JBG
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Simo Sorce
On Wed, 23 Feb 2011 10:33:26 -0500
Josef Bacik jo...@toxicpanda.com wrote:

 On Wed, Feb 23, 2011 at 10:19 AM, Dennis Jacobfeuerborn
 denni...@conversis.de wrote:
  On 02/23/2011 03:27 PM, Josef Bacik wrote:
  On Wed, Feb 23, 2011 at 9:18 AM, John Reiserjrei...@bitwagon.com
   wrote:
  On 02/23/2011 05:07 AM, drago01 wrote:
  Defaults should be chooses on the metric what provides the best
  experience for the users not based on what we have been doing
  in the past (i.e stagnation).
 
  *One* data corruption constitutes EPIC FAIL.  Btrfs is too young,
  and will be for yet a while longer.
 
 
  Well if data corruption is the test then we shouldn't be using Ext4
  either, there was one fixed as recently as the beginning of this
  month.  File systems are software like anything else, there will be
  bugs.  Off the top of my head I can think of 3 data corrupters
  we've had in 4 years of working on BTRFS, and they've all been
  hard to hit and have not to my knowledge been seen by users, only
  us developers in testing.  BTRFS is young, but we have to start
  somewhere.  Thanks,
 
  I'm actually not that worried about corruption as that is something
  that can be fixed once discovered. What creeps me out about btrfs
  at the moment is this:
 
  https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21__Btrfs_claims_I.27m_out_of_space.2C_but_it_looks_like_I_should_have_lots_left.21
 
  The fact that the FS needs manual rebalance operations and that
  these can take a while (even tough this can be done online)
  doesn't exactly make btrfs the ideal candidate for an end-user
  desktop system that should pretty much be able to look after itself.
  I'm actually quite interested in btrfs especially for servers
  because of it's features but this problem really worries me.
 
 
 Yes this is one of the more complicated areas of BTRFS and tends to
 blow up in our faces a lot.  That being said it's only a big deal if
 you tend to run your filesystem close to full a lot, which most people
 do not.  It is an area that we work very hard to make sure it's not a
 problem, hopefully we have eliminated all of the big problems and you
 should really only see ENOSPC when you actually fill up the disk.
 Thanks,

Sorry josef,
but I do that all the time with my Virtual Machines, as I
do not give them more space then strictly needed.

I did that to the point that I needed to uninstall a few devel packages
in order to upgrade from f14 to rawhide on a VM ...

I am, not sure how common it is on a desktop, just wanted to point out
it is a use case to be able to run with little disk at least for
development VMs. (I guess I can manually run whatever tool there, as
long as it is clearly recognizable when I need to do so).

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-23 Thread Jon Masters
On Wed, 2011-02-23 at 07:15 -0500, Josh Boyer wrote:
 On Tue, Feb 22, 2011 at 10:25 PM, Jon Masters jonat...@jonmasters.org wrote:
  On Tue, 2011-02-22 at 14:51 -0500, Josef Bacik wrote:
 
  2) Fedora 16 ships without LVM as the volume manager and instead use
  BTRFS's built in volume management, again just for the default.
 
  In my personal opinion, this is a poor design decision. Yes, BTRFS can
  do a lot of volume-y things, and these are growing by the day, but I
  don't want my filesystem replacing a full volume manager and I am
  concerned that this will lead to less testing and exposure to full LVM
  use within the Fedora community. Instead, I'd like to counter-propose
  that everything stay exactly as it is, with users being able to elect to
  switch to BTRFS (sub)volumes if they are interested in doing so.
 
  Should the switch to BTRFS by default happen, this will be one more
  thing I will have to fix immediately during installation. The list grows
  longer and longer over time - please don't make this change.
 
 You seem to spend a lot of time during your installs undoing all the
 new things that were done for the release.  Perhaps a rapid changing,
 bleeding-edge distribution isn't quite suited to your needs.  Maybe
 you would be more comfortable with Debian or CentOS?

I'll bite. I am, indeed, a fan of various Enterprise Linux distributions
and I make no pretense that I am not. I do run an Enterprise Linux on
one desktop, and it's also true that I intentionally run my primary
desktop several Fedora releases behind so as to avoid many of the
problems I see from some of these changes. However, I also run more
recent Fedora releases, and on those releases, I typically have to undo
changes such as the one originally being proposed (replacing LVM).

Again, I feel the solution is to have a Fedora architect whose role is
to realize the problems caused by seemingly isolated changes, and stop
them from propagating. You don't just replace years of UNIX (or Linux)
history/heritage overnight without bothering a chunk of the users.

Jon.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Lennart Poettering
On Wed, 23.02.11 11:41, Jon Masters (jonat...@jonmasters.org) wrote:

  You seem to spend a lot of time during your installs undoing all the
  new things that were done for the release.  Perhaps a rapid changing,
  bleeding-edge distribution isn't quite suited to your needs.  Maybe
  you would be more comfortable with Debian or CentOS?
 
 I'll bite. I am, indeed, a fan of various Enterprise Linux distributions
 and I make no pretense that I am not. I do run an Enterprise Linux on
 one desktop, and it's also true that I intentionally run my primary
 desktop several Fedora releases behind so as to avoid many of the
 problems I see from some of these changes. However, I also run more
 recent Fedora releases, and on those releases, I typically have to undo
 changes such as the one originally being proposed (replacing LVM).
 
 Again, I feel the solution is to have a Fedora architect whose role is
 to realize the problems caused by seemingly isolated changes, and stop
 them from propagating. You don't just replace years of UNIX (or Linux)
 history/heritage overnight without bothering a chunk of the users.

We have that already. It's called FESCO, and is pluralistic and
democratic and stuff.

Lennart

-- 
Lennart Poettering - Red Hat, Inc.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Peter Jones
On 02/23/2011 11:41 AM, Jon Masters wrote:

 Again, I feel the solution is to have a Fedora architect whose role is
 to realize the problems caused by seemingly isolated changes, and stop
 them from propagating.

Fedora historically relies on an open source model for this - there
are a lot of smart people working on Fedora, and we depend on each and
every one of them to realize the problems caused by seemingly isolated
changes. With that in mind, I'm wondering if you have any input as to
what problems there might be with the change in question.

 You don't just replace years of UNIX (or Linux) history/heritage
 overnight without bothering a chunk of the users.

This is something what we've historically been willing to do in Linux,
and so far we've had pretty good results.

-- 
Peter

RFC 882 put the dots in .com.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Jesse Keating
On 2/23/11 5:00 AM, Josef Bacik wrote:
 This would be a great thing in general since the default ext* image is
 shrunk down to be installed which creates a bad fs layout which has
 performance implications.

Can you expand upon this more?  The filesystem is shrunk down when the 
live image is built, but then once it is installed the fs is resized to 
fill out the physical disk / partition it is being installed to.

-- 
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Lars Seipel
On Wednesday 23 February 2011 15:07:55 Peter Jones wrote:

 1) can btrfs do encrypted volumes?

Not yet.  Although this was a planned feature at some point, according to 
Josef, nobody has done it yet.

If you want to stack it on top of dm-crypt there are caveats as well.

From btrfs-wiki:
btrfs volumes on top of dm-crypt block devices (and possibly LVM) require
 write-caching to be turned off on the underlying HDD. Failing to do so, in
 the event of a power failure, may result in corruption not yet handled by
 btrfs code. (2.6.33)

Is this still prevailing as of 2.6.38?

While it may work if you're lucky (at least it did for me since F13) it can 
also munch your data. That's what happened to me last night in F15 (didn't 
lose any data though as such stuff was somewhat expected). Btrfs root partition 
(no LVM) on a dm-crypt volume on Intel 32 GB solid state drive.

After an unclean shutdown I was no longer able to mount the filesystem and 
therefore start the system. I even managed to crash a live system when trying 
to mount the decrypted luks volume containing the btrfs.

As I didn't have much time I just dd'ed an image to a spare disk and 
reinstalled. If there is any debugging value in this, I can make it available 
as long as I can be sure all data could be stripped from it (btrfs-image does 
exactly this, right?).

Be aware that the crash-on-shutdown didn't happen with an official Fedora 
kernel. Due to the update embargo on F15 I built an updated Kernel checked out 
with fedpkg.

If btrfs should become the default filesystem for Fedora (which I strongly 
support) a nice and clean solution for encrypted filesystems has to be found. 
Falling back to ext4  lvm when the encryption checkbox gets ticked is just 
plain ugly. Not supporting encrypted disks at all would be even worse.

Any ideas?
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Eric Sandeen
On 2/23/11 11:42 AM, Jesse Keating wrote:
 On 2/23/11 5:00 AM, Josef Bacik wrote:
 This would be a great thing in general since the default ext* image is
 shrunk down to be installed which creates a bad fs layout which has
 performance implications.
 
 Can you expand upon this more?  The filesystem is shrunk down when the 
 live image is built, but then once it is installed the fs is resized to 
 fill out the physical disk / partition it is being installed to.

Shrinking down the filesystem tends to scramble it a bit.  All the allocator 
decisions that were made based on fs size at the time of install go out the 
window as resize2fs fills in every hole it can find to maximally compress the 
fs.

I haven't done a real analysis but I would expect that inode::data locality and 
filesystem fragmentation suffer as a result.

Resizing it back up after the install just attaches free space to the end of 
the shrunken fs.

-Eric
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Matthew Garrett
On Wed, Feb 23, 2011 at 11:41:49AM -0500, Jon Masters wrote:

 Again, I feel the solution is to have a Fedora architect whose role is
 to realize the problems caused by seemingly isolated changes, and stop
 them from propagating. You don't just replace years of UNIX (or Linux)
 history/heritage overnight without bothering a chunk of the users.

LVM is functional for enterprise environments but awful for the common 
home or office cases. If btrfs lives up to its promises it'll give us 
something that provides pretty much all the functional beneft of LVM 
without the additional abstraction that makes seemingly straightforward 
tasks sufficiently awkward that even I find it abstruse (and I hack on 
ACPI). I think that's entirely in keeping with Linux heritage.

(AIX, on the other hand, delights in partying like it's 1979. Are you 
confusing the two?)

-- 
Matthew Garrett | mj...@srcf.ucam.org
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Casey Dahlin
On Wed, Feb 23, 2011 at 11:54:58AM -0600, Eric Sandeen wrote:
 On 2/23/11 11:42 AM, Jesse Keating wrote:
  On 2/23/11 5:00 AM, Josef Bacik wrote:
  This would be a great thing in general since the default ext* image is
  shrunk down to be installed which creates a bad fs layout which has
  performance implications.
  
  Can you expand upon this more?  The filesystem is shrunk down when the 
  live image is built, but then once it is installed the fs is resized to 
  fill out the physical disk / partition it is being installed to.
 
 Shrinking down the filesystem tends to scramble it a bit.  All the allocator
 decisions that were made based on fs size at the time of install go out the
 window as resize2fs fills in every hole it can find to maximally compress the
 fs.
 

Is there some sort of sparsifying defrag we could do after copying the image
over? The dd-based install is so fast that we could actually afford another
pass if we needed it IMHO.

--CJD
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Eric Sandeen
On 2/23/11 12:15 PM, Casey Dahlin wrote:
 On Wed, Feb 23, 2011 at 11:54:58AM -0600, Eric Sandeen wrote:
 On 2/23/11 11:42 AM, Jesse Keating wrote:
 On 2/23/11 5:00 AM, Josef Bacik wrote:
 This would be a great thing in general since the default ext* image is
 shrunk down to be installed which creates a bad fs layout which has
 performance implications.

 Can you expand upon this more?  The filesystem is shrunk down when the 
 live image is built, but then once it is installed the fs is resized to 
 fill out the physical disk / partition it is being installed to.

 Shrinking down the filesystem tends to scramble it a bit.  All the allocator
 decisions that were made based on fs size at the time of install go out the
 window as resize2fs fills in every hole it can find to maximally compress the
 fs.

 
 Is there some sort of sparsifying defrag we could do after copying the image
 over? The dd-based install is so fast that we could actually afford another
 pass if we needed it IMHO.

Not really... e4defrag could maybe, in theory, but its still pretty unloved.

I wonder how it'd go to prep the install on a sparse image of some reasonable 
size,
and then do an efficient sparse copy of that onto the target.  That would save 
the
scrambling, at least, and still give us minimum space usage on the install disc.

-Eric
 
 --CJD

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Peter Jones
On 02/23/2011 01:15 PM, Casey Dahlin wrote:
 On Wed, Feb 23, 2011 at 11:54:58AM -0600, Eric Sandeen wrote:
 On 2/23/11 11:42 AM, Jesse Keating wrote:
 On 2/23/11 5:00 AM, Josef Bacik wrote:
 This would be a great thing in general since the default ext* image is
 shrunk down to be installed which creates a bad fs layout which has
 performance implications.

 Can you expand upon this more?  The filesystem is shrunk down when the
 live image is built, but then once it is installed the fs is resized to
 fill out the physical disk / partition it is being installed to.

 Shrinking down the filesystem tends to scramble it a bit.  All the allocator
 decisions that were made based on fs size at the time of install go out the
 window as resize2fs fills in every hole it can find to maximally compress the
 fs.


 Is there some sort of sparsifying defrag we could do after copying the image
 over? The dd-based install is so fast that we could actually afford another
 pass if we needed it IMHO.

Just as soon as you write it ;)

-- 
 Peter

RFC 882 put the dots in .com.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Peter Jones
On 02/23/2011 12:50 PM, Lars Seipel wrote:
 On Wednesday 23 February 2011 15:07:55 Peter Jones wrote:
 
 1) can btrfs do encrypted volumes?
 
 Not yet.  Although this was a planned feature at some point, according to
 Josef, nobody has done it yet.
 
 If you want to stack it on top of dm-crypt there are caveats as well.

Right, which is what we'd wind up doing in the encrypted case.

 From btrfs-wiki:
 btrfs volumes on top of dm-crypt block devices (and possibly LVM) require
 write-caching to be turned off on the underlying HDD. Failing to do so, in
 the event of a power failure, may result in corruption not yet handled by
 btrfs code. (2.6.33)
 
 Is this still prevailing as of 2.6.38?

So, don't hold me to this, but it looks as if the normal lvm behaviour was
fixed at least as of d87f4c14f2 .  That being said, dm-crypt still just throws
REQ_FLUSH away instead of figuring out the block remaps involved and issuing
the right bios. Of course, this is a problem with dm-crypt and _any_ filesystem.

-- 
Peter

RFC 882 put the dots in .com.

01234567890123456789012345678901234567890123456789012345678901234567890123456789
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


BTRFS on servers (was Re: Plans for BTRFS in Fedora)

2011-02-23 Thread Jonathan Dieter
On Wed, 2011-02-23 at 16:19 +0100, Dennis Jacobfeuerborn wrote:
 I'm actually quite interested in btrfs especially for servers because
 of it's features

For what it's worth, we've been running btrfs on our school fileservers
since September.  After a few teething problems (fixed by
increasing /proc/sys/vm/min_free_kbytes), we've had pretty much zero
trouble.

The best part is we no longer need LVM on those servers and we get the
benefit of painless snapshot generation (we tried LVM snapshots and they
were *slow*).

Jonathan


signature.asc
Description: This is a digitally signed message part
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: BTRFS on servers (was Re: Plans for BTRFS in Fedora)

2011-02-23 Thread Josef Bacik
On Wed, Feb 23, 2011 at 2:00 PM, Jonathan Dieter jdie...@lesbg.com wrote:
 On Wed, 2011-02-23 at 16:19 +0100, Dennis Jacobfeuerborn wrote:
 I'm actually quite interested in btrfs especially for servers because
 of it's features

 For what it's worth, we've been running btrfs on our school fileservers
 since September.  After a few teething problems (fixed by
 increasing /proc/sys/vm/min_free_kbytes), we've had pretty much zero
 trouble.


Wait what?  I know we use lots o ram, but you shouldn't have to bump
min_free_kbytes.  What were you seeing?  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Chris Adams
Once upon a time, Matthew Garrett mj...@srcf.ucam.org said:
 LVM is functional for enterprise environments but awful for the common 
 home or office cases.

Define awful.  I make use of it all the time on home and office
desktops and even my notebook computer.  It makes it easy to reassign
disk space from purpose A to purpose B (it would be easier if there was
a way to shrink a mounted ext3/ext4 FS, but that's a hard problem).
Logical volumes for virtual machines perform better (and virtualization
is something becoming more common on the desktop for compatibility).

-- 
Chris Adams cmad...@hiwaay.net
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: BTRFS on servers (was Re: Plans for BTRFS in Fedora)

2011-02-23 Thread Jonathan Dieter
On Wed, 2011-02-23 at 14:18 -0500, Josef Bacik wrote:
 On Wed, Feb 23, 2011 at 2:00 PM, Jonathan Dieter jdie...@lesbg.com wrote:
  On Wed, 2011-02-23 at 16:19 +0100, Dennis Jacobfeuerborn wrote:
  I'm actually quite interested in btrfs especially for servers because
  of it's features
 
  For what it's worth, we've been running btrfs on our school fileservers
  since September.  After a few teething problems (fixed by
  increasing /proc/sys/vm/min_free_kbytes), we've had pretty much zero
  trouble.
 
 
 Wait what?  I know we use lots o ram, but you shouldn't have to bump
 min_free_kbytes.  What were you seeing?  Thanks,

Sorry, I didn't mean to imply it was btrfs that was causing the problem.
We got new servers when we did our upgrade and ran into problems with
the e1000e kernel module being unable to allocate memory even though
there was loads free.  Bumping up min_free_kbytes fixed it.

See https://bugzilla.redhat.com/show_bug.cgi?id=626851 for more detail.

Jonathan


signature.asc
Description: This is a digitally signed message part
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-23 Thread Matthew Garrett
On Wed, Feb 23, 2011 at 01:38:05PM -0600, Chris Adams wrote:

 Define awful.  I make use of it all the time on home and office
 desktops and even my notebook computer.  It makes it easy to reassign
 disk space from purpose A to purpose B (it would be easier if there was
 a way to shrink a mounted ext3/ext4 FS, but that's a hard problem).
 Logical volumes for virtual machines perform better (and virtualization
 is something becoming more common on the desktop for compatibility).

You can't move PVs. You need a separate /boot. If you use more than one 
disk then it adds significant fragility to the boot process. It slows 
down booting. It provides some functionality that's hugely useful in a 
small number of cases, and in every other case it just makes your life 
more complicated. btrfs does the former without anywhere near as much of 
the latter.

-- 
Matthew Garrett | mj...@srcf.ucam.org
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Matthew Garrett
On Wed, Feb 23, 2011 at 07:49:49PM +, Matthew Garrett wrote:

 You can't move PVs. You need a separate /boot. If you use more than one 
 disk then it adds significant fragility to the boot process. It slows 
 down booting. It provides some functionality that's hugely useful in a 
 small number of cases, and in every other case it just makes your life 
 more complicated. btrfs does the former without anywhere near as much of 
 the latter.

Also, the management tools are worse than AIX had in 1992. I'll accept 
that that's not an inherent flaw in the implementation, merely an 
inconvenient aspect of reality.

-- 
Matthew Garrett | mj...@srcf.ucam.org
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: BTRFS on servers (was Re: Plans for BTRFS in Fedora)

2011-02-23 Thread Josef Bacik
On Wed, Feb 23, 2011 at 2:42 PM, Jonathan Dieter jdie...@lesbg.com wrote:
 On Wed, 2011-02-23 at 14:18 -0500, Josef Bacik wrote:
 On Wed, Feb 23, 2011 at 2:00 PM, Jonathan Dieter jdie...@lesbg.com wrote:
  On Wed, 2011-02-23 at 16:19 +0100, Dennis Jacobfeuerborn wrote:
  I'm actually quite interested in btrfs especially for servers because
  of it's features
 
  For what it's worth, we've been running btrfs on our school fileservers
  since September.  After a few teething problems (fixed by
  increasing /proc/sys/vm/min_free_kbytes), we've had pretty much zero
  trouble.
 

 Wait what?  I know we use lots o ram, but you shouldn't have to bump
 min_free_kbytes.  What were you seeing?  Thanks,

 Sorry, I didn't mean to imply it was btrfs that was causing the problem.
 We got new servers when we did our upgrade and ran into problems with
 the e1000e kernel module being unable to allocate memory even though
 there was loads free.  Bumping up min_free_kbytes fixed it.

 See https://bugzilla.redhat.com/show_bug.cgi?id=626851 for more detail.


Ah ok, good, nevermind then :),

Joesf
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Chris Adams
Once upon a time, Matthew Garrett mj...@srcf.ucam.org said:
 You can't move PVs.

What do you think pvmove does?

 You need a separate /boot.

That's needed for more than just LVM (and probably won't go away, as it
is a lot simpler to handle a single method in the installer).

 If you use more than one 
 disk then it adds significant fragility to the boot process.

How does it do that (any more than any other multi-disk setup)?

 It slows 
 down booting.

Cite numbers?  It was slower early on, but it goes right by now.  I
don't doubt it adds some time (of course), but I don't see it being any
significant amount.

 It provides some functionality that's hugely useful in a 
 small number of cases, and in every other case it just makes your life 
 more complicated.

For most, it doesn't make things any more complicated (because they
never touch it).  I'd say that LVM is useful in a growing number of
cases.

 btrfs does the former without anywhere near as much of 
 the latter.

Oh, I don't object to btrfs and having the basic volume management in
the filesystem layer.  AdvFS on DEC Unix was great in that respect.  I
object to your painting of LVM aw awful.
-- 
Chris Adams cmad...@hiwaay.net
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Ralf Ertzinger
Hi.

On Wed, 23 Feb 2011 13:38:05 -0600, Chris Adams wrote

 Define awful.  I make use of it all the time on home and office
 desktops and even my notebook computer.  It makes it easy to reassign
 disk space from purpose A to purpose B (it would be easier if there
 was a way to shrink a mounted ext3/ext4 FS, but that's a hard
 problem). Logical volumes for virtual machines perform better (and
 virtualization is something becoming more common on the desktop for
 compatibility).

If you never tried the kind of freedom BTRFS and ZFS give you for
shifting around disk space, try it. Seriously. Then you'll see where
the awful comes from. In perspective it really is.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Matthew Garrett
On Wed, Feb 23, 2011 at 02:08:08PM -0600, Chris Adams wrote:
 Once upon a time, Matthew Garrett mj...@srcf.ucam.org said:
  You can't move PVs.
 
 What do you think pvmove does?

Move PEs from one PV to another. You can't move a PV.

  You need a separate /boot.
 
 That's needed for more than just LVM (and probably won't go away, as it
 is a lot simpler to handle a single method in the installer).

Unless you're booting off iSCSI or really, really love software RAID 
(which btrfs also kindly simplifies), the only reason you need a 
separate /boot is because we're slow at putting updated filesystem 
support in grub because there's no need because we default to a separate 
/boot. There's a certain amount of circularity there.

  If you use more than one 
  disk then it adds significant fragility to the boot process.
 
 How does it do that (any more than any other multi-disk setup)?

My experience with LVM is that any number of error states dump me into 
an awkward recovery scenario. Again it's not an inherent problem with 
the technology, more just that because nobody outside the enterprise 
world really cares about LVM nobody has bothered making it easy for 
non-sysadmins to handle this stuff.

  It slows 
  down booting.
 
 Cite numbers?  It was slower early on, but it goes right by now.  I
 don't doubt it adds some time (of course), but I don't see it being any
 significant amount.

Talk to Lennart about it. It's on the order of two seconds.

  btrfs does the former without anywhere near as much of 
  the latter.
 
 Oh, I don't object to btrfs and having the basic volume management in
 the filesystem layer.  AdvFS on DEC Unix was great in that respect.  I
 object to your painting of LVM aw awful.

My objections to the technology aren't that strong. My objections to the 
user experience are.

-- 
Matthew Garrett | mj...@srcf.ucam.org
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Peter Jones
On 02/23/2011 03:33 PM, Matthew Garrett wrote:
 On Wed, Feb 23, 2011 at 02:08:08PM -0600, Chris Adams wrote:
 Once upon a time, Matthew Garrettmj...@srcf.ucam.org  said:
 You can't move PVs.

 What do you think pvmove does?

 Move PEs from one PV to another. You can't move a PV.

 You need a separate /boot.

 That's needed for more than just LVM (and probably won't go away, as it
 is a lot simpler to handle a single method in the installer).

 Unless you're booting off iSCSI or really, really love software RAID
 (which btrfs also kindly simplifies), the only reason you need a
 separate /boot is because we're slow at putting updated filesystem
 support in grub because there's no need because we default to a separate
 /boot. There's a certain amount of circularity there.

Also we need to have the initramfs on an unencrypted chunk of disk.

 If you use more than one
 disk then it adds significant fragility to the boot process.

 How does it do that (any more than any other multi-disk setup)?

 My experience with LVM is that any number of error states dump me into
 an awkward recovery scenario. Again it's not an inherent problem with
 the technology, more just that because nobody outside the enterprise
 world really cares about LVM nobody has bothered making it easy for
 non-sysadmins to handle this stuff.

Yep.

-- 
 Peter

RFC 882 put the dots in .com.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Chris Adams
Once upon a time, Matthew Garrett mj...@srcf.ucam.org said:
 On Wed, Feb 23, 2011 at 02:08:08PM -0600, Chris Adams wrote:
  Once upon a time, Matthew Garrett mj...@srcf.ucam.org said:
   You can't move PVs.
  
  What do you think pvmove does?
 
 Move PEs from one PV to another. You can't move a PV.

Not exactly; pvmove moves data from one PE to another (either in another
PV or in the same PV).  I guess I don't understand your original
objection though.  You can move data around within a PV (granted the
command usage to do this is non-obvious and it would be good for that to
improve, but it is also not a common use case) and from one PV to
another; what else is there to move?

   You need a separate /boot.
  
  That's needed for more than just LVM (and probably won't go away, as it
  is a lot simpler to handle a single method in the installer).
 
 Unless you're booting off iSCSI or really, really love software RAID 
 (which btrfs also kindly simplifies),

Using software RAID means really, really love it?  How else do you
propose to boot from software RAID (if you just kinda like it)?  We're
talking about the typical home/office desktop, so you aren't going to
have hardware RAID, and Linux software RAID is superior to motherboard
software RAID.

 the only reason you need a 
 separate /boot is because we're slow at putting updated filesystem 
 support in grub because there's no need because we default to a separate 
 /boot. There's a certain amount of circularity there.

Encrypted root filesystem requires a separate (non-encrypted) /boot, and
I think that's only going to increase in popularity (especially on
notebooks and other mobile devices).

 My experience with LVM is that any number of error states dump me into 
 an awkward recovery scenario.

I guess in the years I've used LVM, I haven't run into that any number
of error states (at least AFAIK, but since you didn't identify any of
them, it is hard to say).  Sure, when LVM was new, I had to learn a few
more commands to handle the system didn't boot case, but IIRC only
once was that related to LVM (and that was because I renamed a VG
without rebuilding the initrd, which may be no longer an issue with
dracut).

If you don't know what you are doing, just fsck is going to be a
problem.  That's an issue with documentation and maybe making better
recovery tools, not LVM.  Adding a new set of commands to learn with
BTRFS isn't magically going to make it easier; you still have to learn
new stuff.

-- 
Chris Adams cmad...@hiwaay.net
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Chris Adams
Once upon a time, Ralf Ertzinger fed...@camperquake.de said:
 If you never tried the kind of freedom BTRFS and ZFS give you for
 shifting around disk space, try it. Seriously. Then you'll see where
 the awful comes from. In perspective it really is.

You cut out the parts of my email where I said I don't have any problem
with BTRFS, and I look forward to it making things better.  As I said, I
used AdvFS on DEC Unix for many years; ZFS and BTRFS are late-comers to
the filesystem/LVM party (although they obviously bring more to the
table).

-- 
Chris Adams cmad...@hiwaay.net
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Josef Bacik
On Wed, Feb 23, 2011 at 4:37 PM, Kevin Kofler kevin.kof...@chello.at wrote:
 Jon Masters wrote:
 In my personal opinion, this is a poor design decision. Yes, BTRFS can
 do a lot of volume-y things, and these are growing by the day, but I
 don't want my filesystem replacing a full volume manager and I am
 concerned that this will lead to less testing and exposure to full LVM
 use within the Fedora community. Instead, I'd like to counter-propose
 that everything stay exactly as it is, with users being able to elect to
 switch to BTRFS (sub)volumes if they are interested in doing so.

 And I'd like to counter-counter-propose that we just stop using ANY kind of
 subvolumes or volume management by default and just default to plain old
 partitions. IMHO, LVM causes more problems than it fixes. Sure, you can
 easily add storage from another disk, but in exchange there's no
 straightforward way to resize your partitions, at least none of the common
 partition editors can do it. There's also a performance penalty.


Sorry I should clarify, when I say use Btrfs's volume management stuff
I mean just doing normal partitions and then creating a Btrfs
filesystem and then add disks to the fs as required.  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Milan Broz
On 02/23/2011 07:41 PM, Peter Jones wrote:
 On 02/23/2011 12:50 PM, Lars Seipel wrote:

 If you want to stack it on top of dm-crypt there are caveats as well.
 
 Right, which is what we'd wind up doing in the encrypted case.
 
 From btrfs-wiki:
 btrfs volumes on top of dm-crypt block devices (and possibly LVM) require
 write-caching to be turned off on the underlying HDD. Failing to do so, in
 the event of a power failure, may result in corruption not yet handled by
 btrfs code. (2.6.33)

This is completely nonsense and FUD. We asked several times for reproducer
and nobody from btrfs camp was able to provide one. 

If you have reproducer, please send it to me and I'll fix dmcrypt code.
But I am almost sure there is no such problem in current code.

 So, don't hold me to this, but it looks as if the normal lvm behaviour was
 fixed at least as of d87f4c14f2 .  That being said, dm-crypt still just throws
 REQ_FLUSH away instead of figuring out the block remaps involved and issuing
 the right bios. Of course, this is a problem with dm-crypt and _any_ 
 filesystem.

Another nonsense. See 2.6.37 source code, REQ_FLUSH is processed by dm-crypt
correctly. It supported even barriers some time before (at least in 2.6.36).

Anything else for today?

Milan
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread James Ralston
On 2011-02-22 at 14:51-05 Josef Bacik jo...@toxicpanda.com wrote:
 Fedora 16 ships without LVM as the volume manager and instead use
 BTRFS's built in volume management, again just for the default.

I don't think btrfs subvolumes are capable of replacing LVM
functionality quite yet.

Here are two usage cases that I care about:

1.  Separate LVM logical volumes can help mitigate consumption-based
DoS attacks.

For example: if /tmp and /var/tmp are separate LVM logical volumes,
then a runaway/malicious process cannot fill up the entire filesystem
merely by filling up /tmp or /var/tmp.

In contrast, although the btrfs wiki mentions the ability to set a
quota on subvolumes at creation time, I don't think anything in
btrfs-progs implements that.  Plus, even if there were a way to set
it, I don't see that anything reports it.

2.  Separate LVM logical volumes permit easier installs.

Primarily because I've never trusted the upgrade process, but also for
other reasons, I always perform a full install to upgrade from one
version of Fedora to the next.

With LVM logical volumes, this process is trivial: I do a custom
partition layout, and tell anaconda to format the /boot, /, /tmp,
/usr, /var, and /var/cache logical volumes, and leave all the other
logical volumes in the system (e.g., /home, /var/log) alone.

I can't do this with one big btrfs filesystem with subvolumes.  What I
would have to do is first boot the installer in rescue mode, then run
a bunch of find /subvolume -xdev commands to delete all the data on
the subvolumes that I wanted to reformat.  Then I'd have to re-run
the installer and tell it to use the / btrfs filesystem as-is, without
formatting.

None of these issues is a dealbreaker, but they *are* losses of
functionality versus what LVM offers.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-23 Thread Rahul Sundaram
On 02/23/2011 01:33 AM, Josef Bacik wrote:

 Well I don't think cleaning up the existing patches will be that big
 of a deal, it's mostly a matter of testing.  The problem with GRUB2 is
 it's GPLv3, explicitly to be a giant pain in the ass for porting any
 new fs to GRUB since we're all GPLv2 only.  Aren't open source
 licenses grand?

Refer to

http://lwn.net/Articles/429607/

Rahul

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Kevin Fenzi
On Tue, 22 Feb 2011 14:51:50 -0500
Josef Bacik jo...@toxicpanda.com wrote:

 Hello,
 
 So we're getting close to having a working fsck tool so I wanted to
 take the opportunity to talk about the future of BTRFS in Fedora.
...snip...

 1) GRUB support.  Edward Shishkin did GRUB1 patches for BTRFS a while
 ago, but they were obviously never merged upstream and were also not
 included into fedora.  These would either need to be cleaned up and
 put into our grub package, or we'd need to put /boot on a different
 filesystem.  I personally hate the idea of having a non-btrfs /boot
 partition but I'm not the one in charge of GRUB.

Perhaps if we are going to the pain of getting btrfs patches working
and stable, we could just look at moving to grub2? 

...snip...

 So what are your thoughts?  Thanks,

Sounds good. I'd suggest filing a feature page already and getting it
setup with all the status, etc. That can be a central point for
coordinating that effort. 

 Josef

kevin


signature.asc
Description: PGP signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-22 Thread Chris Lumens
 1) Fedora 16 ships with BTRFS as the default root filesystem.
 2) Fedora 16 ships without LVM as the volume manager and instead use
 BTRFS's built in volume management, again just for the default.
 
 2) Anaconda support.  I've already talked with Will Woods about this
 some.  Really anaconda will format a normal disk with BTRFS with no
 problem today, the biggest issue here is adding the volume management
 stuff and allowing users to create subvolumes via anaconda.

Given the slate of changes we have lined up for F16 anaconda
(https://fedoraproject.org/wiki/Anaconda/Features), I don't know that
adding another major storage change is going to happen.  I think in
large part, we were hoping to be done with major storage changes for a
release or two and work on other less touched areas instead (like the
UI).

But, perhaps we can find some time somewhere.  Knowing what the scope of
changes looks like might help with the planning a bit.

- Chris
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Josef Bacik
On Tue, Feb 22, 2011 at 3:00 PM, Kevin Fenzi ke...@scrye.com wrote:
 On Tue, 22 Feb 2011 14:51:50 -0500
 Josef Bacik jo...@toxicpanda.com wrote:

 Hello,

 So we're getting close to having a working fsck tool so I wanted to
 take the opportunity to talk about the future of BTRFS in Fedora.
 ...snip...

 1) GRUB support.  Edward Shishkin did GRUB1 patches for BTRFS a while
 ago, but they were obviously never merged upstream and were also not
 included into fedora.  These would either need to be cleaned up and
 put into our grub package, or we'd need to put /boot on a different
 filesystem.  I personally hate the idea of having a non-btrfs /boot
 partition but I'm not the one in charge of GRUB.

 Perhaps if we are going to the pain of getting btrfs patches working
 and stable, we could just look at moving to grub2?

Well I don't think cleaning up the existing patches will be that big
of a deal, it's mostly a matter of testing.  The problem with GRUB2 is
it's GPLv3, explicitly to be a giant pain in the ass for porting any
new fs to GRUB since we're all GPLv2 only.  Aren't open source
licenses grand?


 ...snip...

 So what are your thoughts?  Thanks,

 Sounds good. I'd suggest filing a feature page already and getting it
 setup with all the status, etc. That can be a central point for
 coordinating that effort.


Sounds good, I'll do that, thank you,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Chris Lumens
  1) GRUB support.  Edward Shishkin did GRUB1 patches for BTRFS a while
  ago, but they were obviously never merged upstream and were also not
  included into fedora.  These would either need to be cleaned up and
  put into our grub package, or we'd need to put /boot on a different
  filesystem.  I personally hate the idea of having a non-btrfs /boot
  partition but I'm not the one in charge of GRUB.
 
 Perhaps if we are going to the pain of getting btrfs patches working
 and stable, we could just look at moving to grub2? 

It's planned:

https://fedoraproject.org/wiki/Anaconda/Features/Grub2Migration

- Chris
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Genes MailLists
On 02/22/2011 02:51 PM, Josef Bacik wrote:
 Hello,
 
 So we're getting close to having a working fsck tool so I wanted to
 take the opportunity to talk about the future of BTRFS in Fedora.
 Coming up in F15 we're going to have the first release of Fedora where


...

 
 So what are your thoughts?  Thanks,
 
 Josef

 Exciting indeed - also do you happen to know the status of RAID 5
support ? If I recall correctly,  the kernel part was completed some
time back .. (several months) but the higher level stuff was not ready
back then.

 thanks

gene

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Lennart Poettering
On Tue, 22.02.11 14:51, Josef Bacik (jo...@toxicpanda.com) wrote:

 Hello,
 
 So we're getting close to having a working fsck tool so I wanted to
 take the opportunity to talk about the future of BTRFS in Fedora.
 Coming up in F15 we're going to have the first release of Fedora where
 we don't need the special boot option to have the ability to format
 you filesystem as BTRFS.  This is in hopes that we can open it up for
 wider testing before possibly making it the default filesystem.  I
 realize we're in the early stages of F15, but since filesystems are
 big and important I'd like to get an idea of the amount of work that
 needs to still be done to get BTRFS in shape for being Fedora's
 default filesystem.  So here are my goals
 
 1) Fedora 16 ships with BTRFS as the default root filesystem.

Hmm, what are your plans regarding hierarchy layout for this?

My hope is that one day we can ship a read-only root dir by default, or
more specifically a btrfs file system with three subvolumes in it: one
read-only one mounted to /, and two writable ones mounted to /home and
/var, with /tmp mounted from tmpfs.

We came a long way with supporting read-only root dirs and it should
mostly work now. In F15 for example /etc/mtab is a symlink, and even
smaller stuff like /etc/nologin got moved to /var/run, to make write
accesses to /etc unnecessary during normal operation. /etc/resolv.conf
is the only thing often updated that's left, but with NM using dnsmasq
it's static too.

By using btrfs subvolumes doing this kind of seperation into writable
and non-writable subtrees we don't have to think anymore about the sizes
for those file systems at install time, since they all live in the same
fs.

If this is adopted package managers and system configuration UIs would
need to invoke mount / -o rw,remount before doing their work.

So, I'd like to see this implemented one day, maybe the adoption of
btrfs is the right time to push this through too?

I have not filed a feature page for this, as I am not sure I want to
push this on F16, and I don't even know if people in general are onboard
with this idea. The benefits of this are mostly security and robustness
since we know that the actual subvolume the OS is booted from is always
in a consistent state during operation and cannot normally be
changed. And of course, we get all kind of magic by doing this because
we can easily use snapshots to roll back system upgrades while leaving
/home completely untouched.

Sorry for hijacking your thread like this,

Lennart

-- 
Lennart Poettering - Red Hat, Inc.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Jim Meyering
[...btrfs and read-only root,/etc...]

Music to my ears.
Glad you're working on this.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Sam Varshavchik

Josef Bacik writes:


2) Fedora 16 ships without LVM as the volume manager and instead use
BTRFS's built in volume management, again just for the default.


Just to clarify -- F16 will still have LVM, but not used by default.

I believe that there's a huge number of existing systems that are 
partitioned using LVM.




pgpY1YS4gcSHd.pgp
Description: PGP signature
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Plans for BTRFS in Fedora

2011-02-22 Thread B.
On Tue, 2011-02-22 at 14:51 -0500, Josef Bacik wrote:
 So what are your thoughts?  Thanks,

Will there be any performance penalties making this move?

I ran it on my workstation when F13 came out with updates+snapshots and while 
updating my desktop responsiveness was well not so good. 

Novice end users tools ( GUI ) and policy to clean those snapshots and perhaps 
storing those snapshots in it's own directory and or partition rather than 
directly on / need be in place before we implement this as an default. 

I suggest people try it during the F15 atleast those that are considering as an 
default to catch any potential usability issues it did not eat any babes from 
me when I ran it. 

I think we should also ask ourselves these questions.. 

What benefit will this switch bring to the novice desktop end users? 

Will the novice desktop end user ever take advantages of any of the features 
that btrfs brings? 
  
Just my 0.0.2 cents.. 

JBG

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Josef Bacik
On Tue, Feb 22, 2011 at 5:07 PM, Lennart Poettering
mzerq...@0pointer.de wrote:
 On Tue, 22.02.11 14:51, Josef Bacik (jo...@toxicpanda.com) wrote:

 Hello,

 So we're getting close to having a working fsck tool so I wanted to
 take the opportunity to talk about the future of BTRFS in Fedora.
 Coming up in F15 we're going to have the first release of Fedora where
 we don't need the special boot option to have the ability to format
 you filesystem as BTRFS.  This is in hopes that we can open it up for
 wider testing before possibly making it the default filesystem.  I
 realize we're in the early stages of F15, but since filesystems are
 big and important I'd like to get an idea of the amount of work that
 needs to still be done to get BTRFS in shape for being Fedora's
 default filesystem.  So here are my goals

 1) Fedora 16 ships with BTRFS as the default root filesystem.

 Hmm, what are your plans regarding hierarchy layout for this?

 My hope is that one day we can ship a read-only root dir by default, or
 more specifically a btrfs file system with three subvolumes in it: one
 read-only one mounted to /, and two writable ones mounted to /home and
 /var, with /tmp mounted from tmpfs.


Yeah the hope is to separate out various things into different
subvolumes so we can have things that can be independently
snapshottable things.

 We came a long way with supporting read-only root dirs and it should
 mostly work now. In F15 for example /etc/mtab is a symlink, and even
 smaller stuff like /etc/nologin got moved to /var/run, to make write
 accesses to /etc unnecessary during normal operation. /etc/resolv.conf
 is the only thing often updated that's left, but with NM using dnsmasq
 it's static too.

 By using btrfs subvolumes doing this kind of seperation into writable
 and non-writable subtrees we don't have to think anymore about the sizes
 for those file systems at install time, since they all live in the same
 fs.

 If this is adopted package managers and system configuration UIs would
 need to invoke mount / -o rw,remount before doing their work.

 So, I'd like to see this implemented one day, maybe the adoption of
 btrfs is the right time to push this through too?

 I have not filed a feature page for this, as I am not sure I want to
 push this on F16, and I don't even know if people in general are onboard
 with this idea. The benefits of this are mostly security and robustness
 since we know that the actual subvolume the OS is booted from is always
 in a consistent state during operation and cannot normally be
 changed. And of course, we get all kind of magic by doing this because
 we can easily use snapshots to roll back system upgrades while leaving
 /home completely untouched.

 Sorry for hijacking your thread like this,


It's cool, just hilights all the cool stuff we can do with BTRFS :).  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Josef Bacik
2011/2/22 Jóhann B. johan...@gmail.com:
 On Tue, 2011-02-22 at 14:51 -0500, Josef Bacik wrote:
 So what are your thoughts?  Thanks,

 Will there be any performance penalties making this move?


Who knows, thats what testing is for :).  There are some things that
suck with BTRFS, but so it goes with filesystems.

 I ran it on my workstation when F13 came out with updates+snapshots and while 
 updating my desktop responsiveness was well not so good.

 Novice end users tools ( GUI ) and policy to clean those snapshots and 
 perhaps storing those snapshots in it's own directory and or partition rather 
 than directly on / need be in place before we implement this as an default.


They would be helpful, but not necessary.

 I suggest people try it during the F15 atleast those that are considering as 
 an default to catch any potential usability issues it did not eat any babes 
 from me when I ran it.

 I think we should also ask ourselves these questions..

 What benefit will this switch bring to the novice desktop end users?


Various things, better data integrity to start with, and if you
install the yum-fs-snapshot you have the ability to rollback easily.

 Will the novice desktop end user ever take advantages of any of the features 
 that btrfs brings?


At the very least you get better data integrity via our checksumming
and duplicate metadata on single spindle fs's.  Thanks,

Josef
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Jon Masters
On Tue, 2011-02-22 at 14:51 -0500, Josef Bacik wrote:

 2) Fedora 16 ships without LVM as the volume manager and instead use
 BTRFS's built in volume management, again just for the default.

In my personal opinion, this is a poor design decision. Yes, BTRFS can
do a lot of volume-y things, and these are growing by the day, but I
don't want my filesystem replacing a full volume manager and I am
concerned that this will lead to less testing and exposure to full LVM
use within the Fedora community. Instead, I'd like to counter-propose
that everything stay exactly as it is, with users being able to elect to
switch to BTRFS (sub)volumes if they are interested in doing so.

Should the switch to BTRFS by default happen, this will be one more
thing I will have to fix immediately during installation. The list grows
longer and longer over time - please don't make this change.

Jon.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


Re: Plans for BTRFS in Fedora

2011-02-22 Thread Bruno Wolff III
On Tue, Feb 22, 2011 at 14:51:50 -0500,
  Josef Bacik jo...@toxicpanda.com wrote:
 
 3) All the various little tools that we have for putting together
 LiveCD's that are very ext* centered.  I've not even looked at this
 yet, but I assume it's going to be kind of a pain.

I like to see live CDs just use squashfs directly and not a squashed copy
of an ext3/4 image. However this will break using dd to copy the image
for installs which will slow things down. Further however, there is at least
some interest in having live CDs and ananconda share the same install method,
so this might be happening in any case.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel