subject:"Re\: Linux filesystems was \[Re\: Debian cd supporting ext4.\]"

Lisi put forth on 7/27/2010 2:23 AM:
 On Tuesday 27 July 2010 08:10:15 Stan Hoeppner wrote:
 XFS which is superior to all other Linux filesystems.
 
 Stan - 
 
 Have you the time to give a rationale for this?

Sure.

1. Best overall performance for most systems, large and small, and the FS
creation and mounting parameters are super configurable to match the system
hardware for best performance.  One recent set of recent benchmarks
demonstrating so:
http://btrfs.boxacle.net/repository/raid/2010-04-14_2004/2.6.34-rc3/2.6.34-rc3.html

man mkfs.xfs
man mount

Older benchmarks:
http://www.debian-administration.org/articles/388
http://everything2.com/index.pl?node_id=1479435

In regard to this last benchmark, some(many?) of the default XFS filesystem
creation parameters and mounting parameters have changed.  Note the testing
was performed in 2005.  A lot changes in 5 years.  Read all you can and ask
questions on the XFS mailing list before tweaking parameters based on what you
find in old forum posts and benchmarks such as this.

Guaranteed Rate I/O for streaming and other critical applications--unique to
XFS amongst all filesytems, ever, not just on Linux--this feature was born on
IRIX XFS for the broadcasting industry where video stutter was basically death
to a TV station or network such as CNN, CBS, etc.  This single feature from
SGI allowed broadcast media to wholesale convert from tape to disk (this and
SGI FC storage arrays)

2. Commercial origin and backing.  SGI is a fantastic technology compay:
http://oss.sgi.com/projects/xfs/

3. Maturity/history/longevity, IRIX birth in 1993, Linux birth 2001, included
in mainline in late 2003:
http://en.wikipedia.org/wiki/XFS
http://marc.info/?l=linux-kernelm=107088371607817w=2

4. Equal/superior user space toolset:
xfsprogs - includes online defragmentation tool xfs_fsr and online growth tool
xfs_growfs.  No other stable Linux FS has an online defragmenter.  Ext4 has
e4defrag but AFAIK it's not complete nor close to maturity or stability.
xfs_fsr has been both for a decade.

5. Very active developer community and thorough documentation:
http://xfs.org/index.php/Main_Page

 I'm not in any way impugning your knowledge.  But I am at the stage of 
 accepting the default that Lenny gives me, for no better reason than that the 
 developers chose it and it is there.  It is time I understood better the 
 reasons for each file system.  (I gave up Reiserfs because Reiser murdered 
 his wife - hardly a logical measure of how good his filesystem is!!)

Debian will _always_ default to an EXT* filesystem--until the end of time.
Then again, I thought the same of LILO, so what do I know eh?  But expert
install mode allows you whatever you want.  I never cared for ReiserFS and
never used it.  Hans actions are just further justification after the fact.

I've only used XFS on servers.  I've never used it on laptops or desktops.  I
know of many people who have, but they are die hard propeller heads and know
how to fix anything if/when it breaks.  If you want to run XFS on a laptop or
desktop, especially on Debian, your only downside is going to be getting quick
help, if you get jammed, from folks on this list, or the community in general.
 That process will probably not be as quick and fruitful as with EXT2/3
issues, simply because there are a lot less people using XFS, thus the pool of
helpers is much smaller.

If you're adventurous, take XFS for a spin.  Partition a 100MB /boot and
install everything else on a big partition running XFS.  And, learn the
utility set, and learn about XFS, just as you have (or should have) with
EXT2/3 and ReiserFS.  As always, knowledge is power.

-- 
Stan


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c4e9e29.4050...@hardwarefreak.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread Kelly Clowers

On Tue, Jul 27, 2010 at 01:51, Stan Hoeppner s...@hardwarefreak.com wrote:

 Debian will _always_ default to an EXT* filesystem--until the end of time.

Nope, btrfs will replace ext3/4 as default soon enough.


Cheers,
Kelly Clowers


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/aanlkti=hgrom682lbgmypx44uqj2dg25jwxzhivva...@mail.gmail.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes:
1. Best overall performance for most systems, large and small, and the FS
creation and mounting parameters are super configurable to match the system
hardware for best performance. One recent set of recent benchmarks
demonstrating so:
http://btrfs.boxacle.net/repository/raid/2010-04-14_2004/2.6.34-rc3/2.6.34-rc3.html

man mkfs.xfs
man mount

Older benchmarks:
http://www.debian-administration.org/articles/388
http://everything2.com/index.pl?node_id=1479435

In regard to this last benchmark, some(many?) of the default XFS filesystem
creation parameters and mounting parameters have changed. Note the testing
was performed in 2005. A lot changes in 5 years. Read all you can and ask
questions on the XFS mailing list before tweaking parameters based on what you
find in old forum posts and benchmarks such as this.

Guaranteed Rate I/O for streaming and other critical applications--unique to
XFS amongst all filesytems, ever, not just on Linux--this feature was born on
IRIX XFS for the broadcasting industry where video stutter was basically death
to a TV station or network such as CNN, CBS, etc. This single feature from
SGI allowed broadcast media to wholesale convert from tape to disk (this and
SGI FC storage arrays)

2. Commercial origin and backing. SGI is a fantastic technology compay:
http://oss.sgi.com/projects/xfs/

3. Maturity/history/longevity, IRIX birth in 1993, Linux birth 2001, included
in mainline in late 2003:
http://en.wikipedia.org/wiki/XFS
http://marc.info/?l=linux-kernelm=107088371607817w=2

4. Equal/superior user space toolset:
xfsprogs - includes online defragmentation tool xfs_fsr and online growth tool
xfs_growfs. No other stable Linux FS has an online defragmenter. Ext4 has
e4defrag but AFAIK it's not complete nor close to maturity or stability.
xfs_fsr has been both for a decade.

5. Very active developer community and thorough documentation:
http://xfs.org/index.php/Main_Page

You are missing a very important point: Durability to power failures.
(Excuse me, but a majority of GNU/Linux users are not switched to a UPS
or something.) And that's where XFS totally fails[1][2]. And considering
my personal experiences, reiserfs is the fastest fs (among ext3 and xfs)
in terms of boot recovery phase times.

Regards.

[1] http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html
[2]
http://xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_see_binary_NULLS_in_some_files_after_recovery_when_I_unplugged_the_power.3F

--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87bp9tuki8@alamut.alborz.net

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

On Tue, Jul 27, 2010 at 3:22 PM, Volkan YAZICI yazic...@ttmail.com wrote:


 You are missing a very important point: Durability to power failures.
 (Excuse me, but a majority of GNU/Linux users are not switched to a UPS
 or something.) And that's where XFS totally fails[1][2].


Ext3 has the same problems when not properly configured:

Ext3 does not do checksumming when writing to the journal. If barrier=1 is
not enabled as a mount option (in /etc/fstab), and if the hardware is doing
out-of-order write caching, one runs the risk of severe filesystem
corruption during a crash.
http://en.wikipedia.org/wiki/Ext3#No_checksumming_in_journal

For the record I use ext3, I remember XFS as not being reliable enough
(with power failures etc).

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread Aaron Toponce

On 7/27/2010 1:23 AM, Lisi wrote:
 On Tuesday 27 July 2010 08:10:15 Stan Hoeppner wrote:
 XFS which is superior to all other Linux filesystems.
 
 Stan - 
 
 Have you the time to give a rationale for this?  

Except XFS filesystems can't shrink, only grow. Sucks when you need to
resize partitions/volumes, and they're all XFS.

Further, XFS makes more system calls to the kernel than standard
Ext2/3/4. Export an XFS filesystem on LVM over NFS, and you'll get a
kernel oops on a 32-bit kernel. Trace it, and you'll see the plethora of
nested system calls XFS makes. You won't oops with Ext2/3/4 in the same
scenario. This can be mitigated by running a 64-bit system, if you have
the hardware to do so.

XFS has also had a history for randomly corrupting data. While this
might have improved over time, I don't trust it.

XFS does have dynamic inode allocation, and better data storage
algorithms than the Ext-family. It's also a good performer, but Ext4
give XFS a run for its money.

-- 
. O .   O . O   . . O   O . .   . O .
. . O   . O O   O . O   . O O   . . O
O O O   . O .   . O O   O O .   O O O



signature.asc
Description: OpenPGP digital signature

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Volkan YAZICI put forth on 7/27/2010 8:22 AM:

You are missing a very important point: Durability to power failures.
(Excuse me, but a majority of GNU/Linux users are not switched to a UPS
or something.) And that's where XFS totally fails[1][2].

[1] http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html

1. Never quote forum or email posts as empirical or reliable evidence of
anything. They are opinion, unless they quote fact from reliable sources to
back that opinion (which I did in my original post). There are of course
exceptions to this rule of thumb, the main one being when the post is made by
a developer who is a recognized authority on the piece of software being
discussed. In the case of XFS this would be Dave Chinner, Alex Elder, Eric
Sandeen, Christoph Hellwig, and others. In the case of EXT2/3/4 this would be
Ted Tso, who happens, BTW, to work hand in hand with the XFS developers
because they rely on each others patches to other parts of the Linux kernel.
Not the last two entries:
https://kerneltrap.org/mailarchive/linux-fsdevel/2010/7/16/expand

In the case of Postfix this would be Wietse Venema and Viktor Duchovni. In
the case of Linux this would be Linus Torvalds, Marcelo Tosatti, Alan Cox,
Andrew Morton and others. Etc.

2. If you are going to quote opinions from unreliable sources, at least read
the entire thread before quoting it. In this case, again, your source
contradicts what you state, and then, oddly, himself:

PS. Apparently they've improved some power-failure problems with XFS
since I used it, but then they also said that *before* I started using
it so I can't say I'd put any trust in that.

Aneurin Price admits he is aware of new changes that fix the problem, but then
discounts the fact and gives an anecdotal story as to why the fix isn't really
a fix.

[2]
http://xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_see_binary_NULLS_in_some_files_after_recovery_when_I_unplugged_the_power.3F

You didn't even read the information you linked to, which contradicts what you
state: The fix for this issue has been in mainline since May 2007, over 3
years ago.

Q: Why do I see binary NULLS in some files after recovery when I unplugged
the power?

Update: This issue has been addressed with a CVS fix on the 29th March 2007
and merged into mainline on 8th May 2007 for 2.6.22-rc1.

And considering
my personal experiences, reiserfs is the fastest fs (among ext3 and xfs)
in terms of boot recovery phase times.

So, given that XFS recovery of any size filesystem is less than one second,
reiserfs recovery is so much less than 1 second that you notice the
difference? From:
http://www.enterprisenetworkingplanet.com/netos/article.php/623661/XFS-Its-worth-the-wait.htm

XFS also provides file system journaling. This means that XFS uses database
recovery techniques to recover a consistent file system state after a system
crash. Using journaling, XFS is able to accomplish this recovery in under a
second, regardless of the file system size.

[1] http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html

Here you quoted misinformation, because the opinions were based on experience
the OPs had with the software before it was patched to fix the problem, i.e.
more than 3 years ago. You treated this as empirical evidence in your
argument against XFS. I have shown this to be incorrect.

[2]
http://xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_see_binary_NULLS_in_some_files_after_recovery_when_I_unplugged_the_power.3F

You quoted this FAQ item solely based on the tile, without reading it, in your
effort to denounce XFS. The article clearly states the problem was fixed over
3 years ago, which you conveniently ignored.

From now on, please get your facts straight, with proper documentation, before
trying to denounce a fantastic piece of FOSS into which many top-of-their-game
kernel engineers have put tens of thousands of man hours, striving to make it
the best it can be--and are wildly succeeding.

Join the xfs mailing list and you might learn something useful in place of
this trash you're talking about it. Better yet, read what Hans Reiser had to
say about XFS. He was totally enamored with it, and wanted to duplicate many
of its features:

http://www.osnews.com/story/69

Hans Reiser:
XFS is an excellent file system, and there is an important area where XFS is
higher performance than we are...ReiserFS does a complete tree traversal for
every 4k block it writes, and then it inserts one pointer at a time into the
tree, which means that every 4k write incurs the overhead of a balancing of
the tree (which means it moves data around). For this reason, XFS has better
very large file performance...If you want to stream multi-media data for
Hollywood style applications, or use ACLs now rather than wait for Reiser4,
you might want to use XFS.

This is an area we are still experimenting with. We currently do what ext2
does, and preallocate blocks. What XFS does is much better, they

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread Lisi

On Tuesday 27 July 2010 09:51:53 Stan Hoeppner wrote:
 Lisi put forth on 7/27/2010 2:23 AM:
  On Tuesday 27 July 2010 08:10:15 Stan Hoeppner wrote:
  XFS which is superior to all other Linux filesystems.
 
  Stan -
 
  Have you the time to give a rationale for this?

 Sure.

Thanks, Stan, for a lucid and erudite exposition.  Much appreciated.

Lisi


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/201007271731.34069.lisi.re...@gmail.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

On Tue, Jul 27, 2010 at 6:19 PM, Stan Hoeppner s...@hardwarefreak.comwrote:

 Volkan YAZICI put forth on 7/27/2010 8:22 AM:

  You are missing a very important point: Durability to power failures.
  (Excuse me, but a majority of GNU/Linux users are not switched to a UPS
  or something.) And that's where XFS totally fails[1][2].

  [1]
 http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html

 

  a fantastic piece of FOSS into which many top-of-their-game
 kernel engineers have put tens of thousands of man hours, striving to make
 it
 the best it can be--and are wildly succeeding.

 That's was very informative, thanks. You got me curious and I will test XFS
on my home system. To be honest I am still  little wary of using XFS in a
production environment. For years now I have heard stories of power failures
with catastrophic results when using XFS. Anyone who using XFS in
a mission critical production environment? Anyone has experience with that?

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Aaron Toponce put forth on 7/27/2010 10:41 AM:

 XFS has also had a history for randomly corrupting data. While this
 might have improved over time, I don't trust it.

Can you cite or reference anything to back your claim?  Time frame?  Irix or
Linux?  Serious users reported this or casual/hobbyist users?  If this was
ever the case the situation could not have lasted long before patches fixed
it.  Have you seen SGI's customer list and the size of the systems and storage
they run with nothing but XFS?  For instance, NAS has over 1.4PB of XFS
filesystems, 1PB CXFS and over 400TB XFS:

http://www.nas.nasa.gov/Resources/Systems/columbia.html
http://www.nas.nasa.gov/Resources/Systems/archive_storage.html

NASA trusts it with over 1PB of storage, but _you_ don't trust it?  Who are
you again?  How many hundreds of TB of storage do you manage on EXT3/4? ;)

-- 
Stan


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c4f1551.8010...@hardwarefreak.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes:
 Volkan YAZICI put forth on 7/27/2010 8:22 AM:
 1.  Never quote forum or email posts as empirical or reliable evidence of
 anything.

You're right, my bad.

 You quoted this FAQ item solely based on the tile, without reading it,
 in your effort to denounce XFS. The article clearly states the problem
 was fixed over 3 years ago, which you conveniently ignored.

I read the very same sentence, but AFAIK, default kernel for xfs bundled
with lenny doesn't have that fix.

 From now on, please get your facts straight, with proper
 documentation, before trying to denounce a fantastic piece of FOSS
 into which many top-of-their-game kernel engineers have put tens of
 thousands of man hours, striving to make it the best it can be--and
 are wildly succeeding.

 Join the xfs mailing list and you might learn something useful in
 place of this trash you're talking about it.

About a year ago, in a similar rush to yours, I ported two of our
PostgreSQL database servers to XFS. During testing period, I even
couldn't *recover* the / fs after the very first power failure test.
Whole testing period took 1 week and the result was negative. This is my
experience with XFS, and not much more thrash than your technical
knowledge. And instead of being a technology zealot, you'd be better put
forward some real world case scenarios. Try unplugging your xfs machineS
that many timeS, and let's discuss this topic again. Yep, my findings
might be deprecated, but I don't know any others investigating the same
subject with recent versions.

BTW, I still couldn't understand your temper and rudeness. I just share
my experience, and try it, it works.


Regards.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87sk34u9j7@alamut.alborz.net

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread Aaron Toponce

On 7/27/2010 11:20 AM, Stan Hoeppner wrote:
 Aaron Toponce put forth on 7/27/2010 10:41 AM:
 
 XFS has also had a history for randomly corrupting data. While this
 might have improved over time, I don't trust it.
 
 Can you cite or reference anything to back your claim?  Time frame?  Irix or
 Linux?  Serious users reported this or casual/hobbyist users?  If this was
 ever the case the situation could not have lasted long before patches fixed
 it.  Have you seen SGI's customer list and the size of the systems and storage
 they run with nothing but XFS?  For instance, NAS has over 1.4PB of XFS
 filesystems, 1PB CXFS and over 400TB XFS:

We have used it three times in the past, and lost about 5TB worth of
data due to corruption. The data corruption appeared to not be the
result of lost power to the drive. Imperical evidence is enough for me
to stop trusting it.

I've also had friends who are admins that have complained of XFS data
corruption, mainly with regards to booting. I don't know their specific
scenarios, but they stopped using XFS as well.

 NASA trusts it with over 1PB of storage, but _you_ don't trust it?  Who are
 you again?  How many hundreds of TB of storage do you manage on EXT3/4? ;)

I guess NASA has us beat.  Nothing in the PB range, that's for sure.

Currently, at my location, we have about 40 TB of SAN, with another 50
TB on the way. In production, we have about 200 TB SAN. We'll be
building a federated shadowing infrastructure that well have Oracle
databases in 16 different locations across the United States. We're
currently targeting about 20 TB in each of the 16 locations.

We won't be deploying XFS.

-- 
. O .   O . O   . . O   O . .   . O .
. . O   . O O   O . O   . O O   . . O
O O O   . O .   . O O   O O .   O O O



signature.asc
Description: OpenPGP digital signature

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Aniruddha put forth on 7/27/2010 9:43 AM:
 On Tue, Jul 27, 2010 at 3:22 PM, Volkan YAZICI yazic...@ttmail.com wrote:
 

 You are missing a very important point: Durability to power failures.
 (Excuse me, but a majority of GNU/Linux users are not switched to a UPS
 or something.) And that's where XFS totally fails[1][2].

 
 Ext3 has the same problems when not properly configured:
 
 Ext3 does not do checksumming when writing to the journal. If barrier=1 is
 not enabled as a mount option (in /etc/fstab), and if the hardware is doing
 out-of-order write caching, one runs the risk of severe filesystem
 corruption during a crash.
 http://en.wikipedia.org/wiki/Ext3#No_checksumming_in_journal
 
 For the record I use ext3, I remember XFS as not being reliable enough
 (with power failures etc).

This isn't a filesystem problem, or a kernel problem, or any other technical
problem.  This is a user problem.  You will _never_ get computing technology
the fully does what you _think_ it should upon loss of power.  Period.  XFS
will prevent filesystem corruption (lookup the definition) but it will not
prevent data loss.  These are two completely different things.  _No_
filesystem will fully prevent data loss when power is lost, but most will
prevent filesystem corruption.  Again, these are two different things.

If you want maximum performance, you have to enable drive caches.  Doing so
causes more data loss when the power goes, and again, it's not the fault of
the filesystem.  If you want maximum protection against data loss, you have to
disable drive caches, reduce the size of the in memory journal log buffer,
etc, etc.  Doing all of these things will absolutely murder your FS
performance.  This is a balancing act folks.  You can't have your cake and eat
it too.

I'd also like to add that anyone smart enough to be on this list is smart
enough to know you should have a UPS, regardless of what filesystem you use.
If you're not you shouldn't be here.  If you disagree on the technical merits
(not cost), you're uneducated and/or stubborn.  If you disagree on a cost
basis, your data isn't valuable, period.  A decent low end UPS for a desktop
system that will get you through all brown outs and far enough through a storm
outage (15-30 minutes) to do a proper shutdown costs about $50 USD.  That's
less than a carton of cigarettes in New York City, less than 3 regular price
large pizzas at Dominos, and $25 less than a tank of gas for a full size
pickup, which would last most people one week of commute.  The cost of one
tank of gas for 3-5 years of power protection before needing a battery
replacement.

I guess I should evangelize UPS as much as XFS given the benefits.  Except XFS
is free. ;)

-- 
Stan


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c4f17db.7010...@hardwarefreak.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes:
 NASA trusts it with over 1PB of storage, but _you_ don't trust it?  Who are
 you again?  How many hundreds of TB of storage do you manage on EXT3/4? ;)

NASA also trusts Windows and NTFS too? Who are you again?

I think you are confusing apples and oranges. Everbody's requirements
might differ, and hence do their tools. Instead of being a tech zealot,
one just need to choose the right tool for the right job.

I don't even think Linus is using XFS too. Isn't he a technical person
in terms of your definition? So what should we do in that case? Ask to
RMS?


Regards.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87ocdsu93o@alamut.alborz.net

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread Paul Cartwright

On Tue July 27 2010, Stan Hoeppner wrote:
 I'd also like to add that anyone smart enough to be on this list is smart
 enough to know you should have a UPS, regardless of what filesystem you
 use. If you're not you shouldn't be here.  If you disagree on the technical
 merits (not cost), you're uneducated and/or stubborn.  If you disagree on a
 cost basis, your data isn't valuable, period.  A decent low end UPS for a
 desktop system that will get you through all brown outs and far enough
 through a storm outage (15-30 minutes) to do a proper shutdown costs about
 $50 USD.  That's less than a carton of cigarettes in New York City, less
 than 3 regular price large pizzas at Dominos, and $25 less than a tank of
 gas for a full size pickup, which would last most people one week of
 commute.  The cost of one tank of gas for 3-5 years of power protection
 before needing a battery replacement.

this is something that I preach to EVERYONE who has a computer. Some people 
don't understand that I leave my computer plugged in  running 24/7. they 
give me a deer-in-the-headlights look when I tell them I don't turn my 
computer off.  But I live in Georgia, home of MASSIVE thunderstorms. I also 
live at the end of a street with 110 foot tall oak  pine trees along side 
the road, and right next to our electric poles. In 5 years we have had 3 
trees drop on wires  cause loss of power, and I've had up to 20+ entries in 
the apcupsd.log file in ONE day, from thunder boomers. I have THREE UPSes in 
my house, not just my PC, but ALL electronic equipment, TV, stero AND Dish 
satellite receiver. I would NEVER plug anything electronic in, in MY house 
WITHOUT an UPS.

 I guess I should evangelize UPS as much as XFS given the benefits.  Except
 XFS is free. ;)
UPSes are really cheap and EXCELLENT insurance for not only your hardware, but 
your DATA. I won't even mention a company, and I DON'T work for them!


-- 
Paul Cartwright
Registered Linux user # 367800
Registered Ubuntu User #12459


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/201007271342.01889.deb...@pcartwright.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread Mark

On Tue, Jul 27, 2010 at 10:29 AM, Volkan YAZICI yazic...@ttmail.com wrote:

 On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes:
  NASA trusts it with over 1PB of storage, but _you_ don't trust it?  Who
 are
  you again?  How many hundreds of TB of storage do you manage on EXT3/4?
 ;)

 NASA also trusts Windows and NTFS too?


NASA also backs up their data on  5.25 floppy disks [1].

[1] *completely made up information

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread B. Alexander

We use XFS in production at work. Where I work, we are routinely dealing
with hundreds of terabytes of data (I have heard the word petabyte bandied
about in several meetings), so we are beyond or hovering on the edge of the
size limits and performance limits of the ext filesystems.

At home, I primarily do reiserfs, for the simple reason that I have had need
in the past (more than one would guess) where I have needed to shrink a
filesystem. In fact, I needed to do so on a box at work.

Right now, I am trying to get my brain around the improvements in btrfs, and
hoping that will take off as many say it will.



On Tue, Jul 27, 2010 at 1:03 PM, Aniruddha mailingdotl...@gmail.com wrote:

 On Tue, Jul 27, 2010 at 6:19 PM, Stan Hoeppner s...@hardwarefreak.comwrote:

 Volkan YAZICI put forth on 7/27/2010 8:22 AM:

  You are missing a very important point: Durability to power failures.
  (Excuse me, but a majority of GNU/Linux users are not switched to a UPS
  or something.) And that's where XFS totally fails[1][2].

  [1]
 http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html

 

  a fantastic piece of FOSS into which many top-of-their-game
 kernel engineers have put tens of thousands of man hours, striving to make
 it
 the best it can be--and are wildly succeeding.

 That's was very informative, thanks. You got me curious and I will test
 XFS on my home system. To be honest I am still  little wary of using XFS in
 a production environment. For years now I have heard stories of power
 failures with catastrophic results when using XFS. Anyone who using XFS in
 a mission critical production environment? Anyone has experience with that?

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes:
 I'd also like to add that anyone smart enough to be on this list is smart
 enough to know you should have a UPS, regardless of what filesystem you use.
 If you're not you shouldn't be here.  If you disagree on the technical merits
 (not cost), you're uneducated and/or stubborn.

You are still not getting it, don't you? We have thousands of embedded
linuxes in the wild and they are just simple data aggragetors. You can't
have a power backup unit in such a condition.

I'd also like to add that anyone smart enough to be on this list is
smart enough to know you cannot have a UPS for embedded systems,
regardless of what filesystem you use. If you're not you shouldn't be
here. If you disagree on the technical merits (not cost), you're
uneducated and/or stubborn.


Regards.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87k4ogu7on@alamut.alborz.net

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread Klistvud


Dne, 27. 07. 2010 19:48:53 je Mark napisal(a):


 NASA also trusts Windows and NTFS too?


NASA also backs up their data on  5.25 floppy disks [1].

[1] *completely made up information




NASA as an authority on reliable storage? C'mon, the bozos can't even  
be trusted with their own Challengers, Columbias, OR the astronauts  
therein!


--
Regards,

Klistvud
Certifiable Loonix User #481801
http://bufferoverflow.tiddlyspot.com


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/1280254750.196...@compax

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Aniruddha put forth on 7/27/2010 12:03 PM:
 On Tue, Jul 27, 2010 at 6:19 PM, Stan Hoeppner s...@hardwarefreak.comwrote:
 
 Volkan YAZICI put forth on 7/27/2010 8:22 AM:

 You are missing a very important point: Durability to power failures.
 (Excuse me, but a majority of GNU/Linux users are not switched to a UPS
 or something.) And that's where XFS totally fails[1][2].

 [1]
 http://linux.derkeiler.com/Mailing-Lists/Debian/2008-11/msg00097.html

 
 
  a fantastic piece of FOSS into which many top-of-their-game
 kernel engineers have put tens of thousands of man hours, striving to make
 it
 the best it can be--and are wildly succeeding.

 That's was very informative, thanks. You got me curious and I will test XFS
 on my home system. To be honest I am still  little wary of using XFS in a
 production environment. For years now I have heard stories of power failures
 with catastrophic results when using XFS. Anyone who using XFS in
 a mission critical production environment? Anyone has experience with that?

How about, and this will probably shock many of you:

1. Kernel.org

All of Linux source, including what becomes the Debian kernel, and the kernels
of all other Linux distros, is served from XFS filesystems:

A bit more than a year ago (as of October 2008) kernel.org, in an ever
increasing need to squeeze more performance out of it's machines, made the
leap of migrating the primary mirror machines (mirrors.kernel.org) to XFS. We
site a number of reasons including fscking 5.5T of disk is long and painful,
we were hitting various cache issues, and we were seeking better performance
out of our file system.

After initial tests looked positive we made the jump, and have been quite
happy with the results. With an instant increase in performance and
throughput, as well as the worst xfs_check we've ever seen taking 10 minutes,
we were quite happy. Subsequently we've moved all primary mirroring
file-systems to XFS, including www.kernel.org , and mirrors.kernel.org. With
an average constant movement of about 400mbps around the world, and with peaks
into the 3.1gbps range serving thousands of users simultaneously it's been a
file system that has taken the brunt we can throw at it and held up
spectacularly.


2. NASA Advanced Supercomputing Facility, NASA Ames Research Center
See my other post for details

3.  Industrial Light and Magic -- ILM
At one time ILM had one of the largest installed SGI SAN storage systems on
the planet, may have been _the_ largest, running XFS.  It backed their render
farm(s).  They don't currently have any render system info on their site that
I can find.  Given the number, size, and scope of their animation projects and
the size to which their rendering farm has grown, they may have very well
switched SAN vendors over the years.  I don't know if they still use XFS or
not.  I would think so given that they're working with multi hundred gigabyte
files daily.


Many, many others.  What you have to understand is that XFS has been around a
long long time, 17 years in both IRIX and Linux.  It's older than EXT2.  Back
before cheap Intel/AMD clusters took over the supercomputing marketplace, SGI
MIPS IRIX systems with XFS owned upwards of 30-40% of that market.  XFS in
various platforms and versions has been in government labs, corporations and
academia for over a decade.  At one time Prof Stephen Hawking had his own
personal 32 CPU SGI Origin 3800 for running cosmology calculations in order
to prove his theories.  It had XFS filesytems, as have all SGI systems since
1994.

Here's a list of organizations that have volunteered information to xfs.org.
It is by far not a complete list, and most of the major SGI customers with XFS
on huge SAN systems aren't listed.  Note NAS at NASA Ames isn't listed.

http://xfs.org/index.php/XFS_Companies

-- 
Stan


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c4f25d5.1040...@hardwarefreak.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread John Hasler

Stan Hoeppner writes:
 I'd also like to add that anyone smart enough to be on this list is
 smart enough to know you should have a UPS, regardless of what
 filesystem you use.  If you're not you shouldn't be here.

I guess I don't belong here then...
-- 
John Hasler


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87ocdsrdt0@thumper.dhh.gt.org

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Volkan YAZICI put forth on 7/27/2010 12:19 PM:

 About a year ago, in a similar rush to yours, I ported two of our
 PostgreSQL database servers to XFS. During testing period, I even
 couldn't *recover* the / fs after the very first power failure test.

What write operations were you performing at the time you pulled the plug?
Unless you were writing the superblock it'd be almost impossible to hose the
filesystem to the point it couldn't mount.  Were you doing a resize operation
when you pulled the plug?  xfs_growfs?  As far as recovery, it's automatic
upon mounting the XFS filesystem.  What do you mean, precisely, by couldn't
*recover* the / fs?

And apologies if my tone seemed rude.  I'm basically a one man army trying to
defeat misinformation WRT XFS and attempt to educate ppl with the correct
information.  I guess I'm feeling outnumbered and thus being more aggressive.
 Again, apologies.

-- 
Stan


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c4f27d6.1070...@hardwarefreak.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Volkan YAZICI put forth on 7/27/2010 12:29 PM:

 I don't even think Linus is using XFS too. Isn't he a technical person

Linus uses them all.  You should know that.

 in terms of your definition? So what should we do in that case? Ask to
 RMS?

kernel.org servers all run XFS as well.

-- 
Stan


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c4f299a.9090...@hardwarefreak.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes:
 What write operations were you performing at the time you pulled the plug?
 Unless you were writing the superblock it'd be almost impossible to hose the
 filesystem to the point it couldn't mount.  Were you doing a resize operation
 when you pulled the plug?  xfs_growfs?  As far as recovery, it's automatic
 upon mounting the XFS filesystem.  What do you mean, precisely, by couldn't
 *recover* the / fs?

Vanilla XFS with noatime,notail like basic mount options. The test was
simple, I was just typing SELECT 1 from a psql command line (this
query shouldn't even hit to disk, it just basically returns 1) and
unplugged machine. At boot, I dropped to fsck command line. At command
prompt, I manually fiddled around with fsck of xfs to recover the
unmounted / filesystem, but had no luck. (I also tried recommendations
and informative messages supplied by manpages and command
outputs/warnings.) Also if you would Google, it shouldn't be hard to
spot similar experiences from other people.

At NASA, they might have genius technicians; but, IMHO a majority of the
linux users would want a filesystem to recover without a prompt from the
user.

 I'm basically a one man army trying to defeat misinformation WRT XFS
 and attempt to educate ppl with the correct information.

I am glad -users ml have you; and I'd be really, really appreaciated if
somebody having experience and knowledge on fs issues can shed some
light to our ignorance. I also support the replacement of default fs
with something that is much more recent. From this point of view, XFS is
a superior alternative. You are totally right with your claims about its
advantages over other alternatives. But as you can see, people still
complain about XFS's sensitivity to power failures. Assuming a majority
of your users aren't behind a UPS, you can sell/ship your product with
such a default filesystem choice. But as you said, there are no
published concrete benchmarks about this issue. It is all what people
claim in the mailing lists. If you would share some of your findings
about Power Failures and XFS to convince us, I'm sure most of us will
be happy to advocate XFS's this achievement.


Regards.


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87fwz4u4oi@alamut.alborz.net

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Volkan YAZICI put forth on 7/27/2010 12:59 PM:
 On Tue, 27 Jul 2010, Stan Hoeppner s...@hardwarefreak.com writes:
 I'd also like to add that anyone smart enough to be on this list is smart
 enough to know you should have a UPS, regardless of what filesystem you use.
 If you're not you shouldn't be here.  If you disagree on the technical merits
 (not cost), you're uneducated and/or stubborn.
 
 You are still not getting it, don't you?  We have thousands of embedded

I get it now.  This is the first you mentioned embedded systems.

 linuxes in the wild and they are just simple data aggragetors. You can't
 have a power backup unit in such a condition.

Sure you can:
http://www.embeddedarm.com/products/board-detail.php?product=TS-BAT3

And they make 'em even much smaller than that.  If you get the right books,
_you_ can make 'em smaller than that. :)

-- 
Stan


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c4f3082.8090...@hardwarefreak.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Time for some testing, I will put Debian stable with  XFS on my laptop and
see how well it deals with power failures :)

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Volkan YAZICI put forth on 7/27/2010 2:04 PM:

 unplugged machine. At boot, I dropped to fsck command line. At command

Were you forced to the command line or did you manually select to go to the
command line?  It sounds like you chose to, not forced to.

 prompt, I manually fiddled around with fsck of xfs to recover the
 unmounted / filesystem, but had no luck. 

Did you read the xfs documentation before embarking on this power loss
experiment?  Or did you it should just work regardless of your actions, or
lack of action?  It sounds like you ran xfs_repair on a filesystem in an
inconsistent state and forced changes, which is a no-no.

(I also tried recommendations
 and informative messages supplied by manpages and command
 outputs/warnings.) Also if you would Google, it shouldn't be hard to
 spot similar experiences from other people.

I'm guessing most of them didn't look before taking the XFS leap.

 At NASA, they might have genius technicians; but, IMHO a majority of the
 linux users would want a filesystem to recover without a prompt from the
 user.

So the system wouldn't boot and you were dropped to a prompt.  You manually
fiddled around with fsck of xfs and made no progress.  It would be nice to
have seen all of that at the time.

What were your results when you did this same power yank test with ext2/3,
ReiserFS, and the other filesystems you tested in this way?

 I'm basically a one man army trying to defeat misinformation WRT XFS
 and attempt to educate ppl with the correct information.
 
 I am glad -users ml have you; and I'd be really, really appreaciated if
 somebody having experience and knowledge on fs issues can shed some
 light to our ignorance. I also support the replacement of default fs
 with something that is much more recent. From this point of view, XFS is
 a superior alternative. You are totally right with your claims about its
 advantages over other alternatives. But as you can see, people still
 complain about XFS's sensitivity to power failures. Assuming a majority
 of your users aren't behind a UPS, you can sell/ship your product with
 such a default filesystem choice. But as you said, there are no
 published concrete benchmarks about this issue. It is all what people
 claim in the mailing lists. If you would share some of your findings
 about Power Failures and XFS to convince us, I'm sure most of us will
 be happy to advocate XFS's this achievement.

I've tried to dig up accurate accounts of the power loss corruption issue post
2007 (when it was supposed to have been fixed) a few times but couldn't find
anything concrete enough to be worth referencing.  I freely admit I've done no
power loss testing of XFS myself.  This probably has to do with the fact that
I'm a firm believer in orderly shutdowns and redundant power, and that I don't
really have any test systems available.  I'll ask around on the XFS list and
see what folks have to say.

I'm somewhat interested in seeing where BTRFS is in 2-3 years.  It may be
stable enough for production by then, and should be as fast or faster than XFS
on some workloads.  Maybe it'll even handle sudden power loss gracefully. :)

-- 
Stan



-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c4f3ce4.2080...@hardwarefreak.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

Aniruddha put forth on 7/27/2010 2:47 PM:
 Time for some testing, I will put Debian stable with  XFS on my laptop and
 see how well it deals with power failures :)

Thanks for picking up the torch/gauntlet/whatever.  How do you laptop test
this issue?  It's a laptop.  Yank the battery?  Yanking the wal-wart
(transformer) won't do diddly.

-- 
Stan



-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c4f3dd8.9070...@hardwarefreak.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

On Tue, Jul 27, 2010 at 10:13 PM, Stan Hoeppner s...@hardwarefreak.com wrote:
 Aniruddha put forth on 7/27/2010 2:47 PM:
 Time for some testing, I will put Debian stable with  XFS on my laptop and
 see how well it deals with power failures :)

 Thanks for picking up the torch/gauntlet/whatever.  How do you laptop test
 this issue?  It's a laptop.  Yank the battery?  Yanking the wal-wart
 (transformer) won't do diddly.

I'll use it 'till runs out of battery. :)


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/aanlktimnqtfz4ozsbym+28wcotynoufhgrqea+bep...@mail.gmail.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

On Tue, Jul 27, 2010 at 11:06 PM, Aniruddha mailingdotl...@gmail.com wrote:
 On Tue, Jul 27, 2010 at 10:13 PM, Stan Hoeppner s...@hardwarefreak.com 
 wrote:
 Aniruddha put forth on 7/27/2010 2:47 PM:
 Time for some testing, I will put Debian stable with  XFS on my laptop and
 see how well it deals with power failures :)

 Thanks for picking up the torch/gauntlet/whatever.  How do you laptop test
 this issue?  It's a laptop.  Yank the battery?  Yanking the wal-wart
 (transformer) won't do diddly.

 I'll use it 'till runs out of battery. :)


Or even better, I'll use virtualbox to see what happens when I plug the power.


--
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/aanlk...@kemojxop36vj-wtcr7910eowbfm0bu9ejz@mail.gmail.com

Re: Linux filesystems was [Re: Debian cd supporting ext4.]

2010-07-27 Thread Rob Owens

On Tue, Jul 27, 2010 at 01:39:18PM -0500, Stan Hoeppner wrote:
 Volkan YAZICI put forth on 7/27/2010 12:19 PM:
 
  About a year ago, in a similar rush to yours, I ported two of our
  PostgreSQL database servers to XFS. During testing period, I even
  couldn't *recover* the / fs after the very first power failure test.
 
 What write operations were you performing at the time you pulled the plug?
 Unless you were writing the superblock it'd be almost impossible to hose the
 filesystem to the point it couldn't mount.  Were you doing a resize operation
 when you pulled the plug?  xfs_growfs?  As far as recovery, it's automatic
 upon mounting the XFS filesystem.  What do you mean, precisely, by couldn't
 *recover* the / fs?
 
Some anecdotal evidence in support of ext3's resilience to power loss:

I recently lost power while my system was running.  When power was
restored, an fsck was automatically performed.  During that fsck, I lost
power again!  I thought for sure I'd be hosed, but after the power came
back and an fsck completed, everything seems to be working normally.

-Rob


-- 
To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org 
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20100727213610.ga14...@aurora.owens.net

Re: Linux filesystems was [Re: Debian cd supporting ext4.]