Re: Filesystem for backup system

[Roberto Ragusa]

On 10/10/2016 05:25 PM, Alex wrote:
> Hi,
> 
> I've built an 11TB RAID5 array using fedora23 (soon to be fedora24) to
> be used for backup for a bunch of mail and web servers, and thought
> I'd inquire about the best filesystem to use.
> 
> I suppose I'm most familiar with ext4, although I understand xfs is
> the default filesystem during install now.

I've been doing this for more than ten years: massive backups with
rsync and --link-dest.

I started with reiserfs and it went on for more than ten years without
any problem.
Then I rebuilt the machines and decided to move to ext4.
Ironically enough, I got into a serious bug in ext4 online expansion
that caused data corruption. It is now solved, of course.
Today, I would go for ext4. I have just a couple of system still
on reiserfs, with very old (read: proven and trusted) software versions.

Regards.

-- 
   Roberto Ragusamail at robertoragusa.it
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Gordon Messmer]

On 10/16/2016 05:42 AM, George N. White III wrote:



I don't remember the date, specifically, and it's not clear what
the circumstances are.  One of the consequences of on-line fs
checking is that errors tend to be detected long after they
actually occur.


Not sure what you mean -- certainly with most linux distros fsck gets 
run at intervals on clean reboot and does on rare occasions find 
problems to repair.




Yes, but as we've discussed, fsck.xfs (and hence, fsck -t xfs) is a 
no-op.  XFS is checked and repaired on-line, as it's used, in the case 
of simple errors.  If corruption is introduced, it may not be found 
until some time later, in which case xfs_repair will be needed.

___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[George N. White III]

On Sat, Oct 15, 2016 at 6:25 PM, Gordon Messmer 
wrote:

> On 10/13/2016 11:49 PM, c...@zip.com.au wrote:
>
>> Out of interest, how recent was your need to recover XFS, and what were
>> the circumstances that caused it to be necessary?
>>
>
>
> I don't remember the date, specifically, and it's not clear what the
> circumstances are.  One of the consequences of on-line fs checking is that
> errors tend to be detected long after they actually occur.


Not sure what you mean -- certainly with most linux distros fsck gets run
at intervals on clean reboot and does on rare occasions find problems to
repair.

I've never seen a problem with XFS filesystems on Scientific Linux 7, but I
have a lot more system hours experience with xfs on SGI Irix, where the
first indication that a disk was failing would come from errors running the
nightly fsr.  When XFS was first ported to linux I used it on 32-bit linux
to recover data disks removed from dead SGI workstations.  The XFS
utilities were extremely slow in that environment.  One reason given for
the poor performance was that XFS was written with deeply nested function
calls with long argument lists that were very inefficient on 32-bit Intel
hardware at that time.


-- 
George N. White III 
Head of St. Margarets Bay, Nova Scotia
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Gordon Messmer]

On 10/13/2016 11:49 PM, c...@zip.com.au wrote:
Out of interest, how recent was your need to recover XFS, and what 
were the circumstances that caused it to be necessary? 



I don't remember the date, specifically, and it's not clear what the 
circumstances are.  One of the consequences of on-line fs checking is 
that errors tend to be detected long after they actually occur.

___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[cs]

On 13Oct2016 14:29, Gordon Messmer  wrote:

On 10/12/2016 03:57 PM, c...@zip.com.au wrote:

Except it the wildest scenarios, XFS fsks at mount, almost immediately.


Is that different from fsck.ext4 replaying the journal?


Not fundamentally.


Go and cat (yes, cat) the fsck.xfs command.


OK.  I'm not sure what you think I'll learn by doing so.


Only that it is a shell script that handles the typically used boot options, 
but doesn't do anything else.


fsck.xfs exists because Linux's "fsck" supports multiple filesystems and 
Irix's did not.  On Irix, fsck was exclusively for EFS.  XFS used xfs_check 
and xfs_repair.  When XFS was ported to Linux, the tools were not renamed; 
instead fsck.xfs was put in place to direct users to the correct tool when 
they ran "fsck -t xfs".


And when they really need a repair that the XFS mount process doesn't 
accomplish.


None of that makes XFS immune to corruption, nor reduce the time or 
the memory required to fix an XFS filesystem if it's damaged.


Of course not, though I have the impression that the repair tool is less memory 
intensive than the ext4 equivalent.


"Never" is indeed not _literally_ true, but it is effectively true, 
far far far far more than is so with ext4. Ext4 really needs fsck 
after an unclean unmount, and it is not cheap for large filesystems.


With journaling and write barriers in place, I don't know any reason 
that ext4 would be any more affected by a power loss than XFS would 
be, and having had to perform recoveries on both, I don't find one to 
be significantly better than the other.


Fair enough. We may be into the realm where a question on an XFS list may be 
more illuminating about the modern state of play.


Thus, my advice remains that users should test both because they do 
differ in performance in different workloads.  If one will be faster 
under normal operation, 99.95% of the time, and require slightly more 
down time to recover in the other .05% of the time, then selecting the 
option which is superior 99.95% of the time is a perfectly rational 
thing to do.


That's especially true of backup systems where down time does not 
carry the same impact as down time in a production system does.


The two are like night and day in the recovery scenario (== xfs 
pretty much never needs manual recovery, and recover is very fast).


That seems subjective.  I've personally had to recover more XFS 
filesystems than I have ext4 filesystems, and I use ext4 filesystems more 
often.  My subjective experience is quite different than yours.


Apparently so. I've had very good experiences with XFS, and plan to stick with 
it at present.


Out of interest, how recent was your need to recover XFS, and what were the 
circumstances that caused it to be necessary?


Cheers,
Cameron Simpson 
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Gordon Messmer]

On 10/12/2016 03:57 PM, c...@zip.com.au wrote:

Except it the wildest scenarios, XFS fsks at mount, almost immediately.


Is that different from fsck.ext4 replaying the journal?


Go and cat (yes, cat) the fsck.xfs command.


OK.  I'm not sure what you think I'll learn by doing so.  fsck.xfs 
exists because Linux's "fsck" supports multiple filesystems and Irix's 
did not.  On Irix, fsck was exclusively for EFS.  XFS used xfs_check and 
xfs_repair.  When XFS was ported to Linux, the tools were not renamed; 
instead fsck.xfs was put in place to direct users to the correct tool 
when they ran "fsck -t xfs".


None of that makes XFS immune to corruption, nor reduce the time or the 
memory required to fix an XFS filesystem if it's damaged.


"Never" is indeed not _literally_ true, but it is effectively true, 
far far far far more than is so with ext4. Ext4 really needs fsck 
after an unclean unmount, and it is not cheap for large filesystems.


With journaling and write barriers in place, I don't know any reason 
that ext4 would be any more affected by a power loss than XFS would be, 
and having had to perform recoveries on both, I don't find one to be 
significantly better than the other.


Thus, my advice remains that users should test both because they do 
differ in performance in different workloads.  If one will be faster 
under normal operation, 99.95% of the time, and require slightly more 
down time to recover in the other .05% of the time, then selecting the 
option which is superior 99.95% of the time is a perfectly rational 
thing to do.


That's especially true of backup systems where down time does not carry 
the same impact as down time in a production system does.


The two are like night and day in the recovery scenario (== xfs pretty 
much never needs manual recovery, and recover is very fast).


That seems subjective.  I've personally had to recover more XFS 
filesystems than I have ext4 filesystems, and I use ext4 filesystems 
more often.  My subjective experience is quite different than yours.

___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

RE: Filesystem for backup system

[J.Witvliet]

-Original Message-
From: c...@zip.com.au [mailto:c...@zip.com.au]
Sent: donderdag 13 oktober 2016 0:58
To: Community support for Fedora users
Subject: Re: Filesystem for backup system

On 12Oct2016 09:40, Mike Wright <nob...@nospam.hostisimo.com> wrote:
>On 10/12/2016 08:53 AM, Heinz Diehl wrote:
>>On 10.10.2016, Gordon Messmer wrote:
>>
>>>Ext4 is probably a better option for a filesystem with a large number
>>>of small files.  XFS continues to be slower for metadata operations.
>>
>>It was, some years ago. This is no longer the case.
>
>Given those, would you go so far as to recommend xfs in lieu of ext4
>for general usage?

Yes.

Cameron Simpson <c...@zip.com.au>
___
users mailing list -- users@lists.fedoraproject.org To unsubscribe send an 
email to users-le...@lists.fedoraproject.org

How about btrfs, (at least the stable parts of it :-)


Dit bericht kan informatie bevatten die niet voor u is bestemd. Indien u niet 
de geadresseerde bent of dit bericht abusievelijk aan u is toegezonden, wordt u 
verzocht dat aan de afzender te melden en het bericht te verwijderen. De Staat 
aanvaardt geen aansprakelijkheid voor schade, van welke aard ook, die verband 
houdt met risico's verbonden aan het elektronisch verzenden van berichten.

This message may contain information that is not intended for you. If you are 
not the addressee or if this message was sent to you by mistake, you are 
requested to inform the sender and delete the message. The State accepts no 
liability for damage of any kind resulting from the risks inherent in the 
electronic transmission of messages.
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[cs]

On 12Oct2016 09:40, Mike Wright  wrote:

On 10/12/2016 08:53 AM, Heinz Diehl wrote:

On 10.10.2016, Gordon Messmer wrote:


Ext4 is probably a better option for a filesystem with a large number of small 
files.  XFS
continues to be slower for metadata operations.


It was, some years ago. This is no longer the case.


Given those, would you go so far as to recommend xfs in lieu of ext4 for 
general usage?


Yes.

Cameron Simpson 
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[cs]

On 12Oct2016 08:40, Gordon Messmer  wrote:

On 10/12/2016 06:40 AM, Michael Cronenworth wrote:

"Never needs fsck"? What crazy alternate reality do you live in?


It's only slightly exaggerated. XFS has online fsck, which means that 
the kernel can fix some errors as it encounters them. Others... well, 
I *have* seen XFS require an offline fsck on a Linux NAS appliance, so I know 
"never" isn't literally true.


Except it the wildest scenarios, XFS fsks at mount, almost immediately. Go and 
cat (yes, cat) the fsck.xfs command. Its data processes are reliable and well 
behaved, and it is extensively tested.


"Never" is indeed not _literally_ true, but it is effectively true, far far far 
far more than is so with ext4. Ext4 really needs fsck after an unclean unmount, 
and it is not cheap for large filesystems.


The two are like night and day in the recovery scenario (== xfs pretty much 
never needs manual recovery, and recover is very fast). Even mkfs is light 
night and day with xfs vs ext4.


Cheers,
Cameron Simpson 
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[George N. White III]

On Wed, Oct 12, 2016 at 2:03 PM, Gordon Messmer 
wrote:

> On 10/12/2016 08:53 AM, Heinz Diehl wrote:
>
>> It was, some years ago. This is no longer the case.
>>
>
>
> I suggest that users evaluate their options under their own workload.
> When I ran tests last year on CentOS 7 for rsnapshot storing maildirs (the
> exact workload in question here), ext4 was significantly faster on the
> hardware I was using.



There are some other considerations.  XFS has a good track record for large
numerical calculations, video production, and remote sensing.  These
are applications where the data are lost due to downtime recovering from a
crash are much larger than the data that was "in flight" when the crash
occurred.  XFS puts priority on metadata consistency, so filesystems are up
and running very quickly after a crash.

There are applications where losing a few bytes of data should "never"
happen, even it results in substantial downtime.






>
> ___
> users mailing list -- users@lists.fedoraproject.org
> To unsubscribe send an email to users-le...@lists.fedoraproject.org
>



-- 
George N. White III 
Head of St. Margarets Bay, Nova Scotia
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Gordon Messmer]

On 10/12/2016 08:53 AM, Heinz Diehl wrote:

It was, some years ago. This is no longer the case.



I suggest that users evaluate their options under their own workload.  
When I ran tests last year on CentOS 7 for rsnapshot storing maildirs 
(the exact workload in question here), ext4 was significantly faster on 
the hardware I was using.

___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Mike Wright]

On 10/12/2016 08:53 AM, Heinz Diehl wrote:

On 10.10.2016, Gordon Messmer wrote:


Ext4 is probably a better option for a filesystem with a large number of small 
files.  XFS
continues to be slower for metadata operations.


It was, some years ago. This is no longer the case.


Given those, would you go so far as to recommend xfs in lieu of ext4 for 
general usage?

___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Heinz Diehl]

On 10.10.2016, Gordon Messmer wrote: 

> Ext4 is probably a better option for a filesystem with a large number of 
> small files.  XFS
> continues to be slower for metadata operations.

It was, some years ago. This is no longer the case.
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Gordon Messmer]

On 10/12/2016 06:40 AM, Michael Cronenworth wrote:
"Never needs fsck"? What crazy alternate reality do you live in? 



It's only slightly exaggerated. XFS has online fsck, which means that 
the kernel can fix some errors as it encounters them. Others... well, I 
*have* seen XFS require an offline fsck on a Linux NAS appliance, so I 
know "never" isn't literally true.

___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Michael Cronenworth]

On 10/10/2016 04:58 PM, c...@zip.com.au wrote:
Use XFS. It is stable; never needs fsck. If ext4 needs to repair it will take 
days/weeks on a filesystem that size, and need insane amounts of RAM (if the NAS 
is hosting this, it may not have much RAM).


Both will work until you need to fsck (eg power outage or other instant/unclean 
shutdown). After that, you will wish you were using XFS. 


"Never needs fsck"? What crazy alternate reality do you live in?

Outrageous claims like this just don't have any place on this list. If you want to 
have a rant you're welcome to start a blog.

___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Alex]

Hi,

On Mon, Oct 10, 2016 at 5:58 PM,   wrote:
> On 10Oct2016 15:59, Michael Cronenworth  wrote:
>>
>> On 10/10/2016 03:50 PM, Alex wrote:
>
> Where did you read that? AFAIK the default continues to be ext4.
>>>
>>> I thought I recalled it being the default during install the last time
>>> I performed one.
>>
>>
>> The Fedora Server installation defaults to XFS. Workstation defaults to
>> ext4.
>>
>> For your use case either filesystem will work. Use what you are
>> comfortable with. Both are stable, supported, and will give you full
>> performance from your setup.
>
>
> Use XFS. It is stable; never needs fsck. If ext4 needs to repair it will
> take days/weeks on a filesystem that size, and need insane amounts of RAM
> (if the NAS is hosting this, it may not have much RAM).
>
> Both will work until you need to fsck (eg power outage or other
> instant/unclean shutdown). After that, you will wish you were using XFS.

Awesome, thanks guys.
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[cs]

On 10Oct2016 15:59, Michael Cronenworth  wrote:

On 10/10/2016 03:50 PM, Alex wrote:

Where did you read that? AFAIK the default continues to be ext4.

I thought I recalled it being the default during install the last time
I performed one.


The Fedora Server installation defaults to XFS. Workstation defaults to ext4.

For your use case either filesystem will work. Use what you are 
comfortable with. Both are stable, supported, and will give you full 
performance from your setup.


Use XFS. It is stable; never needs fsck. If ext4 needs to repair it will take 
days/weeks on a filesystem that size, and need insane amounts of RAM (if the 
NAS is hosting this, it may not have much RAM).


Both will work until you need to fsck (eg power outage or other instant/unclean 
shutdown). After that, you will wish you were using XFS.


Cheers,
Cameron Simpson 
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Michael Cronenworth]

On 10/10/2016 03:50 PM, Alex wrote:

>Where did you read that? AFAIK the default continues to be ext4.

I thought I recalled it being the default during install the last time
I performed one.



The Fedora Server installation defaults to XFS. Workstation defaults to ext4.

For your use case either filesystem will work. Use what you are comfortable with. 
Both are stable, supported, and will give you full performance from your setup.

___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Alex]

Hi,

On Mon, Oct 10, 2016 at 12:17 PM, Patrick O'Callaghan
 wrote:
> On Mon, 2016-10-10 at 11:25 -0400, Alex wrote:
>> I suppose I'm most familiar with ext4, although I understand xfs is
>> the default filesystem during install now.
>
> Where did you read that? AFAIK the default continues to be ext4.

I thought I recalled it being the default during install the last time
I performed one.

>> We're using a homegrown backup script that uses rsync with the
>> --link-dest option to preserve space. I'd like to move to bacula, but
>> don't have the time/resources right now.
>
> You might want to look at rsnapshot (www.rsnapshot.org), which is based
> on rsync.

Thanks, I'll check it out.
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Gordon Messmer]

On 10/10/2016 09:17 AM, Patrick O'Callaghan wrote:

On Mon, 2016-10-10 at 11:25 -0400, Alex wrote:

>I suppose I'm most familiar with ext4, although I understand xfs is
>the default filesystem during install now.

Where did you read that? AFAIK the default continues to be ext4.



Alex is thinking of RHEL/CentOS 7, whose default is XFS.  ext4 is 
probably a better option for a filesystem with a large number of small 
files.  XFS continues to be slower for metadata operations.


___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Re: Filesystem for backup system

[Patrick O'Callaghan]

On Mon, 2016-10-10 at 11:25 -0400, Alex wrote:
> I suppose I'm most familiar with ext4, although I understand xfs is
> the default filesystem during install now.

Where did you read that? AFAIK the default continues to be ext4.

> I believe most files will be maildir style email files, as well as
> maillogs, and some tar.bz2 files, so the file size ranges will be
> varied.
> 
> We're using a homegrown backup script that uses rsync with the
> --link-dest option to preserve space. I'd like to move to bacula, but
> don't have the time/resources right now.

You might want to look at rsnapshot (www.rsnapshot.org), which is based
on rsync.

poc
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org

Filesystem for backup system

[Alex]

Hi,

I've built an 11TB RAID5 array using fedora23 (soon to be fedora24) to
be used for backup for a bunch of mail and web servers, and thought
I'd inquire about the best filesystem to use.

I suppose I'm most familiar with ext4, although I understand xfs is
the default filesystem during install now.

I believe most files will be maildir style email files, as well as
maillogs, and some tar.bz2 files, so the file size ranges will be
varied.

We're using a homegrown backup script that uses rsync with the
--link-dest option to preserve space. I'd like to move to bacula, but
don't have the time/resources right now.

I'm concerned about recovery time for failed disks, but that's
probably a conversation for another post.

I thought the info below would be helpful - /proc/mdstat and hdparm output.

md125 : active raid5 sdh1[4] sde1[0] sdf1[1] sdg1[2]
  11720656896 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] []
  bitmap: 0/30 pages [0KB], 65536KB chunk

# hdparm -i /dev/sdh

/dev/sdh:

 Model=WDC WD4000FYYZ-01UL1B3, FwRev=01.01K04, SerialNo=WD-WMC130F0NJV0
 Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=7814037168
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}

Thanks,
Alex
___
users mailing list -- users@lists.fedoraproject.org
To unsubscribe send an email to users-le...@lists.fedoraproject.org