Re: Homebuilt NAS Advice

2020-08-09 Thread Leslie Rhorer
On 8/9/2020 5:26 PM, David Christensen wrote: On 2020-08-09 13:38, David Christensen wrote: Then again, many small drives have higher performance than fewer large drives of the same total capacity. And a higher probability of failure. Not necessarily. If the smaller drives have a greater

Re: Homebuilt NAS Advice

2020-08-09 Thread David Christensen
On 2020-08-09 15:22, Stefan Monnier wrote: I wasn't thinking of recovery, but rather the case where you're at the office and need to access a file that you accidentally left on your home computer which is now turned off. I have pondered putting my home directory on the Samba server, which is

Re: Homebuilt NAS Advice

2020-08-09 Thread David Christensen
On 2020-08-09 13:38, David Christensen wrote: Then again, many small drives have higher performance than fewer large drives of the same total capacity. And a higher probability of failure. David

Re: Homebuilt NAS Advice

2020-08-09 Thread rhkramer
On Sunday, August 09, 2020 05:53:56 PM Celejar wrote: > On Sun, 9 Aug 2020 17:42:43 -0400 > rhkra...@gmail.com wrote: > > On Sunday, August 09, 2020 12:22:16 PM Celejar wrote: > > > Are you saying that classic forward-delta > > > based systems don't store backup metadata (or whatever the right

Re: Homebuilt NAS Advice

2020-08-09 Thread Stefan Monnier
> Perhaps, but that has nothing to do with whether the host is down or up > - any time you're doing a recovery, you're going to run into this > issue, insofar as it's an issue. I wasn't thinking of recovery, but rather the case where you're at the office and need to access a file that you

Re: Homebuilt NAS Advice

2020-08-09 Thread David Christensen
On 2020-08-09 07:53, Stefan Monnier wrote: it's convenient to make your latest backup readily accessible so it's very easy to get back yesterday's version of a file in case your fingers fumbled (which might be much more frequent than drive failures). With ZFS and zfs-auto-snapshot, recovery

Re: Homebuilt NAS Advice

2020-08-09 Thread Celejar
On Sun, 9 Aug 2020 17:42:43 -0400 rhkra...@gmail.com wrote: > On Sunday, August 09, 2020 12:22:16 PM Celejar wrote: > > On Sun, 09 Aug 2020 10:53:30 -0400 > > Stefan Monnier wrote: > > > > Another advantage is that it lets you access a host's files even when > > > that host is down (as long as

Re: Homebuilt NAS Advice

2020-08-09 Thread rhkramer
On Sunday, August 09, 2020 12:22:16 PM Celejar wrote: > On Sun, 09 Aug 2020 10:53:30 -0400 > Stefan Monnier wrote: > > Another advantage is that it lets you access a host's files even when > > that host is down (as long as the backup server is up, which is likely > > to always be the case). > >

Re: Homebuilt NAS Advice

2020-08-09 Thread David Christensen
On 2020-08-09 02:07, Jonathan Dowland wrote: We should all be in the habit of regularly performing a restore to test our backups are working. I would add -- "to a spare server". I am loath to overwrite my primary server data, out of subliminal fear conditioned by past bad experiences.

Re: Homebuilt NAS Advice

2020-08-09 Thread David Christensen
On 2020-08-07 16:33, Leslie Rhorer wrote: Nonetheless, it is still true a collection of smaller disks can be less expensive than a large disk In the past, that was usually true. But, take a look at the current pricing of Seagate Exos drives on Amazon [1]: $372.30 / 16 TB = 23.27 $/TB

Re: Homebuilt NAS Advice

2020-08-09 Thread Stefan Monnier
>> Another advantage is that it lets you access a host's files even when >> that host is down (as long as the backup server is up, which is likely >> to always be the case). > > Perhaps I'm misunderstanding something here, but why can't you do that > with ordinary forward deltas, as long as all

Re: Homebuilt NAS Advice

2020-08-09 Thread Celejar
On Sun, 9 Aug 2020 10:07:33 +0100 Jonathan Dowland wrote: > On Sat, Aug 08, 2020 at 10:22:51PM -0400, Celejar wrote: > >Fair enough. But less applicable in the case of backups, since restores > >are quite rare, as I've been pointing out. > > I'd agree they're probably rare, and certainly less

Re: Homebuilt NAS Advice

2020-08-09 Thread Stefan Monnier
>>Fair enough. But less applicable in the case of backups, since restores >>are quite rare, as I've been pointing out. > > I'd agree they're probably rare, and certainly less common than backups. > But in my experience when you do perform a restore it's almost always > the latest version you

Re: Homebuilt NAS Advice

2020-08-09 Thread David Christensen
On 8/7/20 6:24 PM, Leslie Rhorer wrote: On 8/7/2020 6:23 PM, David Christensen wrote: ??Filesystem?? Size?? Used Avail Use% Mounted on Your editor seems to replace multiple spaces with two question marks for each leading space (?). Please disable the feature if you can. The NAS

Re: Homebuilt NAS Advice

2020-08-09 Thread Jonathan Dowland
On Sat, Aug 08, 2020 at 10:22:51PM -0400, Celejar wrote: Fair enough. But less applicable in the case of backups, since restores are quite rare, as I've been pointing out. I'd agree they're probably rare, and certainly less common than backups. But in my experience when you do perform a

Re: Homebuilt NAS Advice

2020-08-08 Thread Celejar
On Fri, 7 Aug 2020 18:02:11 -0400 rhkra...@gmail.com wrote: > On Friday, August 07, 2020 01:36:21 PM Celejar wrote: > > On Fri, 7 Aug 2020 17:21:44 +0100 > > > > Jonathan Dowland wrote: > > > On Thu, Aug 06, 2020 at 12:03:56PM -0400, Celejar wrote: > > > >Ah, okay. So IIUC, each time you backup

Re: Homebuilt NAS Advice

2020-08-08 Thread deloptes
Stefan Monnier wrote: > I have a total of 9 drives currently "in use" (i.e. which I'd have to > replace if they were to die).  I suspect you have many more. Also the quality of the drives is important. Leslie puts a 10G network interface, but by the speed you calculated, he has either poor

Re: Homebuilt NAS Advice

2020-08-08 Thread Stefan Monnier
> Oh, for goodness sake! The BPi uses less than 1 ampere at 5V. > That is under 5W, or 120 watt-hours per day. In a year, it will use > less than 40 KWH. At $0.12 per KWH, that is $5 or less, for full > year in operation. The external enclosure will no doubt use more, >

Re: Homebuilt NAS Advice

2020-08-08 Thread Leslie Rhorer
On 8/7/2020 9:48 PM, Stefan Monnier wrote: No, I am talking about "Extra space taken, extra power used 24/7". An external device just does not use that much power It can easily end up consuming about as much power as my BananaPi, so it risks doubling the power consumption.

Re: Homebuilt NAS Advice

2020-08-08 Thread Jonathan Dowland
On Wed, Aug 05, 2020 at 06:18:16PM -0500, Leslie Rhorer wrote: Not only that, the system is simply unavailable until the file are restored. This can take hours or even days. Much more likely weeks in my case, but that's fine. This NAS has been running for 5 years with one interruption to

Re: Homebuilt NAS Advice

2020-08-07 Thread Stefan Monnier
> No, I am talking about "Extra space taken, extra power used 24/7". > An external device just does not use that much power It can easily end up consuming about as much power as my BananaPi, so it risks doubling the power consumption. > or take a lot of space. I suspect my

Re: Homebuilt NAS Advice

2020-08-07 Thread Leslie Rhorer
On 8/7/2020 6:23 PM, David Christensen wrote: ??Filesystem?? Size?? Used Avail Use% Mounted on ??/dev/md0 28T 22T?? 6.0T?? 79% /RAID ??Backup:/Backup 44T 44T?? 512K 100% /Backup The NAS array is 8 @ 5 TB live drives and 1 @ 5 TB hot spare? It

Re: Homebuilt NAS Advice

2020-08-07 Thread Leslie Rhorer
On 8/7/2020 10:48 AM, Stefan Monnier wrote: Extra space taken, extra power used 24/7 (which in turn requires an extra plug because the poor BananaPi can't provide all that power), Now it is my turn to ask, "Seriously?" [ See, our use cases *are* very different. ] Yes, in my

Re: Homebuilt NAS Advice

2020-08-07 Thread David Christensen
On 2020-07-29 16:41, Leslie Rhorer wrote: I run a pair of Debian servers.  One is essentially a NAS, and the other is a backup system.  Both have 30TB (soon to be 48TB) arrays.  I am running XFS Filesystem  Size  Used Avail Use% Mounted on /dev/md0 28T   22T  6.0T  79%

Re: Homebuilt NAS Advice

2020-08-07 Thread rhkramer
On Friday, August 07, 2020 01:36:21 PM Celejar wrote: > On Fri, 7 Aug 2020 17:21:44 +0100 > > Jonathan Dowland wrote: > > On Thu, Aug 06, 2020 at 12:03:56PM -0400, Celejar wrote: > > >Ah, okay. So IIUC, each time you backup you do a full backup, and you > > >then convert the previous backups

Re: Homebuilt NAS Advice

2020-08-07 Thread Celejar
On Fri, 7 Aug 2020 17:21:44 +0100 Jonathan Dowland wrote: > On Thu, Aug 06, 2020 at 12:03:56PM -0400, Celejar wrote: > >Ah, okay. So IIUC, each time you backup you do a full backup, and you then > >convert the previous backups into the reverse of the more common > >incremental / differential

Re: Homebuilt NAS Advice

2020-08-07 Thread Jonathan Dowland
On Thu, Aug 06, 2020 at 12:03:56PM -0400, Celejar wrote: Ah, okay. So IIUC, each time you backup you do a full backup, and you then convert the previous backups into the reverse of the more common incremental / differential backups. It's an implementation detail: rdiff-backup does it too, I

Re: Homebuilt NAS Advice

2020-08-07 Thread Jonathan Dowland
On Thu, Aug 06, 2020 at 10:04:28AM -0500, Leslie Rhorer wrote: And how useful is that? There are very few duplicate files on my systems, because I use applications to eliminate duplicates. Eliminating duplicates in a live data repository is far more important than doing it on backup media.

Re: Homebuilt NAS Advice

2020-08-07 Thread Stefan Monnier
>> Extra space taken, extra power used 24/7 (which in turn requires an >> extra plug because the poor BananaPi can't provide all that power), > Now it is my turn to ask, "Seriously?" [ See, our use cases *are* very different. ] Yes, in my experience Banana Pis quickly become unreliable if

Re: Homebuilt NAS Advice

2020-08-07 Thread Leslie Rhorer
On 8/6/2020 11:08 AM, Stefan Monnier wrote: For better performance, more space, and higher throughput, I would probably create a RAID 4 or RAID 6 array from the external enclosure and use it as the data repository. And you suggest I put a 4-drive enclosure in my

Re: Homebuilt NAS Advice

2020-08-07 Thread deloptes
Leslie Rhorer wrote: > Come on.  Be honest.  How many messages in this or any other forum can > you specifically remember from a year ago?  When was the last time you > read a post more than a year old?  My servers have a permanent and > non-volatile memory (thank heavens), but I don't, and

Re: Homebuilt NAS Advice

2020-08-06 Thread Leslie Rhorer
On 8/6/2020 10:36 PM, Stefan Monnier wrote: If I were one to use a laptop - which I most certainly do not - I think that's why you don't consider it unthinkable to carry around such a thing along with your laptop. I carried around a 90 lb tool case everywhere for nearly 20 years.

Re: Homebuilt NAS Advice

2020-08-06 Thread Stefan Monnier
>> I don't argue about your approach, it is your design, your servers, but in >> your case I would use definitely deduplication. > And save what? About 0.002%. I don't know if it's important for your use case (probably not), but in backups, deduplication is (mostly) not about finding

Re: Homebuilt NAS Advice

2020-08-06 Thread Stefan Monnier
> If I were one to use a laptop - which I most certainly do not - I think that's why you don't consider it unthinkable to carry around such a thing along with your laptop. What you're suggesting looks like the following to me: - I go to my favorite café to work for the afternoon [ ah, the

Re: Homebuilt NAS Advice

2020-08-06 Thread Leslie Rhorer
On 8/6/2020 11:09 AM, deloptes wrote: Leslie Rhorer wrote: And how useful is that? There are very few duplicate files on my systems, because I use applications to eliminate duplicates. Eliminating duplicates in a live data repository is far more important than doing it on backup media.

Re: Homebuilt NAS Advice

2020-08-06 Thread Leslie Rhorer
On 8/6/2020 8:12 PM, deloptes wrote: I don't know from which universe you came here, honestly :D, sorry, I don't mean to hurt you ... You would find that quite impossible. I would never be upset over a simple disagreement. Or a complex one, for that matter. Any forum without

Re: Homebuilt NAS Advice

2020-08-06 Thread Leslie Rhorer
On 8/6/2020 11:15 AM, deloptes wrote: Leslie Rhorer wrote: A few copies of what size? ??The backup server is an exact mirror of the main server, plus several T of additional files I don't need on the main server. The question is how much back in time you can go. If you have just a mirror -

Re: Homebuilt NAS Advice

2020-08-06 Thread deloptes
Leslie Rhorer wrote: > First of all, it is one small chassis, not 2 -3 separate disks. > Secondly, so what?  You act as if a small box sitting next to a laptop > is some sort of elephant sitting in your lap.  It is not difficult at > all to imagine.  What's difficult to imagine is using a laptop

Re: Homebuilt NAS Advice

2020-08-06 Thread Leslie Rhorer
On 8/6/2020 11:25 AM, deloptes wrote: Stefan Monnier wrote: And you suggest I put a 4-drive enclosure in my backpack next to my laptop? Seriously? Why not? Stefan, I have the same situation - on the Laptop only backup works :D Basically you can forget RAID in USB2 context. I have

Re: Homebuilt NAS Advice

2020-08-06 Thread deloptes
Stefan Monnier wrote: > And you suggest I put a 4-drive enclosure in my backpack next to my > laptop? Seriously? :D +1 Stefan, I have the same situation - on the Laptop only backup works :D Basically you can forget RAID in USB2 context. I have not tried USB3, but I heard that even the spead is

Re: Homebuilt NAS Advice

2020-08-06 Thread Stefan Monnier
>> I find it hard to believe that it's a rare feature. >> The system I use (`bup`) does support that as well (every backup is >> (more or less) a Git commit, so it doesn't distinguish full-backups from >> incrementals), but it doesn't bother to mention it probably because it's >> very basic. > >

Re: Homebuilt NAS Advice

2020-08-06 Thread deloptes
Leslie Rhorer wrote: > A few copies of what size?  The backup server is an exact mirror of the > main server, plus several T of additional files I don't need on the main > server. The question is how much back in time you can go. If you have just a mirror - it is one copy, so you can go one step

Re: Homebuilt NAS Advice

2020-08-06 Thread deloptes
Leslie Rhorer wrote: > And how useful is that? There are very few duplicate files on my > systems, because I use applications to eliminate duplicates. > Eliminating duplicates in a live data repository is far more important > than doing it on backup media. > May be you misunderstand

Re: Homebuilt NAS Advice

2020-08-06 Thread Celejar
On Thu, 6 Aug 2020 11:11:41 -0400 rhkra...@gmail.com wrote: > On Thursday, August 06, 2020 10:07:59 AM Celejar wrote: > > * Incremental and differential backups are backups of the delta between > > the last full backup and the current system state (either individually > > [differential] or

Re: Homebuilt NAS Advice

2020-08-06 Thread Stefan Monnier
> For better performance, more space, and higher throughput, I would > probably create a RAID 4 or RAID 6 array from the external enclosure > and use it as the data repository. And you suggest I put a 4-drive enclosure in my backpack next to my laptop? Seriously? >> For the

Re: Homebuilt NAS Advice

2020-08-06 Thread Celejar
On Thu, 6 Aug 2020 10:04:28 -0500 Leslie Rhorer wrote: > On 8/6/2020 9:07 AM, Celejar wrote: > > On Thu, 6 Aug 2020 05:02:17 -0500 > > Leslie Rhorer wrote: > > > * Incremental and differential backups are backups of the delta between > > the last full backup and the current system state

Re: Homebuilt NAS Advice

2020-08-06 Thread rhkramer
On Thursday, August 06, 2020 10:07:59 AM Celejar wrote: > * Incremental and differential backups are backups of the delta between > the last full backup and the current system state (either individually > [differential] or collectively [incremental]) I am not Leslie Rhorer, I'm just coming out of

Re: Homebuilt NAS Advice

2020-08-06 Thread Leslie Rhorer
On 8/6/2020 9:07 AM, Celejar wrote: On Thu, 6 Aug 2020 05:02:17 -0500 Leslie Rhorer wrote: * Incremental and differential backups are backups of the delta between the last full backup and the current system state (either individually [differential] or collectively [incremental]) * I have no

Re: Homebuilt NAS Advice

2020-08-06 Thread Celejar
On Thu, 6 Aug 2020 05:02:17 -0500 Leslie Rhorer wrote: > On 8/5/2020 9:11 AM, Stefan Monnier wrote: > >>I prefer DAR for several reasons. First of all, as I mentioned > >>before, DAR is the only backup solution of which I am aware that can > >>restore not only deleted or corrupted

Re: Homebuilt NAS Advice

2020-08-06 Thread Leslie Rhorer
On 8/5/2020 9:11 AM, Stefan Monnier wrote: I prefer DAR for several reasons. First of all, as I mentioned before, DAR is the only backup solution of which I am aware that can restore not only deleted or corrupted files, but which can also restore deletions. This

Re: Homebuilt NAS Advice

2020-08-05 Thread Leslie Rhorer
On 8/6/2020 12:06 AM, Leslie Rhorer wrote: For better performance, more space, and higher throughput, I would probably create a RAID 4 or RAID 6 array from the external enclosure and use it as the data repository. Sorry, that was a typo. That should be RAID 5 or RAID 6.

Re: Homebuilt NAS Advice

2020-08-05 Thread Leslie Rhorer
On 8/5/2020 11:13 PM, Stefan Monnier wrote: - RAID would require extra hardware in my machines, for some of them that would be a non-trivial constraint (e.g. my BananaPi servers and my laptops). How do you figure? Adding an external USB drive enclosure to a laptop or a

Re: Homebuilt NAS Advice

2020-08-05 Thread Kushal Kumaran
Leslie Rhorer writes: > On 8/5/2020 1:51 PM, deloptes wrote: > >> Imagine you have classical backup: daily incrementals, full weekly and full >> monthly. > > This is not required with DAR. One full backup is all that is > required. Anything else is a waste of time and space. One can

Re: Homebuilt NAS Advice

2020-08-05 Thread Stefan Monnier
>> - RAID would require extra hardware in my machines, for some of them >>that would be a non-trivial constraint (e.g. my BananaPi servers and >>my laptops). > How do you figure? Adding an external USB drive enclosure to > a laptop or a Banana Pi is pretty trivial. Are you

Re: Homebuilt NAS Advice

2020-08-05 Thread Leslie Rhorer
- RAID would require extra hardware in my machines, for some of them that would be a non-trivial constraint (e.g. my BananaPi servers and my laptops). How do you figure? Adding an external USB drive enclosure to a laptop or a Banana Pi is pretty trivial. RAID does not require any

Re: Homebuilt NAS Advice

2020-08-05 Thread Stefan Monnier
> And I disagree with you, because if you do not have RAID, you loose the data > instantly and in best case your backup is a few hours old. Plus you have > the effort to restore probably the last monthly, then the last weekly and > then few incrementals - compared to replacing just one disk and

Re: Homebuilt NAS Advice

2020-08-05 Thread Leslie Rhorer
On 8/5/2020 1:51 PM, deloptes wrote: Imagine you have classical backup: daily incrementals, full weekly and full monthly. This is not required with DAR. One full backup is all that is required. Anything else is a waste of time and space. One can employ either differential or

Re: Homebuilt NAS Advice

2020-08-05 Thread Leslie Rhorer
On 8/5/2020 5:25 PM, deloptes wrote: Jonathan Dowland wrote: For the majority of use-cases, I really disagree that RAID is ever more essential than backup. I can cook up some scenarios where this isn't true, but they are not common ones. And I disagree with you, So do I. Both are

Re: Homebuilt NAS Advice

2020-08-05 Thread deloptes
Jonathan Dowland wrote: > For the majority of use-cases, I really disagree that RAID is ever more > essential than backup. I can cook up some scenarios where this isn't > true, but they are not common ones. And I disagree with you, because if you do not have RAID, you loose the data instantly

Re: Homebuilt NAS Advice

2020-08-05 Thread Jonathan Dowland
On Wed, Aug 05, 2020 at 09:09:07PM +0200, deloptes wrote: Sorry to answer a question that was not addressed to me, but I was just reading this topic as I had experience in the past few years with this topic in a project I worked on. No problem — it's a public list :) IMO you should do RAID

Re: Homebuilt NAS Advice

2020-08-05 Thread deloptes
Jonathan Dowland wrote: > In brief my rationale is: without RAID, when a drive failure occurs I > lose access to my NAS until I've replaced the drive and restored the > files. I judge that the inconvenience of system downtime is outweighed > by the increased cost (upfront and running),

Re: Homebuilt NAS Advice

2020-08-05 Thread deloptes
Leslie Rhorer wrote: > In this context what, exactly, is de-duplication?  I fail to see how > any meaningful interpretation of the term is salient to backups.  To > compression, yes, to symbolic interpretation, surely, and to saving > space on a drive and reducing access times, you bet.  To

Re: Homebuilt NAS Advice

2020-08-05 Thread Stefan Monnier
> Can bup expire old backups yet (and remove the corresponding objects)? Yes ;-) Stefan

Re: Homebuilt NAS Advice

2020-08-05 Thread Jonathan Dowland
On Wed, Aug 05, 2020 at 10:11:51AM -0400, Stefan Monnier wrote: The system I use (`bup`) does support that as well (every backup is (more or less) a Git commit, so it doesn't distinguish full-backups from incrementals), but it doesn't bother to mention it probably because it's very basic. Can

Re: Homebuilt NAS Advice

2020-08-05 Thread Stefan Monnier
> I prefer DAR for several reasons. First of all, as I mentioned > before, DAR is the only backup solution of which I am aware that can > restore not only deleted or corrupted files, but which can also > restore deletions. This means DAR can restore any or all files one >

Re: Homebuilt NAS Advice

2020-08-05 Thread Jonathan Dowland
On Mon, Aug 03, 2020 at 04:59:22PM -0500, Leslie Rhorer wrote: I concur. One area in wich I disagree is your decision not to employ RAID. As you very properly comment in the linked page, RAID is not a backup. They are *VERY* different things. Therefore both,in my estimation, are absolutely

Re: Homebuilt NAS Advice

2020-08-05 Thread Jonathan Dowland
On Wed, Aug 05, 2020 at 09:16:15AM +0200, deloptes wrote: Leslie Rhorer wrote: DAR allows for not only incremental backups, but also incremental deletions.  I find it extremely useful, and allows for as much space saving on the live system as one likes.  I suggest you check it out. someone

Re: Homebuilt NAS Advice

2020-08-05 Thread Leslie Rhorer
On 8/5/2020 2:16 AM, deloptes wrote: Leslie Rhorer wrote: DAR allows for not only incremental backups, but also incremental deletions. ??I find it extremely useful, and allows for as much space saving on the live system as one likes. ??I suggest you check it out. someone should ask them to

Re: Homebuilt NAS Advice

2020-08-05 Thread deloptes
Leslie Rhorer wrote: > DAR allows for not only incremental backups, but also incremental > deletions.  I find it extremely useful, and allows for as much space > saving on the live system as one likes.  I suggest you check it out. someone should ask them to implement deduplication, because

Re: Homebuilt NAS Advice

2020-08-04 Thread Leslie Rhorer
On 8/4/2020 10:18 AM, Andrei POPESCU wrote: On Ma, 04 aug 20, 01:17:13, deloptes wrote: I don't backup music and video - too big and not changing. The raid is enough for that. I'm guessing it depends on the music and videos. If you made them yourself they are basically irreplaceable and

Re: Homebuilt NAS Advice

2020-08-04 Thread Leslie Rhorer
On 8/4/2020 8:52 AM, Celejar wrote: On Tue, 4 Aug 2020 10:26:32 +0100 Jonathan Dowland wrote: On Tue, Aug 04, 2020 at 01:32:24AM -0700, David Christensen wrote: jdupes looks interesting, and should work on any file system that supports hard links. I expect BorgBackup either calls jdupes or

Re: Homebuilt NAS Advice

2020-08-04 Thread Leslie Rhorer
On 8/3/2020 6:17 PM, deloptes wrote: Leslie Rhorer wrote: My main server's data is backed up every morning by a nearly identical backup server using rsync, which in turn is backed up in its entirety on a multi-volume offline, off-site hard drive set using DAR every few months.  All the boot

Re: Homebuilt NAS Advice

2020-08-04 Thread Andrei POPESCU
On Ma, 04 aug 20, 01:17:13, deloptes wrote: > > I don't backup music and video - too big and not changing. The raid is > enough for that. I'm guessing it depends on the music and videos. If you made them yourself they are basically irreplaceable and RAID will replicate any bitrot to the other

Re: Homebuilt NAS Advice

2020-08-04 Thread Stefan Monnier
> "To actually perform the repository-wide deduplication, a hash of each > chunk is checked against the chunks cache, which is a hash-table of all > chunks that already exist." > > https://borgbackup.readthedocs.io/en/stable/internals.html Same approach as used in `bup` (which all come from

Re: Homebuilt NAS Advice

2020-08-04 Thread Celejar
On Tue, 4 Aug 2020 10:26:32 +0100 Jonathan Dowland wrote: > On Tue, Aug 04, 2020 at 01:32:24AM -0700, David Christensen wrote: > >jdupes looks interesting, and should work on any file system that > >supports hard links. I expect BorgBackup either calls jdupes or > >implements similar

Re: Homebuilt NAS Advice

2020-08-04 Thread Jonathan Dowland
On Tue, Aug 04, 2020 at 01:32:24AM -0700, David Christensen wrote: jdupes looks interesting, and should work on any file system that supports hard links. I expect BorgBackup either calls jdupes or implements similar functionality: https://linuxcommandlibrary.com/man/jdupes.html I'm fairly

Re: Homebuilt NAS Advice

2020-08-04 Thread David Christensen
On 2020-08-03 16:17, deloptes wrote: any thoughts on using deduplication? For example I started using borg some time ago. It saves a lot of space and makes it possible to have multiple backups and longer retention. ZFS supports de-duplication, but the documents warn about enabling it. So,

Re: Homebuilt NAS Advice

2020-08-03 Thread Patrick Bartek
On Mon, 3 Aug 2020 16:14:46 +0100 Jonathan Dowland wrote: > On Wed, Jul 29, 2020 at 12:40:03PM -0700, Patrick Bartek wrote: > >Anyone currently using OpenMediaVault, or have recommendations for > >another package, or advice, in general, on homebuilt NAS? > > My advice is to keep it boring and

Re: Homebuilt NAS Advice

2020-08-03 Thread deloptes
Leslie Rhorer wrote: > My main server's data is backed up every morning by a > nearly identical backup server using rsync, which in turn is backed up > in its entirety on a multi-volume offline, off-site hard drive set using > DAR every few months.  All the boot systems employ two disk RAID 1 >

Re: Homebuilt NAS Advice

2020-08-03 Thread Leslie Rhorer
On 8/3/2020 4:45 PM, Stefan Monnier wrote: A BPi can stand as a media server, but I don't recommend it in general. A media server to a very limited number of clients does not need much I/O power, but if it needs to recode a video on the fly, a BPi is going

Re: Homebuilt NAS Advice

2020-08-03 Thread Leslie Rhorer
On 8/2/2020 11:07 PM, Stefan Monnier wrote: I absolutely agree, no matter what the file system, I would definitely up the memory to the 16GB max, especially if this is to be a media server. Unless you're serving some quite demanding clients, I'd expect 16GB in an NAS to

Re: Homebuilt NAS Advice

2020-08-03 Thread Leslie Rhorer
On 8/3/2020 10:14 AM, Jonathan Dowland wrote: On Wed, Jul 29, 2020 at 12:40:03PM -0700, Patrick Bartek wrote: Anyone currently using OpenMediaVault, or have recommendations for another package, or advice, in general, on homebuilt NAS? My advice is to keep it boring and mundane, and avoid

Re: Homebuilt NAS Advice

2020-08-03 Thread Stefan Monnier
> A BPi can stand as a media server, but I don't recommend it in > general. A media server to a very limited number of clients does > not need much I/O power, but if it needs to recode a video on the > fly, a BPi is going to be really hard pressed to keep up, >

Re: Homebuilt NAS Advice

2020-08-03 Thread tomas
On Mon, Aug 03, 2020 at 02:54:57PM -0500, Leslie Rhorer wrote: > On 8/3/2020 12:00 PM, to...@tuxteam.de wrote: > >On Mon, Aug 03, 2020 at 12:36:43PM -0400, Stefan Monnier wrote: > >>>Work. Part of the problem is one person's NAS is another person's media > >>>server, with radically different

Re: Homebuilt NAS Advice

2020-08-03 Thread Leslie Rhorer
On 8/3/2020 12:00 PM, to...@tuxteam.de wrote: On Mon, Aug 03, 2020 at 12:36:43PM -0400, Stefan Monnier wrote: Work. Part of the problem is one person's NAS is another person's media server, with radically different requirements. And here I am, with my home server used as jukebox (running MPD)

Re: Homebuilt NAS Advice

2020-08-03 Thread tomas
On Mon, Aug 03, 2020 at 12:36:43PM -0400, Stefan Monnier wrote: > > Work. Part of the problem is one person's NAS is another person's media > > server, with radically different requirements. > > And here I am, with my home server used as jukebox (running MPD) as well > as serving that same music

Re: Homebuilt NAS Advice

2020-08-03 Thread Stefan Monnier
> Work. Part of the problem is one person's NAS is another person's media > server, with radically different requirements. And here I am, with my home server used as jukebox (running MPD) as well as serving that same music collection via DAAPD, and my video collection as well. Would that count

Re: Homebuilt NAS Advice

2020-08-03 Thread Dan Ritter
Jonathan Dowland wrote: > On Mon, Aug 03, 2020 at 12:07:32AM -0400, Stefan Monnier wrote: > > > I absolutely agree, no matter what the file system, I would > > > definitely up the memory to the 16GB max, especially if this is to > > > be a media server. > > > > Most commercial "home NAS"

Re: Homebuilt NAS Advice

2020-08-03 Thread Jonathan Dowland
On Mon, Aug 03, 2020 at 12:07:32AM -0400, Stefan Monnier wrote: I absolutely agree, no matter what the file system, I would definitely up the memory to the 16GB max, especially if this is to be a media server. Unless you're serving some quite demanding clients, I'd

Re: Homebuilt NAS Advice

2020-08-03 Thread Jonathan Dowland
On Wed, Jul 29, 2020 at 12:40:03PM -0700, Patrick Bartek wrote: Anyone currently using OpenMediaVault, or have recommendations for another package, or advice, in general, on homebuilt NAS? My advice is to keep it boring and mundane, and avoid the temptation to try some flashy technology Just

Re: Homebuilt NAS Advice

2020-08-02 Thread Stefan Monnier
> I absolutely agree, no matter what the file system, I would > definitely up the memory to the 16GB max, especially if this is to > be a media server. Unless you're serving some quite demanding clients, I'd expect 16GB in an NAS to be a complete waste of money. IOW it all

Re: Homebuilt NAS Advice

2020-08-02 Thread Leslie Rhorer
On 8/1/2020 8:06 PM, David Christensen wrote: On 2020-08-01 16:30, Leslie Rhorer wrote: I am a big proponent of having a separate boot drive +1 no matter what the file system, I would definitely up the memory to the 16GB max, 1.  I try very hard not to spend money on obsolete technology.

Re: Homebuilt NAS Advice

2020-08-02 Thread Leslie Rhorer
On 8/1/2020 11:05 PM, David Christensen wrote: On 2020-08-01 19:30, Keith Bainbridge wrote: On 2/8/20 11:06 am, David Christensen wrote: On 2020-08-01 16:30, Leslie Rhorer wrote: I am a big proponent of having a separate boot drive +1 +1 more For players-around like me, it is also

Re: Homebuilt NAS Advice

2020-08-02 Thread Patrick Bartek
On Sat, 1 Aug 2020 18:39:49 -0500 Leslie Rhorer wrote: > I don't think SyncThing is what you want, per your stated requirements. > SyncThing synchronizes data among multiple hosts. You said you wanted > a NAS, which implies a solitary host. After a little purusing of the docs on

Re: Homebuilt NAS Advice

2020-08-01 Thread David Christensen
On 2020-08-01 19:30, Keith Bainbridge wrote: On 2/8/20 11:06 am, David Christensen wrote: On 2020-08-01 16:30, Leslie Rhorer wrote: I am a big proponent of having a separate boot drive +1 +1 more For players-around like me, it is also easier, in general terms, to multi boot. No

Re: Homebuilt NAS Advice

2020-08-01 Thread Keith Bainbridge
On 2/8/20 11:06 am, David Christensen wrote: On 2020-08-01 16:30, Leslie Rhorer wrote: I am a big proponent of having a separate boot drive +1 +1 more For players-around like me, it is also easier, in general terms, to multi boot. -- Keith Bainbridge keith.bainbridge.3...@gmail.com

Re: Homebuilt NAS Advice

2020-08-01 Thread David Christensen
On 2020-08-01 16:30, Leslie Rhorer wrote: I am a big proponent of having a separate boot drive +1 no matter what the file system, I would definitely up the memory to the 16GB max, 1. I try very hard not to spend money on obsolete technology. 2. I would wait until the computer is put

Re: Homebuilt NAS Advice

2020-08-01 Thread Leslie Rhorer
I don't think SyncThing is what you want, per your stated requirements. SyncThing synchronizes data among multiple hosts. You said you wanted a NAS, which implies a solitary host. On 7/30/2020 4:28 PM, Patrick Bartek wrote: On Thu, 30 Jul 2020 09:40:29 -0700 Peter Ehlert wrote: This

Re: Homebuilt NAS Advice

2020-08-01 Thread Leslie Rhorer
This is one reason, among several others, why I am a big proponent of having a separate boot drive - or better yet, a boot array - from my data array. I can quickly and easily build a very plain Jane boot system using the Debian distro DVD without having to worry about inconsistencies with

Re: Homebuilt NAS Advice

2020-07-31 Thread Celejar
On Thu, 30 Jul 2020 13:21:00 -0600 Charles Curley wrote: > On Thu, 30 Jul 2020 09:40:29 -0700 > Peter Ehlert wrote: > > > This whole conversation is a bit over my head. > > I suggest you look into Syncthing. > > > > It's not in the Debian repos, but it is open source and it just works. > >

  1   2   >